Reflections on SIA V2 and generic cutout services

[François Bonnarel]

Hi all,

Le 07/03/2016 16:16, Tom McGlynn (NASA/GSFC Code 660.1) a écrit :
> Hi Markus,
>
> While Walter may have a different response, my view is that while SODA 
> gets us started on defining how we do subsetting, there's no obvious 
> way to tie it to SIA to provide the kind of complete capability that 
> the current SkyView SIA services (and other SIA cutout services) do.  
> So right now the latest version of our primary image access protocol 
> cannot naturally access many of our image resources.
>
As I tried to explain in my previous answer this is true. Just because 
we decided to refactor SIA in a multi-d perspective, we stayed on very 
basic functionalities as a first step: discovery of archived datasets 
and cutouts/selection. We are in an incremental process while extending 
the datatype scope of SIA a lot. Hopefully there will be  next steps. 
For 2D surveys you can always keep you SIA1 services.
>   - SODA looks only at the subsetting aspects of the query.  While it 
> might be possible to use SIA to expose SODA resources this brings us 
> back to requiring a two step process where users enter data twice 
> first to discover resources and then to subset them. This seems pretty 
> clumsy.
As I wrote in my previous answer, we could decide that the SIA Query 
discover virtual data instead of archived datasets. In that case the 
access reference will probably be an URL with a full SODA request.
But we will have to find  a mecanism (OPTION=VIRTUAL parameter ?) to 
distinguish Archived data discovery from virtual data discovery. And 
somewhat the SIAV2 (Or OBsTAP) has to know about the real capacities of 
the associated SODA service to give pertinent answers. Another reason to 
postpone to 2.1
>
>   - The names for parameters in SODA duplicate the names of the 
> parameters in the SIA call.  Thus we're likely to get some confusion 
> between parameters used for data discovery versus parameters used for 
> subsetting.
>
Hummm. This was intentional. The idea is that if the ROI for the 
discovery is also the limits you want to use for an output, the 
transition from SIAV2 query to SODA request is easy. The closest thing 
for accessing to virtual data we could imagine at this step.
>   - The spatial subsetting is not well defined.  E.g., if I'm asking 
> for a spatial subset where I've given  POS=CIRCLE 12 34 0.5 what do I 
> get back?  Do I get back a list of pixels that are within (or 
> partially within) that circle?
No
> Do I get back a rectangle that encloses that circle? How is such a 
> rectangle defined?  What coordinates, projection?
Yes, it was always the intention. If it is not obvious we have to 
clarify this in the text.
>   The last bit is probably less a problem for most cutout services 
> since they'd do rectangles in the frame of the observations.  But for 
> SkyView where the whole idea is that the user gets pick these things, 
> it's quite relevant.
Adding some recomputing facilities modifying grids and pixel values has 
been postponed to next version of SODA.
> SODA should provide some way to select these, or at least to describe 
> the defaults.
>
> What I could see working is defining a service that is simultaneously 
> a SODA and SIA service.  That would imply that the SODA parameters 
> should have distinct names from the SIA names. 
This is the idea of virtual data discovery. If you have a solid and 
stable SODA service you can discover the datasets it is able to provide 
you in a given N-D box
Unfortunately for Skyview, at the IVOA level, we have to provide the 
firs step first for cubes before going further.
> Specifically for image cutouts there should be some way that the SODA 
> POS could be inferred from the SIA POS fields. 
Immediatly, in the two step perspective, the SODA descriptor included in 
the SIAV2 query response will allow you to do something like that.

Cheers
François
> The actual meaning of the cutout for a given POS field would also need 
> to be defined at least semi-rigorously so users understand what they 
> are going to get back.
>
>     Tom
>
>
> Markus Demleitner wrote:
>> Hi Walter,
>>
>> On Thu, Mar 03, 2016 at 12:39:24PM -0800, Walter Landry wrote:
>>> "Tom McGlynn (NASA/GSFC Code 660.1)" <tom.mcglynn at nasa.gov> wrote:
>>>> 1. SkyView is a cutout and mosaicking service.  So in terms of
>>>> retrieving an image SkyView needs two kinds of inputs: those that
>>>> select the survey or surveys we are interested in (e.g., bandpass and
>>>> resolution) and the WCS parameters that define the region to be
>>>> generated.
>>> I had the same problem with our Planck image generation service.  SIA2
>>> is really not set up for this.  SIA2 is much better for retrieving
>>> links to existing images.  For example, your mosaicing service might
>>> have a multitude of options on how exactly to stitch images together.
>>> The best conceptual match I could find was something like SimDAL,
>>> which allows for this sort of extensive computation in order to
>>> generate an artifact for the end-user.  With that said, I have not
>>> actually implemented anything like SimDAL for our service.
>> These kinds of things are very much what I'm sure we have to cover in
>> SODA and why I'm so keen to keep it easily *and interoperably*
>> extensible.  And no, I'm totally against postponing "support" for
>> this to SODA 1.1.  Sure, we won't give the parameter prototypes
>> ("three factors") for such parameters in 1.0.  But SODA 1.0 clients
>> should already support this by virtue of discovering parameter
>> metadata by dataset and building UIs based on that.
>>
>> Can't you just try to formulate the paramters you need and see how
>> they come out in SODA?  And what's missing in case it's not
>> straightforward?
>>
>> It'd be a great test to see if SODA clients actually do the right
>> thing, it'd be great as models for other services, and from the
>> collected experience we could then derive how to actually
>> standardise parameters for this kind of thing.
>>
>> Cheers,
>>
>>         Markus