Multi-dimensional Data Access minimal requirements

[Douglas Tody]

On Mon, 5 May 2014, Patrick Dowler wrote:

> From the discovery use cases summarised below, the current WD-SIA-2.0 has 
> parameters to support most of the axis fields (with the applicable ObsCore 
> column in brackets):
>
>   RA,Dec               -> POS  (s_region)
>   Frequency/wavelength -> BAND (em_min, em_max)
>   Polarization states  -> POL  (pol_states)
>   Time of observation  -> TIME (t_min, t_max)
>
> We will need 3 additional parameters to query these:
>
>   Spatial size         -> ? (s_fov)
>   Angular resolution   -> ? (s_resolution)
>   Integration time     -> ? (t_exptime)

Spectral resolution and resolving power is also quite important,
especially for spectral cubes.  PubDID (publisher dataset identifier) is
also essential for the operation of the service, but it is more like
MAXREC, UPLOAD, etc.

> My inclination is to define parameters with a prefix taken from the axis, 
> such as POS_???, BAND_???, TIME_??? etc. There are also existing usages (eg. 
> SPATRES in SSA) that mean the same thing; I'm not sure if trying to use 
> existing parameter names will make it easier or get us into a bind later 
> on...
>
> Action Item for WG: propose 1-3 parameter names to query the above 3 numeric 
> fields of ObsCore; think about how more parameters might be defined for 
> querying other ObsCore fields (including ObsCore-1.x if something gets 
> added).

It seems unlikely that I can prevent a complete redo of the DAL interface
patterns, but I will nontheless argue for providing some consistency with
current interfaces, e.g., SSA.

In our implementations here for example, SIA and SSA are both essential
data service interfaces.   We will be putting up SSA services for
example for GBT spectra and for spectra extracted from ALMA and EVLA
spectral data cubes.  Our DALServer framework already supports both SIA
and SSA as well as other DAL interfaces.  It is desirable for parameters
that do the same thing to appear the same in both interfaces.

Consistency with legacy services is also an issue.  Even if we come out
with V2 versions of SIA and SSA, most of the actual services out there
in the community will continue to be V1 services for some time, probably
years.  Hence both V1 and V2 versions have to be supported.  Consistency
is again important.  Good service software and client bindings will need
to support both versions for some time, probably years.

Major changes to the service interfaces will have major impact, from the
service level all the way up into client APIs and applications.  For our
current SIAV2 prototype here it was actually quite easy to implement the
service interface as we just cloned our SSA implementation and edited it
for what is different for the SIA semantics.  To provide support for the
eventual IVOA SIAV2 (which is actually going to require 5-6 linked
services) will be a major undertaking, requiring software changes at all
levels.

Some of the things we are adding, such as integration of the data model
with ObsCore/ObsTAP and support for new capabilities such as DataLink,
and broader support for DALI capabilities such as UPLOAD and UWS, and
important enough to be worth the effort.  But many of these changes are
mere changes to things like parameter look and feel, with no actual
change in capability.  We are ensuring inconsistent interfaces and
requiring considerable software changes for little gain.

For SIA I suggest that we retain the parameter names currently in use
unless there is a pretty compelling reason to change:

     SPATRES                     Spatial resolution
     SPECRES, SPECRP             Spectral resolution and resolving power
     [TIMERES - if we ever manage to add TimeSeries]

     PubDID                      Publisher dataset identifier

COLLECTION can also be important if the service is exposing data from
multiple collections.

UPLOAD, MAXREC, RESPONSEFORMAT, etc. I think are already in DALI.

The above are the most basic parameters.  MODE has already been
discussed to little effect, but we will need to do something about
virtual data, e.g., simple use of SIA to access wide field survey data
where we do not care about individual archival images (simple use means
the optional advanced accessData is not required).  INTERSECT has not
been discussed at all, but addresses an issue with spatial region
queries.  We should at least define the degree of intersection of ROI
and target image that a service is required to implement.

         - Doug



> thanks
>
>
> On 06/03/14 04:14 PM, Gaudet, Severin wrote:
>> Multi-dimensional Data Access minimal requirements:
>>
>>   * Data Discovery (Query)
>>       o A service shall be able to receive queries regarding its data
>>         collection(s) from a client, with the client placing one or more
>>         of the following constraints:
>>           + RA,Dec
>>           + Frequency/wavelength
>>           + Polarization states
>>           + Spatial size
>>           + Angular resolution
>>           + Integration time
>>           + Time of observation
>
> -- 
>
> Patrick Dowler
> Canadian Astronomy Data Centre
> National Research Council Canada
> 5071 West Saanich Road
> Victoria, BC V9E 2E7
>
> 250-363-0044 (office) 250-363-0045 (fax)
>