Beside (Re: ObsCore update discussion : adding Axes information in Obscore table) ---> WCS in nDAL

[Marco Molinaro]

Dear all,
regarding this topic I have a small use case that comes from a (currently
custom) set of services whose aim is to allow velocity spectra analysis of
galactic FITS cubes.
The full service (the combination of the set of services) itself can fit
into a "TAP/ObsCore-Datalink and/or SIAv2-AccessData" scenario.

The use case can be described as:
a - a super-set of FITS cubes from non-homogeneous galactic surveys and
pointed archives in the radio band is deployed to allow velocity spectra
analysis
b - the first step for the user is to search in this set for available data
along a line-of-sight, with possible filtering on a cone around it, or a
box around it, limiting the velocity range, selecting explicitly one/more
survey(s) by name, species, transition, ...
c - the search output (which includes something along the lines of a
PublisherDID) is then used to explicitly cut the needed cube(s) to make
data transfer affordable (in the near future merging of adjacent
"same-survey" cubes will be also implemented)

The need for WCS information in the output of the search comes from the
idea of allowing the client side to build correct cutout queries to the
system in terms of spatial boundaries but also for the available velocity
ranges and units available for each stored dataset.
That is, given the surveys are heterogeneous, a broad filtering will return
datasets with different boundaries and the client should be able to
understand the limits in coverage before asking for the actual data.
As a last, but not less important feature, the fits cutout can be performed
using both positional and velocity constraints, together or independently.

I take the chance of this mail thread to give here also my 2 cents on the
regexp approach Markus described in the DM-listed starting post on this
topic (http://mail.ivoa.net/pipermail/dm/2015-April/005150.html).
I tend to agree with Laurent's reply content.
I agree adding fields to a table is something we should care about, but
packaging information is not usually my preferred solution.
Anyway, it could work in discovery, but it's not enough for the results
output I need in my above use case.

Cheers,
    Marco

2015-04-16 9:09 GMT+02:00 François Bonnarel <
francois.bonnarel at astro.unistra.fr>:

>  Dear all,
>
> Beside the discussion on OBscore upgrade for datatype specific description
> of cubes or whatever (see belox Mireille's email), a lot of arguments were
> developped about the question of access to WCS information with DAL
> protocol.
>     In general WCS information is not usefull for data discovery but
> nonetheless there is a lot of use cases requiring WCS knowledge in advance
> to data retrieval or access (among others: preparation of pixel cutouts or
> conveing WCS information with png images).
>
>     The original DAL idea is that SIA2 Getmetadata method will provide a
> full serialisation of  ImageDm when this will be available. The WCS (or
> Transfom) classes in ImageDM will be based on STC2.0 and encompass all the
> possible complex situations.
>     However, I am aware of several  experiments under study in differents
> groups   which could provide quick  solutions for DAL WCS access in most of
> the simpler cases including simple and multiple array calibrated with
> linear solutions.
>
>      It would be a good idea to start a  discussion on this topic  based
> on description  of  various proposals and prototypes. It would be also
> usefull to better specify the GetMetadata concept in this perspective and
> to highlight the relationships of these proposals with the ImageDM effort
> in order to clarify  the strategy and DAL roadmap.
>
>      A dedicated DAL session on this topics could be held in Sesto.
> According to what DM chairs think it could be alos co-organized as a DAL/DM
> joint session.
>
> Regards
> François Bonnarel after discussion with Marco and Mireille
> Le 15/04/2015 19:50, Louys Mireille a écrit :
>
> Dear all ,
> Here is a summary of discussions on the Obscore data model update, with
> several iterations
> with Doug Tody, François Bonnarel, Laurent Michel, Daniel Durand and Pat
> Dowler.
> After the suggestion I made at the last interop in Banff,
> (see
> http://wiki.ivoa.net/internal/IVOA/InterOpOct2014DM/obcore-ObsdatasetDM_compatibility.pdf
> )
> we iterated with Doug Tody to enhance the axes' description of  the data
> part exposed in a dataproduct distributed by Obscore.
>
> These add_ons are meant to enrich the query response of ObsTAP serving
> N-D datasets in order to expose the
> dimensions and the nature of axes involved in the data part.
>
> The way the data values are organized in files ( File format, single or
> multiple extensions in FITS, etc, ..) is not taken into account here,
> as the goal is discovery  and selection of  candidate datasets from the
> query response.
>
> These values are not meant to be exact but represent a very good
> approximation of the dimension in each physical axis.
> So we are proposing the addition of
>
>    - s_dim1, s_dim2 = the coverage in sampling elements ( pixels) for
>    each spatial axis
>    - em_dim = the coverage in spectral elements along the energy axis
>    - t_dim = the coverage in the time axis, as number of time bins
>     - pol_dim = the coverage in the polarization axis, as number of
>    polarization states
>
> The nature of each axis is given by their prefix (s (spatial), em
> (energy), t (time) and pol (polarization) s mentionned in the Obscore
> previous version.
>
> These parameters are not describing the packaging (this is not providing
> enough information for cutout yet.
> If the value of the <n>_dim is 1, it means that this is a degenerate axis.
> One cannot have a <n>_dim =0 but NULL is allowed (interpreted as 1).
>
> Please note that the default polarization for any imaging data is I,
> intensity.
>
> It is important to note that the <n>_dim values are the overall
> characteristic of the dataset and NOT the value of its WCS axis
> (it might be true for simple cases, i.e. simple image but certainly not
> for mosaic observation where the gap
>  and/or the presence/absence of a given chip is not specified) .
>
> Here are some examples of various datasets exposed with this strategy :
>
>    - MUSE data cube
>
>     s_dim1   = 300
>     s_dim2   = 300
>     em_dim   = 3463
>     pol_dim      = 1
>     pol_state = I
>     t_dim      = 1
>
>
>    - 2MASS: 2D image
>
>     s_dim1   = 300
>     s_dim2   = 300
>     em_dim   = 1
>     pol_dim  = 1
>     pol_state = I
>     t_dim = 1
>
>
>
>    - MEGACAM: mosaic image
>
>     s_dim1   = 20000
>     s_dim2   = 20000
>     em_dim   = 1
>     pol_dim  = 1
>     pol_state = I
>     t_dim = 1
>
>
>
>    - STIS spectroscopy (1D):
>
>     s_dim1   = 1
>     s_dim2   = 1
>     em_dim   = 1024
>     pol_dim  = 1
>     pol_state = I
>     t_dim = 1
>
>
>
>    - STIS spectroscopy (2D long slit):
>
>     s_dim1    = 1024
>     s_dim2    = 1
>     em_dim    = 1024
>     pol_dim   = 1
>     pol_state = I
>     t_dim = 1
>
>
>
>    - ALMA:
>
>     s_dim1    = 1000
>     s_dim2    = 1000
>     em_dim   = 3000
>     pol_dim  = 4
>     pol_state = I/U/V/Q
>     t_dim = 1
>
> The more detailed mapping of s, em , etc on NAXIS1, NAXIS2, etc. in terms
> of WCS values could be handled in
> a future *getmetadata *capability of SIAv2, either via ADQL / Obstap or
> with a query by parameter, awaiting a
> more complete WCS solution for all astronomical observation.
> A discussion on WCS implementation for instance  at the IVOA interop
> meeting in Sesto in June will help to go forward.
>
> There are suggestions to use an optional UCD tag to specify the flavour of
> these axes , like *em_ucd*, for instance , already in ObsCoreDM
> to help to disentangle between frequency , wave and energy in the query
> response.
>
> A draft is currently updated along these lines.
>
> Your comments , and suggestions welcome .
> Mireille
>
> --
> Mireille Louys	, Maître de conférences
> Centre de Données ( CDS)		Icube & Télécom Physique Strasbourg, Pôle API
> Observatoire de Strasbourg 		300, boulevard Sébastien Brant
> 11, Rue de l'Université			CS 10413
> 67000 Strasbourg 				F - 67412 ILLKIRCH Cedexhttp://astro.unistra.fr			http://www.telecom-physique.fr
> tel : 03 68 85 24 34
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.ivoa.net/pipermail/dal/attachments/20150423/2ff64044/attachment.html>