On the relationship of DAL standard input PARAMETERS and IVOA Data models

[François Bonnarel]

Dear all,

     This email (with some ideas in it partially discussed with Laurent 
Michel last week) is motivated by the discussion on the relationship 
between SODA input parameter Domain metadata and some of the Obscore 
attributes of course.  But the scope is neither limited to SODA nor to 
Obscore Data Model. SSA implemented a view of the Spectrum data model 
while SIAV2.1 is supposed to implement the Cube and dataset models in a 
near future.

      In the case  of Data discovery services like SIAV2.0, the Obscore 
data model is not only present in the query response as a description of 
the available dataset, it is also underlying the discovery process 
itself. The process of finding out datasets included in a given region 
of the parameter space is actually the process of looking for datasets 
for which the support or bounds are included in the ROI on all 
considered axes.

      Cuting-out or selecting some values on some axes, like SODA is 
supposed to do, is actually forcing the generation of sub-datasets whose 
bounds or support are matching the INput parameter values as much as 
possible.

      Actually Obscore attributes do not have only a power of 
description of existing datasets but also have the power of generating 
virtual datasets. Isn't that really interesting ?

      An important point is that the direct relationship between the 
SODA INPUT parameters and the Obscore model attributes is not for the 
archived discovered dataset it is with virtual dataset. We are actually 
describing the dataset which will come out of the SODA service.

      The relationship between standard parameters and the underlying 
model also explain why it was important to keep consistency between 
SIAV2 and SODA (previously AccessData) INPUT parameters. Same parameter, 
same model concept. But operated in a different way!

       In the case of simple cuting-out where there is no reprocessing 
of the pixel values (only selection) the Obscore bounds and support 
attributes values are giving the very limits of the possible "cutouts" 
ones. If we were an institution operating from the same place both a 
discovery service and a SODA service it would be strange to give bounds 
wider than the extractable limits. If we can be sure that data on the 
edges are irrelevant for extraction we should adapt bounds consequently 
in our discovery service.
If we were operating only a SODA service, while another institution will 
operate the Data discovery service, we are able to provide the correct 
bound values for the whole archived dataset and can inform this other 
institution to adopt these correct values in their discovery service. If 
we have SODA, whe also have the dataset home and we are the source of 
"true" dataset metadata.

In the case of resampling or regriding of the dataset or other 
Server-side accessdata recomputing (some functionalities which will come 
in SODA1.1) some other standard parameters will be in charge : spatial, 
spectral or time resolution, WCS-Mapping. All will be relying on other 
Obscore or Cube/dataset dm attributes and features.

       For the current standard parameters (POS, BAND,TIME, POL), it 
would be usefull to relate in some way the input PARAMETERS   to the 
corresponding Obscore model attributes. A month ago, I have proposed a 
"ref/ID" mechanism, which has been considered by Markus on this list as 
too heavy. Maybe it is. So let's try another solution.
One could also imagine to give the Obscore model attribute utype as a 
the utype PARAM attribute in the SODA service descriptor ,like this


<?xml version="1.0" encoding="UTF-8" ?>
<VOTABLE version="1.2" xmlns:xsi = 
"http://www.w3.org/2001/XMLSchema-instance" 
xsi:noNamespaceSchemaLocation = 
"xmlns:http://www.ivoa.net/xml/VOTable-1.2.xsd" >
       <RESOURCE type="meta" utype="adhoc:service" name="this">
           <PARAM name="standardID" datatype="char" arraysize="*" 
value="ivo://ivoa.net/std/SODA#sync-1.0" />
           <PARAM name="accessURL" datatype="char" arraysize="*" 
value="http://example.com/SODA/sync" />
           <GROUP name="inputParams">
                <PARAM name="ID" ucd="meta.id" datatype="char" 
arraysize="*"  utype="obscore:Curation.PublisherDID" />
                <PARAM name="POS" ucd="pos" unit="deg" datatype="char" 
arraysize="*" utype="obscore:Char.SpatialAxis.Coverage.Support.Area"   />
                <PARAM name="BAND" ucd="em" unit="m" datatype="double" 
arraysize="*"  utype=" obscore: Char.SpectralAxis.Coverage.Bounds.Limits" />
                <PARAM name="TIME" ucd="time" unit="d" datatype="double" 
arraysize="*" utype="obscore:Char.TimeAxis.Coverage.Bounds.Limits/>
                <PARAM name="POL" ucd="pol" datatype="char" 
arraysize="*" utype="obscore:CharPolarizationAxis.stateList" />
           </GROUP>
       </RESOURCE>



Having such a utype  in the context of standard input PARAMS in a SODA 
service descriptor could have the special meaning of forcing the 
corresponding feature of generated data to the INPUT parameter value. 
For the domain metadata, in the current context of POS,BAND,TIME, POL, 
the <VALUES> element inside the <PARAM> could be written with a ref to 
the appropriate FIELD/GROUP in the dataset metadata section.
<?xml version="1.0" encoding="UTF-8" ?>
<VOTABLE version="1.2" xmlns:xsi = 
"http://www.w3.org/2001/XMLSchema-instance" 
xsi:noNamespaceSchemaLocation = 
"xmlns:http://www.ivoa.net/xml/VOTable-1.2.xsd" >
       <RESOURCE type="meta" utype="adhoc:service" name="this">
           <PARAM name="standardID" datatype="char" arraysize="*" 
value="ivo://ivoa.net/std/SODA#sync-1.0" />
           <PARAM name="accessURL" datatype="char" arraysize="*" 
value="http://example.com/SODA/sync" />
           <GROUP name="inputParams">
                <PARAM name="ID" ucd="meta.id" datatype="char" 
arraysize="*"  utype="obscore:Curation.PublisherDID" />

                <PARAM name="POS" ucd="pos" unit="deg" datatype="char" 
arraysize="*" utype="obscore:Char.SpatialAxis.Coverage.Support.Area"   >
                       <VALUES ref="sreg" />
                </PARAM>
                <PARAM name="BAND" ucd="em" unit="m" datatype="double" 
arraysize="*"  utype=" obscore: Char.SpectralAxis.Coverage.Bounds.Limits" >
                             <VALUES ref="sbounds"/>
                </PARAM>
                <PARAM name="TIME" ucd="time" unit="d" datatype="double" 
arraysize="*" utype="obscore:Char.TimeAxis.Coverage.Bounds.Limits>
                              <VALUES ref="tbounds"/>
                </PARAM>
                <PARAM name="POL" ucd="pol" datatype="char" 
arraysize="*" utype="obscore:CharPolarizationAxis.stateList" >
                             <VALUES ref="pstates" />
                </PARAM>
           </GROUP>
       </RESOURCE>



<RESOURCE name="DatasetMetadata">
           <GROUP ID="tbounds">
              <FIELDref ref="tmin"/>
              <FIELDref ref="tmax"/>
           </GROUP>
          <GROUP ID="sbounds">
             <FIELDref ref="smin"/>
             <FIELDref ref="smax"/>
           </GROUP>
          <PARAM ID="sreg" name="s_region" datatype="char" 
ucd="phys.angArea;obs" 
utype="obscore:Char.SpatialAxis.Coverage.Support.Area" arraysize="*" 
unit="deg" value="POLYGON 30.0 200.0 32.0 200.0 32.0 198.0 30.0 198.0"/>
          <PARAM ID="tmin" name="t_min" datatype="double" 
ucd="time.start;obs.exposure" 
utype="obscore:Char.TimeAxis.Coverage.Bounds.Limits.StartTime" unit="s" 
value="52300.5"/>
           <PARAM ID="tmax" name="t_max" datatype="double" 
ucd="time.end;obs.exposure" 
utype="obscore:Char.TimeAxis.Coverage.Bounds.Limits.StopTime" unit="s" 
value="52300.6"/>
            <PARAM ID="smin" name="em_min" datatype="double" 
ucd="em.wl;stat.min" utype=" obscore: 
Char.SpectralAxis.Coverage.Bounds.Limits. LoLimit " unit="m" value="0.20"/>
           <PARAM ID="smax" name="em_max" datatype="double" 
ucd="em.wl;stat.max" 
utype="obscore:Char.SpectralAxis.Coverage.Bounds.Limits.HiLimit" 
unit="m" value="0.22"/>
           <PARAM ID="pstates" name="pol_states" datatype="char" 
ucd="meta.code;phys.polarization" 
utype="obscore:Char.PolarizationAxis.stateList" arraysize="*" 
value="StokesQ,StokesU,StokesV"/>
        </GROUP>
     </RESOURCE>

CUSTOM parameters :
     In this general  view, CUSTOM Parameters are Service parameters 
which are not relying on an IVOA datamodel such as Obscore, spectrum or 
soon  Cube/ Dataset. In that case they will have no corresponding utype 
and will not be associated  with dataset metadata.
      Actually if there is no utype in the PARAM definition, client 
software should discover the domain metadata using <VALUES> <MIN/> 
<MAX/> </VALUES> . So the solution Markus is proposing in its version of 
the draft (http://docs.g-vo.org/SODA-r3192.pdf).

       Although it should be nice to find a standard way to define 
custom parameters, I am not sure we have to do it now, as I am not sure 
we really need to define DOMAIN metadata right now (see my previous 
emails). But at least we can imagine that a solution relating standard 
parameters and the model attributes exist. And at least custom 
parameters and standard ones could be defined in similar ways.

       This could also allow a standard way to define non standard or 
custom services, because these ones are supposed to be all made of non 
standard parameters

Cheers
François