Obstap / about optional fields

[Douglas Tody]

TAP_SCHEMA.columns should fully describe the actual table, which *must*
include all mandatory ObsCore fields, and which *may* include optional
ObsCore fields or additional custom fields as defined by the data provider.

 	- Doug


On Thu, 17 Mar 2011, Mireille Louys wrote:

> Hi François, all
>
> In the current version of the document the content of TAP_SCHEMA.columns is 
> described in table 6 for the mandatory column_names and in table 7 for the 
> optional ones.
> I suggest adding a paragraph that states :
> TAP_SCHEMA.columns must contain all mandatory fields of table 6 and may 
> contain an extra list of columns as described in Table 7 and possibly some 
> additional fields defined by the data provider.
>
> is it correct?
> understandable?
>
> thanks, mireille
>
> Bonnarel <francois.bonnarel at astro.unistra.fr> a écrit :
>
>> Hi JuanDe
>> I think the simplest for version 1.0 is solution 1 with a minor
>> modification. A standard error message is enough
>> for services not supporting the non mandatory fields.
>> The idea is to ALLOW the discovery of datasets which will stay headen
>> otherwise....
>> The idea is also to minimize the extra work for this version ...
>> Cheers
>> François
>> Le 17/03/2011 10:05, Juande Santander Vela a écrit :
>>> Perhaps it has eluded me and is already discussed in the ObsCore 
>>> document, but I cannot recall having read it.
>>> 
>>> I have the following problem with optional fields: when we say that  a 
>>> given ObsTAP field is optional, what does exactly mean:
>>> 
>>> a) The field is not mandatory in the ivoa.ObsCore table,  and
>>>   therefore no entries exist in TAP_SCHEMA.columns with
>>>   table_name = "ivoa.ObsCore".
>>> 
>>> b) The field is mandatory in the ivoa.ObsCore table, but the
>>>   content for all rows is NULL.
>>> 
>>> The problem with approach a) is that we are defeating the ability  of 
>>> sending the same ADQL query in all ObsTAP compliant services,  unless the 
>>> ObsTAP implementations return NULL by default to columns  they do not know 
>>> about in a given schema.
>>> 
>>> So, for solving that issue, I see three approaches:
>>> 
>>> 1) TAP services return NULL for columns they know nothing about.
>>>   This seems like a hack, and an extra burden for TAP
>>>   implementations.
>>> 
>>> 2) Take approach b) above. But then we would have to check
>>>   that the semantics for NULL do not interfere with anything
>>>   we have already in place.
>>> 
>>> 3) Group optional fields, so that they exist or not together
>>>  (even if some of them need to be NULL), and the first discovery
>>>   of relevant ObsTAP services is made through capabilities search
>>>   in the registry.
>>> 
>>> I think 3) is more natural: we have more information in the  registry for 
>>> service discovery, and we ensure that the fields exist  for the use case 
>>> we are interested about.
>>> 
>>> What do you think?
>>> 
>>> Am 17/03/2011 um 00:14 schrieb François Bonnarel:
>>> 
>>>> It was admitted a long time ago that the obstap parameters should  alllow 
>>>> the same ADQL
>>>> query to be performed on all Obstap/compliant services. Therefore  a 
>>>> limited set of mandatory
>>>> parameters has been defined covering most of the Initially  proposed use 
>>>> cases.
>>>> 
>>>> However a few of them (polarization data, Cubes with Radial  Velocity, 
>>>> specific dataproduct subtypes, etc...) were missing.
>>>> A small set of additional optional parameters have been defined in  the 
>>>> document. (Some new ones have been proposed
>>>> in the current discussion)
>>>> 
>>>> I see two interests in these optional fields:
>>>>    * complete the description and help the data discovery phase  by 
>>>> displaying more complete descriptions of the data sets
>>>>
>>>>    * extend the Obstap Query mechanism for the benefit of small 
>>>> specialized communities...
>>>> ADQL query on optional field will make no harm to "minimal" Obstap 
>>>> services and will return no
>>>> data from them. But data providers of these kind of data will  probably 
>>>> fill these optional
>>>> fields for a better exposition of them, and data consumers may use  them 
>>>> in their queries if they are interested  and get a chance to  discover 
>>>> some .....
>>>> 
>>> --
>>> Juande Santander Vela
>>> Software Engineer, ALMA Archive Subsystem
>>> Data Flow Infrastructure Department, Software Development Division
>>> European Southern Observatory (Germany)
>>> 
>>> George Carlin: Si es cierto que la Humanidad está sola en el  Universo, 
>>> habría que decir que el Universo no tenía miras muy  altas, y se conformó 
>>> con bastante poco.
>>> 
>>> 
>
>
>
>