Obstap / about optional fields

[Juande Santander Vela]

Perhaps it has eluded me and is already discussed in the ObsCore document, but I cannot recall having read it.

I have the following problem with optional fields: when we say that a given ObsTAP field is optional, what does exactly mean:

a) The field is not mandatory in the ivoa.ObsCore table,  and 
   therefore no entries exist in TAP_SCHEMA.columns with 
   table_name = "ivoa.ObsCore".

b) The field is mandatory in the ivoa.ObsCore table, but the
   content for all rows is NULL.

The problem with approach a) is that we are defeating the ability of sending the same ADQL query in all ObsTAP compliant services, unless the ObsTAP implementations return NULL by default to columns they do not know about in a given schema.

So, for solving that issue, I see three approaches:

1) TAP services return NULL for columns they know nothing about. 
   This seems like a hack, and an extra burden for TAP 
   implementations.

2) Take approach b) above. But then we would have to check
   that the semantics for NULL do not interfere with anything
   we have already in place.

3) Group optional fields, so that they exist or not together
  (even if some of them need to be NULL), and the first discovery
   of relevant ObsTAP services is made through capabilities search
   in the registry.

I think 3) is more natural: we have more information in the registry for service discovery, and we ensure that the fields exist for the use case we are interested about.

What do you think?

Am 17/03/2011 um 00:14 schrieb François Bonnarel:

> It was admitted a long time ago that the obstap parameters should alllow the same ADQL
> query to be performed on all Obstap/compliant services. Therefore a limited set of mandatory
> parameters has been defined covering most of the Initially proposed use cases.
> 
> However a few of them (polarization data, Cubes with Radial Velocity, specific dataproduct subtypes, etc...) were missing.
> A small set of additional optional parameters have been defined in the document. (Some new ones have been proposed
> in the current discussion)
> 
> I see two interests in these optional fields:
>     * complete the description and help the data discovery phase by displaying more complete descriptions of the data sets
> 
>     * extend the Obstap Query mechanism for the benefit of small specialized communities...
> ADQL query on optional field will make no harm to "minimal" Obstap services and will return no
> data from them. But data providers of these kind of data will probably fill these optional
> fields for a better exposition of them, and data consumers may use them in their queries if they are interested  and get a chance to discover some ..... 
>  

--
Juande Santander Vela
Software Engineer, ALMA Archive Subsystem
Data Flow Infrastructure Department, Software Development Division
European Southern Observatory (Germany)

George Carlin: Si es cierto que la Humanidad está sola en el Universo, habría que decir que el Universo no tenía miras muy altas, y se conformó con bastante poco.