release of a new version for the WD Observation Core Components data model and ObsTAP

[Douglas Tody]

On Wed, 9 Mar 2011, Anita M. S. Richards wrote:

> p14 Table 1 and elsewhere
>
> What about Solar System or other data which cannot be described at all by RA 
> and Dec (because the body moves too fast, for example?) For example, many 
> ALMA observation sets will contain a moon as amplitude calibrator along with 
> extragalactic sources.  It would be nice to have an option e.g. 'coosys 
> other' if not all the options.  Alternatively, one could allow RA and Dec to 
> be nil and identify Solar System objects by name and time-date.

A target name query should still work in this case.

> p15 3.3.1 Data Product Type
> Is it worth stating explicitly that visibility data is likely to be in 
> formats such as FITS, Measurement Sets (MS) or Science Data Models (SDM, ASDM 
> etc.) (usually distributed as tar or zip directories) - just so that 
> implementors can recognise the type of the latter, relatively new formats?

ObsTAP per se does not specify the possible file formats, it just allows
them to be described.  The plan is to address this file format issue
mainly in the access_format specification.  This is still being
specified, but we are in the process of looking at ALMA, EVLA and others
to see how well they map onto {collection, dataproduct_type/subtype,
calib_level, access_format} which should be sufficient to fully specify
what a data product is.

> p17 3.3.3
>
> I find that the discussion in paras 3, 4 adds confusion to the nice clear 
> description in 3.3.2 - why are aggregates of multiple files limited to levels 
> 0 or 1 (e.g. MS, CASA-format images are directories but can be completely 
> calibrated and science-ready or even advanced products)? Surely the 
> approaches adopted are up to the provider.  Either cut it down or leave it 
> out, or replace it with more fully described  examples from specific 
> archives?

If the "data product" (tar, directory, etc.) being described is a
collection of instrument-specific files, regardless of their individual
calibration level, it is still an instrument-specific data product.
Hence probably level 1.  The instrument signature has to be removed to
get to level 2-3.  Level 1 may be calibrated, it is whether it is an
insrument-specific data product which is the issue here.

If instead of a tar or a dir the individual data products are exposed
then they can have separate calibration levels.  An image or cube for
example could be level 2-3.  But if they are all together as an
instrument "observation" grouping (tar or dir) then they are probably
instrument specific.  Of course, it is up to the DP to make the final
decision on the calibration level.

> I am not sure that a combination of real observations is a 'software' 
> observation - I would reserve that term for simulated or model data.  Back to 
> real combinations, surely it is up to the provide whether it gets a new ID, 
> but if the purpose of the ID is to allow tracing the observational history 
> this seems counter-productive.  Omit this para?

The point was that we talk about "observation", but a composite of
multiple actual observations is a more complex derived product, and we
want to describe that here too.  Perhaps the wording could be improved;
we will revisit it in light of your comments.

> 4.6 Does 'data product' include metadata-only responses, e.g. when the 
> product is very large and the user should be warned or the data need staging 
> or user registration at a web form or whatever? (the Introduction does 
> mention 'retrieving or otherwise accessing')

ObsTAP does not address these more complex access use cases.  More
generally, ObsTAP does not do virtual data.  This is reserved to the
typed interfaces such as SIAV2.  ObsTAP describes only "archival"
(actual) data products in an archive.  These can be directly retrieved
as files, but to do anything more complex the typed DAL interfaces must
be used.

 	- Doug