[obs-tap]:updates on the Proposed recommendation

[Arnold Rots]

This is becoming unwieldy.
Trying to make X-ray data (and I suspect the same is true for aperture
synthesis data) fit into something that is designed with optical
images in mind is reminiscent of round pegs and square holes.

Service providers are free to define subtypes and titles, but you are
saying that if they don't follow rules that are not spelled out,
things won't work as envisaged.
Also, if I understand the argument correctly, if data discovery
software is to be helpful at all, it needs to be able to extract some
information from the title field - but that is intended for human
consumption.

If I see this, it looks like I need to generate at least eight records
for a single observation, some containing a mix of levels, and all
duplicating pretty much the same metadata.

This is not going to make it attractive to provide ObsTAP services.


Maybe I should do what you did and provide an example of how I thought
it should have worked.

Here is how I would envisage data discovery of Chandra data to work:
  A single record per Obsid that provides the observational metadata and:
    ObsId
      12345
    Dataset Identifier
      ivo://ADS/Sa.CXO#obs/12345
    Data Types available
      Package
      Event list
      Image
    Calibration level
      2
    Title
      Chandra/ACIS ObsId 12345

Then a data access protocol that allows querying the archive using any
of the above in a where clause, with either ObsId or DID required, and
returning:
  ObsId  DataType   Contents   Level   Format      URL
  -----------------------------------------------------------
  12345  Pkg_1      evt,img    2       tar         http://...
  12345  Pkg_2      evt,img    1       tar         http://...
  12345  Pkg_12     evt,img    2,1     tar         http://...
  12345  evt        evt        2       fits-bin    http://...
  12345  evt        evt        1       fits-bin    http://...
  12345  img        img        2       fits        http://...
  12345  img        img        2       jpg         http://...
  12345  img        img        2       fits        http://...
  12345  img        img        2       jpg         http://...
This is an example where the client specified ObsId or DID, but no
data type or format.

Never mind the terms and abbreviations I used - you get the picture.

Cheers,

  - Arnold


Douglas Tody wrote:
> More precisely what you might have is something like (display in a wide view):
> 
>      ObsId     Type     Subtype               Level     Format                         Title
>      ----------------------------------------------------------------------------------------------------------
>      123      event    chandra.hrc.pkg         1      application/x-tar-gzip   Chandra ACS-XYZ observation package (event,refimage)
>      123      image    chandra.hrc.refimage    2      image/fits               Chandra ACS-XYZ reference image
>      123      image    chandra.hrc.preview     2      image/jpeg               Chandra ACS-XYZ preview image
>      345      event    rosat.foo.pkg           1      application/x-tar-gzip   ROSAT whatever observation package (xxx)
> 
> and so forth.  The subtype could in principle be more generic but will
> likely be instrument-specific for a level 1 observation.
> 
> The Title should concisely describe the data product, e.g., origin,
> instrument, ID, what it is (observation package, calibration, standard
> view, etc.).  The title string is what one normally wants to output on a
> displayed image or plot to identify to a human the data being shown.
> You can put whatever you want in there to describe the data product so
> long as it is concise (one line of text).
> 
>          - Doug
> 
> 
> 
> 
> On Mon, 11 Jul 2011, Douglas Tody wrote:
> 
> > On Thu, 7 Jul 2011, Arnold Rots wrote:
> >
> >> Aside from what I reported in a previous message, quoted below, there
> >> are more discrepancies between Table 5 and Tables 6 and 7:
> >> 
> >> obs_creator_did is missing from Table 7
> >> o_units in Table 5 should be o_unit
> >> pol_states is missing from Table 6
> >> facility_name and instrument_name are spelled differently;
> >>  even though required, they show up in Table 7, rather than 6
> >> em_unit is missing from Table 5
> >> o_stat_error is missing from Table 7
> >> 
> >> Also, note the comment I made on MJD in use case 1.6
> >> and on the uselessness of bib_reference because of its murky
> >> definition
> >> 
> >> I still lament the fact that the data access functionality is
> >> compromising the self-consistency and usefulness of the data discovery
> >> function, but decided for our tarred packages to use:
> >>  dataproduct_type = NULL
> >>  dataproduct_subtype = package:event,image
> >>  access_format = application/x-tar
> >> As far as I can tell, this is within the specifications.
> >
> > Well we don't specify what the subtypes you provide for your archive
> > should be so I suppose you could get away with this, but this example is
> > not at all what we had in mind.  The subtype should be the science type
> > of the specific data product, *not* details about the content of the
> > data product.  I would expect the type to be "event" (meaning "event
> > data" not "event list") and the subtype to be something more like
> > "chandra.hrc.package", "chandra.hrc.refimage (or "rosat.XX" etc.).
> >
> > Note subtypes are supposed to be fixed strings so that one can search
> > the local archive for a particular type of data product; if you try to
> > describe what is included in a particular data product then such
> > selection won't be possible.  So for example a client will do a generic
> > query to see what subtypes Chandra defines, and then they can pose a
> > more specific query to get a certain type of Chandra-specific data
> > product.  Likewise for ALMA etc.
> >
> > Note you also have obs.title where you can provide a short description
> > of the data product and for this you can provide whatever you want.
> >
> >       - Doug
> >
> 
--------------------------------------------------------------------------
Arnold H. Rots                                Chandra X-ray Science Center
Smithsonian Astrophysical Observatory                tel:  +1 617 496 7701
60 Garden Street, MS 67                              fax:  +1 617 495 7356
Cambridge, MA 02138                             arots at head.cfa.harvard.edu
USA                                     http://hea-www.harvard.edu/~arots/
--------------------------------------------------------------------------