[QUANTITY] and [OBSERVATION] data models: new drafts

Fri Apr 23 16:35:37 PDT 2004

Dear DM  partners,
    As I mailed allready I am a little concerned about the lack of reactions
on the two drafts: Quantity and Observation. Not to speak about STC!
    Because even if there is no real opposition to these drafts, are we sure 
that the people  will use this work if they are not convinced?
    So let me tell you a few questions I have , specially about Observation.
   where I participated 
   A ) Generalities on Data models 

    In a private mail Gerard Lemson  introduced concerns  on the Observation
data model 
by noticing the lack of relation of our work with a more general data
model:
-------------------------------------------------------------------------
Gerard:
I do maintain that the current model should be viewed as one 
"view/representation/denormalization" of a more fundamental model. Such a 
more fundamental model was what I set out to define in the domain model.
-----------------------------------------------------------------

     Is this point of view similar to Tom McGlynn's and Brian Thomas'
one, two weeks ago, on the DM List? (although Tom's one appeared more 
functionnal than the "derivation" paradigm of Gerard)
---------------------------------------------------------------------------
Tom:
As you may see this is completely different in approach.
We haven't even begun to discuss the VOTable.   First we have
defined the high level abstract data models which the data can satisfy.
Only now do we even begin to build the structure for defining concrete
instantiations of this data model. (Tom)

and      

Brian:
Simply put, thats just not the reality of the astrophysical concepts. Many 
(if not most) concepts belong to more than one group, and when they belong 
to a group, their relationship may be qualified (ex. I belong to this group 
under the following conditions..)
        As a result, I don't see how pushing the strings as id's/concepts 
within UCD is going to be effective in the long run in terms of meeting VO 
requirements. We need to take a more advanced approach, such as has been 
frequently suggested by UCD3. (Brian)

with this answer of Jonathan to Tom:

Jonathan:
Right, and that's a fine approach - but we *have* a VOTable, and
I think it will help VOTable users understand data models if
we see where it sits in the currently defined
`high level abstract data models which the data can satisfy.'
Your complaint is that we have done SIA backwards, creating the
serialization before creating the data model. I agree, but that's
what we did, (and we had to, for speed)
- I don't see this as an argument against discussing
how it maps to the model.

and 
Jonathan:
Yes, but I think it is more useful to think of concrete serializations
as representing a data model that is close to their actual structure.
In other words, an event list "satisfies the image array data model"
via a transformation that bins the list on the fly, and it "satisfies
the event catalog model". But I think it is far from meaningless to
work in terms of specific models that reflect the data storage structure,
as well as in terms of higher level models that are just interfaces.

> - Defining the high level interfaces for spectra, images and time series.
> - Defining the high level interfaces for basic transactions between VO
> entities.

I think these are essentially the Observation and Quantity model documents
respectively.

_______________________________________________________________________________

Now are coming my two first questions: are we really facing a pyramid of 
datamodels ? From very general to really specific ? Observation being 
somewhat in the middle? But how far arre we in all these  ?

   For me IVOA Observation draft data model was a certain level of abstraction 
and generallity higher than previous attempts and could deal with data 
providers needs as well as data processing ones. If We can consider that I am 
on the data provider side, I feel rather well with the draft as it is. 
What about the other people on the list?

   The second general point is: What is the relationship between a Gerard's 
like point of view and a Tom's like point of view. Gerard or Tom would you
 like to comment on that?

  B ) UCDs in data model

----------------------------------------------------------------------------
Gerard private point of view:
In that model UCDs are themselves derived quantities, expressible in terms
of domain model entities. UCDs are useful in that they give a shorthand,
data dictionnary kind of description of common concepts. If one wants to 
to think of what UCDs should be added, I think in fact that the distinction
between a corrected or a raw observable value is as important as wheter
we know how that value is expressed.

   How easily one answers the questions "what is the data now" is a question 
of implementation. What it comes down to is what views/queries the  archives
support. Yes it seems that adding a singlre UCD for a particular phenomenon
to a certain dataset is easy. What is not easy is to anticipate all the 
possible questions someone may want to ask of a dataset and have UCDs for
all of them. This is not argument against UCDs and I think this particular
example definitly warrants discussion with the UCD group
--------------------------------------------------------------------------

I am not sure to understand this "definition" of UCDS. Or maybe you say
"UCD" and think "utype" ? UCDs are concept somewhat independant from any
kind of datamodel and loose enough to be used for comparisons and cross-
matching of quantities of various provenances. How can they be derived
from the datamodel? Except if we have the omniscient Astronomy wide data
model ???

    C ) Hierarchical  Structure and Data Tree 

-------------------------------------------------------------------------
  Jonathan's private point of view:

This is just meant to be the AVO/IDHA demo thing. So yes, tree-like
was in my mind. That's the most typical way people visualize archives
right now. I agree the actual structure can be modelled in a more
complex way. The model doesn't prevent that but I don't have a specific
use case requiring it to hand.

and

Ch 3 "Data Tree"  At Garching several people (e.g. Anita Richards)
emphasized that they found the IDHA tree model extremely
useful and that they are expecting to see it in some form
in the output of the DM group. So here it is.
------------------------------------------------------------------------

    Tree or Hierarchy have to be seen at the viewing method level, as
it is dicussed now on the list around "SIAP Evolution" and "Structured SIAP"
The very nature of the model (and the data) is indeed graph-like, but user
defined point of view may be tree-like.
      Tree-like has more to do with the request and interpretation than
with the data themselves.

     D )   Serialization ?

    What kind of serialization are we looking for ? In a SIAP or "AVO demo"
oriented perspective, we are looking for a serialization which is a 
standardized data description. But I think the full XML serialization in 
the Quantity/Brian/Ed perspective is serialization for the data themselves
(replacing older data formats like FITS).

Other question: is the serialization controlled by Developper/Astronomer like
in the various VOTable serialization proposals or automatically generated?

     And for serialization we need attributes: Jonathan defined a few of them
by defining utypes for SIA. Should we go on before the next version of the
draft? 

     E ) Packaging

     This is somewhat related to the previous point. If we are using the
datamodel for data description, we need a description of the data which
will come next? Is Quantity what we need for that ? What about the coding
compression/format stuff? Don't we have to add a packaging class to the
draft ?

   Just to be discussed here.

Cheers
François

SAUVONS LA RECHERCHE :  <http://recherche-en-danger.apinc.org/>

=====================================================================
Francois   Bonnarel               Observatoire Astronomique de Strasbourg
CDS (Centre de donnees          11, rue de l'Universite
astronomiques de Strasbourg)    F--67000 Strasbourg (France)

Tel: +33-(0)3 90 24 24 11       WWW: http://cdsweb.u-strasbg.fr/people/fb.html
Fax: +33-(0)3 90 24 24 25       E-mail: bonnarel at astro.u-strasbg.fr
---------------------------------------------------------------------