Revised ImageDM-ObsCore architecture

[Douglas Tody]

On Wed, 13 Nov 2013, CresitelloDittmar, Mark wrote:

> The "ObsCore" class of the right hand side corresponds to the Observation
> class of the ObsCore model document.  Future versions of the diagram will
> restore that name.  The left hand side (ObservationDM) is the Observation
> model used by Spectral, and presumably ImageDM.  The goal of the diagram
> is to show exactly what we need outside of ObsCore to support those
> models.
> Explicitly stating the linkage.
>
> The best description of ObsCore is that it should be a 'view' of the
> larger
> Observation model which serves ObsTap... so while the diagram may be
> ugly, I think it shows the relationships and provides all the information
> needed
> to inform any request to modify the Observation model.

None of this was obvious to me from the diagram.  It is clear that the
general root model is Observation, rather or not this is adequately
documented in our original ObsCore/ObsTAP document (there just wasn't
enough time, it took 1-2 years as it was).

I like the characterization of ObsCore as a "view" of Observation; that
is what it is.

> On Wed, Nov 13, 2013 at 8:10 PM, Douglas Tody <dtody at nrao.edu> wrote:
>       Just to amplify on this point: in ObsTAP (ObsCore), it is
>       permitted
>       for a record to describe either an Observation or a specific
>       dataset
>       (data product).  So for example the Observation may be
>       described, and a
>       "package" can be downloaded containing multiple data
>       products.  Or the
>       individual datasets (data products) may be exposed, linked by
>       a common
>       obs_id.
> 
> 
> This seems to be echoing section 3.3.3 of the document.  If I understand
> the doc
> and diagrams, the only way to get multiple individual data products from
> the same
> observation is to have multiple instances, each having the same obs_id
> value.
> Alternatively, one can have a tar file of products, but then the
> observation details
> cannot be reported in detail for each product.

Correct.  But in model terms there is still one Observation with
multiple data products / datasets.  We have multiple instances because
we are mapping it into table records using obs_id.  If the serialization
were hierarchical then the association would be more obvious (not that I
am promoting that view).  The higher level model is still an Observation
with multiple data products.

> I don't see an option for having 1 Observation with >1 Datasets fully
> represented.

This is easily done via association in (for example) a query response.

>        Or some combination of the two.  So there is a distinction
>       between an observation and a dataset.  An observation may
>       have multiple
>       datasets.  An individual dataset has a type, a subtype, a
>       calibration
>       level, a size, and so on.  (it is also possible to have
>       datasets that
>       are not observations).  - Doug
> 
> 
> I don't think there is a use case for a Dataset that is not associated
> with an Observation..
> The above is saying that Observation "has" 0+ Datasets.. rather than
> Observation "is" a Dataset.

A Theory / simulation dataset for example has nothing to do with an
Observation.  This is why "Dataset" is a more general concept than
"Observation", just as "Image" is more general than for example a
traditional image of the sky measuring flux.  But most of the standard
metadata is the same in either case so in practice we use the same model
for both.

 	- Doug