[Observation] relation to Dataset
Douglas Tody
dtody at nrao.edu
Thu Nov 21 09:54:26 PST 2013
There is no question that real data has this many-to-many mapping: an
observation may have multiple datasets, and an individual dataset may be
produced from mutliple observations or other datasets. However, this
does not mean that the Observation model for example, needs to itself
describe more than a single observation or dataset. The OBS_ID
attribute in ObsCore for example already provides some simple support
for multiple datasets within a single observation. Provenance could in
the future provide a way to determine the progenitor datasets used to
compute a dataset, as well as the operation performed.
I just suggest that we need to think more broadly about this. The
solution may involve relationships between more than one model. ObsCore
may already provide a sufficient solution for the more narrow problem it
is dealing with.
- Doug
On Thu, 21 Nov 2013, François Bonnarel wrote:
> Hi all,
> I think I agree with Arnold here. More, I suspect the dataset to
> observation relationship could be M<------>N relationship.
> - Optical CCD mosaics (eg MEGACAM images archived at CADC) are datasets
> made of several observations (independant reduction for each CCD ?)
> - On the other side an "observation" can be made oif several datasets. We
> have studied the CTA/HESS case (Gamma ray Cherenkov obsevatories)
> An "observation" (one night, one target) is made of several "runs" (or
> some other could say exposures) each rendered by one or several
> "datasets" (= event list) . Something like that seem to exist in Xray
> domain if I'm not mistaking ?
> A consequence of this could be that an observation is made of several
> datasets
>
> So in the DataSet Class, the ObservationID attribute should be a list
> And in OBservation Class, DatasetID should also be a list.
>
> Best regards
> François
> Bonnarel
>
> Le 15/11/2013 15:59, Arnold Rots a écrit :
> If multiple observations have to be taken care of through
> provenance,
> then why should a single observation not be handled the same way?
> Don't get me wrong: I think neither should be handled through
> provenance.
>
> Examples are: VLA multi-configuration images; stacked images;
> multi-observation event files.
>
> It is much clearer and more intuitive if we just simply allow a
> Dataset
> to be associated with multiple Observations.
> Actually, I think this is absolutely a requirement.
>
> - Arnold
>
> --------------------------------------------------------------------------------
> -----------------------------
> Arnold H. Rots Chandra
> X-ray Science Center
> Smithsonian Astrophysical Observatory tel: +1
> 617 496 7701
> 60 Garden Street, MS 67 fax:
> +1 617 495 7356
> Cambridge, MA 02138
> arots at cfa.harvard.edu
> USA
> http://hea-www.harvard.edu/~arots/
> --------------------------------------------------------------------------------
> ------------------------------
>
>
>
> On Thu, Nov 14, 2013 at 6:29 PM, Douglas Tody <dtody at nrao.edu>
> wrote:
> On Thu, 14 Nov 2013, Arnold Rots wrote:
>
> >From this description I am
> beginning to suspect that a
> Dataset can be
>
> derived from
> (associated with) no more than one
> Observation.
> That seems utterly wrong; multiple
> Observations can be combined into a
> single Dataset.
> Or did I misunderstand?
>
>
> Multiple Observations can be and often are combined to
> produce a new
> Dataset, however describing that history would be likely be
> the
> responsibility of the Provenance model. At the level of
> Observation it
> would probably be a new "Observation" (or at least Dataset).
> Depends
> upon how strict we are with the concept of Observation. The
> CreationType and calibration level say something about it
> being a
> synthesized/derived data product.
>
> I think it is OK to require that a Dataset is
> associated with at least one
> Observation,
> provided that a model or simulation can be
> described as an Observation.
>
>
> In practice that is what we are doing, to keep things simple;
> DataSource
> can be something like "theory".
>
> - Doug
>
> Cheers,
>
> - Arnold
>
> --------------------------------------------------------------------------------
> -----------------------------
> Arnold H. Rots
> Chandra X-ray
> Science Center
> Smithsonian Astrophysical Observatory
> tel: +1 617 496
> 7701
> 60 Garden Street, MS 67
> fax: +1 617
> 495 7356
> Cambridge, MA 02138
> arots at cfa.harvard.edu
> USA
> http://hea-www.harvard.edu/~arots/
> --------------------------------------------------------------------------------
> ------------------------------
>
>
>
> On Thu, Nov 14, 2013 at 12:08 PM,
> CresitelloDittmar, Mark <
> mdittmar at cfa.harvard.edu> wrote:
>
> All,
> This thread is for discussion on
> the relation between Observation and
> Dataset.
>
> ref: ObsCoreDM -
> http://www.ivoa.net/documents/ObsCore/20111028/index.html
> ref: diagram illustrating relation of
> Image/Spectral Observation to
> ObsCoreDM (draft)
>
> http://www.ivoa.net/pipermail/dm/attachments/20131113/c9ef7581/attachment-0001.p
> ng
>
> motivation
> It is clear that there is a
> relationship between "Observation"
> and a
> more generic "Dataset". This
> "Dataset" would contain elements such
> as the
> dataProductType, and
> dataProductSubtype, presumably
> others. This object
> has not been formally defined.
>
> In ObsCore, there is an implied
> relationship for Observation as an
> Extension of Dataset in the location
> of these attributes. So, I have
> always interpreted that Observation
> "is" a Dataset. This is reflected in
> my choice of the name
> "ObservationDataset" in the left hand
> package of my
> diagram. It implies that it is a
> Dataset extended for Observation
> purposes.
>
> Recent discussion brings this
> relationship into question, with
> assertions that an Observation can be
> associated with 0 or more Datasets.
>
> This has real ramifications for the
> Image and Spectral models..
>
> Seed:
>
> If the relation is Observation "has"
> 0..* Dataset, then all the diagrams
> to date are wrong.
> It feels like this would be a
> fundamental change to all these
> models.
>
> - there would need to be a
> bi-directional relation between
> Observation
> and Dataset
> (observation has 0..* Dataset;
> Dataset associated with 1
> Observation)
> Hmm.. since there can be Datasets
> not associated with Observations,
> this would
> need to be a specialization of
> Dataset.. (ObservationDataset.. but
> not
> the one in my diag.)
>
> - the Char associated with
> Observation would characterize the
> total
> space of all included Datasets.
> (0..1) relation to Observation. If
> no
> Datasets, no Char
>
> - each Dataset would require it's
> own Characterisation, specific to
> it's
> space.
> (so there is another attribute
> for Dataset).
>
> - we would need to specify which of
> the elements are associated to the
> Dataset, and which to the
> Observation. e.g. DataModel =>
> Dataset; Target
> => Observation
>
> Thoughts?
> Mark
>
>
>
>
>
>
>
>
More information about the dm
mailing list