[Observation] relation to Dataset
Pierre Didelon
pdidelon at cea.fr
Fri Nov 22 07:16:39 PST 2013
I agree, for me to it is no longer an observation... but a data product,
which processing history has perhaps to be kept in memory...
DataProduct Id is perhaps more appropriate/convenient
observation has/is a data product, but dataproduct (from simulation,
engenerring...) is/corresponds not always an observation :-/
PiR
Le 22/11/2013 16:04, Arnold Rots a écrit :
> I strongly object to this statement:
>
> "the data product may be the result of combining data from multiple
> primary (physical) observations. In this case the resulting data
> product is a new processed "observation" to which a new unique
> observation identifier should be assigned."
>
> We really need to distinguish clearly between Datasets and Observations.
> An Observation represents an operation that is characterized by a
> configuration
> - instrument characteristics, coordinate volume and properties,
> calibration, etc.
> A Dataset is a container of bytes that may have resulted from an
> Observation
> (the byte stream that came out of the telescope or various direct
> processing
> products of it), a simulation, or the processing and analysis of
> (possibly a subset)
> of one or more parent Datasets.
> Each Dataset also carries metadata detailing coordinate
> characteristics, the nature
> of the Dataset and its components, and its provenance regarding its
> parents.
>
> Blurring the line between Observations and Datasets and carelessly
> forcing one
> to assume the characteristics of the other is going to get us into
> major trouble.
>
> Cheers,
>
> - Arnold
>
> -------------------------------------------------------------------------------------------------------------
> Arnold H. Rots Chandra X-ray Science Center
> Smithsonian Astrophysical Observatory tel: +1 617 496 7701
> 60 Garden Street, MS 67 fax: +1 617 495 7356
> Cambridge, MA 02138 arots at cfa.harvard.edu <mailto:arots at cfa.harvard.edu>
> USA http://hea-www.harvard.edu/~arots/
> <http://hea-www.harvard.edu/%7Earots/>
> --------------------------------------------------------------------------------------------------------------
>
>
>
> On Thu, Nov 21, 2013 at 6:00 PM, CresitelloDittmar, Mark
> <mdittmar at cfa.harvard.edu <mailto:mdittmar at cfa.harvard.edu>> wrote:
>
> All,
>
> I've been thinking about this and some comments Arnold made on the
> Provenance thread which are closely related.
> 1) there is general agreement that Observation *has* 0 or more
> Datasets (rather than *is* a Dataset)
>
> 2) Dataset can exist without an Observation (can be created by
> something else).
>
> 3) The definition of Observation is pretty fuzzy, but lets
> assume that there could be an "Analysis" or "Simulation" step
> which could create a Dataset. These may be parts of the larger
> domain that all these objects live in, but are not modeled.
> Currently, the ObsCore model does say (pg 19) "the data product
> may be the result of combining data from multiple primary
> (physical) observations. In this case the resulting data product
> is a new processed "observation" to which a new unique observation
> identifier should be assigned."
> So the relation of Dataset to 'the thing which created it', is not
> clear to me yet. I keep going back to the 'Experiment' concept in
> Gerard's mail (provenance thread).
>
> I don't think that a Dataset should have a bi-directional relation
> to the full Observation(s) as I noted at the head of this thread,
> but should
> a) have an association back to components of the Observation (
> ObsConfig, Proposal ) which become part of the Dataset 'provenance'.
> (which is what I think Arnold was saying in the other thread).
> b) have metadata identifying the relevant Observation(s)
> comprising Dataset (DataID.ObservationID), as Francois notes.
> but this gets tricky because ObsCore expects a singular
> (well unique) obs_id for each Dataset.
> c) if the Dataset were created by something else, then it would
> add associations to components of those things holding the
> relevant information to fold into the 'provenance'. Like the
> progenitor Datasets.
>
>
>
>
> On Fri, Nov 15, 2013 at 9:59 AM, Arnold Rots
> <arots at cfa.harvard.edu <mailto:arots at cfa.harvard.edu>> wrote:
>
> If multiple observations have to be taken care of through
> provenance,
> then why should a single observation not be handled the same way?
> Don't get me wrong: I think neither should be handled through
> provenance.
>
> Examples are: VLA multi-configuration images; stacked images;
> multi-observation event files.
>
> It is much clearer and more intuitive if we just simply allow
> a Dataset
> to be associated with multiple Observations.
> Actually, I think this is absolutely a requirement.
>
> - Arnold
>
> -------------------------------------------------------------------------------------------------------------
> Arnold H. Rots Chandra X-ray Science Center
> Smithsonian Astrophysical Observatory tel:
> +1 617 496 7701 <tel:%2B1%20617%20496%207701>
> 60 Garden Street, MS 67 fax: +1 617 495 7356
> <tel:%2B1%20617%20495%207356>
> Cambridge, MA 02138 arots at cfa.harvard.edu
> <mailto:arots at cfa.harvard.edu>
> USA http://hea-www.harvard.edu/~arots/
> <http://hea-www.harvard.edu/%7Earots/>
> --------------------------------------------------------------------------------------------------------------
>
>
>
> On Thu, Nov 14, 2013 at 6:29 PM, Douglas Tody <dtody at nrao.edu
> <mailto:dtody at nrao.edu>> wrote:
>
> On Thu, 14 Nov 2013, Arnold Rots wrote:
>
> From this description I am beginning to suspect
> that a Dataset can be
>
> derived from
> (associated with) no more than one Observation.
> That seems utterly wrong; multiple Observations can be
> combined into a
> single Dataset.
> Or did I misunderstand?
>
>
> Multiple Observations can be and often are combined to
> produce a new
> Dataset, however describing that history would be likely
> be the
> responsibility of the Provenance model. At the level of
> Observation it
> would probably be a new "Observation" (or at least
> Dataset). Depends
> upon how strict we are with the concept of Observation. The
> CreationType and calibration level say something about it
> being a
> synthesized/derived data product.
>
>
> I think it is OK to require that a Dataset is
> associated with at least one
> Observation,
> provided that a model or simulation can be described
> as an Observation.
>
>
> In practice that is what we are doing, to keep things
> simple; DataSource
> can be something like "theory".
>
> - Doug
>
>
> Cheers,
>
> - Arnold
>
> -------------------------------------------------------------------------------------------------------------
> Arnold H. Rots Chandra X-ray
> Science Center
> Smithsonian Astrophysical Observatory
> tel: +1 617 496
> 7701
> 60 Garden Street, MS 67
> fax: +1 617
> 495 7356
> Cambridge, MA 02138
> arots at cfa.harvard.edu <mailto:arots at cfa.harvard.edu>
> USA
> http://hea-www.harvard.edu/~arots/
> <http://hea-www.harvard.edu/%7Earots/>
> --------------------------------------------------------------------------------------------------------------
>
>
>
> On Thu, Nov 14, 2013 at 12:08 PM, CresitelloDittmar,
> Mark <
> mdittmar at cfa.harvard.edu
> <mailto:mdittmar at cfa.harvard.edu>> wrote:
>
> All,
> This thread is for discussion on the relation
> between Observation and
> Dataset.
>
> ref: ObsCoreDM -
> http://www.ivoa.net/documents/ObsCore/20111028/index.html
> ref: diagram illustrating relation of
> Image/Spectral Observation to
> ObsCoreDM (draft)
>
> http://www.ivoa.net/pipermail/dm/attachments/20131113/c9ef7581/attachment-0001.png
>
> motivation
> It is clear that there is a relationship between
> "Observation" and a
> more generic "Dataset". This "Dataset" would
> contain elements such as the
> dataProductType, and dataProductSubtype,
> presumably others. This object
> has not been formally defined.
>
> In ObsCore, there is an implied relationship for
> Observation as an
> Extension of Dataset in the location of these
> attributes. So, I have
> always interpreted that Observation "is" a
> Dataset. This is reflected in
> my choice of the name "ObservationDataset" in the
> left hand package of my
> diagram. It implies that it is a Dataset extended
> for Observation purposes.
>
> Recent discussion brings this relationship into
> question, with
> assertions that an Observation can be associated
> with 0 or more Datasets.
>
> This has real ramifications for the Image and
> Spectral models..
>
> Seed:
>
> If the relation is Observation "has" 0..* Dataset,
> then all the diagrams
> to date are wrong.
> It feels like this would be a fundamental change
> to all these models.
>
> - there would need to be a bi-directional
> relation between Observation
> and Dataset
> (observation has 0..* Dataset; Dataset
> associated with 1
> Observation)
> Hmm.. since there can be Datasets not
> associated with Observations,
> this would
> need to be a specialization of Dataset..
> (ObservationDataset.. but not
> the one in my diag.)
>
> - the Char associated with Observation would
> characterize the total
> space of all included Datasets. (0..1) relation
> to Observation. If no
> Datasets, no Char
>
> - each Dataset would require it's own
> Characterisation, specific to it's
> space.
> (so there is another attribute for Dataset).
>
> - we would need to specify which of the elements
> are associated to the
> Dataset, and which to the Observation. e.g.
> DataModel => Dataset; Target
> => Observation
>
> Thoughts?
> Mark
>
>
>
>
>
>
>
--
Pierre
-------------------------------------------------------------------
DIDELON :@: pdidelon_at_cea.fr Phone : 33 (0)1 69 08 58 89
CEA SACLAY - Service d'Astrophysique 91191 Gif-Sur-Yvette Cedex
-------------------------------------------------------------------
Aidez les enfants Tibétains : http://www.a-e-t.org/actions/
d'autres : http://www.sosesf.org/ ou avec une autre ONG
-------------------------------------------------------------------
- C'est de l'espérance que naît le désespoir.
Haruki Murakami - La fin des temps
-------------------------------------------------------------------
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.ivoa.net/pipermail/dm/attachments/20131122/291d51fb/attachment-0001.html>
More information about the dm
mailing list