[timeDomain: model for Time series] discussion on Timeseries Data model Note / how are ND-Cube DM and Timeseries DM connected ?

François Bonnarel francois.bonnarel at astro.unistra.fr
Fri Jul 14 16:01:34 CEST 2017


Dear all,


Le 14/07/2017 à 13:39, Jiří Nádvorník a écrit :
>
> Hi,
>
> Thank you, see inline.
>
> Jiri
>
> *From:*François Bonnarel [mailto:francois.bonnarel at astro.unistra.fr]
> *Sent:* Thursday, July 13, 2017 11:52 PM
> *To:* Jiří Nádvorník <nadvornik.ji at gmail.com>; 
> mireille.louys at unistra.fr; dm at ivoa.net; voevent at ivoa.net; dal at ivoa.net
> *Subject:* Re: [timeDomain: model for Time series] discussion on 
> Timeseries Data model Note / how are ND-Cube DM and Timeseries DM 
> connected ?
>
> Hi Jiri,
>    Thanks for having given a look to these emails. This is a very 
> important discussion and we have to all agree on these grounds before 
> going to more TimeSeries specific features of the model we need.
>
> Le 13/07/2017 à 15:02, Jiří Nádvorník a écrit :
>
>     Hi Mireille,
>
>     Thank you very much for the input.
>
>     Your diagram is almost correct, but I believe that the relationship
>
>     TimeSeriesCube  <is a > NDCubeDM::SparseCubeDataset
>
>     Is not correct, even in the original idea Mark Cresitello Ditmar
>     had (please correct me here if I’m wrong, Mark). The correct
>     relationship is:
>
>     TimeSeriesCube <is a> NDCubeDM::SparseCube and
>
>     NDCubeDM::SparseCube <is collected by> NDCubeDM::SparseCubeDataset
>
>     As seen on the following image:
>
>     Meaning that the SparseCubeDataset is describing a collection of
>     data cubes, e.g., time series data, e.g., light curves, **not**
>     one cube, e.g., one time series, e.g., one light curve. If we
>     agree that we don’t need collections of time series (because they
>     can by themselves be multi-dimensional), we can change it to <is
>     a> relationship as you propose.
>
> OK. This is were we differ. I don't think SparSecubedataset is made 
> for tackling collections. My interpretation of the ND-Cube diagram 
> helped by MCD' text is that SparSeCubeDataset inheritance from 
> ObsDataSet contains all generic metadata for a dataset which may be 
> COMPLEX. That is it contains : curation, provenance, identification, 
> characterisation.
>
> Such  a SparseCubeDataset may contain one (simple dataset) or several 
> (complex dataset) "SparseCubes"
>
> Each of these SparseCube(s) contains ND-points where the actual data 
> are stored and inherits from the DataProduct class the coordinate 
> systems and mappings of the data to physical coordinates.
>
> As Mireille said the "/dataproduct_type/ " of your TimeSeriesCube (or 
> TimeSeriesDataSet) can only be inherited from ObsDataSet through 
> SparseCube Dataset and not from DataProduct via SparseCube.
>
> */[[Jiri Nadvornik]] Ha, so this means that each row we see in ObsCore 
> table is describing a DataSet, meaning Spectrum <is a> DataSet, Image 
> <is a> DataSet, SED <is a> DataSet, event <is a> Dataset, etc.? I can 
> also see a valid option dataproduct_type=cube in the ObsCore document 
> (http://www.ivoa.net/documents/ObsCore/20111028/REC-ObsCore-v1.0-20111028.pdf 
> )/*
>
> *//*
>
/*Yes that is it.*/
>
> */If the assumptions above are correct, I concur that changing the 
> relationship to TimeSeries <is a> Cube <is a> DataSet is not a bad 
> idea. From my point of view, this can be changed in the model rather 
> easily – it will just introduce a more tight coupling between DataSet 
> and TimeSeriesCube data model. TimeSeriesCube data model will need to 
> import everything from the DataSet DM and if something changes in 
> DataSet DM, it will need to change in TimeSeriesCube DM too. In other 
> words, new major versions of DataSet DM will require new major 
> versions of TimeSeriesCube DM to follow./*
>
>
I think DataSet DM is made to be reused in different specific contexts.

Cheers
François
>
>
>
> Cheers
> François
>
>
>
>
>     Now the Time series class is described in the attached UML.pdf
>     (please note that this one is different from the original note,
>     this version was last updated after Shanghai Interop in May). Main
>     difference is that the TimeSerieCubeDM::CubeAxis custom class was
>     replaced just by a generic columnRef (yellow) saying where can I
>     find the data for this axis and that axis is described by Quantity
>     class (yellow).
>
>     The Quantity class indeed provides the **richer description** on
>     the cube axis (not only the time axis). This is indeed correlated
>     by STC2.0::CoordMeasurement, but we got into conflict in here, as
>     we would like to use it not for describing only **uncertainties**
>     in the Measurement, but for statistical distribution in the whole
>     axis, that’s why we are trying to create an abstraction above both
>     CharacterisationDM::ObservableAxis and STC2.0::CoordMeasurement
>     describing only the statistical properties of both. The Quantity
>     class is just a sketch what could be described by it – the final
>     solution would be to store a mixture of gaussians in it,
>     describing the distribution in a generic way.
>
>     I completely agree with the rest – we can discover TimeSeries data
>     cubes by /dataproduct_type/  and /target_name, s_region, s_resol,
>     t_min, t_max, t_resol, em_min, em_max, em_resol, etc. /Attributes
>     right now.
>
>     How to extend these Obscore discovery parameters to discover time
>     series by more details of their axes, we need to agree on how the
>     distribution of values on them will be described in the time
>     series. From the data point of view, a **mixture of gaussian**
>     based abstraction above measurement uncertainties and axis
>     statistical distributions would be perfect, but I don’t know
>     whether we can provide that description for any type of time
>     series axis.
>
>     Cheers,
>
>     Jiri
>
>     *From:* dm-bounces at ivoa.net <mailto:dm-bounces at ivoa.net>
>     [mailto:dm-bounces at ivoa.net] *On Behalf Of *Mireille Louys
>     *Sent:* Wednesday, July 12, 2017 10:57 AM
>     *To:* dm at ivoa.net <mailto:dm at ivoa.net>; voevent at ivoa.net
>     <mailto:voevent at ivoa.net>; dal at ivoa.net <mailto:dal at ivoa.net>
>     *Subject:* [timeDomain: model for Time series] discussion on
>     Timeseries Data model Note / how are ND-Cube DM and Timeseries DM
>     connected ?
>
>     Dear DM and Time Domain followers,
>
>     I am trying, together with my CDS colleagues,  to recap on the
>     various DMs available in the IVOA and understand the possible
>     links between the future Time Series Model ( as sketched in
>     Jiris's Note) and existing DMs like ND-Cube and STC 2.
>
>     Here is a graph proposed by Laurent Michel to clarify the links in
>     3 main parts :
>
>       * /DataSetMetadata DM/, which has the main ObsDataset Class ,
>       * /ND-CubeDM/, which defines a SparseCubedataset
>       * /TimeSerieCubeDM/, which highlights the special properties of
>         a Cube depending on a Time axis
>
>     I think this is essential to highlight the inheritance path
>     between these 3 DM building blocks:
>     a TimeSeriesCube  <is a > NDCubeDM::SparseCubeDataset
>     a NDCubeDM::SparseCubeDataset <is a > DatasetMetadaDM::ObsDataset
>
>     ObsDataset has a /dataproduct_type/ attribute which allows to
>     discover all dataproducts of type ' timeseries'.
>     this provides the container object for time-dependent data.
>
>     If we need to select /timeseries dataproducts/ according to some
>     properties extracted from their data we can:
>      - reuse what Obscore DM provides to explain general axes properties
>     target_name, s_region, s_resol, t_min, t_max, t_resol, em_min,
>     em_max, em_resol, etc. are the basic properties for discovery
>
>      - provide a richer description of the TimeAxis and ObservableAxis.
>     For that , extracting  a statistical profile from the data
>     contained in the Cube could do the job.
>     this means to access and analyse the Data part in ND-Cube , i. e
>     the ND-Points gathered in a SparseCube Object
>
>
>
>     I guess more properties can be exposed to qualify the axes present
>     in the Timeseries dataset , but for the moment , I see some
>     overlap of notions between
>     CharacterisationDM::ObservableAxis, STC2.0::CoordMeasurement (??)
>     and TimeSerieCubeDM::CubeAxis.
>
>     This would be great if we could sort this out,
>     but currently , I would appreciate your feedback on the attached
>     diagram , in order to proceed on the data model structure.
>
>     Cheers, Mireille ( after discussions together with Laurent,
>     François, Ada)
>
>     -- 
>
>     --
>
>     Mireille Louys
>
>     CDS                                          Laboratoire Icube
>
>     Observatoire de Strasbourg    Telecom Physique Strasbourg
>
>     11 rue de l'Université               300, Bd Sebastien Brandt CS 10413
>
>     F- 67000-STRASBOURG                  F-67412 ILLKIRCH Cedex
>
>     tel: +33 3 68 85 24 34
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.ivoa.net/pipermail/dm/attachments/20170714/8c845937/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/jpeg
Size: 14767 bytes
Desc: not available
URL: <http://mail.ivoa.net/pipermail/dm/attachments/20170714/8c845937/attachment-0001.jpe>


More information about the dm mailing list