[timeDomain: model for Time series] discussion on Timeseries Data model Note / how are ND-Cube DM and Timeseries DM connected ?

[François Bonnarel]

Hi Jiri,
    Thanks for having given a look to these emails. This is a very 
important discussion and we have to all agree on these grounds before 
going to more TimeSeries specific features of the model we need.
Le 13/07/2017 à 15:02, Jiří Nádvorník a écrit :
>
> Hi Mireille,
>
> Thank you very much for the input.
>
> Your diagram is almost correct, but I believe that the relationship
>
> TimeSeriesCube  <is a > NDCubeDM::SparseCubeDataset
>
> Is not correct, even in the original idea Mark Cresitello Ditmar had 
> (please correct me here if I’m wrong, Mark). The correct relationship is:
>
> TimeSeriesCube <is a> NDCubeDM::SparseCube and
>
> NDCubeDM::SparseCube <is collected by> NDCubeDM::SparseCubeDataset
>
> As seen on the following image:
>
> Meaning that the SparseCubeDataset is describing a collection of data 
> cubes, e.g., time series data, e.g., light curves, **not** one cube, 
> e.g., one time series, e.g., one light curve. If we agree that we 
> don’t need collections of time series (because they can by themselves 
> be multi-dimensional), we can change it to <is a> relationship as you 
> propose.
>
OK. This is were we differ. I don't think SparSecubedataset is made for 
tackling collections. My interpretation of the ND-Cube diagram helped by 
MCD' text is that SparSeCubeDataset inheritance from ObsDataSet contains 
all generic metadata for a dataset which may be COMPLEX. That is it 
contains : curation, provenance, identification, characterisation.

Such  a SparseCubeDataset may contain one (simple dataset) or several 
(complex dataset) "SparseCubes"

Each of these SparseCube(s) contains ND-points where the actual data are 
stored and inherits from the DataProduct class the coordinate systems 
and mappings of the data to physical coordinates.

As Mireille said the "/dataproduct_type/ " of your TimeSeriesCube (or 
TimeSeriesDataSet) can only be inherited from ObsDataSet through 
SparseCube Dataset and not from DataProduct via SparseCube.


Cheers
François



> Now the Time series class is described in the attached UML.pdf (please 
> note that this one is different from the original note, this version 
> was last updated after Shanghai Interop in May). Main difference is 
> that the TimeSerieCubeDM::CubeAxis custom class was replaced just by a 
> generic columnRef (yellow) saying where can I find the data for this 
> axis and that axis is described by Quantity class (yellow).
>
> The Quantity class indeed provides the **richer description****on the 
> cube axis (not only the time axis). This is indeed correlated by 
> STC2.0::CoordMeasurement, but we got into conflict in here, as we 
> would like to use it not for describing only **uncertainties** in the 
> Measurement, but for statistical distribution in the whole axis, 
> that’s why we are trying to create an abstraction above both 
> CharacterisationDM::ObservableAxis and STC2.0::CoordMeasurement 
> describing only the statistical properties of both. The Quantity class 
> is just a sketch what could be described by it – the final solution 
> would be to store a mixture of gaussians in it, describing the 
> distribution in a generic way.
>
> I completely agree with the rest – we can discover TimeSeries data 
> cubes by /dataproduct_type/ and /target_name, s_region, s_resol, 
> t_min, t_max, t_resol, em_min, em_max, em_resol, etc. /Attributes 
> right now.
>
> How to extend these Obscore discovery parameters to discover time 
> series by more details of their axes, we need to agree on how the 
> distribution of values on them will be described in the time series. 
> From the data point of view, a **mixture of gaussian** based 
> abstraction above measurement uncertainties and axis statistical 
> distributions would be perfect, but I don’t know whether we can 
> provide that description for any type of time series axis.
>
> Cheers,
>
> Jiri
>
> *From:*dm-bounces at ivoa.net [mailto:dm-bounces at ivoa.net] *On Behalf Of 
> *Mireille Louys
> *Sent:* Wednesday, July 12, 2017 10:57 AM
> *To:* dm at ivoa.net; voevent at ivoa.net; dal at ivoa.net
> *Subject:* [timeDomain: model for Time series] discussion on 
> Timeseries Data model Note / how are ND-Cube DM and Timeseries DM 
> connected ?
>
> Dear DM and Time Domain followers,
>
> I am trying, together with my CDS colleagues,  to recap on the various 
> DMs available in the IVOA and understand the possible links between 
> the future Time Series Model ( as sketched in Jiris's Note) and 
> existing DMs like ND-Cube and STC 2.
>
> Here is a graph proposed by Laurent Michel to clarify the links in 3 
> main parts :
>
>   * /DataSetMetadata DM/, which has the main ObsDataset Class ,
>   * /ND-CubeDM/, which defines a SparseCubedataset
>   * /TimeSerieCubeDM/, which highlights the special properties of a
>     Cube depending on a Time axis
>
> I think this is essential to highlight the inheritance path between 
> these 3 DM building blocks:
> a TimeSeriesCube  <is a > NDCubeDM::SparseCubeDataset
> a NDCubeDM::SparseCubeDataset <is a > DatasetMetadaDM::ObsDataset
>
> ObsDataset has a /dataproduct_type/ attribute which allows to discover 
> all dataproducts of type ' timeseries'.
> this provides the container object for time-dependent data.
>
> If we need to select /timeseries dataproducts/ according to some 
> properties extracted from their data we can:
>  - reuse what Obscore DM provides to explain general axes properties
> target_name, s_region, s_resol, t_min, t_max, t_resol, em_min, em_max, 
> em_resol, etc. are the basic properties for discovery
>
>  - provide a richer description of the TimeAxis and ObservableAxis.
> For that , extracting  a statistical profile from the data contained 
> in the Cube could do the job.
> this means to access and analyse the Data part in ND-Cube , i. e the 
> ND-Points gathered in a SparseCube Object
>
>
>
> I guess more properties can be exposed to qualify the axes present in 
> the Timeseries dataset , but for the moment , I see some overlap of 
> notions between
> CharacterisationDM::ObservableAxis, STC2.0::CoordMeasurement (??) and 
> TimeSerieCubeDM::CubeAxis.
>
> This would be great if we could sort this out,
> but currently , I would appreciate your feedback on the attached 
> diagram , in order to proceed on the data model structure.
>
> Cheers, Mireille ( after discussions together with Laurent, François, 
> Ada)
>
>
> -- 
> --
> Mireille Louys
> CDS                                          Laboratoire Icube
> Observatoire de Strasbourg    Telecom Physique Strasbourg
> 11 rue de l'Université               300, Bd Sebastien Brandt CS 10413
> F- 67000-STRASBOURG                  F-67412 ILLKIRCH Cedex
> tel: +33 3 68 85 24 34

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.ivoa.net/pipermail/dm/attachments/20170713/1e401ad7/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/jpeg
Size: 14767 bytes
Desc: not available
URL: <http://mail.ivoa.net/pipermail/dm/attachments/20170713/1e401ad7/attachment-0001.jpe>