[timeDomain: model for Time series] discussion on Timeseries Data model Note / how are ND-Cube DM and Timeseries DM connected ?
Jiří Nádvorník
nadvornik.ji at gmail.com
Fri Jul 14 13:39:57 CEST 2017
Hi,
Thank you, see inline.
Jiri
From: François Bonnarel [mailto:francois.bonnarel at astro.unistra.fr]
Sent: Thursday, July 13, 2017 11:52 PM
To: Jiří Nádvorník <nadvornik.ji at gmail.com>; mireille.louys at unistra.fr; dm at ivoa.net; voevent at ivoa.net; dal at ivoa.net
Subject: Re: [timeDomain: model for Time series] discussion on Timeseries Data model Note / how are ND-Cube DM and Timeseries DM connected ?
Hi Jiri,
Thanks for having given a look to these emails. This is a very important discussion and we have to all agree on these grounds before going to more TimeSeries specific features of the model we need.
Le 13/07/2017 à 15:02, Jiří Nádvorník a écrit :
Hi Mireille,
Thank you very much for the input.
Your diagram is almost correct, but I believe that the relationship
TimeSeriesCube <is a > NDCubeDM::SparseCubeDataset
Is not correct, even in the original idea Mark Cresitello Ditmar had (please correct me here if I’m wrong, Mark). The correct relationship is:
TimeSeriesCube <is a> NDCubeDM::SparseCube and
NDCubeDM::SparseCube <is collected by> NDCubeDM::SparseCubeDataset
As seen on the following image:
Meaning that the SparseCubeDataset is describing a collection of data cubes, e.g., time series data, e.g., light curves, *not* one cube, e.g., one time series, e.g., one light curve. If we agree that we don’t need collections of time series (because they can by themselves be multi-dimensional), we can change it to <is a> relationship as you propose.
OK. This is were we differ. I don't think SparSecubedataset is made for tackling collections. My interpretation of the ND-Cube diagram helped by MCD' text is that SparSeCubeDataset inheritance from ObsDataSet contains all generic metadata for a dataset which may be COMPLEX. That is it contains : curation, provenance, identification, characterisation.
Such a SparseCubeDataset may contain one (simple dataset) or several (complex dataset) "SparseCubes"
Each of these SparseCube(s) contains ND-points where the actual data are stored and inherits from the DataProduct class the coordinate systems and mappings of the data to physical coordinates.
As Mireille said the "dataproduct_type " of your TimeSeriesCube (or TimeSeriesDataSet) can only be inherited from ObsDataSet through SparseCube Dataset and not from DataProduct via SparseCube.
[[Jiri Nadvornik]] Ha, so this means that each row we see in ObsCore table is describing a DataSet, meaning Spectrum <is a> DataSet, Image <is a> DataSet, SED <is a> DataSet, event <is a> Dataset, etc.? I can also see a valid option dataproduct_type=cube in the ObsCore document ( <http://www.ivoa.net/documents/ObsCore/20111028/REC-ObsCore-v1.0-20111028.pdf> http://www.ivoa.net/documents/ObsCore/20111028/REC-ObsCore-v1.0-20111028.pdf )
If the assumptions above are correct, I concur that changing the relationship to TimeSeries <is a> Cube <is a> DataSet is not a bad idea. >From my point of view, this can be changed in the model rather easily – it will just introduce a more tight coupling between DataSet and TimeSeriesCube data model. TimeSeriesCube data model will need to import everything from the DataSet DM and if something changes in DataSet DM, it will need to change in TimeSeriesCube DM too. In other words, new major versions of DataSet DM will require new major versions of TimeSeriesCube DM to follow.
Cheers
François
Now the Time series class is described in the attached UML.pdf (please note that this one is different from the original note, this version was last updated after Shanghai Interop in May). Main difference is that the TimeSerieCubeDM::CubeAxis custom class was replaced just by a generic columnRef (yellow) saying where can I find the data for this axis and that axis is described by Quantity class (yellow).
The Quantity class indeed provides the *richer description* on the cube axis (not only the time axis). This is indeed correlated by STC2.0::CoordMeasurement, but we got into conflict in here, as we would like to use it not for describing only *uncertainties* in the Measurement, but for statistical distribution in the whole axis, that’s why we are trying to create an abstraction above both CharacterisationDM::ObservableAxis and STC2.0::CoordMeasurement describing only the statistical properties of both. The Quantity class is just a sketch what could be described by it – the final solution would be to store a mixture of gaussians in it, describing the distribution in a generic way.
I completely agree with the rest – we can discover TimeSeries data cubes by dataproduct_type and target_name, s_region, s_resol, t_min, t_max, t_resol, em_min, em_max, em_resol, etc. Attributes right now.
How to extend these Obscore discovery parameters to discover time series by more details of their axes, we need to agree on how the distribution of values on them will be described in the time series. From the data point of view, a *mixture of gaussian* based abstraction above measurement uncertainties and axis statistical distributions would be perfect, but I don’t know whether we can provide that description for any type of time series axis.
Cheers,
Jiri
From: dm-bounces at ivoa.net <mailto:dm-bounces at ivoa.net> [mailto:dm-bounces at ivoa.net] On Behalf Of Mireille Louys
Sent: Wednesday, July 12, 2017 10:57 AM
To: dm at ivoa.net <mailto:dm at ivoa.net> ; voevent at ivoa.net <mailto:voevent at ivoa.net> ; dal at ivoa.net <mailto:dal at ivoa.net>
Subject: [timeDomain: model for Time series] discussion on Timeseries Data model Note / how are ND-Cube DM and Timeseries DM connected ?
Dear DM and Time Domain followers,
I am trying, together with my CDS colleagues, to recap on the various DMs available in the IVOA and understand the possible links between the future Time Series Model ( as sketched in Jiris's Note) and existing DMs like ND-Cube and STC 2.
Here is a graph proposed by Laurent Michel to clarify the links in 3 main parts :
* DataSetMetadata DM, which has the main ObsDataset Class ,
* ND-CubeDM, which defines a SparseCubedataset
* TimeSerieCubeDM, which highlights the special properties of a Cube depending on a Time axis
I think this is essential to highlight the inheritance path between these 3 DM building blocks:
a TimeSeriesCube <is a > NDCubeDM::SparseCubeDataset
a NDCubeDM::SparseCubeDataset <is a > DatasetMetadaDM::ObsDataset
ObsDataset has a dataproduct_type attribute which allows to discover all dataproducts of type ' timeseries'.
this provides the container object for time-dependent data.
If we need to select timeseries dataproducts according to some properties extracted from their data we can:
- reuse what Obscore DM provides to explain general axes properties
target_name, s_region, s_resol, t_min, t_max, t_resol, em_min, em_max, em_resol, etc. are the basic properties for discovery
- provide a richer description of the TimeAxis and ObservableAxis.
For that , extracting a statistical profile from the data contained in the Cube could do the job.
this means to access and analyse the Data part in ND-Cube , i. e the ND-Points gathered in a SparseCube Object
I guess more properties can be exposed to qualify the axes present in the Timeseries dataset , but for the moment , I see some overlap of notions between
CharacterisationDM::ObservableAxis, STC2.0::CoordMeasurement (??) and TimeSerieCubeDM::CubeAxis.
This would be great if we could sort this out,
but currently , I would appreciate your feedback on the attached diagram , in order to proceed on the data model structure.
Cheers, Mireille ( after discussions together with Laurent, François, Ada)
--
--
Mireille Louys
CDS Laboratoire Icube
Observatoire de Strasbourg Telecom Physique Strasbourg
11 rue de l'Université 300, Bd Sebastien Brandt CS 10413
F- 67000-STRASBOURG F-67412 ILLKIRCH Cedex
tel: +33 3 68 85 24 34
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.ivoa.net/pipermail/dm/attachments/20170714/176412ee/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.jpg
Type: image/jpeg
Size: 14767 bytes
Desc: not available
URL: <http://mail.ivoa.net/pipermail/dm/attachments/20170714/176412ee/attachment-0001.jpg>
More information about the dm
mailing list