[timeDomain: model for Time series] discussion on Timeseries Data model Note / how are ND-Cube DM and Timeseries DM connected ?
François Bonnarel
francois.bonnarel at astro.unistra.fr
Fri Jul 14 16:01:34 CEST 2017
Dear all,
Le 14/07/2017 à 13:39, Jiří Nádvorník a écrit :
>
> Hi,
>
> Thank you, see inline.
>
> Jiri
>
> *From:*François Bonnarel [mailto:francois.bonnarel at astro.unistra.fr]
> *Sent:* Thursday, July 13, 2017 11:52 PM
> *To:* Jiří Nádvorník <nadvornik.ji at gmail.com>;
> mireille.louys at unistra.fr; dm at ivoa.net; voevent at ivoa.net; dal at ivoa.net
> *Subject:* Re: [timeDomain: model for Time series] discussion on
> Timeseries Data model Note / how are ND-Cube DM and Timeseries DM
> connected ?
>
> Hi Jiri,
> Thanks for having given a look to these emails. This is a very
> important discussion and we have to all agree on these grounds before
> going to more TimeSeries specific features of the model we need.
>
> Le 13/07/2017 à 15:02, Jiří Nádvorník a écrit :
>
> Hi Mireille,
>
> Thank you very much for the input.
>
> Your diagram is almost correct, but I believe that the relationship
>
> TimeSeriesCube <is a > NDCubeDM::SparseCubeDataset
>
> Is not correct, even in the original idea Mark Cresitello Ditmar
> had (please correct me here if I’m wrong, Mark). The correct
> relationship is:
>
> TimeSeriesCube <is a> NDCubeDM::SparseCube and
>
> NDCubeDM::SparseCube <is collected by> NDCubeDM::SparseCubeDataset
>
> As seen on the following image:
>
> Meaning that the SparseCubeDataset is describing a collection of
> data cubes, e.g., time series data, e.g., light curves, **not**
> one cube, e.g., one time series, e.g., one light curve. If we
> agree that we don’t need collections of time series (because they
> can by themselves be multi-dimensional), we can change it to <is
> a> relationship as you propose.
>
> OK. This is were we differ. I don't think SparSecubedataset is made
> for tackling collections. My interpretation of the ND-Cube diagram
> helped by MCD' text is that SparSeCubeDataset inheritance from
> ObsDataSet contains all generic metadata for a dataset which may be
> COMPLEX. That is it contains : curation, provenance, identification,
> characterisation.
>
> Such a SparseCubeDataset may contain one (simple dataset) or several
> (complex dataset) "SparseCubes"
>
> Each of these SparseCube(s) contains ND-points where the actual data
> are stored and inherits from the DataProduct class the coordinate
> systems and mappings of the data to physical coordinates.
>
> As Mireille said the "/dataproduct_type/ " of your TimeSeriesCube (or
> TimeSeriesDataSet) can only be inherited from ObsDataSet through
> SparseCube Dataset and not from DataProduct via SparseCube.
>
> */[[Jiri Nadvornik]] Ha, so this means that each row we see in ObsCore
> table is describing a DataSet, meaning Spectrum <is a> DataSet, Image
> <is a> DataSet, SED <is a> DataSet, event <is a> Dataset, etc.? I can
> also see a valid option dataproduct_type=cube in the ObsCore document
> (http://www.ivoa.net/documents/ObsCore/20111028/REC-ObsCore-v1.0-20111028.pdf
> )/*
>
> *//*
>
/*Yes that is it.*/
>
> */If the assumptions above are correct, I concur that changing the
> relationship to TimeSeries <is a> Cube <is a> DataSet is not a bad
> idea. From my point of view, this can be changed in the model rather
> easily – it will just introduce a more tight coupling between DataSet
> and TimeSeriesCube data model. TimeSeriesCube data model will need to
> import everything from the DataSet DM and if something changes in
> DataSet DM, it will need to change in TimeSeriesCube DM too. In other
> words, new major versions of DataSet DM will require new major
> versions of TimeSeriesCube DM to follow./*
>
>
I think DataSet DM is made to be reused in different specific contexts.
Cheers
François
>
>
>
> Cheers
> François
>
>
>
>
> Now the Time series class is described in the attached UML.pdf
> (please note that this one is different from the original note,
> this version was last updated after Shanghai Interop in May). Main
> difference is that the TimeSerieCubeDM::CubeAxis custom class was
> replaced just by a generic columnRef (yellow) saying where can I
> find the data for this axis and that axis is described by Quantity
> class (yellow).
>
> The Quantity class indeed provides the **richer description** on
> the cube axis (not only the time axis). This is indeed correlated
> by STC2.0::CoordMeasurement, but we got into conflict in here, as
> we would like to use it not for describing only **uncertainties**
> in the Measurement, but for statistical distribution in the whole
> axis, that’s why we are trying to create an abstraction above both
> CharacterisationDM::ObservableAxis and STC2.0::CoordMeasurement
> describing only the statistical properties of both. The Quantity
> class is just a sketch what could be described by it – the final
> solution would be to store a mixture of gaussians in it,
> describing the distribution in a generic way.
>
> I completely agree with the rest – we can discover TimeSeries data
> cubes by /dataproduct_type/ and /target_name, s_region, s_resol,
> t_min, t_max, t_resol, em_min, em_max, em_resol, etc. /Attributes
> right now.
>
> How to extend these Obscore discovery parameters to discover time
> series by more details of their axes, we need to agree on how the
> distribution of values on them will be described in the time
> series. From the data point of view, a **mixture of gaussian**
> based abstraction above measurement uncertainties and axis
> statistical distributions would be perfect, but I don’t know
> whether we can provide that description for any type of time
> series axis.
>
> Cheers,
>
> Jiri
>
> *From:* dm-bounces at ivoa.net <mailto:dm-bounces at ivoa.net>
> [mailto:dm-bounces at ivoa.net] *On Behalf Of *Mireille Louys
> *Sent:* Wednesday, July 12, 2017 10:57 AM
> *To:* dm at ivoa.net <mailto:dm at ivoa.net>; voevent at ivoa.net
> <mailto:voevent at ivoa.net>; dal at ivoa.net <mailto:dal at ivoa.net>
> *Subject:* [timeDomain: model for Time series] discussion on
> Timeseries Data model Note / how are ND-Cube DM and Timeseries DM
> connected ?
>
> Dear DM and Time Domain followers,
>
> I am trying, together with my CDS colleagues, to recap on the
> various DMs available in the IVOA and understand the possible
> links between the future Time Series Model ( as sketched in
> Jiris's Note) and existing DMs like ND-Cube and STC 2.
>
> Here is a graph proposed by Laurent Michel to clarify the links in
> 3 main parts :
>
> * /DataSetMetadata DM/, which has the main ObsDataset Class ,
> * /ND-CubeDM/, which defines a SparseCubedataset
> * /TimeSerieCubeDM/, which highlights the special properties of
> a Cube depending on a Time axis
>
> I think this is essential to highlight the inheritance path
> between these 3 DM building blocks:
> a TimeSeriesCube <is a > NDCubeDM::SparseCubeDataset
> a NDCubeDM::SparseCubeDataset <is a > DatasetMetadaDM::ObsDataset
>
> ObsDataset has a /dataproduct_type/ attribute which allows to
> discover all dataproducts of type ' timeseries'.
> this provides the container object for time-dependent data.
>
> If we need to select /timeseries dataproducts/ according to some
> properties extracted from their data we can:
> - reuse what Obscore DM provides to explain general axes properties
> target_name, s_region, s_resol, t_min, t_max, t_resol, em_min,
> em_max, em_resol, etc. are the basic properties for discovery
>
> - provide a richer description of the TimeAxis and ObservableAxis.
> For that , extracting a statistical profile from the data
> contained in the Cube could do the job.
> this means to access and analyse the Data part in ND-Cube , i. e
> the ND-Points gathered in a SparseCube Object
>
>
>
> I guess more properties can be exposed to qualify the axes present
> in the Timeseries dataset , but for the moment , I see some
> overlap of notions between
> CharacterisationDM::ObservableAxis, STC2.0::CoordMeasurement (??)
> and TimeSerieCubeDM::CubeAxis.
>
> This would be great if we could sort this out,
> but currently , I would appreciate your feedback on the attached
> diagram , in order to proceed on the data model structure.
>
> Cheers, Mireille ( after discussions together with Laurent,
> François, Ada)
>
> --
>
> --
>
> Mireille Louys
>
> CDS Laboratoire Icube
>
> Observatoire de Strasbourg Telecom Physique Strasbourg
>
> 11 rue de l'Université 300, Bd Sebastien Brandt CS 10413
>
> F- 67000-STRASBOURG F-67412 ILLKIRCH Cedex
>
> tel: +33 3 68 85 24 34
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.ivoa.net/pipermail/dm/attachments/20170714/8c845937/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/jpeg
Size: 14767 bytes
Desc: not available
URL: <http://mail.ivoa.net/pipermail/dm/attachments/20170714/8c845937/attachment-0001.jpe>
More information about the dm
mailing list