Time Series Cube DM - IVOA Note

[CresitelloDittmar, Mark]

Jiri,


On Sun, Mar 19, 2017 at 2:48 PM, Jiří Nádvorník <nadvornik.ji at gmail.com>
wrote:

>
>
>>
>>    + Section 4/Figure 4
>>>       your description of the TimeSeriesCube aligns pretty well with the
>>> SparseCube as it is..
>>>       I'm not sure it is necessary to 'override' the content (btw you
>>> could just extend "PointDataProduct")
>>>       o the cube model SparseCube has a collecton of NDPoints (ie rows),
>>> which contains a collection
>>>          of DataAxis, each referring to an instance of a measurement
>>> type coordinate (value + errors)
>>>          * your representation and mine simply reverse the row/column
>>> order.
>>>          * in cube, the Observable is any instance of Coordinate which
>>> follows the pattern described
>>>            in STC2-coords model.  The modeling of that instance/domain
>>> does NOT have to be in
>>>            any particular model.. so the Axis Domain DMs scenario you
>>> show works fine.
>>>          * But.. for interoperability sake, we do require them to use
>>> the same pattern (by linking
>>>            Observable to the abstract coords:DerivedCoordinate which is
>>> bound to a pattern)
>>>
>>
>> Yes, it is pretty similar - we used it as an inspiration and believe
>> originally that we will just build upon it without changing it. Problems
>> and difficulties listed beneath.
>>
>>    1. Axis Domain models - physical meaning of the data in the axis of
>>    the data cube should not be part of the cube model. The Frame mappings and
>>    CoordSys entities taken from STC are also such domain models IMHO.
>>
>> The cube model does not have any description of domain content.. (other
> than Pixel).
> The Mappings and CoordSys objects are containers for pan-domain
> information defined in the respective models (coordsys, trans).
>
> Well then do I need to keep them as part of the Cube DM, can't we keep it
> only in those respective models? Also connected with the point I put below:
>

If I understand the questions..
The coordsys model defines the "CoordSys" and "AstroCoordSys" objects.
These are collections of associated Frames which fully define a coordinate
space.
In the cube model, I want to say: "A DataProduct has one or more Coordinate
system specifications, and the DataProduct owns its instances of CoordSys"
To do that, I must show a composition relation from cube:DataProduct to the
coordsys:CoordSys elements.
The vo-dml rules restrict limit how compositions are done, the proper
method is what is shown in the cube model... this is the aggregation
pattern described in the vo-dml spec.

The same is true for FrameMappings... the Cube has a collection of
Transform definitions which convert data from one Frame to another... this
is where they go.

My impression is not that you object to the items per se, but rather that
they are explicitly connected in the model.. that it would be sufficient to
simply serialize a coordsys instance in my cube, and since CoordSys is a
valid, modeled object, that is all I need to do.  If this is so.. what is
lost is the ability to validate the data product.  How do I know if the
instance has all the expected components?

To do what I think you are suggesting, would require a change to the VO-DML
specification.


>>
>     + Section 4.2.2
>>>        cube model does not have dependency on specifc axis domains any
>>> more (since ~Nov)
>>
>>
>> What about those FrameMappings and CoordSys entities? These are storing
>> physical meaning of the data, not just describing the structure of the data
>> (the data cube) itself.
>>
>>>
>>>
> Yes, they are containers for storing the physical meaning of the data.
>> The content of a Cube or pixelated image is not devoid of physical meaning,
>> if a sparse cube has data axes, each of these are of some physical
>> quantity.. these elements allow you to describe that content without
>> imposing restrictions about specific domains.
>
>
> I understand the purpose of the FrameMappings and CoordSys entities but
> again I don't think we need to make them part of the Cube DM. I belive that
> the physical domain model itself (Photometry DM, STC, Probability DM, ...)
> should take care of their own FrameMappings and CoordSys objects (still, if
> it's possible, I encourage them to use the existing models (coordsys,
> trans).
>
> Anyway, could you point me to the documents where those are specified? I
> didn't find it in your document and am not sure how exactly is that
> coordsys or trans defined.
>
>

CoordSys is in the coordsys model:
  vodml-html : https://volute.g-vo.org/svn/trunk/projects/dm/vo-dml/
models/STC2/prototype/coordsys/STCCoordSys-2.0.html

There are 2 types
  CoordSys = collection of GenericFrames
  AstroCoordSys = extension of CoordSys with 0..1 Frame from each
implemented domain.

Transforms are in the trans model:
  https://volute.g-vo.org/svn/trunk/projects/dm/vo-dml/
models/STC2/prototype/trans/STCTrans-2.0.html



>
>>>    1. Row/column inversion - this is a bigger difference than it seems
>>>    on the first look. Logically we are storing the same information but I
>>>    don't want to explicitely store a reference to an axis in every single
>>>    point of the data cube. By this the Sparse Cube Model is IMHO saying that
>>>    the data can be unordered and so we need to store the reference for every
>>>    point? Doing it the other way around, the axis can just specify where I can
>>>    find the axis coordinates in the data element of the cube, no matter how we
>>>    serialize it.
>>>
>>> I think the annotation should end up pretty similar, except that
>> Row-based will sit in a grouping for the NDPoint object, while column based
>> will not.
>> The axis reference is stored in the template next to the COLUMN spec.
>> Each iteration of the Template (table row) fills in the appropriate
>> elements of the template.. in my case the values and errors of the DataAxis
>> observables in an NDPoint.. in the column-based approach each iteration
>> will fill in the next element of the the observables.. outside of an
>> NDPoint.
>>
>
> What exactly do you keep in the NDPoint then? Is it only a simple Coord,
> or does it contain some metadata?
>

It boils down to a collection of Coordinate-s, the Coordinate has reference
back to the Frame/Axis metadata.


>
>>>    1. What is the Observable entity's purpose in the diagram? Please
>>>    explain..
>>>
>>> Figure 2, 6, 7. Observable entity is a collection of Coordinates
>> associated with the data product.  This is something to make sure is
>> explained well.
>> The premise is that a DataProduct should OWN all of its coordinates/data.
>> The vo-dml rules for composition state that a class/object may not be in
>> more than one composition relation.
>>   + so one does not make composition relations across models.
>>   + it defines an aggregation pattern to use  (as shown for DataProduct
>> -> Observable -> DerivedObject ) for these situations
>>
>> Since there are multiple types of Data Axis types, I modeled it this
>> way.. where the DataProduct owns ALL its data (Observables), and the data
>> axis types (DataAxis, DependentAxis) are organizational objects which refer
>> to the instances of the same axis.
>>
>> This could be organized differently.. having the Observables owned by the
>> DataAxis (which is directly or indirectly owned by the DataProduct), and
>> extend that for various types of axis.. adding constraints as needed.  The
>> sub-classes of DataAxis would then each be in composition with some
>> container (NDPoint, Voxel).  I don't know if this would be
>> better/worse/indifferent.
>>
>>      <DataAxis> O---------- Cell  (constrain mult=1)
>>          +              |__ DependentAxis (constrain mult=1)
>>          |              |__ Column ??
>>          V  *
>>      Observable
>>          |
>>          V
>>    coords:DerivedCoordiante
>>
>>
>> I want to note one distinction.  The DataAxis here, is NOT the same as a
>> coordinate space axis.
>> If I have a 3D cartesian Space, with coordinate axes x,y,z.. there is 1
>> DataAxis referring to a Position3D in that space.
>> I think this maps well to statements below.. where the 'Observable' maybe
>> need not be a DerivedCoordinate, but allow for complex objects like
>> "Spectrum"
>>
>
> We need to draw this on a *sheet* and talk about it - I can think of
> multiple instantiations of the abstractions that you used above (some would
> actually be really close to the ones I want to make with the TimeSeriesCube
> DM).
>

>
>>
>>
>>>>
>>  <snip>
>>
>> So, I see we have 2 points of discussion for the cube model itself
>>   1) relation between Dataset and DataProduct
>>       Currently modeled as according to Section 3.. extend Dataset add
>> reference to DataProduct == MyDataset
>>
>>       Alternates include:
>>         a) loose coupling
>>             verbal statement that MyDataset includes an instance of
>> Dataset + instance of MyDataProduct
>>         b) referenced coupling
>>             MyDataSet == reference to Dataset + reference to MyDataProduct
>>             (allows validators to know what is expected, but allows
>> flexibility w.r.t. Dataset flavor )
>>
>>      I personally think a) is too loose, but b) might be a good way to
>> go..
>>
>>   2) relation of DataProduct, DataAxis and the 'Observable'
>>       I haven't explored the alternative shown above very much, but like
>> the possibility
>>       In the current relation, the DataProduct-Observable relation is a
>> potential source of confusion to readers.
>>
>>
> I agree we need to discuss these - we can on the Asterics meeting next
> week if you will be there, otherwise a video call where we can share our
> drawings would be also good.
>
> Output of such discussion should be a model of the abstractions we are
> using *with* the actual examples of data that are instances of those
> abstractions. This is I feel the main source of my confusion as a reader of
> other IVOA DMs, the lack of direct example that puts away the shadiness and
> shortens up the discussions about it a lot.
>
>
No.. I won't be there.
I agree we would benefit from some shared focus time.. perhaps we can
arrange a time off-line to work this?
Mark
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.ivoa.net/pipermail/dm/attachments/20170320/9308ca1f/attachment-0001.html>