Time Series Cube DM - IVOA Note

[Jiří Nádvorník]

Hi Mark,

thank you for the response. My comments/answers below.


2017-03-16 17:19 GMT+01:00 CresitelloDittmar, Mark <mdittmar at cfa.harvard.edu
>:
>
>
> Yes.. the N-Dimensional Cube model is the continuation of that effort..
> when I took over that project it was folded into the effort of extracting
> DatasetMetadata from these Product models and forming the family of Cube,
> Dataset, STC( coordsys, coords, trans ); and using vo-dml compliant
> modeling.   In that process, since the word 'Image' invokes the 'pixelated
> image type', and the model covers a broader scope than that, it was renamed
> Cube.  The Cube model history includes the ImageDM iterations.
>

Thanks for the explanation. Now I understand - I will correct it in my
document.


>
>
>>
>>    + Section 4/Figure 4
>>>       your description of the TimeSeriesCube aligns pretty well with the
>>> SparseCube as it is..
>>>       I'm not sure it is necessary to 'override' the content (btw you
>>> could just extend "PointDataProduct")
>>>       o the cube model SparseCube has a collecton of NDPoints (ie rows),
>>> which contains a collection
>>>          of DataAxis, each referring to an instance of a measurement
>>> type coordinate (value + errors)
>>>          * your representation and mine simply reverse the row/column
>>> order.
>>>          * in cube, the Observable is any instance of Coordinate which
>>> follows the pattern described
>>>            in STC2-coords model.  The modeling of that instance/domain
>>> does NOT have to be in
>>>            any particular model.. so the Axis Domain DMs scenario you
>>> show works fine.
>>>          * But.. for interoperability sake, we do require them to use
>>> the same pattern (by linking
>>>            Observable to the abstract coords:DerivedCoordinate which is
>>> bound to a pattern)
>>>
>>
>> Yes, it is pretty similar - we used it as an inspiration and believe
>> originally that we will just build upon it without changing it. Problems
>> and difficulties listed beneath.
>>
>>    1. Axis Domain models - physical meaning of the data in the axis of
>>    the data cube should not be part of the cube model. The Frame mappings and
>>    CoordSys entities taken from STC are also such domain models IMHO.
>>
>> The cube model does not have any description of domain content.. (other
> than Pixel).
> The Mappings and CoordSys objects are containers for pan-domain
> information defined in the respective models (coordsys, trans).
>

Well then do I need to keep them as part of the Cube DM, can't we keep it
only in those respective models? Also connected with the point I put below:


>
>
    + Section 4.2.2
>>        cube model does not have dependency on specifc axis domains any
>> more (since ~Nov)
>
>
> What about those FrameMappings and CoordSys entities? These are storing
> physical meaning of the data, not just describing the structure of the data
> (the data cube) itself.
>
>>
>>
Yes, they are containers for storing the physical meaning of the data.  The
> content of a Cube or pixelated image is not devoid of physical meaning, if
> a sparse cube has data axes, each of these are of some physical quantity..
> these elements allow you to describe that content without imposing
> restrictions about specific domains.


I understand the purpose of the FrameMappings and CoordSys entities but
again I don't think we need to make them part of the Cube DM. I belive that
the physical domain model itself (Photometry DM, STC, Probability DM, ...)
should take care of their own FrameMappings and CoordSys objects (still, if
it's possible, I encourage them to use the existing models (coordsys,
trans).

Anyway, could you point me to the documents where those are specified? I
didn't find it in your document and am not sure how exactly is that
coordsys or trans defined.


>
>>    1. Row/column inversion - this is a bigger difference than it seems
>>    on the first look. Logically we are storing the same information but I
>>    don't want to explicitely store a reference to an axis in every single
>>    point of the data cube. By this the Sparse Cube Model is IMHO saying that
>>    the data can be unordered and so we need to store the reference for every
>>    point? Doing it the other way around, the axis can just specify where I can
>>    find the axis coordinates in the data element of the cube, no matter how we
>>    serialize it.
>>
>> I think the annotation should end up pretty similar, except that
> Row-based will sit in a grouping for the NDPoint object, while column based
> will not.
> The axis reference is stored in the template next to the COLUMN spec.
> Each iteration of the Template (table row) fills in the appropriate
> elements of the template.. in my case the values and errors of the DataAxis
> observables in an NDPoint.. in the column-based approach each iteration
> will fill in the next element of the the observables.. outside of an
> NDPoint.
>

What exactly do you keep in the NDPoint then? Is it only a simple Coord, or
does it contain some metadata?


>
>
>>    1. What is the Observable entity's purpose in the diagram? Please
>>    explain..
>>
>> Figure 2, 6, 7. Observable entity is a collection of Coordinates
> associated with the data product.  This is something to make sure is
> explained well.
> The premise is that a DataProduct should OWN all of its coordinates/data.
> The vo-dml rules for composition state that a class/object may not be in
> more than one composition relation.
>   + so one does not make composition relations across models.
>   + it defines an aggregation pattern to use  (as shown for DataProduct ->
> Observable -> DerivedObject ) for these situations
>
> Since there are multiple types of Data Axis types, I modeled it this way..
> where the DataProduct owns ALL its data (Observables), and the data axis
> types (DataAxis, DependentAxis) are organizational objects which refer to
> the instances of the same axis.
>
> This could be organized differently.. having the Observables owned by the
> DataAxis (which is directly or indirectly owned by the DataProduct), and
> extend that for various types of axis.. adding constraints as needed.  The
> sub-classes of DataAxis would then each be in composition with some
> container (NDPoint, Voxel).  I don't know if this would be
> better/worse/indifferent.
>
>      <DataAxis> O---------- Cell  (constrain mult=1)
>          +              |__ DependentAxis (constrain mult=1)
>          |              |__ Column ??
>          V  *
>      Observable
>          |
>          V
>    coords:DerivedCoordiante
>
>
> I want to note one distinction.  The DataAxis here, is NOT the same as a
> coordinate space axis.
> If I have a 3D cartesian Space, with coordinate axes x,y,z.. there is 1
> DataAxis referring to a Position3D in that space.
> I think this maps well to statements below.. where the 'Observable' maybe
> need not be a DerivedCoordinate, but allow for complex objects like
> "Spectrum"
>

We need to draw this on a *sheet* and talk about it - I can think of
multiple instantiations of the abstractions that you used above (some would
actually be really close to the ones I want to make with the TimeSeriesCube
DM).


>
>
>>>
>  <snip>
>
> So, I see we have 2 points of discussion for the cube model itself
>   1) relation between Dataset and DataProduct
>       Currently modeled as according to Section 3.. extend Dataset add
> reference to DataProduct == MyDataset
>
>       Alternates include:
>         a) loose coupling
>             verbal statement that MyDataset includes an instance of
> Dataset + instance of MyDataProduct
>         b) referenced coupling
>             MyDataSet == reference to Dataset + reference to MyDataProduct
>             (allows validators to know what is expected, but allows
> flexibility w.r.t. Dataset flavor )
>
>      I personally think a) is too loose, but b) might be a good way to
> go..
>
>   2) relation of DataProduct, DataAxis and the 'Observable'
>       I haven't explored the alternative shown above very much, but like
> the possibility
>       In the current relation, the DataProduct-Observable relation is a
> potential source of confusion to readers.
>
>
I agree we need to discuss these - we can on the Asterics meeting next week
if you will be there, otherwise a video call where we can share our
drawings would be also good.

Output of such discussion should be a model of the abstractions we are
using *with* the actual examples of data that are instances of those
abstractions. This is I feel the main source of my confusion as a reader of
other IVOA DMs, the lack of direct example that puts away the shadiness and
shortens up the discussions about it a lot.

Cheers,

Jiri
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.ivoa.net/pipermail/dm/attachments/20170319/ebc68946/attachment.html>