Time Series Cube DM - IVOA Note

[CresitelloDittmar, Mark]

Markus,

In this and Mark's mail, I'm seeing a distinction which maybe has been
missed?
There is nothing preventing one from annotating a cube with 'alternate'
content and this could be widely usable by clients who don't care about
validating the cube.

My objection is to the statements that the model itself should not
facilitate validation for the times when it IS desired.  With the loose-non
coupled modeling scheme, one could literally put ANYTHING in the CoordSys
element and it would be 'valid' so long as it is has a vo-dml
representation.  I'll note that this behavior is only true at the model
boundaries.. if the elements are in the same model then the vo-dml must
resolve to a specific - defined element.

more below:


On Wed, Mar 22, 2017 at 9:20 AM, Markus Demleitner <
msdemlei at ari.uni-heidelberg.de> wrote:

>
> I'm not strictly arguing against saying in an appropriate place
> (e.g., a DAL protocol, an endorsed note, or perhaps even a short REC
> containing mainly examples): "A fully-compliant spectrum-1 instance has to
> have valid annotations for STC-2 (spectral axis; for the record, I still
> think the spectral axis should have a separate DM), photometry-1 (flux
> axis), Dataset-1, and NDCube (axis organisation).  For observed spectra,
> additionally, the Dataset-1 target position annotation and an
> Observation-1 annotation".
>
> IMO - that is the job of the model... then if you annotate to the model,
it is valid.
if you don't annotate to the model, it is not.


> Note that the last clause also provides a nice solution to what were
> discussing back in the Spectrum DM 2.0 days, where SDM made the
> target position required and it was unclear what to do with
> theoretical spectra.
>

Yeah.. in one of the earlier mails, I agree that we probably want to change
this relation so that different types of Dataset could be used.
My favorite being:
  NDCubeDataset
    + reference to Dataset  ( could be ObsDataset, or SimDataset, or
whatever )
    + reference to NDCube DataProduct
  which defines the required pieces, but allows more flexibility.

 <snip>

>
> > The ownership relations are there for various applications which
> > implement the model.  When implementing a library, I would want to
> > know when it is safe to free the memory space for particular
> > elements.  I think this is most true for database applications, but
> > that is outside my wheelhouse.
>
> Hm -- I think memory management, whether reference counting as
> proposed here, a conventional garbage collector, or whatever else is
> so far removed from data modelling that I doubt the concept of
> ownerwhip in data model elements will actually help implementations
> in memory management.
>

My understanding is that defining memory management is one of the primary
uses of data models.
That information is not used in all cases, and elements may be
flattened/combined on implementation, but the model defines the fundamental
relations between objects.


> > > > >> If I have a 3D cartesian Space, with coordinate axes x,y,z..
> there is
> > > 1
> > > > >> DataAxis referring to a Position3D in that space.
> > >
> > > Uh -- that sounds... dangerous.  In the spirit of my preference to
> > > ideally reference native entities (i.e., FIELDs here): How does this
> > > DataAxis grouping help a client?  What is it supposed to do with it?
> > > How does the grouping help it over just having three axis (that, of
> > > course, might still be related through one or more separate STC
> > > annotations, but I'd like that to be uncorrelated if at all
> > > possible).
> > >
> > >
> > 1 FIELD -> 1 DataAxis (Coordinate) works fine only for the simplest
> > case (1D value with no errors).  For the 2D/3D cases, the errors
> > may be correlated, so the bundle of FIELDs for the value must be
> > grouped above the errors.  And then there is the errors...  A '2D
> > Coordinate' with 2 sources of error, both symmetric.. would have 4
> > FIELDs feeding the DataAxis/Coordinate content  ( x, y, xy_staterr,
> > xy_ranerr ).
>
> Yes, correlations are a bit of a challenge, but you'll have
> correlated errors in all kinds of contexts, so I'd argue that's for
> quantity again, not for coordinates themselves.

The Quantity you outline above (value + error) IS what is called Coordinate
in these models.  With the distinction that 'value' includes the axis/frame
reference.


>   Since doing this
> kind of advanced annotation right is difficult, my preference would
> be to postpone their definition until we have some experience with
> the simple cases.  You can compatibly add things later, but taking
> them away will probably mean a new major version, which would be
> painful even without lots of links between DMs.
>

I think the way we have modeled errors will facilitate growth/change on a
minor-version level.


> Having said that, I think these correlated errors are a case where DM
> objects need to be referenced; but that would be references *within*
> (hopefully) quantity, not between different DMs.
>
> I admit that I don't really look forward to defining quantity (i.e.,
> modelling errors, ranges, and that kind of thing) either, but I
> maintain it's much better to do this once properly (and then
> carefully evolve it in this single place) than to solve these
> analogous problems in each DM and/or domain separately.
>

Agreed.. again, since your Quantity == my Coordinate, the 'coords' model
would be that place.
If we agree on consolidating out the 'domain' aspect of that model, it
becomes a single set of objects which can be evolved as you suggest.
(the 'domain' variations move entirely to the coordsys model.. 'values +
axis refs' are domain savvy)

Mark
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.ivoa.net/pipermail/dm/attachments/20170322/3781b159/attachment.html>