Time Series Cube DM - IVOA Note

[Laurino, Omar]

Markus,

at some point I thought you were arguing that a model should be able to
import another model without caring about its version. I think I now
understand you weren't. Rather, you are arguing that modeling should be
done in such a way as to reduce the amount of importing and reuse, at least
in some instances. If that's the case, then indeed we are on the same page.

> Still, if
> > they want to enable compatibility with a new major version of a model,
> they
> > need to create a new xml descriptor that describes the relationship with
> > the new model. It could be trivial to do so, but this depends on the


> Uh -- you lost me at the part with the compatibility with new
> major versions of a model.  Can you try to come up with an example?


Again, I was responding to something you probably weren't arguing in the
first place. What I meant is that if ModelA-1.0 imports ModelB-1.0 today,
and a year from now the DM group decides to support ModelB-2.0, they should
create a new VODML description file (and a new version) of ModelA that
explicitly imports ModelB-2.0. I was wondering (and doubting) this could be
done with a minor increment of ModelA. If there is no link, i.e. ModelA
does not import ModelB to begin with, a valid annotation could indeed
follow ModelA-1.0, ModelB-1.0, and even ModelB-2.0 in the same file. Which
is what you were pointing out in the first place, right?

Just to be sure we're on the same page: Within one instance, since
> we've agreed on a fixed mapping from prefixes to minor DM versions,
> you can only "use" one minor version (i.e., it's impossible (and
> would be silly) to have annotations for both NDCube-1.0 and
> NDCube-1.2).  Agreed?


Yes.

Thanks for the clarifications,

Omar.

On Fri, Feb 24, 2017 at 3:56 AM, Markus Demleitner <
msdemlei at ari.uni-heidelberg.de> wrote:

> Hi Omar,
>
> On Thu, Feb 23, 2017 at 09:47:30AM -0500, Laurino, Omar wrote:
> > On Thu, Feb 23, 2017 at 4:46 AM, Markus Demleitner <
> > msdemlei at ari.uni-heidelberg.de> wrote:
> >
> > > Now, my appeal is to have the same property across DMs: We should
> > > build things so that a document can be valid NDCube-1.0 regardless of
> > > whether it uses Dataset-1.0 or Dataset-2.4 or both, or even none
> > > of them.  The point of my previous mail was that that's feasible;
> > >
> >
> > The problem with this is that it would depend on how *exactly* the
> > importing model imports and uses the imported model.
>
> Exactly -- this is why I'm making all the noise here.  I'm
> advertising a model in which we minimize dependencies, and I argue
> that's the big dividend of all the pain we've gone into with VO-DML
> and proper annotation.
>
> > If a TimeSeries *is* a Dataset (the example is arbitrary), then it is
> > important to know what version of a Dataset it is, because Dataset-1.0
> and
> > Dataset-2.0 may be, by design, not compatible. Your point is still
> relevant
>
> Right.  And I'm arguing that we should keep inheritance and embedding
> at an absolute minimum, using it only when technically necessary.  In
> your example, I don't believe there actually should be a type
> TimeSeries (what would it convey?).  There's Dataset (and that would
> have an attribute dataproduct_type="timeseries" as in obscore).
>
> Time series clients would then know it's something for them.  They'd
> then look for annotation they know.  Typically, they'll look for an
> NDCube annotation, but that's independent of the fact that we have
> time series here.  And in particular, things remain time series
> whether there's NDCube-1.0 or NDCube-2.4 annotation in there, or
> both.
>
> Finally, the time series will contain a time, so we need STC
> annotation, and there are dependent axis that could have STC
> annotations (for time series over positions), or photometry, or may
> have spigot annotation.  Neither NDCube nor Dataset need to know
> anything about the details here.
>
> > for minor increments, though. Depending on the type of parser, it might
> not
> > really matter if a TimeSeries is expressed in terms of Dataset-1.0 or
> > Dataset-1.2, because by design a Dataset-1.2 valid annotation is also a
> > valid Dataset-1.0 annotation, and vice versa. A validator would still be
> > able to validate the instance, for example.
>
> Well, I'd argue that by definition, Dataset-1.2 and Dataset-1.0 must
> be interchangable (with 1.2 being able to express a bit more, of
> course).  That, I claim, is the definition of a minor update, and
> that's what we're trying to reflect on the schema side (RFC still
> open: http://wiki.ivoa.net/twiki/bin/view/IVOA/XMLVersRFC).
>
> > In any case an importing VODML model MUST say what models it is
> importing,
> > otherwise it cannot refer to their elements, not even to state that
> > "something" has "such and such" datatype, e.g. that a ts:TimeSeries is a
>
> True.  Which is why this should be avoided, except for a well-defined
> set of well-thought-out, rarely changing basic classes.  Well, of
> course, there's the other application, where individual institutions
> take an IVOA data model and adapt it to their needs.
>
> Even then multiple imports should, by extension, be strongly
> discouraged, so the Academy of Sciences of Paraphernalia would
> separately extend NDCube to paraph-ndcube: and STC to paraph-stc: if
> they absolutely have to.
>
> As I said a few mails back, four points with three connections each
> result in a completely rigid tetrahedron in which you can't move
> anything any more.
>
> > Now, the designers of the importing model can be wise and reduce the
> number
>
> s/can/should/
>
> > of references to an imported model if the coupling between the models is
> > inherently loose. This might involve using object composition over type
> > inheritance, if this makes sense from a modeling point of view. Still, if
> > they want to enable compatibility with a new major version of a model,
> they
> > need to create a new xml descriptor that describes the relationship with
> > the new model. It could be trivial to do so, but this depends on the
>
> Uh -- you lost me at the part with the compatibility with new
> major versions of a model.  Can you try to come up with an example?
>
> > Most likely, if we wanted to enable this (new) requirement we would need
> to
> > make some tweaks to the VODML spec, and possibly to the mapping spec. One
> > might say that TimeSeriesDM imports the Dataset type from any version of
> > DatasetDM, and the binding is done at serialization time, i.e. when a
>
> No, I don't think anything needs to change in the draft specs to
> accomodate this.  There should simply be as few imports as
> possible, and as I said above I'm pretty sure there's no need for
> imports of DMs like DatasetDM at all.  STC *might* be a slightly
> different matter if we keep a Characterisation DM, but even there I'd
> advocate (until someone shows me an example where that paradigm
> breaks) there char: should only talk about generic "limits", and the
> actual physical annotation of these limits can be kept out of char: and
> just live in STC, phot:, or spigot:, keeping char: nicely generic.
>
> > However, I could see a model (probably not DatasetDM) removing a type
> > between major increments, possibly leaving importing models with dangling
> > references to non-existent types, e.g. ad absurdum DatasetDM-2.0 might
> > remove the Dataset type altogether, so TimeSeries would be left with a
> > dangling reference to a Dataset type that does not exist anymore.
>
> Yes, obviously, such references cannot go across major versions;
> that's what major versions are about: They break compatibility, and
> the big art is to make that possible without breaking clients.  I
> think VO-DML and out annotation plans have all it takes to pull off
> this trick.
>
> Whether references to and within DMs contain minor versions or not is
> orthogonal to this discussion, but since we've touched it I take the
> liberty of mentioning that I've  argued in
> http://mail.ivoa.net/pipermail/dm/2017-January/005458.html they
> should not.
>
> Just to be sure we're on the same page: Within one instance, since
> we've agreed on a fixed mapping from prefixes to minor DM versions,
> you can only "use" one minor version (i.e., it's impossible (and
> would be silly) to have annotations for both NDCube-1.0 and
> NDCube-1.2).  Agreed?
>
> > The only way I can see this working at this point is by removing the link
> > altogether. It could be that TimeSeries is just an additional layer of
> > metadata that can be added to a file on top of the existing ones. So
> > something might be annotated as being both a Dataset (of any version)
> and a
>
> Yes! Except, as I said:
>
> TL;DR: I don't think we need the notion of TimeSeries except as
> a term in the values of dataproduct_type at all: it's just a Dataset,
> and NDCube; and if you look, you'll see that there's just one
> independent axis that happens to be a time in STC).
>
>           -- Markus
>



-- 
Omar Laurino
Smithsonian Astrophysical Observatory
Harvard-Smithsonian Center for Astrophysics
100 Acorn Park Dr. R-377 MS-81
02140 Cambridge, MA
(617) 495-7227
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.ivoa.net/pipermail/dm/attachments/20170224/bee4a369/attachment.html>