Time Series Cube DM - IVOA Note

[Laurino, Omar]

On Mon, Mar 27, 2017 at 1:55 PM, Markus Demleitner <
msdemlei at ari.uni-heidelberg.de> wrote:

> I agree with Omar that concrete serialisations is what we should now
> look at; let's do some!
>

I am working on them. I got stuck home sick, unfortunately, but hopefully
I'll have something to share by the end of the week.

That is also why I think the equation coordinate = quantity that
> Mark makes in a parallel branch of this thread means that this part
> of the model needs to move "up"; we'll have this pattern for
> temperatures and photometry as well (and whatever else), and their
> errors will be correlated as well, and people will want to represent
> their statistcal properties as well, and perhaps correlate value and
> derivative, and so on.  It would be shame if STC and photometry, say,
> had different models for their, well, quantities.


Agreed. Which is one of the reasons why I think this conversation has been
very constructive and productive.

Our standards live a lot longer than the last new-fangled
> distributed peer-to-peer NoSQL social blockchain web glitz.  And
> therefore for us dependencies are even more expensive.


I can't disagree with this one, but I wonder if we really agree in
practice. We'll see when we get to the concrete implementations.

Well, yes.  But of course when someone does a major version of a
> model you depend on, you'll have to issue a new major version of your
> own data model (and break *your* clients in turn) where otherwise you
> could probably have just left your DM alone.  So, my urging is
> exactly to avoid having to build a lot of major DM versions
> just because a new major version was necessary for one DM (ivoa,
> quantity, and STC excepted).


At the same time, I think we should make sure that when such an update is
really necessary, our framework makes it easy to update the downstream
model. It's always a trade-off. The easier it is to adopt healthy patterns,
the easier it is to fall into disruptive anti-pattern pits.

In the time series example, more than a time series *data model* I think
time series can just be seen as *instances* of a common, more generic data
model, that is itself a lightweight one. A client could specialize into
dealing with a simpler serialization (e.g. my app only deals with time
series, for some reason) while at the same time a more general client (a
plotting app, a fitting app) should be able to recognize there are axis,
their generic properties, and so on. This is part of my example(s).

Oh, by the way, this *is* something I'd like to change in the VO-DML
> spec: I think ivoa:quantity is not very helpful while hogging the term in
> a way that might become confusing later.  Is anyone really attached
> to it?  Using it already, perhaps?


VO-DML's "quantity" maps, by design, to VOTable's PARAM and FIELD, i.e. it
has a value, a unit, and implicitly a data type. I am not particularly
attached to the name, but I think we are dealing with two different things:
one is the (value, unit, data type) tuple, one is a more complex thing that
can have possibly multiple values, possibly a complex error object
associated to different kinds of errors, e.g. statistical, systematic, etc.
In simple cases, this is just "3.5 +- 0.3 microns". In my mind, the first
thing is named "quantity" and the second one is named "measurement". I am
not attached to the names themselves, but let's make sure we are on the
same page on the meaning before we, possibly, change the signifier.

We might also need an ivoa:NumericQuantity and an ivoa:number types that
abstract all numeric quantities and their values, to simplify both the
modeler's and the client's life.

  <GROUP vodml-type="stc:Coordinate">
>     <PARAMref vodml-role="value" vodml-type="Coord2" ref="pt"/>
>   </GROUP>
>   <PARAM ID="pt" xtype="POINT" datatype="real"
>     arraysize="2" value="23.3 41"/>
>   <GROUP vodml-type="stc:Coordinate">
>     <GROUP vodml-role="value" vodml-type="Coord2">
>       <PARAMref vodml-role="C1" ref="ra"/>
>       <PARAMref vodml-role="C2" ref="dec"/>
>     </GROUP>
>   </GROUP>
>   <PARAM ID="ra" value="23.3"/>
>   <PARAM ID="dec" value="41"/>


Would you have both annotations in the same file? How should a client
(unaware of the enclosing model) know this is two different representations
of the same coordinate rather than two distinct coordinates? I would rather
be in favor of specific mapping rules for certain types, if that makes
sense, which is what we already do for ivoa:Quantity. Coord2 would be
serialized as a DALI POINT, if that makes sense. Admittedly, I haven't
given this possibility enough thought, so I am not sure how convenient that
would be or what repercussions it might have down the road.

Omar.


-- 
Omar Laurino
Smithsonian Astrophysical Observatory
Harvard-Smithsonian Center for Astrophysics
100 Acorn Park Dr. R-377 MS-81
02140 Cambridge, MA
(617) 495-7227
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.ivoa.net/pipermail/dm/attachments/20170328/83ba8dbb/attachment.html>