Time Series Cube DM - IVOA Note

Fri Mar 17 13:24:14 CET 2017

Markus/Jiri,

This review yesterday has inspired me to try something.
If the usage case of 'finding all coordinates in the document' is a valid
one (and I think it is), then it does force some constraints on the
modeling and/or annotation since all coordinates must be identifiable as
being related.. or sharing some generic core elements.

So, I'm going to try something out which may help in this regard.
More later..

Mark

On Thu, Mar 16, 2017 at 4:58 PM, CresitelloDittmar, Mark <
mdittmar at cfa.harvard.edu> wrote:

> Markus,
>
> Sorry for my earlier frustration.. I have no issue with implementation
> feedback and am enjoying this interchange with you and Jiri.. it is
> invaluable.
>
> first, w.r.t. to Quantity.. that is included in the 'ivoa' model described
> in the vo-dml document.
> The description thereof has been moved to the normative part of the spec
> (Section 5)
> and the vo-dml representation and schema are to be provided as per other
> models.
>
> I started this response by extracting the corresponding elements from the
> SparseCube example I have generated.
> see: https://volute.g-vo.org/svn/trunk/projects/dm/CubeDM-1.0/
> examples/chandra_events_annotated.vot
>
> I will attach this, but when I got to the STC+Quantity stuff.. I started
> realizing there were more questions,
> so I'll put those inline below:
>
> It's important to note that the vo-dml annotation syntax is isolated from
> the VOTable, so the Group/Param
> elements are NOT tagged with vodml-type/role.. but I think the relation
> can be seen.
>
> The closest analogy is annotating where it copies the Param info, rather
> than refer to the existing Param, so I'll show that.
>
>
>
> On Thu, Mar 16, 2017 at 7:03 AM, Markus Demleitner <
> msdemlei at ari.uni-heidelberg.de> wrote:
>
>>
>> Now, here's how I'd like such an annotation to look like.  I'm using
>> vodml-type and -role as attributes rather than elements here for
>> readability, and I'm interspersing the comments; please allow me to
>> indulge in improvising class and attribute names for the time being.
>> I hope if they don't match current models they should at least
>> readily map to them:
>>
>> sure..
>
>
>> ================= Dataset annotation =======================
>>
> <snip>  Dataset stuff has good agreement
>
>
>> ================== Cube annotation =========================
>>
>> <GROUP vodml-type="ndcube:Cube">
>>    <!-- No reason to have an extra type for time series; that's
>>     already defined in ds:Dataset.dataproductType and unlikely to
>>     be of relevance to a cube-only client (e.g., a plot program)
>>     anyway. -->
>>
>
> This particular Instance is a TimeSeriesCube from the diagrams, so the
> vodml-type would be <model>:TimeSeriesCube.
> I don't think this can be the generic SparseCube, as the axes of that
> object are essentially equivalent.
> I think this is the big distinction between the SparseCube and the
> specialized views such as TimeSeries, Spectrum, etc..
> there is no dominant (independent) axis..
>
>   <FIELDref vodml-role="independent-axis" ref="obs_date"/>
>>   <!-- that's it; a client just counts
>>   *[@vodml-role="independent-axis" and knows the number of dimensions
>>   in the cube.  All additional annotation is on the FIELD itself. -->
>>
>>   <FIELDref vodml-role="dependent-axis" ref="FLX"/>
>>   <FIELDref vodml-role="dependent-axis" ref="MAG"/>
>>   <!-- that's it; a single reference defines a "value" in this cube,
>>   and any further annotation is on the field itself, where non-cube
>>   clients can also use it. -->
>> </GROUP>  <!-- I don't think much further metadata is needed here -->
>>
>>
> Comparing with the Note diagram, the dependent/independent axes have 2
> components
>   1) field (presumably the 'value')
>   2) error
> The annotation above does not say whether "obs_date" is the 'value' or
> 'error' of the independent axis.
> <similar for "FLX" and "MAG">
>
>
> ============== STC+Quantity annotation =====================
>>
>> <GROUP vodml-type="stc:Time">
>>   <!-- the one place STC metadata is collected -->
>>   <FIELDref vodml-role="value" ref="obs_date"/>
>>   <!-- note how we "amend" metadata on obs_date here; by co-reference
>>   with the ndcube:Cube annotation, obs_date is *both* an independent
>>   axis *and* a time. -->
>>
>
> I do not believe one can "amend" metadata on objects. This instance is
> entirely independent of the above instance of "independent-axis".  They
> both use the same column (obs_date) for the 'value', but they are not the
> same thing.
>
> This is defining an 'Absolute Time coordinate'.. a value along the 'time'
> axis in a particular TimeFrame.   The value is coming from a FIELD, so
> there are many of them, but who/what is using this Time 'column'?  ie: This
> GROUP must be assigned to a role with multiplicity greater than one.  (eg.
> in a CubeAxis.field in your diagrams or NDPoint.axis.observable in mine.
>
>
>>   <PARAM vodml-role="timescale" value="TT"/>
>>   <PARAM vodml-role="timeformat" value="MJD"/>
>>   <!-- timeformat is an invention; STC 1.0 uses classes to
>>   distinguish between JD, MJD, "ISO". -->
>>   <PARAM vodml-role="referencePosition" value="BARYCENTER"/>
>>   ...
>>
>
> Translating this.. This is Frame information in the Time domain.. and
> would be bundled within a grouping identified with the appropriate
> vodml-type.
> Otherwise, there is no way of telling which object this 'timescale' role
> belongs to.  This Group reverse-engineers to an Object with spec:
>   Time
>     +value:RealQuantity[*]
>     +timescale:TimeScale
>     +timeformat:string
>     +referencePosition:StdReferencePosition
>
> Maybe I'm interpreting it too literally.  My point is that the structure
> must be represented in the annotation.. a Type has content which are
> assigned Roles, this is 1 level deep.  One cannot assume that 'timescale'
> maps to "TimeFrame.timescale".
>
>
>> </GROUP>
>>
>> <GROUP vodml-type="ivoa:Quantity">
>>   <!-- all measurements (can) have errors, min/max vals, etc, so
>>   there's no point separately modelling this in cube, stc,
>>   photometry, etc.; let's have ivoa:Quantity for that. -->
>>   <FIELDref vodml-role="value" ref="obs_date"/>
>>   <FIELDref vodml-role="standard-deviation" ref="err_time"/>
>>   <PARAM name="minimum" value="56493.339"/>
>>   <PARAM name="maximum" value="56498.341"/>
>>   <!-- which also does much of char:, without introducing 1000s of
>>   utypes -->
>> </GROUP>
>>
>>
> OK.. so this maps more to the 'coords' model content (this is NOT in
> cube).  Specifically, this would be 'coords:domain.temporal.Time' .
> I see here ref="obs_date" is used again...  is this supposed to
> 'cross-reference' to the stc:Time object above?, so that this
> Quantity.value IS an stc:Time object with value and associated Frame, which
> is also cross-referenced to the independent axis of the Cube?  How does the
> user know if this Quantity or the stc:Time element is the cube "independent
> axis"?
>
> The ivoa Quantity is an atomic physical value.  It is used in both the
> value and error elements of a 'measurement'.
>
> The coords model defines a pattern which associates the coordinate value
> with associated errors.  (the pattern maps to your assertion that 'there is
> no point separately modeling this in cube, stc, photometry, etc.').  The
> pattern establishes the framework for all Coordinates (measurements) and
> each domain implementation must follow the pattern, adding any
> specialization needed.  We are starting with a simple Error model, but as
> things develop, it can grow to accommodate more complex things (like
> probability distribution errors).  All domain implementations should follow
> this pattern.
>
> I'm not sure what you mean by min/max here
>   + a single 'measurement' can have a min/max uncertainty (error)
>   + a group of 'measurements' can have a min/max range (char)
>   + an axis can have a min/max range in the domain (axis.domainMin/Max in
> the coordsys model)
>
> The identification of the 'error' as 'standard-deviation' is interesting..
> this implies the value is a sampling and the error is calculated by the
> spread in the sampling.
> The current error model is simpler than that, it allows for specifying a
> symmetrical error type, but not HOW that value was determined (standard
> deviation).
>
>
>
>> ============ Photometry+Quantity annotation ==================
>>
>> <snip>
>
>
>>
>> <GROUP vodml-type="ivoa:Quantity">
>>   <!-- it's exactly the same thing as above for obs_date; if
>>   a client understands Quantity, it'll need no extra code, no extra
>>   utypes to interpret this as well as the obs_date annotation. -->
>>   <FIELDref vodml-role="value" ref="FLX"/>
>>
>>   <FIELDref vodml-role="standard-deviation" ref="FLXERR"/>
>>   <!-- here, and not in a custom thing within ndcube or somewhere
>>   else, is the connection made between FLX and FLXERR; that way, a
>>   Quanitity-knowing client can figure out the error, not just one
>>   for NDCube -->
>>   <PARAM name="minimum"...
>> </GROUP>
>>
>
> So, the theme seems to be that you don't want the various model elements
> annotated within the annotation of another model?
> If you look at my example,  a user who understands Time measurements
> "coords:domain.temporal.Time" can find
> them by scanning for INSTANCE with that dmtype.. even if it resides within
> an object they are not familiar with.
>
> What one cannot do is find all coords (your Quantity here) because the
> underlying pattern is not actualized.  We are not assuming that all
> measurements are modeled the same, and have the same error model.  The
> dimensionality of some domains also hinders that because it requires
> multiple 'values' (not a problem) with cross-correlated errors (again,
> different error model than simple 1D domains).
>
> Ah! that may not be entirely true.  In my examples, I short-cut a relation.
>   cube:Observable has reference to coords:DerivedCoordinate
>
> I short-cut it because this is the aggregation relation, and I figured one
> would know that cube:Observable is a coords:DerivedCoordinate.
> But going through your example, I see an important tag would be missing
> that would allow a coords savvy, but non-cube savvy user to FIND all
> "coords:DerivedCoordinate" types.  (This type is bound to follow the coords
> pattern.. so if a user has implemented the pattern, they should be able to
> handle at least some content of any implementations)
>
> Hmm.. Nope.. that wouldn't do it (sorry for the train of thought
> rambling).  Even with that relation, the element at the target of the
> reference would be of the implemented type, not the base type.  (so, it
> would show "coords:domain.temporal.Time", but not
> "coords:DerivedCoordinate").  One would have to interrogate the model
> definition to see that Time extends DerivedCoordinate.
>
>
>> ============= Field declarations
>> <FIELD ID="dateObs" name="dateObs"/>   << obs_date
>> <FIELD ID="FLX" name="FLX"/>
>> <FIELD ID="FLXERR" name="FLXERR"/>
>>
>>
>>
> Wow.. that's a lot to take in..
> Mark
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.ivoa.net/pipermail/dm/attachments/20170317/e1768013/attachment-0001.html>