<div dir="ltr"><div><div><div><div><div><div>Markus/Jiri,<br><br></div>This review yesterday has inspired me to try something.<br></div>If the usage case of 'finding all coordinates in the document' is a valid one (and I think it is), then it does force some constraints on the modeling and/or annotation since all coordinates must be identifiable as being related.. or sharing some generic core elements.<br><br></div>So, I'm going to try something out which may help in this regard.<br></div>More later..<br></div><br><br></div>Mark<br><br><div><div><div><div><br></div></div></div></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Mar 16, 2017 at 4:58 PM, CresitelloDittmar, Mark <span dir="ltr"><<a href="mailto:mdittmar@cfa.harvard.edu" target="_blank">mdittmar@cfa.harvard.edu</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><div><div><div><div><div><div>Markus,<br><br></div><div>Sorry for my earlier frustration.. I have no issue with implementation feedback and am enjoying this interchange with you and Jiri.. it is invaluable.<br><br>first, w.r.t. to Quantity.. that is included in the 'ivoa' model described in the vo-dml document.<br></div></div>The description thereof has been moved to the normative part of the spec (Section 5)<br></div>and the vo-dml representation and schema are to be provided as per other models.<br><br></div>I started this response by extracting the corresponding elements from the SparseCube example I have generated.<br></div>see: <a href="https://volute.g-vo.org/svn/trunk/projects/dm/CubeDM-1.0/examples/chandra_events_annotated.vot" target="_blank">https://volute.g-vo.org/svn/<wbr>trunk/projects/dm/CubeDM-1.0/<wbr>examples/chandra_events_<wbr>annotated.vot</a><br><br></div><div>I will attach this, but when I got to the STC+Quantity stuff.. I started realizing there were more questions,<br></div><div>so I'll put those inline below:<br></div><div><br></div>It's important to note that the vo-dml annotation syntax is isolated from the VOTable, so the Group/Param<br></div>elements are NOT tagged with vodml-type/role.. but I think the relation can be seen.<br><div><div><br><div>The closest analogy is annotating where it copies the Param info, rather than refer to the existing Param, so I'll show that.<br></div><div><br></div><div><div><br><div><div><div><div><div class="gmail_extra"><br><div class="gmail_quote"><span class="">On Thu, Mar 16, 2017 at 7:03 AM, Markus Demleitner <span dir="ltr"><<a href="mailto:msdemlei@ari.uni-heidelberg.de" target="_blank">msdemlei@ari.uni-heidelberg.<wbr>de</a>></span> wrote:<br><blockquote style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex" class="gmail_quote"><br>Now, here's how I'd like such an annotation to look like. I'm using<br>
vodml-type and -role as attributes rather than elements here for<br>
readability, and I'm interspersing the comments; please allow me to<br>
indulge in improvising class and attribute names for the time being.<br>
I hope if they don't match current models they should at least<br>
readily map to them:<br>
<br></blockquote></span><div>sure..<br> <br></div><blockquote style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex" class="gmail_quote">
================= Dataset annotation =======================<br></blockquote><div><snip> Dataset stuff has good agreement <br> <br></div><span class=""><blockquote style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex" class="gmail_quote">
================== Cube annotation =========================<br>
<br>
<GROUP vodml-type="ndcube:Cube"><br>
<!-- No reason to have an extra type for time series; that's<br>
already defined in ds:Dataset.dataproductType and unlikely to<br>
be of relevance to a cube-only client (e.g., a plot program)<br>
anyway. --><br></blockquote><div><br></div></span><div>This particular Instance is a TimeSeriesCube from the diagrams, so the vodml-type would be <model>:TimeSeriesCube.<br></div><div>I don't think this can be the generic SparseCube, as the axes of that object are essentially equivalent.<br></div><div>I think this is the big distinction between the SparseCube and the specialized views such as TimeSeries, Spectrum, etc..<br></div><div>there is no dominant (independent) axis..<br><br></div><span class=""><blockquote style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex" class="gmail_quote">
<FIELDref vodml-role="independent-axis" ref="obs_date"/><br>
<!-- that's it; a client just counts<br>
*[@vodml-role="independent-axi<wbr>s" and knows the number of dimensions<br>
in the cube. All additional annotation is on the FIELD itself. --><br>
<br>
<FIELDref vodml-role="dependent-axis" ref="FLX"/><br>
<FIELDref vodml-role="dependent-axis" ref="MAG"/><br>
<!-- that's it; a single reference defines a "value" in this cube,<br>
and any further annotation is on the field itself, where non-cube<br>
clients can also use it. --><br>
</GROUP> <!-- I don't think much further metadata is needed here --><br>
<br></blockquote><div><br></div></span><div>Comparing with the Note diagram, the dependent/independent axes have 2 components<br></div><div> 1) field (presumably the 'value')<br></div><div> 2) error<br></div><div>The annotation above does not say whether "obs_date" is the 'value' or 'error' of the independent axis.<br></div><div><similar for "FLX" and "MAG"><br></div><span class=""><div> <br><br></div><blockquote style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex" class="gmail_quote">
============== STC+Quantity annotation =====================<br>
<br>
<GROUP vodml-type="stc:Time"><br>
<!-- the one place STC metadata is collected --><br>
<FIELDref vodml-role="value" ref="obs_date"/><br>
<!-- note how we "amend" metadata on obs_date here; by co-reference<br>
with the ndcube:Cube annotation, obs_date is *both* an independent<br>
axis *and* a time. --><br></blockquote></span><div><div><br>I do not believe one can "amend" metadata on objects. This instance is entirely independent of the above instance of "independent-axis". They both use the same column (obs_date) for the 'value', but they are not the same thing. <br><br></div></div><div>This is defining an 'Absolute Time coordinate'.. a value along the 'time' axis in a particular TimeFrame. The value is coming from a FIELD, so there are many of them, but who/what is using this Time 'column'? ie: This GROUP must be assigned to a role with multiplicity greater than one. (eg. in a CubeAxis.field in your diagrams or NDPoint.axis.observable in mine.<br><br></div><span class=""><blockquote style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex" class="gmail_quote">
<br>
<PARAM vodml-role="timescale" value="TT"/><br>
<PARAM vodml-role="timeformat" value="MJD"/><br>
<!-- timeformat is an invention; STC 1.0 uses classes to<br>
distinguish between JD, MJD, "ISO". --><br>
<PARAM vodml-role="referencePosition" value="BARYCENTER"/><br>
...<br></blockquote><div><br></div></span><div>Translating this.. This is Frame information in the Time domain.. and would be bundled within a grouping identified with the appropriate vodml-type.<br></div><div>Otherwise, there is no way of telling which object this 'timescale' role belongs to. This Group reverse-engineers to an Object with spec:<br></div><div> Time<br></div><div> +value:RealQuantity[*]<br></div><div> +timescale:TimeScale<br></div><div> +timeformat:string<br></div><div> +referencePosition:<wbr>StdReferencePosition<br><br></div><div>Maybe I'm interpreting it too literally. My point is that the structure must be represented in the annotation.. a Type has content which are assigned Roles, this is 1 level deep. One cannot assume that 'timescale' maps to "TimeFrame.timescale".<br></div><span class=""><div> </div><blockquote style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex" class="gmail_quote">
</GROUP><br>
<br>
<GROUP vodml-type="ivoa:Quantity"><br>
<!-- all measurements (can) have errors, min/max vals, etc, so<br>
there's no point separately modelling this in cube, stc,<br>
photometry, etc.; let's have ivoa:Quantity for that. --><br>
<FIELDref vodml-role="value" ref="obs_date"/><br>
<FIELDref vodml-role="standard-deviation<wbr>" ref="err_time"/><br>
<PARAM name="minimum" value="56493.339"/><br>
<PARAM name="maximum" value="56498.341"/><br>
<!-- which also does much of char:, without introducing 1000s of<br>
utypes --><br>
</GROUP><br>
<br></blockquote></span><div><br>OK.. so this maps more to the 'coords' model content (this is NOT in cube). Specifically, this would be 'coords:domain.temporal.Time' .<br></div><div>I see here ref="obs_date" is used again... is this supposed to 'cross-reference' to the stc:Time object above?, so that this Quantity.value IS an stc:Time object with value and associated Frame, which is also cross-referenced to the independent axis of the Cube? How does the user know if this Quantity or the stc:Time element is the cube "independent axis"?<br></div><div><br>The ivoa Quantity is an atomic physical value. It is used in both the value and error elements of a 'measurement'.<br></div><div><br>The coords model defines a pattern which associates the coordinate value with associated errors. (the pattern maps to your assertion that 'there is no point separately modeling this in cube, stc, photometry, etc.'). The pattern establishes the framework for all Coordinates (measurements) and each domain implementation must follow the pattern, adding any specialization needed. We are starting with a simple Error model, but as things develop, it can grow to accommodate more complex things (like probability distribution errors). All domain implementations should follow this pattern.<br><br></div><div>I'm not sure what you mean by min/max here<br></div><div> + a single 'measurement' can have a min/max uncertainty (error)<br></div><div> + a group of 'measurements' can have a min/max range (char)<br></div><div> + an axis can have a min/max range in the domain (axis.domainMin/Max in the coordsys model)<br><br></div><div>The identification of the 'error' as 'standard-deviation' is interesting.. this implies the value is a sampling and the error is calculated by the spread in the sampling.<br></div><div>The current error model is simpler than that, it allows for specifying a symmetrical error type, but not HOW that value was determined (standard deviation).<br></div><div><br> </div><blockquote style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex" class="gmail_quote">
============ Photometry+Quantity annotation ==================<br>
<br></blockquote><div><snip><br> <br></div><span class=""><blockquote style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex" class="gmail_quote">
<br>
<GROUP vodml-type="ivoa:Quantity"><br>
<!-- it's exactly the same thing as above for obs_date; if<br>
a client understands Quantity, it'll need no extra code, no extra<br>
utypes to interpret this as well as the obs_date annotation. --><br>
<FIELDref vodml-role="value" ref="FLX"/><br>
<br>
<FIELDref vodml-role="standard-deviation<wbr>" ref="FLXERR"/><br>
<!-- here, and not in a custom thing within ndcube or somewhere<br>
else, is the connection made between FLX and FLXERR; that way, a<br>
Quanitity-knowing client can figure out the error, not just one<br>
for NDCube --><br>
<PARAM name="minimum"...<br>
</GROUP><br></blockquote><div><br></div></span><div>So, the theme seems to be that you don't want the various model elements annotated within the annotation of another model?<br></div><div>If you look at my example, a user who understands Time measurements "coords:domain.temporal.Time" can find<br></div><div>them by scanning for INSTANCE with that dmtype.. even if it resides within an object they are not familiar with.<br><br></div><div>What one cannot do is find all coords (your Quantity here) because the underlying pattern is not actualized. We are not assuming that all measurements are modeled the same, and have the same error model. The dimensionality of some domains also hinders that because it requires multiple 'values' (not a problem) with cross-correlated errors (again, different error model than simple 1D domains).<br></div><div><br></div><div>Ah! that may not be entirely true. In my examples, I short-cut a relation.<br></div> cube:Observable has reference to coords:DerivedCoordinate<br><br></div><div class="gmail_quote">I short-cut it because this is the aggregation relation, and I figured one would know that cube:Observable is a coords:DerivedCoordinate.<br></div><div class="gmail_quote">But going through your example, I see an important tag would be missing that would allow a coords savvy, but non-cube savvy user to FIND all "coords:DerivedCoordinate" types. (This type is bound to follow the coords pattern.. so if a user has implemented the pattern, they should be able to handle at least some content of any implementations)<br></div><div class="gmail_quote"><br></div><div class="gmail_quote">Hmm.. Nope.. that wouldn't do it (sorry for the train of thought rambling). Even with that relation, the element at the target of the reference would be of the implemented type, not the base type. (so, it would show "coords:domain.temporal.Time", but not "coords:DerivedCoordinate"). One would have to interrogate the model definition to see that Time extends DerivedCoordinate.<br></div><div class="gmail_quote"><div> </div><blockquote style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex" class="gmail_quote">
============= Field declarations<br>
<FIELD ID="dateObs" name="dateObs"/> << obs_date<span class=""><br>
<FIELD ID="FLX" name="FLX"/><br>
<FIELD ID="FLXERR" name="FLXERR"/><br>
<br>
<br></span></blockquote><div><br></div><div>Wow.. that's a lot to take in..<span class="HOEnZb"><font color="#888888"><br></font></span></div><span class="HOEnZb"><font color="#888888"><div>Mark<br><br></div><div> </div></font></span></div></div></div></div></div></div></div></div></div></div></div>
</blockquote></div><br></div>