<html>
<head>
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
</head>
<body>
<p><br>
</p>
<div class="moz-forward-container"><br>
<br>
-------- Message transféré --------
<table class="moz-email-headers-table" cellspacing="0"
cellpadding="0" border="0">
<tbody>
<tr>
<th valign="BASELINE" nowrap="nowrap" align="RIGHT">Sujet :
</th>
<td>Re: MIVOT: fully qualified attribute names</td>
</tr>
<tr>
<th valign="BASELINE" nowrap="nowrap" align="RIGHT">Date : </th>
<td>Fri, 17 Feb 2023 17:12:33 +0100</td>
</tr>
<tr>
<th valign="BASELINE" nowrap="nowrap" align="RIGHT">De : </th>
<td>BONNAREL FRANCOIS
<a class="moz-txt-link-rfc2396E" href="mailto:francois.bonnarel@astro.unistra.fr"><francois.bonnarel@astro.unistra.fr></a></td>
</tr>
<tr>
<th valign="BASELINE" nowrap="nowrap" align="RIGHT">Pour : </th>
<td>Laurent Michel <a class="moz-txt-link-rfc2396E" href="mailto:laurent.michel@astro.unistra.fr"><laurent.michel@astro.unistra.fr></a>,
<a class="moz-txt-link-abbreviated" href="mailto:dm@ivoa.net">dm@ivoa.net</a></td>
</tr>
</tbody>
</table>
<br>
<br>
Hi Markus, Laurent, all,<br>
Le 16/02/2023 à 18:28, Markus Demleitner a écrit :<br>
<blockquote type="cite">Dear Laurent,<br>
<br>
On Thu, Feb 16, 2023 at 04:40:32PM +0100, Laurent Michel wrote:<br>
<blockquote type="cite">We could continue playing ping-pong, but
we reached a point where<br>
there is no longer great benefits to expect by doing so.<br>
</blockquote>
Hm, I don't know about that -- at least I'm learning quite a
bit, and<br>
I think figuring things out in this way will help many others,
too.<br>
<br>
First, it made me notice another (perhaps small) snag:<br>
<br>
<blockquote type="cite">- all needed VODML files<br>
</blockquote>
These are a bit hairy to find at this point. Admittedly, they
*are*<br>
on <a class="moz-txt-link-freetext" href="https://ivoa.net/xml/">https://ivoa.net/xml/</a>, but they're intermingled with all
kinds of<br>
XSDs, and it's really hard to see what's ancient and (excuse me)<br>
stillborn and what's fresh and current. My suggestion would be:
Have an<br>
extra section for VO-DML files in the Schema repo so they are
not<br>
drowned in the legacy of more or less rusty XSD from the DM
group.<br>
And perhaps use a slightly different table pattern so there are
links<br>
to both the .vo-dml.xml and the generated HTML documentation.<br>
</blockquote>
<br>
Yes this has been pointed out in the TCG a couple of years ago.
But no decision was taken.<br>
<br>
Anyway Vo-dml.xml are not xsd schema. they are xml DOCUMENTS,
following the vodml xsd schema.<br>
<br>
vo-dml xml documents could be attachedd to the fron page of each
data model specification.<br>
<br>
<blockquote type="cite">I'd suggest "Formal VO-DML models" as a
section header, perhaps with<br>
VO-DML being a link to the spec.<br>
<br>
<blockquote type="cite">At this point, the annotator knows that
it must build an instance with a role given by the model; 2
options:<br>
taking the NAME (Markus option) —> @dmrole=error<br>
taking the VODML-ID prefixed with the model name (MIVOT) —>
@dmrole=meas:Measure.error<br>
</blockquote>
Ah! See, this is what I had missed -- sorry about that. The<br>
attribute names are not computed, they're taken from the VO-DML<br>
files, more specifically, their vodml-id elements (the existence
of<br>
which I had forgotten since when I had contributed to VO-DML).<br>
Still, for all I can tell, the syntax of the dmrole attribute is
not<br>
specified in MIVOT yet (at least searching for vodml-id just
turned<br>
up<br>
<br>
element_model.tex:All models that define vodml-ids used in the
annotation must be declared.<br>
<br>
for me). Perhaps you should put, somewhere above Table 2, some<br>
language like:<br>
<br>
The values of both dmrole and dmtype are constructed by taking<br>
the vodml-id of the respective VO-DML elements and prefixing it
with<br>
the canonical short name of the model and a colon.<br>
<br>
I can't *promise* that would have prevented me from having asked
for<br>
a clarification, but I'd claim there's a good chance.<br>
</blockquote>
Yes dm-roles are the vodml-id of classes and attributes. It's
unambiguous<br>
<blockquote type="cite"><br>
<blockquote type="cite">with the roles ala MIVOT , the client
selecting<br>
“.//INSTANCE[@dmrole=meas:Measure.error]: is sure of what it
gets<br>
with the roles ala Markus , the client selecting<br>
“.//INSTANCE[@dmrole=error]: will return anything playing any
role<br>
named error, either from the meas:Measure class or any other.
This<br>
is the price for using non-unique identifiers.<br>
</blockquote>
I note in passing that no client probably ever could learn
anything<br>
useful when it does this in either case, so that's a weak
argument.<br>
But then: While I'm still not particularly happy with the choice
of<br>
vodml-id, and I still think there is no good reason for that
choice,<br>
given the dmrole values are at least well-defined, it is
certainly<br>
not a blocker any more.<br>
<br>
<blockquote type="cite">We can add a section in the spec
relating the process above. I<br>
admit that would help.<br>
</blockquote>
A non-normative appendix should indeed say where to find models
and<br>
where to find the vodml-ids.<br>
<br>
<blockquote type="cite">I will be less open-minded about the
requirement of assigning the<br>
RFC ending with the release of software pieces operating all
steps<br>
of the model mapping process with a good test coverage
furthermore;<br>
very new in the IVOA landscape!<br>
</blockquote>
Well, the role string-computing function is no longer needed,
because<br>
it's now clear where the role strings come from. So scratch
that.<br>
<br>
What's not new in the VO is the demonstration of two (or, when
there<br>
are producers and consumers, three) interoperating
implementations.<br>
<br>
I *am* trying to provide one on the server side (that's why I'm<br>
here), but so far failed, partly because I couldn't figure out
how to<br>
use the existing models to annotate even the basic aspects of
the<br>
Gaia source catalogue (linking positions and proper motions,
linking<br>
scalar errors, defining photometries; note that I'm not asking
for<br>
annotation of the correlations any more -- sigh!).<br>
<br>
If I can't do that, I'd say it indicates *some* sort of problem:<br>
Perhaps our models are still insufficient -- but then there's no<br>
particular hurry to pass MIVOT (because we can't do anything
useful<br>
with it until the models are there).<br>
<br>
Another possibility is that I'm too dumb or too grumpy --<br>
</blockquote>
Markus !!!<br>
<blockquote type="cite">in that<br>
case someone else should jump in and provide a... well, "second"<br>
implementation.<br>
</blockquote>
<br>
It is true that we don't have all the data models to annotate all
our VOTables<br>
<br>
The missing level is made of integrating datamodels such as Mango,
Cube or maybe also TimeSeries, Spectrum (as specialisation of Cube
or Mango)<br>
<br>
But at least we have PhoTDM, Meas;, Coords which allow to
demonstrate some annotation capability with MIVOT.<br>
<br>
We have plenty of working examples for that.<br>
<br>
My view on this :<br>
<br>
It's possible to generate xml components for each class from
vo-dml-xml documents . Laurent's code does it.<br>
<br>
Now integrating annotation in any VO service framework, Dachs
or whatever, requires new software development. That's
unavoidable, with MiVOT or with an alternative which doesn't
exist.<br>
<br>
One way to do it with MIVOT is to add these xml components in
specific tables of the relational schema of the database (or the
TAP schema in case of a TAP service)<br>
<br>
Then the pieces of code which build the VOtable responses can
be modified to use on the fly these components to build the
appropriate annotation for each VOTable response.<br>
<br>
<br>
<br>
<br>
<blockquote type="cite">I'm using quotes here because very frankly
what I have seen so far as<br>
implementation examples is either outdated and does not
correspond to<br>
the spec as present, or it strongly looks like hand-crafted
examples.<br>
Whether or not the hand-crafted is right: For "implemenation",
I'd<br>
strong prefer something that exists in the wild (i.e., in an
actual<br>
data centre) and is consumed by something that is not completely<br>
contrived.<br>
</blockquote>
<br>
We have to break the deadlock loop.<br>
<br>
Validating an annotation mechanism only when we have a fully
integrated datamodel versus specify an annotation mechanism
validated by partial models and then go forward by building more
complete datamodels<br>
<br>
<blockquote type="cite"> only<br>
<br>
Remember my suggestion for an annotation syntax back in the
Workshop<br>
days, <a class="moz-txt-link-rfc2396E" href="https://github.com/msdemlei/astropy"><https://github.com/msdemlei/astropy></a>? You know,
that *would*<br>
have worked as a PR against astropy, and the things it consumed
where<br>
generated by a machine from an abstract representation in an
actual<br>
data centre. It's that sort of thing I'd really like to see for<br>
something as wide-reaching and ambitious as MIVOT.<br>
<br>
But, really, at this point I'd already be content with just
something<br>
coming out of an actual data centre (like mine) that some of
your<br>
client-side code can consume. But that's really the minimum in
terms<br>
of what I'll take as a "reference implementation" (as required
by<br>
Stdproc forever).<br>
<br>
<blockquote type="cite">The purpose of MIVOT is to map data on
models serialised in VODML.<br>
All the implementations we provide show that this requirement
is<br>
achieved even if we could dream about tools making the job
easier.<br>
</blockquote>
That's the second point: I have no idea just *what part* of
MIVOT is<br>
actually exercised by the examples mentioned on the RFC page.
There<br>
is simply *soo* much material in MIVOT that I feel unable to
decide<br>
what is and what is not covered by implementation examples. To<br>
cite the (to me) scariest part of the spec: There is a 5-way<br>
definition by cases for COLLECTION in mostly very dense
language, and<br>
I freely admit I can't understand most of it.<br>
<br>
I'd trust it a lot more if there were a non-author implementor
who<br>
had looked at it and found that it at least does not contradict<br>
anything else in the spec. I expect I'd have to spend days
working<br>
out what that is and writing test cases and all -- and sorry,
I'll<br>
not spend that time on something I think we should think about
in<br>
version 1.4 or version 1.5 of MIVOT, after we know *a lot* more.<br>
<br>
<blockquote type="cite">Actually I’ve code generating MIVOT
instances from VODML<br>
(github:ivoa/modelinstanceinvot-code<br>
branch:feature/instance-generator), i’m using it to build the<br>
snippets I was talking about i my previous mail. This code is
not<br>
ready to be published, I would be happy to do it but I’ve just
no<br>
time to complete the job; but anyway, this must not block the
REC.<br>
</blockquote>
But on the other hand, no urget use case is actually waiting for
this<br>
to become REC (or is one?), and given we're betting our
coordinate<br>
annotation on it, we can at least wait with REC until we
successfully<br>
exchange a 6-parameter astrometric solution with it and do an
epoch<br>
propagation, with one client and two services implemented by
three<br>
different people. I'm volunteering to be one on the server side,
but<br>
I'll need help making sense of MCT.<br>
</blockquote>
<br>
Photometry at least can strongly benefit from MIVOT<br>
<br>
Cheers<br>
<br>
François<br>
<br>
<blockquote type="cite"><br>
Even then I'd feel *a lot* more comfortable if MIVOT 1.0 just<br>
contained whatever is necessary for that exercise, because I am<br>
almost 100% sure anything not exercised in that way will contain
lots<br>
of contradictions and errors that will bug us later (that's<br>
invariably been the case for unused parts of specs that I have<br>
written).<br>
<br>
But if you really cannot be moved to postpone the complex stuff
until<br>
we better understand what we're doing, I'd not stand in the way
once<br>
we have the epoch propagation use case properly demonstrated.
But<br>
that, I feel, is not asking too much.<br>
<br>
Thanks,<br>
<br>
Markus<br>
</blockquote>
<br>
<br>
</div>
</body>
</html>