<div dir="ltr"><div class="gmail_extra"><p class="gmail-MsoPlainText">Hi Markus<span></span></p>
<p class="gmail-MsoPlainText"><span> </span></p>
<p class="gmail-MsoPlainText">Tl,dr: In the mapping spec for LITERAL we must pay
careful attention to the serialization of LITERAL values, and may want to
"just" use PARAM.<span></span></p>
<p class="gmail-MsoPlainText">Details to be worked out, but we may not be able to keep
the separation between a VOTable and separate vo-dml-maping schema.<span></span></p>
<p class="gmail-MsoPlainText">More comments below.<span></span></p>
<p class="gmail-MsoPlainText"><span> </span></p>
<p class="gmail-MsoPlainText">> -----Original Message-----<span></span></p>
<p class="gmail-MsoPlainText">> From: <a href="mailto:dm-bounces@ivoa.net">dm-bounces@ivoa.net</a> [<a href="mailto:dm-bounces@ivoa.net">mailto:dm-bounces@ivoa.net</a>] On Behalf Of <span></span></p>
<p class="gmail-MsoPlainText">> Markus Demleitner<span></span></p>
<p class="gmail-MsoPlainText">> Sent: Thursday, April 13, 2017 5:18 AM<span></span></p>
<p class="gmail-MsoPlainText">> To: <a href="mailto:dm@ivoa.net">dm@ivoa.net</a><span></span></p>
<p class="gmail-MsoPlainText">> Subject: Re: ivoa: DM type system<span></span></p>
<p class="gmail-MsoPlainText">> <span></span></p>
<p class="gmail-MsoPlainText">> Hi Gerard,<span></span></p>
<p class="gmail-MsoPlainText">> <span></span></p>
<p class="gmail-MsoPlainText">> On Wed, Apr 12, 2017 at 10:45:38AM -0400, Gerard
Lemson wrote:<span></span></p>
<p class="gmail-MsoPlainText">> > It seems that at least it is agreed that we
should distinguish the <span></span></p>
<p class="gmail-MsoPlainText">> > VO-DML spec from the serialization/mapping
spec.<span></span></p>
<p class="gmail-MsoPlainText">> ><span></span></p>
<p class="gmail-MsoPlainText">> > And that the issue Markus brings up is one for
the latter only, <span></span></p>
<p class="gmail-MsoPlainText">> > apart from possible discussions on the contents
of the ivoa model, <span></span></p>
<p class="gmail-MsoPlainText">> > which belongs to the RFC page?<span></span></p>
<p class="gmail-MsoPlainText">> <span></span></p>
<p class="gmail-MsoPlainText">> Well, I've put it there, and as regards pulling the
ivoa data model <span></span></p>
<p class="gmail-MsoPlainText">> out of VO- DML proper, I think there still is a
point. Keeping what <span></span></p>
<p class="gmail-MsoPlainText">> it specifies fluid until we have actual experience
how these things <span></span></p>
<p class="gmail-MsoPlainText">> work out while fundamental things like stc and
whatever we use to <span></span></p>
<p class="gmail-MsoPlainText">> model value+error are in flux would seem wise to me.<span></span></p>
<p class="gmail-MsoPlainText">> <span></span></p>
<p class="gmail-MsoPlainText">Will have a look at RFC page again and reply there as
well.<span></span></p>
<p class="gmail-MsoPlainText"><span> </span></p>
<p class="gmail-MsoPlainText">> But you're right, my primary concern here is LITERAL
within VOTable, <span></span></p>
<p class="gmail-MsoPlainText">> and thus a mapping issue.<span></span></p>
<p class="gmail-MsoPlainText">> <span></span></p>
<p class="gmail-MsoPlainText">And especially a serialization issue I think. For indeed
VO-DML says NOTHING about how to serialize instances.<span></span></p>
<p class="gmail-MsoPlainText">So you're right that when using a LITERAL value, only
specifying its VO-DML primitive type may not be sufficient in general.<span></span></p>
<p class="gmail-MsoPlainText">It would be nice to have a common way to specify this,
which so far in the mapping doc we did not do.<span></span></p>
<p class="gmail-MsoPlainText">Whether VOTable elements would suffice I don't know,
maybe it is up to protocols to define this as you seem to be doing below in
DALI?<span></span></p>
<p class="gmail-MsoPlainText">> <span></span></p>
<p class="gmail-MsoPlainText">> > > Markus Demleitner <Wednesday, April 12,
2017 5:04 AM> Are there <span></span></p>
<p class="gmail-MsoPlainText">> > > any other cases except ENUMs?<span></span></p>
<p class="gmail-MsoPlainText">> ><span></span></p>
<p class="gmail-MsoPlainText">> > datetime!
Any custom primitive types some model defines. For <span></span></p>
<p class="gmail-MsoPlainText">> > example Mark CD has been wanting to add a URL
type as separate from anyURI).<span></span></p>
<p class="gmail-MsoPlainText">> <span></span></p>
<p class="gmail-MsoPlainText">> I feared it would come to this. After long contortions, we eventually <span></span></p>
<p class="gmail-MsoPlainText">> reached agreement on using<span></span></p>
<p class="gmail-MsoPlainText">> <span></span></p>
<p class="gmail-MsoPlainText">> datatype="char",
arraysize="*", xtype="timestamp"<span></span></p>
<p class="gmail-MsoPlainText">> <span></span></p>
<p class="gmail-MsoPlainText">> in DALI, sect. 3.3.3, for these.<span></span></p>
<p class="gmail-MsoPlainText">> <span></span></p>
<p class="gmail-MsoPlainText">Ok, sorry for not keeping up with that. Is it DALI 3.1.2?<span></span></p>
<p class="gmail-MsoPlainText">Does this mean you're
favoring passing such choices off to the protocols then? <span></span></p>
<p class="gmail-MsoPlainText">For interpreters that do not know that protocol this
would not help. <span></span></p>
<p class="gmail-MsoPlainText">But if this is allowed, could this not extend to the
VO-DML literal serializations as well?<span></span></p>
<p class="gmail-MsoPlainText"><span> </span></p>
<p class="gmail-MsoPlainText"><span> </span></p>
<p class="gmail-MsoPlainText">> > Fields outside of astronomy introduce
"zip-code" type as a string <span></span></p>
<p class="gmail-MsoPlainText">> > obeying certain restrictions.<span></span></p>
<p class="gmail-MsoPlainText">> ><span></span></p>
<p class="gmail-MsoPlainText">> > Similarly decimal can be seen as a rational
with denominator a power of 10.<span></span></p>
<p class="gmail-MsoPlainText">> ><span></span></p>
<p class="gmail-MsoPlainText">> > No doubt there are many others.<span></span></p>
<p class="gmail-MsoPlainText">> <span></span></p>
<p class="gmail-MsoPlainText">> Sure.
Geometries in VOTable.<span></span></p>
<p class="gmail-MsoPlainText">> <span></span></p>
<p class="gmail-MsoPlainText">> I appreciate it would be great if we could introduce
some clean, well- <span></span></p>
<p class="gmail-MsoPlainText">> thought-out system for doing what's now done
(admittedly<span></span></p>
<p class="gmail-MsoPlainText">> suboptimally) with xtypes in VOTable.<span></span></p>
<p class="gmail-MsoPlainText">> <span></span></p>
<p class="gmail-MsoPlainText">> But if the price for this is that people, within one
VOTable, have to <span></span></p>
<p class="gmail-MsoPlainText">> recognise timestamps in LITERALs by seeing it's <span></span></p>
<p class="gmail-MsoPlainText">> vodml-type="ivoa:datetime" and using one
literal parser, while having <span></span></p>
<p class="gmail-MsoPlainText">> to check xtype and use a different literal parser
when it's in PARAM makes me cringe.<span></span></p>
<p class="gmail-MsoPlainText">> <span></span></p>
<p class="gmail-MsoPlainText">> Plus, it won't stop with datetimes.<span></span></p>
<p class="gmail-MsoPlainText">> <span></span></p>
<p class="gmail-MsoPlainText">Sure. So are you arguing we should rely on xtype as a
kind of free-for-all label plus a protocol-based serialization prescription? For datatype, even with arraysize will in general not be sufficient.</p>
<p class="gmail-MsoPlainText">> <span></span></p>
<p class="gmail-MsoPlainText">> > Introduction of such custom primitives is
easily accommodated in our <span></span></p>
<p class="gmail-MsoPlainText">> > VO-DML mechanism, where the basic primitives
are explicitly defined <span></span></p>
<p class="gmail-MsoPlainText">> > as types and<span></span></p>
<p class="gmail-MsoPlainText">> <span></span></p>
<p class="gmail-MsoPlainText">> I have no doubts that it's easy in VO-DML. If it stays there, I don't <span></span></p>
<p class="gmail-MsoPlainText">> have a major problem with this. If it bleeds out into other <span></span></p>
<p class="gmail-MsoPlainText">> standards, that's a different matter; if we're not
careful there, the <span></span></p>
<p class="gmail-MsoPlainText">> complete system will be an inconsistent mess of
duplicate <span></span></p>
<p class="gmail-MsoPlainText">> specifications.
And people will hate us or, worse, ignore us.<span></span></p>
<p class="gmail-MsoPlainText">> <span></span></p>
<p class="gmail-MsoPlainText">> Yes, it sucks to always have to compromise. But the alternative is worse.<span></span></p>
<p class="gmail-MsoPlainText">> <span></span></p>
<p class="gmail-MsoPlainText"><span> </span></p>
<p class="gmail-MsoPlainText"><span> </span></p>
<p class="gmail-MsoPlainText">> > Also, VO-DML aims to be representation neutral,
hence restricts for <span></span></p>
<p class="gmail-MsoPlainText">> > example its numeric types to the mathematical
number systems <span></span></p>
<p class="gmail-MsoPlainText">> > (N,Z,Q,R,C), rather than using the software
representations,<span></span></p>
<p class="gmail-MsoPlainText">> short/int/long(/longer/longest?).<span></span></p>
<p class="gmail-MsoPlainText">> > This is one reason why rational is included.
Also it is datetime, <span></span></p>
<p class="gmail-MsoPlainText">> > not date, or time, or MJD, UTC, or whatever.
Again, those are deemed <span></span></p>
<p class="gmail-MsoPlainText">> > to be serialization issues and it completely
proper for VOTable to <span></span></p>
<p class="gmail-MsoPlainText">> > define those, but no need to do so in VO-DML.<span></span></p>
<p class="gmail-MsoPlainText">> <span></span></p>
<p class="gmail-MsoPlainText">> Again, it's great if VO-DML has conceptual clarity
here, but if <span></span></p>
<p class="gmail-MsoPlainText">> VOTable implementors have to support another type
system with lots of <span></span></p>
<p class="gmail-MsoPlainText">> rules of its own that's a major implemenation
liability. And given <span></span></p>
<p class="gmail-MsoPlainText">> there's CONSTANT and LITERAL, somehow the mapping
document will still <span></span></p>
<p class="gmail-MsoPlainText">> have to talk about VOTable types and literals.<span></span></p>
<p class="gmail-MsoPlainText">> <span></span></p>
<p class="gmail-MsoPlainText">It will, it is "only" in the formal definition
that it might be nice to be able to separate out the schemas.<span></span></p>
<p class="gmail-MsoPlainText"><span> </span></p>
<p class="gmail-MsoPlainText">> > > (a) normal clients just mind that they can
parse the thing. For <span></span></p>
<p class="gmail-MsoPlainText">> > > that, they just need the VOTable datatype
(and they already have <span></span></p>
<p class="gmail-MsoPlainText">> > > parsers for<span></span></p>
<p class="gmail-MsoPlainText">> > > that)<span></span></p>
<p class="gmail-MsoPlainText">> ><span></span></p>
<p class="gmail-MsoPlainText">> > I expect with 'normal clients' you mean those
that will ignore the <span></span></p>
<p class="gmail-MsoPlainText">> > whole VODML element? If so there is no issue as
only there LITERALs <span></span></p>
<p class="gmail-MsoPlainText">> > will occur<span></span></p>
<p class="gmail-MsoPlainText">> <span></span></p>
<p class="gmail-MsoPlainText">> No. Since
we're using VO-DML to annotate basic STC, I don't think <span></span></p>
<p class="gmail-MsoPlainText">> there should be clients ignoring VODML at all. A normal client, for <span></span></p>
<p class="gmail-MsoPlainText">> instance, is one that just wants to figure out what
Epoch and Frame a <span></span></p>
<p class="gmail-MsoPlainText">> coordinate is for, but has built-in knowledge about
STC and hence does <span></span></p>
<p class="gmail-MsoPlainText">> not need to retrieve and parse the VO-DML data model
itself.<span></span></p>
<p class="gmail-MsoPlainText">> <span></span></p>
<p class="gmail-MsoPlainText">> Call it "non-validating" if you will.<span></span></p>
<p class="gmail-MsoPlainText">> <span></span></p>
<p class="gmail-MsoPlainText">> > > -- which of course begs the question
whether we can't scupper <span></span></p>
<p class="gmail-MsoPlainText">> > > LITERAL again and just use PARAMs instead
of them.<span></span></p>
<p class="gmail-MsoPlainText">> ><span></span></p>
<p class="gmail-MsoPlainText">> > That was what I originally used in th rewrite
of the mapping syntax, <span></span></p>
<p class="gmail-MsoPlainText">> > but we (focus team) decided, I think correctly,
that we'd like to <span></span></p>
<p class="gmail-MsoPlainText">> > remove any dependency on VOTable elements,
something Laurent and<span></span></p>
<p class="gmail-MsoPlainText">> Mark<span></span></p>
<p class="gmail-MsoPlainText">> > touched upon in their replies I think.<span></span></p>
<p class="gmail-MsoPlainText">> ><span></span></p>
<p class="gmail-MsoPlainText">> > Doing so makes it possible to separate the
schema for VO-DML mapping <span></span></p>
<p class="gmail-MsoPlainText">> > elements out from the main VOTable schema.<span></span></p>
<p class="gmail-MsoPlainText">> <span></span></p>
<p class="gmail-MsoPlainText">> Is that a benefit proportional to forcing VOTable
implementors to <span></span></p>
<p class="gmail-MsoPlainText">> supporting a second, incompatible type system with
its own rules for serialisation?<span></span></p>
<p class="gmail-MsoPlainText">> <span></span></p>
<p class="gmail-MsoPlainText">> As I said, VODML annotation will do extremely basic
things in the <span></span></p>
<p class="gmail-MsoPlainText">> future. I
don't see how any VOTable library could get away with not supporting it.<span></span></p>
<p class="gmail-MsoPlainText">> <span></span></p>
<p class="gmail-MsoPlainText">> > Also, it was decided that the dmtype would be
mandatory even if the <span></span></p>
<p class="gmail-MsoPlainText">> > type is not being cast to a more specific type
than the one declared <span></span></p>
<p class="gmail-MsoPlainText">> > for a<span></span></p>
<p class="gmail-MsoPlainText">> Role?<span></span></p>
<p class="gmail-MsoPlainText">> ><span></span></p>
<p class="gmail-MsoPlainText">> > Would you want to remove that requirement form
the mapping?<span></span></p>
<p class="gmail-MsoPlainText">> <span></span></p>
<p class="gmail-MsoPlainText">> Since the requirement doesn't seem to be something
that makes client's <span></span></p>
<p class="gmail-MsoPlainText">> lives easier, I'm all for removing it, yes.<span></span></p>
<p class="gmail-MsoPlainText">> <span></span></p>
<p class="gmail-MsoPlainText">Good point for a vote?<span></span></p>
<p class="gmail-MsoPlainText"><span> </span></p>
<p class="gmail-MsoPlainText"><span> </span></p>
<p class="gmail-MsoPlainText"><span> </span></p>
<p class="gmail-MsoPlainText">> > > > Finally, I do not think that renaming
ivoa types as VOTable <span></span></p>
<p class="gmail-MsoPlainText">> > > > datatypes will help that much.<span></span></p>
<p class="gmail-MsoPlainText">> ><span></span></p>
<p class="gmail-MsoPlainText">> > > Well, it's certainly saving clients the
trouble of having to come <span></span></p>
<p class="gmail-MsoPlainText">> > > up with parsers for all the literals of
the VO-DML types when they <span></span></p>
<p class="gmail-MsoPlainText">> > > already have parsers for the VOTable
types. I'd say that's, if <span></span></p>
<p class="gmail-MsoPlainText">> > > nothing else, a courtesy to implementors,
and given they're the <span></span></p>
<p class="gmail-MsoPlainText">> > > ones that eventually control takeup, that
ain't so bad.<span></span></p>
<p class="gmail-MsoPlainText">> ><span></span></p>
<p class="gmail-MsoPlainText">> > Main issue seems to be LITERAL-s and the fact
that, being <span></span></p>
<p class="gmail-MsoPlainText">> > standalone, one can currently not use the
VOTable datatypes.<span></span></p>
<p class="gmail-MsoPlainText">> ><span></span></p>
<p class="gmail-MsoPlainText">> > One solution as you say is to use PARAM, other
is to add the VOTable <span></span></p>
<p class="gmail-MsoPlainText">> > datatype as an attribute to LITERAL.<span></span></p>
<p class="gmail-MsoPlainText">> <span></span></p>
<p class="gmail-MsoPlainText">> If you consider dmtype as critical and PARAM really
unsuitable for <span></span></p>
<p class="gmail-MsoPlainText">> your purposes, that would be a solution that at
least would still keep <span></span></p>
<p class="gmail-MsoPlainText">> things manageable (you'd have to copy datatype,
arraysize, and xtype).<span></span></p>
<p class="gmail-MsoPlainText">> <span></span></p>
<p class="gmail-MsoPlainText">Ok. Note, arraysize may not be required, as a LITERAL is ALWAYS inside an ATTRIBUTE spec
and has multiplicity unbounded.<span></span></p>
<p class="gmail-MsoPlainText">(Actually a bit more generic than that even).<span></span></p>
<p class="gmail-MsoPlainText"><span> </span></p>
<p class="gmail-MsoPlainText"><span> </span></p>
<p class="gmail-MsoPlainText">> But it seems a lot of effort for a use case that
we're not even sure <span></span></p>
<p class="gmail-MsoPlainText">> is terribly common (validating in the case of
derived data models).<span></span></p>
<p class="gmail-MsoPlainText">> <span></span></p>
<p class="gmail-MsoPlainText">So choice seems to be between using PARAM as valid
instance inside of an ATTRIBUTE iso new LITERAL, or adding PARAM attributes
'datatype' and 'xtype' (and 'unit' and 'arraysize')?<span></span></p>
<p class="gmail-MsoPlainText"><span> </span></p>
<p class="gmail-MsoPlainText">Another vote?<span></span></p>
<p class="gmail-MsoPlainText"><span> </span></p>
<p class="gmail-MsoPlainText"><span> </span></p>
<p class="gmail-MsoPlainText">> > Anyway, note also that VOTable mappers will
have the option of using <span></span></p>
<p class="gmail-MsoPlainText">> > a CONSTANT (i.e. the VO-DML-mapping equivalent
of a PARAMref), i.e.<span></span></p>
<p class="gmail-MsoPlainText">> > create a VOTable-typed PARAM somewhere and
refer to it. I would <span></span></p>
<p class="gmail-MsoPlainText">> > definitely NOT propose this as a solution
though.<span></span></p>
<p class="gmail-MsoPlainText">> <span></span></p>
<p class="gmail-MsoPlainText">> Since you mention it: Why not? Sure, an extra reference is involved, <span></span></p>
<p class="gmail-MsoPlainText">> which is always bad, but as far as I can see there's
nothing you can <span></span></p>
<p class="gmail-MsoPlainText">> do with LITERAL that you can't do with CONSTANT, and
one feature less is always a big win.<span></span></p>
<p class="gmail-MsoPlainText">> <span></span></p>
<p class="gmail-MsoPlainText">You'd have to always add PARAMs outside the VODML spec to
be linked to from within.<span></span></p>
<p class="gmail-MsoPlainText">So in spec it may be one element less, but in instances
it'd be one extra element for every literal value.<span></span></p>
<p class="gmail-MsoPlainText"><span> </span></p>
<p class="gmail-MsoPlainText">> $ python -c "import this" | grep
preferably There should be one-- and <span></span></p>
<p class="gmail-MsoPlainText">> preferably only one --obvious way to do it.<span></span></p>
<p class="gmail-MsoPlainText">> <span></span></p>
<p class="gmail-MsoPlainText">FWIW, I don't like CONSTANT very much and would rather do
away with it.<span></span></p>
<p class="gmail-MsoPlainText"><span> </span></p>
<p class="gmail-MsoPlainText"><span> </span></p>
<p class="gmail-MsoPlainText">Gerard<span></span></p>
<p class="gmail-MsoPlainText"><span> </span></p>
<p class="gmail-MsoPlainText"><span> </span></p>
<p class="gmail-MsoPlainText"><span> </span></p>
<p class="gmail-MsoPlainText">> --
Markus<span></span></p>
<p class="gmail-MsoPlainText">> <span></span></p>
<p class="gmail-MsoPlainText">> (Who apologises for playing the troublemaker all
time; but somehow <span></span></p>
<p class="gmail-MsoPlainText">> almost the entire VO comes together in DaCHS, and so
when conflicts <span></span></p>
<p class="gmail-MsoPlainText">> between different grand or little designs start,
there's always craters in my front lawn.<span></span></p>
<p class="gmail-MsoPlainText">> Little innocent me, I'd be happy if only I had a way
to tell my <span></span></p>
<p class="gmail-MsoPlainText">> clients that a position is in ICRS for Epoch J2000,
and where the <span></span></p>
<p class="gmail-MsoPlainText">> proper motions are.
A way the clients actually bother to use, that <span></span></p>
<p class="gmail-MsoPlainText">> is.)<span></span></p></div></div>