ivoa: DM type system

[Gerard Lemson]

Hi Markus



Tl,dr: In the mapping spec for LITERAL we must pay careful attention to the
serialization of LITERAL values, and may want to "just" use PARAM.

Details to be worked out, but we may not be able to keep the separation
between a VOTable and separate vo-dml-maping schema.

More comments below.



> -----Original Message-----

> From: dm-bounces at ivoa.net [mailto:dm-bounces at ivoa.net
<dm-bounces at ivoa.net>] On Behalf Of

> Markus Demleitner

> Sent: Thursday, April 13, 2017 5:18 AM

> To: dm at ivoa.net

> Subject: Re: ivoa: DM type system

>

> Hi Gerard,

>

> On Wed, Apr 12, 2017 at 10:45:38AM -0400, Gerard Lemson wrote:

> > It seems that at least it is agreed that we should distinguish the

> > VO-DML spec from the serialization/mapping spec.

> >

> > And that the issue Markus brings up is one for the latter only,

> > apart from possible discussions on the contents of the ivoa model,

> > which belongs to the RFC page?

>

> Well, I've put it there, and as regards pulling the ivoa data model

> out of VO- DML proper, I think there still is a point.  Keeping what

> it specifies fluid until we have actual experience how these things

> work out while fundamental things like stc and whatever we use to

> model value+error are in flux would seem wise to me.

>

Will have a look at RFC page again and reply there as well.



> But you're right, my primary concern here is LITERAL within VOTable,

> and thus a mapping issue.

>

And especially a serialization issue I think. For indeed VO-DML says
NOTHING about how to serialize instances.

So you're right that when using a LITERAL value, only specifying its VO-DML
primitive type may not be sufficient in general.

It would be nice to have a common way to specify this, which so far in the
mapping doc we did not do.

Whether VOTable elements would suffice I don't know, maybe it is up to
protocols to define this as you seem to be doing below in DALI?

>

> > > Markus Demleitner <Wednesday, April 12, 2017 5:04 AM> Are there

> > > any other cases except ENUMs?

> >

> > datetime!  Any custom primitive types some model defines. For

> > example Mark CD has been wanting to add a URL type as separate from
anyURI).

>

> I feared it would come to this.  After long contortions, we eventually

> reached agreement on using

>

>   datatype="char", arraysize="*", xtype="timestamp"

>

> in DALI, sect. 3.3.3, for these.

>

Ok, sorry for not keeping up with that. Is it DALI 3.1.2?

Does this mean you're  favoring passing such choices off to the protocols
then?

For interpreters that do not know that protocol this would not help.

But if this is allowed, could this not extend to the VO-DML literal
serializations as well?





> > Fields outside of astronomy introduce "zip-code" type as a string

> > obeying certain restrictions.

> >

> > Similarly decimal can be seen as a rational with denominator a power of
10.

> >

> > No doubt there are many others.

>

> Sure.  Geometries in VOTable.

>

> I appreciate it would be great if we could introduce some clean, well-

> thought-out system for doing what's now done (admittedly

> suboptimally) with xtypes in VOTable.

>

> But if the price for this is that people, within one VOTable, have to

> recognise timestamps in LITERALs by seeing it's

> vodml-type="ivoa:datetime" and using one literal parser, while having

> to check xtype and use a different literal parser when it's in PARAM
makes me cringe.

>

> Plus, it won't stop with datetimes.

>

Sure. So are you arguing we should rely on xtype as a kind of free-for-all
label plus a protocol-based serialization prescription? For datatype, even
with arraysize will in general not be sufficient.

>

> > Introduction of such custom primitives is easily accommodated in our

> > VO-DML mechanism, where the basic primitives are explicitly defined

> > as types and

>

> I have no doubts that it's easy in VO-DML.  If it stays there, I don't

> have a major problem with this.  If it bleeds out into other

> standards, that's a different matter; if we're not careful there, the

> complete system will be an inconsistent mess of duplicate

> specifications.  And people will hate us or, worse, ignore us.

>

> Yes, it sucks to always have to compromise.  But the alternative is worse.

>





> > Also, VO-DML aims to be representation neutral, hence restricts for

> > example its numeric types to the mathematical number systems

> > (N,Z,Q,R,C), rather than using the software representations,

> short/int/long(/longer/longest?).

> > This is one reason why rational is included. Also it is datetime,

> > not date, or time, or MJD, UTC, or whatever. Again, those are deemed

> > to be serialization issues and it completely proper for VOTable to

> > define those, but no need to do so in VO-DML.

>

> Again, it's great if VO-DML has conceptual clarity here, but if

> VOTable implementors have to support another type system with lots of

> rules of its own that's a major implemenation liability.  And given

> there's CONSTANT and LITERAL, somehow the mapping document will still

> have to talk about VOTable types and literals.

>

It will, it is "only" in the formal definition that it might be nice to be
able to separate out the schemas.



> > > (a) normal clients just mind that they can parse the thing.  For

> > > that, they just need the VOTable datatype (and they already have

> > > parsers for

> > > that)

> >

> > I expect with 'normal clients' you mean those that will ignore the

> > whole VODML element? If so there is no issue as only there LITERALs

> > will occur

>

> No.  Since we're using VO-DML to annotate basic STC, I don't think

> there should be clients ignoring VODML at all.  A normal client, for

> instance, is one that just wants to figure out what Epoch and Frame a

> coordinate is for, but has built-in knowledge about STC and hence does

> not need to retrieve and parse the VO-DML data model itself.

>

> Call it "non-validating" if you will.

>

> > > -- which of course begs the question whether we can't scupper

> > > LITERAL again and just use PARAMs instead of them.

> >

> > That was what I originally used in th rewrite of the mapping syntax,

> > but we (focus team) decided, I think correctly, that we'd like to

> > remove any dependency on VOTable elements, something Laurent and

> Mark

> > touched upon in their replies I think.

> >

> > Doing so makes it possible to separate the schema for VO-DML mapping

> > elements out from the main VOTable schema.

>

> Is that a benefit proportional to forcing VOTable implementors to

> supporting a second, incompatible type system with its own rules for
serialisation?

>

> As I said, VODML annotation will do extremely basic things in the

> future.  I don't see how any VOTable library could get away with not
supporting it.

>

> > Also, it was decided that the dmtype would be mandatory even if the

> > type is not being cast to a more specific type than the one declared

> > for a

> Role?

> >

> > Would you want to remove that requirement form the mapping?

>

> Since the requirement doesn't seem to be something that makes client's

> lives easier, I'm all for removing it, yes.

>

Good point for a vote?







> > > > Finally, I do not think that renaming ivoa types as VOTable

> > > > datatypes will help that much.

> >

> > > Well, it's certainly saving clients the trouble of having to come

> > > up with parsers for all the literals of the VO-DML types when they

> > > already have parsers for the VOTable types.  I'd say that's, if

> > > nothing else, a courtesy to implementors, and given they're the

> > > ones that eventually control takeup, that ain't so bad.

> >

> > Main issue seems to be LITERAL-s and the fact that, being

> > standalone, one can currently not use the VOTable datatypes.

> >

> > One solution as you say is to use PARAM, other is to add the VOTable

> > datatype as an attribute to LITERAL.

>

> If you consider dmtype as critical and PARAM really unsuitable for

> your purposes, that would be a solution that at least would still keep

> things manageable (you'd have to copy datatype, arraysize, and xtype).

>

Ok. Note, arraysize may not be required, as  a LITERAL is ALWAYS inside an
ATTRIBUTE spec and has multiplicity unbounded.

(Actually a bit more generic than that even).





> But it seems a lot of effort for a use case that we're not even sure

> is terribly common (validating in the case of derived data models).

>

So choice seems to be between using PARAM as valid instance inside of an
ATTRIBUTE iso new LITERAL, or adding PARAM attributes 'datatype' and
'xtype' (and 'unit' and 'arraysize')?



Another vote?





> > Anyway, note also that VOTable mappers will have the option of using

> > a CONSTANT (i.e. the VO-DML-mapping equivalent of a PARAMref), i.e.

> > create a VOTable-typed PARAM somewhere and refer to it. I would

> > definitely NOT propose this as a solution though.

>

> Since you mention it: Why not?  Sure, an extra reference is involved,

> which is always bad, but as far as I can see there's nothing you can

> do with LITERAL that you can't do with CONSTANT, and one feature less is
always a big win.

>

You'd have to always add PARAMs outside the VODML spec to be linked to from
within.

So in spec it may be one element less, but in instances it'd be one extra
element for every literal value.



> $ python -c "import this" | grep preferably There should be one-- and

> preferably only one --obvious way to do it.

>

FWIW, I don't like CONSTANT very much and would rather do away with it.





Gerard







>       -- Markus

>

> (Who apologises for playing the troublemaker all time; but somehow

> almost the entire VO comes together in DaCHS, and so when conflicts

> between different grand or little designs start, there's always craters
in my front lawn.

> Little innocent me, I'd be happy if only I had a way to tell my

> clients that a position is in ICRS for Epoch J2000, and where the

> proper motions are.  A way the clients actually bother to use, that

> is.)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.ivoa.net/pipermail/dm/attachments/20170413/5863afba/attachment-0001.html>