ivoa: DM type system

[Markus Demleitner]

Hi Gerard,

On Wed, Apr 12, 2017 at 10:45:38AM -0400, Gerard Lemson wrote:
> It seems that at least it is agreed that we should distinguish the VO-DML
> spec from the serialization/mapping spec.
> 
> And that the issue Markus brings up is one for the latter only, apart from
> possible discussions on the contents of the ivoa model, which belongs to
> the RFC page?

Well, I've put it there, and as regards pulling the ivoa data model
out of VO-DML proper, I think there still is a point.  Keeping what
it specifies fluid until we have actual experience how these things
work out while fundamental things like stc and whatever we use to
model value+error are in flux would seem wise to me.

But you're right, my primary concern here is LITERAL within VOTable,
and thus a mapping issue.


> > Markus Demleitner <Wednesday, April 12, 2017 5:04 AM>
> > Are there any other cases except ENUMs?
> 
> datetime!  Any custom primitive types some model defines. For example Mark
> CD has been wanting to add a URL type as separate from anyURI).

I feared it would come to this.  After long contortions, we
eventually reached agreement on using 

  datatype="char", arraysize="*", xtype="timestamp"

in DALI, sect. 3.3.3, for these.

> Fields outside of astronomy introduce "zip-code" type as a string obeying
> certain restrictions.
> 
> Similarly decimal can be seen as a rational with denominator a power of 10.
> 
> No doubt there are many others.

Sure.  Geometries in VOTable.

I appreciate it would be great if we could introduce some clean,
well-thought-out system for doing what's now done (admittedly
suboptimally) with xtypes in VOTable.

But if the price for this is that people, within one VOTable, have to
recognise timestamps in LITERALs by seeing it's
vodml-type="ivoa:datetime" and using one literal parser, while having
to check xtype and use a different literal parser when it's in PARAM
makes me cringe.

Plus, it won't stop with datetimes.


> Introduction of such custom primitives is easily accommodated in our VO-DML
> mechanism, where the basic primitives are explicitly defined as types and

I have no doubts that it's easy in VO-DML.  If it stays there, I
don't have a major problem with this.  If it bleeds out into other
standards, that's a different matter; if we're not careful there,
the complete system will be an inconsistent mess of duplicate
specifications.  And people will hate us or, worse, ignore us.

Yes, it sucks to always have to compromise.  But the alternative is
worse.

> Also, VO-DML aims to be representation neutral, hence restricts for example
> its numeric types to the mathematical number systems (N,Z,Q,R,C), rather
> than using the software representations, short/int/long(/longer/longest?).
> This is one reason why rational is included. Also it is datetime, not date,
> or time, or MJD, UTC, or whatever. Again, those are deemed to be
> serialization issues and it completely proper for VOTable to define those,
> but no need to do so in VO-DML.

Again, it's great if VO-DML has conceptual clarity here, but if
VOTable implementors have to support another type system with lots of
rules of its own that's a major implemenation liability.  And given
there's CONSTANT and LITERAL, somehow the mapping document will still
have to talk about VOTable types and literals.

> > (a) normal clients just mind that they can parse the thing.  For that,
> > they just need the VOTable datatype (and they already have parsers for
> > that)
> 
> I expect with 'normal clients' you mean those that will ignore the whole
> VODML element? If so there is no issue as only there LITERALs will occur

No.  Since we're using VO-DML to annotate basic STC, I don't think
there should be clients ignoring VODML at all.  A normal client, for
instance, is one that just wants to figure out what Epoch and Frame a
coordinate is for, but has built-in knowledge about STC and hence
does not need to retrieve and parse the VO-DML data model itself.

Call it "non-validating" if you will.

> > -- which of course begs the question whether we can't scupper LITERAL
> > again and just use PARAMs instead of them.
> 
> That was what I originally used in th rewrite of the mapping syntax,  but
> we (focus team) decided, I think correctly, that we'd like to remove any
> dependency on VOTable elements, something Laurent and Mark touched upon in
> their replies I think.
> 
> Doing so makes it possible to separate the schema for VO-DML mapping
> elements out from the main VOTable schema.

Is that a benefit proportional to forcing VOTable implementors to
supporting a second, incompatible type system with its own rules for
serialisation?

As I said, VODML annotation will do extremely basic things in the
future.  I don't see how any VOTable library could get away with not
supporting it.

> Also, it was decided that the dmtype would be mandatory even if the type is
> not being cast to a more specific type than the one declared for a Role?
> 
> Would you want to remove that requirement form the mapping?

Since the requirement doesn't seem to be something that makes
client's lives easier, I'm all for removing it, yes.

> > > Finally, I do not think that renaming ivoa types as VOTable
> > > datatypes will help that much.
> 
> > Well, it's certainly saving clients the trouble of having to come up
> > with parsers for all the literals of the VO-DML types when they
> > already have parsers for the VOTable types.  I'd say that's, if
> > nothing else, a courtesy to implementors, and given they're the ones that
> > eventually control takeup, that ain't so bad.
> 
> Main issue seems to be LITERAL-s and the fact that, being standalone, one
> can currently not use the VOTable datatypes.
> 
> One solution as you say is to use PARAM, other is to add the VOTable
> datatype as an attribute to LITERAL.

If you consider dmtype as critical and PARAM really unsuitable for
your purposes, that would be a solution that at least would still
keep things manageable (you'd have to copy datatype, arraysize, and
xtype).

But it seems a lot of effort for a use case that we're not even sure
is terribly common (validating in the case of derived data models).

> Anyway, note also that VOTable mappers will have the option of using a
> CONSTANT (i.e. the VO-DML-mapping equivalent of a PARAMref), i.e. create a
> VOTable-typed PARAM somewhere and refer to it. I would definitely NOT
> propose this as a solution though.

Since you mention it: Why not?  Sure, an extra reference is involved,
which is always bad, but as far as I can see there's nothing you can
do with LITERAL that you can't do with CONSTANT, and one feature less
is always a big win.

$ python -c "import this" | grep preferably
There should be one-- and preferably only one --obvious way to do it.

      -- Markus

(Who apologises for playing the troublemaker all time; but somehow
almost the entire VO comes together in DaCHS, and so when conflicts
between different grand or little designs start, there's always
craters in my front lawn.  Little innocent me, I'd be happy if only I
had a way to tell my clients that a position is in ICRS for Epoch
J2000, and where the proper motions are.  A way the clients actually
bother to use, that is.)