ivoa: DM type system

[Gerard Lemson]

HI
Sorry for missing early part of the discussion, was travelling.
It seems that at least it is agreed that we should distinguish the VO-DML spec from the serialization/mapping spec.
And that the issue Markus brings up is one for the latter only, apart from possible discussions on the contents of the ivoa model, which belongs to the RFC page?


> -----Original Message-----
> From: dm-bounces at ivoa.net [mailto:dm-bounces at ivoa.net] On Behalf Of
> Markus Demleitner
> Sent: Wednesday, April 12, 2017 5:04 AM
> To: dm at ivoa.net
> Subject: Re: ivoa: DM type system
> 
> Hi,
> 
> On Tue, Apr 11, 2017 at 06:41:46PM +0200, Laurent Michel wrote:
> > I agree with Markus, having LITERAL @dmtypes compliant with VOTable
> > datatypes would be valuable.  Unfortunately, LITERAL @dmtypes are not
> > necessarily in the ivoa name space (ivoa:xx).
> > They can refer to dataTypes defined into the model. This is the case
> > for ENUM literals e.g..
> 
> Are there any other cases except ENUMs?  
datetime!  Any custom primitive types some model defines. For example Mark CD has been wanting to add a URL type as separate from anyURI).
Fields outside of astronomy introduce "zip-code" type as a string obeying certain restrictions.
Similarly decimal can be seen as a rational with denominator a power of 10. 
No doubt there are many others.

Introduction of such custom primitives is easily accommodated in our VO-DML mechanism, where the basic primitives are explicitly defined as types and hence can be extended. NOT as a listing of valid values as in VOTable. My guess would be that it is because the VOTable datatypes are defined in the schema and we're so afraid of backwards compatibility that we do still not have a datetime datatype there?
Also, VO-DML aims to be representation neutral, hence restricts for example its numeric types to the mathematical number systems (N,Z,Q,R,C), rather than using the software representations, short/int/long(/longer/longest?). This is one reason why rational is included. Also it is datetime, not date, or time, or MJD, UTC, or whatever. Again, those are deemed to be serialization issues and it completely proper for VOTable to define those, but no need to do so in VO-DML.


> If not, I'd say there's no big
> incentive to introduce any complication here.  
> When we just use VOTable types:
I hope that with "just" you mean, allow one to use the VOTable datatypes in LITERAL? Not doing something to 'ivoa' data model?

> 
> (a) normal clients just mind that they can parse the thing.  For that, they just
> need the VOTable datatype (and they already have parsers for that)
> 
I expect with 'normal clients' you mean those that will ignore the whole VODML element? If so there is no issue as only there LITERALs will occur
> (b) validators can infer the the set of allowed values from the VO-DML
> document, because the literal's role is known from where it stands in the
> instance document, and the VO-DML states what type that role has (i.e., in
> this case the allowed literals).
> 
> -- which of course begs the question whether we can't scupper LITERAL again
> and just use PARAMs instead of them.
> 
That was what I originally used in th rewrite of the mapping syntax,  but we (focus team) decided, I think correctly, that we'd like to remove any dependency on VOTable elements, something Laurent and Mark touched upon in their replies I think.
Doing so makes it possible to separate the schema for VO-DML mapping elements out from the main VOTable schema. 

Also, it was decided that the dmtype would be mandatory even if the type is not being cast to a more specific type than the one declared for a Role? 
Would you want to remove that requirement form the mapping?

> > Finally, I do not think that renaming ivoa types as VOTable datatypes
> > will help that much.
> 
> Well, it's certainly saving clients the trouble of having to come up with parsers
> for all the literals of the VO-DML types when they already have parsers for
> the VOTable types.  I'd say that's, if nothing else, a courtesy to implementors,
> and given they're the ones that eventually control takeup, that ain't so bad.
> 
Main issue seems to be LITERAL-s and the fact that, being standalone, one can currently not use the VOTable datatypes.
One solution as you say is to use PARAM, other is to add the VOTable datatype as an attribute to LITERAL.
Both break the nice feature of 'vo-dml part is independent of VOTable elements and hence can be separated easily and hence VOTable schema readers who don't want to have to skip past lots of VO_MDL elements don't have to'.  This separation would also enable us to reuse the VO-DML-only part of the mapping spec to be used outside of VOTable contexts. 

I presonally would prefer to *not* have to pollute the VO-DML part this way. To me any parser interested in the VO-DML mapping worth its salt should be able to deal with vodmlref elements, hence should be able to parse 'ivoa:string' and should be able to translate this to a type in the language it is coded in. And anyone not interested in the VO-DML annotation would ignore LITERALs anyway. Yes it might be another impedance mismatch, but one could argue that resolving those is what VO is all about.
In fact, I always thought the main role of data models in the VO would be to simplify the resolution of those impedance mismatches, both as semantic and syntactic level. ISO having to be able to translate every representation into every other one, O(N^2) complexity, one has a central, core representation into and from which one needs to be able to translate these representations, providing "only"  O(N) complexity. (See section 5 in the OLD http://wiki.ivoa.net/internal/IVOA/IvoaTheory/Theory_in_the_VO_whitepaper.pdf or section 2  in http://wiki.ivoa.net/internal/IVOA/IvoaDataModel/DomainModelv0.9.1.doc proposing data models as a kind of eparanto).

Anyway, note also that VOTable mappers will have the option of using a CONSTANT (i.e. the VO-DML-mapping equivalent of a PARAMref), i.e. create a VOTable-typed PARAM somewhere and refer to it. I would definitely NOT propose this as a solution though.


Cheers

Gerard

>         -- Markus