datatypes (effects all 3 WDs to some extent)

[Markus Demleitner]

On Tue, Apr 01, 2014 at 09:27:48AM -0700, Patrick Dowler wrote:
> 
> I don't see how anything but some kind of syntax for the value can
> ever provide the ability to deliver footprints/polygons as values in
> a simple table, such as one gets when doing
> 
> select s_region from ivoa.ObsCore

I would claim that's a different use case than the question of how we
write service parameters.  For instance, table columns need less
metadata than service parameters (i.e., you don't generally need to
specify valid values there); the specific case of s_region where
clients need to inspect table values to decide on a suitable storage
type (CIRCLE? POLYGON?  BOX?) is particularly striking (and may point
to features we should probably not keep when revising TAP and
company).  

Having said that, the beauty of deferring the definition of complex
types to VO-DML is that it'll work for tabular outputs just as well
(indeed, that's what it's designed for).

Geometries, of course, aren't just any complex datatype.  Each of
ADQL, TAP, and actual DBMSes (incomaptibly) define atomic geometry
types, which are fairly lamely mapped to VOTable right now.  I agree
that that will need to be revisited at some point, but we got that
wrong twice before.  Let's think this though carefully and thoroughly
this time and *not* rush it.  Whatever we come up with should come as
a part of a reform of the TAP stack and include a well thought-out
VOTable serialization.  It definitely should not come as rushed
ad-hoc definitions of parameters.  

Doing those would be particularly annoying as I still cannot see a
compelling use case for having those in our new protocols that could
not otherwise be satisfied.


> So, keeping the concept of "geometry datatypes" does not require
> votable changes and I appreciate the arguments against such a change.
> They seem to be correct in principle. That leaves micro-format (xtype
> hack).

...or, IMHO much preferably,

double CIRCLE_RA, double CIRCLE_DEC, double CIRCLE_RADIUS

double[] POLYGON_RAS, double[] POLYGON_DECS

and whatever else you wish, with the obvious advantage that it is
clear what a given service actually supports, and you can give proper
metadata (no more stinking maxSearchRadius items hidden somewhere in
a registry record).  

Ok, it's a bunch of parameters, but on both sides much easer to
handle than some magic POS parameter that supposedly supports many
forms, but where a client cannot discover which these are and which
of them actually work.  And where ranges and other constraints have
to be pulled out of a registry record or, if you're lucky, a
capabilities endpoint on the service itself.


Case in point is Arnold's interjection:

On Tue, Apr 01, 2014 at 02:46:10PM -0400, Arnold Rots wrote:
> I don't have a whole lot of appetite to enter into this discussion,
> but I wonder whether anybody has taken into account footprints
> consist of composite regions.

-- by explicitely specifying the *actual*, *primitive* parameters we
exactly delimit what we do and what we don't support.  With the
parameters above (and the constraint, admittedly yet to be made
explicit, that these are parameters of multiplicity 0..1), it is
clear that no complex shapes are supported.  Which is good, as that
requires a different class of code (basically, query generation has
to deal with trees instead of sequences), and that shouldn't go in
through some string forms.

*If* we decide we must have complex shapes, I suspect we'd be looking
at MOCs given the way current IVOA standardisation goes.  Which
brings me to a proposed mantra:

  Let's not go where FITS is stuck: representing always more complex
  data structure in key-value pairs.

HTTP parameters are just that, key-value pairs.  With VOTable and
VO-DML we have tools to express complex data structures in a
suitable serialization.  With UPLOAD, we have have a way proven in
TAP to allow the transmission of (relatively) complex  compound data
(in the case of TAP, database tables).  Let's use these rather than
ending up where FITS has. And let's not dump arrays and tables in
HISTORY or COMMENT cards in ad-hoc serializations utterly
incompatible between IRAF, MIDAS, and the home-grown nightmare from
the friendly engineer next door.

Cheers,

        Markus