STC-S in DataLink

[Markus Demleitner]

Dear DAL folks,

I take the liberty to reply to Pat and Tim in one mail:

On Thu, Jun 20, 2013 at 04:08:07PM +0000, Tim Jenness wrote:
> 
> On Jun 20, 2013, at 07:46 , Markus Demleitner <msdemlei at ari.uni-heidelberg.de>
>  wrote:
> > 
> > And here's now my offer: I'd write up EBNF and accompanying prose for
> > something that's "pretty much" like STC-S according to the current note,
> > leaving existing usages of STC-S intact and simplifying the remaining
> > rules to, e.g., allow both (1) and (2).  I'd have it ready for Hawaii,
> > complete with an implementation that at least can move the stuff to
> > STC-X and VOTable utypes.
> > 
> > Both encouragement and, erm, well, let's say discouragement is welcome
> > (since this definitely would not be a standard I'd enjoy writing, I'd
> > actually appreciate the latter a bit more?)
> 
> This was an interesting discussion. So are you saying you would be
> trying to update the STC-S definition to make it easier to parse,
> but backwards compatible, or that you would be creating a whole new
> variant? Would STC-S be deprecated? Do you have a list of all the
> software that deals with STC-S and what features they have
> implemented?

The intention would be to not break existing software; total
backwards compatibility is probably not even desirable -- I'm
thinking of the strict sequence conditions on the one hand  and too
much reliance on closed sets of keywords on the front side of the
clauses on the other.  

It's clear, though, that we cannot really go back behind what's in
TAP (which, to be honest, I'm not totally happy about, but there you
go).  I would guess that covers a large part of what's actually
implemented out there.  I'd gladly hear of applications not covered
by this.

On to Pat:

On Thu, Jun 20, 2013 at 11:24:03AM -0700, Patrick Dowler wrote:
> the scope of an STC-S standard is still to be decided and since this
> is being done now to support some current use cases that scope could
> be quite highly constrained. For example, we could decide to
> constrain it enough that we do not feel bad about making most/all of
> it mandatory.

Even just supporting what TAP allows for things like cutouts is a
fairly steep agenda, I must say, in particular since for most use
cases simple coordinate intervals will go a long way.

> As for Markus' soliloquy, I do agree that we have in the past lied
> when we represent things as being simpler than they really are. But

Ah, that was not my concern.  My concern is much more concrete: Don't
say you expect a float when you're really expecting something else
than a float literal.  This problem needs to be fixed in some way
anyway, and I maintain the "structured" parameters are the simplest
and cleanest way to go.

> accepting and using data types beyond primitive integers and floats
> and strings. At CADC we have been treating shapes (circles, polygons,
> etc) and intervals a real datatypes** for a long time now and once
> you do that all the confusion goes away -- and astronomers are fine
> with this (we consult our users while developing these features)!! I
> do not agree at all with the idea of trying to do everything with
> primitive datatypes and hordes of parameters.

I'm not against defining types for whatever inputs you need.  I'm
highly skeptical, though, that a type calculus involving intervals is
something you want in IVOA protocols or that would simplify them, in
particular when an almost trivial convention can do basically the
same thing (and is what service deployers out there have been doing
when left to their own devices).  Without having this kind of
calculus includinga strict definitions of literals of the values,
we'll have the situation of SSAP over again, with inconsistent and
spotty support of a syntax full of can, might, and may.

Allowing geometry primitives is a different matter -- though
spherical geometry is hard to get right, we certainly cannot ignore
use cases in which we just have to support them.  However, geometry
primitives in protocols don't mean STC-S and don't mean binding
together actual coordinates and all kinds of metadata.  We didn't get
this totally right in ADQL, but having geometries in there definitely
is a good thing.

For DataLink, on the other hand, I still seriously doubt whether
non-interval cutouts -- which complicate the services severely, for
the full STC-S geometry expressions by probably an order of magnitude
-- are on the right side of the 80/20 line.

> I just got Arnold's latest draft and will be tidying it up as a WD
> and posting it asap. Once that is done, we can discuss what the scope
> of 1.0 should be, but it really has to support the current science
> use cases (cubes and time series) and the ones from ObsTAP.

Are there really any of those that could not be satisfied by simple
interval (ie., _MIN/_MAX) cutouts?  Frankly, even for the cutouts
of extracted sources (where I can see some use in being able to
select spheres or similar), we're probably not talking about factors
of 10 in reduction of data transfer between the user's actual ROI vs.
the bounding box -- and hence I doubt it makes sense burdening all
the servers with such requirements when those cases that actually
want this can about as well do their magic on the client side.


Let me not in closing that by just explicitely defining the axes you
have in a concrete cube, you're also much more generic -- generalized
coordinates of some sort?  A breeze.  Polarization axis?  Don't need
to change STC-S at all.  Just give a ucd and maybe an STC utype
(where applicable) in your metadata declaration, and clients have
about the same modelling power as with STC-S inside of STC and much
more outside of it.


Cheers,

        Markus