New SIA v2 proposal

[Markus Demleitner]

Dear DAL,

On Wed, Sep 16, 2015 at 10:21:01AM -0700, Walter Landry wrote:
> Markus Demleitner <msdemlei at ari.uni-heidelberg.de> wrote:
> > On Tue, Sep 15, 2015 at 12:14:00PM -0700, Patrick Dowler wrote:
> > Well, users shouldn't even see that unless they're non-naive -- this
> > is a *protocol* that's implemented by libraries and clients, *not* a
> > user interface unless users are devious, in which case they can be
> > expected to study a spec.
> 
> I strongly disagree with this sentiment.  The implementors are people
> too.  Moreover, we have plenty of ordinary users writing simple
> scripts to fetch results.  We should not be booby trapping the
> protocols.

...and the implementors will appreciate it if our protocols have
simple, straightforward, and easy-to-implement conventions.  Why, as
was my point, NULL would be preferable to +/-Inf, which always
requires extra logic, both on the server and the client.  And even
for script writers.

Except...

I actually went back to read the VOTable 1.3 spec, which I probably
should have done more carefully before, rather than relying on my
understanding of the discussions we had back then.  And, OMG, you're
right, NaN no longer is NULL for floats (at least outside of BINARY).
I'm a bit surprised it worked out like that, as I fairly distinctly
remember nobody really spoke out for making that distinction.  Well,
I should read the text more carefully next time.

This means that we can't have NULLs in arrays cells any more (ok,
it's been marginal before, too).  Which in turn means I've been
wrong.  With the xtype=INTERVAL thing there simply is no way to
signify a missing value with floating point numbers unless we wanted
to revive VALUES null for floats, which I definitely don't want to
do.

> > I happen to believe going for Inf is not a good idea, as it won't
> > work for integers (where with NULL-is-open we at least can use
> > the nullvalue mechanism, bad as it is).
> 
> I am confused.  I do not know of any special values for null integers.
> NaN is certainly not allowed.

Exactly (and neither are +/-Inf).  This is why in VOTable (as in FITS
binary) there's an extra facility to define a NULL value for
integer-valued columns; this would look like this:

<PARAM name="echelle_orders" datatype="integer" xtype="INTERVAL">
  <VALUES null="-1"><MIN value="10"/><MAX value="100"/></VALUES>
</PARAM>

-- now, -1 can't be used for NULL in general, obviously, and so this
isn't quite as nice as I'd like it to be (where clients could simply
dump in a constant "something" for NULL), but it works and as that's
already in VOTable, people will have to implement it anyway.

With this, a client would serialise a left-open interval to -1 50,
and a right-open interval to 50 -1 in that case.  Again, admittedly
not beautiful, but if atomic parameters are out, I fail to see a
better way, and at least it's in line with current practices.  Except
that if we, as it seems to me now, have to use +/-inf for floats,
it's special rules left and right again.

[Oh, and, no, this isn't contrived: I already have a data access
service that needs integer intervals if intervals it is]

> > It's also less straightforward, as you'll have to map an "empty" on
> > some input widget to +Inf or -Inf depending on whether it's a lower
> > or upper bound, whereas mapping "empty string" to NULL and then
> > serialize that into whatever NULL representation you happen in an
> > output format is very natural.
> 
> The solution is to not allow empty strings to have meaning as an
> upper/lower bound.  Then you do not have to map it.

Well, the value will have to come from somewhere, right?  If you
offer your user a text input box, you'll get an empty string.  For
most other UIs, I'd also expect a "no limit" to come out as a "not
given", which doesn't even need any mapping to become NULL.

Put given we can't have NULLs in arrays and hence in intervals since
we chose to represent them as arrays, all this is moot.  Sigh.

Well, have your +/-Inf then, with the proviso we'll need special
rules for integer intervals in server side data processing
("AccessData").  And a large, deep siiiiiiggggghhhhh from me.

Cheers,

        Markus

Who's now discovering his sympathies for Cato the elder: ceterum
censeo all this wouldn't have happened had we built intervals with
with atomic parameters.