ADQL versioning

[Markus Demleitner]

Dear DAL,

While reviewing ADQL 2.1, I stumbled across the question: "What is a
minor update to a language?"

Background: ADQL 2.1 defines several new features (LOWER, ILIKE, Set
operations, common table expressions, bitwise operators...), but all
of them are optional.  This is mainly because it was felt we cannot
introduce mandatory new features in a minor version.

My experience with optional features in a standards is that they
suck.  Users can never quite rely on them, in particular when doing
cross-service work, they're a pain in documentation, and they are a
liability for validators that, in essence, have to be able to
validate against 2^n different standards, when n optional features
can independently be present or missing.

So, I wondered: Is there really no way we can introduce mandatory
features in 2.1?  Based on usage scenarios, I'd say we can.  The
reason we're restricting the changes allowed in minor versions is so
legacy clients continue to work.  As long as we don't un-require (or
even outlaw) anything that was in ADQL 2.0, I claim we're fine with
new mandatory features.

Consider:

* Any client, any user only speaking ADQL-2.0 can then still operate
  any service implementing ADQL-2.1 (as 2.0 is a subset of 2.1)
* Clients aware of ADQL-2.1 can figure out support for ADQL-2.1 
  on the service side (by inspecting the capabilities), if desired, 
  restrict itself to ADQL-2.0.

The migration matrix then would be

   Client:       2.0          2.1
   Server
    2.0       of course    by metadata inspection
    2.1       by subset    of course


I'd say all is fine.  And hence I'd like to make mandatory as much as
we can agree on.  Here's my personal wishlist based on sect 4:

* Geometry functions: Are already optional in 2.0, and we've somewhat
  coped.  I'd not object if we made POINT, CIRCLE, POLYGON, DISTANCE,
  and CONTAINS mandatory, but I'd be fine with not touching the mess
  that this already is.
* but (not that this really belongs here) deprecate BOX and in
  particular not change it to also accept a POINT first argument.
* and make it clear that if you support a certain geometry, support
  for all the different polymorphous arguments is mandatory.
* LOWER: mandatory
* ILIKE: either mandatory or drop; there's already RegTAP's
  ivo_nocase_match which does the same thing, and the less operators
  the better.
* Set operators: Ouch.  I'll get to these in a later mail
* Common Table Expressions: Probably drop (services can still put
  them in; or perhaps they are a valid case for an optional feature)
* CAST: provided there's grammar for it, make it mandatory (but note:
  the grammar may be non-trivial if we want more than "anything but a
  )" the second argument -- SQL types can have funny type names like
  "DOUBLE PRECISION")
* IN_UNIT: As cool as I consider it, I can't honestly stand here and
  demand it's made mandatory at the current state of takeup (it *is*
  a considerable implementation effort even if you have a VOUnits
  library).  So: optional.  Or drop.
* Bitwise Operators: Mandatory.
* OFFSET: Mandatory.

We'd do our future users a big favour if we kept ADQL a single-ish
language with a dependable feature set.  No?

         -- Markus