ADQL 2.1 and TAP LANG

[Markus Demleitner]

Dear DAL,

DaCHS has been supporting what I think is pretty close to full ADQL
2.1 support for a while now.  What I've postponed so far is
reflecting this in the TAP LANG parameter and the registry record.
And as usual, the problem user-visible minor versions that might
break services.

Background: TAP clients must specify a LANG parameter when submitting
a query; for ADQL 2.0, they can either use LANG=ADQL or
LANG=ADQL-2.0.  What this latter means is described in sect 2.7.1 of
TAP (current PR):

  The service should return an “unknown query language” error [...]
  if an unsupported language or an incompatible language version is
  specified.

You could argue I'm good since ADQL 2.0 is probably "compatible" 
with ADQL 2.1 in the sense of this statement.  So, I'd accept all
or ADQL, ADQL-2.0, and ADQL-2.1 as LANG even if my implementation is
2.1, and I will never employ a 2.0 parser.  

In order to not risk anything with clients trying to do the right
thing, I'd even declare support for 2.0 in my TAPRegExt record even
though there's not a shard of 2.0 code in the implementation.

Does everyone agree that is the right thing to do?  Are there use
cases for which it might be desirable if a 2.1 service rejects a
query in ADQL 2.0?  Or use cases for which it might be desirable that
LANG=ADQL-2.0 actually select a strict ADQL-2.0 grammar without 2.1
extensions?


If there were such use cases, I'd argue we must have some
semi-required procedure for language version negotiation in the TAP
specification ("Clients giving an explicit version SHOULD inquire the
supported versions from the TAP capability element").

Saying something like this not great -- negotiations are difficult to
get right in general, and in, ahem, scientific programming, they're a
recipe for chaos.  If we can do without them, that'd be great.  

But there's another thing why we might still want such version
negotiation: What should my current code do when a request with
LANG=ADQL-2.2 comes in?

That question is non-trivial -- quite a few DaCHS deployments
(regrettably) only get updated every few years, if at all.  So, it's
totally possible that code I'm writing today will still run five
years from now when, I hope, we'll have ADQL 2.2 with a more reliable
grammar.  Then, when a query with LANG=ADQL-2.2 comes in, the
likelihood that the service would do the right thing if it just went
ahead is quite high, and in general I'd like to avoid breaking things
hard that would probably just work without the check.

We could of course say that explicit versions SHOULD only be used
when the client has information that the query uses features of that
higher version (and/or if it's known that the service supports the
version).  But that puts a fairly high load at least on generic
clients again.


Here's my take: As usual, we should avoid making minor versions
user-visible.  So, TAP should recommend writing LANG=ADQL without a
version (and then perhaps say that if people really want to give a
version, they MUST check in the capability first).

And when there's a new major version of ADQL, it would no longer use
the language name ADQL but (say) ADQL3 instead.

Opinions?  Advice?

         -- Markus