ADQL-2.1 internal draft

[Markus Demleitner]

Hi Dave,

On Thu, Jun 11, 2015 at 04:58:47PM +0100, Dave Morris wrote:
> >... The grammar impact of features like the
> >set operators or common table expressions isn't isolated to one
> >"standard" rule needing a change and a set of extra rules otherwise
> >ignored; fairly typically several standard rules are impacted.  In the
> >end, even allowing quite a bit of after-parse logic, the current spec
> >induces at least eight nontrivially different grammars.
> 
> I agree that dynamically adding and removing bits from the grammar is not an
> option, it would cause way too much complexity and confusion.

Phewy.

> ADQL-2.1 should have a single standard grammar, which includes *all* of the
> features, including the optional ones.

I think that is workable (though I'd still like to make as much of
our feature set madatory as possible in order to keep user experience
as uniform as possible across the various services).

> Two things that I would like to propose :
> 
> The first is that we remove the requirement that ADQL has to be
> implementable on all our target platforms without requiring a grammar parser
> or rewriting parts of the query.

Has that been a requirement?  If so, it hasn't worked out; something
as trivial as Postgres not supporting TOP prevents that.

Also, such a service would produce very poor VOTables indeed.  I'd
still be curious if anyone has tried to run a TAP services without an
ADQL parser.

> However, as far as I know, it not been explicitly specified as a formal
> requirement in any of the documents.

I'd certainly hope so.  Given the state of SQL engines, the
intersection of their features is so limited that I doubt it'd make a
useful language.

> Therefore I propose we formally reject the requirement and clearly state it
> in the specification.

Absolutely.

> The second is that we formally define the acceptance criteria for mandatory
> features.
> 
> Accepting something as a mandatory part of the specification has a big
> impact on the range of platforms that can support ADQL.
> 
> For example, both the set operators and the bit wise operations have
> implementation issues on one or more of the RDBMS platforms currently in use
> by TAP services.

As in -- "there's no way to teach them bitwise operations, whether via
operators or functions"?  Note that if you'd have to write a few
lines of C for a DB extension I'd still count ADQL as implementable
on that platform.  I have a hard time believing there's a reasonable
platform that wouldn't let you do that.

Common Table Expressions, however...  Depending on what we allow
there, that's an entirely different matter, so I'm definitely in
favour of keeping an eye on implementabilty in the back end.

> This should go in the standard, probably in an appendix, along with a list
> of the target platforms on which it must be possible to implement a feature
> before it can be accepted as mandatory.

I'm not fully sold on this, but I suspect you could talk me into it.
Problems I can see include:

* how much work could be offloaded to DB extensions to still count as
  "implementable"?  Would we require PoC implementations for them?
* do we state minimal (maximal?) versions?
* is there an "efficiently" somewhere in there (e.g., plain CTEs can be
  simulated through copying subtrees, but unless you have a good
  query planner, performance may go down the drain)?
* how do we take things off that list?  Put things on it?  Given that
  lifetime(standard]>>lifetime(RDBMS version), this can be a bit
  tricky if we went about things legalistically.

Still, I think the idea is good, and it would help rationalise
development.

In that vein, I assume all of you know http://sqlfiddle.com ?

Cheers,

       Markus