less required ADQL support [TAP-1.1]

[Markus Demleitner]

Dear DAL,

On Mon, Sep 28, 2015 at 11:13:28AM -0700, Patrick Dowler wrote:
> The goal/implication is that other tables could be provided which can be
> queried by ADQL and/or some other languages... but the idea to open up TAP
> to non-SQL type systems is to make it so they don't have to support ADQL for
> all tables.
> 
> So, without thinking too hard about *how* to do it, do we generally agree
> that this is a suitable goal for TAP?

Frankly, I'm not really sure, because of course once you drop the
requirement to support ADQL, there is a substantial risk that
everyone will just support their own SQL dialect, and users will have
to learn a new query language on each server they want to use.

That would suck.

On the other hand, perhaps ADQL now has enough traction that that
wouldn't happen, and there certainly are use cases where supporting
the full relational model may result in unreasonable requirements on
the back end. Then, it's better to have something with a proprietary
query language than nothing.

So -- neither yes nor no from me on this.  If we don't plan to have
multiple languages in the near future, we should at least clearly say
so and un-require LANG.  If we do plan to, we need to think hard and
have implemenations.

> So, it's easy enough to make ADQL less required, but we also need a
> mechanism to tell users/clients which LANG can be used:
> 
> 1- this would be easy enough in TAP_SCHEMA

Would it?  I'd expect a fairly typical case is that people support
some proprietary dialect *in addition to* ADQL.  So, langs:tables is
n:n, and hence we'd need a new table.  Perhaps:

TAP_SCHEMA.langs_available:

table_name  language_available.


> 2- need to decide what to do with VOSI-tables (VODataService model)

<rummaging for Registry chair hat>
<donning it>
We-ell.  From a pure Registry position, I'm not really wild on
including this into VODataService; that's really something very much
TAP-specific, and I don't know that there's a plausible scenario in
which "what language can I use to query this particular data" really
becomes a discovery use case.  <tossing Registry chair hat away>

Having said that, I have some sympathy for trying to keep VOSI tables
and TAP_SCHEMA (if we have to have them both) roughly equivalent in
representative power.

I'd say: let's see how this works out in TAP_SCHEMA, and then we'll
see what to do about VOSI.

> 3- at one extreme, you need to specify it for each table and TAPRegExt. See
> the "TAPRegExt fine grained differences between tables" email thread for a
> discussion of that...

Ouch.  Well, one thing I'm certain about: It should remain easy to
"do ADQL", as that's the social thing to do.  Anything that makes
that still harder needs a lot of justification.  On the other hand,
using non-ADQL can be a bit cumbersome IMHO, as you should really
have  a strong reason to do so (as opposed to "ah, I just don't feel
like looking at the existing ADQL engines").

> 4- at the other extreme, you could group all your ADQL tables under one
> capability and all your LANG=NoSQL (eg) tables under another capability
> (recall that TAP-1.1 defines standardID values per capability and you will
> be able to have multiple async and/or multiple sync capabilities - each with
> their own TAPRegExt metadata)... but this isn't without implementation
> complications and it seems like a pretty artificial way to do it to me

...in particular we should deprecate LANG then.  But I claim that
wouldn't help you, because you'd then have to have multiple tables
endpoints, too, and a mechanism to couple those and the respective
capabilities.

This might help if you defined different *resources*, but given that
I'd expect a common case is that people allow multiple languages on
their tables, that would require duplication of the potentially
massive table metadata, and so <not picking up the Registry hat, but
seriously considering it> I'm against that.

> - if these aren't enough, there are certain to be other complications

Yes.  And therefore I suspect we shouldn't push it into TAP 1.1.  TAP
1.0 has almost all that's required to support proprietary  languages.
If someone starts prototyping TAP_SCHEMA.langs_available and its
implications on TAPRegExt now, great.  If things work out
spectacularly nicely and TAP 1.1 is slower that we hope, it may
go in, but otherwise I think trying things out first is highly
advisable.

But let's not design such a thing in thin air.

Cheers,

        Markus