TAP1.0 Comments

[Markus Demleitner]

On Fri, Jul 17, 2009 at 02:04:20PM +0200, Francois Ochsenbein wrote:
> 
> >
> >On Thu, Jul 16, 2009 at 10:22:16AM +0200, Francois Bonnarel wrote:
> >> I was also wondering about this single table output issue because I
> >> had use  cases in mind where it could be a great constraint
> >> (Observation metadata in the larger extent) What would be the price
> >> to pay to remove this limitation?
> >
> >Well, at least the nice symmetry that the output of a TAP service can
> >be used as the upload for another would be sacrificed, unless one
> >lifted the requirement on single-table uploads.
> 
> ===> But hopefully you can upload several tables to be used in a query
>      (section 2.5.1 of TAP document) -- and anyway apart from 
>      cosmetic aspects, there is no reason for a symetry here.
It's not only cosmetic -- from what I've seen with users, the magic
happening when you use one service's output as the input to the next
is a major selling point for the VO.

Allowing multi-table VOTables in uploads would require severe changes
in the UPLOAD parameter(s) (2.5.1 and 2.5.2 in TAP1.0) with quite a
few consequential issues (VOTable does not mandate id and/or name on
tables; even if you mandate them for TAP, figuring them out for each
VOOTable instance at query writing time is a nasty burden, etc).  I
still think it's too late in the TAP process to iron all that out.

So, *if* we allow multi table VOTables to be returned, there should
be a reliable way for clients to figure out if the TAP results are
suitable for uploading.  This could be as easy as "If LANG is empty,
PQL or ADQL, services MUST return a single-table VOTable.  For other
values of LANG, arbitrarily complex VOTables could be returned."


> >Because of that -- and of Gerard's observation that TAP currently
> >defines no query language that would allow you to generate
> >multi-table responses -- I'd suggest we leave the single-table
> >requirement and postpone multi-table responses until we have more
> >experience with TAP.
> 
> ===> Here I think it's a mixture between TAP and the query language;
>      if neither AQDL not PQL currently can return more than one table,
>      the conclusion should not be, imho, a requirement that TAP returns
>      a single table. 
True.  On the other hand, I'm not sure whether TAP -- a rather
complex beast already -- should be required to cover even more ground
than the (obviously nontrivial) "Deliver pieces that lead to *one*
SQL query to a web service and retrieve the results".

And it's not only philosophy: Once you allow multiple tables, you'd
have to say what should happen if someone orders CSV output and the
result consists of multiple tables.  There are probably more
pitfalls like those when you go through the document.

> There are really many cases where getting as a result more than
> one table is useful and brings a lot more efficiency: the example
[Non-normative: I've already had users come to me complaining they
"couldn't read" a VOTable they got back from NED -- and sure enough,
it was such a multi-table beast, and I had to admit that using
generic tools like topcat, such tables aren't very attractive.  So, I
think it pays to try hard to deliver single tables from generic
services.  Of course, when communicating with special tools (say,
Vizier <-> Aladin), that situation is completely different.]

> Maybe the conclusion for us should be, in the case of a single-table
> output, that TAP is not the solution for these two services 
> (Simbad and Vizier) ? 
Well, my take on it is that many users will just *love* to be able to
run ADQL queries against Vizier's data holdings, and maybe Simbad's as
well. I'd actually be pretty curious what kind of queries people
will come up with once they can.

A "give me all catalogue entries you have 20 arcminutes around
Rigel"-type query on the other hand is, I believe, something else --
it's a cone search, so it has a (simple) data model behind it, where
TAP is essentially data model free.

Sure, one could re-introduce data models by defining a new query
language (e.g., you could strip down the WHERE-equivalent to just a
cone spec, but would allow really complex expressions in the FROM).
I still maintain such a query language is a bad match for TAP (think
TAP_SCHEMA), however.

So, summing up: *If* you really feel (and I don't) that TAP will at
some point have to support languages specifying results with multiple
tables, at least clearly specify which languages must return single
table-VOTables; and make sure there are nothing in the rest of TAP
that breaks with such a change in model (like the CSV issue mentioned
above).

Cheers,

         Markus