comments on contents of TAP 0.31

[Douglas Tody]

Hi Pat, All -

On Mon, 23 Feb 2009, Patrick Dowler wrote:
(responding to mail from Gerard)
> 
> That was a very long list and this is great. I read through it and I
> estimate that about 3/4 of the points can be addressed by clarifying
> the document. I am in the process of integrating such improvements
> now and hope to post a new revision of the document asap.

I also thought that Gerard had a number of good points, e.g., async
is more complex than sync and it could be more logical to make the
simple mandatory and the complex and advanced functionality which is
still being evolved optional.  The TAP schema and its VODataService
analogue also need further refinement.  Just a reminder though that
NVO also responded last week.  We plan to address this more fully
(probably by proposing specific document changes) once your updated
draft is available.

Our comments (submitted by Bob last Thursday, but prepared by several
of us) are available here:

     http://www.ivoa.net/forum/dal/0902/0991.htm

To reiterate:

     We do have some reservations, however, about the notion of turning
     TAP param-based queries into a generic query language that is
     supposed to work across all DAL interfaces. There are different
     parameters for different purposes depending on whether one is
     accessing images, spectra, or tables. The same parameter can have
     different semantics depending on the type of data. (Example:
     SIZE, which in TAP defines a circle in which to search and in
     SIAP defines a box that can be the requested size of a dynamically
     generated image.) We could quite easily tie ourselves into semantic
     knots and contradict the design of the DAL2 interfaces if we go
     down this road too far. This is why LANG, which suggests that
     the external query specification is a fully defined language,
     seems less appropriate than the alternatives mentioned above.

     Simply put, we are ok moving the specification for param-based
     queries into its own document, but not with attempting to
     generalize this into "PQL". The specification of param-based
     queries should focus on how to query table data and metadata within
     the context of TAP, including capabilities such as multi-position
     queries and table metadata queries. There are additional details
     that need to be added to the param-based query specification
     along these lines, and we are happy to contribute to this work.

Basically we agreed, in the spirit of compromise and to help move
things along, to split the TAP spec into three documents (core TAP,
AQ/ADQL, PQ).  As you, Keith and others have proposed, this would
essentially replace AdqlQuery and ParamQuery by a single "doQuery"
service operation (or whatever we decide to call it), with the
QUERYTYPE or QUERYMETHOD parameter defining the type of query being
posed, with ADQL/QL and parameter-based queries at the same level in
the interface.  We would probably want to replace LANG with QUERYTYPE
or QUERYMETHOD (whatever we pick) since this more generalized query is
no longer being restricted to parsed language syntaxes, as for example
the param query formulation is not an actual general query language.
We could quibble over the form of the interface but it does not make
a great deal of difference so long as the basic functionality provided
is comparable.  The actual functionality provided remains much has been
described over the past several months, i.e., a general syntax-based
query method (defaulting to ADQL) and simpler param-based queries.

In other words a generalized PQL is not the right concept, and is
not adequate to fully specify parameter-based table data and metadata
queries for TAP.  While such a mechanism might be useful in the longer
term, our immediate problem is to produce a useful TAP interface (we
certainly need this to support NVO TAP prototypes and applications very
shortly) and we already have things like the proposed DAL2 "generic
dataset" concept which address this more general problem of developing
standards for generic queries which apply to all DAL interfaces.

Hence while we can agree with the proposed revision to TAP, we need
to replace the proposed PQL with something which fully specifies how
to do simple parameter based TAP queries, supporting "multicone",
basic filter type catalog queries, TAP_SCHEMA based table metadata
queries, and so forth.  This is a high priority for NVO at least,
to provide something robust to use for applications while we develop
the more advanced functionality, as well as to provide a simple client
interface after all this technology is mature.

Perhaps we could call this something like "Table Access Protocol;
Simple Parameter Queries", or some such, instead of posing this as
a general parameter query language which tries to apply to all data
while not being specific enough to be useful for any actual data such
as a tableset, image, or spectrum.

It could also be useful to have something like this addressing in more
detail how to use ADQL queries within the TAP context.  ADQL unlike
TAP param query actually is a general query language, and it is not
at all clear how to use to use to query tables within TAP.  How to use
such a general facility within the TAP context is not clear and should
be clarified if we expect service providers to actually implement this,
or client application writers to use such a facility effectively.

Ultimately we would end up with a core TAP interface and engine,
plus documents detailing ADQL/other QL based queries, as well
as the simpler astronomy oriented parameter based queries, plus
auxiliary documents defining the general ADQL language, UWS, VOSI,
VOTable, etc. etc.  But in the meantime we need to have enough of TAP
specified to support all our joint use cases and prototyping efforts.
We urgently need to do this *now* in order to complete another round
of prototypes in advance of the Strasbourg interop.

 	- Doug