DAL/TAP

[Doug Tody]

Hi Keith -

I am a bit disappointed as I thought we had agreed in our meeting
last November on the basic form and scope of the TAP interface,
but now we appear to be at risk of starting over yet again.
How much of our agreements from the November meeting (see
http://www.ivoa.net/twiki/bin/view/IVOA/TapJhu) do we consider to
still be in effect?

Within NVO we have been proceeding as agreed last fall, with the
goal of having a partial draft interface for discussion plus some
prototyping done in time for the interop.  It remains a top priority
for NVO to have progress on a TAP working draft plus usable prototype
services later this year, including both a simple parameter-based
capability (which we could really use now if it was available), as well
as progress an ADQL-based capability with a start at integration of
Grid capabilities (async execution, vospace integration).  Both types
of capabilities are needed for different use-cases, and both are
essential.

A key problem with having two separate interfaces is that, even for
an ADQL-based interface, it will still be desirable to have a simple
table metadata query capability.  Since the exact same table metadata
is being queried in both cases, it would be unfortunate to have a
different approach and interface for querying table metadata for the
parameter and QL-based interfaces (we had exhaustive discussions of
this issue and the associated TAP schema in our November meeting
as detailed at the link above).  If we include a common, simple
parameter-based mechanism and associated TAP schema for table metadata
queries in the "TAP/QL" interface, then we are right back to what we
agreed last fall, with ADQL and parameter-based queries within the
same fully-scoped interface.

One thing I do agree with is that prototyping of the advanced
Grid-enabled, ADQL-based table query capability can proceed largely
in parallel with use of a simpler parameter-based query capability for
both table data and metadata queries.  It would be unfortunate though
to have these develop as two separate service interfaces - I think
that would merely indicate failure to agree within the IVOA on a TAP
interface, with two groups instead going off in different directions.

A possible compromise here which has some of the characteristics of
your two-track approach might be to proceed towards and integrated
TAP interface as we discussed last fall, focusing initially on
synchronous parameter-based table data and metadata queries, while
prototyping asynchronous ADQL-based queries with VOSpace integration
as a parallel effort.  The goal would be to standardize the basic
interface and capability for simpler parameter-based table data and
metadata queries capabilities as soon as possible, so that we can
all begin using this, folding in the ADQL and Grid capabilities later
once this technology is sufficiently advanced.

"TAP V1.0" would support parameter-based table data and metadata
queries, possibly with prototype support for ADQL, and "TAP V2.0"
would add all the advanced capabilities.  We would all proceed with
prototyping of the ADQL/Grid capabilities beginning this year, but
would not attempt to finalize this portion of the interface until
prototyping is further along.  This would still provide the two-track
approach you describe, but with a more clearly defined path toward
getting to TAP V2.0.   Does that sound workable?

	- Doug








On Mon, 10 Mar 2008, Keith Noddle wrote:

> Dear DALers and UWSers,
> 
> I have deliberately been quiet re: TAP since last autumn to allow the dust to
> settle and mature reflection to take place. Clearly we have a wide range of
> table access requirements and it is my responsibility to ensure they are all
> equitably met.
> 
> After much thinking, discussion and water-testing regarding TAP I believe I
> have a proposal that meets the needs of all concerned and which will deliver
> solutions to table access in general that we can all support. My thinking is
> as follows:
> 
> (A) We have misled ourselves by calling parametrised table access SimpleTAP
> (it was called STAP a long time back you may recall). That led to the
> development of the VOQL-WG STAP proposal that (correctly if with hindsight,
> misguidedly) adhered closely to the other "Simple" protocols but which was
> rejected at Beijing. Henceforth, I propose that parametrised table access be
> known as "TAP/Param" and we drop all reference to the word "Simple"
> 
> (B) For completeness, I shall call table access using ADQL: "TAP/QL"
> 
> (C) By trying to include parametrised access along with ADQL access in the
> same specification, we inadvertently put two sets of requirements together
> that are in many ways incompatible - how much time have we spent gently
> circling the issues of synchronous/asynchronous access, staged versus streamed
> result returns etc?
> 
> I therefore propose that we separate the two table access mechanisms into
> separate standards: TAP/Param and TAP/QL and that we progress them
> independently, each unencumbered by the sometimes incompatible requirements of
> the other. However, there are obvious common areas between the two (output
> formats spring to mind) that both must specify but which are easy to agree.
> Hence my proposal has two qualifications:
> 
> (1) We include in both specifications a Contract that defines the required
> common areas (such as output formats)
> 
> (2) We include a Statements of Intent to: "endeavour to conflate the two
> specifications into one as part of the definition of TAP V2.0".
> 
> I realise that at first glance this appears to double our workload, but in
> practise I believe it will clear the logjam - we've spent far too long on this
> already and without some radical change, I don't foresee comfortable
> resolution any time soon. Qualification (1) above will not involve much extra
> work (mostly an exercise in documentation) whilst Qualification (2) will allow
> us to assess both standards in operation and decide the best way to harmonise
> the them once we know in practise how they work. Essentially I want to define
> a standard based upon working practise: in this I adhere to the observations
> of Marshall Rose, "The Pied Piper of OSI" :-)
> 
> I want to make progress before the Trieste Interop so that we have details to
> discuss, not principles to agree and I believe taking this approach will allow
> us to do so.
> 
> Comments?
> 
> Keith.
> 
> -- 
> Keith Noddle                    Phone:  +44 (0)116 223 1894
> AstroGrid Project manager       Fax:    +44 (0)116 252 3311
> Dept of Physics & Astronomy     Mobile: +44 (0)7721 926 461
> University of Leicester         Email:  ktn at star.le.ac.uk
> Leicester, UK   LE1 7RH         Web:    http://www.astrogrid.org
>