DAL/TAP

[Keith Noddle]

Hi Doug,

 > I am a bit disappointed as I thought we had agreed in our meeting
 > last November on the basic form and scope of the TAP interface,
 > but now we appear to be at risk of starting over yet again.
 > How much of our agreements from the November meeting (see
 > http://www.ivoa.net/twiki/bin/view/IVOA/TapJhu) do we consider to
 > still be in effect?
I'm disappointed you are disappointed - let me seek to assuage your 
disappointment! Nothing I have proposed contradicts or derails anything 
we said last November as far as I am concerned. However, we hardy 
represented a DAL/TAP quorum and it is important for me, as DAL Chair, 
to recognise that the VO projects have differing expectations of TAP. 
Some have compelling Use Cases that call for swift progress on 
TAP/Param, others that drive TAP/QL.

I am seeking to ensure the definition of TAP/QL has the same parity as 
that of TAP/Param. I have decoupled them to ensure each can progress - 
or stumble - without affecting the other. The work done within NVO on 
what is effectively TAP/Param is excellent news and something to brought 
to the Interop for demonstration and discussion, so I really feel 
nothing is endangered. I do accept that it is highly desirable to have a 
single TAP standard which is why I included my Qualifications to try to 
ensure we do nothing to preclude such a merger nor allow our intentions 
to be forgotten.


With the above in mind, I'll just make a few brief observations about 
the rest of your email (thanks for replying so quickly btw!) as maybe 
they are no longer in context:

 > Within NVO we have been proceeding as agreed last fall, with the
 > goal of having a partial draft interface for discussion plus some
 > prototyping done in time for the interop.  It remains a top priority
 > for NVO to have progress on a TAP working draft plus usable prototype
 > services later this year, including both a simple parameter-based
 > capability (which we could really use now if it was available), as well
 > as progress an ADQL-based capability with a start at integration of
 > Grid capabilities (async execution, vospace integration).  Both types
 > of capabilities are needed for different use-cases, and both are
 > essential.

Agreed - and some projects (AstroGrid included but I am trying very hard 
to keep my AG PM hat *off*) are subject to similar time pressures with 
respect to TAP/QL.


 > A key problem with having two separate interfaces is that, even for
 > an ADQL-based interface, it will still be desirable to have a simple
 > table metadata query capability. Since the exact same table metadata
 > is being queried in both cases, it would be unfortunate to have a
 > different approach and interface for querying table metadata for the
 > parameter and QL-based interfaces (we had exhaustive discussions of
 > this issue and the associated TAP schema in our November meeting
 > as detailed at the link above).  If we include a common, simple
 > parameter-based mechanism and associated TAP schema for table metadata
 > queries in the "TAP/QL" interface, then we are right back to what we
 > agreed last fall, with ADQL and parameter-based queries within the
 > same fully-scoped interface.

Hmmm, that's not the conclusion I drew from what was said. My 
understanding was that we agreed upon the desirability of supporting a 
simple call that returned all metadata as a single entity (VOTable) and 
we circled the question of whether the the metadata would be returned in 
an empty VOTable (proposed by François) or as the content of a VOTable.

We further suggested that support for more intricate metadata querying 
was most desirable, but that would be optional in TAP V1.x to allow 
prototyping and evaluation and that we would define the exact details in 
TAP V2. I remember this most clearly as it seemed to me a good 
compromise, allowing the VO Projects that manage their resource metadata 
in various ways to provide a single solution to the problem despite 
their very different back-end implementations.

Neither of these indicate that  "simple" (that word again - meaning 
TAP/Param) querying is required. So provided the getMetadata call (or 
whatever we decide to name it) is supported by both TAP/Param and 
TAP/QL, we have enough commonality AND enough diversity to justify 
progressing each standard in (closely-linked) parallel.


 > One thing I do agree with is that prototyping of the advanced
 > Grid-enabled, ADQL-based table query capability can proceed largely
 > in parallel with use of a simpler parameter-based query capability for
 > both table data and metadata queries.  It would be unfortunate though
 > to have these develop as two separate service interfaces - I think
 > that would merely indicate failure to agree within the IVOA on a TAP
 > interface, with two groups instead going off in different directions.

See comment above. Once the details of complex metadata querying are 
agreed, I would venture to suggest that TAP/Param may not be 
sufficiently comprehensive to provide the level of metadata 
interrogation you described. However, as that is not yet on the radar, I 
cannot speak with authority.


 > A possible compromise here which has some of the characteristics of
 > your two-track approach might be to proceed towards and integrated
 > TAP interface as we discussed last fall, focusing initially on
 > synchronous parameter-based table data and metadata queries, while
 > prototyping asynchronous ADQL-based queries with VOSpace integration
 > as a parallel effort.  The goal would be to standardize the basic
 > interface and capability for simpler parameter-based table data and
 > metadata queries capabilities as soon as possible, so that we can
 > all begin using this, folding in the ADQL and Grid capabilities later
 > once this technology is sufficiently advanced.

Much of this ADQL-based querying is already prototyped, deployed and 
working. Not to progress it now is to miss a golden opportunity.


 > "TAP V1.0" would support parameter-based table data and metadata
 > queries, possibly with prototype support for ADQL, and "TAP V2.0"
 > would add all the advanced capabilities.  We would all proceed with
 > prototyping of the ADQL/Grid capabilities beginning this year, but
 > would not attempt to finalize this portion of the interface until
 > prototyping is further along.  This would still provide the two-track
 > approach you describe, but with a more clearly defined path toward
 > getting to TAP V2.0.   Does that sound workable?

I don't think there's any real impediment to us progressing both 
simultaneously. Both approaches have been worked on and progress made. 
I'm sure we can keep two closely bound standards on track until such 
time as we can successfully merge them!

Keith.

-- 
Keith Noddle                    Phone:  +44 (0)116 223 1894
AstroGrid Project manager       Fax:    +44 (0)116 252 3311
Dept of Physics & Astronomy     Mobile: +44 (0)7721 926 461
University of Leicester         Email:  ktn at star.le.ac.uk
Leicester, UK   LE1 7RH         Web:    http://www.astrogrid.org