DAL/TAP Response

[Keith Noddle]

There has been very little activity on the DAL mailing list regarding 
TAP and I confess I too have been at a loss as to how to respond.

Firstly, thank you François and Bob for your input, I will address the 
points you raise below. However, I am struggling with how to respond to 
certain aspects of Bob's posting of 17th December without causing 
another perturbance, but...

I am concerned that we are recycling ideas and not progressing.

Let me make clear my aims as DAL Chair:

(1) provide a TAP specification that meets the needs of *every member of 
the IVOA* - even if that means rapidly producing a number of incremental 
revisions to the standard

(2) have a first-version TAP standard that meets, as closely as 
possible, the expectations of the DAL audience at the next Interop.

I am firmly of the opinion that we have to compromise, agree what can be 
agreed and finalise that, then break up what remains into manageable pieces.

===
Following the last three Interops and the Trieste one in particular, my 
interpretation of the intent of the IVOA regarding TAP is this:

======================================================================
TAP should define an asynchronous service that consumes ADQL queries and 
returns VOTable formatted results. There are additional bells and 
whistles that also should be included. These are well understood.
======================================================================

To address NVO concerns that not all Data Centres have the resources to 
implement TAP compliant interfaces, we included the notion of simple, 
synchronous, parametrised querying as an optional TAP capability. To 
distinguish between the required and optional query interfaces, we 
called the former TAP/QL and the latter TAP/Param.

All this is recorded in my Trieste Plenary talk.

Between Interops, when activity within the IVOA traditionally 
diminishes, focus has again returned to the importance of parametrised 
querying. It seems clear to me that this is a strongly held view but 
which is seemingly in conflict with the collective decisions of the 
IVOA. However, simply losing a vote does not diminish the importance of 
an issue _in the view of those that regard it as essential_.

Whatever we decide to do next, the current TAP definition process is not 
working nor will it ever truly be successful; it simply tries to bite 
off too much in one go. We are in difficulties and it seems to me there 
are at least 3 possibilities:

===
(1) We adhere to the decisions of the IVOA and define a TAP standard 
that includes only asynchronous ADQL-based querying. We follow that 
quickly with the next revision of the TAP document that includes 
optional parametrised querying. Separating them out will allow progress 
on each to be made.

(2) We revisit the IVOA decision, turn it on its head and make 
parametrised querying mandatory, define a standard that enshrines that 
and in a later revision add an ADQL querying option.

(3) We revisit the whole notion of TAP and accept two standards that 
define Table Access: SimpleTAP that defines Parametrised querying and 
TAP that defines ADQL based querying (actual names TBD). We bring both 
to the IVOA as and when they are ready as independent standards. We then 
allow (recommend, cajole) the Data Centres to choose which they 
implement - or more likely, which they deploy based upon the 
off-the-shelf services the various VO projects offer.
===

Option (2) is a non-starter. We cannot, after all this time, make this 
U-turn.

Option (3) flies directly in the face of all that the IVOA stands for. 
If we introduce two "standards" to achieve the same goal, what then 
homogeneous access to astronomical resources? It too is self eliminating.

I actually feel that Option (1) is the best and quickest way to address 
the needs of those wanting to get parametrised querying right. Various 
correspondents seem confident that a TAP V1.0 with QL only can be 
completed very quickly. With that done, we can concentrate our energies 
on adding PARAM in V1.1 as an optional optional, additional capability.

_It is VERY important to recognise that defining IVOA standards is an 
on-going journey, that the standards we define will evolve and that this 
is to the good. We are not about defining standards that are then cast 
in stone for all eternity. As an organisation, the IVOA must adapt or die._

I, and I'm sure most of those involved with TAP, am enormously 
frustrated that we are spinning our wheels on this when DAL could be 
successfully addressing much bigger opportunities.

===
To address the issues raised in the postings:

Some of the points in Bob's email will be addressed by simplifying TAP 
V1.0 as described above. These include Section 1, various 
inconsistencies and verbiage. The comments regarding section 2.1, 
2.4.12-14, 2.7 and 7 are all well observed and need to be fixed. Section 
2.9.1 needs to be clarified. It relates to versioning of the protocol 
supported by the service. In light of agreements made after this draft 
was written, this section can be suitably revised. Other suggestions 
pertaining to TAP results, section ordering and extra description will 
be taken into consideration during editing.

François make very specific points, each of which will be taken into 
account during preparation of the next version of TAP. Specifically 
those relating to VOTable V1.2 need to be addressed. It would be silly 
to provide exceptions related to VOTable V1.1 if the next (imminent) 
version fixes them. The same applies to the MIME type comments. Also, we 
must be sure we are consistent with other standards (VODataService in 
particular) but again, this will be undertaken as part of the next 
revision. Other comments about TAP_SCHEMA are equally well observed. 
Finally, our definition of time format should indeed be consistent 
across the document.
===


-- 
Keith Noddle                    Phone:  +44 (0)116 223 1894
AstroGrid Project manager       Fax:    +44 (0)116 252 3311
Dept of Physics & Astronomy     Mobile: +44 (0)7721 926 461
University of Leicester         Skype:  keithnoddle
Leicester                       Email:  ktn at star.le.ac.uk
LE1 7RH, UK                     Web:    http://www.astrogrid.org