Issues relating to simple data and metadata queries

[Doug Tody]

Hi All -

It is good to see some conceptual/technical discussion and analysis,
even if we don't all agree yet.

The asynchronous mechanism (stageData) is still only a concept and
has not been specified, so any analysis of the requirements for
long running table queries is welcome.  Since the specification of
a long running job is data dependent though, and can be specified
differently for an image or a table, while the basic job control
and data management mechansims (probably based on the UWS pattern
and VOSpace) should be largely independent of the data type, it is
not clear why there should be any fundamental issue with a similar
approach.  Basically the pattern is job estimation, submission,
monitoring, and eventual data retrieval or delivery.

Considering only the synchronous part of the interface, I wonder if
there is any possibility of agreement if we permit both VOTable and
XML/VOResource output for metadata queries?  This would provide both
a uniform query mechanism to make things simpler for most client
applications, plus XML for those who have existing software which
requires it.  So long as the information content is the same (or
a subset), serialization to either format (or CSV for that matter)
is pretty simple.

 	- Doug


On Tue, 1 May 2007, Kona Andrews wrote:

> Hi Pat,
>
>> - the difference in opinions on various TAP issues is not all that large
>> conceptually, just lots of details differ
>
> I agree that there are places where our collective opinions on
> the synchronous features of TAP are probably closer than they seem.
>
>> My thought right now is that TAP is somewhat different from other DAL services
> I also agree that TAP is somewhat different from other DAL services.
> In particular, I will shortly post a discussion (by an external author)
> of asynchronous querying for TAP.  This discussion explains very clearly
> why the current DAL model works very well for e.g. SIAP and SSAP, but
> not so well for long-running tabular data queries.  I'm afraid it
> is a reasonably long message (sorry!), but that is mostly because it
> takes the time to express itself very clearly and understandably. I do
> hope everyone can at least glance at it.
>
> If, as that discussion suggests, the DAL model is not a good fit for
> asynchronous ADQL querying, I don't think we should be *constrained* to
> make any/every part of TAP concur with the pre-existing DAL protocols
> (however well they work for other kinds of data).  It may be that some
> aspects *do* concur naturally with the existing DAL-based protocols for
> other kinds of data, of course; but let's not force the whole thing to
> be a square peg for a round hole.
>
> Cheers,
> Kona
>