TAP RFC [VOSI]

[Guy Rixon]

Hi Alberto,

I'm making a distinction between a generic TAP-client (which might be  
a software library or might be an application or might be in a web  
portal) and one intended only for services known to be fast. E.g. if I  
write a 10-line, disposable Python script, I try a basic GET on the / 
sync resource before I even consider UWS. If I write a shared,  
maintainable app for a group of 20 services which are all fast, then I  
use synchronous query. But if write a general portal on behalf of  
EuroVO then I better damn well implement the asynchronous client,  
because 5% of all use cases is too much to lose, especially when those  
might be the most productive. It would be rude to the users not to  
provide this.

To get this in perspective, the client part for a web portal can be  
really, really simple. You could try it with the test app I mentioned  
earlier.

Try this: write a web for that submits  HTTP POST to

http://casx020-zone2.ast.cam.ac.uk:8080/wfs-objects/TAP/async

with TAP query-parameters. A suitable test-query would be

select top 1000 * from maincat

which would be quick enough for synchronous querym, or you can scale  
up a little so that it takes longer. (Caveat: pre-beta implementation,  
so you can probably break it if you try.)

When you submit this, the browser will get redirected to the jobs  
resource and that resource will show up as a web page (HTML ex XML via  
XSLT in the browser) that says whether the job is QUEUED, RUNNING,  
COMPLETED etc. Keep pressing reload in the browser until the job-phase  
goes to COMPLETED and then the results are hyperlinked from the job  
page.

It's ugly as sin, but the only thing you have to write in the portal  
is the form for the initial post. If the portal author wants to pretty  
it up, then he can run the XML-HTML transformation inside the portal  
and maybe do the polling with Javascript.

I'll post the basic XSTL for the transformation if anybody's interested.

Cheers,
Guy

On 30 Sep 2009, at 12:11, Alberto Micol wrote:

>
>
> Hi Guy,
>
> Again, I'm not disputing on the usefulness of a ASYNC.
> I'm saying that the client should not ask for it (in 95% of the  
> cases), and let the service decide how to react.
>
> Is it for that 5% of the cases that we want to build a porsche  
> instead of a the famous Andy's bike?
>
> Alberto
>
>
> On 30 Sep 2009, at 13:02, Guy Rixon wrote:
>
>>
>> On 30 Sep 2009, at 11:33, Alberto Micol wrote:
>>
>>> [...]
>>> All that going back and forth is completely unnecessary (unless of  
>>> course there is a real
>>> bug in the question, but not otherwise).
>>> - If the service decides upfront to use ASYNC (because it offers a  
>>> huge catalog) the user
>>> will simply send his query, and the answer will be to please use  
>>> ASYNC.
>>> - If the service decides upfront to use SYNC (because it offers  
>>> access to a small catalog) the user
>>> will simply send his query, and will receive her answer shortly.
>>> Of course, the problem arises if a huge catalog is served only in  
>>> SYNC mode, or if
>>> a small catalog is served and the network connection is not that  
>>> good.
>>> Timeouts will likely  happen often in those two cases. In such  
>>> case, yes, the user will have to limit
>>> using MAXREC, if not done so by the provider herself. Some  
>>> handling of the kind proposed by Pat
>>> will always happen, but we should limit the number of cases to  
>>> only the strictly necessary ones,
>>> balancing it out with the burden otherwise imposed to data  
>>> providers.
>>>
>>> In one sentence: Why complicating things at both ends?
>>>
>>> Alberto
>>
>> There is a third case where an asynchronous query is needed: when  
>> the query is known a priori to be long-running and the client  
>> doesn't intend to stay connected.
>
>> This is possibly rare, but it's exactly the sort of special case -  
>> large-scale data-mining, basically - where the VObs can be worth  
>> more than its its predecessors.
>>
>> We need to make the asynchronous gear optional in services (Gerard  
>> proposed this months ago), for the reasons you yourself have stated  
>> in the last few mails. If we did make asynchronous queries optional  
>> in 1.00, then we urgently need a way to denote this in the service  
>> metadata, as per my earlier email today. This must be part of the  
>> TAP standard, not the VOSI standard.
>>
>> However, a generic TAP client is much, much less useful if it  
>> ignores UWS where offered. It would be a crying shame if the VObs  
>> became limited to piffling little queries because the application  
>> writers wimped out. It's a lot simpler to write a UWS client for  
>> TAP than it is to write the service end.
>>
>> Cheers,
>> Guy
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.ivoa.net/pipermail/dal/attachments/20090930/ecbe9111/attachment-0003.html>