[TAP] sync vs async

[Patrick Dowler]

(in response to a point in Gerard's email of 2009-02-19)

In TAP-0.4 both the asynchronous and synchronous query execution are required. 
Gerard proposes that they be optional and services implement (at least) one.

The reason for this is that in the past (currently) we have services with 
arbitrary limits that make it hard for users to do bigger queries (bigger 
results or longer waits). These limits are all over the place (http timeouts, 
maximum result size, network stability while downloading large results using 
wifi in the airport lounge or back row of the interop meeting, temporary 
resource limits on the service, etc).

Async queries following the UWS pattern aim to solve these sorts of issues and 
in previous interops it was deemed essential to the success of TAP to solve 
these issues. Async as specified requires server-side state. 

* implementation note: When implementing a prototype I started out with a 
simple UwsJob class and stored instances in memory. That's pretty fragile and 
short-lived so I then moved implemented storing the jobs (query and uws 
state) in a database. In between, one could store the entire job state in 
small XML files (the XML for the UWS part is specified in the UWS doc and the 
TAP doc will have to describe the XML representation of TAP-specific stuff). 
That would work fine to but it seemed that it would be harder to deal with 
subtle changes and maybe bugs in the way the XML was formulated. 

Sync requires no server-side state (in the sense above). Sync is easier to 
implement on it's own, but does not solve the long-standing problems noted 
above in a generic way. It is true that a user could figure out how to 
work-around those issues by executing many smaller sync queries, but that 
moves a large burden onto the client (not just the tools - you usually need 
some expert knowledge of the content to perform such a work-around). 


Implementing /sync in addition to /async is a small amount of extra work. In 
fact, I think a recent version of the UWS doc discusses this. This and the 
utility of sync for simple queries (such as metadata queries) lead to making 
it required. That is much easier for users than having major optional 
features.

It should be noted that currently the VOSI requests are all made via the /sync 
endpoint (GET and POST to the /async endpoint is already fully specified by 
UWS). While this is largely a cosmetic issue, it does make the /sync endpoint 
and other sync-only DAL services (eg SSA) highly consistent. The VOSI 
requests are always executed synchronously and putting them here makes for 
minimal amount of surprise. Given that the VOSI capabilities would have to 
describe which optional things are implemented, it would be odd for /sync to 
be optional... this might be a non-costmetic issue of we were talking about a 
service where sync made no sense at all.

Summary: 

Comments?



-- 

Patrick Dowler
Tel/Tél: (250) 363-0044
Canadian Astronomy Data Centre
National Research Council Canada
5071 West Saanich Road
Victoria, BC V9E 2M7

Centre canadien de donnees astronomiques
Conseil national de recherches Canada
5071, chemin West Saanich
Victoria (C.-B.) V9E 2M7