[TAP] sync vs async - time outs

[Doug Tody]

On Wed, 4 Mar 2009, Patrick Dowler wrote:

> On Wednesday 04 March 2009 07:48:19 Tom McGlynn wrote:
>> I agree with what you had said more elegantly in your earlier message.
>> My point was to emphasize my sense that that people were saying
>> asynchronous calls somehow obviated the need for careful timeout
> 
> The UWS spec already has a mechanism for specifying/negotiating
> how long the job will actually be allowed to run, which covers the
> TIMEOUT behaviour for async jobs. This is independent of a (larger)
> time that the service will keep the job (and results) before destroying
> it (and the result or error document).
> 
> Otherwise, DAL generally uses MAXREC to specify "how much work" but I
> think we all agree that this does not capture the it very well at all.

There has never been any question that an async mechanism is needed
for large jobs.

MAXREC serves a different purpose, providing a simple means to limit
the amount of output that comes back in a query response.  This was
developed originally for DAL data queries, which are always quick to
compute and for which an interactive response is generally desired even
if it is incomplete (as with a google query for example).  Such queries
are often iterative and it may not be necessary to receive the full
query response in each iteration, as the output will only be thrown
away anyway as the query is refined.  If eventually a full response
is desired MAXREC can be increased, e.g., to allow a large synchronous
streaming transfer that might run for several minutes, but the sensible
default policy is to limit the query response to some reasonable value.

Providing MAXREC in TAP is desirable if only for reasons of
consistency, so that code can be shared for processing the query
response - in terms of formatting and returning an output table,
TAP, SIA, SSA etc. are all very similar.  But even in TAP, having
a simple means to limit the size of a query response is desirable
(SQL TOP provides a similar mechanism but is SQL dependent and only
applies if it is explicitly included in the query expression).

Note also that the limit on the size of the output table and the job
time limit are in general independent.  It could take a very long time
to compute a small output table, or an enormous amount of data could be
returned from a very simple query which takes little time to compute.
Hence we need different mechanisms to control each.

 	- Doug