Running TAP/async jobs immediately: Proposal Text

[Brian Major]

On Sun, Oct 2, 2016 at 11:33 PM, Marco Molinaro <molinaro at oats.inaf.it>
wrote:

> 2016-09-30 23:02 GMT+02:00 Walter Landry <wlandry at caltech.edu>:
> > Patrick Dowler <pdowler.cadc at gmail.com> wrote:
> >> If people are doing quick queries that respond immediately and they
> >> want to minimise connection overhead... TAP-sync? That's what it is
> >> for. With TAP-async you have to stage the results someplace, which
> >> adds way more latency than an extra connection... so, use TAP-sync!
> >
> > TAP-sync is not appropriate for queries which might have unbounded
> > execution time.
>

I can see the use case for that.  Sometimes clients don't have enough
information about the job being run to make a decent estimate on execution
time, so you send it to async to be safe.

But, I also agree that the cost of doing the extra POST to start the job
usually is minor compared to the overhead that comes with async support.


> Also, staging happens at the archive in server farms.
> > Connections happen over the internet over spotty wifi.  The result does
> > not even have to be physically written to disk before the user fetches
> > it.  So I can easily see places where staging is much faster than
> > making connections.
> >
> > Moreover, an optimistic user can get by with only 2 connections
> > (submit + fetch), because they can tell if they get back a 404 or
> > incomplete result.  We should not be making our protocols needlessly
> > slower.
> >
> >> While TAP-async could mandate support for this, we cannot remove
> >> support for the multi-request UWS way (so you need that too) and thus
> >> we can't mandate clients use this... which means Brian is right: you
> >> can do this and your own client/portal can take advantage of it with
> >> no spec change.
> >
> > Mandating support for single-request means that people can rely on it.
> > It makes TAP-async faster and easier to use.  I do not know the details
> > of everyone's implementation, but I would fall off my chair if this is
> > actually more work than writing these emails ;)
>
> Probably it's not, but it's a change in standard.
> Re-reading Mark's comment, it may push TAP to 2.0.
> Not sure it's the right timing.
>

Maybe the language in UWS should be tightened up too?  It could say that
job control parameters need to be recognized on job creation unless the
implementing protocol disallows it.  But again, this would be 1.1 or 2.0.

Brian


>
> Marco
>
>
> >
> > Cheers,
> > Walter Landry
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.ivoa.net/pipermail/dal/attachments/20161003/c0355fc0/attachment.html>