Running TAP/async jobs immediately

[Patrick Dowler]

I have added this to the WD-TAP-1.1 topics for discussion on Sat in DAL.

Pat

On 21 October 2016 at 06:41, Paul Harrison
<paul.harrison at manchester.ac.uk> wrote:
>
>> On 2016-10 -21, at 12:28, Brian Major <major.brian at gmail.com> wrote:
>>
>> Hi grid, dal,
>>
>> I'm restarting the conversation originally posted by Walter here:
>>
>>     http://mail.ivoa.net/pipermail/dal/2016-September/007593.html
>>
>> The request is for this to be a requirement for TAP services.
>>
>> UWS mentions this ability as a "possibility", but has no strong wording on the matter.
>>
>> I believe that having the ability to start asynchronous jobs on the initial job posting could be useful when one is uncertain about the amount of time it will take to execute the job.  This applies to TAP because queries are tough to estimate.  However, there are some questions to be answered:
>>
>> 1)  Would this use case be applicable to all applications of UWS, or is it more of a TAP specific issue?
>>
>> 2)  Similarly, would such a feature belong in UWS or TAP?
>>
>
> I think that the feature does not belong in UWS - it says all that it meant to on the subject - however, I will expand on the explanation why here.
>
> UWS describes the job initialisation step as being done by posting a “blob” (as far as UWS is concerned) of what it terms the "job description language”, and in general this could be anything - it is not necessarily application/x-www-form-urlencoded  - it could be an xml document for example - therefore there is no universal way for UWS to encode the job control metadata of a PHASE=RUN into this unspecified JDL . I have been tempted in the past to say that it could be encoded into the URL of the POST,  i.e. POST to {jobs}?PHASE=RUN however I am not sure that is strictly allowed in HTTP and anyway definitely goes against the usual practice of the query part of the URL being for a GET, and some web toolkits might not interpret it properly.
>
> I think that all the IVOA standards that use the UWS pattern do specify that they are using  application/x-www-form-urlencoded as the initial post, and as long as there is no clash of parameter names with its JDL then the dependent standard can say that it is ok to place PHASE=RUN in the job creation POST
>
>
>> 2)  How do implementers feel about possibly updating their TAP (and perhaps other) services to do this?
>
> I do not believe that it is particularly difficult to implement - given that the code must exist to do this for a subsequent post on a job - I certainly found it trivial to do in my implementation of a more general UWS server in Java.
>
>>
>> 3)  Does having such a feature outweigh having more required functionality in UWS (or TAP) services?  Or is it too small a change to cause inconvenience?
>>
>> Your thoughts on this would be appreciated.
>>
>> Regards,
>> Brian
>
> Cheers,
>         Paul.
>
> Dr. Paul Harrison
> JBO, Manchester University
> http://www.manchester.ac.uk/jodrellbank
>
>
>



-- 
Patrick Dowler
Canadian Astronomy Data Centre
Victoria, BC, Canada