[DALI] posting parameters to a DAI-async (aka UWS) job

[Paul Harrison]

On 2013-04 -24, at 17:43, Patrick Dowler <patrick.dowler at nrc-cnrc.gc.ca> wrote:

> 
> The PR-DALI and the UWS spec (interpretting the latter liberally) allow the caller to POST job parameters in three ways:
> 
> 1. included with the initial post to the job list (eg during job creation). This is useful and efficient as it allows one to completely specify a job in one POST and (for DALI) it makes sync and async that much more similar to use. We definitely want this.
> 
> 
> 2. to the parameters resource (eg /srv/joblist/<jobid>/parameters). This is unambiguous: you are modifying that list of parameters.
> 
> 
> 3. directly to the job url (eg /srv/joblist/<jobid>). This is kind of sketchy to implement, especially if one starts thinking about allowing the UWS job control params here (and in #1) and then "putting things where they belong"...
> 
> *** TODO ***
> 
> What we can do right now in DALI is restrict what the UWS-1.0 spec allows. The suggestion is to completely forbid #3 and only allow #1 and #2.
> 
> 
> Disclosure: The OpenCADC library (cadcUWS) does make all of these work, but while we do use feature #3 I'm willing to make that change as I did always consider it overly complex (to implement) and lazy (on the caller side).
> 
> 
> Notes on #2: For this to be really purely RESTful, one should require PUT to add a parameter to the list and POST (to /srv/joblist/<jobid>/parameters/FOO) would be used to modify an existing param. DALI allows multiple instances of a param so the current UWS/interpretation does not have a way to distinguish between adding a param and modifying one. BUT: I think this is something for a future UWS spec.
> 
> 


Initially the intention of UWS was that all job parameters were only specified at job creation - i.e. as #1 above and that the endpoint 
/srv/joblist/<jobid> was only to be used to set "job control" parameters (e.g. execution duration) - nothing was said about the contents of the initial POST that created the job, and as that was the only way to specify the functional parameters for a job it was possible for a client to resubmit a job by simply sending the same POST.

Then when TAP wanted to use UWS it became clear that UWS ought to support the notion of the simple name/value parameter pairs that appear in typical DAL services, so this was added to UWS 1.0, and additionally the ability to set parameters individually has been added (as in #2) to the latest draft (1.1) see (http://volute.googlecode.com/svn/trunk/projects/grid/uws/doc/UWS.html) of UWS following discussions at the Napoli interop.

It should be noted that this current 1.1 draft says
8<------
A particular implementation of UWS may choose to allow the parameters to be updated (but not created) after the initial job creation step, before the PHASE is set to the executing state. It is up to the individual implementation to specify exactly how these parameters may be updated, but good practice (in the REST binding) would be to choose one of the following options.

	• HTTP POST an application/x-www-form-urlencoded parameter name, value pair to either
		• /{jobs}/{job-id}
		• /{jobs}/{job-id}/parameters
	• HTTP PUT the parameter value to /{jobs}/{job-id}/parameters/(parameter-name)
8<------

i.e. it is not very prescriptive, only suggestive - From what I remember it was rather difficult to find common ground between those who wanted lots of flexibility and those who thought that there are too many options for setting parameters. For my part I would like more simplicity and clarity and would definitely like to ban some of the possibilities.

Paul