Feedback from UWS session - REST criticisms

[Paul Harrison]

Hi,

It was a shame that we did not have a little more time to discuss some of the issues raised.  As was mentioned I believe that there are two basic categories of issue raised in the session;

1. Clarifications - I think that these can probably be dealt with by issuing a new version of the UWS standard - this might even be only a version 1.1 with a strong emphasis on maintaining strict compatibility (with perhaps some small extensions)
2. Enhancements - some of these might require a new version of UWS or even result in a new standard to be layered on UWS.

In this email I will address the first of these categories with specific reference to the criticisms of the use  of REST within UWS 

Firstly the REST "purity" characteristics were constrained by two requirements

1. The UWS should be drivable from a browser (without use of javascript)
2. It should fit in with the existing patterns of usage of DAL services.

In general I agree that there needs to be some clarification - particularly of returned status codes, but by and large we cannot consider making most of the changes suggested to the REST implementation of UWS at this stage anyway, because it would be very disruptive to existing clients and servers.

In response to the specific criticisms of http://www.ivoa.net/internal/IVOA/InterOpMay2011GWS/normand_malapert_lesidaner_uwsIVOA2011V2.pdf

* Section 2.1.3 HELD status - whilst this might appear to have little utility in current implementations, in future versions where there might be quotas or priorities in the UWS then HELD is a way of expressing within the UWS that the job is accepted in principle, but will not be run until some action (like freeing up some of the quota) is taken.

* Section 2.1.11 - I agree that the text is a little vague here, but the use cases are clear;
   * having a way to "read back" a parameter that  was supplied in line with the job creation post - i.e. the parameter element in the job description can be used to point to a standard place with a "by reference" URL (though it should be noted that any URL would do, as long as it offered storage that would last as long as the job object).
   * having a way to set an individual parameter value after the initial job was created.

  It is probably not made clear enough that the initial values of the parameters (and certainly the possible parameter names) are all established during the initial POST that creates the job and in most cases this is how the job should be driven - The ability to set an individual parameter after job creation is an additional capability that the UWS may offer - it should not offer the ability to create new parameters nor delete existing parameters - in this way a client that just creates the job with the initial POST does not "miss" out on setting a crucial parameter. We could make this clearer by removing the ability to set the individual parameter, as I believe that it was added as a "would be nice" feature without a strong use case. There is only one guaranteed way to set a parameter that all UWS services must implement - in the initial POST that creates the job.

   The general relationship between a UWS and its parameters is somewhat complex as it depends upon the job description language (JDL) - It will be an active topic of standardization in a standard that is layered on top of UWS to provide a model of a so called "parameterized application" which will be a specific JDL that will hopefully create a common language that will work in a wide variety of possible use cases (Though there could exist UWS implementations that might still need their own specialized JDL).

* Section 2.3.1 - the PHASE=RUN option was added so that a UWS could offer a "one hit" way of creating a running a job - this was a breaking of the REST purity that was requested during standardization - there was strong feeling that this should be allowable.

* Section 2.2.3.2 Deleting a Job - the joblist is returned to make the UWS interface nicer when driven purely by a browser, and again the POST was added to the pure REST DELETE to make it easier to drive from the browser.

* Section 2.2.3.2 & 2.2.3.3 Changing execution duration & destruction time - if a service choses not to implement these features, then the standard is clear that a value of 0 should be returned for the execution duration, but I agree it is not clear what should be returned for the destruction time - in the job schema the DestructionTime element is nillable, so that would be appropriate representation in the job XML - however for the value returned at the resource URL then I agree that there is no description of what should be returned in the case where the UWS never deletes a job - you could return a value far in the future.

* Section 2.2.3.5 Starting a Job - agreed the language should be made stronger

* Section 2.2.3.6 Aborting a Job  -the suggestion to use the URI /{jobs}/(job-id)/abort to abort a job does not seem very RESTful to me as abort is a verb not an object. I do not see that there is anything non-RESTful with the way that UWS handles changing the job state via the PHASE property. Incidentally POST is used in this manipulation (and others like the ExecutionDuration) and not PUT because by RESTful principles if you do a PUT followed by a GET you would expect to get the value that you PUT, but in many of these cases the UWS is allowed to respond with another value (in the case of ExecutionDuration) or may take some time to actually carry out the request (eg. Aborting the job).

* pagination of the job list - this was discussed, but it was left out to make implementation easier, and incidentally pagination is very non-REST-ful (requires state saving, which disables caching).

Then some responses (prefixed by ->) to issues in http://voparis-srv.obspm.fr/doc/uws-v2.0.txt

- Job cannot be deleted when phase != [COMPLETED|ABORTED|ERROR]  => need to be stopped before, status ?
-> I think that job can be deleted at any time - it is up to the UWS server side to clean up appropriately.

- Job cannot be started  because its current status is not pending, status ?
-> Although the current wording of the document does not make this clear enough in every case, the intention is that changing the PHASE of the job is a request by the client to the server, and the client sees whether it has been successful by examining the XML returned by the redirect to the URI /{jobs}/(job-id)/. The allowable transitions are shown by the state diagram within the document - I am undecided whether is is better for the server to simply ignore disallowed transitions of PHASE or to issue a an error status.

- Job cannot be cancelled because its current status is not [pending|queued|executing], status ?
-> see above.

- Cannot update a parameter after a pending phase, status ?
-> I can see that allowing update of a parameter whilst a job was executing could be a useful feature for some services - it depend on the JDL as to whether this might be allowable/useful. If updating a parameter is not allowed at some time in the job lifecycle then I think that a 403 [Forbidden] status should be returned

- Cannot create parameters after a pending phase, status ?
-> the text needs updating to say that creating a parameter at any stage other than the initial job creation POST is not allowed.

- Status when a resource is optional ?
-> I think that most resources are not optional - some value should be returned, but otherwise a 404.

I will stop here as I think I have covered most of the issues (and this email is getting long), and will update the UWS document on the volute site with some of these clarifications in the next few days

Paul.