UWS POST PHASE=RUN

[Paul Harrison]

Hi,

I think that it might be worth a recounting of UWS history to perhaps understand where “parameters” fit into UWS. The original intention was that UWS was a general framework that would work for any “Job description language” (in the words of the standard) - Meaning that particular UWS instances could chose to interpret the initial POST content in any way that they wanted. It was, however, clear that the there would be a large number of DAL services that would want to use simple parameter/value pairs, so some support for this pattern was also added to UWS (whilst at the same time trying to be consistent with servers that did not want to use the simple parameter/value pattern)

Having said the above, I have some more specific responses to your questions below…

On 2014-10 -30, at 16:23, Grégory Mantelet <gmantele at ari.uni-heidelberg.de> wrote:

> Dear UWS authors, users and implementors,
> 
>     Currently working on the new version of the UWS library, I would like to be sure that the way I get/read parameters in my library is correct and if not, jump on the occasion of this new release to make the appropriate corrections.
> 
>     Indeed, according to the UWS Recommendation 1.0 and the UWS draft for the version 1.1, parameters can be provided in several ways to a UWS service. These ways are for me, because of the possible desire to make them free, still partially unclear. So I will try in the following lines to summarise them as I have understood (and I apologise in advance for any eventual stupid question, but I need to be sure on the answers). I encourage anybody - and particularly UWS authors - to tell me where I am wrong, how it should really work and if I have forgotten something.
> 
> A. When?
> 
> It is possible to create parameters for a job only at its creation.
> After creation, they may be modified, but never created, only if the PHASE "is not set the executing state".
> 
> My questions:
> 
>     * Is a parameter modification possible if the phase is QUEUED? Since it is not an executing state, I would say that it is still possible, isn't it?
> 
>     * I know it is stupid, but what about the phases COMPLETED, ERROR and ABORTED? I am sure it should not be possible, but they are neither an executing state...so theoretically and according to the standard document, it would be possible to change parameters in those phases.
> 

The intention was that the parameters could be changed at any time up until the job was in the executing state (so a  QUEUED job can have its parameter values changed, but a job in any of the “terminal” states cannot)  - I will try to make this more explicit in the 1.1 draft.

> B. How?
> 
>     1. At the job creation
> 
> The only thing defined by the UWS standard is that parameters must be submitted in an HTTP POST request. No content-type is defined, which would be then totally free.
> 
>     2. After creation, but before the executing state
> 
> Here, no specific way or format is defined, but a good practice is recommended. It lets update parameters 1 by 1, by providing a       name-value pair with one of the following methods:
>     - HTTP POST with application/x-www-form-urlencoded to /{jobs}/{job-id} or /{jobs}/{job-id}/parameters
>     - HTTP PUT with just the value to /{jobs}/{job-id}/parameters/{parameter-name}
> 
> 
> For 1. and 2., it is said that it is possible to POST an XML document to create or update parameters. The structure/schema of this document is let free, as the content-type to use to submit it to the UWS service.
> 
> C. XML representation
> 
> A textual parameter must be represented like that in /{jobs}/{job-id}:
>     <parameter id="{param-name}">{param-value}</parameter>
> 
> A non-textual parameter or at least a parameter value that can not be written legally in an XML document must be represented:
>     <parameter id="{param-name}" byReference="true">{URL toward the param-value}</parameter>
> 
> 
> My questions about B. and C.:

This is the area where the distinctions between the POST a BLOB approach and the parameter/value pair approach do get a bit blurry - I think that the intention of generic server/client components should be to adopt a “least surprise” approach and do as little interpretation of the contents as possible to produce a reasonable API. There probably do need to be some more conventions about what should be done in various circumstances though.
> 
>     * When an XML document is submitted as single parameter, how must it be stored in the UWS service: as one parameter (with byReference=true) or as some individual parameters which has been extracted from the XML document?

I think that it should be stored as a single “parameter” - there should be no attempt by the generic UWS to interpret the document.

I think that it would probably be a good convention that if the initial POST has a application/x-www-form-urlencoded encoding, then the UWS can assume that the parameter/value pair mechanism can be used.
> 
>     * In the case where a submitted XML document must be stored as one parameter, what must be its name/id (the id which must be written in the "id" attribute of the XML node "parameter")? Since it has been submitted alone, I presume that no name has been given, and so that one must be invented and associated to it by the UWS service. That method could work because it is not possible to create parameters after the job creation. Is it correct?

It would probably be a good idea to give a conventional name to this parameter - e.g. JDL, and the UWS 1.1 document should specify what the conventional name should be.

> 
>     * Should it be possible to mix the usage of content-types? For instance, creating a job in POST multipart and updating parameters afterward with individual POST in application/x-www-form-urlencoded.

I am not as certain I have a definitive answer for this - I know that it is what TAP does - I think that we should probably adopt some simplifying conventions based on what is already done for TAP - however it seems to me that if POST multipart has been used in the first place then for those parameters in particular probably have got complex structure that should not be interpreted by generic UWS clients.
	
> 
>     * If the way to create or update parameters is such free that it allows even POSTing XML documents with no defined schema or structure to set them, why just XML documents are allowed? I am not at all in favor of having other kinds of document to set parameters, but I just wonder why allowing just XML documents while not defining its schema...I do not see the interest of fixing the MIME type without providing any clue to parse its content and use it smartly in the UWS service.

I do not think that the UWS standard does say that only XML documents are allowed for updating parameters - it might be the case that a single parameter is an XML document, but so far at least it has UWS has not proposed that multiple parameters can be updated with a single XML document. The section on updating the parameters should probably be clarified further, but I think that the basic rule should still be that if the POST encoding is application/x-www-form-urlencoded then the UWS can interpret it as parameter/value pairs - otherwise it should be interpreted as a BLOB (i.e. just stored as is).


> 
> D. PHASE=RUN
> 
> This parameter name-value pair must be POSTed to /{jobs}/{job-id}/phase in order to start the specified job (going first through the QUEUED phase).
> It may be possible to post it also at the initial POST in order to start the job immediately after its creation.
> 
> My question:
> 
>     * As has said Matthew, there is no content-type defined for this POST. So should there be one? I personally think that this special parameter, must always be submitted using application/x-www-form-urlencoded, when submitted after the job creation.
> 

I agree with this as the UWS Job control “parameters” are separate from the parameters that might make up the business logic of what the job does.


I think that you have brought up some important points here, and I would be interested to hear other opinions on what should be done.

Regards,
	Paul.