UWS POST PHASE=RUN

[Grégory Mantelet]

Hi,

 From all what have been said until now, I see the 3 following methods 
to submit parameters:

     1/ "application/x-www-form-urlencoded"
Only parameter/value pairs can be submitted with this method. However, 
these parameters can be the parameters needed to the job (JDL), the UWS 
parameters (execution duration, destruction time, PHASE=RUN, ...) or both.

     2/ "multipart/form-data"
Here, using DALI UPLOAD (like in TAP), parameters can be JDL, UWS 
parameters or both, as in 1/. So, the JDL can be submitted as 
parameter/value pairs or as one file (e.g. XML document). But it is also 
possible to submit other files which could be necessary to the job (for 
instance, if it is a job doing some processing over one or several files 
; those files could be provided inline or as URL).

     3/ other
Only the JDL can be provided. It is a blob, XML or other kind of 
document which is completely opaque and could not be interpreted by the 
UWS server. This document must be stored as provided and with no 
interpretation of the UWS server. It implies that UWS parameters 
(duration, destruction, ...) can not be provided in the initial POST and 
if the user want to modify their default value, it must do it after 
creation and before execution with one or several POST in 
"application/x-www-form-urlencoded".


Like Markus, I also think that 3/ could be a problem for large files and 
that's why 2/ is better. Then, allowing the 2 methods with the rule "if 
the file (JDL) is large, we use DALI UPLOAD", is for me source of 
problem for server, because giving a such choice to the user does 
generally not work. We can indeed see it in TAP with the choice of the 
synchronous and asynchronous execution. Generally users do not know the 
difference between the two modes or have not enough knowledge of DB 
query execution to decide which method should be used, and consequently, 
they generally stick to one method (which is often synchronous because 
the result is returned "immediately"). So, they have the choice, but are 
generally unable to decide which should be used...and that, is only when 
they know they have the choice and what is the rule to use.

So, I think that only the two first methods must be proposed, even if I 
understand the third one. The two first methods are simple, well-known 
and let cover all possible cases, whereas 3/ allows just one opaque file 
- eventually large - to be submitted and without any possibility to 
provide UWS parameters in the initial POST.

Cheers,
Grégory


On 04.11.2014 09:37, Markus Demleitner wrote:
> Hi,
>
> Let me extract these two parts from the conversation:
>
> On Mon, Nov 03, 2014 at 11:33:25PM -0800, Matthew Graham wrote:
>> The obvious exampled of a document-based JDL is VOSpace where data
>> transfers are coordinated via UWS. The transfer resource that is
>> posted to initiate a job is hierarchical and so could be difficult
>> to map to a parameterized form. It gets stored under the jobInfo
>> element for the UWS representation of the job.
>>
>> On Nov 3, 2014, at 11:22 PM, Paul Harrison wrote:
>>
>>> necessarily representable even under the <jobinfo> element -and I
>>> agree that there was no intent that the generic JDL and DAL
>>> ?parameter/value? mechanism could be mixed - which is why I was
>>> suggesting that there be some conventions written into the
>>> standard to make this separation clearer and make the language of
>>> the standard stronger in section 2.1.11 - so that the suggestions
>>> become ?should?s rather than ?best practice?.
> As to Matthew's point, any single file can be represented in the
> trivial parametrised from JOB=<blob>; UWS wouldn't know about any
> internal structure then, but that's probably what's intended in these
> cases anyway.
>
> So the big trick is to represent <blob>, and I'd really like to
> stress Paul's point here.  *If* we want to have these kinds of opaque
> parameters in the UWS job description, that has severe consequences
> in that we must say how non-XML would be represented (or outlaw it).
> Also, what about largeish uploads?  Do we really want job documents
> that are a couple of megabytes?  Gigabytes?
>
> All this is to say: We have DALI UPLOAD, with its built-in option of
> having both URL and inline uploads.  Something fairly akin to it
> already works ok for TAP.  It solves the problem of content not
> easily representable in XML, its representation always is of very
> reasonable size (an upload name and a URL), and there's no
> UWS-special tricks necessary.
>
> Hence: I propose to deprecate POSTing anything but parameters encoded
> browser-style (either form-encoded or as a multipart) and say opaque
> JCL should come in a single parameter (I like recommending the name
> JCL for it) if it's printable and short or be UPLOADed if it's not.
>
> Cheers,
>
>          Markus
>
>
>