vos:group vs. uws:JobInfo

[Matthew Graham]

Hi Paul,

Thanks for the swift response - this certainly helps the discussion. 

Is it your recommendation that any service which uses UWS specifically states what its JDL is? Is there a suitable format for this for validation, etc? In VOSpace 2.0, I have assumed that the <uws:parameters> define the job since there is no initial POST - what is posted is the full XML representation of the Job resource, e.g.:

<uws:job xsi:schemaLocation="http://www.ivoa.net/xml/UWS/v1.0 UWS.xsd "
    xmlns:uws="http://www.ivoa.net/xml/UWS/v1.0"
  xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <uws:jobId/>
  <uws:ownerId xsi:nil="true"/>
  <uws:phase>RUN<uws:phase/>
  <uws:startTime xsi:nil="true"/>
  <uws:endTime xsi:nil="true"/>
  <uws:executionDuration>0</executionDuration>
  <uws:destruction xsi:nil="true"/>
  <uws:parameters>
    <uws:parameter id="target">vos://nvo.caltech!vospace/mydata1</uws:parameter>
    <uws:parameter id="direction">vos://nvo.caltech!vospace/mydata2</uws:parameter>
    <uws:parameter id="keepBytes">false</uws:parameter>
  </uws:parameters>
  <uws:results/>
</uws:job>
or have I misread UWS?
	Cheers,
	Matthew




On Aug 13, 2010, at 6:20 AM, Paul Harrison wrote:

> Hi,
> 
> On first reading of your requirements wrt VOSpace, I am not sure that I entirely understand the context in which you want to use these <uws:parameter> elements. Remember that the elements that occur within <uws:parameters> are purely informational, with the definition of the job input being dependent on (in UWS jargon) the "Job Description Language (JDL)" that is used in the initial POST.  The easiest interpretation comes where the JDL is simply a set of application/x-www-form-urlencoded (name, value) pairs such as what is produced by a HTML form submission, but this model is fairly restrictive when it comes to defining relations between the parameters as you are wanting to do here. I think that the only viable solution with this simple model is to introduce parameter naming rules that indicate the relationships - but this sort of thing quickly reaches its limitations. The more general idea in UWS is to have a more expressive JDL such as can be achieved with XML (though other encodings/ domain specific languages are equally valid), and in this case there might not be a simple mapping into the individual "parameters" within the JDL. As far as I can see one of the main uses of the <uws:parameters> section of the Job information is to be able to re-run the job, and in this case the recommendation of the UWS specification is to list a single parameter with the full text of the JDL. However, despite the listing of the parameters in the Job Information, a generic UWS client cannot in general resubmit the job armed with this information, as the interpretation of the parameter is dependent on the JDL, which is not specified by UWS.
> 
> On your specific queries see comments between the text below...
> 
> 
> On 2010-08 -12, at 19:06, Matthew Graham wrote:
> 
>> Hi,
>> 
>> I posted this to the VOSpace mailing list (vospace at ivoa.net) after the last Interop and have gotten no responses so far so let's try the main GWS list.
>> 
>> 	Cheers,
>> 
>> 	Matthew
>> 
>> Begin forwarded message:
>> 
>>> From: Matthew Graham <mjg at cacr.caltech.edu>
>>> Date: May 24, 2010 10:28:04 PM PDT
>>> To: vospace at ivoa.net
>>> Subject: vos:group vs. uws:JobInfo
>>> 
>>> Hi,
>>> 
>>> As mentioned last week, this remains the outstanding issue with VOSpace 2.0.
>>> 
>>> When multiple protocols are specified, we may need to associate a particular protocolEndpoint parameter with its respective protocol parameter (pushFromVoSpace) but in UWS these are all uws:parameters:
>>> 
>>> <uws:parameters>
>>>  <uws:parameter id="protocol">ivo://ivoa.net/vospace/core#httpput</uws:parameter>
>>>  <uws:parameter id="protocolEndpoint">http://some.server.com/put/the/data/here</uws:parameter>
>>>  <uws:parameter id="protocol">ivo://ivoa.net/vospace/core#ftpput</uws:parameter>
>>>  <uws:parameter id="protocolEndpoint">ftp://some.other.server.com/put/the/data/here</uws:parameter>
>>> </uws:parameters>
>>> 
>>> To handle this, we have added a parameter vos:group to the UWS Job representation so that groups of parameters can be tagged. The attribute should appear on all parameters and have the same value for a particular group, in this case, a particular protocol - protocolEndpoint pairing:
>>> 
>>> <uws:parameters>
>>>  <uws:parameter id="protocol" vos:group="protocol1">ivo://ivoa.net/vospace/core#httpput</uws:parameter>
>>>  <uws:parameter id="protocolEndpoint" vos:group="protocol1">http://some.server.com/put/the/data/here</uws:parameter>
>>>  <uws:parameter id="protocol" vos:group="protocol2">ivo://ivoa.net/vospace/core#ftpput</uws:parameter>
>>>  <uws:parameter id="protocolEndpoint" vos:group="protocol2">ftp://some.other.server.com/put/the/data/here</uws:parameter>
>>> </uws:parameters>
> 
> I am not very keen on this mainly because it means that the UWS response is not schema valid according to the published schema -Whilst experience has shown us it is probably good practice to write clients so that they only interpret the XML that they understand and be fairly lenient about extra attributes etc.,  many people use code generators which are quite rigid.
> 
>>> 
>>> The alternate solution is that the protocol information for a transfer job does not appear in uws:parameters but in uws:jobInfo which can take arbitrary XML (as the spec says: The <job> element has placeholders of all of the standard UWS objects, and in addition there is a <uws:jobinfo> element which can be used by implementations to include any extra information within the job description.):
>>> 
>>> <uws:job>
>>>  ...
>>>  <uws:parameters>
>>>     ...
>>>  </uws:parameters>
>>>  ...
>>>  <uws:jobinfo>
>>>    <vos:protocol id="ivo://ivoa.net/vospace/core#httppput>
>>>      <vos:endpoint>http://some.server.com/put/the/data/here</vos:endpoint>
>>>    <vos:protocol>
>>>    <vos:protocol id="ivo://ivoa.net/vospace/core#ftpput">
>>>      <vos:endpoint>ftp://some.other.server.com/put/the/data/here</vos:endpoint>
>>>    <vos:protocol>
>>> </uws:job>
>>> 
>>> My only concern with this is that my reading of the UWS spec was that uws:jobInfo was something that the implementation used to provide extra information after a job has been specified and that it was not for additional UWS job specification material. All the information for a job should be in the uws:parameters section.
>>> 
> 
> I think that this is the better option of the two, and as said above the <uws:parameters>  is just as informational as  <uws:jobinfo> as far as I am concerned.
> 
> 
> Regards,
> 	Paul.
> 
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.ivoa.net/pipermail/grid/attachments/20100813/732e2a77/attachment-0001.html>