UWS 1.0 suggestions

Tue Nov 24 04:22:25 PST 2009

Thanks Paul for your comments and explanations:

>
> you are free in the description of the job to put any metadata that you want 
> within the jobinfo element - this is where you can add extra information that 
> is not defined within UWS.

>> although there is a "owner" in UML 
>> schema and "ownerId" in the  JobSummary tag

> no the owner of the job MUST be the user identifier of the user that created 
> the job as obtained via IVOA standard authentication mechanisms - you should 
> not use any other type of

OK - I supposed this is not the right place for user name (full)

> you can put any xml that you want within the jobinfo element - it is already 
> defined allow xs:any content - it could not be more extensible.
Sorry I have overlooked this but my guess was correct - use the jobinfo . 
Thanks for confirmation

>> Then when the service returns jobs list - the element "jobs" - it is now 
>> said it returns "uws:ShortJobsDescription"  but is is in fact only
>> the "phase" plus JobIdentifier. Why not return the whole "JobSummary" and 
>> let the client to extract the information neccessary for displaing nice job 
>> roaster (e.g. using xslt style sheet) ?
>
> Mainly to limit the size of the data returned for the list of jobs

I guess that adding little more info like short comment and perhaps 
several other words would not extend the job list too much, so the return 
of almost all contents of  JobSummary (with exception of Parameters 
,Results and Error) might not be so 
dramaticaly overloading the client but gave him a possibility to extract 
information for display of relevant information according to the 
particular requirements.

So I would suggest for future consideration to extend the 
<ShortJobDescription> to include not only Phase and id but as well the 
start End Execution Duration, Destruction, Quote and the full jobinfo
as well as the link to results.

It is question about the parameters as well what to prepare for them 
similar separate reference like for "results" - and there would be 
separate (e.g. web page) listing the links to input parameters (if they 
are files - eg images)  so root_URI/jobs/job-ID/params/

would be reference to list of parameters and

root_URI/jobs/job-ID/params/param1  the value of parametr

This would allow simple rerun of jobs from storage area by changing this 
param list that would be the only argument of job to run ....

Of course this have to be debated after more experience with 
implementations is get...

> it was decided to avoid the need for paging mechansims to retain simplicity - 
> by having the relatively small amount of data for each job in the job list 
> (see last answer) it means that there is of order 100 bytes per job in the 
> list - assuming that ~1MByte is a reasonable maximum download size for the 
> job list, it means that there can be 10,000 jobs in the list before the 
> listing time starts to become intolerable.

I understand but consider the possibility said above ...
>
>
>> Here we see the solution to let the client ask either for list of active 
>> jobs (e.g.   root-URI/jobs?phase=executing ) or all (default root-URI/jobs) 
>> or e.g those already completed etc ...
>
> I think that this might be a good idea for a future revision of UWS.

It could be naturaly the paging implemented by this
e.g. root_URI/jobs?page=2  or even the complexity of the list 
(jobs?info=full ) etc ....
BTW (As it starts to look non-RESTful, I will just add that it is 
still the REST way according to various Internet sources - the REST does 
not require only slashes but question mark and parameters are allowed or 
even recommended).

>> > It is true that it is difficult to establish a absolute comparative meaning 
> for quote in different circumstances, but if you implement your 
> different queues as a set of UWS based services that effectively 
> operated different priority queues as you suggest you can indicated the 
> behaviour of each one by the relative values of the Quote and the 
> Execution Duration. If for a particular set of job parameters you 
> determine that a job will be costly you can set the quote to be greater 
> than the execution duration on the "queue" (= UWS endpoint) for the 
> "high priority" queues, which would indicate that the job would not be 
> run for that endpoint. The client might try to run the job anyway, but 
> in this case the server would simply fail the job immediately.

This is clever idea in the current concept - thanks for it

>
> I think that trying to introduce the concept of queue priorities into a later 
> version of UWS should be considered, but not for UWS 1.0 as it would take too 
> long to fully establish the semantics.

OK thanks for taking this into consideration

>> 
>> 
> There is nothing stopping you from implementing your UWS based service in a 
> way that allows rather long destruction times to allow for a form of "long 
> term" storage of the results.

OK - its clear

You could even operate the quota system that 
> you describe with current UWS - all that is missing from the current standard 
> is some form of semantics for expressing in a uniform way why new jobs are 
> not being allowed to be created so that the client understands that they must 
> destroy some jobs to be able to run new ones.

I agree

Again, I would not want to 
> delay v1.0 to try to sort these issues out, but I think that it would be 
> possible to perhaps add the concept of quotas to a future version of UWS.
>
OK - I would like to participate in the further versions

>> 
>> As the jobs is in fact described by its set of parameters it might be 
>> possible to use the whole "job" element to change particular parametrs and 
>> resubmit
>
> I am not sure that I see exactly how the UWS standard has any bearing on 
> being able to perform an operation such as described above.

Probably the concept of input parameter list (as shown above) have to be 
devised - so the job would have only one parametr (and this would be 
reference to list of all real paramteres, files etc - some complex xml 
structure... I agree it'll be little bit complicated, however..

> is no doubt that the design is biased towards being driven by simple clients 
> rather than web browsers directly - however, I think that we have shown that 
> there are techniques by which a html interface can be created by the same 
> machinery that creates the UWS responses without compromising the standard 
> UWS XML responses.
yes the xslt transoformation is nice idea and if the clien does not 
understand the server may prepare the html for it directly. Really clever.

Indeed my interpretation of section 2.2.2 is that if the 
> client explicitly includes in a request an Accept header including 
> "text/html" then the UWS service may return pure html rather than XML, but 
> the UWS service must return XML or "text/plain" in the cases where HTML has 
> not been explicitly requested.

>
> I have said several times that I think that there are probably some useful 
> concepts that could be added to a future version of the UWS standard (i.e 
> queues and quotas). I would encourage current implementers to try creating 
> services that do include these concepts (to prototype what is necessary) 
> whilst at the same time conforming to UWS 1.0. I believe that it is possible 
> to do this, with the only problem from the client's point of view being that 
> sometimes it does not "understand" why job creation has been refused or that 
> a short destruction time has been allocated.

OK we will try to be compliant at the same time but prototyping the 
requests of scientists as well - (so experimentaly changing the xsd1.0) 
and see what happens ...

Petr

*************************************************************************
*  Petr Skoda                         Phone : +420-323-649201, ext. 361 *
*  Stellar Department                         +420-323-620361           *
*  Astronomical Institute AS CR       Fax   : +420-323-620250           *
*  251 65 Ondrejov                    e-mail: skoda at sunstel.asu.cas.cz  *
*  Czech Republic                                                       *
*************************************************************************