UWS 1.0 suggestions

[Matthew Graham]

Hi Petr,

Firstly could you please post these comments onto the RFC page for UWS.

I am wary about extending not just the UWS RFC period but any RFC  
period just to enable one set of comments to be made. There is an  
agreed process involved here and we were public enough about  
announcing the start of the RFC. If people cannot be attentive to the  
specifics of the process then they have to live with the consequences.  
That having been said, I am sure that Paul and Guy will address your  
comments.

One that I will take umbrage with, however, is your statement: "We  
understand that the UWS is just describing the solution implemented on  
Astrogrid - but lets thik about it as a universal pattern for future  
VO services where part of them might have the complicated workflows  
and both other automatic services and the users through web clients  
would like to control them."

UWS is not just describing the Astrogrid solution but is the result of  
several years hard work within the IVOA - a process that you have  
always been free to be part of - to define a general design pattern  
for asynchronous services. It has been much discussed within both GWS  
and DAL WGs, particularly with regard to TAP.

	Cheers,

	Matthew



On Nov 21, 2009, at 11:32 AM, Petr Skoda wrote:

>
> Hi all,
>
> I hope that the UWS RFC will be extended still to have time make  
> clear some of the not-so-obvious issues. During the implementation  
> phase we may still get some ideas and improvements worth of its  
> standardization.
>
> Having look at the http://www.ivoa.net/Documents/UWS/20090909/PR-UWS-1.0-20090909.html
>
> I have found errors in section 2.2.2.2 and appendix B
> in both cases there is typo "xlmns" instead of "xmlns"
> The xsd on the http://www.ivoa.net/xml/UWS/v1.0 is syntactically OK.
>
> Now about some suggestions:
>
> We try to use the UWS pattern for allowing the people to do specific  
> computations using their datasets driven by some parametric files.  
> So most of the extensions given below are specific cases from our  
> experience - we wish to have the WWW browser as an only user  
> interface to work with such a system (uploading jobs, runing and  
> braking them, reading results, reruning with different parametrs,  
> deleting, zipping, downloading the results, sending the mails with  
> results after disconnecting from the service totally - we suppose  
> the deployment in a distributed environment - using the remote  
> queues submission (e.g. grid engines like qsub...)
>
>
> To facilitate the comfortable interaction between UWS service and  
> web browser (like a client front-end) it would be nice to describe  
> the job by more detailed tags that the client could use to present  
> either in the single job description (root-URI/jobs/jobid) and in  
> jobs list (root-URI/jobs).
>
> The example of this is a name of user (although there is a "owner"  
> in UML schema and "ownerId" in the  JobSummary tag - it may be e.g.  
> full name of the user) or a additional description of a particular  
> job (e.g. in computation of stellar models, the name of a star and  
> additional description like method of computation and some remarks ...
>
> This simply the user enters into the web- based job creation form  
> and might be nice to keep this for housekeeping purposess e.g. on  
> the list of completed jobs as well as on detaild jobid info.
>
> So the question is - how to extend the tags to allow some additional  
> information to embed with job (e,g, make the jobinfo an complexType ?)
>
> Then when the service returns jobs list - the element "jobs" - it is  
> now said it returns "uws:ShortJobsDescription"  but is is in fact only
> the "phase" plus JobIdentifier. Why not return the whole  
> "JobSummary" and let the client to extract the information  
> neccessary for displaing nice job roaster (e.g. using xslt style  
> sheet) ?
>
> And still there seems the issue of paging of long list of jobs...
>
> Here we see the solution to let the client ask either for list of  
> active jobs (e.g.   root-URI/jobs?phase=executing ) or all (default  
> root-URI/jobs)  or e.g those already completed etc ...
>
> Simply it may be requested by the client to make the selection for  
> different purposes and so the server should do it and return what he  
> wants.
>
> We can foresee the  the usefullnes of such operation even without  
> the need for web client for e.g. telling the UWS servers by soem  
> administration application to list a jobs of particular user etc ...
> In principle the joblist ia quite complex compject and so some fine- 
> grained information may be required to extract from it directly.
>
>
> Another issue is the meaning of "quote"
> Instead of using crystal ball to estimate when the job is expected  
> to finish, it would be better to have some idea of allowed priority  
> on different servers - like e.g. the nice value for different queues.
> So if checking among servers i would use that with higher priority  
> queue.
>
> Bat again it may be on the client decision to modify this behaviour.  
> E.g. the higher priority queues might impose shorter Execution  
> Duration.
>
>
> And the concept of Descruction Time is unclear in the document as  
> well.
>
> We think that this should be the time when the results will be  
> removed from the storage space - but suppose you want to use the UWS  
> server as your "external notebook" of your work - e.g. in the  
> concept of PDA- supercomputing.
>
> So many of your experiments are stored on the server and you decide  
> to rerun the particular one with different paramer sets but  
> basically same data sets(e.g. spectra, images ...) So its up to you  
> what experiment will be removed (if you go over quota you are not  
> allowed the create new jobs).
>
> However after some warning time the jobs will be removed anyway  
> (maybe you should receive some warning by the scheduler first to  
> allow you to copy the data).
>
>
> Last issue concerns the possibility of restarting jobs with same  
> datasets but different control parameters. So the client (in a web  
> browser) might have the button for resubmitting the same computation  
> with different number of iterations (change of some control  
> parametrs that may be re-edited in a browser)
>
> As the jobs is in fact described by its set of parameters it might  
> be possible to use the whole "job" element to change particular  
> parametrs and resubmit
>
>
> We understand that the UWS is just describing the solution  
> implemented on Astrogrid - but lets thik about it as a universal  
> pattern for future VO services where part of them might have the  
> complicated workflows and both other automatic services and the  
> users through web clients would like to control them. Here the  
> suggestions given above might make this interaction easier and  
> better controllable.
>
> Petr Skoda and Jan Fuchs
>
> *************************************************************************
> *  Petr Skoda                         Phone : +420-323-649201, ext.  
> 361 *
> *  Stellar Department                          
> +420-323-620361           *
> *  Astronomical Institute AS CR       Fax   :  
> +420-323-620250           *
> *  251 65 Ondrejov                    e-mail:  
> skoda at sunstel.asu.cas.cz  *
> *  Czech  
> Republic                                                       *
> *************************************************************************
>