UWS 1.0 suggestions

[Petr Skoda]

On Sat, 21 Nov 2009, Matthew Graham wrote:

> Hi Petr,
>
> Firstly could you please post these comments onto the RFC page for UWS.
>
Done


> I am wary about extending not just the UWS RFC period but any RFC period just 
> to enable one set of comments to be made. There is an agreed process involved 
> here and we were public enough about announcing the start of the RFC.

I am fully aware of this but many point and limitations started to be 
clear only during the interop where I was discussing some issues in more 
detail both with Paul and Dave and my colleague Fuchs tried to implement 
the UWS in full glory to our KOREL-VO service.

At the same time started to come the requests for polishing the web 
interface from the author (scientist) of KOREL who happened to start the 
teaching of disentangling in Chile (as an Visiting Professor) just a week 
ago and started to use this web service (running on a fast HW)  insted of 
working on his notebook.

So his appreciation of our work was a real victory of a VO technology over 
his (very strong conservatism) ;-) But his comments for improvement are the 
real science cases coming from his experience and student's comments.

The most recent versions of KOREL-VO UWS service tried to address these 
science issues (e.g. more detailed description of a project - star name, 
wavelenght range ...) the extremely long list of jobs that are completed - 
difficult to scroll just to see the runing ones (you may mix one long and 
then rung 20 short trials and the question is how to order the list 
properly... and we have faced the limits as described earlier and posted 
on twiki.

> people cannot be attentive to the specifics of the process then they have to 
> live with the consequences.

It is nice to tell this, but during the RFC period we have read n-times 
the documentation and everything seemed to be crystal-clear from the point 
of view of software developers ! (and my as an astronomer but with just 
avarage idea how the software should work - having tested it on several 
examples from disentangling summer school)

But the experience of a scientist (and author of the primary tool) is 
crucial. - its a real field-test by scientist.



> One that I will take umbrage with, however, is your statement:

I am very sorry of your umbrage - and I understand it.
I have probably done a very big shortcut in  the 
explanation of the suggestions - in fact addressed mainly to Astrogrid 
people.

So I have to  explain this in a more detail. Once again sorry.:

> that the UWS is just describing the solution implemented on Astrogrid

In fact I started to be interested in all the UWS stuff several weeks 
before the May interop when I felt the opportunity to solve many of 
long-time problems with the current KOREL code - its user unfriendliness, 
limitations in computing power, keeping different versions, compiler 
issues - the misuses of source code etc .... by the RESTful web service - 
and by chance I have had seen the TAP appendix describing the UWS.
(the first idea of korel presented at May interop was still in PHP 
unRESTful). In June I have shown this prototype at the astronomical 
conference and it was still in PHP, but I was already looking for more 
info about implementation of REST services.

In August my collegue had found the CherryPy and in fact we started the 
work on a RESTful version, while we did not have idea about details not 
clear in UWS0.4 - but we undestood this rather as an idea than a real 
service. And suddenly the UWS1.0 was announced and the RFC page appeared.
And here an examples of real working implementations were given:

But the Pat's one did not return anything and the only one available for 
testing how it works was the Astrogrid CEA - with this we worked and 
played to understand what it returns and checked with documentation - so 
some ideas started to be clear.
(nevertheless I have found some discrepancies wrt to both 0.4 and 1.0 - 
and have posted it here just before interop)

During the interop I have discussed with Dave and Paul the details and 
their explanation was that the UWS servers of Astrogrid are only called 
from VODesktop - that is the simplicity of a behaviour is given by the UWS 
as a backend for specific application - ie it is for deploying some 
prepared workflow. Dave told me that it was never intended for direct web client 
interaction. So here is the reason of my statement:

<my_opinion>

I think that In Astrogrid there were not implemented the features for 
direct human interaction - only for machine intaraction. It can work well 
for the backend service of e.g. TAP , but the responses are quite sparse 
for human web interaction. If it should be really Universal, I think that 
some more features (described in my comments and probably some others 
would be desirable - and the real scientists usage justifies this.

</my_opinion>


To summarize - I am fully aware of all the work of GWS group and I admit I 
have been interested in some matters only for very short time - but I 
could not find any other working example of UWS than that in the CEA 
server.

The statement "describes the solution implemented by Astrogrid" could be 
read as well as that Astrogrid have implemented the standard described in 
UWS, but the author of the document is Paul (and he IS from Astrogrid) and 
I guess most of the UWS design ideas come from CEA experience (e.g. the 
Trieste interop started this idea as a RESTfull version of CEA - v0.3)

I apologize for not knowing other UWS implementations.


> lets think about it as a universal pattern for future VO services where 
> part  of them might have the complicated workflows and both other 
> automatic services and the users through web clients would like to 
> control them."

I feel that most of the problems with the popularity of VO tools 
among the rather conservative astronomers is the strictly logical design - 
very flexible and clear but done by software engineers and implemented in 
a manner that they consider to be intuitive. But the astronomers want to 
have things resembling something familiar for them. So instead of new 
fancy client (sorry for telling this here - now I wear my head of rather 
experienced astronomer and VO evangelists ;-) like VODesktop they prefer 
the interaction with web browser - but they want the web fancy as well !
(The idea of the KOREL as a graphics tool was burried already many years 
ago due to complicated support of different platforms, its rewritting in 
Java is impossible because of FORTRAN backend and the conservatism of its 
author so the only solution was to let the SW as it is and do the key 
chenges on the level of interaction with the user. And the web is an 
excellent place how to do it - as the browser is the only tool that every 
astronomer is guarranted to know and use.


> UWS is not just describing the Astrogrid solution but is the result of 
> several years hard work within the IVOA

Sorry again - I had in mind the practical behaviour of what was described 
in 1.0

> free to be part of - to define a general design pattern for asynchronous 
> services.

I did not say anything against the pattern for services - I commented just 
the concrete representations of objects and its containers and suggested 
some extensions (as well as in joblist - subqery for state of jobs)


Cheers
Petr

*************************************************************************
*  Petr Skoda                         Phone : +420-323-649201, ext. 361 *
*  Stellar Department                         +420-323-620361           *
*  Astronomical Institute AS CR       Fax   : +420-323-620250           *
*  251 65 Ondrejov                    e-mail: skoda at sunstel.asu.cas.cz  *
*  Czech Republic                                                       *
*************************************************************************