UWS 1.0 suggestions

[Matthew Graham]

Hi Petr,

I think that there are a number of wider issues here that I hope  
others will pick up and run with.

Certainly there is a question of process in what to do when we get  
real feedback/implementation experience with one of our standards that  
clearly shows its limitations/restrictions - this is clearly a driver  
for new versions but it could be more. Do we really want standards  
becoming recommendations based on only two "reference implementations"  
that only potentially demonstrate interoperability? How would we  
define true "field implementations"?

Then there is one of scope. You say below "I think that In Astrogrid  
there were not implemented the features for direct human interaction -  
only for machine intaraction." By definition grid and web services are  
not intended for direct human interaction but for machine interaction.  
Middleware can always be built on top to make them more usable by  
humans. Given this UWS is maybe properly scoped to do what it is meant  
to do and your comments are more relevant for application/middleware  
layers that sit on top and are the provenance of the Applications WG.

We should also review how much emphasis we want to put on web browsers  
as general clients to our services and how much this should influence  
our specification designs.

Lastly my feeling is that UWS 1.0 should proceed as is but we should  
we factor your commends and feedback into a UWS 1.1.

	Cheers,

	Matthew



On Nov 21, 2009, at 6:46 PM, Petr Skoda wrote:

> On Sat, 21 Nov 2009, Matthew Graham wrote:
>
>> Hi Petr,
>>
>> Firstly could you please post these comments onto the RFC page for  
>> UWS.
>>
> Done
>
>
>> I am wary about extending not just the UWS RFC period but any RFC  
>> period just to enable one set of comments to be made. There is an  
>> agreed process involved here and we were public enough about  
>> announcing the start of the RFC.
>
> I am fully aware of this but many point and limitations started to  
> be clear only during the interop where I was discussing some issues  
> in more detail both with Paul and Dave and my colleague Fuchs tried  
> to implement the UWS in full glory to our KOREL-VO service.
>
> At the same time started to come the requests for polishing the web  
> interface from the author (scientist) of KOREL who happened to start  
> the teaching of disentangling in Chile (as an Visiting Professor)  
> just a week ago and started to use this web service (running on a  
> fast HW)  insted of working on his notebook.
>
> So his appreciation of our work was a real victory of a VO  
> technology over his (very strong conservatism) ;-) But his comments  
> for improvement are the real science cases coming from his  
> experience and student's comments.
>
> The most recent versions of KOREL-VO UWS service tried to address  
> these science issues (e.g. more detailed description of a project -  
> star name, wavelenght range ...) the extremely long list of jobs  
> that are completed - difficult to scroll just to see the runing ones  
> (you may mix one long and then rung 20 short trials and the question  
> is how to order the list properly... and we have faced the limits as  
> described earlier and posted on twiki.
>
>> people cannot be attentive to the specifics of the process then  
>> they have to live with the consequences.
>
> It is nice to tell this, but during the RFC period we have read n- 
> times the documentation and everything seemed to be crystal-clear  
> from the point of view of software developers ! (and my as an  
> astronomer but with just avarage idea how the software should work -  
> having tested it on several examples from disentangling summer school)
>
> But the experience of a scientist (and author of the primary tool)  
> is crucial. - its a real field-test by scientist.
>
>
>
>> One that I will take umbrage with, however, is your statement:
>
> I am very sorry of your umbrage - and I understand it.
> I have probably done a very big shortcut in  the explanation of the  
> suggestions - in fact addressed mainly to Astrogrid people.
>
> So I have to  explain this in a more detail. Once again sorry.:
>
>> that the UWS is just describing the solution implemented on Astrogrid
>
> In fact I started to be interested in all the UWS stuff several  
> weeks before the May interop when I felt the opportunity to solve  
> many of long-time problems with the current KOREL code - its user  
> unfriendliness, limitations in computing power, keeping different  
> versions, compiler issues - the misuses of source code etc .... by  
> the RESTful web service - and by chance I have had seen the TAP  
> appendix describing the UWS.
> (the first idea of korel presented at May interop was still in PHP  
> unRESTful). In June I have shown this prototype at the astronomical  
> conference and it was still in PHP, but I was already looking for  
> more info about implementation of REST services.
>
> In August my collegue had found the CherryPy and in fact we started  
> the work on a RESTful version, while we did not have idea about  
> details not clear in UWS0.4 - but we undestood this rather as an  
> idea than a real service. And suddenly the UWS1.0 was announced and  
> the RFC page appeared.
> And here an examples of real working implementations were given:
>
> But the Pat's one did not return anything and the only one available  
> for testing how it works was the Astrogrid CEA - with this we worked  
> and played to understand what it returns and checked with  
> documentation - so some ideas started to be clear.
> (nevertheless I have found some discrepancies wrt to both 0.4 and  
> 1.0 - and have posted it here just before interop)
>
> During the interop I have discussed with Dave and Paul the details  
> and their explanation was that the UWS servers of Astrogrid are only  
> called from VODesktop - that is the simplicity of a behaviour is  
> given by the UWS as a backend for specific application - ie it is  
> for deploying some prepared workflow. Dave told me that it was never  
> intended for direct web client interaction. So here is the reason of  
> my statement:
>
> <my_opinion>
>
> I think that In Astrogrid there were not implemented the features  
> for direct human interaction - only for machine intaraction. It can  
> work well for the backend service of e.g. TAP , but the responses  
> are quite sparse for human web interaction. If it should be really  
> Universal, I think that some more features (described in my comments  
> and probably some others would be desirable - and the real  
> scientists usage justifies this.
>
> </my_opinion>
>
>
> To summarize - I am fully aware of all the work of GWS group and I  
> admit I have been interested in some matters only for very short  
> time - but I could not find any other working example of UWS than  
> that in the CEA server.
>
> The statement "describes the solution implemented by Astrogrid"  
> could be read as well as that Astrogrid have implemented the  
> standard described in UWS, but the author of the document is Paul  
> (and he IS from Astrogrid) and I guess most of the UWS design ideas  
> come from CEA experience (e.g. the Trieste interop started this idea  
> as a RESTfull version of CEA - v0.3)
>
> I apologize for not knowing other UWS implementations.
>
>
>> lets think about it as a universal pattern for future VO services  
>> where part  of them might have the complicated workflows and both  
>> other automatic services and the users through web clients would  
>> like to control them."
>
> I feel that most of the problems with the popularity of VO tools  
> among the rather conservative astronomers is the strictly logical  
> design - very flexible and clear but done by software engineers and  
> implemented in a manner that they consider to be intuitive. But the  
> astronomers want to have things resembling something familiar for  
> them. So instead of new fancy client (sorry for telling this here -  
> now I wear my head of rather experienced astronomer and VO  
> evangelists ;-) like VODesktop they prefer the interaction with web  
> browser - but they want the web fancy as well !
> (The idea of the KOREL as a graphics tool was burried already many  
> years ago due to complicated support of different platforms, its  
> rewritting in Java is impossible because of FORTRAN backend and the  
> conservatism of its author so the only solution was to let the SW as  
> it is and do the key chenges on the level of interaction with the  
> user. And the web is an excellent place how to do it - as the  
> browser is the only tool that every astronomer is guarranted to know  
> and use.
>
>
>> UWS is not just describing the Astrogrid solution but is the result  
>> of several years hard work within the IVOA
>
> Sorry again - I had in mind the practical behaviour of what was  
> described in 1.0
>
>> free to be part of - to define a general design pattern for  
>> asynchronous services.
>
> I did not say anything against the pattern for services - I  
> commented just the concrete representations of objects and its  
> containers and suggested some extensions (as well as in joblist -  
> subqery for state of jobs)
>
>
> Cheers
> Petr
>
> *************************************************************************
> *  Petr Skoda                         Phone : +420-323-649201, ext.  
> 361 *
> *  Stellar Department                          
> +420-323-620361           *
> *  Astronomical Institute AS CR       Fax   :  
> +420-323-620250           *
> *  251 65 Ondrejov                    e-mail:  
> skoda at sunstel.asu.cas.cz  *
> *  Czech  
> Republic                                                       *
> *************************************************************************
>