TAP Implementation issues (cont'd): UWS

[Guy Rixon]

On 30 Oct 2009, at 13:45, Tom McGlynn wrote:

> I've had a few questions with the implementation of the asynchronous  
> access for TAP.  Most of these are relevant to UWS document  
> generally rather than just TAP so I've copied the GWS group in this  
> mail.
>
> 	Tom McGlynn
>
> -- UWS general questions --
>
> - As defined a user needs to always do two web actions to start the  
> service.  Is there some reason that the user cannot simply request  
> the service to start running immediately?  I suspect that that is  
> what the user wants to do in 99% of the cases.  It would be much  
> easier for clients too.  The example given in the UWS document of  
> starting a job omits the error checking that the a user presumably  
> should do after starting the job.  Why not allow
>     {root}/tap/async?request=doQuery&query=...&phase=RUN
> to both create and start a query?  [Describe it as a POST if you  
> prefer.]

That would work, but would have to be POSTed.


>
> - I'm continue to be confused by the benefits conferred by various  
> practices.  Why do we require POSTs specifically in a number of  
> intances?  E.g., what would be wrong with using
>    ../jobid/phase?phase=RUN
> as an HTTP GET rather than an HTTP POST.  Since my code cannot tell  
> the difference between these I certainly will be supporting both,  
> but, other than bowing to the mantra REST, I'm not sure why it's  
> supposed to matter.

Whether or not a web service follows REST principles, it has to  
distinguish between requests that change the service state and  
requests that are idempotent. This is a basic part of HTTP. Starting a  
job changes the service state by creating web resources for that job.  
Sending the same query twice gets you two jobs doing the same query;  
they have separate web-resources. Therefore, not idempotent; therefore  
a POSTed request.

GET responses can be cached, and the caching is out of your control as  
a service provider - it may be on the user's LAN (HTTP proxy) or in  
their client (browser cache). If you send the same query twice then  
via GET, for the second request you could get the response for the  
first, pulled from the cache, and no new job. This doesn't happen too  
often but when it does it's brain-bendingly harder to debug.

I suggest that your code must not accept UWS create-job requests via  
HTTP GET. Your users won't like it if they get given the wrong answer  
from a cache. And Google tend to spider all the GETable URLs so you  
don't want them creating jobs.



>
> - Similarly I don't think that there should be a strict limitation  
> of the coding used in sending requests.  There may be a requirement  
> that a given encoding be supported but there should not be a  
> requirement that it be used.  As with POST/GET this level of HTTP  
> detail is handled below the UWS logic in our implementation, so for  
> services that the HEASARC supports we'll be allowing multipart/form  
> encoding as well unless someone can tell us why we should reject  
> such requests.

You're free to support this as well as the stated encoding because it  
doesn't break anything. However, if you write a client that assumes  
this encoding then it won't work on all implementations. So it seems  
pointless to add the feature even in the service.  Personally, I think  
that supporting broken clients in this way is not helpful.

>
> - From what states is a user allowed to start a job?  E.g., can a  
> user attempt to restart a job that has previously had an error or  
> aborted? Could the user change the parameters and then rerun the  
> same job?  I'm guessing this isn't supposed to happen, but I didn't  
> see where it was forbidden.

You can only change things while it's pending. If you need to re-run a  
failed job then you have to resubmit it.


>
> - What is supposed to happen if there is a problem in creating the  
> job.
> Should a job be created with an immediate status of ERROR?  Is there  
> any way of flagging an error if the system cannot create even an  
> error job?  E.g., we're going to use the database to store all job  
> information. What are we supposed to do if the database is down?  It  
> would be nice to be able to inform the user of an error in a  
> standard way.

If you can create the job at all then you should immediately set the  
phase to ERROR and make the error document available. If you can't do  
this, then I guess giving up with a 500 "I'm completely stuffed" error  
is reasonable. By extension, UWS clients need to deal minimally with  
500 errors as well as with proper error-documents.


>
> - Is there supposed to be a one-one mapping between there being an  
> error resource and the job failing with error or aborted.  I.e., if  
> I have a job with phase COMPLETED can it still have an error  
> document?  Can I have phase ABORTED without an error document?

Not formally defined. I don't see why you can't have an "error"  
document saying "OK, no problem" for status COMPLETED.



>
> - Is is allowed to have an empty error document?  E.g., can I have a  
> URL
>  ".../jobid/error"
> which exists but has no content (either as an actually zero length  
> string, or as an empty VOTable)?

In UWS, yes, because the form of the document is not fixed. In TAP, no  
you may not do that. It has to be a VOTable with the status filled in.


>
>
>
> -- TAP specific questions. --
>
> - The description of where to get the TAP result in an async request  
> is not given (as far as I can see) in what is described as the  
> normative parts of the document.  There it says that result will be in
>   root/async/jobid/results/
> but this is the list of results and can, in principle, contain a  
> number of results. Only in section 5.2 which is described as  
> informative does it say that the result document is .../results/ 
> result.  Is this actually  a requirement or can the result be named  
> anything?

In my service implementation, I take it to be a requirement. In my  
client implementation it currently assumes the one result with the  
standard name but I plan to make it parse the list. (In case we add to  
the results list in future TAP versions.)


>
> - The UWS standard discusses the naming of results.  Does TAP  
> require a specific name for the result?  In fact it looks like the  
> way UWS is supposed to be used the jobid/results returns a document  
> that looks like
>
>    <results>
>       <result id='someid' xlink:href='someurl' />
>       <result id='anotherid' xlink:href='anotherurl' />
>    </results>
>
> and the user is supposed to find the id of the desired result and  
> use whatever URL is given there, not use a specifically defined  
> URL.  I'm guessing the the ID attributes of the <result> fields is  
> the UWS name. The UWS standard says
>  "When a protocol specifies standard results it must do so by
>   naming those results; the names appear in the Results list in
>   addition to the URI's.  Not all results need to be named, sometimes
>   the meaning of the result is obvious from the context and the
>   name is omitted."
> Since the second sentence here seems to contradict the first it's a  
> bit hard to follow, but my reading of this is that it would be  
> better for TAP to specify a name for the output result rather than a  
> specific URL.

For a given service-protocol incorporating UWS, a result can be in one  
of thread cases:

  - formally named and mandated by the protocol: name is fixed; result  
must be present when status=COMPLETED; clients can assume these things  
and bypass the results list;

  - formally named and made optional by the protocol: name is fixed,  
result might not be present on job completion; clients can either use  
the results list to find whether it's there or just get its and handle  
the 404 if it's missing;

  - not formally named: neither URI nor presence is predictable:  
clients must use the links in the results lists to find these results.

TAP has one result that is both named and mandated and nothing in the  
other two categories.


>
> - The TAP RFC document still links to the June version of the  
> document.  While there is a comment before the TCG review section  
> indicating that there is a new document, this should probably be  
> noted and linked to at the very top of the document.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.ivoa.net/pipermail/dal/attachments/20091030/789cc1bd/attachment-0003.html>