Fwd: Mail delivery failed: returning message to sender

[Guy Rixon]

>
> On 30 Oct 2009, at 13:45, Tom McGlynn wrote:
>
>> I've had a few questions with the implementation of the asynchronous
>> access for TAP.  Most of these are relevant to UWS document
>> generally rather than just TAP so I've copied the GWS group in this
>> mail.
>>
>> 	Tom McGlynn
>>
>> -- UWS general questions --
>>
>> - As defined a user needs to always do two web actions to start the
>> service.  Is there some reason that the user cannot simply request
>> the service to start running immediately?  I suspect that that is
>> what the user wants to do in 99% of the cases.  It would be much
>> easier for clients too.  The example given in the UWS document of
>> starting a job omits the error checking that the a user presumably
>> should do after starting the job.  Why not allow
>>    {root}/tap/async?request=doQuery&query=...&phase=RUN
>> to both create and start a query?  [Describe it as a POST if you
>> prefer.]
>
> That would work, but would have to be POSTed.
>
>
>>
>> - I'm continue to be confused by the benefits conferred by various
>> practices.  Why do we require POSTs specifically in a number of
>> intances?  E.g., what would be wrong with using
>>   ../jobid/phase?phase=RUN
>> as an HTTP GET rather than an HTTP POST.  Since my code cannot tell
>> the difference between these I certainly will be supporting both,
>> but, other than bowing to the mantra REST, I'm not sure why it's
>> supposed to matter.
>
> Whether or not a web service follows REST principles, it has to
> distinguish between requests that change the service state and
> requests that are idempotent. This is a basic part of HTTP. Starting a
> job changes the service state by creating web resources for that job.
> Sending the same query twice gets you two jobs doing the same query;
> they have separate web-resources. Therefore, not idempotent; therefore
> a POSTed request.
>
> GET responses can be cached, and the caching is out of your control as
> a service provider - it may be on the user's LAN (HTTP proxy) or in
> their client (browser cache). If you send the same query twice then
> via GET, for the second request you could get the response for the
> first, pulled from the cache, and no new job. This doesn't happen too
> often but when it does it's brain-bendingly harder to debug.
>
> I suggest that your code must not accept UWS create-job requests via
> HTTP GET. Your users won't like it if they get given the wrong answer
> from a cache. And Google tend to spider all the GETable URLs so you
> don't want them creating jobs.
>
>
>
>>
>> - Similarly I don't think that there should be a strict limitation
>> of the coding used in sending requests.  There may be a requirement
>> that a given encoding be supported but there should not be a
>> requirement that it be used.  As with POST/GET this level of HTTP
>> detail is handled below the UWS logic in our implementation, so for
>> services that the HEASARC supports we'll be allowing multipart/form
>> encoding as well unless someone can tell us why we should reject
>> such requests.
>
> You're free to support this as well as the stated encoding because it
> doesn't break anything. However, if you write a client that assumes
> this encoding then it won't work on all implementations. So it seems
> pointless to add the feature even in the service.  Personally, I think
> that supporting broken clients in this way is not helpful.
>
>>
>> - From what states is a user allowed to start a job?  E.g., can a
>> user attempt to restart a job that has previously had an error or
>> aborted? Could the user change the parameters and then rerun the
>> same job?  I'm guessing this isn't supposed to happen, but I didn't
>> see where it was forbidden.
>
> You can only change things while it's pending. If you need to re-run a
> failed job then you have to resubmit it.
>
>
>>
>> - What is supposed to happen if there is a problem in creating the
>> job.
>> Should a job be created with an immediate status of ERROR?  Is there
>> any way of flagging an error if the system cannot create even an
>> error job?  E.g., we're going to use the database to store all job
>> information. What are we supposed to do if the database is down?  It
>> would be nice to be able to inform the user of an error in a
>> standard way.
>
> If you can create the job at all then you should immediately set the
> phase to ERROR and make the error document available. If you can't do
> this, then I guess giving up with a 500 "I'm completely stuffed" error
> is reasonable. By extension, UWS clients need to deal minimally with
> 500 errors as well as with proper error-documents.
>
>
>>
>> - Is there supposed to be a one-one mapping between there being an
>> error resource and the job failing with error or aborted.  I.e., if
>> I have a job with phase COMPLETED can it still have an error
>> document?  Can I have phase ABORTED without an error document?
>
> Not formally defined. I don't see why you can't have an "error"
> document saying "OK, no problem" for status COMPLETED.
>
>
>
>>
>> - Is is allowed to have an empty error document?  E.g., can I have a
>> URL
>> ".../jobid/error"
>> which exists but has no content (either as an actually zero length
>> string, or as an empty VOTable)?
>
> In UWS, yes, because the form of the document is not fixed. In TAP, no
> you may not do that. It has to be a VOTable with the status filled in.
>
>
>>
>>
>>
>> -- TAP specific questions. --
>>
>> - The description of where to get the TAP result in an async request
>> is not given (as far as I can see) in what is described as the
>> normative parts of the document.  There it says that result will be  
>> in
>>  root/async/jobid/results/
>> but this is the list of results and can, in principle, contain a
>> number of results. Only in section 5.2 which is described as
>> informative does it say that the result document is .../results/
>> result.  Is this actually  a requirement or can the result be named
>> anything?
>
> In my service implementation, I take it to be a requirement. In my
> client implementation it currently assumes the one result with the
> standard name but I plan to make it parse the list. (In case we add to
> the results list in future TAP versions.)
>
>
>>
>> - The UWS standard discusses the naming of results.  Does TAP
>> require a specific name for the result?  In fact it looks like the
>> way UWS is supposed to be used the jobid/results returns a document
>> that looks like
>>
>>   <results>
>>      <result id='someid' xlink:href='someurl' />
>>      <result id='anotherid' xlink:href='anotherurl' />
>>   </results>
>>
>> and the user is supposed to find the id of the desired result and
>> use whatever URL is given there, not use a specifically defined
>> URL.  I'm guessing the the ID attributes of the <result> fields is
>> the UWS name. The UWS standard says
>> "When a protocol specifies standard results it must do so by
>>  naming those results; the names appear in the Results list in
>>  addition to the URI's.  Not all results need to be named, sometimes
>>  the meaning of the result is obvious from the context and the
>>  name is omitted."
>> Since the second sentence here seems to contradict the first it's a
>> bit hard to follow, but my reading of this is that it would be
>> better for TAP to specify a name for the output result rather than a
>> specific URL.
>
> For a given service-protocol incorporating UWS, a result can be in one
> of thread cases:
>
>  - formally named and mandated by the protocol: name is fixed; result
> must be present when status=COMPLETED; clients can assume these things
> and bypass the results list;
>
>  - formally named and made optional by the protocol: name is fixed,
> result might not be present on job completion; clients can either use
> the results list to find whether it's there or just get its and handle
> the 404 if it's missing;
>
>  - not formally named: neither URI nor presence is predictable:
> clients must use the links in the results lists to find these results.
>
> TAP has one result that is both named and mandated and nothing in the
> other two categories.
>
>
>>
>> - The TAP RFC document still links to the June version of the
>> document.  While there is a comment before the TCG review section
>> indicating that there is a new document, this should probably be
>> noted and linked to at the very top of the document.
>
>
> --Apple-Mail-14--568047058
> Content-Type: text/html;
> 	charset=US-ASCII
> Content-Transfer-Encoding: quoted-printable
>
> <html><body style=3D"word-wrap: break-word; -webkit-nbsp-mode:  
> space; =
> -webkit-line-break: after-white-space; "><br><div><div>On 30 Oct  
> 2009, =
> at 13:45, Tom McGlynn wrote:</div><br =
> class=3D"Apple-interchange-newline"><blockquote  
> type=3D"cite"><div>I've =
> had a few questions with the implementation of the asynchronous  
> access =
> for TAP.  Most of these are relevant to UWS document generally =
> rather than just TAP so I've copied the GWS group in this =
> mail.<br><br><span class=3D"Apple-tab-span" style=3D"white- 
> space:pre">	=
> </span>Tom McGlynn<br><br>-- UWS general questions --<br><br>- As =
> defined a user needs to always do two web actions to start the  
> service. =
>  Is there some reason that the user cannot simply request the =
> service to start running immediately?  I suspect that that is  
> what =
> the user wants to do in 99% of the cases.  It would be much  
> easier =
> for clients too.  The example given in the UWS document of  
> starting =
> a job omits the error checking that the a user presumably should do =
> after starting the job.  Why not allow<br> =
>     {root}/tap/async? 
> request=3DdoQuery&query=3D...=
> &phase=3DRUN<br>to both create and start a query?   
> [Describe it =
> as a POST if you prefer.]<br></div></blockquote><div><br></ 
> div><div>That =
> would work, but would have to be =
> POSTed.</div><div><br></div><br><blockquote type=3D"cite"><div><br>-  
> I'm =
> continue to be confused by the benefits conferred by various  
> practices. =
>  Why do we require POSTs specifically in a number of intances? =
>  E.g., what would be wrong with using<br> =
>    ../jobid/phase?phase=3DRUN<br>as an HTTP GET  
> rather =
> than an HTTP POST.  Since my code cannot tell the difference =
> between these I certainly will be supporting both, but, other than =
> bowing to the mantra REST, I'm not sure why it's supposed to =
> matter.<br></div></blockquote><div><br></div><div>Whether or not a  
> web =
> service follows REST principles, it <i>has </i>to distinguish  
> between =
> requests that change the service state and requests that are  
> idempotent. =
> This is a basic part of HTTP. Starting a job changes the service  
> state =
> by creating web resources for that job. Sending the same query twice =
> gets you two jobs doing the same query; they have separate =
> web-resources. Therefore, not idempotent; therefore a POSTed =
> request.</div><div><br></div><div>GET responses can be cached, and  
> the =
> caching is out of your control as a service provider - it may be on  
> the =
> user's LAN (HTTP proxy) or in their client (browser cache). If you  
> send =
> the same query twice then via GET, for the second request you could  
> get =
> the response for the first, pulled from the cache, and no new job.  
> This =
> doesn't happen too often but when it does it's brain-bendingly  
> harder to =
> debug.</div><div><br></div><div>I suggest that your code must not  
> accept =
> UWS create-job requests via HTTP GET. Your users won't like it if  
> they =
> get given the wrong answer from a cache. And Google tend to spider  
> all =
> the GETable URLs so you don't want them creating =
> jobs.</div><div><br></div><div><br></div><br><blockquote =
> type=3D"cite"><div><br>- Similarly I don't think that there should  
> be a =
> strict limitation of the coding used in sending requests.  
>  There =
> may be a requirement that a given encoding be supported but there  
> should =
> not be a requirement that it be used.  As with POST/GET this  
> level =
> of HTTP detail is handled below the UWS logic in our implementation,  
> so =
> for services that the HEASARC supports we'll be allowing multipart/ 
> form =
> encoding as well unless someone can tell us why we should reject  
> such =
> requests.<br></div></blockquote><div><br></div><div>You're free to =
> support this as well as the stated encoding because it doesn't break =
> anything. However, if you write a client that assumes this encoding  
> then =
> it won't work on all implementations. So it seems pointless to add  
> the =
> feature even in the service.  Personally, I think that  
> supporting =
> broken clients in this way is not helpful.</div><br><blockquote =
> type=3D"cite"><div><br>- =46rom what states is a user allowed to  
> start a =
> job?  E.g., can a user attempt to restart a job that has  
> previously =
> had an error or aborted? Could the user change the parameters and  
> then =
> rerun the same job?  I'm guessing this isn't supposed to  
> happen, =
> but I didn't see where it was =
> forbidden.<br></div></blockquote><div><br></div><div>You can only  
> change =
> things while it's pending. If you need to re-run a failed job then  
> you =
> have to resubmit it.</div><div><br></div><br><blockquote =
> type=3D"cite"><div><br>- What is supposed to happen if there is a =
> problem in creating the job.<br>Should a job be created with an =
> immediate status of ERROR?  Is there any way of flagging an  
> error =
> if the system cannot create even an error job?  E.g., we're  
> going =
> to use the database to store all job information. What are we  
> supposed =
> to do if the database is down?  It would be nice to be able to =
> inform the user of an error in a standard =
> way.<br></div></blockquote><div><br></div><div>If you can create the  
> job =
> at all then you should immediately set the phase to ERROR and make  
> the =
> error document available. If you can't do this, then I guess giving  
> up =
> with a 500 "I'm completely stuffed" error is reasonable. By  
> extension, =
> UWS clients need to deal minimally with 500 errors as well as with =
> proper error-documents.</div><div><br></div><br><blockquote =
> type=3D"cite"><div><br>- Is there supposed to be a one-one mapping =
> between there being an error resource and the job failing with error  
> or =
> aborted.  I.e., if I have a job with phase COMPLETED can it  
> still =
> have an error document?  Can I have phase ABORTED without an  
> error =
> document?<br></div></blockquote><div><br></div><div>Not formally =
> defined. I don't see why you can't have an "error" document saying  
> "OK, =
> no problem" for status =
> COMPLETED.</div><div><br></div><div><br></div><br><blockquote =
> type=3D"cite"><div><br>- Is is allowed to have an empty error  
> document? =
>  E.g., can I have a URL<br>  ".../jobid/error"<br>which  
> exists =
> but has no content (either as an actually zero length string, or as  
> an =
> empty VOTable)?<br></div></blockquote><div><br></div><div>In UWS,  
> yes, =
> because the form of the document is not fixed. In TAP, no you may  
> not do =
> that. It has to be a VOTable with the status filled =
> in.</div><div><br></div><br><blockquote  
> type=3D"cite"><div><br><br><br>-- =
> TAP specific questions. --<br><br>- The description of where to get  
> the =
> TAP result in an async request is not given (as far as I can see) in =
> what is described as the normative parts of the document.  
>  There it =
> says that result will be in<br> =
>   root/async/jobid/results/<br>but this is the list of  
> results =
> and can, in principle, contain a number of results. Only in section  
> 5.2 =
> which is described as informative does it say that the result  
> document =
> is .../results/result.  Is this actually  a requirement or  
> can =
> the result be named =
> anything?<br></div></blockquote><div><br></div><div>In my service =
> implementation, I take it to be a requirement. In my client =
> implementation it currently assumes the one result with the standard =
> name but I plan to make it parse the list. (In case we add to the =
> results list in future TAP =
> versions.)</div><div><br></div><br><blockquote  
> type=3D"cite"><div><br>- =
> The UWS standard discusses the naming of results.  Does TAP  
> require =
> a specific name for the result?  In fact it looks like the way  
> UWS =
> is supposed to be used the jobid/results returns a document that  
> looks =
> like<br><br>    <results><br> =
>       &l