Home for the UWS Registry Extension?

Sat Jul 28 00:06:51 CEST 2018

Hi Grid,

(Dropped registry, dal from this thread)

I thought I'd offer my opinion on the support of synchronous job running in
UWS...

On Mon, Jun 11, 2018 at 3:14 PM Mark Taylor <m.b.taylor at bristol.ac.uk>
wrote:

> My feeling is that if you're going to use sync mode, it's because
> you don't want to muck about with all that job ID business.
> I always thought of UWS as a protocol for clients that want job metadata
> (either to support long-running jobs or for other reasons)
> and synchronous services as for clients that don't.
>

Yes, that's a good point.  The UWS 1.1 spec does provide a suggested
(informative) recipe for implementing a sync endpoint.  If a service were
written with that recipe then clients could indeed ignore all the jobID
business by simply choosing to follow all redirects on the initial job
request.

http://ivoa.net/documents/UWS/20161024/REC-UWS-1.1-20161024.html#SynchronousService

I think it is often in the hands of the user as to whether a job should be
run synchronously or asynchronously.  They would prefer an immediate result
so would favour synchronous but would understand that for some tasks
asynchronous would be better (error handling, timeouts, lots of output,
etc...) . For these types of services where the need for asynchronous
execution is required, you may want to offer a similar experience for
clients to potentially run the same things synchronously.  That is, the
same way of gathering the task parameters and the same way of getting at
results.  So, I think there's a case to be made that, if a there is a need
for async support and a desire to offer sync for convenience, they should
be be offered in a similar manner.

(Looking at it another way: would there ever be value in writing a
sync-only service using the UWS pattern?  I would argue probably yes.)

>
> But at least I would be wary of, e.g., requiring something like
> a TAP sync endpoint to be marked up in such a way that it has
> to be implemented in a UWS-like way, rather than just delivering
> the result and forgetting about it.  I know some services do
> implement sync on top of async so that would not be an implementation
> issue, but others may not.
>

Yes, I think official UWS sync support would have to be the same as simply
delivering the results.  (Maybe with the condition that clients must choose
to follow all redirects.)

>
> I am not violently opposed to using UWS as a place to to hang
> these sync/async markers, but I still find it a bit unintuitive
> to have UWS talking about synchronous services.
>

I can understand this point but I also think that since it is the same work
being executed (with sync or async) we may as well make them as similar as
possible in the other ways for clients.

More below...

>
> Mark
>
> On Wed, 30 May 2018, Patrick Dowler wrote:
>
> > The  home for these concepts depend on a couple of predictions we might
> > make:
> >
> > 1. Is it useful for UWS to be extended to specify interface behaviour for
> > synchronous job execution? TAP, SODA, and VOSpace all have "jobs" that
> can
> > run  in either mode and s/w choses the endpoint with the behaviour it
> > wants... the minimal "specification" of UWS-sync would be a standard way
> > for a client executing a sync job to get the jobID and access it via a
> > joblist (eg in UWS-next)
>

As per above, my feeling is that there is a benefit to UWS offering sync
support.

> >
> > 2. Ae there common metadata about the UWS job endpoint that could be
> > conveyed by adding content to the UWSRegExt? e.g. will UWSRegExt go
> beyond
> > providing tagging interface in future? This one I am less sure about.
> > TAPRegExt already describes some of the UWS job control limits, for
> > example, but that reg-ext defines a new capability type (the type of
> > job)... I do have some UWS-defined features that are interface-specific
> so
> > there are use cases.
>

I know that there are a couple of UWS client software tools out there but I
haven't taken a close look at them.  I am wondering if these tools can be
used (without plugins or customization) to speak to UWS services such as
TAP and SODA independently.  If so, then I imagine it puts a lot of
responsibility in the hands of the user to formulate the correct input for
the different services.  Perhaps this is an area where extended UWS
metadata could help...

> >
> > I do think it is important for us, as a community, to find ways to do
> small
> > things and get them done in a short time, and then iterate more rapidly
> on
> > minor versions.
>

+1 on that.

> >
> > my 2c,
> >
> > Pat
>

Cheers,
Brian

> >
> > On 29 May 2018 at 17:21, Brian Major <major.brian at gmail.com> wrote:
> >
> > > Hi DAL, Registry and Grid,
> > >
> > > Sorry for the wide distribution but this touches all three working
> groups.
> > >
> > > An initial note for the UWS Registry Extension has been written and
> can be
> > > viewed here:
> > >
> > >     http://wiki.ivoa.net/internal/IVOA/IvoaGridAndWebServices/
> > > UWSRegExt.pdf
> > >
> > > It is very simple: the essence of the note is to allow registry
> interface
> > > entries to 'tag' interfaces as being UWS synchronous or asynchronous
> > > endpoints.  This is to allow implementers of UWS services (TAP is the
> > > working example) the freedom of having separate URLs for interfaces
> with
> > > various security methods (supported authentication) and to have custom
> URLs
> > > for sync and async endpoints.
> > >
> > > Comments are welcome.  There is still uncertainty about how the
> > > information in this note should be brought in as a document for
> reference
> > > by TAP 1.1, so comments on that are welcome too.
> > >
> > > Brian
> > >
> > >
> > > On Thu, Feb 22, 2018 at 4:58 PM Brian Major <major.brian at gmail.com>
> wrote:
> > >
> > >> Hi Grid, Registry,
> > >>
> > >> At the Santiago Interop we decided to identify the sync and async
> > >> endpoints of a TAP 1.1 capability by setting the 'type' of the
> Interface
> > >> element as one of either
> > >>
> > >>     uws:Sync or
> > >>     uws:Async
> > >>
> > >> instead directly in the standardID of the capability like
> > >>
> > >>     ivo://ivoa.net/std/tap#sync-1.1 and
> > >>     ivo://ivoa.net/std/tap#async-1.1
> > >>
> > >> We have implemented this in our TAP (1.1) services using the XSD here:
> > >>
> > >>     https://github.com/opencadc/reg/blob/master/cadc-registry/
> > >> src/main/resources/UWSRegExt-v0.1.xsd
> > >>
> > >> And it is all working as expected, no issues.
> > >>
> > >> Now I am wondering now where this XSD can be placed within the IVOA
> and
> > >> how it can be documented to describe the intended use.  This will
> need to
> > >> go somewhere official ahead of TAP 1.1.  Any suggestions are welcome.
> > >>
> > >> Thanks,
> > >> Brian
> > >>
> > >>
> > >>
> > >>
> >
> >
> > --
> > Patrick Dowler
> > Canadian Astronomy Data Centre
> > Victoria, BC, Canada
> >
>
> --
> Mark Taylor   Astronomical Programmer   Physics, Bristol University, UK
> m.b.taylor at bris.ac.uk +44-117-9288776  http://www.star.bris.ac.uk/~mbt/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.ivoa.net/pipermail/grid/attachments/20180727/c34178f8/attachment.html>