TAP discovery in 1.1

[Markus Demleitner]

Hi,

On Mon, Sep 11, 2017 at 10:20:08AM -0700, Patrick Dowler wrote:
> On 11 September 2017 at 05:13, Markus Demleitner
> <msdemlei at ari.uni-heidelberg.de> wrote:
> > will then return each set of interfaces twice.  Also, at least I will
> > declare a TAP 1.0 capability for quite a while in addition because
> > otherwise my service will disappear from legacy clients.  In sum,
> > you'll have
> >
> > <capability standardID="ivo://ivoa.net/std/TAP">
> >   ...
> >
> > <capability standardID="ivo://ivoa.net/std/TAP#async-1.1">
> >   ...
> >
> > <capability standardID="ivo://ivoa.net/std/TAP#sync-1.1">
> >   ...
> 
> The main reason for having these is that you have a set of endpoints
> with different names to implement different authentication methods and
> the client needs to figure out "which endpoint is the sync endpoint
> that supports username/password" (for example).

Ah, I see; in the end it's a grouping issue, where there are two
conflicting groupings: there's the grouping "endpoints providing
access to (essentially) the same service that should only be shown
once in an enumeration", and the grouping "these endpoints make up a
full TAP service (sync/async/cap/tables) together".  Hm...

> Now, you could declare an old-style ivo://ivoa.net/std/TAP capability
> and rely on the async - sync naming convention. We *used to do that
> with paths like this*:
> 
> /tap/async
> /tap/sync
> /tap/capabilities
> /tap/auth/async
> /tap/auth/sync

-- having these as separate capabilities of course also alleviates
the issue of multiple interfaces within each capability.  

In a way, I suspect it's even worse.  For instance, I would expect
the the availability endpoint could have quite different responses
for the /tap/auth and the /tap endpoints (e.g., because the SSO
infrastructure might be down, or the SSL cert has expired and the
registrar is broke, whatever).  And, really, I'd expect the
capabilties to be different, too -- for instance as regards hard
limits on downloads and uploads.

I'm not yet quite sure what this means, but

> That starts to look like /tap and /tap/auth are just two separate TAP
> services... so it doesn't really solve the RegTAP situation.

would certainly be straightforward in implementation, although it
would raise the question of how to come up with an enumeration of
"all TAP services out there" to a whole new level (in such a world, I
suppose the enumeration query would have to go through
rr.relationship -- shudder).

> 1. query "registry" by resourceID and find the VOSI-capabilities location
> 2. get the capabilities document and use that to figure out how to
> actually invoke the service
> 
> Our client tools are targeted at a specific service so we do not
> generally try to solve the part RegTAP does:
> 
> 0. discovery interesting/useful services (find/pick resourceID)
> 
> and I realise that existing clients probably do not perform 1 and 2
> before trying to call a service. That's too bad because that part
> really works quite well.

I suppose it would be not totally unreasonable to strongly urge
clients to do 1 and 2 in TAP 1.1 if that's the sanest way to cope
with restricted services (much as I dream of a world in which we
didn't have that complication).


But I'd propose we take a step back and try to figure out the
discovery and usage scenarios and what would work for which users.
Perhaps that would really work better interactively.  Anyone in
for a brainstorming in the ADASS week?  I suppose TAP 1.1 can wait
that long, in particular as implementations are being worked on all
around.

Anyway, here's my first attempt at "scenarios for a VO with
proprietary services" (perhaps we should do a note on this?):

(1) Ad-hoc usage.  I'm pretty sure that for the forseeable future
"normal" users expect clients to work as TOPCAT does right now: You
get to browse a service list or type in a few keywords, and whatever
comes back is a service you can "just use".  This would suggest to me
that systems that are all closed (or just have an unusably small,
"teaser" public part) or that require registration should be easily
filterable or, preferably, not show up in the queries current clients
use.

(2) VIP data discovery.  I'm using VIP here for a person who, in one
or more SSO schemes, may have access to proprietary data and wants do
do searches like "quasar list with redshifts and UV photometry".  In
an ideal-minus-one world (almost everything is fine, there's just
still proprietary data left), I think the expectation would be that
one could say "I have these sets of credentials, now do these ad-hoc
discovery queries but include everything proprietary I have access
to."

This would require the proprietary services to publish, e.g., table
metadata (will they?) as well as declarations as to which sorts of
credentials are accepted where, presumably in an extended
vr:SecurityMethod.  We're far from there on the side of
infrastructure, but I *think* these people would be best served if
the restricted access modes were described in extra services (they
have extra /tables endpoints after all).

(3) VIP service discovery.  Here (and I think that's a much closer
thing to happen), the VIPs have a solid idea what service they're
looking for; let's say the ExpensoSat archives.  Then, clients
should, I think, be able to figure out with reasonable registry
queries (and/or preferably from the service's capability endpoint)
that there's a public part and a proprietary part, and how to access
them; clients could then ask the user if they have any of the
required credentials and choose the appropriate endpoint.

Are these reasonable scenarios?  Have I missed some?

        -- Markus