prototype: scalable VOSI-tables-1.1

[Markus Demleitner]

Hi,

On Mon, May 04, 2015 at 01:56:08PM -0700, Patrick Dowler wrote:
> The basic aproach is to define a RESTful resource tree following the
> VODataService model: tableset, schema, table. All the example URLs below
> work but note that the table names are fully qualified because that is how
> it works in TAP right now. There are separate discussions on that so it
> isn't really relevant here.

Well... depends.  If you're using fully qualified names, there's no
strong reason to talk about schemas in your URL scheme at all, as the
table names are guaranteed unique.

There is a good reason not to talk about schemas: ADQL table
references have one to three components, and mapping those components
to a constant number of two levels on the tables endpoint may be
painful with empty or magic path elements.

True, the problem of a mismatch between the VODataService model and
ADQL table references already exists in  VODataService itself.
Given that valid table references can have one to three "segments"
(and one could imagine query languages with even more exotic table
references behind a TAP service), I now believe it was an error to
require schema in VODataService 1.1 (1.0 didn't have it), and I
propose to re-allow (and even recommend) table elements as direct
children of tableset there.

I've just added as much on
http://wiki.ivoa.net/twiki/bin/view/IVOA/VODataServiceNext --
comments are welcome there.

In short, I'd like to reduce explicit modeling of naming hierachies
in our protocols, and I think there's little to be gained by it here,
too.  So, I'd say, instead of

> http://www.cadc-ccda.hia-iha.nrc-cnrc.gc.ca/tap/tables/TAP_SCHEMA/TAP_SCHEMA.tables

we should have

http://www.cadc-ccda.hia-iha.nrc-cnrc.gc.ca/tap/tables/TAP_SCHEMA.tables

or, if trolls come to haunt us,

http://example.com/tap/tables/%22%2B%2Bdatabase%22/%22example%20schema%22/%22VI/23.4%22

for a table referenced by

"++database"."example schema"."VI/23.4"

In consequence, detail=schema could go, too, and detail would
essentially become a boolean (show columns or don't show them).  I
still don't find detail per se pretty, but as I have no better idea
to offer, I'll shut up about it.

> Given the size of the documents when detail=schema or detail=table is used
> (very small), I don't see an obvious need for a pagination mechanism.
> VOSpace has such a feature so we could model something on that but I'd like
> to see the practical need.

I'm all for trying to avoid pagination as best we can -- pagination
of potentially changing resources is *really* hard to get right over
HTTP, as it implies transactionality over a stateless protocol.

> think the only compliant choice (for TAPvizier, for example) would be to
> respond with a 403 "Forbidden": only authorized callers can get the whole

I like the idea of telling clients: "If you get a 403, try again with
less details".  Services doing that won't look good in 1.0 clients,
but at least they'll still work.

> The change to the VOSI-tables XSD is to add <schema> and <table> as valid
> document root elements with types taken from VODataService-1.1 xsd.

Or keep it as it is and just dump things in a tableset consisting of
just one schema and one table?  I don't think it's much uglier and as
long as we keep the "schema change means namespace change" rule, I'd
rather tread lightly there lest we unnecessarily break clients.

On the other hand, if we change VODataService in parallel (and that
might well be a good idea with TAP evolution), the schema would need
to change anyway, and such a change would come for free.

Cheers,

        Markus