prototype: scalable VOSI-tables-1.1

[Patrick Dowler]

I have implemented a prototype VOSI-tables-1.1 resource to deal with the 
issues that came up in the TAP discussion in Banff: some services have 
many tables and many columns and the top-level VOSI-tables document can 
be very large.


The basic aproach is to define a RESTful resource tree following the
VODataService model: tableset, schema, table. All the example URLs below 
work but note that the table names are fully qualified because that is 
how it works in TAP right now. There are separate discussions on that so 
it isn't really relevant here.

The VOSI-tables-1.0 resource returns a <tableset> document, so the 
"tables" resource name really means "the tableset". This remains unchanged:

http://www.cadc-ccda.hia-iha.nrc-cnrc.gc.ca/tap/tables

Any schema name can be used as a child; this returns a <schema> document:

http://www.cadc-ccda.hia-iha.nrc-cnrc.gc.ca/tap/tables/TAP_SCHEMA

Any table name found in a schema can be used as a child; this returns a 
<table> document:

http://www.cadc-ccda.hia-iha.nrc-cnrc.gc.ca/tap/tables/TAP_SCHEMA/TAP_SCHEMA.tables

There is no REST binding to get a single <column> within  a <table> as I 
don't see the use case for that. The abvoe by itself is nice but doesn't 
solve the scalability problem. For that we need to ask for less than 
"all the details" of VOSI-tables-1.0 using the "detail" parameter (name 
taken from the VOSpace spec as it seems to be the same sort of thing: 
ask for a certain amount of detail. The default (as the above
URLs show) is to get all the details below the resource. The detail 
parameter can taken two values: schema, table.

To get a <tableset> document with a list of <schema> only:

http://www.cadc-ccda.hia-iha.nrc-cnrc.gc.ca/tap/tables?detail=schema

To get a <tableset> docuemnt with <schema> and <table> (no <column>):

http://www.cadc-ccda.hia-iha.nrc-cnrc.gc.ca/tap/tables?detail=table

Somewhat useless*: get a <schema> document without the <table>(s):

http://www.cadc-ccda.hia-iha.nrc-cnrc.gc.ca/tap/tables/TAP_SCHEMA?detail=schema

To get a <schema> doccument with only the list of <table> (no <column>):

http://www.cadc-ccda.hia-iha.nrc-cnrc.gc.ca/tap/tables/TAP_SCHEMA?detail=table

To get a <table> document (a single table):

http://www.cadc-ccda.hia-iha.nrc-cnrc.gc.ca/tap/tables/TAP_SCHEMA/TAP_SCHEMA.tables

Somewhat useless*: get a <table> document without the <column>(s):

http://www.cadc-ccda.hia-iha.nrc-cnrc.gc.ca/tap/tables/TAP_SCHEMA/TAP_SCHEMA.tables?detail=table

* it was simpler to implement this response than to come up with some 
logic for denying such a request; it can check the existence of the 
schema or table.

Given the size of the documents when detail=schema or detail=table is 
used (very small), I don't see an obvious need for a pagination 
mechanism. VOSpace has such a feature so we could model something on 
that but I'd like to see the practical need.

These would all return a 404 response code:

testSchemaNotFound:
http://www.cadc-ccda.hia-iha.nrc-cnrc.gc.ca/tap/tables/no_such_schema

testTableNotFound:
http://www.cadc-ccda.hia-iha.nrc-cnrc.gc.ca/tap/tables/TAP_SCHEMA/no_such_table

testExtraPathComponentNotFound (trying to get a <column>):
http://www.cadc-ccda.hia-iha.nrc-cnrc.gc.ca/tap/tables/TAP_SCHEMA/tables/table_name

The base resource complies to VOSI-tables-1.0 so 1.0 clients could use a 
VOSI-tables-1.1 resource. The only outstanding issue is how would a 
service force clients to use 1.1 and forbid them from getting the whole 
<tableset> document from the base resource? That is already an issue and 
right now I think the only compliant choice (for TAPvizier, for example) 
would be to respond with a 403 "Forbidden": only authorized callers can 
get the whole tableset in full detail. This is already how some services 
handle things like a GET to a UWS job-list. I think any oter kind of 
response to indicate that some of the requests are not supported would 
mess wit backwards compatibility. Making the default "detail" not be 
"all" would likewise trod on compatibility. Obviously, the tap 
capabilities would express that the VOSI-tables resource implemented 1.0 
or 1.1 and I don't see any obvious reason to require a specific one in 
TAP-1.1 (at least).

The change to the VOSI-tables XSD is to add <schema> and <table> as 
valid document root elements with types taken from VODataService-1.1 xsd.



-- 

Patrick Dowler
Canadian Astronomy Data Centre
National Research Council Canada
5071 West Saanich Road
Victoria, BC V9E 2E7

250-363-0044 (office) 250-363-0045 (fax)