prototype: scalable VOSI-tables-1.1

[Patrick Dowler]

Hi Mark,


On 07/05/15 06:21 AM, Mark Taylor wrote:
>> >The change to the VOSI-tables XSD is to add <schema> and <table> as valid
>> >document root elements with types taken from VODataService-1.1 xsd.
> Not necessarily.  You could just require that every response from
> this (modified) tables endpoint still has the tableset top-level
> element, but only contains the elements that have been requested
> (e.g. the ancestor schema and tableset of a requested table, but
> not the sibling schemas).

I did consider this option (it is easy to implement) but something about 
having the same document with stuff just missing rubs me the wrong way. 
And writing integration tests (sort of a client) I found different 
document roots to make more sense...

Given that the  VOSI-tables xsd does nothing but import VODataService 
and define which element(s) can be root, adding to it is very simple and 
safe.

 > Although I'm still experimenting, I currently
 > use a hybrid metadata acquisition policy that uses the /tables
 > endpoint for small services, and TAP_SCHEMA for large ones:
 >
 >     ncol = (SELECT COUNT(*) FROM TAP_SCHEMA.columns)
 >     if ncol < 5000
 >        slurp entire VODataService doc from /tables endpoint
 >     else
 >   read all schemas and tables, but not columns, from TAP_SCHEMA in one go
 >        read per-table column/foreign key info from TAP_SCHEMA as required

The count query seems a good way to gauge the amount of content. The 
"else" could be accomlished with VOSI-tables-1.1 with

http://www.cadc-ccda.hia-iha.nrc-cnrc.gc.ca/tap/tables?detail=table

which gives you all the schema and table names (easier than 2 queries to 
TAP_SCHEMA? I guess so) followed by calls like

http://www.cadc-ccda.hia-iha.nrc-cnrc.gc.ca/tap/tables/TAP_SCHEMA/TAP_SCHEMA.tables

to get individual table metadata. Whether VOSI-tables documents are 
easier or harder to parse than TAP_SCHEMA query output... I guess that's 
up to you.

-- 

Patrick Dowler
Canadian Astronomy Data Centre
National Research Council Canada
5071 West Saanich Road
Victoria, BC V9E 2E7

250-363-0044 (office) 250-363-0045 (fax)