prototype: scalable VOSI-tables-1.1

[Mark Taylor]

On Thu, 7 May 2015, Patrick Dowler wrote:

> 
> Hi Mark,
> 
> 
> On 07/05/15 06:21 AM, Mark Taylor wrote:
> > > >The change to the VOSI-tables XSD is to add <schema> and <table> as valid
> > > >document root elements with types taken from VODataService-1.1 xsd.
> > Not necessarily.  You could just require that every response from
> > this (modified) tables endpoint still has the tableset top-level
> > element, but only contains the elements that have been requested
> > (e.g. the ancestor schema and tableset of a requested table, but
> > not the sibling schemas).
> 
> I did consider this option (it is easy to implement) but something about
> having the same document with stuff just missing rubs me the wrong way. And
> writing integration tests (sort of a client) I found different document roots
> to make more sense...

I think it rubs me the right way :-), and would probably require
fewer changes to my existing code - but I don't have very strong
feelings, could live with either.

> Given that the  VOSI-tables xsd does nothing but import VODataService and
> define which element(s) can be root, adding to it is very simple and safe.

Agreed.

> > Although I'm still experimenting, I currently
> > use a hybrid metadata acquisition policy that uses the /tables
> > endpoint for small services, and TAP_SCHEMA for large ones:
> >
> >     ncol = (SELECT COUNT(*) FROM TAP_SCHEMA.columns)
> >     if ncol < 5000
> >        slurp entire VODataService doc from /tables endpoint
> >     else
> >   read all schemas and tables, but not columns, from TAP_SCHEMA in one go
> >        read per-table column/foreign key info from TAP_SCHEMA as required
> 
> The count query seems a good way to gauge the amount of content. The "else"
> could be accomlished with VOSI-tables-1.1 with
> 
> http://www.cadc-ccda.hia-iha.nrc-cnrc.gc.ca/tap/tables?detail=table
> 
> which gives you all the schema and table names (easier than 2 queries to
> TAP_SCHEMA? I guess so) followed by calls like
> 
> http://www.cadc-ccda.hia-iha.nrc-cnrc.gc.ca/tap/tables/TAP_SCHEMA/TAP_SCHEMA.tables
> 
> to get individual table metadata. Whether VOSI-tables documents are easier or
> harder to parse than TAP_SCHEMA query output... I guess that's up to you.

Yes, probably your extended VOSI-tables route is easier to write
client code for than doing it with multiple TAP_SCHEMA TAP queries,
as long as you can rely on it being present.  And it may have
different (better?) performance characteristics too.
I just wanted to note that I have found it *possible* to write a
working client metadata presentation GUI using only existing standards.

--
Mark Taylor   Astronomical Programmer   Physics, Bristol University, UK
m.b.taylor at bris.ac.uk +44-117-9288776  http://www.star.bris.ac.uk/~mbt/