table metadata and the registry

[Tony Linde]

> No doubt by now you have seen my posting to the gws list commenting on

Doesn't seem to have made it yet, Ray. Only recent one up there is about
using MAY for the getCapabilities (or whatever it is called).

> (getRegistration() or getMetadata()) to include it in is the service

I thought there was only one method to get metadata and it returned the
VOResource record. I cannot see the need for more than one such.

> Furthermore, with no guideline as to what information should go in

I would certainly mandate that the full VOResource record be returned with
all the optional bits of that made mandatory. As has been said elsewhere,
the service provider is the only source of the metadata: whether they
provide it through the service or by typing it into the registry is not
especially relevant, but it must ultimately be available from the registry.

> is all or nothing.  Not only does the URL solution allow a registry to
> choose what fine-grained information it collects, but also it does not
> require that that information fit into the VOResource format.

Why would the registry care about non-VOResource information? And what use
is a registry which cannot supply the information a calling service
requires? Do we now have to specify all the levels of metadata that a
registry can and cannot supply?

> You are neglecting the effort of incorporating that fine-grained
> information VOResource.  The point is that if we can just point to the
> information, it does not have to be in VOResource format.

I neglected it because it is negligible. The coding effort of creating a
VOResource record from available metadata is minimal, and exposing the
record to a getWhatever method is less than coding up several cgi services
to respond to metadata URLs.

And how many URLs do we need? Do we have one for tables and one for columns
or just one for combined tables/columns? And what standard do these URLs
follow? It seems we're expecting a whole load of new effort by service
providers, registry providers and the workgroups.

> existing standard catalog services have methods that return table
> metadata.  A simple service (provided by a registry) can translate that
> information into a standard format, so off the bat you get good

How can the registry do that? None of the catalog services have *standard*
ways of providing metadata: a registry will have to implement separate code
for every potential service unless we specify new standards for these
URL-based metadata retrieval methods.

> I don't understand this.  Anybody who wants the fine-grained
> information
> can get it by following the URLs.  Anybody who doesn't want this

But this is an enormous waste of time. I thought the VO was supposed to make
things better. What you propose will mean that anyone who wants to provide a
general query builder will have to query the registry for resources and,
when the user selects a resource, find the service and query it for table
information, then, for each table, query the service for column information:
all while the user patiently stares at a spinning hourglass. This is not,
IMO, an improvement on existing services.


Bottom-line, Ray. I think what you are proposing is a radical change to the
way the VO works. This turns the registry into a simple pointer to resources
and puts the onus on VO applications to do all the searching for metadata,
something that will seriously damage those who wish to provide complex and
useful applications rather than simple front-ends to existing data sources. 

Yes, as you say, a super-registry can decide to cache this metadata, and
certainly the AstroGrid, ESO, ESA etc registries will probably do this (and,
ultimately, most NVO ones I think), but in what format? If a service
standard only has to specify its own ways of presenting metadata then we
will not get any more VOResource capabilities being specified. 

So every registry will have to come up with its own ways of presenting the
real service metadata to its users. And any application will be tied to one
specific registry. Not a very desirable end I think.

Cheers,
Tony.

> -----Original Message-----
> From: owner-registry at eso.org [mailto:owner-registry at eso.org] On Behalf
> Of Ray Plante
> Sent: 08 May 2007 10:16
> To: IVOA Registry WG
> Subject: Re: table metadata and the registry
> 
> Hi Tony,
> 
> On Tue, 8 May 2007, Tony Linde wrote:
> > I'm not sure we do not have a better version of what you suggest
> already.
> > The GWS proposal for getCapabilities will return the full registry
> record
> > for a resource and this is, in effect, a URL for getting the extra
> metadata.
> 
> No doubt by now you have seen my posting to the gws list commenting on
> this standard as an alternative approach to the problem.  The main
> problem
> with the VOSI solution is that the choice of which method
> (getRegistration() or getMetadata()) to include it in is the service
> provider's.  Since the one of the objections to including fine-grained
> information in the registry is the curation costs incurred by the
> registry, this needs to be the choice of the registry.
> 
> Furthermore, with no guideline as to what information should go in
> either,
> the provider is hardly motivated to make any distinction at all between
> them.  The effect will be that either the information will not be
> included in either method, or it will be included in both.
> 
> To resolve this issue, we need allow a way for a registry to get at
> fine-grained information without effectively forcing other registries
> to
> carry the same burden of this information.  The VOSI standard does do
> enough to prevent registries that do not want this information from
> having
> to deal with it.  Even if we later specified what information should go
> into getMetadata() (and managed to get providers to follow that
> specification), the registry's choice of pulling in this extra
> information
> is all or nothing.  Not only does the URL solution allow a registry to
> choose what fine-grained information it collects, but also it does not
> require that that information fit into the VOResource format.
> 
> > If we were to implement your suggestion of a url for the fine-grained
> > metadata this would require:
> > -          that all resources implement these new URLs (effort
> matched by
> > the getCapabilities implementation anyway)
> 
> (I think you mean getMetadata().  It's not clear whether the DAL
> getCapabilities() will include table metadata.)
> 
> Not so.  First of all, providing this URL is optional.  Second, all of
> our
> existing standard catalog services have methods that return table
> metadata.  A simple service (provided by a registry) can translate that
> information into a standard format, so off the bat you get good
> coverage
> with no effort by the providers.
> 
> One relevent future catalog protocol, TAP, will also have these
> methods.
> The conversion of this information into the standard format can be
> accomplished on the fly from these methods.  Again, there is no
> extra per-service instance effort necessary.
> 
> > -          future resources which have fine-grained metadata not of a
> > table/column nature would have to implement other URL methods which
> > registries then have to implement rather than just using
> getCapabilities on
> > those resources (a method which will work on all existing and future
> > resource structures)
> 
> You are neglecting the effort of incorporating that fine-grained
> information VOResource.  The point is that if we can just point to the
> information, it does not have to be in VOResource format.
> 
> > -          the registry interface will need changing to take account
> of
> > getting full resource information rather than the skeleton record
> 
> I don't understand this.  Anybody who wants the fine-grained
> information
> can get it by following the URLs.  Anybody who doesn't want this
> information is not saddled with it anyway.  No change the the registry
> interface is required.
> 
> cheers,
> Ray