table metadata and the registry

[Ray Plante]

Hi Tony,

On Tue, 8 May 2007, Tony Linde wrote:
> 3. we had already agreed to have fine- and coarse-grained registries back in 
> Victoria: I'm not sure why this has come up again (this in response to your 
> comments below about forcing you to do registries our way or getting detailed 
> metadata from your services!). As long as these registries are identifiable 
> and applications can tie themselves to the one they need, what is the 
> problem?
>
> Am I missing anything else?

I tried to explain how a fine-grained regsitry like AstroGrid affects a 
coarse-grained registry in the NVO.  It comes down to how much information 
appears in the records we exchange in harvesting.  If you export 
VOResource records through your harvesting interface that include table 
metadata, we are obligated to preserve that information and make it 
available to users.  That was part of the Victoria deal.  Furthermore, if 
the only way you get table metadata from NVO services is through our 
harvesting interface, then we have to provide this information in order to 
make them usable by the AstroGrid query builder.

Do you see how we have effectively been forced into storing table metadata 
in our registry?

Now, is there a way for you to get at the table metadata--regardless of 
where it came from, integrate it into your registry, and use it as you do 
now, without NVO having to collect and manage this information within our 
registry as well?  This is what I am trying to answer (with a 'yes').

Do you agree that we should try to find a way to do this?

> 1. You've proposed returning a limited subset but in the same format as 
> VOResource table schema. Why do you stop at that level? Surely everything 
> within Capability describes the service. If we're going to pick an arbitrary 
> level, why not Capability or, better, VOResource? What is the issue with the 
> service returning *all* its metadata?

I am currently picking on table metadata, because
   *  this is what critics are specifically pointing to as an example of
        information that is too fine-grained.
   *  it is not included among the RM metadata, our core set intended for
        registries.
   *  it is part of a schema and resource types that are too important to
        ignore completely.

If we can come up with the way to meet the above-mentioned goal, then we 
may have a model for dealing with other controversial information.  This 
might also include detailed detailed coverage descriptions.  What we 
really need is a shared understanding of what is fine-grained so that 
if/when we define new resource metadata and new extensions, we not only 
have a way to evaluate information that might be considered fine-grained, 
but also have a recipe for accessing that information if we can't agree 
whether to include it in the extension.

> 2. I have no problem how the metadata is returned but only want the most 
> efficient method. I would have thought it was more efficient to have all 
> services implement a single standard interface aligned to the way the service 
> is implemented: soap method if a soap service; url if a cgi service.

I suggested a URL as the handle because this is the simplest way to 
retrieve information when inputs are not required.  It can be associated 
with a kind of resource, be it a service or data collection, but it can be 
made as complex as necessary (e.g. to generate the information 
dynamically) under the covers.

> That's fine, but let's make sure the applications and services that rely on 
> registry information *now* can continue to work while these techniques are 
> being developed.

My suggestion had this in mind from the beginning.  You are currently 
collecting table metadata explicitly within VOResource records.  To 
implement my idea, we make a place in the schema to hold the new URL.  We 
begin populating the new metadatum.  We begin using the URL, and then we 
begin deprecating the explicit inclusion of table metadata.  There is 
no interruption in the support for you query builder, and all during 
this transition, we see an increase in the availability of table metadata 
because the URL for Cone Searches, SIAs, and SkyNodes can be 
auto-generated.

cheers,
Ray