getCapabilities() and graininess

[Keith Noddle]

Just a few thoughts:

>> anything; however, once the URL is in place and used, it requires a
>> lady-and-gentleman's agreement among our registries to prefer the URL
>> over explicit inclusion of table metadata in the records we harvest.
> 
> This could be a problem for registries which do store the table/column
> metadata: they're not going to want to hit all the services so registered if
> the registry already contains the information. So we'd need a separate way
> of harvesting full information - yes?
I contend that we need to get the full information from the service. 
Whether and where it is subsequently cached is not within the remit of 
this discussion!

> Also, how does a service/registry indicate that it has updated the
> table/column metadata? Do we say that the current mechanism applies to all
> metadata, whether the home registry holds detailed metadata or not?
There's a VOSI defined mechanism a service can use to indicate it has
changed its metadata. If the service is registered with a fine-grained
(we must find a better description!) Registry, it would be good practise
for the data curator to tell the Registry operator the change has taken
place and so trigger an out of band update. Either way, a fine-grained
registry will need to periodically poll services to see if they have
been updated and harvest accordingly. This is a light-weight operation
which 99% of the time will result in a "no change" response. The network
and service overhead will be minimal.

>>        want AstroGrid users to use NVO resources in the query builder, we
>>        have to provide table metadata from our registries.
> 
> No, I thought we'd agreed that if the home registry does not keep the fine
> metadata, it is up to the harvesting registry to get it from the service.
YES, YES, YES!

>> Clearly from the opinions raised regarding the role of registries, it
>> is not simply a DAL issue; registries have a role in this as well.
>> It's the curation of the metadata in the registries that has folks
>> bothered.  And even in the DAL and VOQL groups there is division about
>> whether table metadata should be managed by the registry.
> 
> I'm still not sure registry curators can better define the fine/coarse split
> than the service providers and users. I'd still prefer the registry folk to
> define generic ways of handling fine/coarse metadata and the service folk to
> settle what they are.
Actually, I'm not sure there is a division. Doug clearly states that he
feels metadata should come from the service and I agree with him 100%.

>> As mentioned above, I am not calling for the removal of the table-related
>> terms in the schema.  Leaving them in there will allow you to continue
> 
> That's good to know.
Seems to me the whole issue is founded on what data the registries
exchange but is clouded by the inclusion of a discussion about the
definition of schemata and the provision of metadata.

I contend the Registry schemata *should* take account of the so-called
fine-grained metadata, but I would venture to suggest that the
definition of those metadata are best sub-contracted to the domain
experts.

If we accept that...

1. Registries MUST exchange the minimum of information needed to satisfy
their resource location obligations
2. Services MUST provide their own metadata
3. It's OK for the Registry Work Group to co-ordinate the definition of
the metadata schemata
4. Projects can cache metadata or not as they see fit, in the Registry
or some other service, again, as they see fit

...then, once we agree consistent mechanisms for gathering these
metadata, aren't we about done?

Keith.

-- 
Keith Noddle                    Phone:  +44 (0)116 223 1894
AstroGrid Project manager       Fax:    +44 (0)116 252 3311
Dept of Physics & Astronomy     Mobile: +44 (0)7721 926 461
University of Leicester         Email:  ktn at star.le.ac.uk
Leicester, UK   LE1 7RH         Web:    http://www.astrogrid.org