table metadata and the registry

[Ray Plante]

Hi Tony,

On Tue, 8 May 2007, Tony Linde wrote:
> One last query, if the URL only returns the 'extra' metadata, where does the 
> core service metadata come from? The registry only? Does this mean the 
> service provider has to maintain metadata in two locations? Surely one 
> additional benefit of the getWhatever method is that a service provider can 
> update their registry record simply by changing the VOResource record served 
> up by getWhatever?

Let's look at this in two evolutionary stages.  In the near future, the 
user still goes to a publishing registry and fills out a form. 
Consider...
   ...an SIA service:  the user need not enter URL for the table metadata;
        this can be set automatically based on the base URL and the
        translater service.
   ...a theoretical TAP service:  some near-future standard protocol could
        provide this URL-based access as part of its definition; in this
        case the URL is set automatically in the record.  If the standard
        does not provide this, it can still be handled like an SIA through
        a translator.
   ...a Data Collection:  since there is not an associated service to
        provide this information, the publisher can optionally enter this
        URL explicitly.  A registry may provide tools for formatting this
        information.
When the form is submitted, the registry decides whether it wants to 
retrieve the table metadata.

In the next phase, when VOSI is more widely supported, essentially the 
same process happens at service machine when the publisher sets up the 
VOResource description getRegistration().  The VOSI standard provides a 
straightforward way to recognize service resource descriptions that 
support getRegistration() (the VOSI capability extension).  The publisher 
provides the VOSI URL to the registry.  The registry retrieves the 
VOResource description.  At that time the registry decides whether it 
wants to retrieve the table metadata.

In either case, the registry has access to the table metadata if it wants 
it, but if it doesn't want it, it doesn't get it anyway.

>> to see is that all of the table information be retrievable in one URL.
>
> That is certainly more reasonable. But what I don't understand is why, if 
> we're to have a single method of getting the metadata, it cannot be returned 
> in the format recognised by the registry: VOResource? It would take no more 
> work to format the metadata as VOResource than some other format so why not 
> stick with what we have, why come up with another format?

I hope you noticed my suggestion that the VOResource table schema could 
provide the format because we are already using it.  This is my 
preference.

The issue is not about another format.  It's about providing access to the 
information that is not exclusively through the registry.

> But, as I said previously and above, this will lead to registries serving 
> information in ways no other does and so, applications which rely on that 
> information will only be able to work against specific registries.

Are you going to force us to do registries your way?  We need 
interoperability, but this does not mean our registries must behave in 
identical ways.  We need--you need--to be able to explore new ways to 
support discovery and access that best suits the needs of our respective 
communities without sabotaging each others efforts.  That's what I'm 
looking for.  Do you agree with this?

>> To put it in concrete terms, the AstroGrid registry effectively
>> pressures the NVO registries into supporting fine-grained table
>> ...  First, you
>> encourage your publishers to provide table metadata.  We harvest these
>> records which in turn go out to our users as a result of queries.  We
>> have to then help users make sense of this information.
>
> I don't understand this - the information is the same whether you get it from 
> the registry alone or from the registry plus the service. What is the 
> difference?

If the fine-grained information is in our registry to begin with, then we 
do not have to deal with the metadata problems within the context of the 
registry.

>> When there are
>> problems with the information, it reflects poorly on us, not you.
>
> Poor metadata information is a problem we all have to tackle, not just 
> AstroGrid: it is exacerbated by poor registry population applications and has 
> nothing to do with the type of registry.

Do you see that we have to bear some responsibility for the metadata you 
export from your registries?  And vice-versa?

>> Second, your application in effect encourages our publishers to provide
>> table metadata to our registry if they are to be used in your
>> application, because your application only gets this information from
>> the registry.
> This is not going to change, however the metadata is collected, whether by 
> entry into the registry, by VOSI:getWhatever from the service or by the 
> metadata URLs you propose. We will continue to provide full registry 
> information and applications that rely on it: it is the only effective way to 
> build responsive applications.

I want you to be able to do this.  I want you to have a way to get the 
information you need from our services without requiring that you get it 
directly from our registries (via the harvesting interface).

> It is also, I might add, the only way to discover resources based on the 
> additional metadata. If someone wants to discover x-ray catalogs with a given 
> type of informaiton (specified by a ucd), how does it do this from a 
> pointer-only registry, apart from getting every possible x-ray source and 
> querying every one of them?

It is the only way today.  NVO is working on other techniques for drilling 
down into the information for discovery outside the context of the 
Registry.

> I guess if we do follow your proposal it will, over the next couple of years, 
> show what the application developers really want by the number that tie 
> themselves to AG-style registries vs NVO-style registries.

That is fine.  We only ask for the freedom to discover this ourselves.

> I'd like to expose more to the list since I won't be at the meeting.

(I'm hoping to capture it in a Note.)

cheers,
Ray