getCapabilities() and graininess

[Ray Plante]

Hi Tony,

On Wed, 9 May 2007, Tony Linde wrote:
> I know you want this discussion to stop on the list and continue in Beijing,
> but I need it to continue as long as you're accessible. Neither Kevin nor I
> can be in Beijing and we need to brief those who will be there on the affect
> your proposal might have on AstroGrid, so I'll have to keep digging I'm
> afraid (does your flight have wireless? :) ).

I'm running out of daylight here, but I'll see what I can do.

>>    o  At the heart of the fine-grained metadata issue is the method we
>> use to share metadata: registry harvesting.
>
> So why are we talking about changing the schema? Why not just change the
> harvesting spec: instead of a getMetadata method (or whatever it is), add a
> getCoarseMetadata method so that those registries which do not want to
> maintain the fine-grained metadata do not need to.

My suggestion involves adding one URL.  It does not require removing 
anything; however, once the URL is in place and used, it requires a 
lady-and-gentleman's agreement among our registries to prefer the URL over 
explicit inclusion of table metadata in the records we harvest.

Clearly this is a much less intrusive solution than changing the 
harvesting spec.

Please recall we are using the OAI standard for harvesting.  In principle, 
we could define another format type that limits the information in the 
records returned; however, this has problems:

   1)  We have no mechanism for identifying and controlling what
       information gets put into the coarse harvest and into the fine
       harvest.  This is the same problem as with the VOSI interface.

   2)  It doesn't address how AstroGrid gets table metadata from NVO
       resources, since we are not keeping that information in our
       registries.  That's the 2nd term in the pressure equation: if we
       want AstroGrid users to use NVO resources in the query builder, we
       have to provide table metadata from our registries.

   3)  Finally, it's an all or nothing affair.  No registry can choose what
       fine grained information they want.

>>    o  We need a common understanding of what fine-grained means.
>
> Why not leave this to the people who define the service standard: so whoever
> defines SIAP, SSAP, TAP etc can define the fine and coarse level of metadata
> (if necessary - they might be equivalent), depending on the needs of those
> who use the services.

Clearly from the opinions raised regarding the role of registries, it 
is not simply a DAL issue; registries have a role in this as well. 
It's the curation of the metadata in the registries that has folks 
bothered.  And even in the DAL and VOQL groups there is division about 
whether table metadata should be managed by the registry.

> But the definition of it MUST remain part of the IVOA standard for a tabular
> data service. To go back on that after the months of tiger team and
> workgroup wrangling to derive the standard cannot be right. It would render
> all the registries and applications that rely on it non-standard and thereby
> ineffective in the eyes of other apps developers - no-one could countenance
> that.

As mentioned above, I am not calling for the removal of the table-related 
terms in the schema.  Leaving them in there will allow you to continue to 
manage your records and deliver them to users just as you do now.

cheers,
Ray