Alternative proposal

[Paul Harrison]

On 11.05.2007, at 00:44, Robert Hanisch wrote:

> I _think_ things are going in a reasonable direction, but if I  
> agree with
> Tony there must be something wrong.  ;-)
>
Despite all the noise I think things are actually very little changed  
from before....or that is the impression I am getting i.e. it is  
still legal in the registry to have a "fine-grained" view or not, so  
clients will need to pick their registry carefully.

the only issues are that

  1. we do not have a standardized way of finding out the "fine- 
grained" metadata across all service types.
  2. there needs to be a complication to the harvesting mechanism to  
avoid harvesting "fine-grained" metadata (and it is not at all clear  
what is meant by "fine-grained").

> I think we need the current level of resource metadata, as defined  
> in the RM
> document.  Anita was in accord with this.  This is core, and is  
> sufficient
> for discovery, location, and access.  This is what it was intended  
> to do.

I dispute that the core metadata are sufficient for access. In fact  
that is the nub of this whole "fine-grained" argument. The proposal  
for TAP is that you have to ask the service for more metadata before  
you can actually access the data.

>
> Tony said that a registry of core metadata would be of " little use to
> scientists or applications."  Perhaps we are talking about  
> different cores,
> but based on what we have now, I most vehemently disagree.  When we  
> have
> properly populated, well curated resource metadata we have a  
> tremendous
> asset for astronomers and applications, and we have already built a  
> number
> of useful applications based on this.

By core I think that he means the metadata that are formally  
contained within the VOResorce.xsd schema. They are sufficient for  
searching human readable descriptions of the data, so that they are a  
great resource for finding out what is available - "find all the TAP  
services", but not for making a distinction between the services  
"find all TAP services with a redshift column".  Intriguingly the  
current draft of VODataService-v1.0.xsd extension schema does have  
structures to register the basic TAP fine-grained metadata that would  
allow the second query.

>
> Anita, what I meant when I said that "the simple stuff is not done  
> well or
> completely" is that resource providers don't do a very good job of  
> entering
> metadata.  They have poor tools (our fault), astronomer-unfriendly
> documentation (our fault), and are well-intentioned (usually) and  
> lazy (not
> our fault).  As a result, relevant resources are not always found, and
> irrelevant resources show up (e.g., something claims to be a SIAP  
> service
> but it is not).  I came across many many situations like this  
> during an
> intense period of testing ~2 months ago.  Our resource metadata is  
> in poor
> shape.  Resource and service providers, I found, were eager to fix  
> things
> once they were aware of the problems.  But reviewing and testing  
> all of this
> is very labor intensive.
>
Indeed if the software components and the data already exist, the  
only work left is to define the metadata properly, and we are all  
agreed that it is an arduous job. However, part of the motivation for  
suggesting the VOSI is that then the service "owns" the metadata - it  
becomes central and integrated into the functioning of the service,  
and even better if the service writers can be persuaded to configure  
the service itself using the same metadata - then the registry  
description of  and the functioning of the service can never out of  
line - CEA takes this approach.

Cheers,
	Paul.

p.s. As an amusing aside, this hit the German news recently (http:// 
www.dw-world.de/dw/article/0,2144,2483499,00.html) - nothing to do  
with metadata, but it illustrates the problems of a big data  
reconstruction effort where the computers do the mind-numbingly  
tedious part before the humans do the clever bit - which is sort of  
what we want the VO to achieve. The analogy with the registry  
functionality is that with the current registry, you can find all the  
pieces of paper that  belong on the same page, but it does not fit  
them together.


Paul Harrison
ESO Garching
www.eso.org