Feedback on Harvesting

[Ray Plante]

Hi Tony,

You've seen my summary that was input to your WG report.  (For others, 
see http://www.ivoa.net/forum/registry/0813.htm, particularly the second 
paragraph.)  

You might also have a look at my posting on "OAI coordination" 
(http://www.ivoa.net/forum/registry/0779.htm) in which I enumerate the 
open interoperability issues related to OAI as of 25 Nov.  I believe we 
came up with resolutions for these--either on this list or within NVO for 
the purposes of our demo.  Let me attempt to enumerate them here.  (The 
numbers correspond with those in my original list.)

  1. The OAI metadata format name we agreed on for the VOResource schema 
     is "ivo_vor".  (This name has no bearing outside the context of OAI.)

  2. The "root" element for VOResource metadata within an OAI response 
     envelope should be <VOResource>.  More specifically, the 
     <oai:metadata> element should contain a single <VOResource> child.

  3. The value of the <oai:identifier> element should be an IVOA 
     identifier in the URI format.  

Other bits:

  *  All implementations should emit at least one Registry record.  This 
     record describe the registry itself and should include a 
     <ManagedAuthority> element for each AuthorityID that originates from 
     that registry.

  *  All implementations should emit one Authority record for each 
     AuthorityID that originates from that registry.

While the NCSA registry conforms to these, I don't think we have 
conformance across all known implementations.  Despite the small 
differences though, we were still able to harvest.

On Thu, 29 Jan 2004, Tony Linde wrote:
> >From informal feedback, I understand that the standard we adopted was not
> detailed enough to develop an interface. So, what worked, what didn't work,
> what was easy, what was difficult, what do we need to specify to make an
> implementation unambiguous?

On our end (from the feedback I got), the difficulty was not so much with 
OAI as with dealing with our schemas (e.g. generating valid VOResource 
documents).  

> At the end of the feedback, please add recommendations that will help us
> revise the standard.

The are a variety of items regarding the schemas that I have received, but 
those should be discussed independent of the harvesting topic.  

My recommendations specific to harvesting, assuming we are sufficiently 
happy with OAI, are:

  o  We should start drafting a list like above as part of a 
     standard for IVOA harvesting.

  o  We should prototype a Web Services version of the OAI interface, 
     starting with a WSDL document.

I'm of the understanding that AstroGrid did not put up an OAI interface.  
Is that correct?  (That is, Wil grabbed records via Kevin's Web Service.)
I'd be very interest in hearing Kevin's take on harvesting and OAI.

hope this helps,
Ray