discussion status & my recommendations

[Ray Plante]

Hi RWGers,

With the interop meeting quickly approaching I thought it would be good
summarize the status of currently unresolved issues with the Registry
Interface (RI) standard and make some recommendations so that we can move
forward.

Given the state of the discussion, I think we should be aiming at a v1.0 
Working Draft for RI rather than a Proposed Recommendation (it's 
currently an internal WD).  Once we have confirmed that we all have 
working implementations, we should elevate it to PR status.  Changes we 
make as part of that transition (which could include some of the 
suggestions below that I am recommending we table) should be based on our 
working experience.  

1.  harvestedFrom

It was suggested that we add this as an attribute to the base Resource 
type and would be used to help track problem records.  It was meant to 
address an ambiguity when tracing a record that may have made multiple 
hops.  Some registries keep this information already internally.  By 
making this part of VOResource it would expose it through our standard 
interfaces.  

I should note that in the discussions, it was not made clear exactly what 
the value should be, an IVOA registry identifier or an OAI URL base 
(current implementations use the latter).  

Recommendation:  Given the lack of clarity of its usefulness and what the 
value should be, I think this suggestion should be tabled for now.

2.  registry record curation/stamping

This thread started with the concern that there would be parts of a
VOResource record that a particular registry could update (as opposed to
the original publisher of the record) and that, subsequently, VOResource
records would not be identical regardless of where it was retrieved.  The
registry-updatable information would include "harvestedFrom", 
"verificationLevel", and possibly "status", all encoded as attributes to 
the base Resource type.  

The above concerns led to the suggestion to pull this information out of 
the VOResource record and into a special sibling node called something 
like "stamping".  The "stamping" part and the VOResource part 
would be combined to be our registry record.  This would allow this 
stamping information to be used to select resources in a registry 
search.  The motivation for the "stamping" node is to clearly 
separate what is provided by the publisher and what is provided by the 
registry.  It also provides a place for other registry-updatable 
information (e.g., "liveliness").  

Recommendation:  I recommend that we table the creation of the "stamping" 
node.  We added "verificationLevel" as an attribute to the base Resource 
type but not "harvestFrom" (see #1), and we indicate that "status" only be 
updateable by the original publisher.  

Although the "stamping" node is a very nice idea, it represents a major 
change to our metadata standard.  The cost of updating our 
implementations would be high--too high for something that has not been 
prototyped, yet.  Also, it opens the door to add lots of other information 
that hasn't be sufficiently motivated as being needed nor prototyped and 
demonstrated.  

If a "verificationLevel" is the only thing we allow to be 
registry-updatable, we can add it as an optional attribute and have 
minimal impact on the current implementations.  We can see how this simple 
adjustment works out to see if a separate "stamping" node is needed.  
Meanwhile, if we want to start prototyping a "stamping" node, we can do it 
using a separately defined (OAI) format.  

3.  verificationLevel

This is a new item that is proposed to be added as an attribute (see 
also #2).  Fully defined in the v1.1 of the RM (WD), it is intended to 
indicate a level at which the metadata (and service interface, when 
appropriate) has been checked to be "good" according to the registry 
holding the record.  Some of the criteria for this evaluation is set by 
the registry.  A registry may choose to preserve the value it finds in the 
harvested record or override it according to its own evaluations.  
Users/applications may use this attribute to select resources as part of a 
search.

Recommendation:  I recommend we add this to VOResource as an attribute 
to the base Resource type.  This is critical for putting into place good 
registry curation practices which is a major area of focus now that we 
have some operating registries.  

4.  ADQL => RQL

The RI current specifies restricted forms of ADQL (only a Where node is 
given) and XPath to enable constraint-based searching.  The need for these 
restrictions and presence of XMatch in ADQL, which is not needed by 
registries, motivated a suggestion that we branch off from the ADQL and 
maintain our own Registry Query Language based on ADQL.  

Recommendation:  We should not yet break off from ADQL for v1.0.  We 
should update RI to use ADQL v0.91.  

The issue of how our standards should depend on each other is a profound 
one that we can't fully understand until more of our standards become 
stable (i.e. reach rec status).  I recommend that we build to ADQL v0.91 
and reexamine this issue based on our experience before we go to PR 
status.  

--------

Your comments, as always, are welcome.  In this last week before Kyoto, we 
should concentrate on things that help us wrap up the RI specification.

cheers,
Ray