The IVOA in 2006: Assessment and Future Roadmap - Registries

[Paul Harrison]

On 08.06.2006, at 19:52, Gretchen Greene wrote:

> To expand a little in the direction of xml mapping to relational DB
>
> for the RDBMS at STScI, we will be upgrading the registry backend with
> the schema changes.  There is advanced XML support including  
> Xquery. As
> long as the XML structures are well-formed,  the columns can be mapped
> to xml types (also Un-typed,  but that's not as ideal for query
> performance).  Just as years ago the RDBMS adopted the ootypes in a
> hybrid fashion,  the xml trends are influencing the implementations.
>
> While the xml schemas can be standardized,  I'm not sure how much
> overlap there will be in the table mapping for different DB
> implementations yet we are planning to discuss with ESAC the
> capabilities ...
>
> So some of the issues with mapping and flattening will depart from the
> classical thinking and experience.
> It's not simply treating xml blobs vs. hierarchical storage vs. flat
> tables,  the features will be combined and the advantage of course is
> optimal query performance and load times.
>
> -Gretchen
>
>

This is good news as there has been a difference between the  the  
registry implementations up to now. Try searching for the resource  
(which is published in the AstroGrid registry) with identifier ivo:// 
org.astrogrid/SExtractor which is the CEA definition (a non-standard  
registry schema extension) of the web service wrapped SExtractor in  
http://nvo.stsci.edu/voregistry/ - it is not there, but it is in the  
http://nvo.caltech.edu:8080/carnivore/ registry. The essential  
difference being that the STScI registry is RDBMS based with and  
Carnivore is native XML based - the STScI registry has a static  
mapping between the table structure storage which does not include  
the CEA schema, so rejects CEA types when harvesting. Whilst I agree  
with Brian that the mapping approach between the data model and an  
underlying RDBMS structure is the right approach for established  
static data models, the registry is a slightly special case as it has  
always allowed the extensions to core model (a 'good thing' as it  
fosters innovation = don't have to wait an eternity for a standard to  
be published before being able to publish information about a new  
type of resource) - for this to happen the mapping technology needs  
to be dynamically able to cope with new objects, and it is my  
assertion that up till now this has not been the case - the mapping  
has been created statically at registry build time. It is great that  
the RDBMS vendors are adding 'native' XML support to their products  
as this should allow us to have the best of both worlds.

Incidentally doing the exercise of trying to search for ivo:// 
org.astrogrid/SExtractor in various registry did reinforce another of  
my original points about the registry implementations as far as end  
users are concerned - i.e. a single standardised User interface that  
could talk to any of the registries would be a great help....

The user interface for each of the registries is completely  
different, and it can be difficult (even for someone like me who  
knows the registry schema well) to formulate the query that they want  
in the different interfaces

* ideas of "quick search" and "advanced search" differ between the  
interfaces
* quick search does in most cases just allow you just to specify a  
few keywords, but it is not clear whether the search scope is over  
the full text of the registry record, or over selected fields
* the ideas of advanced search vary - sometimes the user interface  
allows you to perform selections for certain fields from lists,  
sometimes you need to type to full text of the query.
* for the advanced text input style queries the exact syntax of the  
query language varies
      STScI   -     identifier like '%SExtractor%'
      ESAVO -  Where #identifier# like '%SExtractor%'
     AstroGrid - Select * from Registry where vr:identifier like '% 
SExtractor%'