RofR extended validation changes

Theresa Dower dower at stsci.edu
Tue Mar 12 20:43:29 CET 2019 

Hello Registry!


Catching up on discussion about validation in the RofR, I went over the additional validation checks that it runs against registries apart from the main OAI-PMH 2.0 and VO resource .xsd standards validation steps.


I only see a couple of changes we should make *given the current snapshot of standards*, one of which has been temporarily done already. I'm posting this to the whole list not just for transparency but in hopes folks will double-check this before we revisit it at the May 2019 interop and weigh in.


Proposed changes (and proposed lack of changes) are below.

--Theresa




Additional RESOURCE validation:

Change:

- Registry resource should have (at least) one capability with either xsi:type='vg:Search' or xsi:type='vg:Harvest' (temporarily removed, needs to include RegTAP)

Change this to xsi:type 'vg:Harvest' or 'vg:Search' or a capability with standardID='ivo://ivoa.net/std/TAP' including dataModel ivo-id='ivo://ivoa.net/std/RegTAP#[ANY NUMBER]'


Add:

- Recommend setting VOResource xsi:type='vs:CatalogService' on service with SimpleSpectralAccess capability
- SimpleSpectralAccess capability must include interface with xsi:type='vs:ParamHTTP' and role='std'


These are fine:

- VOResource record should have a status attribute
- VOResource record should have an updated attribute
- VOResource record should have a created attribute
- Recommend setting VOResource xsi:type='vs:CatalogService' on service with SimpleImageAccess capability
- SimpleImageAccess capability must include interface with xsi:type='vs:ParamHTTP' and role='std'
- Recommend setting VOResource xsi:type='vs:CatalogService' on service with ConeSearch capability
- ConeSearch capability must include interface with xsi:type='vs:ParamHTTP' and role='std'
- Recommend setting VOResource xsi:type='vg:Registry' on service with a Search or Harvest capability
- Harvest capability must include interface with xsi:type='vg:OAIHTTP' and role='std'
- Search capability must include interface with xsi:type='vr:Webservice' and role='std'
- Recommend setting VOResource xsi:type='vs:CatalogService' on service with SkyNode capability
- SimpleImageAccess capability must include interface with xsi:type='vr:WebService' and role='std'
- Service resource should have at least one capability element
- capability element should have at least one interface element
- The date for the created attribute must be in the Past
- The date for the updated attribute must be in the Past
- The Registry must not serve records authorities it does not claim in its Identify response ("managedAuthority")
- A harvesting registry must support version 1.0 of the OAIHTTP interface.

(I am against adding further TAP capability checks here that aren't already checked via the schema (as per Cone, SIA, etc), as there are some changes between 1.0 and 1.1 that I don't want us to have to worry about staying in sync with right now)




Additional REGISTRY validation:

Change:

        - Registry resource should have (at least) one capability with either xsi:type='vg:Search' or xsi:type='vg:Harvest'

This was in both lists provided by the RofR; make it match the above.



These are all fine:

- Look for oai:error. Service must respond to a legal OAI command and support the ivo_vor metadata prefix and the ivo_managed set.
        - Identify/baseurl must be equal to the URL being tested
        - Identifier must include a description containing a Registry Resource record
        - Registry record must have an element name of 'Resource'
        - Recommend using Resource from the RegistryInterface schema (http://www.ivoa.net/xml/RegistryInterface/v1.0) for full OAI schema compliance
        - A Harvesting Registry should declare a Harvest capability
        - A Harvesting Registry must return VOResource records when metadataPrefix=ivo_vor
        - When metadataPrefix=ivo_vor, each VOResource must be wrapped in a ri:Resource element.
        - The date for the created attribute must be in the Past
        - The date for the updated attribute must be in the Past
        - VOResource record should have a status attribute
        - VOResource record should have an updated attribute
        - VOResource record should have a created attribute
        - A harvesting registry must support version 1.0 of the OAIHTTP interface.
        - Recommend setting VOResource xsi:type='vg:Registry' on service with a Search or Harvest capability
        - Harvest capability must include interface with xsi:type='vg:OAIHTTP' and role='std'
        - The OAI record/header/identifiers must match the VOResource identifier.
        - Service must respond to a legal OAI [verb] query.
        - ListRecords must include the record for the registry being tested.
        - ListRecords must include an Authority record for all authority IDs
        - ListMetadataFormats must declare support for the ivo_vor format
        - Recommend setting schema to http://www.ivoa.net/xml/VOResource/v1.0 for the ivo_vor format
        - Recommend setting metadataNamespace to http://www.ivoa.net/xml/VOResource/v1.0 for the ivo_vor format
        - ListSets must declare support the ivo_managed set
        - A registry cannot declare new set names that begin with ivo_










________________________________
From: registry-bounces at ivoa.net <registry-bounces at ivoa.net> on behalf of Theresa Dower <dower at stsci.edu>
Sent: Tuesday, December 4, 2018 11:23 AM
To: registry at ivoa.net; Janet Evans
Subject: Action items for registry maintainers / interested parties re: RofR meeting IVOA College Park interop



Hello Registry!


First, my apologies for the delay on getting back to this. A lot of much-appreciated work was done before and at the RofR operations meeting held at College Park, by Janet, Yulie, Menelaos, and Euro-VO registry maintainers. We are all working  toward an error-free whole registry ecosystem for the RofR, to provide good-quality metadata the IVOA can be proud of, and to make inclusion in the RofR a much easier process for the people who have to maintain it and the registries in it.


For those not in attendance, several people involved in operations for searchable and publishing registries, registry validators, and with the Registry of Registries, met to address long-standing and widespread problems with validation of registries and inclusion in the RofR. Yulie's excellent and detailed notes are attached. I'll also be forwarding some of the resultant discussion.


In summary, we decided to tackle some of these issues for the May 2019 Interop, where we will report on progress and further discuss approaches for handling various levels of validation failure (OAI, resource authority provenance issues, resource XML, etc). Known issues exist with individual resources, with outdated standards schemas the validator uses, and with some registry publishing service software. There is a little bit for everybody to work on.  I am currently working on errata and issues within the NAVO searchable registry at STScI to keep it fresh in my mind. I've also tentatively started a page linked in the wiki under RegistryOperations; time will tell if we use this and actually updated the Ops guide or remove it. (https://wiki.ivoa.net/twiki/bin/view/IVOA/RegistryOperations)


Action items:

Everyone interested - participate in discussion, come to the Interop!

Anyone who has written/edited VOResource and extension updates, RofR folks, kind Registry citizens - review the extra rules the RofR uses for validation (attached) for outdated checks.

All publishing registry maintainers - please run the validator at http://rofr.ivoa.net/regvalidate and clean up what resource and software issues we can. Discuss remaining problems at next Interop, especially if issues are found in open software that runs multiple registries. I'll send out reminders in the months leading up. Feel free to contact Menelaus or the list if a validation error is not easily understood.

Menelaos Perdikas - field questions from maintainers having issues with the registry, provide stack traces as needed.

Theresa Dower - write errata for VOResource ivoid to include #fragments, an oversight in the most recent version's xsd. This causes validation failures where there should not be any.

Yulie Zografou - once errata and any other schema document changes are made, include the new version in the RofR's validation suite.




Thanks so much to everyone for the work that's already gone into this,


--Theresa Dower

Archives, STScI

________________________________
From: Zografou, Panagoula <pzografou at cfa.harvard.edu>
Sent: Wednesday, November 14, 2018 1:09 PM
To: Janet Evans; Theresa Dower; Pierre Le Sidaner; Christophe Arviset; Kostandin Vrioni; Juan Gonzalez; Markus Demleitner; Menelaos Perdikeas; Harbo, Peter
Subject: RofR meeting notes

Hi All,

Please see the attached notes. Two lists of additional validations referenced in the notes are also attached.

Yulie

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.ivoa.net/pipermail/registry/attachments/20190312/672c722a/attachment-0001.html>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: additional-registry-validations.txt
URL: <http://mail.ivoa.net/pipermail/registry/attachments/20190312/672c722a/attachment-0002.txt>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: additional-resource-validations.txt
URL: <http://mail.ivoa.net/pipermail/registry/attachments/20190312/672c722a/attachment-0003.txt>

More information about the registry mailing list