RofR meeting notes

[Theresa Dower]

________________________________
From: Markus Demleitner <msdemlei at ari.uni-heidelberg.de>
Sent: Monday, November 19, 2018 5:20 AM
To: Zografou, Panagoula
Cc: Janet Evans; Theresa Dower; Pierre Le Sidaner; Christophe Arviset; Kostandin Vrioni; Juan Gonzalez; Menelaos Perdikeas; Harbo, Peter
Subject: Re: RofR meeting notes


> On Wed, Nov 14, 2018 at 01:09:46PM -0500, Zografou, Panagoula wrote:
> > - Should we re-harvest all registries, by-passing the validator?

> There's not much point running the whole enrollment procedure for
> every update.  What needs to be done, though, is regularly
> re-harvesting the registry record itself.  I *suppose* just going for
>what comes back from verb=Identify would be enough, but I'd really
> prefer trying verb=GetRecord&identifier=<registry-id>, as that
> would catch more subtle failures.

> If what's coming back there fails some basic validation, that's a
> reason for concern and should probably block updates to the RofR.

>The most important thing to check, I think, would be if a registry
>claims someone else's managedAuthorities.

Off the top of my head, the only thing in the RofR that could update and affect an individual registry owner's validation status outside of their own local updates are

1: new standards updates, which could validate previously in-valid records, which doesn't happen often
2: other registries' managedAuthority changes, which, well, might?

Therefore I think while ongoing validation monitoring by third parties (whether it be the RofR or the Euro-VO weather report, or something else) is a good thing, a good low and useful bar would be to periodically check the Identify record for validation and managedAuthority issues, with manual followup as Markus mentioned.

>> - Should we set a time by when all currently registered registries
>> must succeed validation and after that any new or updated
>> registries will be required to comply?

>While it would be useful to impress the importance of correct
>registry records on the operators, I suppose there's no real need to
>be heavy-handed.  The searchable registries deal ok with the sorts of
>minor failures that are fairly common.

>I like Juan's proposal of sending, perhaps twice a year, a text
>version of the validation report to the contact addresses of the
>registries.

I believe at the meeting consensus was leaning towards something like the latter, out of concerns being too heavy-handed on validation would knock very useful and compliant services in registries with unrelated issues out of search listings, and since we still have issues with the validation itself this isn't the time anyway. So, perhaps more importantly we came to a consensus about RofR and other maintainers of resources and software doing our best-effort by the next interop, seeing where we are then and what intractable problems remain (including technical issues but also support level for various registries' operations), and letting that inform the process moving forward.

>> - For resource XSD validation, it was agreed to keep it and also
>> provide lower level errors like a stack trace.

>Is that really useful?  We should, anyway, make sure the reports
>don't become even more hard to read than they already are.

>What I think would be helpful would be a reviving and reviewing the
>explanations for the various errors (the ones you get when you follow
>the links under the error codes; right now, examples look like
>http://rofr.ivoa.net/regvalidate/not_available.htm#OAIvalid<http://rofr.ivoa.net/regvalidate/not_available.htm#OAIvalid>).  I'd
>volunteer for having a go at what *I* understand if you tell me where
>what's still there is.

With my registry devops hat on, there are cases where I'd like to see the stack trace or just more error information in general at the bottom of the links under the error codes. Some of them were really not obvious, though a few minutes of explanation by Menelaos cleared it up. I agree we shouldn't just dump the whole thing on the user in what could be an already crowded page.

>> - There is a discrepancy in some IVOA documents that causes some of
>> the XSD validation failures. Standards documents give examples of
>> ivo-id's that include a '#' but VOResource-v1.0.xsd or

> These should be fixed in later versions or errata.  If you find
>anything that's still wrong, let me know.

This particular errata is on my todo list, though someone can pick it up if they want to!



>>         - A Harvesting Registry must return VOResource records when metadataPrefix=ivo_vor

>This should be "Publishing Registry", right?

(I'm assuming yes here, but possibly arguing below that searchable registries also must have an endpoint for this)

>I actually have an issue with this requirement (and it's still being
>imposed) on searchable registries.  For these, it may be perfectly
>sensible to have an oai-pmh endpoint without any records of its own;
>the ones associated with my RegTAP services are examples.  I'd like
>it a lot if only publishing registries were requried to actually have
>at least one record.

>Incidentally, I don't register these OAI-PMH endpoints for exactly
>that reason.

>>         - Registry resource should have one capability with either xsi:type='vg:Search' or xsi:type='vg:Harvest'

>That one should be dropped.  RegTAP registries, for instance, can
>perfectly work with just a TAP or auxiliary TAP capabilitiy.


>>         - ListRecords must include the record for the registry being tested.

>That one should also only be enforced for publishing registries, not
>of OAI endpoints of searchable ones.


I'm either misunderstanding or disagreeing with a key point in this bit about RegTAP. IVOA registries are, on a fundamental level, built on the OAI-PMH design, and the concept of a searchable registry is built on top of a publishing one, whether it has any records beyond Identity (and associated authority and/or organisation, etc) of its own or not. Even a searchable registry with only RegTAP and basic OAI endpoints should have self-reflective resources (i.e. identify) for contact information, and an authority record would go with it. And we need to serve up SOME VOResource record in ivo_vor format (i.e. at least a vg:Harvest capability) in that case; a RegTAP query can't fully construct that XML resource. Are you arguing that those bookkeeping records don't count, or don't have to exist? Or does this come from having the RegTAP service resource be separate from what you would register the OAI one as if it could be registered without records to list? Also this seems to go counter to the comment above suggesting we pull from GetRecord instead of Identify. I know I'm missing something here, can you explain a bit more?


Thanks for writing this up and sorry it took me a bit to get to, hopefully we can spin this discussion, and the work to resolve the issues, started back up.
--Theresa


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.ivoa.net/pipermail/registry/attachments/20181204/25d768de/attachment.html>