Registry Interfaces 1.1: WG comments?

[Theresa Dower]

-----Original Message-----
From: registry-bounces at ivoa.net [mailto:registry-bounces at ivoa.net] On Behalf Of Markus Demleitner
Sent: Monday, August 31, 2015 4:17 AM

>  I've built the document, and all is well -- the document with references is at http://docs.g-vo.org/RegistryInterface.pdf

Much obliged, thank you.

> I'd much rather have a generic URI that would be mapped by ScriptAliases or whatever URI mapping mechanisms future servers have.  I'd propose

>  http://rofr.ivoa.net/oai

> -- what do the current maintainers of the RofR think?  Would you see an issue with adding

> ScriptAlias /oai <path-to-wherever>/cgi-bin/oai.pl
 
> into your apache config[1]?

> Similarly, I'd much prefer http://rofr.ivoa.net/add-registry (or 
> similar) to http://rofr.ivoa.net/regvalidate/regvalidate.html -- which again contains a bit too much implementation detail.

I agree with Markus and Norman here; if this is acceptable to the RofR maintainers, the simpler, configurable, back-end-agnostic URLs would be a marked improvement and one I'd be far more comfortable including directly in the document as 'well-known'.


> (2) The current text has

>  The Registry of Registries includes the VOResource records directly
>  representing each currently active registry of IVOA resources, be they
>  fully searchable or providing only an OAI-PMH harvesting interface.

> -- this is a problem I've been thinking about for a while now without having stumbled on a deep and satisfying insight.  While I believe it makes a whole lot of sense to have > the searchable registries in RofR, too, with current RegTAP these don't have a specific registry record any more, let alone a vg:Registry-typed one.

> Right now, RegTAP registries are just TAP services with a proper data model declaration.  Pulling these into the RofR is probably not a good idea (there can be, and already
 > is in some cases, lots of other junk in the services), plus of course these records are not vg:Registry-typed.

> I could well imagine just putting in a TAP capability into a vg:Registry element, plus perhaps the relational registry table schema.  In that case, we'd have to change the 
> annotation on vg:Registry in the schema, which currently says:

>  A registry is considered a publishing registry if it contains a
> capability element with xsi:type="vg:Harvest". It is considered a
> searchable registry if it contains a capability element with
>  xsi:type="vg:Search".

> With the proposed change, this would have to be something like

>  A registry is considered a publishing registry if it contains a
> capability element with xsi:type="vg:Harvest". Other capabilities
> may be given, including the legacy vg:Search (for the old, RI 1.0
> search interface) or a TAP capability (Dowler et al...) with a
> dataModel http://ivoa.net/...RegTAP... for a RegTAP-searchable
> registry.

> I'd feel a lot more relaxed with that if we implemented proper XSLT for that on the RofR first.  Another thing I really don't like about this is that so far, endpoint discovery
> usually works via a standard id, not via a resource type.  This stuff would be the first excpetion.

> Hence: Good ideas solicited.

> Anyway, whatever we do for extra, stand-alone RegTAP records would have to be specified in RI for the time being -- we don't want to wait for RegTAP-Next with that.  
> When we reach a consensus on what to do here, I'll happily contribute.

I think adding potential RegTAP capability into the vg:Registry record is cleanest here.  RegTAP capabilities can also be part of general TAP resources with many unrelated tables. I don't think it's problematic for that to be redundant.

Either way, we also need to clean up, from the vg:Registry xsd for Search:
<xs:element name="optionalProtocol" type="vg:OptionalProtocol"
minOccurs="0" maxOccurs="unbounded">
<xs:annotation>
<xs:documentation>
the name of an optional advanced search protocol
supported.
</xs:documentation>
<xs:documentation>
Only one optional protocol is currently allowed
(XQuery). It is assumed that the required protocols
(simple keyword search and ADQL) are supported.
</xs:documentation>
</xs:annotation>
</xs:element>

....and, importantly, also leave room for the upcoming simple keyword search interface.

Good ideas solicited.


> (3) The current text has 

>   Upon successful validation, the publishing registry will be
 >  automatically included in the RofR listing...

> -- which I believe is not what happens nor what should happen.
> Developers are welcome to validate their Registry often and early without having to fear premature publication (I, for one, was happy I could do that and still can with my 
> development site).  I've already changed that to

>  Upon successful validation, the publishing registry may be
>  included in the RofR listing at the user's option,...

> In Rev. 3051. Agreed?

Agreed. It has been a long time since I ran it myself; thanks for the correction!


>  (4) The current text has:

>  ...updates to the Registry record itself are not automatically
 > reflected in the RofR contents.

> ...which has bugged me for a while now.  I am fairly convinced the whole system would work better if the RofR went out to re-havest the records of the publishing 
> registries and warned somebody (I'd volunteer for the duration of my term) when things break (e.g., two registries claiming the same authority).

> I don't think this spec is a good place for laying down such operational aspects (or is it?  After all, this is all about the operation of the VO Registry... hm), but at least we  
> shouldn't indicate that you can expect to get away with anything once you are in the RofR.  Hence, I'd vote for striking this.

It bugs me too, especially as an operator with occasional changes to our vg:Registry record; there have been times our local copy gets out of sync with the RofR copy, especially when adding new managedAuthorities or cleaning up deleted/migrated ones. I would appreciate it if the RofR would re-harvest identity records on a known, regular basis, but I'm not the RofR operator who would have to do the work.

I do want to keep the caveat about no further automated validation <strike>currently being done</strike>guaranteed by the RofR: I hadn't so much considered it indicating that you can expect to get away with anything once you're in the RofR as warning potential harvesting clients that  registries listed in the RofR, despite initial validation efforts, cannot be assumed to be entirely valid at the time of harvest. Instead of omitting the text, I think your note (5) would serve the purpose of additionally warning registry operators they don't get a pass after initial inclusion: we should also mention separate ongoing validation efforts throughout the IVOA (US VO/HEASARC and VO-Paris)  with a process for IVOA-Exec-requested deletion of old invalid records. 

> (5) In a similar vein, I believe we'd help the RofR operators if we indicated somewhere under which conditions records will be removed from the RofR.  I'd roughly say:
>  When a Registry produces invalid content or announces dead services as active for X months and the operators haven't responded to at least three communication 
> attempts, their registry will be removed from the RofR as dead.  Or something like that.  I realise that's a highly political issue, so we should, in TCG review, say, explicitly
> point to this question.

Right. While we keep coming back to this issue on different scales and with different timelines, we have at Sesto already once gone through a round of IVOA Exec action for the deletion of individual bad records throughout the registries. I'd propose the same standards and timeline for de-registering the RofR registry record itself.  However, even for individual records this process doesn't appear in any of the Registry documents or its own official note (as far as I can find skimming quickly), and is in many ways more of an Operations matter (as is the ongoing validation process itself, and I don't see a finished note on that either.) Possibly we are better off coordinating with Ops on this point even before the TCG.

--Theresa