deleting a record from the registry

Fri Sep 20 18:28:34 CEST 2019

Pat,

Looks good on first glance! We got 6 changes, 3 of them deletions. And I see the change to "persistent" in the CADC registry Identify response, to match. Thanks for putting in the extra effort to track these rather than simply disappearing them, it makes the registry searches IVOA-wide cleaner.

--Theresa

9/20/2019 4:23:01 PM Harvesting http://www.cadc-ccda.hia-iha.nrc-cnrc.gc.ca/reg/oai?verb=ListRecords&set=ivo_managed&metadataPrefix=ivo_vor&from=2019-09-18T00:00:00Z Got 6 recs Delete id: ivo://cadc.nrc.ca/vospace Delete id: ivo://cadc.nrc.ca/hips2 Delete id: ivo://cadc.nrc.ca/tap Loaded 6 RESOURCES.

________________________________
From: Patrick Dowler <pdowler.cadc at gmail.com>
Sent: Friday, September 20, 2019 11:48:09 AM
To: Theresa Dower; registry at ivoa.net
Subject: Re: deleting a record from the registry

CAUTION: This email originated from outside of STScI. Do not click links or open attachments unless you recognize the sender and know the content is safe.

Hi again,

I have finally got to releasing the changes in our registr service with some updated and deleted records and declaring support for persistent deletion tracking in the Identify record.
Timestamps are ~2019-09-20T16:00:00 for anyone that wants to peek at their logs and let me know that thinsg went OK.

The summary:
- a few updated records
- one new record
- three deleted records
--
Patrick Dowler
Canadian Astronomy Data Centre
Victoria, BC, Canada

On Fri, 13 Sep 2019 at 14:14, Patrick Dowler <pdowler.cadc at gmail.com<mailto:pdowler.cadc at gmail.com>> wrote:
I'll probably be ready to push this out on Mnday or Tuesday next week. I'll let you know when it happens so you can see how things work when harvested.

have a good weekend,

--
Patrick Dowler
Canadian Astronomy Data Centre
Victoria, BC, Canada

On Fri, 13 Sep 2019 at 12:09, Theresa Dower <dower at stsci.edu<mailto:dower at stsci.edu>> wrote:

Pat,

Good catch! Yes, also NOT including the element with status 'deleted' in GetRecord is correct. (It is in fact a thing the ST registry did wrong for a long time, and does generate RofR validation errors.)  And yes, persistent is the right tag for long-term deletion support, not 'full'; that's for  the harvesting/searching end, sorry.

--Theresa

________________________________
From: registry-bounces at ivoa.net<mailto:registry-bounces at ivoa.net> <registry-bounces at ivoa.net<mailto:registry-bounces at ivoa.net>> on behalf of Patrick Dowler <pdowler.cadc at gmail.com<mailto:pdowler.cadc at gmail.com>>
Sent: Thursday, September 12, 2019 6:55:08 PM
To: registry at ivoa.net<mailto:registry at ivoa.net>
Subject: Re: deleting a record from the registry

Theresa,

Thanks for the info. It wasn't too hard to add status="deleted" support and will change deletedRecord to persistent in the Identify response as well.

I see in the link you sent about Deleted record support that OAI says I must not include metadata (element) in GetRecord responses when status="deleted" -- just the header element. That's fine for me as I wanted to remove the actionable information anyway and as I interpret it I must remove all if it.  Am I reading that correctly?

--
Patrick Dowler
Canadian Astronomy Data Centre
Victoria, BC, Canada

On Thu, 12 Sep 2019 at 10:11, Theresa Dower <dower at stsci.edu<mailto:dower at stsci.edu>> wrote:

Pat,

Short answer: Yes, please! Also try to ensure this removes the item from your ListHeaders response, or add deletion status information to the ListIdentifiers response (http://www.openarchives.org/OAI/openarchivesprotocol.html#ListIdentifiers).

Long answer: You don't strictly have to do any of this, as your registry is marked "transient" in its Identify response (http://www.cadc-ccda.hia-iha.nrc-cnrc.gc.ca/reg/oai?verb=Identify), regarding the treatment of deleted records. "Transient" is the OAI-PMH tag for promising nothing regarding deleted records, not even consistency (http://www.openarchives.org/OAI/openarchivesprotocol.html#DeletedRecords).  The very existence of this tag suggests a more lax curation atmosphere for some fields using OAI technology than we strive for.

However, to be a good IVOA registry citizen, I would greatly encourage going through this effort rather than just quietly deleting the records. The will keep your ghost records from persisting in full registries' searches until such time as each builds its own snapshot of all registries from scratch, which does not happen often.  If you do go through this effort, you might as well change "transient" to "full" in your registry record to note other registries can trust record status. If you take the time to make this change and want some end-to-end testing, I'm happy to run the STScI harvester against it and check the logs/status on our end.

Thanks!

--Theresa

________________________________
From: registry-bounces at ivoa.net<mailto:registry-bounces at ivoa.net> <registry-bounces at ivoa.net<mailto:registry-bounces at ivoa.net>> on behalf of Patrick Dowler <pdowler.cadc at gmail.com<mailto:pdowler.cadc at gmail.com>>
Sent: Thursday, September 12, 2019 11:07:29 AM
To: registry at ivoa.net<mailto:registry at ivoa.net>
Subject: deleting a record from the registry

As you recall, we are operating a really simple hack OAI publishing registry and I have reached the point where I have to delete some old records. Looking at VOResource, I think the minimum I have to do is:

- set the status attr on the resource to "deleted"
- set the updated attr of the resource to current timestamp so it gets harvested
- set the datestamp in the header to match

Is that it?

Can/should I remove other metadata? I'd like to remove the content and capability elements, but would leave the title, shortName, identifier, and curation in the "deleted" record".

thanks,

--
Patrick Dowler
Canadian Astronomy Data Centre
Victoria, BC, Canada
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.ivoa.net/pipermail/registry/attachments/20190920/7bb079ed/attachment-0001.html>