<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 14 (filtered medium)">
<!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]--><style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
@font-face
{font-family:Consolas;
panose-1:2 11 6 9 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
pre
{mso-style-priority:99;
mso-style-link:"HTML Preformatted Char";
margin:0in;
margin-bottom:.0001pt;
font-size:10.0pt;
font-family:"Courier New";}
tt
{mso-style-priority:99;
font-family:"Courier New";}
p.MsoAcetate, li.MsoAcetate, div.MsoAcetate
{mso-style-priority:99;
mso-style-link:"Balloon Text Char";
margin:0in;
margin-bottom:.0001pt;
font-size:8.0pt;
font-family:"Tahoma","sans-serif";}
span.HTMLPreformattedChar
{mso-style-name:"HTML Preformatted Char";
mso-style-priority:99;
mso-style-link:"HTML Preformatted";
font-family:Consolas;}
span.EmailStyle20
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
span.BalloonTextChar
{mso-style-name:"Balloon Text Char";
mso-style-priority:99;
mso-style-link:"Balloon Text";
font-family:"Tahoma","sans-serif";}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Menelaus,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Thanks, that’s good to know. I’ll set up the STScI registry on a similar schedule. Currently we do incremental harvests daily, it’s just the full harvests that
are irregular and infrequent.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal" style="margin-left:.5in"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> Menelaus Perdikeas [mailto:mperdikeas@sciops.esa.int]
<br>
<b>Sent:</b> Tuesday, March 15, 2016 3:40 AM<br>
<b>To:</b> Theresa Dower<br>
<b>Cc:</b> Thomas Boch; registry@ivoa.net; ops@ivoa.net<br>
<b>Subject:</b> Re: [Ops] Inconsistency between full searchable registries<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal" style="margin-left:.5in"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-family:"Arial","sans-serif";color:black">Hi Theresa,<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-family:"Arial","sans-serif";color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-family:"Arial","sans-serif";color:black">Just to clarify to avoid any misunderstandings: The new EuroVO is currently in production and is configured to perform incremental harvests on a more or
less daily basis and full harvests once a month.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-family:"Arial","sans-serif";color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-family:"Arial","sans-serif";color:black">Cheers,<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-family:"Arial","sans-serif";color:black">Menelaus.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-family:"Arial","sans-serif";color:black"><o:p> </o:p></span></p>
</div>
<div class="MsoNormal" align="center" style="margin-left:.5in;text-align:center">
<span style="font-family:"Arial","sans-serif";color:black">
<hr size="3" width="100%" align="center" id="zwchr">
</span></div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><b><span style="font-family:"Arial","sans-serif";color:black">From:
</span></b><span style="font-family:"Arial","sans-serif";color:black">"Theresa Dower" <<a href="mailto:dower@stsci.edu">dower@stsci.edu</a>><br>
<b>To: </b>"Thomas Boch" <<a href="mailto:thomas.boch@astro.unistra.fr">thomas.boch@astro.unistra.fr</a>>,
<a href="mailto:registry@ivoa.net">registry@ivoa.net</a><br>
<b>Cc: </b><a href="mailto:ops@ivoa.net">ops@ivoa.net</a><br>
<b>Sent: </b>Monday, March 14, 2016 7:26:06 PM<br>
<b>Subject: </b>RE: [Ops] Inconsistency between full searchable registries<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-family:"Arial","sans-serif";color:black"><o:p> </o:p></span></p>
</div>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:.5in">
<span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Thomas,</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:.5in">
<span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:.5in">
<span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Hello! I know we at STScI have had occasional issues for years with VizieR records slipping through the daily harvesting cracks. The last time I worked on this with Sebastien Derriere,
we determined that the way VizieR records are published, sometimes the ‘created’ date is not set to the date of publishing records into the VizieR registry, thus when we harvest incrementally daily, records can be missed. This may still be an issue. I am curious
whether this is still happening, or whether we have an ingest bug on our end that is for some reason not being logged.</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:.5in">
<span style="color:black"><o:p> </o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:.5in">
<span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">I know that Euro-VO re-harvests entire registries from scratch (not incrementally) on a regular but infrequent basis, which would explain them having these records we have missed.
I will re-harvest VizieR registry by hand and see how many new records we get. We at STScI are also publishing an update to our registry software and database in the coming weeks, and will get a new fresh harvest from every registry at that time. Once we
have the new registry software running operationally I will work on making that re-harvest a semi-regular automated event, as Euro-VO does.</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:.5in">
<span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:.5in">
<span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Thank you for bringing this issue up!</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:.5in">
<span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">--Theresa</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:.5in">
<span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"> </span><span style="color:black"><o:p></o:p></span></p>
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:1.0in">
<b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">
<a href="mailto:ops-bounces@ivoa.net">ops-bounces@ivoa.net</a> [<a href="mailto:ops-bounces@ivoa.net">mailto:ops-bounces@ivoa.net</a>]
<b>On Behalf Of </b>Thomas Boch<br>
<b>Sent:</b> Monday, March 14, 2016 9:44 AM<br>
<b>To:</b> <a href="mailto:registry@ivoa.net">registry@ivoa.net</a><br>
<b>Cc:</b> <a href="mailto:ops@ivoa.net">ops@ivoa.net</a><br>
<b>Subject:</b> [Ops] Inconsistency between full searchable registries</span><span style="color:black"><o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:1.0in">
<span style="color:black"> <o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:1.0in">
<span style="color:black">Hi Registry-enthusiasts,<br>
<br>
I would like to report on an inconsistency I found between resources available in the EuroVO registry and in the VAO/STScI registry.<br>
<br>
I am performing daily a full harvesting (through OAI PMH) of the registry in order to retrieve and filter out services of interest to Aladin Desktop. I used to query the STScI registry for this task until I found out some active resources were missing (for
instance </span><tt><span style="font-size:10.0pt;color:black">ivo://cfa.tdc/hectospec/hectospec_public.ssap.q/ssa</span></tt><span style="color:black">). I then switched to the EuroVO registry and just found out that some other resources, for instance</span><tt><span style="font-size:10.0pt;color:black">
</span></tt><tt><span style="font-size:9.0pt;color:black">ivo://nasa.heasarc/skyview/skyview</span></tt><span style="color:black">, were also missing (but available in the STScI registry).
<br>
<br>
The full list of missing resources for each registry is attached to this message. From a quick look:<br>
<br>
- STScI registry is mostly missing 1300 VizieR resources<br>
<br>
- EuroVO registry is mostly missing HEASARC services. Menelaus confirmed me that they had an issue with querying the HEASARC registry.<br>
<br>
<br>
What should I do ? I am not really keen on querying the two registries and merging the results, as I feel this should not be done at my side. I would expect consistency between full registries, at least for resources older than 1 week. Am I missing something
?<br>
<br>
<br>
Cheers,<br>
<br>
Thomas<o:p></o:p></span></p>
</div>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-family:"Arial","sans-serif";color:black"><o:p> </o:p></span></p>
</div>
</div>
<pre style="margin-left:.5in">This message and any attachments are intended for the use of the addressee or addressees only.<o:p></o:p></pre>
<pre style="margin-left:.5in">The unauthorised disclosure, use, dissemination or copying (either in whole or in part) of its<o:p></o:p></pre>
<pre style="margin-left:.5in">content is not permitted.<o:p></o:p></pre>
<pre style="margin-left:.5in">If you received this message in error, please notify the sender and delete it from your system.<o:p></o:p></pre>
<pre style="margin-left:.5in">Emails can be altered and their integrity cannot be guaranteed by the sender.<o:p></o:p></pre>
<pre style="margin-left:.5in"><o:p> </o:p></pre>
<pre style="margin-left:.5in">Please consider the environment before printing this email.<o:p></o:p></pre>
</div>
</body>
</html>