<html>
<head>
<meta content="text/html; charset=utf-8" http-equiv="Content-Type">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<div class="moz-cite-prefix"><br>
Dear Registry members,<br>
<br>
May I suggest a constructive solution for the IVOID issue that I
tried to expose in my two last mails, and raised by the current
HiPS standardization process. <br>
<br>
Why not continue to authorize any <b>ivo://authority.id/A/B/C/etc</b>
at the condition that the full id is VO resolvable or, <i>at
least, a left prefix of it</i>.<br>
<br>
This identification method is comparable to the DNS resolver
mechanism. You can query it by a full hostname
(host1.network1.gouv.fr), or by network (network1.gouv.fr) even by
sub-domain (gouv.fr) or domain (fr). The number of members is not
fixed (generally 3 or 4), and it uses an unique separator '.' If
the DNS resolver has no information about the host, it can provide
at least the network information, and so on.<br>
<br>
For VO resource, a simple recursive resolution could be used for a
metadata query. For instance, imagine that a client tries to
retrieve meta data on "<b>ivo://CDS.VizieR/I/221/smc</b>". It
queries the VO registry with this full id. The registry returns
the information concerning the longer prefix found in the VO
registry. In this case "<b>ivo://CDS.VizieR/I/221</b>" with a
dedicated flag alerting the client that this sub-resource is
unknown, but its "parents" is described.<br>
<br>
<br>
Thanks to this more flexible constraint we are sure that any
resource can continue to be identified with a simple, stable,
evolutive and canonical (only one way to write the id) method. We
avoid to introduce the articificial separator "?" for delimiting
"fragment" or "query syntax" (strange for an identifier and not
necessary canonical). And we will be able to manage the possible
evolutions of the VO registry content without having to
potentially modify the identifiers (for instance if the SMC table
is, at the end, described itself in the VO registry. And we insure
that any existing IVOID has, at least, a declared authority id.<br>
<br>
How does it sounds ?<br>
Pierre<br>
<br>
Le 18/01/2016 20:09, Pierre Fernique a écrit :<br>
</div>
<blockquote cite="mid:569D387F.2040302@astro.unistra.fr" type="cite">
<meta content="text/html; charset=utf-8" http-equiv="Content-Type">
<div class="moz-cite-prefix">Hi Markus and others,<br>
<br>
For helping to understand the problem at the good level, you can
consider that the HiPS challenge is comparable to the TAP
challenge.<br>
As TAP, we can imagine that some VO resources will provide
directly a dedicated HiPS capability in their corresponding VO
registry records. For these resources, there is no problem. But
as the VO registry is not really designed for hierarchical
resources (catalog/table, survey/band, ...), this approach will
be unfortunatelly limited.<br>
<br>
Like for TAP table descriptions, it is presently
difficult/impossible, to describe all HiPS resources in the VO
registry and we use a 2 step mecanisms to discover and
manipulate the HiPS descriptions. As your own TAP global schema
developement (GLoTS), we deployed a HiPS Global List server
harvesting each HiPS servers (called HiPS nodes, comparable to
TAP services) and used finally by the HiPS clients.<br>
So concretly, in the VO registry, we plan to register the HiPS
servers (6 presently), rather than all HiPS resources provided
by these HiPS servers (about 20 000 HiPS in a few months,
probably mirrored in several places).<br>
<br>
But even if these HiPS resources are not present individually in
the VO registry we need to identify them (in the client
interfaces, for script commands, for internal HiPS mirroring
management...). And I arrive to my identifier issue : logically
we chose IVOA IVORN identification scheme, now called IVOID in
your last doc. But as I said in my previous mail, the last
Identifier 2.0 doc does no longer allow the usage of IVOID
without a VO real registration (section 2.4). For instance, the
way that we identified the "smc" table of the <font
color="#000099">I/221</font> catalog is now invalid<font
color="#000099">:</font> the identification<font
color="#000099"> ivo://CDS.VizieR/I/221/smc</font> is no
longer a valid IVOID because the "<font color="#000099">smc</font>"
table is not described in the VO registry. Only the catalog <font
color="#000099">I/221 </font>catalog is declared<font
color="#000099"> =></font> <a moz-do-not-send="true"
class="moz-txt-link-freetext"
href="http://registry.euro-vo.org/GetResource.jsp?identifier=ivo://CDS.VizieR/I/221">http://registry.euro-vo.org/GetResource.jsp?identifier=ivo://CDS.VizieR/I/221</a>.<br>
<br>
So we can decide, as you did in your last note, to have a
"cavalier approach" (section 3 - <a moz-do-not-send="true"
class="moz-txt-link-freetext"
href="http://ivoa.net/documents/Notes/DataCollect/20160108/NOTE-discovercollections-1.0-20160108.pdf">http://ivoa.net/documents/Notes/DataCollect/20160108/NOTE-discovercollections-1.0-20160108.pdf</a>)
and just ignore the constraint. But I would like to use a more
coherent approach. May be, we have just to change the "MUST"
word by "SHOULD" in the IVOA PR-identifier 2.0 document -
section 2.4 ?<br>
<br>
I am also wondering if we should start to discuss how the VO
registry could also manage natively complex resources, notably
hierarchical resources and mirrored resources. In fact most of
the resources that we manipulate are hierarchical and mirrored.
Perhaps it is a too big challenge and the usage of specifical
additional registry mecanisms such as GLoTS and Global HiPS list
is fine.<br>
<br>
I hope this HiPS identifier problem related to the VO registry
is a little bit more clear.<br>
Cheers<br>
Pierre<br>
<br>
PS. Also - and that's why I still publish in the Registry
mailing list - it seems that the EURO-VO registry is presently
out of date, at least for the example cited below, and since
several months, I'm afraid. The up-to-dated record from VizieR
is this one: <a moz-do-not-send="true"
class="moz-txt-link-freetext"
href="http://cdsweb.u-strasbg.fr/reg-bin/vizier/oai_CS.pl?verb=GetRecord&metadataPrefix=ivo_vor&identifier=ivo%3A%2F%2FCDS.VizieR%2FI%2F221">http://cdsweb.u-strasbg.fr/reg-bin/vizier/oai_CS.pl?verb=GetRecord&metadataPrefix=ivo_vor&identifier=ivo%3A%2F%2FCDS.VizieR%2FI%2F221</a>
notably by providing two capabilities for CONE SEARCH, one for
each individual table (not really easy to use, but there is
presently no other good VO registry solution for that). EURO-VO
still have the old record version providing only one CONE SEARCH
capability declaration (only for the first table I am afraid). <br>
<br>
<br>
<br>
Le 15/01/2016 13:31, Markus Demleitner a écrit :<br>
</div>
<blockquote cite="mid:20160115123145.GG15598@victor" type="cite">
<pre wrap="">[Suggestion: Let's continue this on apps; I've set a corresponding
reply-to]
Dear colleagues,
On Fri, Jan 15, 2016 at 09:23:08AM +0100, Pierre Fernique wrote:
</pre>
<blockquote type="cite">
<pre wrap="">Last week, we started the process of the IVOA HiPS standardization, notably
by homogeneizing their identifiers already used by the HiPS providers.
In the existing implementation, HiPS are already identified by an IVORN
</pre>
</blockquote>
<pre wrap="">Although Pierre mentioned it in his mail, just as a reminder IVORN is
deprecated (just as URN is deprecated). Be hip, say IVOID.
</pre>
<blockquote type="cite">
<pre wrap="">following the last IVOA recommendation, normally it should have been easy to
adjust the authority-id (first word of the IVORN) which the corresponding
one in the VO registry or by creating the missing authority-id in the VO
registry and that's all.
/HiPS IVORN examples with their associated properties/
1. ivo://ov-gso/P/DIRBE/ZSMA8
<a moz-do-not-send="true" class="moz-txt-link-rfc2396E" href="http://cade.irap.omp.eu/documents/Ancillary/4Aladin/DIRBE_ZSMA_8/properties"><http://cade.irap.omp.eu/documents/Ancillary/4Aladin/DIRBE_ZSMA_8/properties></a>
: Band8 of COBE HiPS provided by IRAP
</pre>
</blockquote>
<pre wrap="">[...]
The central question is: What should a client get when it resolves
the IVOID? Is it meant to be resolved at all? If not, what's the
function of the identifier?
</pre>
<blockquote type="cite">
<pre wrap="">In a few months, we will have about 20000 HiPS available (tables with
coordinates [15000?] + survey bands [500] + specifical HiPS images [>4000])
</pre>
</blockquote>
<pre wrap="">So, I assume you want to make them discoverable through the Registry?
If so, then (something in) the HiPS' IVOIDs will have to resolve anyway.
</pre>
<blockquote type="cite">
<pre wrap="">name for IVORN). Simply because there is presently no HIPS described in the
VO registry, so all these IVOIDs are invalid. But more, *most of the HiPS
IVOIDs will never be valid as they will never be described individually in
the VO registry *(catalog tables ? outreach individual images ? HiPS
</pre>
</blockquote>
<pre wrap="">What is the plan for how people will discover them?
</pre>
<blockquote type="cite">
<pre wrap="">associated to an article ?) Additionnally, during the last IVOA meeting, we
were thinking to add HiPS in the VO registry as a new capability, and not
necessary as a new separated VO entry (comparable to a cone search or a TAP
access point for a dedicated resource)
</pre>
</blockquote>
<pre wrap="">I would definitely recommend this, in particular since it seems the
overwhelming majority of HiPSes are now for resources that already
are in the registry (VizieR tables).
</pre>
<blockquote type="cite">
<pre wrap="">*
Three solutions *
1.
Declare all resources having a HiPS access in the VO registry
notably all VizieR tables (and not only catalogs), plus all outreach
images, ... Does it technically and politically possible ? May be by
</pre>
</blockquote>
<pre wrap="">If these things are intended to be discoverable, then yes, that is
what needs to be done. But as already mentioned that doesn't mean
they all need new resource records. Where the underlying data
already has an RR, just adding an additional capability will do (see
below).
</pre>
<blockquote type="cite">
<pre wrap=""> probably introduce structural incoherences (HiPS table level on one
side, and catalog description on other side, HiPS per band survey,
and not HiPS per survey... this point is clearly related to the last
Markus's discovery resource note)
</pre>
</blockquote>
<pre wrap="">I'm not sure I understand what you're saying here, but what would
really help is a clear picture of how clients in the future are
expected to discover HiPSes.
Gut feeling: if we could construct things such that, say, SDSS has a
resource record but not the individual bands, that'd be great. There
are various mechanisms conceivable, e.g., a discovery service for
HiPSes, presumably simply SIA. Alternatively, you could have multiple
capabilities in a registry record, though I don't really like the
idea of regularly distinguishing capabilities by waveband. Also, all
capabilities of a service right now share the same IVOID, and there
is no portable way to reference them from the outside yet -- although
id/fragment could easily work
</pre>
<blockquote type="cite">
<pre wrap="">2. Modify the HiPS identification by trying to use the new "fragment"
syntax introduced in the new PR-Identifiers2.0. For instance,
replaceivo://CDS.VizieR/I/284/out by
ivo://CDS.VizieR/I/284?/some_keyword/=out. Unfortunately, this
</pre>
</blockquote>
<pre wrap="">This would make sense if you went for the intermediate discovery
service. This is one of the recommended patterns for SIA and SSA.
</pre>
<blockquote type="cite">
<pre wrap=""> solution will also mean to adapt the internal service nomenclature
if they already use the previous IVORN "/" separator and not the
"?something" new syntax. For instance, the notation for tables in
VizieR and all VizieR clients (Aladin, TOPcat, DS9, Web sites...)
used for years is I/284/out and not I/284?/some_keyword/=out. Does
it means that CDS VizieR and associated clients have to support
both, or remove the usage of previous one ? I suppose no.
</pre>
</blockquote>
<pre wrap="">Again, I suspect I don't quite understand the problem here -- do HiPS
clients parse the identifiers? If so, what do they do with the
parsed parts?
If they don't parse the identifiers, they shouldn't care if there's a
syntax change, should they?
</pre>
<blockquote type="cite">
<pre wrap="">3. Accept the definition of an IVOID not necessary declared in the VO
registry.
</pre>
</blockquote>
<pre wrap="">Depending on what clients do with the IVOID, that may be an option,
and the Registry WG, perhaps unfortunately, does not control the
Death Star and thus has no way to blackmail you into creating
registry records.
However, again, think of the discoverability of the HiPS, using
standard Registry mechanisms. I suspect you'll find out it's a good
idea to play it nice...
</pre>
<blockquote type="cite">
<pre wrap="">1. Add VizieR tables in the VO Registry for avoiding to break the
VizieR notation (=> ivo://CDS.VizieR/I/284/out would be valid)
</pre>
</blockquote>
<pre wrap="">But the VizieR tables by and large already are in the Registry -- all
you'd need to do is add a capability to them, no?
</pre>
<blockquote type="cite">
<pre wrap="">2. Use fragment IVOID for really "too small resources" level, notably
the HiPS outreach images (ivo://eso.org/outreach?product=eso1238a)
</pre>
</blockquote>
<pre wrap="">...which would mean that an intermediate discovery step would need to
be there (or perhaps you require ids on the capabilities and
reference those with fragment identifiers, but then a publication of
a new image would require an update of the Registry record, which is
possible, but a little weird).
</pre>
<blockquote type="cite">
<pre wrap="">3. Tolerate the existence of IVOIDs not yet defined in the registry
notably for new HiPS providers (=>ex: ivo://JAXA/xxx), and/or accept
the creation of VO registry entries by delegation (the ivo:JAXA/xxx
VO registry would be generated from its properties HiPS definitions)
</pre>
</blockquote>
<pre wrap="">But how would clients find out about their existence?
</pre>
<blockquote type="cite">
<pre wrap="">In the mean time, and for avoiding to jam the HiPS IVOA standardization
process during this discussionI suggest to stop the usage of IVOID in the
"properties" HiPS file (/publisher_did/ field), and use instead two new
fields splitting the authority-id and the hips-id in two separated fields
(/p//ublisher_id/ and /hips_id/). This solution would allow to continue to
have a usable and homogeneous identification HiPS mechanism without clashing
with the 2.4 constraint, but allowing to generate the associated IVOID
(ivo:// prefix + concatenation with "/" separator) if the associated HiPS is
really defined in the VO registry.
</pre>
</blockquote>
<pre wrap="">Well, my advice is: Work out how discovery of HiPSes should work.
After what I understood of the above, I suspect you might get by
defining one standard ID:
ivo://ivoa.net/std/hips#hips
Such capabilties would have one ParamHTTP interface, the access URL
of which directly points to a HiPS. You'd probably also set the
resultType of that interface to your HiPS media type for consistency
with case 2:
When there are several HiPSes in one resource, you'll have to put in
a discovery service. Let's say it's a SIA service (you could do
something simpler, too, perhaps just an enumeration on an HTML page
or whatever).
Let's say you're using SIA. You'd then define the SIA service in a
normal Registry record. Clients only looking for HiPSes could figure
out which SIA services they should look at by looking at the
interface; a HiPS service's interface would give whatever media type
you reserve for HiPS as the interface's resultType. It's a fairly
easy registry query pattern, I'd say.
In both cases, I don't think a specialized registry extension would
be necessary, just use plain vr:Capability (or a SIA capability).
In the first case, the HiPS' identifier would be the resource
identifier, in the second case, it would be the resource identifier
plus "?<local part>".
</pre>
<blockquote type="cite">
<pre wrap="">For the end, I have to apologize for arriving so late in the document
Identifier process (PR state). But I have to say that I had totally missed
the consequence of the 2.4 contraint until this practical case of the HiPS
IVOA standardization raised it.
</pre>
</blockquote>
<pre wrap="">I hope I'm making a convincing point that there is (almost) no
problem.
Or have I missed something?
Cheers,
Markus
</pre>
</blockquote>
<br>
</blockquote>
<br>
</body>
</html>