HiPS IVOID issue

[Markus Demleitner]

[Suggestion: Let's continue this on apps; I've set a corresponding
reply-to]

Dear colleagues,

On Fri, Jan 15, 2016 at 09:23:08AM +0100, Pierre Fernique wrote:
> Last week, we started the process of the IVOA HiPS standardization, notably
> by homogeneizing their identifiers already used by the HiPS providers.
> In the existing implementation, HiPS are already identified by an IVORN

Although Pierre mentioned it in his mail, just as a reminder IVORN is
deprecated (just as URN is deprecated).  Be hip, say IVOID.

> following the last IVOA recommendation, normally it should have been easy to
> adjust the authority-id (first word of the IVORN) which the corresponding
> one in the VO registry or by creating the missing authority-id in the VO
> registry and that's all.
> 
> 
> /HiPS IVORN examples with their associated properties/
> 
> 1. ivo://ov-gso/P/DIRBE/ZSMA8
>    <http://cade.irap.omp.eu/documents/Ancillary/4Aladin/DIRBE_ZSMA_8/properties>
>    : Band8 of COBE HiPS provided by IRAP
[...]

The central question is: What should a client get when it resolves
the IVOID?  Is it meant to be resolved at all?  If not, what's the
function of the identifier?

> In a few months, we will have about 20000 HiPS available (tables with
> coordinates [15000?] + survey bands [500] + specifical HiPS images [>4000])

So, I assume you want to make them discoverable through the Registry?
If so, then (something in) the HiPS' IVOIDs will have to resolve anyway.

> name for IVORN). Simply because there is presently no HIPS described in the
> VO registry, so all these IVOIDs are invalid. But more, *most of the HiPS
> IVOIDs will never be valid as they will never be described individually in
> the VO registry *(catalog tables ? outreach individual images ? HiPS

What is the plan for how people will discover them?

> associated to an article ?) Additionnally,  during the last IVOA meeting, we
> were thinking to add HiPS in the VO registry as a new capability, and not
> necessary as a new separated VO entry (comparable to a cone search or a TAP
> access point for a dedicated resource)

I would definitely recommend this, in particular since it seems the
overwhelming majority of HiPSes are now for resources that already
are in the registry (VizieR tables).

> *
> Three solutions *
> 
> 1.
> 
>    Declare all resources having a HiPS access in the VO registry
>    notably all VizieR tables (and not only catalogs), plus all outreach
>    images, ... Does it technically and politically possible ? May be by

If these things are intended to be discoverable, then yes, that is
what needs to be done.  But as already mentioned that doesn't mean
they all need new resource records.  Where the underlying data
already has an RR, just adding an additional capability will do (see
below).

>    probably introduce structural incoherences (HiPS table level on one
>    side, and catalog description on other side, HiPS per band survey,
>    and not HiPS per survey... this point is clearly related to the last
>    Markus's discovery resource note)

I'm not sure I understand what you're saying here, but what would
really help is a clear picture of how clients in the future are
expected to discover HiPSes.

Gut feeling: if we could construct things such that, say, SDSS has a
resource record but not the individual bands, that'd be great.  There
are various mechanisms conceivable, e.g., a discovery service for
HiPSes, presumably simply SIA. Alternatively, you could have multiple
capabilities in a registry record, though I don't really like the
idea of regularly distinguishing capabilities by waveband.  Also, all
capabilities of a service right now share the same IVOID, and there
is no portable way to reference them from the outside yet -- although
id/fragment could easily work

> 2. Modify the HiPS identification by trying to use the new "fragment"
>    syntax introduced in the new PR-Identifiers2.0. For instance,
>    replaceivo://CDS.VizieR/I/284/out by
>    ivo://CDS.VizieR/I/284?/some_keyword/=out. Unfortunately, this

This would make sense if you went for the intermediate discovery
service.  This is one of the recommended patterns for SIA and SSA.

>    solution will also mean to adapt the internal service nomenclature
>    if they already use the previous IVORN "/" separator and not the
>    "?something" new syntax. For instance, the notation for tables in
>    VizieR and all VizieR clients (Aladin, TOPcat, DS9, Web sites...)
>    used for years is I/284/out and not I/284?/some_keyword/=out. Does
>    it means that CDS VizieR and associated clients have to support
>    both, or remove the usage of previous one ? I suppose no.

Again, I suspect I don't quite understand the problem here -- do HiPS
clients parse the identifiers?  If so, what do they do with the
parsed parts?

If they don't parse the identifiers, they shouldn't care if there's a
syntax change, should they?

> 
> 3. Accept the definition of an IVOID not necessary declared in the VO
>    registry.

Depending on what clients do with the IVOID, that may be an option,
and the Registry WG, perhaps unfortunately, does not control the
Death Star and thus has no way to blackmail you into creating
registry records.

However, again, think of the discoverability of the HiPS, using
standard Registry mechanisms.  I suspect you'll find out it's a good
idea to play it nice...

> 1. Add VizieR tables in the VO Registry for avoiding to break the
>    VizieR notation (=> ivo://CDS.VizieR/I/284/out would be valid)

But the VizieR tables by and large already are in the Registry -- all
you'd need to do is add a capability to them, no?

> 2. Use fragment IVOID for really "too small resources" level, notably
>    the HiPS outreach images (ivo://eso.org/outreach?product=eso1238a)

...which would mean that an intermediate discovery step would need to
be there (or perhaps you require ids on the capabilities and
reference those with fragment identifiers, but then a publication of
a new image would require an update of the Registry record, which is
possible, but a little weird).

> 3. Tolerate the existence of IVOIDs not yet defined in the registry
>    notably for new HiPS providers (=>ex: ivo://JAXA/xxx), and/or accept
>    the creation of VO registry entries by delegation (the ivo:JAXA/xxx
>    VO registry would be generated from its properties HiPS definitions)

But how would clients find out about their existence?

> In the mean time, and for avoiding to jam the HiPS IVOA standardization
> process during this discussionI suggest to stop the usage of IVOID in the
> "properties" HiPS file (/publisher_did/ field),  and use instead two new
> fields splitting the authority-id and the hips-id in two separated fields
> (/p//ublisher_id/ and /hips_id/). This solution would allow to continue to
> have a usable and homogeneous identification HiPS mechanism without clashing
> with the 2.4 constraint, but allowing to generate the associated IVOID
> (ivo:// prefix + concatenation with "/" separator) if the associated HiPS is
> really defined in the VO registry.

Well, my advice is:  Work out how discovery of HiPSes should work.
After what I understood of the above, I suspect you might get by
defining one standard ID:

ivo://ivoa.net/std/hips#hips 

Such capabilties would have one ParamHTTP interface, the access URL
of which directly points to a HiPS.  You'd probably also set the
resultType of that interface to your HiPS media type for consistency
with case 2:

When there are several HiPSes in one resource, you'll have to put in
a discovery service.  Let's say it's a SIA service (you could do
something simpler, too, perhaps just an enumeration on an HTML page
or whatever).  

Let's say you're using SIA.  You'd then define the SIA service in a
normal Registry record.  Clients only looking for HiPSes could figure
out which SIA services they should look at by looking at the
interface; a HiPS service's interface would give whatever media type
you reserve for HiPS as the interface's resultType.  It's a fairly
easy registry query pattern, I'd say.

In both cases, I don't think a specialized registry extension would
be necessary, just use plain vr:Capability (or a SIA capability).

In the first case, the HiPS' identifier would be the resource
identifier, in the second case, it would be the resource identifier
plus "?<local part>".

> For the end, I have to apologize for arriving so late in the document
> Identifier process (PR state). But I have to say that I had totally missed
> the consequence of the 2.4 contraint until this practical case of the HiPS
> IVOA standardization raised it.

I hope I'm making a convincing point that there is (almost) no
problem.

Or have I missed something?

Cheers,

          Markus