HiPS IVOID issue => proposed solution
Pierre Fernique
Pierre.Fernique at astro.unistra.fr
Fri Jan 29 17:09:46 CET 2016
Hi Markus & all,
Thanks Markus (and all other readers) for your patience. I think that we
are converging. We have arrived at the same conclusion that to correctly
identify astronomical resources we need to manipulate the notion of
creator DID rather than publisher DID to ensure the uniqueness.
I am still convinced that we have to let the HiPS providers decide if
they want to declare their HiPSes in the VO or not. It is their
responsibility, and forcing an a priori declaration would probably
reduce the success of HiPS. However, I totally agree that it will be
definitively better if, at the end, they accept/want to declare their
HiPSes or their HiPS nodes in the VO registry for a wider visibility -
but my position is "a posteriori".
So it is not possible to use directly IVOID syntax (1- to avoid the "a
priori" VO registry declaration, 2-the IVOID syntax depends now to the
way that this declaration is done, 3-the VO registry ambiguity between
publisher and creator). However by basing the HiPS identification on the
pair of ids creator_id and obs_id, I think that we can ensure
compatibility with VO identification in the case that the HiPS provider
wants to VO declare its HiPSes.
So, concretly the HiPS provider will have the choice :
1) to not declare their HiPSes in the VO (for tests, political choice,
not really related to the VO, or ...)
2) to declare their HiPS individually thanks to the appropriated
capability that you suggest (thanks for that). In this case the
associated IVOID would be ivo://creator_id <ivo://creator_id/obs_id>/
<ivo://creator_id/obs_id>obs_id <ivo://creator_id/obs_id> (slash
separator) -> the regular IVOID/./
3) to declare their HiPS node (= HiPS server). In this case I understand
that the IVOIDs would beivo://creator_id <ivo://creator_id?obs_id>?
<ivo://creator_id?obs_id>obs_id <ivo://creator_id?obs_id> (question mark
separator) -> the new IVOID scheme /(//The way that these IVOIDs will be
determined is still a little bit obscure for me and I still believe that
there is a confusion between publisher and creator. Also I'm pretty sure
that we will have potentially various IVOID syntaxes for the same
resource [cf (1)])
/Despite the IVOID possible ambiguities that I see in the case 3, the
HiPS standardization and developpement is no longer jammed, and the
declaration in the VO registry will be possible.
Cheers
Pierre
--
(1) We can imagine that an HiPS provider chooses the solution 3 (HiPS
server declaration) but some of his resources are already defined in the
VO registry. In this case, we will have two different IVOIDs for these
resources, one with the "/" separator and the other with the "?"
separator. It could be notably the case for HiPS VizieR resources for
instance.
However this problem is not limited to HiPS, and we have the same
potential problem with TAP services
Le 28/01/2016 16:14, Markus Demleitner a écrit :
> Hi,
>
> On Mon, Jan 25, 2016 at 07:28:40PM +0100, Pierre Fernique wrote:
>> I need your help Markus (and other Registry's involved persons) concerning
>> the best way to declare HiPSes in the VO registry (independantly to the
>> identifier issue above).
>>
>> I am studying these two ways, not necessary exclusive :
>>
>> 1. *Adding HiPS capabilities* to the VO resources already defined in
>> the VO registry (your suggestion (1)). For instance, we can imagine
>> to add two HiPS capabilities to Simbad VO resources, one for HiPS
>> simbad access to CDS location, and another HiPS capability to Simbad
>> Harvard mirror site => Both of this HiPS access will refer to the
>> same IVOID => ivo://CDS/Simbad. Good ! /*[my first question]*/ Can
>> you provide us a template of the XML capability that we should use ?
>> Similar to cone search capability ?
> First off, this is not independent of what kind of discovery
> scenarios you have in mind. For what I think would be great -- I
> discover a resource that looks interesting to me, and to get an idea
> about its coverage and content, I send its access URL to a
> HiPS-enabled client --, you do not need a special capability type.
> All it takes is
>
> <capability standardID="ivo://ivoa.net/std/hips#hips-1.0">
> <interface role="std" xsi:type="vs:ParamHTTP">
> <accessURL use="base"
> >http://example.com/hipses/data/band</accessURL>
> </interface>
> </capability>
>
> To make this really nice and proper, after publication of your
> standards text, send a StandardsRegExt record to the RofR (if you
> need assistence writing one, kindly ask me). This would then be what
> ivo://ivoa.net/std/hips points to, and it would have
>
> <key>
> <name>hips-1.0</name>
> <description>
> A HiPS cube accessible through HTTP.
> </description>
> </key>
>
> in it (or analogous, I'm improvising based on incomplete knowledge
> about how HiPS really works), which is what the standardID
> references.
>
>
>> This method seems to be more difficult to apply for the VizieR
>> tables. As I was trying to explain in my previous mail, the HiPS
>> capabilities will not be inserted at the good level of the VO
>> Registry records (catalogs not tables). And we are facing the same
>> problem that we already have for the cone search capabilities and
>> footprint descriptions (for years). In this case*/[my second
>> question]/*//do you recommend us to create several HiPS capabilities
>> at the catalog level, one per table (eventually multiplied by the
>> number of mirror sites) ? Or maybe it is time to define individually
>> each VizieR tables in the VO registry ? (in discussion in the CDS team)
> This is a bit of a philosophical question: What's a "resource".
> The W3C people essentially say "if it's got a URI, it's a resource",
> so that's not helpful.
>
> Therefore, I recommend the two criteria: If it's got common metadata,
> it's one resource. If users would expect to discover x1 together
> x2 because otherwise x1 or x2 don't make much sense, they would be in
> one resource. That is, I'd claim, a bit more helpful, but of
> course full of ambiguities either. Indulge me for two examples:
>
> Consider a catalog of cataclysmic variables, consisting of a table of
> objects t_1 and a table of outbursts t_2.
>
> One resource or two? So, author and title are the same, presumably
> also a resource-wide description. But the tables of course can have
> separate descriptions. But, really, t_2 will have a foreign key into
> t_1, so discovering it without t_1 won't help. So, make one resource
> out of it. Indeed, tableset lets you attach detailed table (and
> column) description to each table.
>
> I believe the situation is analogous in most VizieR tables -- but
> really, there's no good reason why it couldn't be different for some.
> Consider, for instance, two teams, each building one catalog of AGNs,
> onein radio, the other in X-Ray, from original data obtained,
> obviously, on two different instruments. For the publication,
> they've teamed up for some reason, so there's a bit of common
> metadata. But since there's so much differening metadata, my take
> here would be: Make it two resources so you can properly support
> discovery by wavelength or instrument.
>
> After all that philosophy: I think it would make a lot of sense to
> just include a capability as given above per HiPS in all but the
> most pathologic cases. Capability lets you add a description
> (about 30% of the capabilities in the current VO use that, so perhaps
> clients should do more with these), so you could give hints on what's
> going on like this:
>
> <capability standardID="ivo://ivoa.net/std/hips#hips-1.0">
> <description>HiPS for the table of X-Ray detections</description>
> <interface role="std" xsi:type="vs:ParamHTTP">
> <accessURL use="base"
> >http://example.com/hipses/data/xray</accessURL>
> </interface>
> </capability>
>
> <capability standardID="ivo://ivoa.net/std/hips#hips-1.0">
> <description>HiPS for the table of K-band detections</description>
> <interface role="std" xsi:type="vs:ParamHTTP">
> <accessURL use="base"
> >http://example.com/hipses/data/kband</accessURL>
> </interface>
> </capability>
>
> <capability standardID="ivo://ivoa.net/std/hips#hips-1.0">
> <description>HiPS for the table of detections with gravitational
> waves</description>
> <interface role="std" xsi:type="vs:ParamHTTP">
> <accessURL use="base"
> >http://example.com/hipses/data/gravity</accessURL>
> </interface>
> </capability>
>
> I guess at least in WIRR we should think about a smart way to use
> capability/description.
>
>
>> 2. *Declaring HiPS servers* in the VO registry (your suggestion (2)),
>> for letting each HiPS server enumerating HiPSes that it is
>> providing. [The way that this enumeration is provided is presently a
>> list of properties records in a specific HiPS syntax; your SIA
>> suggestion is unfortunately not really adaptable for HiPSes, I think
>> - but it is another discussion].
> It is, indeed -- if we can avoid defining yet another protocol, that'd be
> *really* great. If SIA really doesn't work for you, perhaps ObsTAP
> does?
>
>> In this last method I do not see how the same HiPS will not refer
>> potentially to several different IVOIDs. For instance the HiPS of
>> DSS colored is presently distributed by 3 HiPS servers located
>> respectively at CDS, ESAC and IAS. If the 3 HiPS servers are
>> declared in the VO registry, and following your IVOID building
>> mecanism (<main_service_id>?<some_local_id>) each HiPS server will
>> provide their own prefix and syntax of the IVOID and we will have,
>> at the end, three different IVOID associated to the same resource:
>> ivo://CDS/myhipsnode?id=DSSColor,
>> ivo://ESAVO/theirhipsnode?theirid=DSSColor,
>> ivo://IAS/yourhipsnode?yourid=DSSColor. So/*[my third question]*/
>> what I misunderstood ? and if not, do we agree to have several
>> IVOIDs for the same resource ?
> You have not misunderstood anything -- this is the nature of the
> *publisher* DID -- it depends on who's publishing the dataset. If
> you look at, for instance, SSA, in the VO we've also had the notion
> of a *creator* DID. That is a unique id for a dataset, invariant
> with respect to where it comes from, assigned by the creator.
>
> Example: If the Aladin team builds HiPSes with the intent that they are
> mirrored by other publishers, they would get themselves an IVOID,
> ivo://cds/aladin, say, and then stamp their datasets
>
> ivo://cds/aladin?twomass/ks
> ivo://cds/aladin?twomass/h
> ivo://cds/aladin?twomass/j
> ivo://cds/aladin?twomass/technicolor
>
> As they distribute their datasets, this id would never change, even
> if the publisher DID would.
>
> [Note, however, that actual mirrors probably should be handled
> through different interfaces on the same capability and thus would
> share a publisher, with the datasets served therefore having
> identical pubDIDs regardless of the mirror; but that's, I guess, not
> a pressing issue for HiPSes]
>
> Creator DIDs so far haven't been used much. But at least when looked
> at from the outside, your use case looks like it is what they were
> conceived for.
>
>> /HiPS identifier issue//=> normally solved
>> /
>> It is clear that we (HiPS developers) had a wrong conception of the IVORN
>> (IVOID now). We believed that we could use it as a stable and uniform
>> astronomical resource identication mecanism. But as you said, "/they are
>> recipes to locate something/", not to identify something (you are right: URI
>> != URL). The two concepts seems similar but differ. I understand now that we
> No, there is really no difference between URL and URI any more (see
> RFC 3986). And you cannot have one of uniqueness and discovery
> without the other -- there's no way you can find out if some name is
> already taken without some sort of Registry. I still maintain you're
> doing yourself a favour if you don't re-invent that Registry, even if
> at first it may seem handing out some names to people you know anyway
> can't be much of a problem.
>
>> will have difficulties to use it as we need (notably because IVOID is still
>> in evolution).
> Uh, for the record, no, IVOID is *not* in evolution, the only thing
> that has actually changed in the last 10 years[1] is that Identifiers
> 2.0 did away with the XML serialisation that nobody has used anyway.
> The rest hasn't changed. Identifiers 2.0 is just a clarification
> (or sanctioning of existing practice), and it filled in some gaps
> left open by the old spec (comparison of IVOIDs with stop characters,
> mainly). I'd say stability over 10 years isn't at all bad for
> something to do with computers.
>
>> Also, forcing an "a priori" declaration in the VO registry of any HiPSes
>> just for having an identifier is probably not a good idea for three reasons:
>> 1) We have the risk to have a lot of "prototype" HiPS VO registrations -
>> never maintain after (as we had with cone search resources at the beginning
>> of the VO)
> In the creatorDID scheme above the only thing that's in the registry
> is a record for the creator (aladin in the example above), probably
> an organisation. Those don't hurt, and as said above, they're really
> necessary for uniqueness.
>
> For test purposes, you could still register ivo://CDS/hips-testing
> (and forgoe uniqueness guarantees for HiPSes having that as Registry
> part in their creator DID).
>
>> 2) I'm not sure that all HiPS providers will agree.
> That's a larger problem, but they'll have to agree to register their
> id somewhere if you want to maintain uniqueness. Why not use what's
> already there?
>
>> 3) The granularity of the VO registry is probably not always adapted in such
>> HiPS cases (cf below)
> [actually, above, now, because I first wanted to answer the questions]
>
> Oh, the granularity matters for discovery, but not for identification
> purposes. For that, I believe the creatorDID scheme above is as good
> as anything you'd invent just for HiPSes.
>
>> So to avoid jamming the HiPS standardization process and the deployment of
>> the HiPS aware codes, we prefer to use another HiPS identification mecanism,
>> no longer based on IVOID:
>> 1) The publisher_did in the HiPS properties record (HiPS metadata file) will
>> be now optional and no longer used for identifying the HiPS
> If you want to use that id to un-dupe responses from several servers
> then, yes, that is exactly what you should do.
>
>> 2) The HiPS internal unique identifier will be now built by the
>> concatenation of publisher_id et obs_id without the ivo:// prefix
> Well, the publisher id could (and, to control uniqueness, probably
> should) be an ivoid. Then use a ? to concatenate it with the obs_id
> as you propose, and presto, you have an ivoid, the uniqueness of
> which is guaranteed by the Registry (in front of the question mark)
> and the creator (after the question mark).
>
> Nice, clean, no extra work required. What's not to like?
>
> Call that "internal identifier" a creator DID, and you're smack in
> the middle of the VO mainstream.
>
>> 3) We are modifying the various codes and data impacted by this evolution
>> (Hipsgen.jar, Hipsgencat.jar, Aladin V9, Aladin Lite, MocServer, and some
>> various scripts)
>> 4) I will send a memo to all HiPS actors for upgrading their codes and
>> synchronize their HiPS data according to this evolution (notably the mirror
>> actions should be suspended amongst the HiPS sites for avoiding dupplication
>> until the situation is again cleaned.
> If you're still convinced you have to go into all that trouble,
> please let's have a quick chat on the phone -- I think there's not
> terribly much you need to do to get your use cases covered without
> having to invent new tech.
>
> Cheers,
>
> Markus
>
> [1] Approximately; I'd wager with PR-20040621 it's essentially been
> there, though I've not really, formally, checked it.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.ivoa.net/pipermail/apps/attachments/20160129/b2744b7f/attachment-0001.html>
More information about the apps
mailing list