VOResource 1.1: Mirrors?

[Thomas Boch]

Hi Markus,

as you can guess, that's a topic of interest for CDS (VizieR has 7 
mirrors, Simbad has one mirror and HiPS might have several mirrors).

The use cases I had in mind are the following:
- in case the main VizieR service is down, how can a sophisticated 
client find an alternative access URL to query the service?
VizieR services are mirrored in 7 places around the world, but this 
knowledge is not inside the registry. Hence, if the main site breaks, 
VizieR is totally unreachable from a VO point of view, which is too bad 
as working mirrors are there.
- find the nearest (in term of network latency) mirror for a given service.


My comments inline:


>
> (2) Design goal
>
> It'd be fairly important to me to keep "simple" service discovery
> possible.  So, I'd say the design goal for mirrors in the Registry
> would be
>
>    "Let advanced clients or other parts of the VO infrastructure
>    figure out the possible access URLs so it can select one close to
>    them.  Plain clients should just be directed to a primary site."

agreed. Whatever we do, it should not break any existing (simple) client.
>
>
> (3) Alternatives
>
> My suspicion is that the Registry is not the ideal component if your
> goal is geographical load balancing or even some sort of fallback
> scheme.  Here's some other ways I'm aware of:
>
> (a) I guess most commercial services use some sort of GeoIP, i.e., the
> DNS responses depend on the geographic location.  So, for instance
> here in Baden-Württemberg www.google.de (at the moment) resolves to
> 2a00:1450:4013:c01::5e, wheres in Saxonia it is
> 2a00:1450:4001:817::2003.  I've never set something like that up
> before, but I'd be surprised if it was hard.
>
> [...]

GeoDNS would be I think the ideal solution to this problem. It is a 
general solution to a problem that goes well beyond the IVOA registry.
But that requires some technical know-how and to have the full mastery 
of your DNS records, which might not be trivial.
>
> (b) A redirector.  I think some content delivery networks work like
> this.  The access URL then points to
> http://redirector.example.org/svcs/stars, and based on an arbitrary
> heuristics, that one would then, respond with a 301 or 303 redirect
> to a mirror.  In terms of advantages and disadvantages, that's a bit
> like (a), except it would be easier for a client to insist on using
> the main site -- it just would go directly there.  If it's smart
> enough to figure out there is a primary site, it stands to reason
> that it knows its URL, too.
I don't really like this option :
- the redirector becomes a single point of failure
- each query has to go through it, slowing down the whole thing
- very basic clients do not handle redirections quite well

>
> (c) Do nothing.  Perhaps with faster networks and more fiber in the
> oceans, there's not much point any more in putting a large effort
> into mirrors and their maintenance, which always is a bit of a pain.

That's what I was thinking until trying to access our services at 
Strasbourg from countries with sub-optimal network or with high latency 
connection. In those cases, being able to target a nearby mirror does 
make a difference.
Now, I also agree that managing distributed mirrors is a pain but which 
is still worth it.
>
>
>
> (4) VOResource solutions
>
> Here's what I've worked out so far how mirrors could work within the
> registry infrastructure.
>
> (a) accessURL attributes
>
> One could use an attribute to say what's the primary site and what's
> a mirror.  So, perhaps:
>
>    <capability standardID="ivo://example/whatever">
>      <interface std="True">
>        <accessURL use="base" priority="primary"
>          >http://example.eu/svc/stars?</accessURL>
>        <accessURL use="base" priority="fallback"
>          >http://example.za/svc/stars?</accessURL>
>        <accessURL use="base" priority="mirror"
>          >http://example.cn/svc/stars?</accessURL>
>      </interface>
>      <param ...
>    </capability>
>
> [...]
>
> (b) have a separate element, perhaps like this:
>
>    <capability standardID="ivo://example/whatever">
>      <interface std="True">
>        <accessURL use="base">http://example.eu/svc/stars?</accessURL>
>        <mirrorURL>http://example.za/svc/stars?</mirrorURL>
>        <mirrorURL>http://example.cn/svc/stars?</mirrorURL>
>      </interface>
>      <param ...
>    </capability>
>
> I guess something like that would be my winner if we decided to go
> ahead with VOResource mirrors.


This option has my preference too, as it introduces a new element and 
minimizes the risk of a basic client using by mistake the mirror 
accessURL (though you might argue it also introduces a schema change).
>
> Is there any metadata that should go ahead with mirrorURL if we went
> this way?  Perhaps something to help make a choice between the
> mirrors without having to ping them all?

Well, pinging all the mirror alternatives with a basic query (SR=0 for 
cone search) might not be that bad.
Aladin Desktop is doing this kind of things at startup when testing the 
nearest HiPS mirror.

That's all for now.

Thomas
>
>
> (5) Another issue with mirrors: Availability


> If we decide mirrors need to be described interoperably (i.e., make
> the VO mirror-aware), there's a second problem: VOSI availability,
> i.e., the endpoint that says whether a given service is up and if
> not, when one should try again.
>
> Currently, it's modelled as a separated capability, i.e., the
> capabilities of a service with mirrors would look like this:
>
>    <capability standardID="ivo://example/whatever">
>      <interface std="True">
>        <accessURL use="base">http://example.eu/svc/stars?</accessURL>
>        <mirrorURL>http://example.za/svc/stars?</mirrorURL>
>        <mirrorURL>http://example.cn/svc/stars?</mirrorURL>
>      </interface>
>      <param ...
>    </capability>
>
>    <capability standardID="ivo://ivoa.net/std/VOSI#availability">
>      <interface xsi:type="vs:ParamHTTP">
>        <accessURL use="full">http://av.example.eu/av/stars</accessURL>
>      </interface>
>    </capability>
>
> The availability schema doesn't let you specify the status of mirrors
> (or, for that matter, alternative interfaces) yet; including
> additional mirrorURLs probably isn't terribly helpful because it'd be
> hard to match query URL and availability URL.
>
> If we were serious about mirrors, we'd hence need to fix
> availability, too.  This would be a good moment for that because VOSI
> is being reviewed as we speak.   But someone would have to volunteer
> for actually doing it.
>
> Thanks for making it down here,
>
>           Markus


-- 
Thomas Boch
Ingénieur de Recherche

   CDS/Observatoire Astronomique   Phone  : 33 (0)3 68 85 24 42
   11, rue de l'Universite         Fax    : 33 (0)3 68 85 24 17
   F-67000 Strasbourg              Email  : thomas.boch at astro.unistra.fr
   France                          http://cdsweb.u-strasbg.fr/~boch