s_region problem

Tue Nov 28 09:35:27 CET 2017

Hi Apps and DAL,

A few additional points on Pat's summary:

On Mon, Nov 27, 2017 at 01:35:16PM -0800, Patrick Dowler wrote:
> There was not and still is not an STC-S standard. The xtypes in
> TAP-1.0 turned out to be pretty ambiguous because ADQL-2.0 doesn't
> have a type system and we kind of just made stuff up to fill the gaps.
> There is/was quite a lot of inconsistent use even then (to prefix or
> not to prefix?, case?)...

At least from my side, the main motivation to not pursue the
standardisation of the STC-S subset in TAP was twofold:

(a) lack of typing.  Many platforms have separate types for the
various sorts of geometries (point, circle, polygon).  For these, it
is really problematic to ingest tables having STC-S columns (DaCHS
converts everything to polygons, but of course that doesn't work once
people use set operators).  Also, indexing such mixed-type columns
appears problematic.

(b) in-value metadata breaks VOTable semantics.  In VOTable, metadata
like units an reference frames are in header-like material, and never in
the data section (e.g., there is no <TD unit="s">5</TD>.  That's
more or less a precondition for efficient processing.  Analogously,
in current VOTable, we declare reference frames and such somewhat like this:

-- this will certainly look a bit differently when the STC-in-VOTable
thing will be resolved, but what's certain is that the metadata will
reference FIELD and not individual rows, let alone cells.

With that in place, what is a client supposed to do if a value like

  Point GALACTIC_II 12.3 34.1

appears in the cov column?  Is it ICRS?  or GALACTIC_II?  or yet
something else?

Add to this the necessity to have uniform frames once the stuff is in
a database, and I think it's clear that the TAP 1.0 appendix design
isn't terribly great, and I'd be surprised if there were a single
complete and correct implementation.

> So, right now there are two ways a service could output polygons:
> DALI-1.1 xtype="polygon" or TAP-1.0 xtype="adql:REGION" (and de-facto
> STC-S) but only one of those is actually a standard upon which we
> should build. TAP-1.1 strongly recommends following DALI but of course
> it cannot forbid using both older standards and conventions and/or
> custom xtypes/serialisations.

Well, it's clear takeup of the non-normative appendix of TAP was
greater than expected after the (for me surprisingly) smooth
acceptance of DALI 1.1 with the new-style geometries.

In particular, it seems the immediate need for non-simply connected
geometries is there.

Question 1: Is there a need to properly standardise the current
usage?  

If so, my proposal would be to take the TAP 1.0 appendix, remove the
reference system declaration (in a way that existing strings remain
valid, but their reference system part becomes ignored) and dump this
into DALI 1.2 for an xtype="region".

However, in my Registry experience, STC-S footprints never quite took
off for a good reason.  MOCs, on the other hand, appear to work
nicely for resource coverage and even for cases in which STC-S would
have been incredibly tedious.  See the MOC standard for examples.

So, my hope has actually been for a while that complex geometries in
VOTables would eventually be represented with MOCs, and both DaCHS
and pgsphere already contains a prototype for this; try, for
instance,

  select coverage, res_title from 
  rr.resource
  natural join rr.stc_spatial
  where 
	  1=contains(point('', 163, 57.5), coverage)	
	  and 1=ivo_hashlist_has('radio', waveband)

on http://reg.g-vo.org/tap

(this is dominated by all-sky resources at this point, but you can
see at least one non-trivial coverage in there).

Given how MOCs took the VO by storm, I suppose we could quickly see
native support for this kind of thing in many clients, and, again
from the Registry experience, I'd expect MOCs will be much easier to
work with than the current STC-S strings.

So:

Question 2: Assuming some tool support (i.e., "STC-S-to-MOC
converter", and MOC support in their backen databases), how would
current STC-S providers feel about migrating to MOCs on a time frame
of a few years?

*If* such a migration seems acceptable, I'd say we can keep the
standardisation situation with respect to STC-S as it is and plan for
xtype="moc" in DALI 1.2 to cover the complex geometries use case.

          -- Markus