s_region problem

[Patrick Dowler]

As it stands right now, DALI-1.1 provides a standardway to serialise
points, intervals, circles, and simple polygons as arrays of numbers.
The serialisation format is specified in VOTable and is perfectly
usable as output (votable values) and as input (SIAv2 query and SODA
cutout parameters) as they are fully described by standard VOTable
metadata (FIELD/PARAM: datatype/arraysize/xtype and units) which means
that service descriptors (DataLink) can also describe standard and
custom params correctly.

However, just because DALI-1.1 describes these types doesn't mean that
is the end of the story. The datatype="char" arraysize="*"
xtype="adql:REGION" and STC-S serialisation is in use and DALI-1.1
does explicitly say that non-standard xtypes are prefixed, so that
means that TAP services (eg) can continue to use STC-S and just that
it is non-standard (which is true - it is not an IVOA
reccommendation).

It is clear now that there are use cases for more complex geometry. I
have had such in CAOM for years(*) but the consensus during the
development of SIA2, DALI-1.1 and SODA-1.0 was that these were not
needed. If that is no longer true than we can of course start work on
DALI-1.2 according to new use cases. If that was not true back then it
would have been good for more people to speak up at the time. That's
water under the bridge now and we should look forward, not back.

There is certainly no need to panic. SIA and SODA are pretty clearly
defined and individual TAP services can chose to use DALI polygons or
STC polygons. Obviously I think the former are the technically
superior and simpler choice and I hope we can transition to this (or
DALI-1.2) in an orderly fashion. Right now, the main CADC TAP service
outputs DALI polygon from the caom2.Plane.position_bounds column and
the accepted STC-S for from the ivoa.ObsCore.s_region column (same
underlying database column). Both choices are based on what we think
is best for our users and software that uses that service.

So, let's not panic :-)


Pat

* and I have done the necessary work to adapt my content to match the standards

On 18 December 2017 at 08:40, Jesus Salgado
<jesus.salgado at sciops.esa.int> wrote:
> Hi Alberto, all,
>
> Also, at the ESDC, we share the statements from Alberto. If we want to have
> polymorphic ways to describe
> the same kind of entities (like geometry) we have to ensure that the
> applications are prepared for them and
> we have to justify why we are moving away from the previous defacto
> standard.
>
> In case we want to officially standardize STC-S or a similar string format,
> we also consider that reusing a
> format like "Well-known_text" could be an option to prevent inconsistencies
> and simplify implementations.
>
> Cheers,
> Jesus
>
>
> On 18/12/17 17:20, Baptiste Cecconi wrote:
>
> Hi Alberto,
>
> in spite of my delayed answer, I fully agree with your statements (all of
> them, include your extra point).
>
> Baptiste
>
> Le 7 déc. 2017 à 12:14, alberto micol <amicol.ivoa at googlemail.com> a écrit :
>
> Dear All,
>
> I think that this topic has the potential for a very bad impact on the way
> the astronomical community perceives the VO.
>
> 1.- The clients/tools are the face of the VO, they are what the astronomical
> community sees of the VO.
> Exposing these new polygon format when the clients/tools are not yet in the
> position of digesting it
> seems not a good way forward.
> We are losing our face with the astronomical community by acting this way.
> Coordination with the developers of tools/clients is of paramount
> importance.
> This to me is an important lesson learned.
>
> 2.- The ObsTAP promise was and still is:
> - one and the same query to all data centres,
> - one and the same answer (names, formats, units) from all of them
> Easy.
> This is now an unfulfilled promise.
>
> 3.- A multiplication of formats makes it very hard to interoperate.
> Some clients will support all new formats, some will only support one of
> them;
> as a consequence, data providers will have to generate output containing all
> formats
> so to guarantee maximum flexibility to their users. Complexity is nobody’s
> friend.
>
> 4.- At ESO, complementing what Theresa said, there was quite some effort in
> supporting STC-S
> as output format from our SQLServer database. We did that because this is
> required by ObsCore,
> and because this is what the clients understand. Blood went into our design
> and implementations:
> STC-S is formally not an approved standard? It is a de-facto standard, and
> that should be respected.
>
> Sorry all, I might sound a bit hard on all this… apologies but I see a lot
> at stake here.
>
> Conclusions:
> I plea the data centres that already implemented the change in
> Obscore/SSA/SIA/etc
> to retract it asap to avoid this confusion, until the impasse
> --from the user's point of view-- is resolved.
>
> Alberto
>
> PS: One extra point: if you really really really want to move away from
> STC-S,
> then why re-inventing the wheel and coming up with yet a new format?
> Please consider to use the well-established text markup language called
> Well-Known Text (WKT),
> already supported by many DBMSes including postgres, SQLServer,
> Oracle, mysql, and many others;
> see: https://en.wikipedia.org/wiki/Well-known_text (and its RDBMS section).
> Such choice would greatly simplify the adoption of the new standard by the
> data centres:
> there would be no need to develop special functions to handle VO-made
> formats.
> Still, clients would have to implement it, but at least the backend job
> would be already done.
>
>
> On 24 Nov 2017, at 12:26, Pierre Fernique <Pierre.Fernique at astro.unistra.fr>
> wrote:
>
>
> Dear DAL and Apps members,
>
> It seems that we have a problem concerning the s_region usage.
>
> Recently, we discover that some data providers have changed their syntax,
> moving from the regular STC-S string to a new one which seems to be an array
> of doubles, alternating long and lat. Both Aladin V10 and Aladin Lite fail
> to manage a such new syntax, and we had complaints from final users which do
> no longer see the FoV of their observations.
>
> After investigating about this unexpected evolution, it appears that this
> new syntax is generated by some versions of DaCHS testing TAP1.1 future
> possible recommendation.
>
> So, I have several questions for TAP 1.1 authors and DAL members :
>
> Is is really true that STC-S s_region should be removed from VOTable results
> ? Personally, I did not see this consequence in the TAP 1.1 document. or ?
> And if yes : is it only in TAP ? or also in SIA2 or SSA ?
> How the client can know the logic (lon/lat or lat/lon, counter-clock or
> anti-counterclock, ...), the frame and other coordinate parameters for such
> array of double ?
> How providers who already generated complex s_region will do now ? (for
> instance ESO)
> How we will manage the transition period ?
>
> Best regards
> Pierre Fernique
>
>
>
>
> --
> Jesus Salgado
> ESA Astronomy Archives Technical Lead
> QUASAR Science Resources for European Space Agency (ESA)
>
> ESAC Science Data Centre (ESDC), SCI-OPD
> Data and Engineering Division, SCI-OP
> Operations Department, Directorate of Science
> European Space Astronomy Centre (ESAC)
> European Space Agency (ESA)
>
> ESAC - ESA Science Astronomy Centre
> Camino Bajo del Castillo s/n
> Urb. Villafranca del Castillo
> 28692 Villanueva de la Canada - Madrid        Tel: +34 91 813 12 71
> SPAIN                                         Fax: +34 91 813 13 08
> -------------------------------------------------------------------
>
> This message and any attachments are intended for the use of the addressee
> or addressees only.
> The unauthorised disclosure, use, dissemination or copying (either in whole
> or in part) of its
> content is not permitted.
> If you received this message in error, please notify the sender and delete
> it from your system.
> Emails can be altered and their integrity cannot be guaranteed by the
> sender.
>
> Please consider the environment before printing this email.
>



-- 
Patrick Dowler
Canadian Astronomy Data Centre
Victoria, BC, Canada