s_region problem

Mon Nov 27 18:32:43 CET 2017

Hello DAL/Apps,

Speaking as a TAP data service provider at MAST, we have put considerable effort into implementing footprints based on STC-S.  For CAOM/ObsTAP, even our database-internal geometric search functions under the hood use it as input. Adding to this would be considerable effort. We're also planning replacement of some other standard services with a TAP layer, so my concern extends to how those would work as well.

I'm confused as to what is being proposed here; am I correct that some data providers are currently responding with footprints in a style that is not defined by any of our standards? Or is it another known, defined standard that can be referenced? Are we suggesting this alternate format should be accepted in client queries so that other data providers have to deal with it as well, or is the onus entirely on clients to understand either method and take the server's lead? I'm not happy with either of these ideas but admit I might just not understand the use case.

Thanks for your time,

--Theresa Dower

________________________________
From: apps-bounces at ivoa.net <apps-bounces at ivoa.net> on behalf of Felix Stoehr <fstoehr at eso.org>
Sent: Monday, November 27, 2017 10:29 AM
To: apps at ivoa.net; dal at ivoa.net; François Bonnarel
Subject: Re: s_region problem

Dear all,

as far as ALMA is concerned, we are using STC-S and have indeed quite
complex notations with UNION, POLYGON, CIRCLE as well as NOT for holes
in the footprints. While I can not judge whether or not an
additional/modified standard of STC-S is really required (for us at
least STC-S works very well) I can safely say that changing the notation
would entail non-negligible costs.

In general, of course, the less standards evolve (without backwards
compatibility), the more stable our systems can be, the less effort it
is to maintain them and probably also the easier it is for new players
to take these standards up.

Best regards,

Felix

On 27.11.2017 13:12, François Bonnarel wrote:
> Dear all,
>
> My understanding of this context as a DAL chair involved in several of
> the specifications as author and (supposed to be) aware of the others.
>
> Most of this stuff is extracted from a discussion we had in a more
> limited group.
>
> 1 ) Obscore
>
>     The discussion here is actually invloving at least 3 DAL standards +
> Obscore (which is DM oficially but actually "DM-DAL") : TAP, ADQL, DALI
> and Obscore. The DAL dillemma is the following : as it encompasses a
> vast variety of situations and use cases and various interface shapes,
> DAL technology cannot be defined by a single "SPEC_OF_EVERYTHING". DAL
> has defined a great deal of different protocols, which of them evolving
> its own rythm. We have to be carefull all this keep consistent and to be
> carefull also we do not break everything
>
>      Each of the 4 standards involved here have two versions : TAP 1.0,
> 1.1, ADQL 2.0, 2.1, DALI 1.0, 1.1. Obscore 1.1 and DALI 1.1 are IVOA
> recommendations since 2017 May the 17th. TAP1.1 and ADQL 1.1 are still
> in discussion.
>
>      1 ) For s_region what does Obscore spec says ? ObsCore is a model
> (or a view of a larger model) each class/attribute is perfectly defined.
> To map such a model in a TAP service one has to express it in a TAP
> schema. The OBscore spec defines precisely this TAP schema. + xtypes for
> each TAP schema column. In OBscore 1.0 the xtype for s_region is
> "adql:REGION". Obscore 1.1 maintains the same definitions for s_region
> (and other ObsCore attributes) than 1.0. But it adds that the definition
> given in the spec is consistent with TAP 1.0 (the only valid one at the
> moement it was defined) but that the situation may evolve after the
> release of TAP 1.1.
>
>      2 ) So what does TAP says about xtypes ? 1.0 was defining a couple
> of them, all bound to adql functions, including adql:Region. adql:REGION
> WAS AT THE TIME OF WRITING OF 1.0  considered as STC-S. TAP 1.1 gets rid
> of this and claims that as far as possible xtype should map to the
> definitions given in DALI. It says "SHOULD" and not MUST of course. You
> are still allowed to define or use other xtypes if really necessary.
>
>      3 ) DALI 1.0 didn't say anything about xtypes. DALI 1.1 defines a
> couple of them. xtypes CIRCLE and POLYGON are defined as arrays of
> doubles. This has also been discussed to be xtypes of some of SODA input
> parameters (together witn interval). These xtypes are those refered by
> TAP 1.1  as those   that SHOULD be used if no other xtype is required by
> the use case.
>
>      4 ) ADQL 2.0 Defined a couple of functions which TAP was mapping to
> xtypes: Among them REGION. ADQL 2.1 is considering dropping string-typed
> REGION and keep more double array oriented CIRCLE and POLYGON. This has
> consequences for querying. This has no consequence on adql:REGION xtype
> which can remain defined as in ADQL 2.0 if necessary.
>
>
>      Discussion :
>
>                       With the main goal of keeping the defintions and
> formats stable for a while in the vO in a given context ( the
> ObsCore/OBstap one here) I think nothing prevent us to keep adql:REGION
> (equivalent to STC-S string) as an xtype for the s_region attribute in
> ObsCore.
>
>                        I don't know how DACHS is implemented but I
> assume it generates xtypes in the VOTABLE response of the services from
> the TAP schemata. So if the the ObsCore TAP schema keeps adql:REGION as
> the xtype of s_region instead of xtype=polygon, clients like Aladin
> could still recognize and parse it immediatly.
>
>                        Actually it makes more sense than it may appear.
> The data/metadata (coord system and whatever) distinction is not an
> issue for Obscore. Everything is ICRS in there. So the STC-S string is
> unambiguously a polygon, a circle, or any of other stc flavors of space
> area perfectly understandable. s_region is not only a space area. It's
> the support of the observation (as defibed by ObsCore and
> characterisation) expressed as a space area. So it's not that strange to
> have a spoecific xtype for it, and to keep the historical
> adql:REGION/STC-S then.
>
>                         For POlygon or CIRCLES in general, which have
> plenty of usages different from "dataset support" (exemply gratia
> "region of interest", "survey or catalog coverages" , "cross match
> search regions" ,etc... and which may require different coordinate
> systems than ICRS going to the POLYGON/CIRCLE xtypes with arrays is the
> most standard option for the future...
>
> Lets' go to Pierre questions in detail
>
>
>
> Le 24/11/2017 à 12:26, Pierre Fernique a écrit :
>>
>>
>> Dear DAL and Apps members,
>>
>> It seems that we have a problem concerning the *s**_region* usage.
>>
>> Recently, we discover that some data providers have changed their
>> syntax, moving from the regular STC-S string to a new one which seems
>> to be an array of doubles, alternating long and lat. Both Aladin V10
>> and Aladin Lite fail to manage a such new syntax, and we had
>> complaints from final users which do no longer see the FoV of their
>> observations.
>>
>> After investigating about this unexpected evolution, it appears that
>> this new syntax is generated by some versions of DaCHS testing TAP1.1
>> future possible recommendation.
>>
>> So, I have several questions for TAP 1.1 authors and DAL members :
>>
>>  1. Is is really true that STC-S s_region should be removed from
>>     VOTable results ? Personally, I did not see this consequence in
>>     the TAP 1.1 document. or ?
>>
> No. Nothing prevent us to have string field in VOTable
>>
>> 1.
>>
>>
>>  2. And if yes : is it only in TAP ? or also in SIA2 or SSA ?
>>
> TAP itself doens't prevent to use string-typed FIELDS in the  response.
> IT only recommands usage of fIELDS with wtype.
> In the case of s_region the basic thing is not TAP or SIAV2, the
> discovery protocol, but ObsCOre. ObsCore has to be stabilized.
> Common discovery metadata in the case of ADQL queries and the case of
> Parameter query language has been a major progress recently...
> We should not break it/
>
> For SSA si next answer.
>>
>>  1. How the client can know the logic (lon/lat or lat/lon,
>>     counter-clock or anti-counterclock, ...), the frame and other
>>     coordinate parameters for such array of double ?
>>
> I think xtype=POLYGON assumes it's lon/lat. At the moment coordsystem
> can aonly be given by the de-deprecated COOSYS. So reference to a
> COOSSYS element seems to be the only solution.
>>
>>  1. How providers who already generated complex s_region will do now ?
>>     (for instance ESO)
>>
> I think they should not change that. ObsCore has been designed and
> discussed durin a long time to be a stable basis.
>>
>> 1.
>>
>>
>>  2. How we will manage the transition period ?
>>
> In my opinion no transition period is needed for the specific case of
> s_region which is STCS ICRS.
>
> DAL "build up" is always a balance between stability and backward
> compatibility and controlled progress in the definitions.
>
> Cheers
> François
>>
>> Best regards
>> Pierre Fernique
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.ivoa.net/pipermail/dal/attachments/20171127/5413f8a5/attachment.html>