Arrays in TAP_SCHEMA

Wed Jun 7 11:49:27 CEST 2017

Alberto,

Markus is using DACHS, which is completely different from my library. 
So, I fear the blame is on me for that issue.

It is actually due to the way I ask the columns metadata: through the 
TAP_SCHEMA. And since the TAP_SCHEMA, according to TAP 1.0, has no way 
to declare arrays, my library does not know about your arrays and can 
not add any arraysize. In theory, it could be possible to declare the 
column metadata in a more complete way, but unfortunately, the whole TAP 
library is using TAPType and so, suffers from the same lack as TAP 1.0. 
That's why I was asking to DAL about what is planned for TAP 1.1 so that 
I can already start to solve this issue.

According to what has already been discussed here, I think I will very 
soon switch internally to VOTable datatypes. The impact on the current 
library users should be minimal, almost invisible. For those interested 
in having arrays, like you Alberto, there will be a small additional 
step (probably adding some column(s) in TAP_SCHEMA.columns....ideally 
the one of the future TAP 1.1 so that avoiding an annoying transition 
when TAP 1.1 will be released...hence my original questions). On a 
further version of the library, I may propose a different way to declare 
the tables and columns metadata.

In the meantime, Alberto, I may have a quite quick solution for you. 
Since it is starting to be too technical here, I propose to discuss that 
privately so that avoiding to bother the rest of DAL with that ^^

Grégory

On 06/07/2017 11:27 AM, alberto micol wrote:
> Thanks Markus.
>
> I tried your TAP with the query you provided and found that
> - your FIELD declaration contains: datatype=“float” arraysize=“*”
> - votlint is happy with your returned votable
>
> In my case instead:
> - the FIELD has no arraysize and datatype=“double"
> - votlint complains about it:
> ERROR (l.159, c.43): Bad double string '123.269560 -34.578040'
>
>
> Given that I think we are both using Gregory’s library,
> I wonder how you manage to get the arraysize into your votable…
> In my votable only the “char” fields have gotten an arraysize=“*”
> all other fields have no arraysize.
>
> What is that I’m doing wrong? Maybe Gregory can tell me...
>
> Many thanks,
> Alberto
>
>
>
>> On 01 Jun 2017, at 13:37, Markus Demleitner 
>> <msdemlei at ari.uni-heidelberg.de 
>> <mailto:msdemlei at ari.uni-heidelberg.de>> wrote:
>>
>> Hi Pat,
>>
>> On Wed, May 31, 2017 at 10:54:10AM -0700, Patrick Dowler wrote:
>>> I agree with the goal of supporting array-typed columns, I think 1-d
>>> arrays are OK now, and I think this is within the realm of the TAP-1.1
>>> update. It would make TAPType in VODataService obsolete since we could
>>> just use VOTableType.
>>
>> I'm all for that.
>>
>>> And I think this is actually a pretty small change and I'm almost
>>> there in my prototype.
>>>
>>> Thoughts?
>>
>> But then why not just take over the VOTable type system (or at least
>> arraysize) to TAP_SCHEMA, too?  I'm noticing just now, but if
>> something in the vicinity of the VO is called arraysize, as an
>> implementor I'd be somewhat sore if it contained something different
>> than what is in VOTable's arraysize syntax (I'd be ok with dropping the
>> number-plus-star feature, like arraysize="7*").
>>
>> It does all we can possibly need at this point, and since dealing
>> with it is part of VOTable, people already have code to deal with
>> strings like these.
>>
>> For 1.1, I don't think we can switch datatype to VOTable types to
>> complete the transition (or can we?).  But arraysize is new anyway,
>> and I can't see many reasons to keep the semantics of 1.0 size for
>> it.
>>
>> Oh, and:
>>
>>> On 31 May 2017 at 08:08, alberto micol <amicol.ivoa at googlemail.com 
>>> <mailto:amicol.ivoa at googlemail.com>> wrote:
>>>> My use case regarding arraysize=???2" in TAP is here described.
>>>>
>>>> SSA 1.1 requires, in the VOTable output, a single field for the spatial
>>>> location,
>>>> expressed as:
>>>>
>>>> <FIELD ID="SpatialLocation" name="SpatialLocation" datatype=???double"
>>>> ucd="pos.eq" utype="ssa:Char.SpatialAxis.Coverage.Location.Value"
>>>> arraysize="2" unit=???deg???>
>>>>
>>>> At ESO we are implementing SSA on top of TAP.
>>>>
>>>> I therefore created the spatial location column in the 
>>>> TAP_SCHEMA.columns
>>>> table declaring size=2,
>>>> but TAP does not translate this information into the (naively) expected
>>>> arraysize=???2???,
>>>> for the reasons that Gregory explained.
>>>>
>>>> Hence, we are stack with the development of the ESO SSAP...
>>>> unless some smart DAL person comes up with a solution!
>>
>> I'd say don't worry *too* much about that.  At the GAVO data cetner,
>> we've been publishing tables with arrays for ages, and the fact that
>> we've been lying a bit about the TAP_SCHEMA metadata hasn't bothered
>> anyone (we have size NULL for them for var-size arrays, and we've
>> been abusing size for array length otherwise).  If you don't believe
>> me, run
>>
>> select top 1 wcs_refpixel from lsw.plates
>>
>> on http://dc.zah.uni-heidelberg.de/tap.  I'd be surprised if you
>> found a TAP client that wouldn't simply do the right thing (provided
>> they support arrays in VOTable cells at all).
>>
>> For SSA, incidentally, the situation is a bit different, because
>> conceptually what ssa_location is is a (spherical) point.  You can
>> inspect the table flashheros.data (which is an SSA table) on our TAP
>> service to see what we do.  Admittedly, that's not terribly
>> attractive because I doubt many clients do something sensible with
>> STC-S specs.
>>
>> With TAP 1.1, that's going to improve (the POINT xtype); so, that's
>> what I'd put into my VOTable if I wrote new code now:
>>
>> <FIELD datatype="float" arraysize="2" xtype="POINT".../>
>>
>> -- clients will be happy with this for now, and a correct
>> TAP_SCHEMA/VODataService representation can follow when TAP 1.1 is
>> finalized.
>>
>>       -- Markus
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.ivoa.net/pipermail/dal/attachments/20170607/09ed3c35/attachment.html>