Arrays in TAP_SCHEMA

[Patrick Dowler]

Markus? Just poking you to see if you missed my last message in this topic...

Pat

On 1 June 2017 at 13:58, Patrick Dowler <pdowler.cadc at gmail.com> wrote:
> Markus,
>
> So are you proposing that instead of making arraysize in tap_schema
> sort of compatible with "size" from 1.0 we just make it a string and
> use exactly the VOTable syntax? And then we don't need
> tap_schema.columns.varsize at all.
>
> For backwards compat, TAP-1.1 services would keep "size" and put a
> single int or null there.
>
> The only downside is that it seems like we're making tap_schema less
> type-safe by switching to char and having clients convert string to
> int. At the same time, separating varsize means the type system is
> slightly different from VOTable which is annoying for clients in other
> ways.
>
> I'm neutral on this.
>
> Pat
>
> PS-Maybe we should make VOTable more type-safe? :-)
>
>
> On 1 June 2017 at 04:37, Markus Demleitner
> <msdemlei at ari.uni-heidelberg.de> wrote:
>> Hi Pat,
>>
>> On Wed, May 31, 2017 at 10:54:10AM -0700, Patrick Dowler wrote:
>>> I agree with the goal of supporting array-typed columns, I think 1-d
>>> arrays are OK now, and I think this is within the realm of the TAP-1.1
>>> update. It would make TAPType in VODataService obsolete since we could
>>> just use VOTableType.
>>
>> I'm all for that.
>>
>>> And I think this is actually a pretty small change and I'm almost
>>> there in my prototype.
>>>
>>> Thoughts?
>>
>> But then why not just take over the VOTable type system (or at least
>> arraysize) to TAP_SCHEMA, too?  I'm noticing just now, but if
>> something in the vicinity of the VO is called arraysize, as an
>> implementor I'd be somewhat sore if it contained something different
>> than what is in VOTable's arraysize syntax (I'd be ok with dropping the
>> number-plus-star feature, like arraysize="7*").
>>
>> It does all we can possibly need at this point, and since dealing
>> with it is part of VOTable, people already have code to deal with
>> strings like these.
>>
>> For 1.1, I don't think we can switch datatype to VOTable types to
>> complete the transition (or can we?).  But arraysize is new anyway,
>> and I can't see many reasons to keep the semantics of 1.0 size for
>> it.
>>
>> Oh, and:
>>
>>> On 31 May 2017 at 08:08, alberto micol <amicol.ivoa at googlemail.com> wrote:
>>> > My use case regarding arraysize=???2" in TAP is here described.
>>> >
>>> > SSA 1.1 requires, in the VOTable output, a single field for the spatial
>>> > location,
>>> > expressed as:
>>> >
>>> > <FIELD ID="SpatialLocation" name="SpatialLocation" datatype=???double"
>>> > ucd="pos.eq" utype="ssa:Char.SpatialAxis.Coverage.Location.Value"
>>> > arraysize="2" unit=???deg???>
>>> >
>>> > At ESO we are implementing SSA on top of TAP.
>>> >
>>> > I therefore created the spatial location column in the TAP_SCHEMA.columns
>>> > table declaring size=2,
>>> > but TAP does not translate this information into the (naively) expected
>>> > arraysize=???2???,
>>> > for the reasons that Gregory explained.
>>> >
>>> > Hence, we are stack with the development of the ESO SSAP...
>>> > unless some smart DAL person comes up with a solution!
>>
>> I'd say don't worry *too* much about that.  At the GAVO data cetner,
>> we've been publishing tables with arrays for ages, and the fact that
>> we've been lying a bit about the TAP_SCHEMA metadata hasn't bothered
>> anyone (we have size NULL for them for var-size arrays, and we've
>> been abusing size for array length otherwise).  If you don't believe
>> me, run
>>
>> select top 1 wcs_refpixel from lsw.plates
>>
>> on http://dc.zah.uni-heidelberg.de/tap.  I'd be surprised if you
>> found a TAP client that wouldn't simply do the right thing (provided
>> they support arrays in VOTable cells at all).
>>
>> For SSA, incidentally, the situation is a bit different, because
>> conceptually what ssa_location is is a (spherical) point.  You can
>> inspect the table flashheros.data (which is an SSA table) on our TAP
>> service to see what we do.  Admittedly, that's not terribly
>> attractive because I doubt many clients do something sensible with
>> STC-S specs.
>>
>> With TAP 1.1, that's going to improve (the POINT xtype); so, that's
>> what I'd put into my VOTable if I wrote new code now:
>>
>> <FIELD datatype="float" arraysize="2" xtype="POINT".../>
>>
>> -- clients will be happy with this for now, and a correct
>> TAP_SCHEMA/VODataService representation can follow when TAP 1.1 is
>> finalized.
>>
>>        -- Markus
>
>
>
> --
> Patrick Dowler
> Canadian Astronomy Data Centre
> Victoria, BC, Canada



-- 
Patrick Dowler
Canadian Astronomy Data Centre
Victoria, BC, Canada