Arrays in TAP_SCHEMA

[Patrick Dowler]

Markus,

So are you proposing that instead of making arraysize in tap_schema
sort of compatible with "size" from 1.0 we just make it a string and
use exactly the VOTable syntax? And then we don't need
tap_schema.columns.varsize at all.

For backwards compat, TAP-1.1 services would keep "size" and put a
single int or null there.

The only downside is that it seems like we're making tap_schema less
type-safe by switching to char and having clients convert string to
int. At the same time, separating varsize means the type system is
slightly different from VOTable which is annoying for clients in other
ways.

I'm neutral on this.

Pat

PS-Maybe we should make VOTable more type-safe? :-)


On 1 June 2017 at 04:37, Markus Demleitner
<msdemlei at ari.uni-heidelberg.de> wrote:
> Hi Pat,
>
> On Wed, May 31, 2017 at 10:54:10AM -0700, Patrick Dowler wrote:
>> I agree with the goal of supporting array-typed columns, I think 1-d
>> arrays are OK now, and I think this is within the realm of the TAP-1.1
>> update. It would make TAPType in VODataService obsolete since we could
>> just use VOTableType.
>
> I'm all for that.
>
>> And I think this is actually a pretty small change and I'm almost
>> there in my prototype.
>>
>> Thoughts?
>
> But then why not just take over the VOTable type system (or at least
> arraysize) to TAP_SCHEMA, too?  I'm noticing just now, but if
> something in the vicinity of the VO is called arraysize, as an
> implementor I'd be somewhat sore if it contained something different
> than what is in VOTable's arraysize syntax (I'd be ok with dropping the
> number-plus-star feature, like arraysize="7*").
>
> It does all we can possibly need at this point, and since dealing
> with it is part of VOTable, people already have code to deal with
> strings like these.
>
> For 1.1, I don't think we can switch datatype to VOTable types to
> complete the transition (or can we?).  But arraysize is new anyway,
> and I can't see many reasons to keep the semantics of 1.0 size for
> it.
>
> Oh, and:
>
>> On 31 May 2017 at 08:08, alberto micol <amicol.ivoa at googlemail.com> wrote:
>> > My use case regarding arraysize=???2" in TAP is here described.
>> >
>> > SSA 1.1 requires, in the VOTable output, a single field for the spatial
>> > location,
>> > expressed as:
>> >
>> > <FIELD ID="SpatialLocation" name="SpatialLocation" datatype=???double"
>> > ucd="pos.eq" utype="ssa:Char.SpatialAxis.Coverage.Location.Value"
>> > arraysize="2" unit=???deg???>
>> >
>> > At ESO we are implementing SSA on top of TAP.
>> >
>> > I therefore created the spatial location column in the TAP_SCHEMA.columns
>> > table declaring size=2,
>> > but TAP does not translate this information into the (naively) expected
>> > arraysize=???2???,
>> > for the reasons that Gregory explained.
>> >
>> > Hence, we are stack with the development of the ESO SSAP...
>> > unless some smart DAL person comes up with a solution!
>
> I'd say don't worry *too* much about that.  At the GAVO data cetner,
> we've been publishing tables with arrays for ages, and the fact that
> we've been lying a bit about the TAP_SCHEMA metadata hasn't bothered
> anyone (we have size NULL for them for var-size arrays, and we've
> been abusing size for array length otherwise).  If you don't believe
> me, run
>
> select top 1 wcs_refpixel from lsw.plates
>
> on http://dc.zah.uni-heidelberg.de/tap.  I'd be surprised if you
> found a TAP client that wouldn't simply do the right thing (provided
> they support arrays in VOTable cells at all).
>
> For SSA, incidentally, the situation is a bit different, because
> conceptually what ssa_location is is a (spherical) point.  You can
> inspect the table flashheros.data (which is an SSA table) on our TAP
> service to see what we do.  Admittedly, that's not terribly
> attractive because I doubt many clients do something sensible with
> STC-S specs.
>
> With TAP 1.1, that's going to improve (the POINT xtype); so, that's
> what I'd put into my VOTable if I wrote new code now:
>
> <FIELD datatype="float" arraysize="2" xtype="POINT".../>
>
> -- clients will be happy with this for now, and a correct
> TAP_SCHEMA/VODataService representation can follow when TAP 1.1 is
> finalized.
>
>        -- Markus



-- 
Patrick Dowler
Canadian Astronomy Data Centre
Victoria, BC, Canada