Arrays in TAP_SCHEMA

[Markus Demleitner]

Hi Pat,

On Wed, May 31, 2017 at 10:54:10AM -0700, Patrick Dowler wrote:
> I agree with the goal of supporting array-typed columns, I think 1-d
> arrays are OK now, and I think this is within the realm of the TAP-1.1
> update. It would make TAPType in VODataService obsolete since we could
> just use VOTableType.

I'm all for that.

> And I think this is actually a pretty small change and I'm almost
> there in my prototype.
> 
> Thoughts?

But then why not just take over the VOTable type system (or at least
arraysize) to TAP_SCHEMA, too?  I'm noticing just now, but if
something in the vicinity of the VO is called arraysize, as an
implementor I'd be somewhat sore if it contained something different
than what is in VOTable's arraysize syntax (I'd be ok with dropping the
number-plus-star feature, like arraysize="7*").

It does all we can possibly need at this point, and since dealing
with it is part of VOTable, people already have code to deal with
strings like these.

For 1.1, I don't think we can switch datatype to VOTable types to
complete the transition (or can we?).  But arraysize is new anyway,
and I can't see many reasons to keep the semantics of 1.0 size for
it.

Oh, and:

> On 31 May 2017 at 08:08, alberto micol <amicol.ivoa at googlemail.com> wrote:
> > My use case regarding arraysize=???2" in TAP is here described.
> >
> > SSA 1.1 requires, in the VOTable output, a single field for the spatial
> > location,
> > expressed as:
> >
> > <FIELD ID="SpatialLocation" name="SpatialLocation" datatype=???double"
> > ucd="pos.eq" utype="ssa:Char.SpatialAxis.Coverage.Location.Value"
> > arraysize="2" unit=???deg???>
> >
> > At ESO we are implementing SSA on top of TAP.
> >
> > I therefore created the spatial location column in the TAP_SCHEMA.columns
> > table declaring size=2,
> > but TAP does not translate this information into the (naively) expected
> > arraysize=???2???,
> > for the reasons that Gregory explained.
> >
> > Hence, we are stack with the development of the ESO SSAP...
> > unless some smart DAL person comes up with a solution!

I'd say don't worry *too* much about that.  At the GAVO data cetner,
we've been publishing tables with arrays for ages, and the fact that
we've been lying a bit about the TAP_SCHEMA metadata hasn't bothered
anyone (we have size NULL for them for var-size arrays, and we've
been abusing size for array length otherwise).  If you don't believe
me, run

select top 1 wcs_refpixel from lsw.plates

on http://dc.zah.uni-heidelberg.de/tap.  I'd be surprised if you
found a TAP client that wouldn't simply do the right thing (provided
they support arrays in VOTable cells at all).

For SSA, incidentally, the situation is a bit different, because
conceptually what ssa_location is is a (spherical) point.  You can
inspect the table flashheros.data (which is an SSA table) on our TAP
service to see what we do.  Admittedly, that's not terribly
attractive because I doubt many clients do something sensible with
STC-S specs.

With TAP 1.1, that's going to improve (the POINT xtype); so, that's
what I'd put into my VOTable if I wrote new code now:

<FIELD datatype="float" arraysize="2" xtype="POINT".../>

-- clients will be happy with this for now, and a correct
TAP_SCHEMA/VODataService representation can follow when TAP 1.1 is
finalized.

       -- Markus