TAP upload type roundtripping

[Mark Taylor]

Dear DAL, especially TAP experts,

I'm looking at some persistent TAP validation issues relating to
roundtripping VOTable column metadata in an upload query like
"SELECT * FROM TAP_UPLOAD.t1".  In taplint I prepare a VOTable
with columns giving a variety of (datatype,arraysize,xtype)
triples and see if they come back the same from a trivial upload
query like the above.  Applied to DaCHS, that leads to the following
taplint Error report:

   E-UPL-TMCX-1 Upload result column xtype mismatch timestamp != adql:TIMESTAMP

since the column that's uploaded with xtype="adql:TIMESTAMP" comes
back with xtype="timestamp".

I'm working from TAP 1.1 section 3.5 here:

   3.5  Mapping Table Datatypes
   ----------------------------

   The mapping between VOTable and database tables uses the datatype,
   arraysize, and xtype values from the VOTable FIELD elements and the
   TAP_SCHEMA.columns table. Mapping for primitive types (numbers and
   strings) is straightforward; services should ensure that input values
   behave as expected in query processing and output values should have
   correct and complete metadata. Mapping for specially structured values
   use xtype(s) as specified in DALI. The behaviour and usability of such
   structured values in queries depends on the query language (section
   2.7.1) being used and support for optional features of the language.

   Datatype mapping rules apply to input tables supplied via an UPLOAD
   parameter (see section 2.7.6) and to result tables after successful
   query execution. TAP services should at least be able to round-trip
   all values from an uploaded VOTable to the database and back when
   columns in the uploaded table are selected (e.g. SELECT * from
   TAP_UPLOAD.mytable).

My initial reading was that the (datatype,arraysize,xtype) triple
had to be preserved for upload round-trips.  But re-reading this
I'm not so sure: it only talks about round-tripping "all values"
rather than the metadata as such, and moreover there isn't a MUST
in the whole section.

So I'm wondering whether I ought to loosen/downgrade/eliminate
the current taplint behaviour, e.g. to allow an "adql:TIMESTAMP"
xtype to get converted to "timestamp" (and hence, mabye,
any unknown xtype to get converted into any other unknown xtype)
during round-tripping.  For this particular case, I may sidestep
the issue by adjusting taplint to upload the more current
xtype="timestamp".
However, this is a more general issue, e.g. Markus said
"I'd like to be able to round-trip polygon into a moc".

So my question to the TAP authorship and DAL in general is:
should taplint require the (datatype,arraysize,xtype) triple
to be unmodified by upload round-trips (which is its current
behaviour) or not (and if not, what requirements should it make)?
I'm thinking maybe detection of such changes should be
downgraded from Error to Warning.

Thanks for consideration,

Mark


--
Mark Taylor  Astronomical Programmer  Physics, Bristol University, UK
m.b.taylor at bristol.ac.uk          http://www.star.bristol.ac.uk/~mbt/