TAP upload type roundtripping

[Patrick Dowler]

The adql:TIMESTAMP xtype is the legacy of TAP-1.0 and I consider it
equivalent to DALI "timestamp" (in the context of TAP, DALI xtypes are used
more widely than that). So to me any s/w would be justified in converting
from the legacy value to the equivalent DALI value. That would (with
DALI-1.2) apply to adql:REGION and DALI "shape" whenever that moves
forward. I would consider converting from a DALI xtype back to a legacy
xtype to be incorrect behaviour: if the s/w knows they are equivalent then
it knows the DALI xtype is preferred (caveat: we may not have made that
clear in DALI, in which case we should add some text to the DALI-1.2 WD...
I don't think that's a controversial stand).

For an unrecognized xtype, it should be preserved exactly.

If I recall, our TAP services probably behave in about the same way, but if
they do happen to preserve adql:TIMESTAMP it would be an implementation
detail and probably not even my intent. I will check. I do know that our
tap_upload does not correctly preserve timestamp values (UTC vs local tz
issue in the insert). That's a bug.

--
Patrick Dowler
Canadian Astronomy Data Centre
Victoria, BC, Canada


On Fri, 16 Apr 2021 at 02:27, Mark Taylor <m.b.taylor at bristol.ac.uk> wrote:

> Dear DAL, especially TAP experts,
>
> I'm looking at some persistent TAP validation issues relating to
> roundtripping VOTable column metadata in an upload query like
> "SELECT * FROM TAP_UPLOAD.t1".  In taplint I prepare a VOTable
> with columns giving a variety of (datatype,arraysize,xtype)
> triples and see if they come back the same from a trivial upload
> query like the above.  Applied to DaCHS, that leads to the following
> taplint Error report:
>
>    E-UPL-TMCX-1 Upload result column xtype mismatch timestamp !=
> adql:TIMESTAMP
>
> since the column that's uploaded with xtype="adql:TIMESTAMP" comes
> back with xtype="timestamp".
>
> I'm working from TAP 1.1 section 3.5 here:
>
>    3.5  Mapping Table Datatypes
>    ----------------------------
>
>    The mapping between VOTable and database tables uses the datatype,
>    arraysize, and xtype values from the VOTable FIELD elements and the
>    TAP_SCHEMA.columns table. Mapping for primitive types (numbers and
>    strings) is straightforward; services should ensure that input values
>    behave as expected in query processing and output values should have
>    correct and complete metadata. Mapping for specially structured values
>    use xtype(s) as specified in DALI. The behaviour and usability of such
>    structured values in queries depends on the query language (section
>    2.7.1) being used and support for optional features of the language.
>
>    Datatype mapping rules apply to input tables supplied via an UPLOAD
>    parameter (see section 2.7.6) and to result tables after successful
>    query execution. TAP services should at least be able to round-trip
>    all values from an uploaded VOTable to the database and back when
>    columns in the uploaded table are selected (e.g. SELECT * from
>    TAP_UPLOAD.mytable).
>
> My initial reading was that the (datatype,arraysize,xtype) triple
> had to be preserved for upload round-trips.  But re-reading this
> I'm not so sure: it only talks about round-tripping "all values"
> rather than the metadata as such, and moreover there isn't a MUST
> in the whole section.
>
> So I'm wondering whether I ought to loosen/downgrade/eliminate
> the current taplint behaviour, e.g. to allow an "adql:TIMESTAMP"
> xtype to get converted to "timestamp" (and hence, mabye,
> any unknown xtype to get converted into any other unknown xtype)
> during round-tripping.  For this particular case, I may sidestep
> the issue by adjusting taplint to upload the more current
> xtype="timestamp".
> However, this is a more general issue, e.g. Markus said
> "I'd like to be able to round-trip polygon into a moc".
>
> So my question to the TAP authorship and DAL in general is:
> should taplint require the (datatype,arraysize,xtype) triple
> to be unmodified by upload round-trips (which is its current
> behaviour) or not (and if not, what requirements should it make)?
> I'm thinking maybe detection of such changes should be
> downgraded from Error to Warning.
>
> Thanks for consideration,
>
> Mark
>
>
> --
> Mark Taylor  Astronomical Programmer  Physics, Bristol University, UK
> m.b.taylor at bristol.ac.uk          http://www.star.bristol.ac.uk/~mbt/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.ivoa.net/pipermail/dal/attachments/20210416/70901d20/attachment.html>