TAP questions

[Dobos, László]

Hi everyone,

 

I'm implementing a TAP adapter for my cross-match tool [
http://voservices.net/skyquery ] and was running into a few issues with some
of the TAP endpoints so I'd like to ask a few questions about the standard
and some suggestions on possible workarounds.

 

1. The first question is about how schema and table names should be handled
by the TAP_SCHEMA view. For instance, the gavo TAP endpoint at
http://dc.zah.uni-heidelberg.de/tap returns names in the
schemaname.tablename format in the table_name column whereas vizier at
http://tapvizier.u-strasbg.fr/TAPVizieR/tap/ shows the table name only in
the table_name column and schema_name is separate. It is straightforward to
remove the schema name from the table_name column if it is the same as
schema_name but it's not so straightforward to compose a query, which, for
instance, gets the columns of a given table if I only know the schema name
and the table name separately. Or should I go for compatibility across
endpoints with something like:

 

SELECT * FROM TAP_SCHEMA.columns

WHERE schema_name = 'myschema' AND table_name = 'mytable' OR table_name =
'myschema.mytable'

 

2. My other question is about how to properly quote table names in ADQL.
Some endpoint, particularly vizier, requires table name quoting due to the
way table names are constructed (with lots of /-s), e.g.:

 

SELECT TOP 10 * FROM "viz7"."J/AJ/144/129/refs"

 

On the other hand the GAVO tap interface at throws an error for quoted table
names while handles quoted columns correctly:

 

SELECT  TOP 10 "raj2000", "dej2000" FROM "fk6"."part1" -- results in error
"'QuotedName' object has no attribute 'upper'"

 

In contrast, the endpoint of GAIA at http://gaia.ari.uni-heidelberg.de/tap/,
supports table name quoting properly.

 

Quoting names is necessary to avoid keywords and. Would you automatically
quote everything or quote only what requires quoting and hope that all
endpoints understand it?

 

Or is it safer to use the /tables endpoint rather than querying TAP_SCHEMA
views?

 

3. My third question is about how to deal with missing or wrong xml
namespaces. This is often an issue with /capabilities. For example, a number
of services (at least http://heasarc.gsfc.nasa.gov:80/xamin/vo/tap) returns
an xml with the namespace http://www.ivoa.net/xml/TAP/v1.0 which gives me a
404. Is it something that's allowed by the standard or I'm supposed to come
up with a workaround? A similar issue is with the VOTables returned by, for
example by http://datalab.noao.edu/tap and
https://heasarc.gsfc.nasa.gov/xamin/vo/tap/ , which lack the default
namespace:

 

<VOTABLE xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

xsi:noNamespaceSchemaLocation="xmlns:http://www.ivoa.net/xml/VOTable-1.2.xsd
" version="1.2">

 

The default namespace would be
xmlns="http://www.ivoa.net/xml/VOTable-1.2.xsd" and although it is in the
noNamespaceSchemaLocation attribute, many xml processing frameworks (in my
case .Net XmlSerializer) don't recognize it.

 

I implemented a workaround to automagically add the correct namespace based
on the version="1.2" attribute but again, I don't know if it's a good
policy.

 

4. I had a similar issue with XML documents returned by certain services
that implement non-standard or not-yet-standard features, such as the TAP
service at https://heasarc.gsfc.nasa.gov/xamin/vo/tap/capabilities which
returns an xml with namespaces like
xmlns:tr="http://www.ivoa.net/xml/TAP/v1.0" then reference it somewhere as
<capability standardID="ivo://ivoa.net/std/TAP" xsi:type="tr:TableAccess">

 

My workaround is to simply ignore these but the question is whether this
would break anything or not. I assume in case of every TAP service to have a
/async and /sync endpoint so I don't really need the proper capabilities
list.

 

5. What is the suggested way of figuring out the list of supported output
formats? There seems to be no standard list of mime types. For example the
gaia tap interface at "http://gaia.ari.uni-heidelberg.de/tap/ offers this
list:

 

              <outputFormat>

                            <mime>application/x-votable+xml</mime>

                            <alias>votable</alias>

              </outputFormat>

              <outputFormat>

 
<mime>application/x-votable+xml;serialization=BINARY2</mime>

                            <alias>votable/b2</alias>

              </outputFormat>

              <outputFormat>

 
<mime>application/x-votable+xml;serialization=TABLEDATA</mime>

                            <alias>votable/td</alias>

              </outputFormat>

              <outputFormat>

 
<mime>application/x-votable+xml;serialization=FITS</mime>

                            <alias>votable/fits</alias>

              </outputFormat>

              <outputFormat>

                            <mime>application/fits</mime>

                            <alias>fits</alias>

              </outputFormat>

              <outputFormat>

                            <mime>application/json</mime>

                            <alias>json</alias>

              </outputFormat>

              <outputFormat>

                            <mime>text/csv</mime>

                            <alias>csv</alias>

              </outputFormat>

              <outputFormat>

                            <mime>text/tab-separated-values</mime>

                            <alias>tsv</alias>

              </outputFormat>

              <outputFormat>

                            <mime>text/plain</mime>

                            <alias>text</alias>

              </outputFormat>

              <outputFormat>

                            <mime>text/html</mime>

                            <alias>html</alias>

              </outputFormat>

 

whereas other services (e.g. vizier) use different mime types and some don't
even provide the aliases. Are the aliases standardized so I can select the
best format based on that or I need to do some heuristics based on the mime
type strings?

 

6. Finally, is there a collection of votables and xml documents available
somewhere that can be used for regression testing?

 

Thanks a lot for reading all this,

 

-Laszlo

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.ivoa.net/pipermail/dal/attachments/20180228/1675d9b1/attachment.html>