[SIAv2] upload

[Patrick Dowler]

OK, not much discussion on this so I will answer with my CADC hat on 
since we have implemented upload in services and have clients (Our 
AdvancedSearch web UI that queries our TAP service).

Then  there is a call to vote on one of two choices at the bottom so you 
have to read the TL;DR to get a vote :-)

On 22/01/14 11:42 AM, Patrick Dowler wrote:
> ** Some possibilities pulled out of thin air:
>
> Magically get positional constraints out of the table:
>        POS=<name>
> - would work in simple contrived cases
> - table might not contain any identifiable position data
> - table might contain multiple/ambiguous position data

It is really hard to get the magic stuff to work well and certainly not 
currently feasible to resolve ambiguities.

> Get specific constraints out of a column:
>        POS=<name>.<field.name>
> - would resolve the ambiguity, but assumes a single field contains the
> complete position data (no multiple columns with RA and DEC, for example)

I think this needs to be supported because a TAP query can have a column 
with a pair of coordinates:

       select s_region from ivoa.ObsCore ...
       select CENTROID(s_region) from ivoa.ObsCore ...

and such output tables should be usable. Several of the original ObsTAP 
use cases revolved around using ObsCore output tables as input to 
another query and the CSP "upload list" feature echoed the same kind of 
uses.

> Get specific constraints out of multiple columns:
>        POS=CIRCLE <name>.<ra column> <name>.<dec column> <name>.<radius
> column>
> - <name>.<radius column> could be replaced by a numeric constant
> - typical tables would not be feasibly usable with any other geometric
> value types, but circle is > 95% of the use cases anyway

Likewise, there are many VOTables existing or produced by current 
services that have longitude and latitide in separate columns and we 
would have to support them. This is the most complex to support, but 
probably most tables are like this.

So, with my CADC hat on, I would say that query params need to refer to 
the columns in the uploaded table. We should limit SIA-2.0 to a single 
UPLOAD of a VOTable for simplicity. I would favour a shell-script-like 
syntax to identify columns, eg:

UPLOAD=foo,http;//example.com/mytable.xml

POS=CIRCLE $foo.RA $foo.DEC $foo.rad

or (allow constants when table doesn't have all the values):

POS=CIRCLE $foo.RA $foo.DEC 0.2

DALI already requires that the name of an uploaded resource be a very 
simple (SQL-valid) identifier, so the dot notation is safe. The 
shell-script-like $ sign for symbols mans that one could refer to 
columns that contain strings for other parameters that expect string 
values, eg:

POL=$foo.pol_states

I haven't done it (will do) but I am confident this translates fairly 
easily into an equivalent TAP request with an upload and a single join.

** choices **

1. Ignore CSP "upload lists" and say that UPLOAD is not supported in SIA-2.0

2. Define a single votable upload with $symbolic reference to the table 
and columns as described above.

Plus one option: Do we make support for UPLOAD mandatory or optional? 
Note that if it is optional, that opens up needing a way to declare 
support for it, eg VOSI_capabilities, which implies a RegExt. 
SimpleDALRegExt doesn't cover this...


-- 

Patrick Dowler
Canadian Astronomy Data Centre
National Research Council Canada
5071 West Saanich Road
Victoria, BC V9E 2E7

250-363-0044 (office) 250-363-0045 (fax)