TAP 1.0: Substantive comments.

[Tom McGlynn]

Doug Tody wrote:
> On Mon, 31 Aug 2009, Tom McGlynn wrote:
> 
>> - The current design strongly couples the semantics and the transport 
>> protocol.   I.e., all of the data that is transported using a particular 
>> syntax in HTTP POST's is to be treated as a file upload.  This has two major 
>> negative consequences.  It makes TAP more HTTP dependent since rather than 
>> using the abstraction of keyword/value pairs which could easily be 
>> implemented on some other protocol we need to look at exactly how we encoded 
>> data within the transport protocol to understand what certain parameters are. 
>> If we want to layer TAP on some new XXXTP we have to explicitly think about 
>> how file uploads work.  It would come for free if we just use the same 
>> keyword/value abstraction we use everywhere else.
>>
>> Even worse, we've now precluded TAP, and all protocols layered on top of it 
>> (ADQL, PQL, ...), from using in-line file uploads for any purpose other than 
>> table uploads.  What happens when we want to layer in a security protocol 
>> that needs to include a certificate as a file upload?  Or a user 
>> customization file or a file that ADQL6 uses to allow users to dynamically 
>> define a metadata structure or ....  We need to leave space for this kind of 
>> growth.
> 
> If I understand the point correctly, the issue here is that in the
> case of in-line table uploads we are unnecessarily "hiding" the upload
> within the HTTP protocol.
> 
> If instead we were to use a parameter to associate the upload with
> a table what might this look like and what would we gain?  We might
> have something like the following (not yet a carefully thought out
> proposal):
> 
>      UPLOAD=myTable,inline:<multipart-name>
> 
> which would associate the named table myTable with an inline uploaded
> file.

I think I like this.

Just to clarify with an example.  We might have parameters

    UPLOAD=mytable,inline:mytable;yourtable,http://your.host/table.xml
    mytable=<?XML...><VOTABLE>...</VOTABLE>

[where mytable would likely be populated as a file upload but this
is not required.]

This uploads two tables, yourtable which is specified through an
external URL, and mytable which is uploaded as part of the query.

That would be fine with me as a general approach.  It's nice that it 
allows a compatible syntax for both external and inline tables.  There 
is a single keyword to check for to see if there is one or more uploads.
So it bypasses Pat's concern that we need to scan all keywords. The only 
tiny caveat is that we'd want to be careful to use a prefix for the 
local tables, 'inline:' in your example, that's unlikely to conflict 
with a useful HTTP protocol.  I don't know where a list of these are to 
check against.

I think this has all of the advantages that I suggested for my 
namespaced approach, appropriately separates TAP and HTTP and really 
makes it easy for uses to see if they need to worry about uploads.

	Tom