TAP VOSI Information

[Tom McGlynn]

The current TAP specification is a bit unclear about this.  The current 
draft doesn't mention VARBINARY data at all.  I've suggested some 
changes to Pat where  a convention to use the column XTYPE to give the 
internal data type that is hinted at in the current draft is made more 
generally. The data is stored in a CHAR[*] or BYTE[*] array. So you 
could encode your VARBINARY columns in a VOTable (which could also use 
the BINARY or FITS data representations).   I think this probably needs 
to remain a 'convention' within the standard since the internal 
datatypes available differ so widely from database to database.

    Tom

Gretchen Greene wrote:
> Following up on this thread...
>
> we are indeed using SQL Server databases to store varbinary,  
> blobs, e.g. higher performance systems with varying compressed 
> jpg images,  or complex footprint regions.  So for these 
> instances,  it's not clear to me how this content is currently 
> managed in a  VOTable, i.e. what is current  'type handling' 
> for managing binary formats?  This would be lower level than 
> VOSI,  nevertheless a conceivable VOTable content limitation?
>
> ---- Original message ----
>   
>> Date: Mon, 23 Nov 2009 09:48:31 -0700 (MST)
>> From: Doug Tody <dtody at nrao.edu>  
>> Subject: Re: TAP VOSI Information  
>> To: Tom McGlynn <Thomas.A.McGlynn at nasa.gov>
>> Cc: "dal at ivoa.net" <dal at ivoa.net>
>>
>> Hi Tom -
>>
>> On Mon, 23 Nov 2009, Tom McGlynn wrote:
>>     
>>> I hadn't forgotten multi-position...  At least in the 
>>>       
> current PQL draft, the 
>   
>>> only place you can use uploaded tables is in the multi-
>>>       
> position query.  So 
>   
>>> the service would specify that they supported multi-
>>>       
> position queries by 
>   
>>> giving the kinds of tables that a user can upload in PQL.   
>>>       
> A service which 
>   
>>> did not support multi-position queries would specify no 
>>>       
> supported sources for 
>   
>>> table uploads in PQL.
>>>       
>> I think this is too subtle, and would break anyway once we 
>>     
> find
>   
>> another use for an uploaded table. (One such case is querying 
>>     
> the
>   
>> uploaded table as if it were a data table - while not 
>>     
> scientifically
>   
>> useful, this could be very useful as a round-trip test for 
>>     
> example,
>   
>> since we know every detail of the input table.  So in this 
>>     
> case we
>   
>> have FROM=<upload-table> and WHERE=<whatever>.)
>>
>> We may be approaching this wrong in any case.  Table uploads 
>>     
> and
>   
>> VOSpace support are capabilities of the main TAP engine, not 
>>     
> the
>   
>> query language.  To a query they are just another DBMS table.  
>>     
> So it
>   
>> is probably better to just say whether the TAP service 
>>     
> supports these
>   
>> various cases of tables, and separately say whether PQL 
>>     
> suppports
>   
>> the multi-position query capability.
>>
>>     
>>> I had overlooked one of the most critical options though, 
>>>       
> the supported 
>   
>>> Formats.  Presumably the conventions adopted for the other 
>>>       
> DAL protocols can 
>   
>>> be used here.
>>>       
>> Good point.
>>
>>     
>>> I'm not sure that row limits alone are enough.  The 
>>>       
> database may have BLOBs 
>   
>>> or CLOBs which could be very large. Currently we tend to 
>>>       
> store metadata in 
>   
>>> the database and data in the file system, but with modern 
>>>       
> database systems, 
>   
>>> storing the data in the database could make a lot of sense. 
>>>       
> If a user has a 
>   
>>> 500 MB data file in each row, just a few rows might be 
>>>       
> pretty nasty for a 
>   
>>> download.
>>>       
>> Ok, good point.
>>
>>     
>>> I like your suggestion with regard to providing an 
>>>       
> indicator for the 
>   
>>> underlying database.  In addition to being useful, it would 
>>>       
> be fascinating 
>   
>>> just to see what choices our colleagues make.
>>>       
>> Even if SQL pass-through is not supported it would be quite 
>>     
> interesting
>   
>> to know what back-ends are being used.
>>
>> 	- Doug
>>     

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.ivoa.net/pipermail/dal/attachments/20091124/16326b4e/attachment-0003.html>