<div dir="ltr">Dear Juan Carlos, dear all,<div><br></div><div>just a quick reply.</div><div><br></div><div>Have you considered working directly on TAP-1.1?</div><div>It's in <span class="" id=":zn.1" tabindex="-1" style="">WD</span> status now, but I was about to push it to PR this week and it should solve at least the "size" issue in columns instead of adding the array_dim and array_type columns (that you can always leave there, of course).</div><div><br></div><div>Apart from that, accessing array internal elements is not yet covered, that's true, so I'm glad <span class="" id=":zn.2" tabindex="-1" style="">GAIA</span> proposes its own implementation as a use case in that direction.</div><div><br></div><div>Cheers,</div><div> Marco<br><div class="gmail_extra"><br><div class="gmail_quote">2017-08-21 8:46 GMT+02:00 Juan Carlos Segovia <span dir="ltr"><<a href="mailto:juan.carlos.segovia@sciops.esa.int" target="_blank"><span class="" id=":zn.3" tabindex="-1" style="">juan</span>.<span class="" id=":zn.4" tabindex="-1" style="">carlos</span>.<span class="" id=":zn.5" tabindex="-1" style="">segovia</span>@<span class="" id=":zn.6" tabindex="-1" style="">sciops</span>.<span class="" id=":zn.7" tabindex="-1" style="">esa</span>.int</a>></span>:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Dear All,<br>
<br>
This mail is to explain how arrays are going to be handled in the Gaia archive TAP service for DR2.<br>
<br>
We can consider to modify the approach in the future if TAP finally supports this data type but, in our view, the current approach is already providing the basic support required.<br>
<br>
In Gaia we are going to work with arrays of multiple dimensions. It means we need something like (VOTable syntax) 'AxBxC...[x*]' (e.g. '2x3x*') as the size of an array column.<br>
<br>
In the current stable specification (1.0), TAP_SCHEMA.columns.size is an integer. This value cannot be mapped to the required needs.<br>
<br>
Also, the current stable specification assigns to TAP_SCHEMA.columns.size, for numeric types with value > 1, a database type VARBINARY.<br>
<br>
We tried to be compatible with the current specification. So we have decided to assign VARBINARY type for all numeric array types. Note that as the data type VARBINARY is assigned, you loose the original data type.<br>
<br>
In order to be compatible, we have added two new columns to TAP_SCHEMA.columns table:<br>
<br>
array_dim (dimensions)<br>
array_type (numeric data type)<br>
<br>
So, for instance, for an integer array of 4x5, we have the following TAP_SCHEMA.column row (only involved columns are shown):<br>
<br>
datatype: 'VARBINARY'<br>
size: null<br>
array_dim: '4x5'<br>
array_type: 'int'<br>
<br>
(There are some problems with chars, so we finally decided to avoid char arrays.)<br>
<br>
Please, find the attached document with a basic mapping for our types. Only integer, short, char and unsignedByte are shown, but all numeric types (short, int, long, float, double) are handled in the same way. Please, take in mind that the last column, database data type, can be ignored as this is an implementation detail.<br>
<br>
The implementation we have done is quite similar to the specified in VOTable 1.1 working draft (main difference: VARBINARY for arrays data type).<br>
<br>
When retrieving data, the VOTable will contain the right array specification as they are extracted from the two extra columns: array_dim and array_type.<br>
<br>
In order to access an specific element (and until the ADQL syntax allows a direct access) we have implemented a database function named get_array_double_element (only for doubles) and can be used as follows (more information at <a href="http://gea.esac.esa.int/archive-help/index.html" rel="noreferrer" target="_blank">http://gea.esac.esa.int/archiv<wbr>e-help/index.html</a>, section "ADQL syntax"):<br>
<br>
<br>
SELECT get_double_array_element('{{1.<wbr>0,2.6},{0.8,0.1}}','[1][2]') from public.dual<br>
<br>
<br>
Best regards,<br>
Juan-Carlos Segovia.<br>
<br>
-- <br>
Serco for ESA - European Space Agency<br>
<br>
Juan-Carlos Segovia Serrato<br>
<br>
ESAC Science Data Centre<br>
Data and Engineering Division<br>
Operations Department, Directorate of Science<br>
European Space Astronomy Centre (ESAC)<br>
European Space Agency (ESA)<br>
<br>
email: <a href="mailto:juan.carlos.segovia@sciops.esa.int" target="_blank">juan.carlos.segovia@sciops.esa<wbr>.int</a><br>
Phone: (34)-91-8131-175 - 70175 internal<br>
Fax: (34)-91-8131-308<br>
<br>
European Space Astronomy Centre (ESAC)<br>
Camino Bajo del Castillo s/n<br>
Urb. Villafranca del Castillo<br>
28692 Villanueva de la Cañada, Madrid, Spain.<br>
<br>
<br>
This message and any attachments are intended for the use of the addressee or addressees only.<br>
The unauthorised disclosure, use, dissemination or copying (either in whole or in part) of its<br>
content is not permitted.<br>
If you received this message in error, please notify the sender and delete it from your system.<br>
Emails can be altered and their integrity cannot be guaranteed by the sender.<br>
<br>
Please consider the environment before printing this email.<br>
<br>
<br>
</blockquote></div><br></div></div></div>