JSON type in ADQL and TAP

[Pierre Le Sidaner]

On 11/09/2023 09:05, Markus Demleitner wrote:
> Dear DAL,
> 
> On Fri, Sep 08, 2023 at 05:28:39PM +0200, Gregory MANTELET wrote:
>> Personally, I agree with Igor on the fact that JSON data are now everywhere,
>> including in databases. So, at some point, we will have to deal with such
>> things.
>>
>> On the other hand, I also agree with Markus: having JSON data encapsulated
>> inside tables is kind of nasty in term of user experience. Generally, we
>> expect data that we can easily read, but this is not immediately possible
>> with JSON data (especially with long and/or complex data structure) ; a
>> parsing/query step is needed most of the time if the table is exported in
>> something else than a JSON file. A better approach could be to instead
>> provide a link (datalink involved somewhere ?) toward this JSON document ;
>> but not sure, it is a better user experience either.
>>
>> This topic could probably be discussed in more details during a DAL running
>> meeting or during an interop meeting.
> 
> I think a discussion session on this -- specifically, JSON in
> VOTable, not JSON in ADQL, which I consider much less problematic --,
> perhaps with one or two 5-minute introductory talks, would be a
> great thing at this or the next interop.  If it doesn't fit in
> Tucson, perhaps we can have a smaller side meeting during ADASS time?
> 
> By the way, my instinct on externalising JSON is that we'd regret
> that; for one it'd be one HTTP request per VOTable row, and while I
> consider HTTP requests fairly cheap since HTTP/1.1, they're not
> *that* cheap.  And you'll build time bombs into the VOTables for all
> but the most trustworthy producers, which also seems unwise.
> 
> So, for now I'd be in favour of inline JSON (if at all).  But I'm
> afraid it won't be just an xtype; I'm sure we'll have to define at
> least to some extent what clients are expected to do with the
> list/dict structures they'd be parsing out of such columns.
> 
>           -- Markus
Hi Markus

I know that JSON IN/INSTEAD of VOTable is a recurrent discussion in 
IVOA. At least in Apps

As this time it come with Igor request on RCSEDv2, I would prefer that 
we first focus on his need.
If I understand, he have a large collection of spectrum 2D and 3D from 
different instruments and he want to query the collection with deep 
precision.
My first idea is to know what parameters are not in ObsTap that fulfill 
his request. If JSON field is structured by the provider is it the 
"good" answer ?
How client/apps will know about it ?
If the ADQL answer is a combination of different type of metadata 
format, how ObsTAP query can be generic ?
Should this specific JSON field be a complement to the ObsCore 
parameters, and dedicated to a specific service  ?

It seem that you propose a more generic subject, and if I understand the 
apps part of what you proposed. There is that idea to have vectors 
objects and how to handle them.
If JSON format is a good non binary format, it should fit in discussion 
at Interop Apps.
We have to clearly start with simple case.

Regards
Pierre
-- 
-------------------------------------------------------------------------
      	     		   Pierre Le Sidaner
                      Observatoire de Paris - PSL

Directeur de la Direction Informatique de l'Observatoire
Directeur technique de Paris Astronomical Data Centre
tel : 01 40 51 20 82
77 Av Denfert Rochereau 75014 Paris

mailto:pierre.lesidaner at observatoiredeparis.psl.eu
http://dio.obspm.fr  http://padc.obspm.fr

--------------------------------------------------------------------------

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4279 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://mail.ivoa.net/pipermail/dal/attachments/20230911/409d97a2/attachment.p7s>