JSON type in ADQL and TAP

[Markus Demleitner]

Dear DAL,

On Fri, Sep 08, 2023 at 05:28:39PM +0200, Gregory MANTELET wrote:
> Personally, I agree with Igor on the fact that JSON data are now everywhere,
> including in databases. So, at some point, we will have to deal with such
> things.
>
> On the other hand, I also agree with Markus: having JSON data encapsulated
> inside tables is kind of nasty in term of user experience. Generally, we
> expect data that we can easily read, but this is not immediately possible
> with JSON data (especially with long and/or complex data structure) ; a
> parsing/query step is needed most of the time if the table is exported in
> something else than a JSON file. A better approach could be to instead
> provide a link (datalink involved somewhere ?) toward this JSON document ;
> but not sure, it is a better user experience either.
>
> This topic could probably be discussed in more details during a DAL running
> meeting or during an interop meeting.

I think a discussion session on this -- specifically, JSON in
VOTable, not JSON in ADQL, which I consider much less problematic --,
perhaps with one or two 5-minute introductory talks, would be a
great thing at this or the next interop.  If it doesn't fit in
Tucson, perhaps we can have a smaller side meeting during ADASS time?

By the way, my instinct on externalising JSON is that we'd regret
that; for one it'd be one HTTP request per VOTable row, and while I
consider HTTP requests fairly cheap since HTTP/1.1, they're not
*that* cheap.  And you'll build time bombs into the VOTables for all
but the most trustworthy producers, which also seems unwise.

So, for now I'd be in favour of inline JSON (if at all).  But I'm
afraid it won't be just an xtype; I'm sure we'll have to define at
least to some extent what clients are expected to do with the
list/dict structures they'd be parsing out of such columns.

         -- Markus