VOParquet document

[Major, Brian]

Hi Mark, hi Apps,

Thanks for putting that together so quickly.  I just wanted to inform everyone that we are working on an implementation of this in our OpenCADC TAP service.  One question has already arisen about error handling, so expect to see a github issue on that topic soon.

Cheers,
Brian

On 2024-11-26, 4:14 AM, "apps on behalf of Mark Taylor via apps" <apps-bounces at ivoa.net <mailto:apps-bounces at ivoa.net> on behalf of apps at ivoa.net <mailto:apps at ivoa.net>> wrote:


***Attention*** This email originated from outside of the NRC. ***Attention*** Ce courriel provient de l'extérieur du CNRC.


Dear Apps,


I have drafted a document on use of the parquet file format in the VO,
specifically defining a covention for use of VOTable to add rich
metadata to parquet files. You can find it on github here:


https://github.com/ivoa/voparquet <https://github.com/ivoa/voparquet>


I have started a few Issues to invite comments on particular aspects of
the text; if you are interested please contribute to these, create new
issues, or submit pull requests.


As agreed in the Apps telecon of 5 November and at the Malta Interop,
the aim is to get agreement on this on a short timescale, so that
projects that are writing or about to write parquet files in bulk
can do so interoperably.


My suggestion therefore is to gather comments and make updates by
*the end of this calendar year*, so it can be published
(as a Note or possibly towards Endorsed Note, TBD) in the first
weeks of 2025. If there are major disagreements that schedule
might have to slip, but I'm currently optimistic that won't
be the case.


If you think that this timescale is too ambitious, please follow up
this message.


Thanks


Mark


--
Mark Taylor Astronomical Programmer Physics, Bristol University, UK
m.b.taylor at bristol.ac.uk <mailto:m.b.taylor at bristol.ac.uk> https://www.star.bristol.ac.uk/mbt/ <https://www.star.bristol.ac.uk/mbt/>