<!DOCTYPE html>
<html>
<head>
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
</head>
<body>
<p>Dear App,<br>
<br>
</p>
<p>This email to bring recent changes in vot-cli to your attention.
<br>
<br>
vot-cli is a young IVOA command line tool to convert VOTable in
both standard <br>
and non-standard formats and to edit VOTable metadata. <br>
</p>
<p>In short, see:
<a class="moz-txt-link-freetext" href="https://github.com/cds-astro/cds-votable-rust/tree/main/crates/cli">https://github.com/cds-astro/cds-votable-rust/tree/main/crates/cli</a></p>
<p><br>
# Context<br>
<br>
A new VOTable parser, in Rust, was presented at the IVOA meeting
in Bologna a year ago.<br>
Back then, a limited CLI demonstration tool existed to convert
small VOTables<br>
back-on-forth in standard XML + non-standard JSON/TOML/YAML while
preserving all information<br>
(except comments) and the order of the VOTable elements.<br>
<br>
A year later (now), the code has been partly restructured and
cleaned up,<br>
bugs and a few misconceptions (read the full XSD, do not trust the
simplified diagrams!)<br>
have been fixed and new features have been added.<br>
And (the object of this email) an enhanced version of vot-cli was
released a few days ago.<br>
<br>
<br>
# Motivation<br>
<br>
Firstly, both the library and the CLI tool meet internal needs
(for example, we use vot-cli to <br>
convert VOTable metadata into TOML that we then edit both manually
and by script).<br>
<br>
In addition, vot-cli has been made to:<br>
* document some possible usages of the Rust VOTable API<br>
* evaluate performances with respect to existing tools<br>
* be used in discussions about what we want/do not want if the
IVOA decided to make a JSON version of VOTable<br>
* ... <br>
<br>
<br>
# vot-cli features<br>
<br>
First, vot-cli:<br>
* <b>is not</b> a VOTable validator and is (so far) VOTable
version agnostic<br>
+ (for a strict validator, see
<a class="moz-txt-link-freetext" href="https://www.star.bris.ac.uk/~mbt/stilts/sun256/votlint.html">https://www.star.bris.ac.uk/~mbt/stilts/sun256/votlint.html</a>)<br>
*<b> does not</b> check UCDs, units, ... validity<br>
*<b> does not</b> support fancy XML like declared entities<br>
<br>
but it allows to:<br>
* <b>convert in-memory</b> fitting VOTable from/to standard<b>
TABLEDATA/BINARY/BINARY2</b> + non-standard <b>TOML/JSON/YAML</b><br>
* <b>convert large single table </b>VOTable<b> in streaming</b>
from TABLEDATA/BINARY/BINARY2 to TABLEDATA/BINARY/BINARY2/CSV<br>
+ e.g. <b>5 seconds </b>to convert a <b>2.8 GB</b>
TABLEDATA file to a 1.1 GB CSV file (on a server with an SSDs raid
and using <b>multi-threading</b>)<br>
* retrieve a 'structure' view of a votable efficiently<br>
* <b>edit metadata</b> information (add/update/rm almost any tag
except the DATA part) efficiently<br>
<br>
vot-cli will continue to evolve, depending on the needs.<br>
<br>
More information (install instructions, help messages, examples,
...) here<br>
<br>
==>
<a class="moz-txt-link-freetext" href="https://github.com/cds-astro/cds-votable-rust/tree/main/crates/cli">https://github.com/cds-astro/cds-votable-rust/tree/main/crates/cli</a>
<==<br>
<br>
<br>
# Main alternatives I am aware of<br>
<br>
* astropy: one can do almost everything he wants on a VOTable
using python<br>
+ main limitations:<br>
- not a CLI: you have to code<br>
- not (yet) possible to deal with a very large table /
performances ?<br>
<br>
* STILTS: one can convert any file (even very large and
multi-table) efficiently, with a lot of options<br>
+ main limitations:<br>
- if data/metadata editing is required, VOTable structure
may change (see votcopy vs tpipe)<br>
- editing of VOTable-specific metadata is not always
possible<br>
(e.g. adding LINKs; see tpipe with colmeta and setparam
for more details)<br>
<br>
<br>
# vot-cli main limitations<br>
<br>
* not as tested and robust as astropy and STILTS (especially on
arrays, rare datatypes, fancy XML usages)<br>
* no streaming mode (yet) for multi-table files (is it a common
use case?)<br>
* no edition of the data part (cannot remove/add/... columns)<br>
* do not support conversion reading/writing external FITS files<br>
* ...<br>
<br>
</p>
<p># About performances<br>
<br>
The single threaded version of vot-cli shows performances similar
to stilts votcopy.<br>
The multi-threaded version, `vot sconvert`with `--parallel`
option, may increase<br>
performances up to x10 depending on the hardware, the type of
conversion, the votable, ...</p>
<p><br>
</p>
<p>Hoping to have aroused the curiosity of a few people...<br>
</p>
<p><br>
</p>
<p><br>
fx<br>
<br>
</p>
</body>
</html>