Generic FIELD/PARAM metadata items in VOTable

[Mark Taylor]

Dear Applications,

this mail is a summary of a proposed modification to VOTable that has
been discussed on Github (https://github.com/ivoa-std/VOTable/issues/29)
and that may make it into the proposed VOTable 1.5; I'm summarising it
for comment on the apps mailing list at the request of Tom Donaldson,
VOTable editor.

Requirement
-----------

People sometimes want to add arbitrary key=value metadata to VOTable FIELD
or PARAM columns, the sort of thing that doesn't fit into the existing
attributes (unit, UCD, xtype, utype).  Some examples:

   - Labelling DataLink PARAMs as mandatory or optional
     (https://github.com/ivoa-std/DataLink/issues/51)

   - Indicating HEALPix order for a column containing a HEALPix index
     (http://mail.ivoa.net/pipermail/apps/2016-August/001131.html)

   - Domain-specific standard metadata items from outside of astronomy
     (CAIO ATTRIBUTE at https://www.cosmos.esa.int/web/csa-guide/tap-tables-and-views)

At present there's really no way to do this, though in some cases it's
possible to achieve the required effect by ad hoc abuse of some underused
VOTable elements or attributes.

I would like to see a way to associate arbitrary key/value pairs with a
FIELD/PARAM to address issues like the above, and others we haven't foreseen.
The idea would not be to associate any semantics to such per-column metadata 
within the VOTable standard, though other client standards or applications 
could do that using their own key vocabularies if they wanted to.  
I don't think the values need to be typed (i.e. key and value can just
be strings as far as VOTable is concerned).

Solutions
---------

Since multiple instances per FIELD/PARAM might in principle be required,
the obvious thing is to use child elements each with a key and value attribute.
Some possibilities:

   (1) Allow FIELD/PARAM to contain INFO children:

        <FIELD name="healpix_id" datatype="int">
          <INFO name="healpix_order" value="8"/>
        </FIELD>

   (2) Invent a new element for this purpose, say META:

        <FIELD name="healpix_id" datatype="int">
          <META key="healpix_order" value="8"/>
        </FIELD>

   (3) Use the existing LINK element using RDF to indicate semantics:

        <FIELD name="healpix_id" datatype="int">
          <LINK action="rdf" content-role="#healpix_order" value="8"/>
        </FIELD>

(1) and (2) would require modifications to the VOTable schema.
(1) is arguably less disruptive since it doesn't introduce a new element;
however it may be more prone to confusing existing clients, which may assume
that an INFO anywhere within a TABLE represents table-level, rather than
column-level, metadata.

(3) requires no change to the VOTable schema, the only change required is
an explanation somewhere in the document text about what this means, 
and that this pattern is the recommended way to do this sort of thing.

Markus and I have had discussions on the relative merits of these options
at https://github.com/ivoa-std/VOTable/issues/29.
Markus likes (3) because it fits into RDF semantic technology;
I find (3) obscure (not obvious when reading what it means, not obvious
when writing that this is how to communicate key=value intent)
and therefore tend to favour (1) or (2) (probably (2)).
But the fact that (3) requires no schema changes is clearly a significant
bonus.

I think either of us could live with either solution.
Markus feel free to correct or clarify any of the above.

Discussion
----------

So, do others have opinions on:

  (a) whether this is a requirement worth expending effort to satisfy
  (b) which of options (1), (2), (3) or (other) is preferred

I guess initial followups should go to this list, but presumably the discussion
will make its way back to https://github.com/ivoa-std/VOTable/issues/29
eventually; feel free to consult that Issue for more detail on the summary
above.

Mark

--
Mark Taylor  Astronomical Programmer  Physics, Bristol University, UK
m.b.taylor at bristol.ac.uk          https://www.star.bristol.ac.uk/mbt/