Generic FIELD/PARAM metadata items in VOTable

[Mark Taylor]

FX,

I hadn't thought of that, it's definitely a possibility.
The semantics of the various ref/ID linkages are rather under-documented
in VOTable, so like the other options it would need to be written
in the standard what the meaning of this construction would be.
Compared to the other options it's less obvious to a human reader 
what's going on, but it's a bonus that it doesn't require any changes 
to the schema.

One negative consideration is that legacy software (e.g. current
version of STIL/STILTS/TOPCAT) would see such PARAMs, ignore the ref, 
and assume that this was table-level rather than column-level metadata -
but the same might happen for option (1).
You could also end up with multiple PARAMs having the same name, but
referring to different columns, but I don't think there is any rule
against that.

Interested in other people's opinions.

Mark

On Mon, 22 May 2023, Francois-Xavier PINEAU wrote:

> Hi Mark and all,
> 
> Only considering the given example (so the following may be irrelevant), what
> about something like:
> 
> <PARAM name="healpix_order" value="8" ref="healpix_id"/>
> <FIELD ID="healpix_id" name="healpix_id" datatype="int"/>
> 
> which popped up as the more natural way of describing this to me
> (PARAM <=> constant column; with a ref to "link" it to another existing
> column).
> 
> If the order is different for each row, it will naturally be described as
> (italic = optional):
> 
> <FIELD name="healpix_order" datatype="int" /ref="healpix_id"//>
> <FIELD /ID="healpix_id"/ name="healpix_id" datatype="int"/>
> 
> Cheers,
> 
> 
> fx
> 
> 
> Le 17/05/2023 à 18:07, Mark Taylor a écrit :
> > Dear Applications,
> > 
> > this mail is a summary of a proposed modification to VOTable that has
> > been discussed on Github (https://github.com/ivoa-std/VOTable/issues/29)
> > and that may make it into the proposed VOTable 1.5; I'm summarising it
> > for comment on the apps mailing list at the request of Tom Donaldson,
> > VOTable editor.
> > 
> > Requirement
> > -----------
> > 
> > People sometimes want to add arbitrary key=value metadata to VOTable FIELD
> > or PARAM columns, the sort of thing that doesn't fit into the existing
> > attributes (unit, UCD, xtype, utype).  Some examples:
> > 
> >     - Labelling DataLink PARAMs as mandatory or optional
> >       (https://github.com/ivoa-std/DataLink/issues/51)
> > 
> >     - Indicating HEALPix order for a column containing a HEALPix index
> >       (http://mail.ivoa.net/pipermail/apps/2016-August/001131.html)
> > 
> >     - Domain-specific standard metadata items from outside of astronomy
> >       (CAIO ATTRIBUTE
> > athttps://www.cosmos.esa.int/web/csa-guide/tap-tables-and-views)
> > 
> > At present there's really no way to do this, though in some cases it's
> > possible to achieve the required effect by ad hoc abuse of some underused
> > VOTable elements or attributes.
> > 
> > I would like to see a way to associate arbitrary key/value pairs with a
> > FIELD/PARAM to address issues like the above, and others we haven't
> > foreseen.
> > The idea would not be to associate any semantics to such per-column metadata
> > within the VOTable standard, though other client standards or applications
> > could do that using their own key vocabularies if they wanted to.
> > I don't think the values need to be typed (i.e. key and value can just
> > be strings as far as VOTable is concerned).
> > 
> > Solutions
> > ---------
> > 
> > Since multiple instances per FIELD/PARAM might in principle be required,
> > the obvious thing is to use child elements each with a key and value
> > attribute.
> > Some possibilities:
> > 
> >     (1) Allow FIELD/PARAM to contain INFO children:
> > 
> >          <FIELD name="healpix_id" datatype="int">
> >            <INFO name="healpix_order" value="8"/>
> >          </FIELD>
> > 
> >     (2) Invent a new element for this purpose, say META:
> > 
> >          <FIELD name="healpix_id" datatype="int">
> >            <META key="healpix_order" value="8"/>
> >          </FIELD>
> > 
> >     (3) Use the existing LINK element using RDF to indicate semantics:
> > 
> >          <FIELD name="healpix_id" datatype="int">
> >            <LINK action="rdf" content-role="#healpix_order" value="8"/>
> >          </FIELD>
> > 
> > (1) and (2) would require modifications to the VOTable schema.
> > (1) is arguably less disruptive since it doesn't introduce a new element;
> > however it may be more prone to confusing existing clients, which may assume
> > that an INFO anywhere within a TABLE represents table-level, rather than
> > column-level, metadata.
> > 
> > (3) requires no change to the VOTable schema, the only change required is
> > an explanation somewhere in the document text about what this means,
> > and that this pattern is the recommended way to do this sort of thing.
> > 
> > Markus and I have had discussions on the relative merits of these options
> > athttps://github.com/ivoa-std/VOTable/issues/29.
> > Markus likes (3) because it fits into RDF semantic technology;
> > I find (3) obscure (not obvious when reading what it means, not obvious
> > when writing that this is how to communicate key=value intent)
> > and therefore tend to favour (1) or (2) (probably (2)).
> > But the fact that (3) requires no schema changes is clearly a significant
> > bonus.
> > 
> > I think either of us could live with either solution.
> > Markus feel free to correct or clarify any of the above.
> > 
> > Discussion
> > ----------
> > 
> > So, do others have opinions on:
> > 
> >    (a) whether this is a requirement worth expending effort to satisfy
> >    (b) which of options (1), (2), (3) or (other) is preferred
> > 
> > I guess initial followups should go to this list, but presumably the
> > discussion
> > will make its way back tohttps://github.com/ivoa-std/VOTable/issues/29
> > eventually; feel free to consult that Issue for more detail on the summary
> > above.
> > 
> > Mark
> > 
> > --
> > Mark Taylor  Astronomical Programmer  Physics, Bristol University, UK
> > m.b.taylor at bristol.ac.uk           https://www.star.bristol.ac.uk/mbt/
> 

--
Mark Taylor  Astronomical Programmer  Physics, Bristol University, UK
m.b.taylor at bristol.ac.uk          https://www.star.bristol.ac.uk/mbt/