[jmmc-tech-group] SAMP meta-data extensions ?

[Mark Taylor]

On Tue, 20 Nov 2012, Laurent Bourgès wrote:

> > One person's simplicity is often another's complexity, and
> > getting the standards right is usually a matter of trade-offs.
> >
> 
> Agreed, but it should also not lead to relaxed standards which satisfies
> all wishes and provide too many options.

Just so.  Getting to that point (and agreeing what consitutes it)
is the trick.

> > The changes to the VOTable standard currently under way, including the
> > introduction of BINARY2, have been motivated by complaints from people
> > attempting to write VOTables (particularly streaming results from TAP)
> > that it was too complicated to do so given the constraints of VOTable 1.2.
> > I have tried to conduct the debate about the best way to address that
> > as much as possible in public on the mailing list and at IVOA meetings
> > over the last year or so, though public engagement especially on the
> > mailing list has not been very wide.
> >
> 
> What does mean streaming results in few words ?

If you're writing the result of a streamed TAP query (SQL result)
you don't in general know when you start to write it what range of
values will be used in each column, because you haven't seen all
the results yet.  But if you want to accommodate the possibility of
null values in an integer column in VOTable <=1.2 (any serialization)
you are obliged to specify at the start (in the FIELD element)
a magic value which will be used to represent null cells.
If you make a choice which later turns up as an actual cell value,
you're in trouble.

> Why BINARY is better than other data serialization for such case ?

It's not, necessarily (though some argue that BINARY2 represents the
RDBMS notion of null elements better than the other serializations).
But the streaming problem applies to TABLEDATA too - in VOTable <=1.2,
it's required to specify the magic values up front in that case too.

>  > Concerning SAMP meta data, it's the same problem: loose standard i.e.
> > SAMP
> > > meta data have no data model (DM WG) whereas it was done few years ago in
> > > Application Registry DM !
> >
> > It looks to me like the opposite problem (if problem is the right term);
> > you're asking for a smaller VOTable standard and a larger SAMP one.
> >
> 
> That's not so trivial: I am asking for VOTable clarification /
> simplification when handling data and limit optional elements / attributes
> i.e. special cases and only enhance SAMP meta data to include current
> custom meta data that are worth to be shared.
> 
> I thought SAMP meta data were defined as an opened list of key / pair
> values defined outside of the SAMP standard itself like MTypes.

Exactly.

> I want to describe best practices related to SAMP meta data because the
> SAMP standard does not requires any meta data: all are optional.

I am completely in favour of this, and that's what you've done on
the SampXSamp wiki page - I'm very pleased to have those keys listed
there so that if people want to express those things they have
some ready-rolled ones to choose from, and to allow them to add to
the list if they have other requirements not covered there.
It's only adding them to the SAMP standard document itself that
I'm not so sure about.  As you observe, the MType vocabulary is
not standardised (in the IVOA sense) but it's pretty stable,
widely used, and successful.

Mark

--
Mark Taylor   Astronomical Programmer   Physics, Bristol University, UK
m.b.taylor at bris.ac.uk +44-117-9288776  http://www.star.bris.ac.uk/~mbt/