Proposed erratum to clarify arraysize="1"

[Markus Demleitner]

Hi Mark,

On Thu, Feb 15, 2018 at 10:36:21AM +0000, Mark Taylor wrote:
> On Mon, 12 Feb 2018, Tom Donaldson wrote:
> > That is what the spec should have said.  (I do see this as a clarification
> > rather than a change.  Section 4.1 does already say, ???The arraysize
> > attribute exists when the corresponding table cell contains more than
> > one??????. )
> 
> Yes well that does make me think.  That spec text which you quote
> implies, though it does not explicitly state, that the use of
> arraysize="1" is simply incorrect.

[copying from the end of Mark's mail:]

> The downside of R2 [outlawing arraysize=1] 
> is that there is then no way to specify a
> 1-element array.  Question: is that a problem?  Does anybody
> actually need to communicate 1-element arrays as distinct from scalars?
> As I pointed out earlier in this thread, FITS BINTABLE has got away
> with that situation since its invention.

Well... Who knows what hacks may have been concocted somewhere to
work around this limitation/ambiguity/whatever?

The mathematician says no to treating 1 somehow special (even more so
since I think we're not outlawing arraysize=0, for which I can also
see use cases, specifically when applied with PARAMs).  So, this
would not be my preferred outcome [1].

> But in practical terms, the Erratum is saying:
>  
>    R1:
>       - if arraysize="1" is present, it means a 1-element array
>       - don't write arraysize="1" unless you mean a 1-element array
> 
> Given that ruling:
> 
>    1a) VOTable producers should reconsider whether they really mean
>           arraysize="1"
>    1b) VOTable validators should present a warning for arraysize="1"
>           that producers may or may not need to do something about
>    1c) Astropy-like clients can carry on unchanged
>    1d) STIL-like clients should go from treating arraysize="1" as a scalar
>           to treating it like an array (but may wish to exercise flexibility)
> 
> As I've said in an earlier message, I'm not going to make the change (1d)
> any time soon, at least in the default configuration of TOPCAT/STILTS,
> because it's likely to break stuff.

Yes.  I see that, and I've conceded that I have every sympathy for
this stance (although I'd appreciate if STILTS-like clients gave
their users some means to have arrays from arraysize=1 FIELDs and
PARAMs in the future).

With this, I'd describe the situation as:

* I'd like to stop diverging behaviour (essentially by getting rid of
  arraysize=1-producing services as fast as possible; let's see what
  we can do on the client side)
* You'd like to not break what people have been doing with your
  programs

I'd suspect there is language for the Erratum that lets us have both
of these.

What about this for "Change of standards text":

  Section 2.2 - append the following text to end of the section:

    Note: arraysize must be written if, and only if, each table cell
    for the FIELD is intended to be treated as an array. arraysize="1"
    hence denotes in the unusual case that the table cells will
    contain single values that are intended to be understood as
    single-value arrays.

    For the time being, parsers MAY interpret FIELDs with arraysize="1"
    as scalars but SHOULD give warnings if they do so.  As long as this
    legacy exception is present in the VOTable standard, VOTable
    creators should avoid writing VOTables with 1-element arrays.

  Section 4.1 - append the following text to the arraysize bullet:

    On writing, the arraysize attribute must be omitted unless the
    corresponding table cell contents is intended to be understood as
    an array.

Or do you forsee trouble when silently swallowing arraysize="1" when
roundtripping a VOTable?

       -- Markus


[1] In case you're looking for a concrete use case: I think a cube
data model should have the lengths of the array axes denoted
somewhere, and I'd feel it would be really appropriate to serialise
these as a VOTable PARAM (proper annotation omitted):

  <PARAM name="axis_length" datatype="integer" arraysize="3"
    value="1024 1024 300"/>

And now imagine you just have a time series with just one
non-degenerate axis...