Arrays of geometries?

[Markus Demleitner]

Hi DAL,

On Tue, Sep 05, 2017 at 03:43:36PM -0700, Patrick Dowler wrote:
> In principle this seems sound because xtype tells you how to interpret
> the (array of) primitives into some other structure. But: IIRC, the
> wildcard is only allowed in the last dimension in VOTable.

Yes, and given the way we're doing our serialisations, there's really
no way to have variable length in anything but the last dimension[1];
in that sense, it's not just a rule, it's an inherent constraint of
VOTable.

> For xtype='timestamp" you could work around that by knowing the length
> (precision) of the timestamp(s) you were going to write.... then it
> woud be:
> 
> <FIELD datatype="char" arraysize="10x*" xtype="timestamp"...>
> 
> for a variable length array of date-only timestamps, and maybe
> 
> <FIELD datatype="char" arraysize="23x*" xtype="timestamp"...>
> 
> for millisecond precision timestamps.
> 
> Also, point, circle, and interval arrays could work, but the one you
> cannot do is polygon[n] because polygons really do have variable
> length.

Well, one *could* argue that there are valid use cases where you know
the length of the vertex sequence (WFPC images, for instance) --
actually, I suspect for the time being these would be the majority.

Now, if there's rough agreement we should strive for allowing
nD-arrays with xtype, do we tell people, and if so, how?

*I* guess we should tell people, because this interpretation isn't
obvious, and at least in my implementation the intended behaviour
doesn't come naturally (it's not in there right now).  So, I don't
think we'll get interoperable VOTable libraries on this point without
additional specification effort.

One way would be an erratum to DALI, saying something like:

  ---++ Rationale

  Sect. 3.3 has rules on how DAL clients and VOTable libraries should
  interpret certain constructs using the xtype attribute.  For
  instance, a 

    <FIELD datatype="double" arraysize="2" xtype="point"/>

  should be deserialised to a native spherical point object if
  available.  The text in the 1.1 Recommendation, however, neglects
  to specify if this special interpretation extends to
  multidimensional arrays, e.g.

    <FIELD datatype="double" arraysize="2x4" xtype="point"/>

  or even

    <FIELD datatype="double" arraysize="2x*" xtype="point"/>

  As this is a potential issue for the interoperability of VOTable
  libraries, this erratum specifies the behaviour in such cases.

  ---++ Erratum Content

  After the opening paragraphs of section 3.3., the following text is
  inserted:

    The xtype attribute applies to the combination of the datatype
    and the first dimension in arraysize in VOTable metadata.  This
    means, in particular, that arrays of such complex objects are
    supported. For example, it is legal to combine xtype="point" with
    an arraysize of 2x5 to obtain a 5-array of points.

    For xtypes with a natural arraysize="*" (in this specification,
    polygon and timestamp), this means that serialisers have to
    determine a maximum length up front, as VOTable does not support
    arrays of variable-length arrays.

  ---++ Impact assessment

  This erratum specifies behaviour that has been undefined so far.
  It is conceivable that previously standards-compliant
  implementations that chose to treat arraysize and xtype differently
  become invalid now.  Given that DALI 1.1 is relatively new, it
  seems unlikely that this could be an issue in practice, in
  particular because it will probably be a while before VOTables
  exploiting this will appear in the wild.  On the other hand, a
  clear definition of the desired behaviour appears useful in
  controlling future interoperability issues.

How do people feel about pursuing this?  Anyone in for sponsoring
this Erratum (or some suitably edited version of it)?

          -- Markus

[1] Which is one reason why I've always dreamed of a proper string type
in VOTable so we can have proper arrays of strings.  In case you're
feeling this itch, too, let me know.