Marking up HEALPix index columns in VOTable

[Thomas Boch]

Thanks Mark for the clarification.
In that case, I would go for having a GROUP describing the parameters 
and involved fields. Something like this:

<GROUP utype="stc:AstroCoordSystem">
         <PARAM
           utype="stc:AstroCoordSystem.SpaceFrame.CoordFlavor"
           value="HEALPIX" name="CoordFlavor" />
         <PARAM
           utype="stc:AstroCoordSystem.SpaceFrame.CoordRefFrame"
           value="ICRS" name="CoordRefFrame" />
	<PARAM
          utype="???:...norder"
          value="8" name="HpxNorder" />
         <PARAM
          utype="???:...ordering"
          value="NESTED" name="HpxOrdering />
         
         <FIELDref
          utype="???:...HpxIdx"
          ref="hpx8" />
</GROUP>

[note: I'm not quite sure what would be the proper utypes to use to 
describe the HEALPix norder, ordering and index.]

In addition to that, you might also want to tag somehow the field nsrc 
to denote it is a cumulative count of something. <FIELD ID="nsrc" 
ucd="src.density" /> might be enough for your example but we need 
something other if we sum on a given parameter (flux for instance).

--
Thomas

Le 08/08/2016 10:49 PM, Mark Taylor a écrit :
> Thomas,
>
> the primary case we're thinking about here is the results of TAP queries.
> For instance, consider the following ADQL, which uses a custom
> User-Defined Function implemented by Markus's TAP service:
>
>      SELECT ivo_healpix_index(raj2000, dej2000, 8) AS hpx8, COUNT(*) as nsrc
>      FROM wise.main
>      GROUP BY hpx8
>      ORDER BY hpx8
>
> This generates a histogram binned by healpix index, or to put it
> another way an all-sky density map.  Since it's TAP, the result
> is a table, typically transferred in VOTable format.  The question is,
> how does the TAP service mark up the column resulting from
> the ivo_healpix_index UDF so that clients know what they're getting
> and can treat them accordingly, e.g. when attempting to make a
> healpix map plot?
>
> More generally, I'm adding various bits of healpix-related
> functionality to topcat/stilts (density map construction and
> visualisation).  This kind of data set really does have the
> form of a table, so it seems like you should be able to serialize
> it with appropriate metadata in VOTable form.  I admit it also
> makes sense to think of some such datasets as image-like in
> some circumstances too.
>
> Mark
>
> On Mon, 8 Aug 2016, Thomas Boch wrote:
>
>> Hi Mark,
>>
>> what's the motivation to have HEALPix maps in VOTable ?
>> Not to say I'm against the idea, but I'd like to understand better the use
>> case first. If had to output HEALPix maps, I'll go with the FITS serialization
>> as binary tables (and associated keywords ORDERING, COORDSYS, NSIDE) which can
>> be produced/is understood by the HEALPix library and the tools using it.
>>
>> Cheers,
>> Thomas
>>
>> Le 08/08/2016 à 18:55, Mark Taylor a écrit :
>>> [I've crossposted this to apps and semantics, but I suggest
>>> followups to Apps only]
>>>
>>> Dear colleagues,
>>>
>>> Gregory Mantelet, Markus Demleitner and I have been having some
>>> discussions about exchange of HEALPix maps within the VO.
>>>
>>> The type of data we're considering is a table for which one column
>>> contains a HEALPix index (an integer which identifies a tile on the sky),
>>> and other columns contain quantities relating to to that tile
>>> (source density, extinction, mean G magnitude, whatever).
>>> The question we're trying to answer is: how does a data producer
>>> mark up the healpix-index column in a VOTable so that consumers
>>> can recognise it as such?
>>>
>>> The following UCD1+ is defined:
>>>
>>>       Q  |  pos.healpix      |  Hierarchical Equal Area IsoLatitude
>>> Pixelization
>>>
>>> So far so good; however, marking up a column like this provides
>>> insufficient information to make it useful, because to make sense
>>> of a HEALPix index you need to know what Order N of tiling it represents
>>> (HEALPix splits the sky into 12*4^N tiles, where N is an integer
>>> that for practical purposes falls in the range 0..29).
>>> Since UCD words cannot be parameterised (beyond composing them with
>>> other UCD words) there doesn't seem to be any way to encode this
>>> Order information in the UCD.
>>>
>>> Markus's clever idea is to do something like this:
>>>
>>>        <FIELD datatype="int" ucd="pos.healpix">
>>>          <VALUES type="legal">
>>>            <MAX>49151</MAX>
>>>          </VALUES>
>>>        </FIELD>
>>>
>>> Since the legal maximum value for the column is 49151 (=12*(4^6)-1),
>>> it's possible to infer that N=6 for this healpix column.  Note that
>>> since type="legal" this does not say that the actual contents of
>>> the column hit that maximum (i.e. some pixels might be absent, but the
>>> declaration would not change).
>>>
>>> This is kind of horrible, but at the same time the meaning is fairly
>>> unambiguous, and it requires no new standardisation activity.
>>> So, we are likely to go ahead with marking up HEALPix columns in
>>> this way (Markus's and Gregory's services will emit VOTables like
>>> this, and topcat will understand them) unless anybody persuades
>>> us different.
>>>
>>> We have considered various other options too, but on balance we
>>> think they're even worse.  Briefly:
>>>
>>>      - Abuse UCDs by trying to parameterise them:
>>>           pos.healpix(6)?
>>>           pos.healpix;index=6?
>>>
>>>      - Introduce new UCD words to (0<=N<=29, so the number of new words
>>>        would be quite, but not ridiculously, large):
>>>           pos.healpix0, pos.healpix1, ... pos.healpix29?
>>>           pos.healpix;arith.0, pos.healpix;arith.1, ... pos.healpix;arith.29
>>>
>>>      - Xtype magic:
>>>           xtype="healpix(6)"
>>>           xtype="tilings:healpix(6)"
>>>
>>>      - Some table-level indication of the order:
>>>           <TABLE>
>>>             <PARAM name="healpix-order" datatype="int" value="6">
>>>             <FIELD datatype=int ucd="pos.healpix"/>
>>>               ...
>>>           </TABLE>
>>>         this would work because in practice, it's hard to imagine you
>>>         could have multiple columns in a single table with different
>>>         healpix orders.  This is somewhat analogous to how the
>>>         ad-hoc FITS healpix serialization works.
>>>         Note it still needs some convention to identify the healpix-order
>>>         PARAM.
>>>
>>>      - Some business with GROUP/FIELDref/PARAM elements at table level.
>>>        Note this also needs some convention to identify the healpix-order.
>>>
>>>      - Reinvent UCD syntax????
>>>
>>>
>>> Actually it is even worse than this: there are two alternative
>>> conventional schemes for assigning HEALPix indices to a tiling
>>> at a given order, known as NESTED and RING, and you need to know
>>> which one is intended.  Furthermore, the actual sky position
>>> corresponding to a given pixel is dependent on how the tiling is
>>> attached to the celestial sphere (e.g. equatorial, ecliptic, galactic).
>>> So even if you know N you don't have enough explicit information
>>> to make sense of a healpix index.  However, we are currently
>>> following MOC in assuming the NESTED scheme, and it's probably
>>> reasonable to take the sky system as implicit, so we're ignoring
>>> those details for now.
>>>
>>> Any bright ideas, wise comments etc welcome.
>>>
>>> Mark
>>>
>>> --
>>> Mark Taylor   Astronomical Programmer   Physics, Bristol University, UK
>>> m.b.taylor at bris.ac.uk +44-117-9288776  http://www.star.bris.ac.uk/~mbt/
> --
> Mark Taylor   Astronomical Programmer   Physics, Bristol University, UK
> m.b.taylor at bris.ac.uk +44-117-9288776  http://www.star.bris.ac.uk/~mbt/