Native support for MOC operations in the Java HEALPix package

[Mark Taylor]

Martin,

that's an interesting research result, and it might be useful for
improving efficiency in certain closed contexts.  However, in my
opinion, it is not a good idea to proliferate format variants or
provide multiple options for exchanging data formats.  So given that
MOC already declares NESTED as standard, I would not like to see
this alternative introduced into MOC I/O libraries in a way which
would lead to incompatible (or less-compatible) MOC instances
being generated routinely, e.g. from data services.

>From a VO perspective, that is in the context of a range of 
heterogeneous data providers and consumers, providing/acquiring
data in forms that can be understood and worked with easily
(e.g. without having to worry about format variants) is a more
difficult and important goal than ensuring maximal computational
efficiency.  Usage of such data in any case tends to be I/O rather
than compute bound.

That's my 2p - others may disagree.

Mark

On Mon, 20 Oct 2014, Martin Reinecke wrote:

> Hi again,
> 
> I did some further experiments with the HEALPix MOC implementation and
> noticed that the MOC sizes (i.e. the number of pixel intervals in a MOC)
> can be reduced by roughly 20% on average if one uses a
> Peano-Hilbert-ordering of the pixels on the sphere instead of the more
> conventional NEST ordering.
> Support for this kind of ordering has been in the HEALPix C++ package
> for many years now, and porting it to the Java package was a matter of
> half an hour.
> 
> Would there be interest in such a kind of MOC?
> 
> Once generated, the MOC objects would behave in exactly the same way as
> the currently used MOCs in NEST scheme, they would just be smaller and
> operations on them would be faster.
> Conversion between MOCs in Peano and NESTED representations is possible,
> but fairly slow.
> 
> Regards,
>   Martin
> 
> On 10/15/14 14:16, Martin Reinecke wrote:
> > Hi,
> > 
> > I was encouraged by Pierre Fernique to send this announcement to the
> > IVOA apps mailing list, since there might be broader interest ...
> > 
> > 
> > After the MOC standard document had been accepted, I started working on
> > native MOC support in the Java HEALPix package.
> > My aim was to provide a compact and highly efficient implementation of
> > the central functionality (MOC construction, input/output and set
> > operations). Here is a fairly complete list of the current capabilities:
> > 
> > - import/export from/to FITS (both files and streams are supported)
> > 
> > - import/export from/to strings (both the basic ASCII and JSON syntax
> > shown in the MOC standard)
> > 
> > - import/export from/to a highly compressed bitstream representation,
> > making use of interpolative coding. These objects are significantly
> > smaller than the FITS and ASCII representations, and can be read/written
> > very quickly; they could be useful on a VO server dealing with many MOCs
> > at the same time.
> > 
> > - import/export from/to RangeSets of UNIQ pixels
> > 
> > - set operations (union, intersection, subtraction, overlap test, subset
> > test, complement) that are even faster than what was available so far,
> > due to improvements in the RangeSet class.
> > 
> > - resolution degradation of a MOC to a desired Healpix order. Partially
> > covered pixels can be kept or discarded, depending on a parameter.
> > 
> > If you are interested, I have attached a source tarball of the current
> > Java HEALPix package. The most up-to-date version can always be found in
> > the subversion repository at http://sourceforge.net/projects/healpix
> > 
> > Please let me know if you think that essential functionality is still
> > missing!
> > Concerning class and function names, I'm still considering various
> > alternatives to make them as unambiguous as possible. Again, if you have
> > suggestions in that area, please tell me.
> > 
> > Best regards,
> >   Martin Reinecke
> > 
> 

--
Mark Taylor   Astronomical Programmer   Physics, Bristol University, UK
m.b.taylor at bris.ac.uk +44-117-9288776  http://www.star.bris.ac.uk/~mbt/