Native support for MOC operations in the Java HEALPix package

Mon Oct 20 14:21:52 CEST 2014

Dear Martin,

Very nice result :)

If I am correct the NESTED ordering uses a Z-order (or Morton) curve 
(which rely on bit interleaving).
It explains why computing a HEALPix index is very fast and does not 
depends on the NORDER.
Do you have the same kind of algorithm for the Peano-Hilbert?
If yes, would you provide a pointer on it?
The only ones I am aware of use recursion or a 'for' loop from 0 to 
NORDER...
If not, what about performances when computing 100_000_000 Peano-Hilbert 
HEALPix indexes?

If performances using Morton and Peano are similar it is (in addition to 
MOCs) interesting to index data (in databases, ...)
since Peano has "better locality-preserving behaviour" than Morton 
(quoting Wikipedia but I heard/read this elsewhere),
so needs less disk access, ...

Even if it will not become a standard, I think it is worth having it in 
the Java HEALPix libraries :)

Best regards,

François-Xavier

Le 20/10/2014 12:37, Mark Taylor a écrit :
> Martin,
>
> that's an interesting research result, and it might be useful for
> improving efficiency in certain closed contexts.  However, in my
> opinion, it is not a good idea to proliferate format variants or
> provide multiple options for exchanging data formats.  So given that
> MOC already declares NESTED as standard, I would not like to see
> this alternative introduced into MOC I/O libraries in a way which
> would lead to incompatible (or less-compatible) MOC instances
> being generated routinely, e.g. from data services.
>
>  From a VO perspective, that is in the context of a range of
> heterogeneous data providers and consumers, providing/acquiring
> data in forms that can be understood and worked with easily
> (e.g. without having to worry about format variants) is a more
> difficult and important goal than ensuring maximal computational
> efficiency.  Usage of such data in any case tends to be I/O rather
> than compute bound.
>
> That's my 2p - others may disagree.
>
> Mark
>
> On Mon, 20 Oct 2014, Martin Reinecke wrote:
>
>> Hi again,
>>
>> I did some further experiments with the HEALPix MOC implementation and
>> noticed that the MOC sizes (i.e. the number of pixel intervals in a MOC)
>> can be reduced by roughly 20% on average if one uses a
>> Peano-Hilbert-ordering of the pixels on the sphere instead of the more
>> conventional NEST ordering.
>> Support for this kind of ordering has been in the HEALPix C++ package
>> for many years now, and porting it to the Java package was a matter of
>> half an hour.
>>
>> Would there be interest in such a kind of MOC?
>>
>> Once generated, the MOC objects would behave in exactly the same way as
>> the currently used MOCs in NEST scheme, they would just be smaller and
>> operations on them would be faster.
>> Conversion between MOCs in Peano and NESTED representations is possible,
>> but fairly slow.
>>
>> Regards,
>>    Martin
>>
>> On 10/15/14 14:16, Martin Reinecke wrote:
>>> Hi,
>>>
>>> I was encouraged by Pierre Fernique to send this announcement to the
>>> IVOA apps mailing list, since there might be broader interest ...
>>>
>>>
>>> After the MOC standard document had been accepted, I started working on
>>> native MOC support in the Java HEALPix package.
>>> My aim was to provide a compact and highly efficient implementation of
>>> the central functionality (MOC construction, input/output and set
>>> operations). Here is a fairly complete list of the current capabilities:
>>>
>>> - import/export from/to FITS (both files and streams are supported)
>>>
>>> - import/export from/to strings (both the basic ASCII and JSON syntax
>>> shown in the MOC standard)
>>>
>>> - import/export from/to a highly compressed bitstream representation,
>>> making use of interpolative coding. These objects are significantly
>>> smaller than the FITS and ASCII representations, and can be read/written
>>> very quickly; they could be useful on a VO server dealing with many MOCs
>>> at the same time.
>>>
>>> - import/export from/to RangeSets of UNIQ pixels
>>>
>>> - set operations (union, intersection, subtraction, overlap test, subset
>>> test, complement) that are even faster than what was available so far,
>>> due to improvements in the RangeSet class.
>>>
>>> - resolution degradation of a MOC to a desired Healpix order. Partially
>>> covered pixels can be kept or discarded, depending on a parameter.
>>>
>>> If you are interested, I have attached a source tarball of the current
>>> Java HEALPix package. The most up-to-date version can always be found in
>>> the subversion repository at http://sourceforge.net/projects/healpix
>>>
>>> Please let me know if you think that essential functionality is still
>>> missing!
>>> Concerning class and function names, I'm still considering various
>>> alternatives to make them as unambiguous as possible. Again, if you have
>>> suggestions in that area, please tell me.
>>>
>>> Best regards,
>>>    Martin Reinecke
>>>
> --
> Mark Taylor   Astronomical Programmer   Physics, Bristol University, UK
> m.b.taylor at bris.ac.uk +44-117-9288776  http://www.star.bris.ac.uk/~mbt/

-- 
François-Xavier Pineau 	Université de Strasbourg 
<http://astro.u-strasbg.fr> 	CDS, Observatoire de Strasbourg
francois-xavier.pineau at astro.unistra.fr 
<mailto:francois-xavier.pineau at astro.unistra.fr> 	11, rue de l'Université
Phone +33 (0)3 68 85 24 14 	F - 67000 Strasbourg
http://snob.u-strasbg.fr/~pineau <http://snob.u-strasbg.fr/%7Epineau> 
http://astro.u-strasbg.fr

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.ivoa.net/pipermail/apps/attachments/20141020/4cff9f5b/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: logoObs_h60l63_Grey.png
Type: image/png
Size: 4261 bytes
Desc: not available
URL: <http://mail.ivoa.net/pipermail/apps/attachments/20141020/4cff9f5b/attachment-0001.png>