Support for data containing NaN values

[Francois Ochsenbein (ext.52429)]

Hi Tom,

Just 2 remarks:

* have you in mind any practical case in which a distinction 
  between NaN (not-a-number IEEE pattern) and NULL (in relational
  database jargon) is meaningful ?

* do you know any relational database which accepts to enter
  NaN (and/or +/-Inf) values ? As far as I know, these IEEE 
  floating-point patterns are not handled (the  relational databases
  I use generate arithmetic errors, and refuse the operation which
  could lead to NaN's or Inf's)

And yes, the VOTable document assumes (as FITS does) that NaN
and NULL have the same semantic meaning -- i.e. that the answer 
is "no" to both questions. 

Cheers, francois

>
>I'm not sure that Mark's comments really addresses the Randy's
>question.  Suppose we have an original dataset, O, in some non-VO
>format and a VO serialization of this dataset, V.  Both O and V may
>contain NaNs.   As Mark points out a NaN in V is the recommended
>representation for a null value. So in any context in which null
>values are distinct from NaNs, VOTables cannot distinguish them, i.e.,
>a NaN in the VOTable does not in general mean that there was a NaN in
>the original data.  So if you wish to preserve NaNs VOTables are not
>currently a safe way to do so.
>
>As alluded to, this particular issue has been discussed before and
>some thoughts have been collected at
>http://www.ivoa.net/cgi-bin/twiki/bin/view/IVOA/VOTableIssues.
>
>	Tom
>
>Mark Taylor wrote:
>> Mike,
>>
>> in fact VOTable does permit NaN values in (single or double) floating
>> point data; I think this is a consequence of the design decision
>> that the VOTable model for data is to be as close as possible to
>> that of FITS.  Furthermore, NaN is how VOTable recommends to represent
>> NULL values in floating point data (again, following FITS) - whether
>> that's a good idea or not is a question that has been debated elsewhere,
>> but that's what VOTable section 6 says
>> (http://www.ivoa.net/Documents/VOTable/20091130/REC-VOTable-1.2.html#ToC41)
>>
>> Mark
>>
>> On Mon, 26 Sep 2011, Mike Fitzpatrick wrote:
>>
>>> I took the question differently:  If VO allows FITS  data, and
>>> FITS data allows NaN, then apps should of course allow
>>> for this.  OTOH, if the question is whether "VO data" as is
>>> serialized in a VOTable allows NaN values the I think the
>>> answer is 'no' (but I'd have to check).  There are similar issues
>>> with how NULL values are handled, but again it depends on
>>> whether it is in the serialized VOTable or the end data product
>>> being accessed.  The DAL protocols don't say anything about
>>> NaN/NULL beyond how they might be serialized, FITS is FITS
>>> and if that's what the app retrieves in the end and then that is
>>> the standard to follow when interpreting the data.  Was that
>>> your question?
>>>
>>> My $0.02,
>>> -Mike
>>>
>>>
>>>
>>> On Mon, Sep 26, 2011 at 2:06 AM, Mark Taylor<m.b.taylor at bristol.ac.uk>wrote:
>>>
>>>> On Thu, 22 Sep 2011, Randy Thompson wrote:
>>>>
>>>>> As a general question, does data containing NaN values
>>>>> violate any VO standards or protocols,and if not, should VO
>>>>> applications be expected to accept them as input?
>>>>
>>>> the question is rather broad ("data" can take many forms), but on
>>>> the whole the answer is that most standards and software in the VO
>>>> should and do behave sensibly in the presence of NaN-valued floating
>>>> point values.
>>>>
>>>> --
>>>> Mark Taylor   Astronomical Programmer   Physics, Bristol University, UK
>>>> m.b.taylor at bris.ac.uk +44-117-928-8776 http://www.star.bris.ac.uk/~mbt/
>>>>
>>>
>>
>> --
>> Mark Taylor   Astronomical Programmer   Physics, Bristol University, UK
>> m.b.taylor at bris.ac.uk +44-117-928-8776 http://www.star.bris.ac.uk/~mbt/
=======================================================================
Francois Ochsenbein    ------   Observatoire Astronomique de Strasbourg
   11, rue de l'Universite 67000 STRASBOURG  Phone: +33-(0)368 85 24 29
Email: francois at astro.u-strasbg.fr (France)    Fax: +33-(0)368 85 24 17
=======================================================================