Support for data containing NaN values

[Tom McGlynn]

I'm not sure that Mark's comments really addresses the Randy's 
question.  Suppose we have an original dataset, O, in some non-VO 
format and a VO serialization of this dataset, V.  Both O and V may 
contain NaNs.   As Mark points out a NaN in V is the recommended 
representation for a null value. So in any context in which null 
values are distinct from NaNs, VOTables cannot distinguish them, i.e., 
a NaN in the VOTable does not in general mean that there was a NaN in 
the original data.  So if you wish to preserve NaNs VOTables are not 
currently a safe way to do so.

As alluded to, this particular issue has been discussed before and 
some thoughts have been collected at
http://www.ivoa.net/cgi-bin/twiki/bin/view/IVOA/VOTableIssues.

	Tom

Mark Taylor wrote:
> Mike,
>
> in fact VOTable does permit NaN values in (single or double) floating
> point data; I think this is a consequence of the design decision
> that the VOTable model for data is to be as close as possible to
> that of FITS.  Furthermore, NaN is how VOTable recommends to represent
> NULL values in floating point data (again, following FITS) - whether
> that's a good idea or not is a question that has been debated elsewhere,
> but that's what VOTable section 6 says
> (http://www.ivoa.net/Documents/VOTable/20091130/REC-VOTable-1.2.html#ToC41)
>
> Mark
>
> On Mon, 26 Sep 2011, Mike Fitzpatrick wrote:
>
>> I took the question differently:  If VO allows FITS  data, and
>> FITS data allows NaN, then apps should of course allow
>> for this.  OTOH, if the question is whether "VO data" as is
>> serialized in a VOTable allows NaN values the I think the
>> answer is 'no' (but I'd have to check).  There are similar issues
>> with how NULL values are handled, but again it depends on
>> whether it is in the serialized VOTable or the end data product
>> being accessed.  The DAL protocols don't say anything about
>> NaN/NULL beyond how they might be serialized, FITS is FITS
>> and if that's what the app retrieves in the end and then that is
>> the standard to follow when interpreting the data.  Was that
>> your question?
>>
>> My $0.02,
>> -Mike
>>
>>
>>
>> On Mon, Sep 26, 2011 at 2:06 AM, Mark Taylor<m.b.taylor at bristol.ac.uk>wrote:
>>
>>> On Thu, 22 Sep 2011, Randy Thompson wrote:
>>>
>>>> As a general question, does data containing NaN values
>>>> violate any VO standards or protocols,and if not, should VO
>>>> applications be expected to accept them as input?
>>>
>>> the question is rather broad ("data" can take many forms), but on
>>> the whole the answer is that most standards and software in the VO
>>> should and do behave sensibly in the presence of NaN-valued floating
>>> point values.
>>>
>>> --
>>> Mark Taylor   Astronomical Programmer   Physics, Bristol University, UK
>>> m.b.taylor at bris.ac.uk +44-117-928-8776 http://www.star.bris.ac.uk/~mbt/
>>>
>>
>
> --
> Mark Taylor   Astronomical Programmer   Physics, Bristol University, UK
> m.b.taylor at bris.ac.uk +44-117-928-8776 http://www.star.bris.ac.uk/~mbt/