detection limit

[Arnold Rots]

I don't think you are drawing the correct conclusions; I would say:

1. What Francois proposes will not work (well) for HEA data
2. We should keep looking for something that works better, but in the
   meantime: 
   3. It may be wise to maintain the distinction between resolution and
      sensitivity
   4. It should definitely be optional

Cheers,

  - Arnold


Rob Seaman wrote:
> Several minutes of rummaging have failed to rustle up the ObsTap use cases to understand the context here.
> 
> Regarding error depending on the flux, most astronomical data including vanilla CCD imaging will have a poisson noise model (or rather poisson + gaussian background).  I presume a requirement for "some estimation of the detection limit" might be expressed similar to the fpack "q" scaling as fractional binning of the standard deviation of the background peak of the pixel ("observable axis"?) histogram.  In photon counting regimes like Arnold is discussing this then becomes an issue of identifying a physically pertinent histogram...
> 
> The question, I guess, is where to draw the line in subsetting the full Platonic observation data model that somehow manages to capture all the richness of the vast panoply of astronomical instrumentation across all physical regimes.
> 
> Should be an interesting TCG meeting :-)
> 
> Rob
> --
> On Apr 27, 2011, at 1:07 PM, Arnold Rots wrote:
> 
> > There are a number of problems associated with this approach, in our
> > case.
> > 
> > As you are undoubtedly aware, we tend to measure individual photons
> > which means (a) that the conversion to energy-expressing units is
> > model-dependent, and (b) the error or resolution depends on the flux -
> > the result of being in the Poisson error domain.
> > 
> > Third, we cannot separate "signal" from flux since our PSF varies
> > very significantly across the field of view.
> > 
> > And fourth, sensitivity (however defined) varies across the FOV, partly
> > (but not exclusively) due to the previous point.
> > 
> > I'm afraid that the bottom line is that we need a more sophisticated
> > model, if this is to be remotely useful.
> > 
> > Cheers,
> > 
> >  - Arnold
> > 
> > 
> > Douglas Tody wrote:
> > [ Charset ISO-8859-1 unsupported, converting... ]
> >> Hi Francois -
> >> 
> >> I have always thought it was important to have this attribute; for 
> >> global/multiwave use cases it is more important than exposure time for
> >> example, although it serves a similar purpose.  However it can be
> >> complex to provide, similar to converting photometric magnitudes to
> >> fluxes.
> >> 
> >> Calling this the "resolution" of the observable axis is ok I guess.
> >> Note it is closely related to the RMS of the background and is often
> >> expressed that way instead.  We would want to define how these two
> >> are related.  Unit probably Jansky.
> >> 
> >> I think it should be a "should" attribute (i.e., optional), due to the
> >> difficulty of computation.  However it is highly desirable for many
> >> data discovery use cases.
> >> 
> >> 	- Doug
> >> 
> >> 
> >> On Thu, 7 Apr 2011, Fran?ois Bonnarel wrote:
> >> 
> >>> Hi all,
> >>> Going on with searching what kind of initial ObsTap driver use cases
> >>> could have been forgotten I exhumated use case 3.1 and 5.2.  which
> >>> requires some estimation of the detection limit. after discussing with
> >>> Mireille, I decided to  ask everybody:
> >>> 
> >>> For datasets we must be talking of "signal"  detection limit measured on
> >>> the Observable axis, which cannot be confused with the detection limit on 
> >>> sources
> >>> magnitudes (or line flux) which implies flux integration on the flux area 
> >>> convolved
> >>> with the psf (lsf).
> >>> The signal detection limit or inverse sensitivity for a given observation is
> >>> defined as the minimal detectable difference of measured quantities
> >>> along the observable axis. In case we are measuring a flux-related
> >>> observable, it is the minimal detectable flux density difference,
> >>> limited by the noise. In term of signal theory this is exactly the flux
> >>> resolution.
> >>> 
> >>> The utype attached to this attribute could then logically be:
> >>> Char.observableAxis.resolution.refval, and the name o_detection_limit
> >>> 
> >>> It will be sometimes difficult for  data managers to provide this field.
> >>> 
> >>> QUESTION: do we implement this field as optional ? Will be described in 
> >>> Annex
> >>> in that case
> >>> 
> >>> Cheers
> >>> Fran?ois
> >>> 
> >>> 
> > --------------------------------------------------------------------------
> > Arnold H. Rots                                Chandra X-ray Science Center
> > Smithsonian Astrophysical Observatory                tel:  +1 617 496 7701
> > 60 Garden Street, MS 67                              fax:  +1 617 495 7356
> > Cambridge, MA 02138                             arots at head.cfa.harvard.edu
> > USA                                     http://hea-www.harvard.edu/~arots/
> > --------------------------------------------------------------------------
> 
> 
--------------------------------------------------------------------------
Arnold H. Rots                                Chandra X-ray Science Center
Smithsonian Astrophysical Observatory                tel:  +1 617 496 7701
60 Garden Street, MS 67                              fax:  +1 617 495 7356
Cambridge, MA 02138                             arots at head.cfa.harvard.edu
USA                                     http://hea-www.harvard.edu/~arots/
--------------------------------------------------------------------------