[Heig] Post running meeting thoughts
Bruno Khelifi
khelifi at apc.in2p3.fr
Sat Feb 7 19:15:19 CET 2026
Hi all,
About "Background images and pixel masks are not response-function data
products", maybe this is the case for X-rays. I won't discuss it.
As reminder, the term `background` is very generic and can be used for
everything. In gamma-ray astronomy, it is from cosmic rays (it is not
broken pixels, that are handled much more earlier during the raw data
processing). In the GeV, TeV, PeV, the background rate is without any
doubt an IRF!
In contrary to X-rays, 3D analysis are routinely made. For that the
counts are compared with the predicted counts, that is the sum of the
ones associated to gamma rays and the ones associated to the background
rate, that are badly classified events as gamma-rays (see our notes).
The estimation of the background rate can not be done on the data,
because they are gamma rays everywhere in the field of view for the
galactic plane (ie one can not use 'OFF' regions). As reminder, the
Fermi bubble or eRosita bubble are going very up in latitudes. Also, one
can not use simulations of cosmic rays to estimate the background,
because the resources would be much too high and also because the
simulations badly reproduce the reality (many studies made since decades
show that). We use a complex pipeline that takes in input data, creates
some exclusion masks iteratively in 3D, generates templates of rate in
an hypercube ( [X,Y] or theta, atmospheric quality observable, optical
efficiency of our instruments, Zenith angles, azimuth angles between of
the geomagnetic effect on the extensive air showers, and reconstructed
energy), curates the data to handle empty bins and low statistics bin,
interpolates this hypercube template to compute the observation-wise
background rate.
For the neutrino telescopes, real data are also used. A specific
pipeline is of use also to compute the background rate.
So, one should keep without any doubt the background rate as data product!
Best,
Bruno
Le 04/02/2026 à 20:38, Dr. Ian N. Evans via heig a écrit :
> Dear Francois,
>
> I consider the arf, rmf, and psf to be response-function data
> products. Background images and pixel masks are not response-function
> data products - they are determined directly from the observation
> event list similarly to a total counts image. Bad pixel is a region
> data product, but it’s something of a gray area since it’s a
> combination of known bad pixel regions plus bad pixel regions derived
> directly from the observation event list.
>
> For the Chandra Source Catalog (CSC) prototype, at least initially we
> plan to expose all of the data products directly to demonstrate that
> the extension provides the flexibility that we need. However in
> production, we likely would not expose all of the data products
> individually but rather combine some of them with the event lists as
> event bundles (at least for the individual observation full-field data
> product set). We would want to expose the individual observation
> event lists individually, but might choose for example to construct an
> event bundle that exposes (at least) the event list, bad pixel
> regions, aspect histogram, and possible aspect solution as a bundle
> since there is very little use for the latter 3 types of data product
> without the event list.
>
> While tying associated and derived data products to an event list in
> an event bundle seems sensible for individual observations, our
> experience is that this isn’t appropriate for the CSC advanced data
> products. Since CSC 2.0 was released we have had millions of catalog
> data product downloads and surveyed our user base as to data product
> usage.
>
> The typical usage patterns for the CSC advanced data products are
> different from the typical usage patterns for individual X-ray
> observation data.
>
> For the latter the user typically downloads the event list and
> ancillary data products (such as responses or other data products that
> can be used to build responses) as a set, and then performs data
> analysis steps directly on the event list using the ancillary data
> products, often after applying spatial/spectral/temporal filters to
> the data. Event bundles facilitate this usage.
>
> For the CSC advanced data products the usage patterns are quite
> different. Many (most) of these advanced data products are derived
> from multiple (in some cases hundreds) observations. Typically the
> users aren’t interested in performing data analysis steps on the event
> lists themselves, and often aren’t interested in knowing which
> observation(s) they are derived from (at least not from the
> perspective of having to perform a data query). They just want (e.g.)
> all the spectra (or light curves, or photometry MPDFs, or ...) in a
> certain region of the sky, or in a given time range, etc. And given
> the data volume that’s all they want. Maybe they’ll come back later
> and ask for a subset of additional data products after they’ve
> performed some preliminary analyses on those data products, but they
> don’t want those up front.
>
> Based on these usage patterns, I think we will likely want to expose
> the remaining CSC data products individually.
>
> Thanks,
> —Ian
>
>> On Jan 27, 2026, at 09:52, BONNAREL FRANCOIS gmail via heig
>> <heig at ivoa.net> wrote:
>>
>> Dear all,
>>
>> After the meeting last week, I was still thinking about what the
>> Chandra prototype could look like
>>
>> For the Paris HESS prototype, I get the idea since a couple of years
>> now.
>>
>> Trying to understand what the CSC data products could be I came back
>> to Ian's Malta interop presentation.
>>
>> I copy/paste here one of the slides where some of these products are
>> described.
>>
>> Before trying to define dataproduct_type vocabulary terms for those
>> products I am wondering if we really need to expose all this data
>> directly in
>>
>> an ObsTAP service.
>>
>> For example background images, psf, pixel mask, bad pixel regions,
>> ARF belong to the "response functions" category if I'm not mistaking.
>>
>> They probably are attached to a photon event list or an image or ....
>>
>> Including all this in the main ObsCore table will overload it very
>> heterogeneously. Some of these response functions will be similar to
>> what we get in other domains (psf) some will be very different and
>> specific to Xray.
>>
>> I understood that the spatial, spectral, time characterization of
>> these specific products could be borrowed from the observation they
>> are associated with. It's ok but is that useful ?
>>
>> For accessing these response functions I can imagine 4 solutions
>> which all will have the advantage to let the OBsTAp service be
>> focused on measurements obtained from the sky at whatever calib level.
>>
>> 1 ) the photon event list and response functions are gathered
>> together in the same tar or archive file (or MEF) which is typed as
>> an event-bundle. Direct access to this bundle from Obstap access_url
>> is then easy. It's the client task to figure out what to do with the
>> content of the bundle.
>>
>> 2 ) the various response material is kept as a set of individual
>> products. All are associated to an event list or an image or a
>> spectrum. In that case ObsTAP point to a datalink response which
>> lists all these different products. The semantics FIELD writes
>> calibration or response function. Content_qalifier FIELD writes the
>> very nature of the product.
>>
>> 3 ) the DataLink reponse content may be organized as a TAP table.
>> It's then possible to query at the same time the ObsTAP table and
>> the DataLink-like table by a join on
>> ObsCore/obs_publisher_did-DataLink/ID
>>
>> 4 ) if we need a more detailed description of the response products
>> to help discover and select them we could imagine creating a specific
>> "response product" table following a specific datamodel as proposed
>> by Mireille in her Gorlitz presentation. This will allow to attach
>> specific eg :
>>
>> - time range to a psf or
>>
>> - specific release date and description to an arf or a bad
>> pixel map
>>
>> -....
>>
>> Natural join on obs_publisher_did in both tables will allow to
>> query those table at the same time with selection criteria from both.
>>
>> Cheers
>>
>> François
>>
>>
>>
>>
>> <ulnvpjC4zXtPT0zn.png>
>>
>> --
>> heig mailing list
>> heig at ivoa.net
>> https://www.google.com/url?q=http://mail.ivoa.net/mailman/listinfo/heig&source=gmail-imap&ust=1770130364000000&usg=AOvVaw1NuTRL3a6Ib8NM2g8f0TM8
>
> —
> Dr. Ian Evans
> *Astrophysicist*
> *Chandra X-ray Center*
> Center for Astrophysics | Harvard & Smithsonian
> Office: (617) 496 7846 | Cell: (617) 699 5152
> 60 Garden Street | MS 81 | Cambridge, MA 02138
>
> PastedGraphic-2.png
>
> PastedGraphic-3.png _
>
> <http://cfa.harvard.edu/>__cfa.harvard.edu
> <http://cfa.harvard.edu/>_ | _Facebook
> <http://cfa.harvard.edu/facebook>_ | _Twitter
> <http://cfa.harvard.edu/twitter>_ | _YouTube
> <http://cfa.harvard.edu/youtube>_ | _Newsletter
> <http://cfa.harvard.edu/newsletter>_
>
>
--
Bruno Khelifi
Physicist at CNRS (laboratory APC, Paris)
Phone: +33.1.57.27.61.58 - Fax: +33.1.57.27.60.71
APC, IN2P3/CNRS - Universite de Paris Cite
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.ivoa.net/pipermail/heig/attachments/20260207/d3efa467/attachment-0001.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: PastedGraphic-2.png
Type: image/png
Size: 581 bytes
Desc: not available
URL: <http://mail.ivoa.net/pipermail/heig/attachments/20260207/d3efa467/attachment-0002.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: PastedGraphic-3.png
Type: image/png
Size: 21717 bytes
Desc: not available
URL: <http://mail.ivoa.net/pipermail/heig/attachments/20260207/d3efa467/attachment-0003.png>
More information about the heig
mailing list