[Heig] Post running meeting thoughts
BONNAREL FRANCOIS gmail
francois.bonnarel at gmail.com
Mon Feb 9 14:26:17 CET 2026
Dear Ian, all,
A small question on one point.
Le 04/02/2026 à 20:38, Dr. Ian N. Evans a écrit :
> Dear Francois,
>
> I consider the arf, rmf, and psf to be response-function data products.
OK
> Background images and pixel masks are not response-function data
> products - they are determined directly from the observation event
> list similarly to a total counts image. Bad pixel is a region data
> product, but it’s something of a gray area since it’s a combination of
> known bad pixel regions plus bad pixel regions derived directly from
> the observation event list.
In that case what could be the "semantics" term for those in case of
DataLink access : auxiliary ? They are not summarized in table 5.1.5 of
the draft
Cheers
François
>
> For the Chandra Source Catalog (CSC) prototype, at least initially we
> plan to expose all of the data products directly to demonstrate that
> the extension provides the flexibility that we need. However in
> production, we likely would not expose all of the data products
> individually but rather combine some of them with the event lists as
> event bundles (at least for the individual observation full-field data
> product set). We would want to expose the individual observation
> event lists individually, but might choose for example to construct an
> event bundle that exposes (at least) the event list, bad pixel
> regions, aspect histogram, and possible aspect solution as a bundle
> since there is very little use for the latter 3 types of data product
> without the event list.
>
> While tying associated and derived data products to an event list in
> an event bundle seems sensible for individual observations, our
> experience is that this isn’t appropriate for the CSC advanced data
> products. Since CSC 2.0 was released we have had millions of catalog
> data product downloads and surveyed our user base as to data product
> usage.
>
> The typical usage patterns for the CSC advanced data products are
> different from the typical usage patterns for individual X-ray
> observation data.
>
> For the latter the user typically downloads the event list and
> ancillary data products (such as responses or other data products that
> can be used to build responses) as a set, and then performs data
> analysis steps directly on the event list using the ancillary data
> products, often after applying spatial/spectral/temporal filters to
> the data. Event bundles facilitate this usage.
>
> For the CSC advanced data products the usage patterns are quite
> different. Many (most) of these advanced data products are derived
> from multiple (in some cases hundreds) observations. Typically the
> users aren’t interested in performing data analysis steps on the event
> lists themselves, and often aren’t interested in knowing which
> observation(s) they are derived from (at least not from the
> perspective of having to perform a data query). They just want (e.g.)
> all the spectra (or light curves, or photometry MPDFs, or ...) in a
> certain region of the sky, or in a given time range, etc. And given
> the data volume that’s all they want. Maybe they’ll come back later
> and ask for a subset of additional data products after they’ve
> performed some preliminary analyses on those data products, but they
> don’t want those up front.
>
> Based on these usage patterns, I think we will likely want to expose
> the remaining CSC data products individually.
>
> Thanks,
> —Ian
>
>> On Jan 27, 2026, at 09:52, BONNAREL FRANCOIS gmail via heig
>> <heig at ivoa.net> wrote:
>>
>> Dear all,
>>
>> After the meeting last week, I was still thinking about what the
>> Chandra prototype could look like
>>
>> For the Paris HESS prototype, I get the idea since a couple of years
>> now.
>>
>> Trying to understand what the CSC data products could be I came back
>> to Ian's Malta interop presentation.
>>
>> I copy/paste here one of the slides where some of these products are
>> described.
>>
>> Before trying to define dataproduct_type vocabulary terms for those
>> products I am wondering if we really need to expose all this data
>> directly in
>>
>> an ObsTAP service.
>>
>> For example background images, psf, pixel mask, bad pixel regions,
>> ARF belong to the "response functions" category if I'm not mistaking.
>>
>> They probably are attached to a photon event list or an image or ....
>>
>> Including all this in the main ObsCore table will overload it very
>> heterogeneously. Some of these response functions will be similar to
>> what we get in other domains (psf) some will be very different and
>> specific to Xray.
>>
>> I understood that the spatial, spectral, time characterization of
>> these specific products could be borrowed from the observation they
>> are associated with. It's ok but is that useful ?
>>
>> For accessing these response functions I can imagine 4 solutions
>> which all will have the advantage to let the OBsTAp service be
>> focused on measurements obtained from the sky at whatever calib level.
>>
>> 1 ) the photon event list and response functions are gathered
>> together in the same tar or archive file (or MEF) which is typed as
>> an event-bundle. Direct access to this bundle from Obstap access_url
>> is then easy. It's the client task to figure out what to do with the
>> content of the bundle.
>>
>> 2 ) the various response material is kept as a set of individual
>> products. All are associated to an event list or an image or a
>> spectrum. In that case ObsTAP point to a datalink response which
>> lists all these different products. The semantics FIELD writes
>> calibration or response function. Content_qalifier FIELD writes the
>> very nature of the product.
>>
>> 3 ) the DataLink reponse content may be organized as a TAP table.
>> It's then possible to query at the same time the ObsTAP table and
>> the DataLink-like table by a join on
>> ObsCore/obs_publisher_did-DataLink/ID
>>
>> 4 ) if we need a more detailed description of the response products
>> to help discover and select them we could imagine creating a specific
>> "response product" table following a specific datamodel as proposed
>> by Mireille in her Gorlitz presentation. This will allow to attach
>> specific eg :
>>
>> - time range to a psf or
>>
>> - specific release date and description to an arf or a bad
>> pixel map
>>
>> -....
>>
>> Natural join on obs_publisher_did in both tables will allow to
>> query those table at the same time with selection criteria from both.
>>
>> Cheers
>>
>> François
>>
>>
>>
>>
>> <ulnvpjC4zXtPT0zn.png>
>>
>> --
>> heig mailing list
>> heig at ivoa.net
>> https://www.google.com/url?q=http://mail.ivoa.net/mailman/listinfo/heig&source=gmail-imap&ust=1770130364000000&usg=AOvVaw1NuTRL3a6Ib8NM2g8f0TM8
>
> —
> Dr. Ian Evans
> *Astrophysicist*
> *Chandra X-ray Center*
> Center for Astrophysics | Harvard & Smithsonian
> Office: (617) 496 7846 | Cell: (617) 699 5152
> 60 Garden Street | MS 81 | Cambridge, MA 02138
>
> PastedGraphic-2.png
>
> PastedGraphic-3.png _
>
> <http://cfa.harvard.edu/>__cfa.harvard.edu
> <http://cfa.harvard.edu/>_ | _Facebook
> <http://cfa.harvard.edu/facebook>_ | _Twitter
> <http://cfa.harvard.edu/twitter>_ | _YouTube
> <http://cfa.harvard.edu/youtube>_ | _Newsletter
> <http://cfa.harvard.edu/newsletter>_
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.ivoa.net/pipermail/heig/attachments/20260209/182812fd/attachment-0001.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: PastedGraphic-2.png
Type: image/png
Size: 581 bytes
Desc: not available
URL: <http://mail.ivoa.net/pipermail/heig/attachments/20260209/182812fd/attachment-0002.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: PastedGraphic-3.png
Type: image/png
Size: 21717 bytes
Desc: not available
URL: <http://mail.ivoa.net/pipermail/heig/attachments/20260209/182812fd/attachment-0003.png>
More information about the heig
mailing list