[Heig] vocabulary update: proposal for dataproduct_type update for high energy data : event-list definition and event-bundle

[Dr. Ian N. Evans]

Hi Markus,

> On May 28, 2025, at 09:55, Markus Demleitner via heig <heig at ivoa.net> wrote:
> 
> Dear Colleagues,
> 
> On Wed, May 21, 2025 at 02:16:52PM -0400, Dr. Ian N. Evans via semantics wrote:
>> In the current draft of the HEA ObsCore note, I access a PSF using
>> 
>> dataproduct_type = ‘response-function’
>> dataproduct_subtype = ‘psf’
>> 
>> because a PSF is a type of response-function (and there are many
>> types of response function so adding all of these separately as
>> different dataproduct_type would grow the list very significantly.
> 
> Let me first retract my earlier appeal to have these in
> datalink/core; that would have been the right choice if these were
> really only auxiliary artefacts.
> 
> Since there seems to be a wide consensus that these beasts ought be
> be able to live within dataproduct-type.  That would mean that, when
> they *are* in datalink, they'd be semantices #calibration and
> content-qualifier something narrower or equal to response-function.
> 
> Are we in rough agreement here?
> 
> If so:
> 
>> “response-function” is technically correct.  Response-functions are
>> widely applicable across multiple wavebands.  For example, a point
>> spread function is a type of response-function that is used across
>> all wavebands.  Similarly, a line spread function is a
>> response-function used in UV through IR spectroscopy.  The term
>> “irf” is not generally used across all high-energy projects.  In
>> the USA, most high-energy projects follow the NASA HEASARC OGIP
>> standards, and so will use “rmf” for the redistribution matrix
>> file” and “arf” for the “auxiliary response file” (and will keep
>> these separate).  Internationally, some projects historically used
>> “irf” to represent the product of the rmf and arf.  More recently
>> some projects have used “irf” as the equivalent of
>> “response-function” giving it a broader interpretation.  So this
>> can be a source of confusion and lack of clarity.  I also note that
>> “irf” stands for “instrument response function” and there are
>> certainly response-functions such as software filters (e.g., a
>> modified Hanning filter used for optimal extraction) where
>> “instrument” would be a misnomer.
> 
> Thank you for that clarification.  Is something like this already in
> the HEIG note?  If not, can I ask you to prepare a PR putting this
> in?

It isn’t at that level of detail, but could easily be expanded up.


> 
>> I might suggest something more like
>> response-function
>> arf as child of response-function
>> rmf as child of response-function
>> psf as child of response-function
>> lsf as child of response-function (not HEA)
> 
> Which begs the question: Do we have cases where we actually need to
> tell those apart *in discovery*?  What sort of obscore query would
> need to constrain discovery to rmf and would be severely less useful
> if you got psfs and arfs in, too?

Well, as an example, let’s query Chandra Source Catalog data products located within 30 arcsec radius of Sgr A*.  A very hypothetical query could be along the lines of 

SELECT * FROM ivoa.obscore
NATURAL JOIN ivoa.obscore-hea
WHERE
(CONTAINS(POINT(s_ra, s_dec), CIRCLE, 266.41681662, -29.00782497, 0.0083333333) = 1)
AND (dataproduct_type EQ ‘response-function’)
AND (dataproduct_subtype EQ ‘rmf’)
AND (obs_collection = ‘CSC2’)

This query would return 6,010 data products totaling 9,363,824,640 bytes.

The same query without the (dataproduct_subtype EQ ‘rmf’) clause (i.e., returning rmfs, arfs, and psfs) would return 104,740 data products totaling 22,396,100,887 bytes.

So I think you would want to tell them apart in discovery.


> 
> Full disclosure up front: I'd be very skeptical about an arument that
> you need it to tell apart the individual artifacts; I'm 100% sure
> that type information needs to be in the dataset itself, and that we
> need to fix the datasets if they are lacking this kind of metadata.

Yes, the datasets themselves include those metadata.


> 
> Thanks,
> 
>         Markus
> 
> -- 
> heig mailing list
> heig at ivoa.net
> https://www.google.com/url?q=http://mail.ivoa.net/mailman/listinfo/heig&source=gmail-imap&ust=1749045378000000&usg=AOvVaw2hXHOoEwqtvMAqDzD7WXGi



Thanks,
—Ian

—

Dr. Ian Evans
Astrophysicist
Chandra X-ray Center
Center for Astrophysics | Harvard & Smithsonian

Office: (617) 496 7846 | Cell: (617) 699 5152
60 Garden Street | MS 81 | Cambridge, MA 02138



 


 <http://cfa.harvard.edu/>cfa.harvard.edu <http://cfa.harvard.edu/> | Facebook <http://cfa.harvard.edu/facebook> | Twitter <http://cfa.harvard.edu/twitter> | YouTube <http://cfa.harvard.edu/youtube> | Newsletter <http://cfa.harvard.edu/newsletter>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.ivoa.net/pipermail/semantics/attachments/20250528/f66f98e8/attachment-0001.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: PastedGraphic-2.png
Type: image/png
Size: 581 bytes
Desc: not available
URL: <http://mail.ivoa.net/pipermail/semantics/attachments/20250528/f66f98e8/attachment-0002.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: PastedGraphic-3.png
Type: image/png
Size: 21717 bytes
Desc: not available
URL: <http://mail.ivoa.net/pipermail/semantics/attachments/20250528/f66f98e8/attachment-0003.png>