<html><head><meta http-equiv="content-type" content="text/html; charset=utf-8"></head><body style="overflow-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;">Hi Mireille,<br class="Apple-interchange-newline"><div><br><blockquote type="cite"><div>On May 23, 2025, at 08:46, Mireille Louys via dm <dm@ivoa.net> wrote:</div><br class="Apple-interchange-newline"><div>
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
<div text="#000000" bgcolor="#FFFFFF"><p>Hi HEIG ,</p><p>Since a while we have been discussing about the way Obscore can
be used for the distribution and discovery of high energy data
sets <br>
</p><p>in the context of multi-wavelength data discovery in the VO.</p><p>Obscore describes how data products are sampled along the main
physical axes involved in astronomical data : </p><p>spatial, spectral, temporal, polarimetric.</p><p>Coverage, resolution and sampling properties are mandatory in the
Obscore table to represent a dataset. <br>
this means we need to have s_ra, s_dec, em_min, em_max , t_min,
t_max, s_region, s_resolution, etc. filled with representative
values .<br>
</p><p>I think this rule does not fully apply for describing instrument
response function as data sets .<br></p></div></div></blockquote><div>There is already a similar case described in the ObsCore document. For dataproduct_type = “measurements” the document states "A few mandatory keywords for the axes description may be non-applicable for such a data product. In this case the coverage on spatial, energy, time, and polarization may inherit the values from the ObsCore description of its progenitor.” </div><div><br></div><div>Practically, however, response-functions (as well as advanced data products) will often have their own coverage, resolution, and sampling properties in terms of spatial, spectral, temporal, and polarimetric physical axes on the sky that they apply to, even if the products themselves do not include all (or in some cases any) of those axes. I sent an example ObsCore entry for a PSF response-function a couple of days ago that demonstrates this.</div><br><blockquote type="cite"><div><div text="#000000" bgcolor="#FFFFFF"><p>
The response functions belong to the instrumental/calibration
information , and probably are stored in Calibration DB . <br>
I think this is not homogeneous in content with the observation
datasets representing flux variation on the sky, <br>
so they should not be considered as Obscore data product type. <br></p></div></div></blockquote><div><div>I don't agree with this suggestion. If only datasets “representing flux variation on the sky” should be included in ObsCore, then that also means that many types of “advanced data products” would also not qualify for inclusion in ObsCore (even if they refer to a constrained spatial/spectral/temporal location such as the region surrounding a detected source).</div><div><br></div><div>I think the IVOA works because it tries to be encompassing. But from the perspective of a science end-user, why would I want to use IVOA interfaces for data discovery if I know I have to use another interface to ensure I can find all of the data products in the high-energy waveband? Conversely, from the perspective of a HEA data provider, why would I want to put the effort into supporting an IVOA interface if it can’t serve all of my data products (knowing that I’m going to have to provide a second interface that can)?</div><div><br></div><div>In principle, one could propose an alternative IVOA-supported data discovery interface that doesn’t have the same restrictions that you are suggesting for ObsCore. But practically, we know how long it takes for anything to become a reality in the IVOA, and so this approach doesn’t seem terribly realistic. But why would any end-user or data provider want to use two interfaces to circumvent what seems to be an artificial constraint?</div></div><br><blockquote type="cite"><div><div text="#000000" bgcolor="#FFFFFF"><p>
</p><p>Currently from our HEIG discussions , we have identified two ways
to distribute data sets in Obscore : <br>
</p>
<ul>
<li>Include the data together with their instrument response
details into a tarball.<br>
this can be represented by a data product type like
eventlist-bundle, spectrum-bundle, etc . depending of the axes
covered in the data</li>
<li>Distribute the data set as a datalink entry which contains a
list of links to the dataset , e.g an event-list and to the
various instrumental response files, arf rmf, psf, energy
dispersion , etc.<br>
accessible through the datalink table .<br>
for instance , reusing examples given by Laurent Michel <br>
</li>
<ul>
<li>spectrum bundle <br>
<br>
a) If this spectrum bundle is exposed as an ObsCore entry in a
data link, we should have something like this: <br>
<br>
Obscore row: <br>
----------- <br>
- dataproduct_type=spectrum-bundle (tbd) <br>
- access_format=application/x-votable+xml;content=datalink <br>
<br>
Datalink response: <br>
----------------- <br>
- link #1 <br>
- semantics=#this <br>
- content_qualifier=spectrum-bundle (TBD) <br>
- content_type=application/tar+gzip <br>
- description="spectrum file + preview + ARF + RMF +
Background spectrum" <br>
<br>
<br>
b) If the spectrum is exposed in a separate row: <br>
<br>
Obscore row: <br>
----------- <br>
- product_type=spectrum <br>
- access_format=application/x-votable+xml;content=datalink <br>
<br>
Datalink response: <br>
----------------- <br>
- link #1 <br>
- semantics=#this <br>
- content_qualifier=spectrum <br>
- content_type=application/fits <br>
- description="spectrum file" <br>
- link #2 <br>
- semantics=#package <br>
- content_qualifier=spectrum-bundle (TBD) <br>
- content_type=application/tar+gzip <br>
- description="spectrum file + preview + ARF + RMF +
Background spectrum" <br>
<br>
</li>
<li>if we define the terms hierarchy as a response vocabulary
like ivoa.net/rdf/instrument-response<br>
</li>
<ul>
<li><i>Instrument_response</i></li>
<ul>
<li><i>arf</i></li>
<li><i>rmf</i></li>
<li><i>energy-disp</i></li>
<li><i>background</i></li>
<li><i>etc.</i><br>
<br>
</li>
</ul>
</ul>
<li>from Bruno's example we can have : <br>
</li>
<ul>
<li><p>Obscore row:<br>
-----------<br>
- product_type=spectrum-bundle<br>
- access_format=application/x-votable+xml;content=datalink<br>
<br>
Datalink response:<br>
-----------------<br>
- link #1<br>
- semantics=#this<br>
- content_qualifier=event-list<br>
- content_type=application/fits<br>
- description="event file"<br>
- link #2<br>
- semantics=#arf<br>
- content_qualifier=#arf <br>
- content_type=application/x-fits-ogip (to be added in
table 1 of ObsCore)<br>
- description="ARF "<br>
- link #3<br>
- semantics=#rmf<br>
- content_qualifier=#rmf <br>
- content_type=application/x-fits-ogip (to be added in
table 1 of ObsCore)<br>
- description="RMF"<br>
- link #4<br>
- semantics=#background<br>
- content_qualifier=image ( if it is a map)<br>
- content_type=application/x-fits-ogip (to be added in
table 1 of ObsCore)<br>
- description="Background" </p><p>doing so , we can discover a data set based on the
Obscore + heig extension columns , <br>
and in a second step access the response files stored in
the calibration part of an archive through the links . <br>
</p>
</li>
</ul>
</ul>
</ul></div></div></blockquote>While these approaches are certainly useful in cases where the end user is interested in retrieving these products together with the event-list (and in many cases the user may want to do just that), that restriction is also a limitation. For example, our experience with the Chandra Source Catalog (that includes ~90 million data products) is that scientists who are doing advanced analysis of large samples (i.e., they are not interested in just one or two observations) typically want to discover and retrieve the data products independently from the associated event-lists.</div><div><br></div><div><blockquote type="cite"><div><div text="#000000" bgcolor="#FFFFFF"><p>Comments, examples, ideas? <br>
</p><p>Mireille<br>
</p>
<pre class="moz-signature" cols="72">--
--
Mireille Louys, MCF (Assistant Professor)
Centre de données Astronomiques (CDS) Equipe Images, ICube
Observatoire de Strasbourg Telecom Physique Strasbourg
11, rue de l' Université 300, Bd Sebastien Brandt CS 10413
F-67000 Strasbourg F-67412 Illkirch Cedex</pre>
</div>
</div></blockquote></div>Thanks,<div>—Ian</div><div><br><div>
<meta charset="UTF-8"><div style="font-family: Monaco; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; text-decoration: none; caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); background-color: rgb(255, 255, 255); line-height: 1.2; margin-top: 0pt; margin-bottom: 0pt;"><span style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-family: Arial; font-size: 9.5pt; font-style: normal; font-variant-caps: normal; font-weight: 700; letter-spacing: normal; text-transform: none; white-space: pre-wrap; word-spacing: 0px; text-decoration: none; -webkit-text-stroke-width: 0px; font-variant-ligatures: normal; font-variant-east-asian: normal; font-variant-position: normal; vertical-align: baseline;">—</span></div><div style="font-family: Monaco; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; text-decoration: none; caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); background-color: rgb(255, 255, 255); line-height: 1.2; margin-top: 0pt; margin-bottom: 0pt;"><span style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-family: Arial; font-size: 9.5pt; font-style: normal; font-variant-caps: normal; font-weight: 700; letter-spacing: normal; text-transform: none; white-space: pre-wrap; word-spacing: 0px; text-decoration: none; -webkit-text-stroke-width: 0px; font-variant-ligatures: normal; font-variant-east-asian: normal; font-variant-position: normal; vertical-align: baseline;"><br class="Apple-interchange-newline">Dr. Ian Evans</span></div><div style="font-family: Monaco; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; text-decoration: none; caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); background-color: rgb(255, 255, 255); line-height: 1.2; margin-top: 0pt; margin-bottom: 0pt;"><font face="Arial"><span style="font-size: 12.666666984558105px; white-space: pre-wrap;"><b>Astrophysicist</b></span></font></div><div style="font-family: Monaco; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; text-decoration: none; caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); background-color: rgb(255, 255, 255); line-height: 1.2; margin-top: 0pt; margin-bottom: 0pt;"><font face="Arial"><span style="font-size: 12.666666984558105px; white-space: pre-wrap;"><b>Chandra X-ray Center</b></span></font></div><div style="font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; text-decoration: none; caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); background-color: rgb(255, 255, 255); font-family: -webkit-standard; line-height: 1.2; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 9.5pt; font-family: Arial; background-color: rgb(255, 255, 255); font-weight: 700; font-variant-ligatures: normal; font-variant-east-asian: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">Center for Astrophysics | Harvard & Smithsonian</span></div><div style="font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; text-decoration: none; caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); background-color: rgb(255, 255, 255); font-family: -webkit-standard; line-height: 1.2; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 9.5pt; font-family: Arial; font-variant-ligatures: normal; font-variant-east-asian: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br></span></div><div style="font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; text-decoration: none; caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); background-color: rgb(255, 255, 255); font-family: -webkit-standard; line-height: 1.2; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 9.5pt; font-family: Arial; font-variant-ligatures: normal; font-variant-east-asian: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">Office: (617) 496 7846 | Cell: (617) 699 5152</span></div><div style="font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; text-decoration: none; caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); background-color: rgb(255, 255, 255); font-family: -webkit-standard; line-height: 1.2; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-family: Arial; font-size: 9.5pt; white-space: pre-wrap;">60 Garden Street | MS 81 | Cambridge, MA 02138</span></div><br class="Apple-interchange-newline" style="font-family: Monaco; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; text-decoration: none; caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);"><span style="font-family: Monaco; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; text-decoration: none; caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);"><span><span><span><span><span><span><span><span><span><span><span><span><span style="color: rgb(0, 0, 0); font-family: Monaco; font-size: 12px; font-weight: normal; font-style: normal;"><span><span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span><span><img alt="PastedGraphic-2.png" src="cid:0499780D-53E4-4E39-BF65-37F7D6FC29DD@cfa.harvard.edu" width="263"></span><meta charset="UTF-8"><span style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-family: Monaco; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; text-decoration: none;"><br class="Apple-interchange-newline"><span style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-family: Monaco; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;"><br></span></span><div style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-family: Monaco; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; text-decoration: none;"><span style="caret-color: rgb(0, 0, 0); font-variant-caps: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><span style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-family: Monaco; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;"> </span><span style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-family: Monaco; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><span><span><span><span><span><span><span><span><span><span><span><span style="color: rgb(0, 0, 0); font-family: Monaco; font-size: 12px; font-weight: normal; font-style: normal;"><span><span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></div><span><img alt="PastedGraphic-3.png" src="cid:14DB9E27-52AB-4470-821D-EF6CA4A0A226@cfa.harvard.edu" width="111"></span><meta charset="UTF-8"><u style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-family: Monaco; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px;"><span lang="EN" style="font-size: 9.5pt; line-height: 14.566667556762695px; font-family: Arial, sans-serif;"><a href="http://cfa.harvard.edu/"><br class="Apple-interchange-newline"><br class="Apple-interchange-newline"></a></span></u><font face="Arial" size="2" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; text-decoration: none;"><u style="caret-color: rgb(0, 0, 0); font-variant-caps: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;"><span lang="EN" style="line-height: 14.566667556762695px;"><a href="http://cfa.harvard.edu/">cfa.harvard.edu</a></span></u><span lang="EN" style="caret-color: rgb(0, 0, 0); font-variant-caps: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; line-height: 14.566667556762695px;"> | <u><a href="http://cfa.harvard.edu/facebook">Facebook</a></u> | <u><a href="http://cfa.harvard.edu/twitter">Twitter</a></u> | <u><a href="http://cfa.harvard.edu/youtube">YouTube</a></u></span><span lang="EN" style="caret-color: rgb(0, 0, 0); font-variant-caps: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; line-height: 14.566667556762695px;"> | <u><a href="http://cfa.harvard.edu/newsletter">Newsletter</a></u></span></font>
</div>
<br></div></body></html>