<!DOCTYPE html>
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<p>Hi Ian, hi folks,<br>
</p>
<p>Thanks for the various querying scenarios. <br>
</p>
<p>I understand the PSF use case you describe would be something
like: <br>
</p>
<p>give me all dataproduct_type='psf' with additional constraints
like<br>
</p>
<div>SELECT TOP 100 * FROM ivoa.ObsCore</div>
<div>WHERE obs_id='22830' and dataproduct_type='psf'</div>
<div><br>
</div>
<div>How is the relationship between a PSF data product with a data
file? <br>
</div>
<div>through obs_id, obs_publisher_id, ? </div>
<div> <br>
</div>
<div>Can we explore how the obscore1.1 table would be filled for
a PSF data product? <br>
</div>
<div>are there file examples? <br>
</div>
<div><br>
</div>
<div>I understand we identify here a vocabulary to identify various
irf file like : "psf , irf, arf, rmf, background noise, etc..."</div>
<div>with a kind of hierarchy like <br>
</div>
<div><i>irf</i></div>
<div><i>arf as child of irf <br>
</i></div>
<div><i>rmf as child of irf <br>
</i></div>
<div><i>psf as child of irf </i></div>
<div><i>back_ground_noise </i><br>
</div>
<div><br>
</div>
<div>with the assumption that em_ucd, sxel1, sxel2, em_xel etc. can
be filled in the Obscore table.</div>
<div>Parametric functions could not be described here.<br>
</div>
<p>so if these are obscore data product_types we should be able to
describe these features. <br>
</p>
<p>the "content_qualifier" used in data link, if I remember well,
can be something else than an obscore data product type also. <br>
</p>
<p>(to be checked)</p>
<p>I will look deeper in this and come back on the topic soon,</p>
<p>Cheers, Mireille<br>
</p>
<p><br>
</p>
<div class="moz-cite-prefix">Le 30/04/2025 à 19:52, Dr. Ian N. Evans
via semantics a écrit :<br>
</div>
<blockquote type="cite"
cite="mid:5BA6DAF3-9C7D-4323-AE77-5E7747BFE758@cfa.harvard.edu">
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
<div dir="auto"
style="overflow-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;">Hi
Markus,
<div><br>
</div>
<div>See inline comments below.<br
id="lineBreakAtBeginningOfMessage">
<div><br>
<blockquote type="cite">
<div>On Apr 25, 2025, at 03:29, Markus Demleitner via heig
<a class="moz-txt-link-rfc2396E" href="mailto:heig@ivoa.net"><heig@ivoa.net></a> wrote:</div>
<br class="Apple-interchange-newline">
<div>
<div>Dear Mireille,<br>
<br>
Thanks for your VEP.<br>
<br>
On Thu, Apr 24, 2025 at 06:16:36PM +0200, Mireille
Louys via semantics wrote:<br>
<blockquote type="cite">•
Proposedefinitionsforaproduct-type *event-bundle:*
An event-bundledataset<br>
is a complex object containing an event-list and
multiple files or<br>
other substructures that are products necessary to
analyse the event-list.<br>
Data in an event-bundle may thus be used to produce
higher leveldata<br>
products such as images or spectra.<br>
</blockquote>
<br>
I think the definition is reasonably clear and
applicable in<br>
practice. Before merging this, however, I'd have a
few requests for<br>
clarification:<br>
<br>
(1) used-in: I really, *really* would like to see
actual, published<br>
data here (always, in all VEPs; it's a pain if we go
into all the<br>
trouble of defining a concept and then nobody's ever
using it in<br>
practice). I see that CSC on
<a class="moz-txt-link-freetext" href="https://www.google.com/url?q=http://cda.cfa.harvard.edu/csctap&source=gmail-imap&ust=1746354088000000&usg=AOvVaw0j-MeVn0RHNoz8KtLXi1tH">https://www.google.com/url?q=http://cda.cfa.harvard.edu/csctap&source=gmail-imap&ust=1746354088000000&usg=AOvVaw0j-MeVn0RHNoz8KtLXi1tH</a>
(or<br>
<a class="moz-txt-link-freetext" href="https://www.google.com/url?q=http://cda.cfa.harvard.edu/csc21tap&source=gmail-imap&ust=1746354088000000&usg=AOvVaw3xeGe6fO6Vz7vemy8dYWD2">https://www.google.com/url?q=http://cda.cfa.harvard.edu/csc21tap&source=gmail-imap&ust=1746354088000000&usg=AOvVaw3xeGe6fO6Vz7vemy8dYWD2</a>
[1]) has an obscore table. It<br>
would really be excellent if they could mark up their
event bundles<br>
with the new term, such that we could say:<br>
<br>
used-in: dataset
ivo://csc.harvard.edu/scsr2?some-obs-id on
<a class="moz-txt-link-freetext" href="https://www.google.com/url?q=http://cda.cfa.harvard.edu/csctap&source=gmail-imap&ust=1746354088000000&usg=AOvVaw0j-MeVn0RHNoz8KtLXi1tH">https://www.google.com/url?q=http://cda.cfa.harvard.edu/csctap&source=gmail-imap&ust=1746354088000000&usg=AOvVaw0j-MeVn0RHNoz8KtLXi1tH</a><br>
<br>
That would help me maintain a clear consciousness when
setting up the<br>
new term[2].<br>
</div>
</div>
</blockquote>
<div><br>
</div>
<div>I’m not sure that we would use event bundles for any of
the CSC data product types, since our catalog users
generally want to discover and retrieve individual data
products (typically, many of the same data product such as
a light curve or aperture photometry MPDF for a set of
sources, such as all high redshift quasars).</div>
<div><br>
</div>
<div>We do make event bundles available through the Chandra
Data Archive (CDA) ObsCore for individual observations,
but we have been doing this for many years and therefore
those bundles do not conform to the proposed ObsCore HE
extension. For example, at <a
href="https://cda.harvard.edu/cxctap"
moz-do-not-send="true" class="moz-txt-link-freetext">https://cda.harvard.edu/cxctap</a> one
might do a query like</div>
<div>SELECT TOP 100</div>
<div> *</div>
<div>FROM ivoa.ObsCore</div>
<div>WHERE obs_id='22830'</div>
<div><br>
</div>
<div>The bundles currently have dataproduct_type = ‘’ and
correspond to the Chandra “primary” and “primary” +
“secondary” data product categories for the observation.
The “primary” package includes a basic set of data
products that an observer would need to analyze the
observation and produce photometrically calibrated
spectra, while the “secondary” set would be needed in
addition if the user wanted to recalibrate the observation
with updated calibrations (which we recommend because some
updated calibrations only become available typically
several months after an observation is completed as
calibrations such as detector gain change with time). The
bundles are tarballs and have access_format
“application/x-tar”.</div>
<div><br>
</div>
<div>Note that our bundles do not include responses, unlike
(e.g.) CTAO. This is partly because the spacecraft
dithers on the sky during an observation so any target
moves across the detector during the observation. So the
responses depend critically on the user’s choice of data
filtering. We follow the HEASARC standards and provide
separate RMF and ARF data products rather than a combined
IRF. For spectral fitting, the integrated ARF (i.e., the
effective area integrated over an energy range) also
depends on the source spectral model, so we provide tools
for the user to compute the responses and the data
products needed to do so in the “primary” bundle. See <a
href="https://cxc.harvard.edu/cda/DataProdList.html#A"
moz-do-not-send="true" class="moz-txt-link-freetext">https://cxc.harvard.edu/cda/DataProdList.html#A</a> for
the list of Chandra primary and secondary products.</div>
<div><br>
</div>
<br>
<blockquote type="cite">
<div>
<div><br>
(2) Relationship: That's an operational field, i.e., I
need to create<br>
an RDF triple from this. The question thus is: is
#event-list wider<br>
than #event-bundle or is it the other way round? I
could conjure up<br>
arguments for both, so, as usual, I'd approach the
question from the<br>
user side: If I'm looking for #event-bundle, do I want
to see<br>
#event-list, too? If I'm looking for #event-list, do
I want to see<br>
#event-bundle, too? Whatever ought to encompass the
other is the<br>
wider term.<br>
</div>
</div>
</blockquote>
<div><br>
</div>
<div>I would guess probably not, but maybe. I think an
event bundle will always include the event list, so if you
are asking for an event bundle then the event list would
be redundant. Note that an event bundle could really be a
physical bundle such as a tar file (as is the case for
Chandra) or perhaps all the products are accessed via
DataLink. If somebody asks for an event list, they
probably just want the event list but perhaps they don’t
know about event bundles and therefore might like to see
both. There are definitely cases where they really do
want just the event list - for example, a lot of folks are
interested only in morphological studies and, especially
if they want all observations of an object or class of
object, bundles may add a lot of extra unwanted data
volume. Would it hurt to see them? Well if your query
returns 500 event lists and also 500 bundles it just
muddies the waters. For users of advanced data products
(e.g., Chandra Source Catalog products) I suspect that
bundles would likely never be desired (and as I said
above, I don’t think we would provide them).</div>
<div><br>
</div>
<div>Conversely, some folks may only want responses and not
the associated event lists. At the catalog level for
example, we have lots of folks who just want to download
point spread functions. Why? The Chandra PSF varies
strongly (~factor of 50) across the field of view (with
off-axis angle and azimuthal angle) and also with energy.
We actually archive the local PSF for every catalog
detection at several energies, and with ~50,000 counts.
So we effectively have a library of (currently) ~1.3
million Chandra PSFs at several energies that are catalog
advanced data products and may save users from having to
create their own PSFs.</div>
<div><br>
</div>
<div>A lot of our actual catalog usage cases start with
users querying the catalog, then refining their sample and
retrieving a subset of catalog data products (likely
including event lists), then retrieving additional data
products such as light curves, spectra, or aperture
photometry products as they work through their analyses.
These products are not part of bundles, but we find this
usage pattern of wanting to retrieve additional products
in stages to be quite common for folks who are doing
archival science (vs. those who are retrieving their own
observation data).</div>
<div><br>
</div>
<br>
<blockquote type="cite">
<div>
<div><br>
(3) Rationale: If the answer to both of the two
questions in the<br>
preceding paragraph is "Yes", then it turns out the
concepts are<br>
identical (A ⊂ B and B ⊂ A implies A = B), and hence
you really don't<br>
want a new concept but augement #event-list to be
something like,<br>
say, "Event list, possibly augmented with ancillary
information".<br>
This points to an issue with your rationale: It
basically argues that<br>
there's something you would like to say.<br>
<br>
An aphorism I'm bringing up rather often these days
is: "In protocol<br>
design, don't think about what you want to say. Think
about what<br>
others want to listen to." Hence, it's be really
great if the<br>
rationale said why someone would want to look for
#event-bundle<br>
*rather than* #event-list (or for #event-list rather
than<br>
#event-bundle, if the the former is the narrower
term). Could you<br>
provide that information in the Rationale section?<br>
<br>
Thanks,<br>
<br>
Markus<br>
<br>
<br>
[1] Regrettably, the CSC TAP services seem to be
mildly broken at the<br>
moment. Coming in with http, they issue https
redirects which<br>
confuse TOPCAT; CXC folks: if you really need the
forced redirects<br>
(see<br>
<a class="moz-txt-link-rfc2396E" href="https://www.google.com/url?q=https://blog.tfiu.de/foced-https-redirects-considered-harmful.html&source=gmail-imap&ust=1746354088000000&usg=AOvVaw2y6JI9YuzGb_dp9cgELJym"><https://www.google.com/url?q=https://blog.tfiu.de/foced-https-redirects-considered-harmful.html&source=gmail-imap&ust=1746354088000000&usg=AOvVaw2y6JI9YuzGb_dp9cgELJym></a>
for<br>
a better alternative) then please update your registry
records to<br>
point to the https URIs. Even with https, however,
I'm getting a<br>
"cscrel2.dbo.obscore not found" error from TOPCAT when
running<br>
<br>
select top 30 * from ivoa.obscore where
dataproduct_type='event-list'<br>
<br>
It would be great if you could fix that (and a regular
run of stilts<br>
taplint is good practice anyway)<br>
</div>
</div>
</blockquote>
<div><br>
</div>
The https to http redirects is a known issue and should be
fixed in the next full data system release, which is
scheduled for mid-June.</div>
<div><br>
</div>
<div>We’ll look into the other issue you reported.</div>
<div><br>
</div>
<div><br>
<blockquote type="cite">
<div>
<div><br>
[2] You see,<br>
<a class="moz-txt-link-rfc2396E" href="https://www.google.com/url?q=https://ivoa.net/documents/Vocabularies/20230206/REC-Vocabularies-2.1.html%23tth_sEcC.2&source=gmail-imap&ust=1746354088000000&usg=AOvVaw0vPm8jxoQ6ePuvFWnKr6ez"><https://www.google.com/url?q=https://ivoa.net/documents/Vocabularies/20230206/REC-Vocabularies-2.1.html%23tth_sEcC.2&source=gmail-imap&ust=1746354088000000&usg=AOvVaw0vPm8jxoQ6ePuvFWnKr6ez></a>,<br>
while not exactly normative, is clear on:<br>
<br>
In particular, ensure [...] resources mentioned in
Used-in can be<br>
reached and reflect the proposed term [...]<br>
<br>
-- <br>
heig mailing list<br>
<a class="moz-txt-link-abbreviated" href="mailto:heig@ivoa.net">heig@ivoa.net</a><br>
<a class="moz-txt-link-freetext" href="https://www.google.com/url?q=http://mail.ivoa.net/mailman/listinfo/heig&source=gmail-imap&ust=1746354088000000&usg=AOvVaw0wTMYFfnqD-UBwfT4NR3kr">https://www.google.com/url?q=http://mail.ivoa.net/mailman/listinfo/heig&source=gmail-imap&ust=1746354088000000&usg=AOvVaw0wTMYFfnqD-UBwfT4NR3kr</a><br>
</div>
</div>
</blockquote>
</div>
<div><br>
</div>
<div><br>
</div>
Thanks,
<div>—Ian</div>
<div><br>
<div>
<meta charset="UTF-8">
<div
style="font-family: Monaco; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; text-decoration: none; caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); background-color: rgb(255, 255, 255); line-height: 1.2; margin-top: 0pt; margin-bottom: 0pt;"><span
style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-family: Arial; font-size: 9.5pt; font-style: normal; font-variant-caps: normal; font-weight: 700; letter-spacing: normal; text-transform: none; white-space: pre-wrap; word-spacing: 0px; text-decoration: none; -webkit-text-stroke-width: 0px; font-variant-ligatures: normal; font-variant-east-asian: normal; font-variant-position: normal; vertical-align: baseline;">—</span></div>
<div
style="font-family: Monaco; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; text-decoration: none; caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); background-color: rgb(255, 255, 255); line-height: 1.2; margin-top: 0pt; margin-bottom: 0pt;"><span
style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-family: Arial; font-size: 9.5pt; font-style: normal; font-variant-caps: normal; font-weight: 700; letter-spacing: normal; text-transform: none; white-space: pre-wrap; word-spacing: 0px; text-decoration: none; -webkit-text-stroke-width: 0px; font-variant-ligatures: normal; font-variant-east-asian: normal; font-variant-position: normal; vertical-align: baseline;">
Dr. Ian Evans</span></div>
<div
style="font-family: Monaco; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; text-decoration: none; caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); background-color: rgb(255, 255, 255); line-height: 1.2; margin-top: 0pt; margin-bottom: 0pt;"><font
face="Arial"><span
style="font-size: 12.666666984558105px; white-space: pre-wrap;"><b>Astrophysicist</b></span></font></div>
<div
style="font-family: Monaco; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; text-decoration: none; caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); background-color: rgb(255, 255, 255); line-height: 1.2; margin-top: 0pt; margin-bottom: 0pt;"><font
face="Arial"><span
style="font-size: 12.666666984558105px; white-space: pre-wrap;"><b>Chandra X-ray Center</b></span></font></div>
<div
style="font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; text-decoration: none; caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); background-color: rgb(255, 255, 255); font-family: -webkit-standard; line-height: 1.2; margin-top: 0pt; margin-bottom: 0pt;"><span
style="font-size: 9.5pt; font-family: Arial; background-color: rgb(255, 255, 255); font-weight: 700; font-variant-ligatures: normal; font-variant-east-asian: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">Center for Astrophysics | Harvard & Smithsonian</span></div>
<div
style="font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; text-decoration: none; caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); background-color: rgb(255, 255, 255); font-family: -webkit-standard; line-height: 1.2; margin-top: 0pt; margin-bottom: 0pt;"><span
style="font-size: 9.5pt; font-family: Arial; font-variant-ligatures: normal; font-variant-east-asian: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">
</span></div>
<div
style="font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; text-decoration: none; caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); background-color: rgb(255, 255, 255); font-family: -webkit-standard; line-height: 1.2; margin-top: 0pt; margin-bottom: 0pt;"><span
style="font-size: 9.5pt; font-family: Arial; font-variant-ligatures: normal; font-variant-east-asian: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">Office: (617) 496 7846 | Cell: (617) 699 5152</span></div>
<div
style="font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; text-decoration: none; caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); background-color: rgb(255, 255, 255); font-family: -webkit-standard; line-height: 1.2; margin-top: 0pt; margin-bottom: 0pt;"><span
style="font-family: Arial; font-size: 9.5pt; white-space: pre-wrap;">60 Garden Street | MS 81 | Cambridge, MA 02138</span></div>
<br class="Apple-interchange-newline"
style="font-family: Monaco; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; text-decoration: none; caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);">
<meta charset="UTF-8">
<meta charset="UTF-8">
<u
style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-family: Monaco; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px;"><span
lang="EN"
style="font-size: 9.5pt; line-height: 14.566667556762695px; font-family: Arial, sans-serif;"><a
href="http://cfa.harvard.edu/"
moz-do-not-send="true"><br
class="Apple-interchange-newline">
<br class="Apple-interchange-newline">
</a></span></u><font face="Arial" size="2"
style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; text-decoration: none;"><u
style="caret-color: rgb(0, 0, 0); font-variant-caps: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;"><span
lang="EN" style="line-height: 14.566667556762695px;"><a
href="http://cfa.harvard.edu/"
moz-do-not-send="true">cfa.harvard.edu</a></span></u><span
lang="EN"
style="caret-color: rgb(0, 0, 0); font-variant-caps: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; line-height: 14.566667556762695px;"> | <u><a
href="http://cfa.harvard.edu/facebook"
moz-do-not-send="true">Facebook</a></u> | <u><a
href="http://cfa.harvard.edu/twitter"
moz-do-not-send="true">Twitter</a></u> | <u><a
href="http://cfa.harvard.edu/youtube"
moz-do-not-send="true">YouTube</a></u></span><span
lang="EN"
style="caret-color: rgb(0, 0, 0); font-variant-caps: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; line-height: 14.566667556762695px;"> | <u><a
href="http://cfa.harvard.edu/newsletter"
moz-do-not-send="true">Newsletter</a></u></span></font>
</div>
<br>
</div>
</div>
</div>
</blockquote>
<pre class="moz-signature" cols="72">--
--
Mireille Louys, MCF (Assistant Professor)
Centre de données Astronomiques (CDS) Equipe Images, ICube
Observatoire de Strasbourg Telecom Physique Strasbourg
11, rue de l' Université 300, Bd Sebastien Brandt CS 10413
F-67000 Strasbourg F-67412 Illkirch Cedex</pre>
</body>
</html>