<!DOCTYPE html>
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<div class="moz-cite-prefix">Dear all,</div>
<div class="moz-cite-prefix">The discussion on this topic went on on
github : <a class="moz-txt-link-freetext" href="https://github.com/ivoa/HighEnergyObsCoreExt/issues/5">https://github.com/ivoa/HighEnergyObsCoreExt/issues/5</a></div>
<div class="moz-cite-prefix"><br>
</div>
<div class="moz-cite-prefix">I tried my own summary there which i
copy/paste here below</div>
<div class="moz-cite-prefix">Cheers</div>
<div class="moz-cite-prefix">François<br>
</div>
<div class="moz-cite-prefix"><br>
</div>
<div class="moz-cite-prefix">
<blockquote type="cite">
<div data-testid="markdown-body"
data-team-hovercards-enabled="true" class="markdown-body"
data-turbolinks="false">
<div
class="Box-sc-g0xbh4-0 markdown-body NewMarkdownViewer-module__safe-html-box--cRsz0">
<p dir="auto">I think we all agree that we have to define
new terms for the instrument response functions.</p>
<p dir="auto">From the discussion I see three different
possible vocabulary integration for these terms</p>
<p dir="auto">Beside this it's also important to distinguish
creation/extension of vocabularies from the FIELDs where
we use them.</p>
<p dir="auto">And we have to find a solution which works for
two different exposition strategies of the irf :</p>
<div
class="snippet-clipboard-content notranslate position-relative overflow-auto">
<pre class="notranslate"><code class="notranslate"> - directly as observation in ObsCore
- via DataLink, attached to a primary science dataproduct in ObsCore (event-list, spectrum, etc...)
</code></pre>
<div
class="zeroclipboard-container position-absolute right-0 top-0"> </div>
</div>
<p dir="auto">1 ) One of the suggestion I have read so far
(Bruno) is to refine the #calibration branch of "DataLink
core" (<a
href="https://www.ivoa.net/rdf/datalink/core/2022-01-27/datalink.html"
rel="nofollow" class="moz-txt-link-freetext">https://www.ivoa.net/rdf/datalink/core/2022-01-27/datalink.html</a>)
to add your new terms for irf . Let's take the #psf term
for example : if we set this term in the "semantics" FIELD
of the DataLink response that means that we are facing the
"#psf of the primary item in ObsCore". But I understood
(from Ian) that sometimes you want to discover irf
directly, as observations. And in the current status of
ObsCore/SSA/SIA/DataLink it's not possible to use <a
href="https://www.ivoa.net/rdf/datalink/core/2022-01-27/datalink.html"
rel="nofollow" class="moz-txt-link-freetext">https://www.ivoa.net/rdf/datalink/core/2022-01-27/datalink.html</a>
outside the "semantics DataLink response" FIELD. Changing
that would be cumbersome.</p>
<p dir="auto">In addition we could imagine in DataLink that
a psf (or any other irf ) link could be used for a
material required to calibrate (process) the primary item
(child of #calibration) or on the other side for material
already used to calibrate the dataset discovered in
ObsCore. (currently would be a child of #progenitor
although I don't like it personally). So this would
require some duplication of all the different irf terms !
one psf-for-calibration and one psf-used-tocalibrate !!! ,
etc ...</p>
<p dir="auto">----> Probably we want to avoid that</p>
<p dir="auto">2 ) Data Product type (<a
href="https://www.ivoa.net/rdf/product-type/2024-05-19/product-type.html"
rel="nofollow" class="moz-txt-link-freetext">https://www.ivoa.net/rdf/product-type/2024-05-19/product-type.html</a>)
: This vocabulary originally extracted from ObsCore is
intended to be used in the dataproduct_type FIELD (in
future version of ObsCore) and can also be used elsewhere.
It can be used in the registry and in the
content_qualifier FIELD of DataLink.<br>
Considering that IRF are "special" observations, it's
possible to create a new "irf" branch in this vocabulary
with all irf children.<br>
It could be used either in ObsCore dataproduct_type or in
DataLink content_qualifier depending on the irf
"exposition" strategy.<br>
For the DataLink exposition strategy, this kind of
combination (completed by content_type for the
media-type/format) is what we experimented with the
DataLink core #coderived and #counterpart terms in this
context (example):</p>
<div
class="snippet-clipboard-content notranslate position-relative overflow-auto">
<pre class="notranslate"><code class="notranslate"> primary item : an astronomical source in a catalog
link1 : semantics=#coderived, content_qualifier=#time_series,content_type=text/csv
link2 : semantics=#counterpart, content_qualifier=#image,content_type=application/fits
</code></pre>
<div
class="zeroclipboard-container position-absolute right-0 top-0"> </div>
</div>
<p dir="auto">However there is a question: should all the
types of irf be potentially exposed in ObsCore as
"observations" ? If some of them are excluded it would be
difficult to mark them as irrelevant : we don't have ways
to exclude some terms from a vocabulary in a given
context.<br>
It's one reason why I think there is also a third solution
to manage this "irf type" vocabulary.</p>
<p dir="auto">3 ) create a new "irf type" IVOA vocabulary.<br>
It can be used in "content_qualifier" FIELD of DataLink in
combination with "semantics=#calibration" (or any other
appropriate term from <a
href="https://www.ivoa.net/rdf/datalink/core/2022-01-27/datalink.html"
rel="nofollow" class="moz-txt-link-freetext">https://www.ivoa.net/rdf/datalink/core/2022-01-27/datalink.html</a>)<br>
This is perfectly valid because content_qualifier is not
REQUIRED to be taken from "Data Product type" vocabulary.
It's only the default ! And the combination between
semantics , content_qualifier and content_type works as in
2)<br>
The drawback is that in that case we cannot use these new
irf terms directly in the dataproduct_type FIELD of
ObsCore, if we want to expose them there.<br>
The compromise could be to create a single new term in <a
href="https://www.ivoa.net/rdf/product-type/2024-05-19/product-type.html"
rel="nofollow" class="moz-txt-link-freetext">https://www.ivoa.net/rdf/product-type/2024-05-19/product-type.html</a>
: #irf<br>
and when she finds this, the user/software has to look in
another ObsCore field (maybe dataproduct_subtype ???) the
accurate term (taken from this independent irf vocabulary)
describing the irf</p>
<hr>
<p dir="auto">In my personal opinion , solution 1 should be
rejected. It's not consistent with what we did when we
creates #counterpart, #coderived and the content_qualifier
FIELD.<br>
2 or 3 ) can be further discussed. I personally prefer 3
because IRF may be either images, or spectra, or whatever,
in such a way that what distinguish them from classical
observation results is the object which is observed:
"something in the sky" versus "measuring a response of the
instrument". SO I am reluctant to add them to data product
type vocabulary where so far the distinction between so
far was "what axis is observed, what axis is sampled".</p>
</div>
</div>
</blockquote>
<br>
</div>
<div class="moz-cite-prefix"><br>
</div>
<div class="moz-cite-prefix"><br>
</div>
<div class="moz-cite-prefix">Le 07/05/2025 à 00:20, BONNAREL
FRANCOIS gmail a écrit :<br>
</div>
<blockquote type="cite"
cite="mid:dce6d545-33fa-4edc-9e83-6c45bf42d156@gmail.com">
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<div class="moz-cite-prefix">Dear all,</div>
<div class="moz-cite-prefix"> I think I agree with Markus that
we should avoid multiple #this in general.</div>
<div class="moz-cite-prefix"> But with one single exception :
different formats for the same product</div>
<div class="moz-cite-prefix"> Exemples : DataLink allows to have
one single line in ObsCore for an image which we can deliver in
FITS, JPEG, PNG etc...</div>
<div class="moz-cite-prefix"> DataLink allows
to one single line for a spectrum in ObscOre which we can
deliver in VOtable, FITS, CSV, etc...</div>
<div class="moz-cite-prefix"> In those cases we will have in
DataLInk table record 1 : ID = ivo://aaa/bbb <br>
</div>
<div class="moz-cite-prefix">
semantics : #this</div>
<div class="moz-cite-prefix">
content-type : application/fits</div>
<div class="moz-cite-prefix">
content-qualifier : image</div>
<div class="moz-cite-prefix">
description " XXX survey image Target so and so in FITS format"</div>
<div class="moz-cite-prefix">
record 2 : ID = ivo://aaa/bbb (same as above)<br>
</div>
<div class="moz-cite-prefix">
semantics : #this
<div class="moz-cite-prefix">
content-type : image/jpeg<br>
</div>
<div class="moz-cite-prefix">
content-qualifier : image</div>
<div class="moz-cite-prefix">
description " XXX survey image Target so and so in Jpeg
format"</div>
<div class="moz-cite-prefix"> ....<br>
</div>
<div class="moz-cite-prefix"> The examples given by Laurent
show that a data producer can choose different strategies
according to the mode of usage it expects from users.</div>
<div class="moz-cite-prefix"> strategy 1 : the bundle
is exposed in ObsCOre an then in DataLink #this in semantics
row can only be the whole bundle</div>
<div class="moz-cite-prefix"> strategy 2 : the spectrum
(or event-list, or whatever science data) is exposed in
ObsCore and then #this is this very spectrum, or event-list,
etc... other material you want to link to the spectrum must
have different semantics (#calibration, #auxiliary ...)<br>
</div>
<div class="moz-cite-prefix"><br>
</div>
<div class="moz-cite-prefix"> Answer to your last question
below<br>
</div>
<div class="moz-cite-prefix"> <br>
</div>
</div>
<div class="moz-cite-prefix">
<br>
</div>
<div class="moz-cite-prefix">Le 05/05/2025 à 22:45, Jaffe, Tess
(GSFC-6601) via heig a écrit :<br>
</div>
<blockquote type="cite"
cite="mid:MN2PR09MB5499BD22D2B4A73887614C47E68E2@MN2PR09MB5499.namprd09.prod.outlook.com">
<meta http-equiv="Content-Type"
content="text/html; charset=UTF-8">
<style type="text/css" style="display:none;">P {margin-top:0;margin-bottom:0;}</style>
<div class="elementToProof"
style="font-family: Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div class="elementToProof"
style="font-family: Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
I can add what our use case was. </div>
<div class="elementToProof"
style="font-family: Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div class="elementToProof"
style="font-family: Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
There was a local team building a web tool for x-ray spectra
viewing and quick line identification, so I asked them to use
our SSA for it. We then added a datalink service descriptor to
our SSA results and served the response matrices in the
datalink table result. (The SSA result at the time had the
link to the FITS file.) It worked internally with their tool
for a while but the tool didn't make it to production. So that
was our use case. </div>
<div class="elementToProof"
style="font-family: Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div class="elementToProof"
style="font-family: Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Now we are working more on our ObsTAP and its datalinks. But
precisely the question of which products get their own row in
our ObsCore table is what we're working on now. Presumably,
whatever we decide, this hypothetical tool could work with as
long as the service is performant. But from the client point
of view, being given a tarball is more annoying to code for.
I would prefer listing the spectrum product alone in ObsCore
and give each ancillary file its own row in the DataLink
result table. I.e., list a spectrum in ObsCore and return
additional rows in the DataLink result for each of the files
that would constitute the bundle. What would the bundle as a
tarball facilitate?</div>
</blockquote>
<p>We discussed examples in the context of CTA and XMM. I think
"event-bundle" records in ObsCore may be completed by DataLink
records #this with content-type=application/OGIP or
content-type=application/GADF (or later VODF)</p>
<p>In that case a dedicated software can directly process these
"standard" formats of "bundles".<br>
</p>
<p><br>
</p>
<p>Of course another valid strategy is the one you suggest :
product type in ObsCore : event-list , spectrum and then several
records with #calibration for the different IRF or ARF</p>
<p>An extended vocabulary for IRF/ARF types (TBD) may be used in
the content-qualifier FIELD in each of those rows for better
identification. <br>
</p>
<p>Probably more adapted for ad hoc processing. <br>
</p>
<p>Cheers</p>
<p>François<br>
</p>
<p><br>
</p>
<blockquote type="cite"
cite="mid:MN2PR09MB5499BD22D2B4A73887614C47E68E2@MN2PR09MB5499.namprd09.prod.outlook.com">
<div class="elementToProof"
style="font-family: Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div
style="font-family: Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div
style="font-family: Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<hr style="display: inline-block; width: 98%;">
<div dir="ltr" id="divRplyFwdMsg" style="color: inherit;"><span
style="font-family: Calibri, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);"><b>From:</b> heig
<a class="moz-txt-link-rfc2396E"
href="mailto:heig-bounces@ivoa.net" moz-do-not-send="true"><heig-bounces@ivoa.net></a>
on behalf of Mireille Louys via heig <a
class="moz-txt-link-rfc2396E" href="mailto:heig@ivoa.net"
moz-do-not-send="true"><heig@ivoa.net></a><br>
<b>Sent:</b> Monday, May 5, 2025 1:50 PM<br>
<b>To:</b> <a
class="moz-txt-link-abbreviated moz-txt-link-freetext"
href="mailto:heig@ivoa.net" moz-do-not-send="true">heig@ivoa.net</a>
<a class="moz-txt-link-rfc2396E" href="mailto:heig@ivoa.net"
moz-do-not-send="true"><heig@ivoa.net></a><br>
<b>Subject:</b> Re: [Heig] [EXTERNAL] [BULK] Re: vocabulary
update: proposal for dataproduct_type update for high energy
data : event-list definition and event-bundle</span>
<div> </div>
</div>
<div style="font-size: 11pt;">CAUTION: This email originated
from outside of NASA. Please take care when clicking links or
opening attachments. Use the "Report Message" button to
report suspicious messages to the NASA SOC.<br>
<br>
<br>
<br>
<br>
Hello,<br>
<br>
Thanks Tess and Laurent for these examples .<br>
<br>
This was a proposal during the HE workshop last week to bring
some examples<br>
<br>
showing an Obscore data discovery ( obscore entry here ) and
exploring<br>
various settings of data link scenarios .<br>
<br>
Any volunteer for examples from other archives ?<br>
<br>
Best , Mireille<br>
<br>
<br>
Le 05/05/2025 à 18:14, Laurent Michel via heig a écrit :<br>
> Hello,<br>
><br>
><br>
><br>
> Le 05/05/2025 à 08:56, Markus Demleitner via semantics a
écrit :<br>
>> Hi Tess,<br>
>><br>
>> On Wed, Apr 30, 2025 at 01:48:29PM +0000, Jaffe, Tess
(GSFC-6601) via<br>
>> semantics wrote:<br>
>>> Possibly dumb question:<br>
>><br>
>> Not at all; you're touching a topic that has been
discussed quite a<br>
>> few times now without a satisfying result yet: What
does it mean if<br>
>> there are multiple items with the same semantics in
Datalink? Is it<br>
>> "all of them together give the thing" or is it "they
are<br>
>> alternatives"? Or yet something else?<br>
>><br>
>> Previous rounds suggested that the interpretation
will probably<br>
>> depend on the concept, but the details turned out to
be fairly messy.<br>
>><br>
>><br>
>>> If an ObsCore table lists an event-bundle as a
separate row with<br>
>>> its own product_type, and the access_url follows
best practice<br>
>>> specifying a datalink that will return the
bundle, what should the<br>
>>> DataLink result include as #this? We are
actively putting this<br>
>>> together now at HEASARC. If the product type is
simply a spectrum,<br>
>><br>
>> That's excellent news!<br>
>><br>
>>> our datalink result has the spectrum file as
#this and the response<br>
>>> matrices, background, etc. as related products in
the same result<br>
>>> table. If the product itself is a bundle, what
is the #this? Do<br>
>>> we have to provide a tarball or something? Or
are there multiple<br>
>>> #this with different dataproduct_subtypes? The
latter doesn't<br>
>>> sound right to me.<br>
>><br>
>> Given my preamble, I'd avoid multi-#this. The ideal
solution would<br>
>> IMHO be a standard archive if the HEIG can commit to
such a thing.<br>
>> Failing that, I think a tar archive of the individual
components<br>
>> would be the second best thing. CADC does something
like this,<br>
>> although the other way round: They're handing out
everything tarred<br>
>> together as a #package. Offering the components
individually,<br>
>> possibly as #progenitor-s, would help cases when
people really only<br>
>> want to fetch a single part.<br>
><br>
> I agree that having multiple #this is confusing (which
one is the good<br>
> one.??).<br>
> In my understanding #this must match the product_type as
in the<br>
> Obscore record.<br>
><br>
> If a spectrum bundle is exposed in a separate row, we
should have<br>
> something like this:<br>
><br>
> Obscore row:<br>
> -----------<br>
> - product_type=spectrum-bundle (tbd)<br>
> -
access_format=application/x-votable+xml;content=datalink<br>
><br>
> Datalink response:<br>
> -----------------<br>
> - link #1<br>
> - semantics=#this<br>
> - content_qualifier=spectrum-bundle (TBD)<br>
> - content_type=application/tar+gzip<br>
> - description="spectrum file + preview + ARF + RMF +
Background<br>
> spectrum"<br>
><br>
><br>
> If the spectrum is exposed in a separate row:<br>
><br>
> Obscore row:<br>
> -----------<br>
> - product_type=spectrum<br>
> -
access_format=application/x-votable+xml;content=datalink<br>
><br>
> Datalink response:<br>
> -----------------<br>
> - link #1<br>
> - semantics=#this<br>
> - content_qualifier=spectrum<br>
> - content_type=application/fits<br>
> - description="spectrum file"<br>
> - link #2<br>
> - semantics=#package<br>
> - content_qualifier=spectrum-bundle (TBD)<br>
> - content_type=application/tar+gzip<br>
> - description="spectrum file + preview + ARF + RMF +
Background<br>
> spectrum"<br>
><br>
> I do not believe we are able to design a standard HEIG
archive because<br>
> this too much mission/tool specific.<br>
> Do we really needs the archive content to be machine
readable?<br>
> Anyway, individual files can be exposed with an adapted
semantics.<br>
><br>
> Laurent<br>
><br>
><br>
>><br>
>> But at least for a prototype (and, if that works
fine, perhaps also<br>
>> as a long-term practice), I think nobody would be
terribly confused<br>
>> if #this were just the time series or spectrum, in
particular if a<br>
>> content-qualifier would let machines figure out what
it is they'll<br>
>> get.<br>
>><br>
>> I'm not too happy with #progenitor for the individual
components,<br>
>> though. Perhaps datalink/core should have a concept
#component with<br>
>> the definition "for datasets where #this is composed
of multiple<br>
>> individual artefacts, #component rows offer access to
individual<br>
>> artefacts. Use local-semantics to consistently mark
up the roles of<br>
>> the components." or so.<br>
>><br>
>> In the end, I think we need to see what will help
clients consuming<br>
>> this. Do we have software that we could use to try
that out? What<br>
>> do people use to work this #event-bundle-s?<br>
>><br>
>> -- Markus<br>
>><br>
><br>
> --<br>
> English version: https:
//<a class="moz-txt-link-freetext"
href="https://gcc02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.deepl.com%2Ftranslator&data=05%7C02%7Ctess.jaffe%40nasa.gov%7C6075ae45df2e4bb5325508dd8bfd5006%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C638820642266465078%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=vYY2ZoY99F8RvAPlu9Ske3PYa0z8E2Ne12cW%2BdF8Kl4%3D&reserved=0"
moz-do-not-send="true">https://gcc02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.deepl.com%2Ftranslator&data=05%7C02%7Ctess.jaffe%40nasa.gov%7C6075ae45df2e4bb5325508dd8bfd5006%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C638820642266465078%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=vYY2ZoY99F8RvAPlu9Ske3PYa0z8E2Ne12cW%2BdF8Kl4%3D&reserved=0</a><br>
<br>
--<br>
--<br>
Mireille Louys, MCF (Assistant Professor)<br>
Centre de données Astronomiques (CDS) Equipe Images,
ICube<br>
Observatoire de Strasbourg Telecom Physique
Strasbourg<br>
11, rue de l' Université 300, Bd Sebastien
Brandt CS 10413<br>
F-67000 Strasbourg F-67412 Illkirch
Cedex<br>
<br>
--<br>
heig mailing list<br>
<a class="moz-txt-link-abbreviated moz-txt-link-freetext"
href="mailto:heig@ivoa.net" moz-do-not-send="true">heig@ivoa.net</a><br>
<a href="http://mail.ivoa.net/mailman/listinfo/heig"
id="OWA2dc2fe19-a333-6255-b857-4440c58b3f43"
class="OWAAutoLink" data-auth="NotApplicable"
moz-do-not-send="true">https://gcc02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.ivoa.net%2Fmailman%2Flistinfo%2Fheig&data=05%7C02%7Ctess.jaffe%40nasa.gov%7C6075ae45df2e4bb5325508dd8bfd5006%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C638820642266487767%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=YfJb4jKT78AkfN4F7QVYmofeRGcMreiSO0uI9ObBHWo%3D&reserved=0</a></div>
<br>
<fieldset class="moz-mime-attachment-header"></fieldset>
</blockquote>
<p><br>
</p>
</blockquote>
<p><br>
</p>
</body>
</html>