<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<div class="moz-cite-prefix">Hi Pat, all,</div>
<div class="moz-cite-prefix"><br>
</div>
<div class="moz-cite-prefix">Le 17/06/2021 à 00:18, Patrick Dowler a
écrit :<br>
</div>
<blockquote type="cite"
cite="mid:CAFK8nrrTa2CFP+8509OOnVM=SidS7-YuTsUU-qJyRtErevi4ug@mail.gmail.com">
<div dir="ltr">
<div dir="ltr">
<div class="gmail_default"><br>
</div>
<div class="gmail_default">I just want to comment on the two
competing interpretations of #calibration (and children) and
how one would use them in datalink.<br>
</div>
<div class="gmail_default"><br>
</div>
<div class="gmail_default">Suppose you have two data products
from ObsCore: raw data at calib_level 1 and calibrated data
at calib_level 2. With the #calibration that could be
applied interpretation, the datalink(s) provided could be
something like:<br>
</div>
<div class="gmail_default"><br>
</div>
<div class="gmail_default">rawID #this {url to raw fits file}</div>
<div class="gmail_default">rawID #dark {url to a suitable dark
frame}</div>
<div class="gmail_default">rawID #flat {url to a suitable flat
field}</div>
<div class="gmail_default">rawID #derivation {url to datalinks
for calibID}</div>
<div class="gmail_default"><br>
</div>
<div class="gmail_default">calibID #this {url to calibrated
fits file}</div>
<div class="gmail_default">calibID #progenitor {url to
datalinks for rawID}</div>
<div class="gmail_default"><br>
</div>
<div class="gmail_default">With this set of links, clients
that find the rawID can find out about the derivation or
they can chose to download #this, #bias, and #flat and then
do the subtraction and division: those rawID links are
"actionable". Clients that find the calibID can navigate to
the progenitor to look at the calibration files associated.
Caveat: this doesn't say those were the actual calibration
files used -- those could be the default recommended
calibration files -- so it is a weaker statement. Knowing
those were actually applied to create "calibID #this"
requires provenance.... for that maybe we really want
something like:<br>
</div>
<div class="gmail_default"><br>
</div>
<div class="gmail_default">calibID #provenance {url to
provenance metadata eg instance of ProvDM}</div>
<div class="gmail_default"><br>
</div>
<div class="gmail_default">On the other hand, with the
#calibration already applied interpretation, you would have
links like this:</div>
<div class="gmail_default"><br>
</div>
<div class="gmail_default">
<div class="gmail_default">rawID #this {url to raw fits
file}</div>
<div class="gmail_default">rawID #derivation {url to
datalinks for calibID}</div>
<div class="gmail_default"><br>
</div>
<div class="gmail_default">calibID #this {url to calibrated
fits file}</div>
<div class="gmail_default">
<div class="gmail_default">calibID #bias {url to the bias
frame}</div>
<div class="gmail_default">calibID #flat {url to the flat
field}</div>
</div>
<div class="gmail_default">calibID #progenitor {url to
datalinks for rawID}</div>
</div>
<div class="gmail_default"><br>
</div>
</div>
</div>
</blockquote>
It can also happen that a discovery service only delivers calibID
and that #rawID is only delivered through DataLink. One example is
ALMA ObsTAP/SIA services where the raw (visibility) data are made
availbale in DataLink only.<br>
<blockquote type="cite"
cite="mid:CAFK8nrrTa2CFP+8509OOnVM=SidS7-YuTsUU-qJyRtErevi4ug@mail.gmail.com">
<div dir="ltr">
<div dir="ltr">
<div class="gmail_default">So, someone with the calibID could
examine the calibration files and in principle someone with
the rawID could navigate to the derivation and find the
calibration files. In this case the interpretation of
calibration (and children) is a little stronger and you
could infer that they were the ones actually used to produce
"calibID #this" and you could use those links to recalibrate
"rawID #this". <br>
<br>
<b>And here is the big BUT:</b> #calibration already applied
is only useful if you actually have the calibrated data! A
data provider with only raw data (yeah, that is still a
thing) has no way to tell users how to calibrate "rawID
#this".<br>
</div>
</div>
</div>
</blockquote>
Sure. <br>
<blockquote type="cite"
cite="mid:CAFK8nrrTa2CFP+8509OOnVM=SidS7-YuTsUU-qJyRtErevi4ug@mail.gmail.com">
<div dir="ltr">
<div dir="ltr">
<div class="gmail_default"><b><br>
</b></div>
<div class="gmail_default"><b>So, there are two use cases
here: assess quality by looking at calibration files
already applied vs perform calibration of raw data. You
really want to do the latter in the case where calibrated
data doesn't exist, which means only one of the above
interpretations works.</b></div>
</div>
</div>
</blockquote>
<p><b>I fully agree. If we have two use cases we need two terms (or
two parallel families of terms). An option could be to admit to
concatenate "applied" or "applicable" to #calibration, #bias,
etc...</b></p>
<p><b>Something like #calibration;#applied, #bias;#applicable, etc
.... If we admit concatenating terms in semantics (open
question) <br>
</b></p>
<blockquote type="cite"
cite="mid:CAFK8nrrTa2CFP+8509OOnVM=SidS7-YuTsUU-qJyRtErevi4ug@mail.gmail.com">
<div dir="ltr">
<div dir="ltr">
<div class="gmail_default"><br>
</div>
<div class="gmail_default">. Aside 1: the top-level concept of
#auxiliary seems to me to indicate "resources needed to
interpret #this" (error, noise, weights are in there) and I
think if calibID above could have some #auxiliary links for
some of the things we've discussed... I don't think
calibration terms belong in there in either interpretation)</div>
</div>
</div>
</blockquote>
+1<br>
<blockquote type="cite"
cite="mid:CAFK8nrrTa2CFP+8509OOnVM=SidS7-YuTsUU-qJyRtErevi4ug@mail.gmail.com">
<div dir="ltr">
<div dir="ltr">
<div class="gmail_default"><br>
</div>
<div class="gmail_default">Aside 2: in the above, the cross
linking with #progenitor and #derivation are intended to
mean "calibID #this to rawID #this" and not to specify that
"calibID #this" was created from all of the rawID links.
That is, I do agree the #progenitor is for the "science
data" and a #progenitor link to another set of links just
means that "rawID" is the progenitor. I'm not actually sure
that's the best way to present a link to other links... it
is just an example. <br>
</div>
<br>
</div>
</div>
</blockquote>
<p>humm. Not sure to catch that. Sorry !</p>
<p><br>
</p>
<p>Cheers</p>
<p>François<br>
</p>
<blockquote type="cite"
cite="mid:CAFK8nrrTa2CFP+8509OOnVM=SidS7-YuTsUU-qJyRtErevi4ug@mail.gmail.com">
<div dir="ltr">
<div dir="ltr">
<div>
<div dir="ltr" class="gmail_signature"
data-smartmail="gmail_signature">
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div>--<br>
</div>
<div>Patrick Dowler<br>
</div>
Canadian Astronomy Data Centre<br>
</div>
Victoria, BC, Canada<br>
</div>
</div>
</div>
</div>
</div>
<br>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Wed, 16 Jun 2021 at
03:48, BONNAREL FRANCOIS <<a
href="mailto:francois.bonnarel@astro.unistra.fr"
moz-do-not-send="true">francois.bonnarel@astro.unistra.fr</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote">
1 ) The current definition of #calibration (and child
elements) is <br>
unambiguous I think. They currently read "resource used to
calibrate <br>
the primary data" , "used to subtract the detector offset
level" (bias), <br>
"used to subtract the accumulated detector dark current"
(dark), "used <br>
to calibrate variations in detector sensitivity" (flat)<br>
To me this looks unambiguous and means that the link's
target HAS been <br>
used to calibrate this. And I think the use-case for that is
quality <br>
checking as Mireille an Paul already enhanced it.<br>
<br>
<br>
</blockquote>
</div>
</div>
</blockquote>
<p><br>
</p>
</body>
</html>