VEP-006: Discussion summary

[BONNAREL FRANCOIS]

Hi Pat, all,

Le 17/06/2021 à 00:18, Patrick Dowler a écrit :
>
> I just want to comment on the two competing interpretations of 
> #calibration (and children) and how one would use them in datalink.
>
> Suppose you have two data products from ObsCore: raw data at 
> calib_level 1 and calibrated data at calib_level 2. With the 
> #calibration that could be applied interpretation, the datalink(s) 
> provided could be something like:
>
> rawID #this {url to raw fits file}
> rawID #dark {url to a suitable dark frame}
> rawID #flat {url to a suitable flat field}
> rawID #derivation {url to datalinks for calibID}
>
> calibID #this {url to calibrated fits file}
> calibID #progenitor {url to datalinks for rawID}
>
> With this set of links, clients that find the rawID can find out about 
> the derivation or they can chose to download #this, #bias, and #flat 
> and then do the subtraction and division: those rawID links are 
> "actionable". Clients that find the calibID can navigate to the 
> progenitor to look at the calibration files associated. Caveat: this 
> doesn't say those were the actual calibration files used -- those 
> could be the default recommended calibration files -- so it is a 
> weaker statement. Knowing those were actually applied to create 
> "calibID #this" requires provenance.... for that maybe we really want 
> something like:
>
> calibID #provenance {url to provenance metadata eg instance of ProvDM}
>
> On the other hand, with the #calibration already applied 
> interpretation, you would have links like this:
>
> rawID #this {url to raw fits file}
> rawID #derivation {url to datalinks for calibID}
>
> calibID #this {url to calibrated fits file}
> calibID #bias {url to the bias frame}
> calibID #flat {url to the flat field}
> calibID #progenitor {url to datalinks for rawID}
>
It can also happen that a discovery service only delivers calibID and 
that #rawID is only delivered through DataLink. One example is ALMA 
ObsTAP/SIA services where the raw (visibility) data are made availbale 
in DataLink only.
> So, someone with the calibID could examine the calibration files and 
> in principle someone with the rawID could navigate to the derivation 
> and find the calibration files. In this case the interpretation of 
> calibration (and children) is a little stronger and you could infer 
> that they were the ones actually used to produce "calibID #this" and 
> you could use those links to recalibrate "rawID #this".
>
> *And here is the big BUT:* #calibration already applied is only useful 
> if you actually have the calibrated data! A data provider with only 
> raw data  (yeah, that is still a thing) has no way to tell users how 
> to calibrate "rawID #this".
Sure.
> *
> *
> *So, there are two use cases here: assess quality by looking at 
> calibration files already applied vs perform calibration of raw data. 
> You really want to do the latter in the case where calibrated data 
> doesn't exist, which means only one of the above interpretations works.*

*I fully agree. If we have two use cases we need two terms (or two 
parallel families of terms). An option could be to admit to concatenate 
"applied" or "applicable" to #calibration, #bias, etc...*

*Something like #calibration;#applied, #bias;#applicable, etc .... If we 
admit concatenating terms in semantics (open question)
*

>
> . Aside 1: the top-level concept of #auxiliary seems to me to indicate 
> "resources needed to interpret #this" (error, noise, weights are in 
> there) and I think if calibID above could have some #auxiliary links 
> for some of the things we've discussed... I don't think calibration 
> terms belong in there in either interpretation)
+1
>
> Aside 2: in the above, the cross linking with #progenitor and 
> #derivation are intended to mean "calibID #this to rawID #this" and 
> not to specify that "calibID #this" was created from all of the rawID 
> links. That is, I do agree the #progenitor is for the "science data" 
> and a #progenitor link to another set of links just means that "rawID" 
> is the progenitor. I'm not actually sure that's the best way to 
> present a link to other links... it is just an example.
>
humm. Not sure to catch that. Sorry !


Cheers

François

> --
> Patrick Dowler
> Canadian Astronomy Data Centre
> Victoria, BC, Canada
>
>
> On Wed, 16 Jun 2021 at 03:48, BONNAREL FRANCOIS 
> <francois.bonnarel at astro.unistra.fr 
> <mailto:francois.bonnarel at astro.unistra.fr>> wrote:
>
>     1 ) The current definition of #calibration (and child elements) is
>     unambiguous I think.  They currently read "resource used to calibrate
>     the primary data" , "used to subtract the detector offset level"
>     (bias),
>     "used to subtract the accumulated detector dark current" (dark),
>     "used
>     to calibrate variations in detector sensitivity" (flat)
>     To me this looks unambiguous and means that the link's target HAS
>     been
>     used to calibrate this. And I think the use-case for that is quality
>     checking as Mireille an Paul already enhanced it.
>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.ivoa.net/pipermail/semantics/attachments/20210702/4cec6dee/attachment.html>