VEP 6 Re: Please review VEPs 6 and 7

[BONNAREL FRANCOIS]

Dear all,
As Markus knows these are my concerns about VEP 6 as it is now.

My views have been expressed in my email  of June the 16th at 12:47 and 
on July the 2nd at 18:47 (answer to Pat Dowler).

They are close to Mireille's ones but not quite identical when finding a 
solution is concerned.

As Pat wrote there are two use cases for "calibration stuff".
Use case A : calibration already used in producing this

Use case B : calibration stuff to be used to reduce #this

With Markus new definition we encompass use case B, while the old one 
was pretty OK for use case A.

Several people (Paul Harrisson, Pat, Baptiste, Mireille, me) claim that 
Use case A exist. I don't have an example under the hand right now but I 
know it exists.

We have to find a solution valid for all use cases. My proposal in June 
(combining calibration terms  with "applied" and ";applicable") may be 
not the best but the problem has to be discussed and solved.

I know that Markus gave up pushing use case A to "progenitor" which is 
already good but still let the problem for use case A open.

Cheers
François

Le 26/07/2021 à 09:10, Markus Demleitner a écrit :

> Dear TCG,
>
> We have two VEPs for TCG review and voting at the next meeting
> (Janet, would you add it to the agenda?).  One was up a while ago
> before,
>
> https://volute.g-vo.org/svn/trunk/projects/semantics/veps/VEP-006.txt
>
>
> ✂ ✂ ✂ ✂ ✂ ✂ ✂ ✂ ✂ ✂ ✂ ✂ ✂ ✂ ✂ ✂ ✂ ✂ ✂ ✂ ✂ ✂ ✂ ✂ ✂ ✂ ✂ ✂ ✂
> Vocabulary: http://www.ivoa.net/rdf/datalink/core
> Author: Markus Demleitner <msdemlei at ari.uni-heidelberg.de>
> Date: 2020-09-09
>
> Term: #calibration
> Action: Modificiation
> Label: Applicable Calibration
> Description: Data products that can be used to remove instrumental
>    signatures from #this.  Note that the calibration steps such data
>    products feed have not been applied to #this yet.
> Used-in: http://dc.g-vo.org/kapteyn/q/dl/dlmeta?ID=ivo%3A//org.gavo.dc/~%3Fkapteyn/data/fits/POT015_000317.fits
>
> Term: #bias
> Action: Modification
> Description: Data products that can be used to remove detector offset levels
>    from #this.
>
> Term: #dark
> Action: Modification
> Description: Data products that can be used to remove detector dark
>    current from #this.
>
> Term: #flat
> Action: Modification
> Description: Data products that can be used to remove the signature of
>    non-homogeneous detector sensitivity from #this.
>
> Rationale:
>    In a discussion on the semantics mailing list (see
>    http://mail.ivoa.net/pipermail/semantics/2020-June/002735.html
>    and follow-ups) it was found that the existing descriptions of
>    #calibration and its narrower terms are ambiguous; "resource used
>    to calibrate" could mean both "resource that has been used" or
>    "resource that can be used".  This VEP tries to make it clear that the
>    "has been used" interpretation is for #progenitor, wheras #calibration
>    is for "can be used".
>
> Discussion:
>    On the Semantics mailing list
>    (http://mail.ivoa.net/pipermail/semantics/2021-March/002774.html and
>    followups), concerns were brought forward that excluding calibration
>    data already applied would unnecessarily complicate the vocabulary;
>    the temporal aspect ("has been applied" vs. "can be applied") should,
>    if possible, be kept out of it.  Against that it was put forward that
>    doing this would leave parts of #calibration within #progenitor (the
>    "has been applied" part), other parts essentially in what some people
>    suggested is #auxiliary (the "can be applied" part").  This violates
>    the conditions for keeping the concepts organised in a tree, which
>    was considered undesirable.
>
>    On the other hand, it was recognised that being able to trace "science
>    data" (as opposed to auxiliary resources like calibration data)
>    through the provenance chain is valuable.  A method proposed to effect
>    this, given that with VEP-006 #calibration is not available for this,
>    could be to narrow the definition of #progenitor to "less calibrated
>    science data".  But even if this step is not taken and #progenitor
>    remains "anything upstream in the provenance chain", a new term
>    #calibration-applied would seem useful (an example given was: when
>    fusing 50 images, people want to tell those apart from, for instance,
>    a master PSF that also went into the fusion).  Parties having use for
>    such a concept are encouraged to author a VEP for it.
>
>    In the end, after a side meeting at the May 2021 Interop consensus was
>    found that #calibration should certainly not contain elements both in
>    and outside of #progenitor; it was agreed that while, if we started
>    again today, we would call the VEP-006 #calibration something like
>    #calibration-applicable.  However, given the label is there, and that
>    the level of detail below #calibration (with #bias, #dark, and #flat)
>    probably mainly is useful (as far as datalink with its focus on
>    actionable semantics is concerned) when a client wants to
>    semi-automatically perform the calibration itself, it was decided that
>    #calibration is kept with its label changed to "Applicable
>    Calibration" and a corresponding definition.
>
>    As we sharpen the definition of #auxiliary ("resources aiding the
>    scientific exploitation of #this"), #calibration should probably
>    become a child of it.  This, however, would be part of a VEP on
>    #auxiliary.
>
>    This VEP recommended originally using #progenitor for labelling
>    calibration already applied.  This was quite severely opposed and
>    hence dropped from the description.
> ✂ ✂ ✂ ✂ ✂ ✂ ✂ ✂ ✂ ✂ ✂ ✂ ✂ ✂ ✂ ✂ ✂ ✂ ✂ ✂ ✂ ✂ ✂ ✂ ✂ ✂ ✂ ✂ ✂
>
>
> The second VEP to review was uncontroversial in WG review:
>
>
> ✂ ✂ ✂ ✂ ✂ ✂ ✂ ✂ ✂ ✂ ✂ ✂ ✂ ✂ ✂ ✂ ✂ ✂ ✂ ✂ ✂ ✂ ✂ ✂ ✂ ✂ ✂ ✂ ✂
> Vocabulary: http://www.ivoa.net/rdf/datalink-core
> Author: Baptiste Cecconi <baptiste.cecconi at obspm.fr>
> Date: 2021-06-09
>
> New Term: #detached-header
> Action: Addition
> Label: Detached Header
> Description: Machine-readable metadata for #this, which in general
> will be necessary for its scientific use.  Examples include FITS
> headers distributed without their data blocks or PDS label files.
>
> Relationships: rdfs:subPropertyOf(documentation)
> Used-in: http://voparis-tap-maser.obspm.fr/voyager_pra/epn/dl/dlmeta?ID=VG1-J-PRA-3-RDR-LOWBAND-6SEC-V1.0%3APRA_I.TAB
>
> Rationale:
>    In some formats and archives, the metadata required to decode the
>    content of #this is in a separate file. In the case of NASA/PDS
>    data products, for example, the PDS label file contains the decoding
>    metadata. In some archives, the FITS header may be stored separately
>    as a plain text file, next to a data file consisting of a binary
>    stream of bytes. Clients would use the content type (MIME type)
>    (e.g.: application/x-pds4-label+xml, or text/x-pds3-label) to enable
>    the processing.
>
> Discussion:
>    In the thread at
>    http://mail.ivoa.net/pipermail/semantics/2021-June/002806.html,
>    the proposal was received favourably.  There was a proposal to perhaps
>    add a wider term *#metadata, but given there already is #documentation
>    which probably covers a large part of such a concept already, no
>    immediate need was brought forward.
> ✂ ✂ ✂ ✂ ✂ ✂ ✂ ✂ ✂ ✂ ✂ ✂ ✂ ✂ ✂ ✂ ✂ ✂ ✂ ✂ ✂ ✂ ✂ ✂ ✂ ✂ ✂ ✂ ✂
>
> So, please review these two proposals; I'd suggest comments
> addressing issues beyond pure formalities should preferably go to the
> semantics list (as opposed to the TCG list, which is non-public).
>
> Thanks,
>
>               Markus