<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<div class="moz-cite-prefix">Hi Pat, all<br>
</div>
<div class="moz-cite-prefix">Le 13/10/2021 à 20:26, Patrick Dowler a
écrit :</div>
<div class="moz-cite-prefix"><br>
</div>
<blockquote type="cite"
cite="mid:CAFK8nrq7Yba1gfJL2F7X=gKL4f_tM-oRxkSa0Bgikkj8UhqEpw@mail.gmail.com">
<div dir="ltr">
<div class="gmail_default"><br>
</div>
<div class="gmail_default">I agree with the intent of VEP-006
that #calibration and it's child properties should cover the
"use this raw data" use case</div>
<div class="gmail_default">and that the current ambiguity will
become an issue of not resolved.</div>
</div>
</blockquote>
<p>As you know I disagree that the previous definition was ambiguous
(tense was Ok). But anyway we didn't have a working implementation
yet in the VO landscape as far as we know.<br>
</p>
<p>So as I told yesterday during TCG I do not oppose to adoption of
VEP-006 to tackle this "applicable" use case.<br>
</p>
<blockquote type="cite"
cite="mid:CAFK8nrq7Yba1gfJL2F7X=gKL4f_tM-oRxkSa0Bgikkj8UhqEpw@mail.gmail.com">
<div dir="ltr">
<div class="gmail_default"><br>
</div>
<div class="gmail_default">I understand the "already applied"
use case and that some providers will want to provide links
with the product. If that use case is "debugging" or "QA" by a
human, then the existing #auxiliary and a decent description
should suffice to support that. Providers can start there and
see what shortcomings they run into... I completely agree with
Markus that when someone actually tries to do this that we'll
find out whether the use case really is QA or something else
and we can't solve it until we find out what something else
is. So, is</div>
<div class="gmail_default"><br>
</div>
<div class="gmail_default">#auxiliary description="flat field
used to calibrate this image"</div>
<div class="gmail_default"><br>
</div>
<div class="gmail_default">sufficient to prototype the other use
case(s)?<br>
</div>
<div class="gmail_default"><br>
</div>
</div>
</blockquote>
<p>I don't think so. With my provenance co-author hat and my
radioastronomy interests and examples I don't think it's a good
idea to mix all those things in auxiliray. at very first
argument, there is a possibility of selection of links using the
semantics terms and their hierarchy which we will lose by doing
that.</p>
<p>As I said yesterday I will propose a new VEP for
calibration-already-applied and this will be an opportunity to
discuss these things further. And I'm sure an implementation will
come soon. We can create a directory on dal-use-cases git
repository to push these use cases.<br>
</p>
<p>By the way #progenitor definition-fix has to be further
discussed. this is VEP-009.</p>
<p>Cheers</p>
<p>François<br>
</p>
<p><br>
</p>
<blockquote type="cite"
cite="mid:CAFK8nrq7Yba1gfJL2F7X=gKL4f_tM-oRxkSa0Bgikkj8UhqEpw@mail.gmail.com">
<div dir="ltr">
<div>
<div dir="ltr" class="gmail_signature"
data-smartmail="gmail_signature">
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div>--<br>
</div>
<div>Patrick Dowler<br>
</div>
Canadian Astronomy Data Centre<br>
</div>
Victoria, BC, Canada<br>
</div>
</div>
</div>
</div>
</div>
<br>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Wed, 13 Oct 2021 at 08:48,
BONNAREL FRANCOIS <<a
href="mailto:francois.bonnarel@astro.unistra.fr"
moz-do-not-send="true">francois.bonnarel@astro.unistra.fr</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote">Hi all, Markus<br>
As I said, I am ready to let VEP-006 go because it's trying to
solve a <br>
real use case if nobody's reluctant to it in the TCG.<br>
But I am confident that the other use case (applied) will get
out very <br>
soon, if its not already somewhere in some DataLink response
and not <br>
noticed by our "radars"<br>
(there is no possibility to check which terms are used in
DataLink <br>
services or capabilities at the registry level)<br>
So I will have warned : by not trying to find a consistent and
global <br>
solution now we may encounter problems in a near future<br>
But However I cannot keep silent when I read some
inconsistences in the <br>
rationale. see below<br>
Le 11/10/2021 à 14:19, Markus Demleitner a écrit :<br>
> François,<br>
><br>
> On Mon, Oct 11, 2021 at 09:58:42AM +0200, BONNAREL
FRANCOIS wrote:<br>
>>> On Fri, Oct 08, 2021 at 07:06:31PM +0200,
BONNAREL FRANCOIS wrote:<br>
>>>> Le 07/10/2021 à 15:24, Markus Demleitner a
écrit :<br>
>>>>> Based on this, could you then explain as
clearly and concisely as you<br>
>>>>> can why VEP-006 impedes that use case?<br>
>>>> A user discovers a calibrated image (HST,
ESO, etc...) . With DataLink<br>
>>>> (#this or #preview) she has a look to the
image and want to see how the<br>
>>>> uncalibrated data and the flat field looked
like to understand some of the<br>
>>>> features. DataLink provides a link to the
#progenitor and also (by some<br>
>>>> record the semantics of which cannot be
anymore "calibration or #flat) to<br>
>>>> the flat field, etc... used to calibrate this
progenitor.<br>
>>> ...but for this use case there is no need to
distinguish between what<br>
>>> you call a progenitor (i.e., non-calibration part
of provenance) and<br>
>>> calibration files applied. Right?<br>
>> Of course it's needed to make this distinction. Even
to obtain the right<br>
>> caption for the display.<br>
> A datalink client will obviously take the caption from
datalink's<br>
> description field, no? I frankly cannot see what role
the semantics<br>
> field could have in this.<br>
><br>
> What else are you thinking of? As I said, it helps to
use the "A<br>
> user wants... the computer does... using..." template
when stating<br>
> such things so other people can follow.<br>
><br>
>> Not to speak about possible reprocessing<br>
> I think we all agree that datalink metadata is *far* too
weak to<br>
> support this; I suspect even full provenance will not
usually let a<br>
> computer work out a reprocessing chain by itself. You
know, workflow<br>
> engines are the complex beasts they are for a reason.
So, datalink<br>
> may help selecting artefacts mentioned in a provenance
instance, but<br>
> for that, a "Part-of-Provenance" concept is enough.
Agreed?<br>
<br>
About the two answers above<br>
<br>
If the "description" display is enough for "applied" why is it
not the <br>
case for "applicable" (VEP-006 definition for #calibration) ?<br>
<br>
What should the client do more in the case of applicable than
for<br>
<br>
You have anyway no idea in general what kind of process you
will really <br>
apply with these calibration data ?<br>
<br>
You write that reprocessing is too complex for datalink in the
case of <br>
"already applied" but I imagine it's excatly the same for
applicable.<br>
<br>
So if "description" are enough why don't we follow Laurent,
Ada and many <br>
other before to relax the defintion and allow both use cases ?<br>
<br>
It is not my prefered solution as you know but it would be
more <br>
consistent with what you are writing there.<br>
<br>
Cheers<br>
<br>
François<br>
<br>
>>> Plus: A client can already do that, no? If you
think not: What do<br>
>>> you see missing?<br>
>> What would be the semantics term able to drive that?
Progenitor alone is<br>
>> not : this, at least, as been discussed extensively
(see below references)<br>
> I do not see why it would not be. "A user wants to debug
a data<br>
> product. The computer takes all #progenitor links and
displays them<br>
> together with their descriptions, offering to download
them for<br>
> inspection or possible use with a full description of the
provenance."<br>
><br>
>>>> Client software is intended to display all
these images (science and<br>
>>>> calibration) together for checking and
comparison. Moreover an advanced<br>
>>>> version could poropose some kind of
reprocessing of progenitor.<br>
>>> Not that that has any relationship to VEP-006 at
all, but we have<br>
>>> provenance for a detailed description of how the
various pieces of<br>
>>> the provenance chain play together; we certainly
do not want to<br>
>>> re-model that in the datalink vocabulary. It's
been compicated<br>
>>> enough to do that modelling once.<br>
>> Of course it is a very interesting use case of
DataLink to provide a link<br>
>> towards a full (or last step) ivoa provenance record.<br>
> Yes. But that doesn't mean we have to re-build
provenance in<br>
> datalink. On the contrary: we can have a nice, clean
separation of<br>
> concerns, where datalink says how to get things and
provenance says<br>
> how they fit together.<br>
><br>
>> What #calibration-applied provides is a kind of
"poor-lady" provenance<br>
>> which only links used datasets without any insight on
the activity and<br>
>> agents involved<br>
>><br>
>> DataLink in itself has a poor but efficient way to
characterize relationship<br>
>> between #this item and the target of the link<br>
> Yes -- it's enough to filter links, which is what we want
in datalink<br>
> semantics. And VEP-006 plus the current state does
exactly this.<br>
><br>
>>> Second, the current #progenitor is clear that if
there were any<br>
>>> "Calibration applied" links, they would be
covered by its concept; see<br>
>>> its description: "data resources that were used
to create this<br>
>>> dataset (e.g. input raw data)". You may not like
the concept or its<br>
>>> label, but we have VEP-009 to discuss that.<br>
>> Let's go back to VEP-009<br>
> Sure, but can we *please* do that outside of the VEP-006
discussion?<br>
> I'm very sure we're not yet proficient enough in this
kind of<br>
> discussion that we can have multiple of them at the same
time. And<br>
> I think you still have not argued why VEP-006 and VEP-009
could not<br>
> be treated separately, i.e., how my elaboration of how we
still have<br>
> all reasonable options even after accepting VEP-006.<br>
><br>
>> Some references<br>
>><br>
>> Paul Harrison May the 5th<br>
>><br>
>> Mireille , March the 23rd<br>
>><br>
>> Stephane Erard March the 25th<br>
>><br>
> All these persons have been at meetings in the meantime,
and (at<br>
> least that's what I took away from these meetings) they
were<br>
> satisified that their concerns were taken into account in
the current<br>
> form of VEP-006.<br>
><br>
> Paul, Mireille, and Stéphane: If I'm misrepresenting you,
please<br>
> correct me.<br>
><br>
>> Not to speak about the solution Pat proposed me in a
private email (see my<br>
>> email last monday for details). I have some, concerns
about it but this is<br>
>> the part I fooly agree with<br>
>><br>
>> Recursive usage of DataLink to provide both science
data and<br>
>> calibration-used data<br>
>><br>
>> #progenitor link followed by #this link to get
science data<br>
>><br>
>> #progenitor link followed by #calibration to get
calibration data associated<br>
>> to these rawr science data<br>
> While I don't believe this belongs into a discussion of
VEP-006, this<br>
> is one reason why I'm rather skeptical of your VEP-009:
With current<br>
> #progenitor, the link from the reduced to the raw
datalink document<br>
> would reasonably be #progenitor. With VEP-009, that is
quite<br>
> certainly no longer the case (unless your definition of
"science<br>
> data" took a surprising turn later). But let's discuss
that with<br>
> VEP-009.<br>
><br>
>> The consequence of this is that #progenitor itself
are science data<br>
> No, in that case it would be a mix of "science data" and
all kinds of<br>
> other things that went into the reduction. Which, mind
you, is fine,<br>
> and I think it's the way such things should be done.
It's just not<br>
> within the concept your description in VEP-009 seems to
try to<br>
> define.<br>
><br>
>>> If you disagree on this assessment: How would
VEP-006 influence this<br>
>>> deliberation?<br>
>> VEP-006 is not proposing new terms it's changing the
definition of old terms<br>
>> in a sense that calibration-applied is now forbidden.<br>
> Well, the concept "pre-VEP-006-#calibration minus<br>
> post-VEP-006-#calibration" apparently is not populated in
the current<br>
> VO, and as I argued in in two mails back, when members of
that<br>
> concept come around, all the options are still around
whether or not<br>
> we accept VEP-006; so, again, I don't see why we can't at
least get<br>
> VEP-006 off the table.<br>
><br>
><br>
>>> If not, François, can you at least agree to: "I
think VEP-006 is<br>
>>> wrong, but I'll not veto it"?<br>
>> Exactly this : if nobody interested I give up. But I
think we will encounter<br>
>> consistency issues in a near future if we don't
discuss the consequences of<br>
>> this major change of definition for #calibration.<br>
> Can you speculate what consistency issues you expect to
see? Because<br>
> you see, the only reason VEP-006 is there is that without
it there's<br>
> the problem that current #progenitor and #calibration
have a nonempty<br>
> intersection and a nonempty difference, which is really
bad in a<br>
> formal vocabulary of this sort.<br>
><br>
> Now, since the only point of the VEP is to fix an
inconsistency, it<br>
> would defeat the purpose if new ones came up.<br>
><br>
><br>
> Anyway, now that Fançoise has chimed in -- what do we do?<br>
><br>
> For me, it would still be helpful to see what problem you
François,<br>
> are trying to solve or you, Françoise, see as well. And
please try<br>
> to be as concrete as possible and to limit things to
VEP-006. And if<br>
> you feel that is *reayll* impossible, at least make a
strong and<br>
> reproducible case for why we have to solve the question
of the<br>
> "Part-of-Provenance" concept together with it.<br>
><br>
> -- Markus<br>
<br>
<br>
</blockquote>
</div>
</blockquote>
<p><br>
</p>
</body>
</html>