about #calibration (VEP-006) : ----> IMPORTANT for DataLInk EXTENDED USAGE

[Francoise Genova]

Le 11/10/2021 à 09:58, BONNAREL FRANCOIS a écrit :
>
>
> Hi Markus, all
> Le 09/10/2021 à 07:43, Markus Demleitner a écrit :
>> Dear François,
>
> I'm really disappointed that nobody else get inside  this discussion.
>
> Semantics terms in DataLink have to be thought with a wide perspective 
> if we want to extend the usage
>
> But anyway, although I think it's useless to answer you I still do it.
>
>> On Fri, Oct 08, 2021 at 07:06:31PM +0200, BONNAREL FRANCOIS wrote:
>>> Le 07/10/2021 à 15:24, Markus Demleitner a écrit :
>>>> Based on this, could you then explain as clearly and concisely as you
>>>> can why VEP-006 impedes that use case?
>>> A user discovers a calibrated image (HST, ESO, etc...) . With DataLink
>>> (#this or #preview) she has a look to the image and want to see how the
>>> uncalibrated data and the flat field looked like to understand some of the
>>> features. DataLink provides  a link to the #progenitor and also (by some
>>> record the semantics of which cannot be anymore "calibration or #flat) to
>>> the flat field, etc... used to calibrate this progenitor.
>> ...but for this use case there is no need to distinguish between what
>> you call a progenitor (i.e., non-calibration part of provenance) and
>> calibration files applied.  Right?
>
> Of course it's needed to make this distinction. Even to obtain the 
> right caption for the display.
>
> Not to speak about possible reprocessing
>
I did not follow the discussion until now but I fully agree with 
Francois on that (and we did not discuss the point beforehand)

Francoise

>> Plus: A client can already do that, no?  If you think not: What do
>> you see missing?
> What would be the semantics term able to drive that? Progenitor alone  
> is not : this, at least, as been discussed extensively (see below 
> references)
>>> Client software is intended to display all these images (science and
>>> calibration) together for checking and comparison. Moreover an advanced
>>> version could poropose some kind of reprocessing of progenitor.
>> Not that that has any relationship to VEP-006 at all, but we have
>> provenance for a detailed description of how the various pieces of
>> the provenance chain play together; we certainly do not want to
>> re-model that in the datalink vocabulary.  It's been compicated
>> enough to do that modelling once.
>
> Of course it is a very interesting use case of DataLink to provide a 
> link towards a full (or last step) ivoa provenance record.
>
> What #calibration-applied provides is a kind of  "poor-lady" 
> provenance which only links used datasets without any insight on the 
> activity and agents involved
>
> DataLink in itself has a poor but efficient way to characterize 
> relationship between #this item and the target of the link
>
>
>>> Where VEP-006 "impedes" that is by letting the "already applied" use case
>>> orphan.  We have no more terms to qualify the calibration files used this
>>> way. With the new definition we can only apply the calibration files to the
>>> discovered image itself (#this in DataLink), and not to the progenitor
>> As stated multiple times, VEP-006 is entirely unconcerned with this
>> problem.
>>
>> First, we don't have any such links now, so nothing is orphaned at
>> this point.  And I maintain it would be prudent to wait until we
>> actually have such links, as the motivation of people publishing such
>> links will inform us if what we think is sensible behaviour (the
>> "pragmatics") actually is in the view of the data providers.
>>
>> Second, the current #progenitor is clear that if there were any
>> "Calibration applied" links, they would be covered by its concept; see
>> its description: "data resources that were used to create this
>> dataset (e.g. input raw data)".  You may not like the concept or its
>> label, but we have VEP-009 to discuss that.
>
> Let's go back to VEP-009
>
> Some references
>
> Paul Harrison May the 5th
>
>> The description of #progenitor in the RDF is just "data resources that were used to create this dataset (e.g. input raw data)”, and this combined with the disjointness of #calibration leads me to interpret #progenitor as being “science data” progenitor. I don’t see anything written inhttps://www.ivoa.net/documents/DataLink/20150617/REC-DataLink-1.0-20150617.html  that makes this interpretation invalid.
>
> Mireille , March the 23rd
>
>> One more reason :
>> #progenitor should be reserved to designate the data in transformation through various steps within a pipeline.
>> this applies to the data stream...
>> calibration, configuration, parameter sets have a distinct nature with respect to the data processing.
>> The two categories should not be mixed, in my view.
>
> Stephane Erard March the 25th
>
>> The use cas I have in mind is a spectral parameter map built from many spectral cubes, but I think the conclusion is identical to Mireille’s example.
>> In any case, I would certainly reserve #progenitor to identify calibrated products used to build a derived product. If this also links to calibration files, it will become difficult to identify the building pieces - at this stage, I’m no longer interested in details of calibration.
>
> Not to speak about the  solution Pat proposed me in a private email 
> (see my email last monday for details). I have some, concerns about it 
> but this is the part I fooly agree with
>
> Recursive usage of DataLink to provide both science data and 
> calibration-used data
>
> #progenitor link followed by #this link to get science data
>
> #progenitor link followed by #calibration to get calibration data 
> associated to these rawr science data
>
> The consequence of this is that #progenitor itself are science data
>
>
> BY the way, ESO ObsTAP service is using #progenitor exclusively for 
> science data in a rawer status.
>
>> The relevant point here is: VEP-006 in no way predetermines whether
>> we do nothing at all about "Calibration applied" (full disclosure: I
>> think that's the right choice), whether we create a child of
>> #progenitor (perhaps after fixing its label) or whether we change its
>> meaning and create a sibling of it.  Or whether we do something
>> entriely different, depending on what the pragmatics turns out to be.
>>
>> If you disagree on this assessment: How would VEP-006 influence this
>> deliberation?
> VEP-006 is not proposing new terms it's changing the definition of old 
> terms in a sense that calibration-applied is now forbidden.
>> So, again, please let's not mix up all these different discussions.
>> We will never get anywhere in semantics if we do.
>>
>> Everyone but François: Do you, as François alluded to, still have
>> concerns with VEP-006?.
>>
>> If not, François, can you at least agree to: "I think VEP-006 is
>> wrong, but I'll not veto it"?
>
> Exactly this : if nobody interested I give up. But I think we will 
> encounter consistency issues in a near future if we don't discuss the 
> consequences of this major change of definition for #calibration.
>
> Regards
>
> François
>
>>              -- Markus
>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.ivoa.net/pipermail/dal/attachments/20211011/1cef7509/attachment-0001.html>