IVOA Provenance DM -RFC- answers to comments

[François Bonnarel]

Hi Ole, all,

Answer on one point


Le 05/11/2018 à 09:57, Ole Streicher a écrit :
> Hi all,
>
> I shorten the text to the points where I have comments:
>
> On 04.11.2018 20:26, Mireille LOUYS wrote:
>> Le 29/10/2018 à 17:26, Markus Demleitner a écrit :
>>> TL;DR: let's only have the core model in 1.0.  We can always add
>>> extensions in 1.1.
>> we need the ActivityDescription class and Parameter class to be able
>> to search for some specific processing type on the data. Activity is
>> only the process launched for the computation. It does not hold the
>> details of the methods , because those details are factorised in the
>> ActivityDescription class.
> We could move (only) the ActivityDescription as a prov:Plan into the
> core model.
>
> Parameters are generally questioned as being special thingies that carry
> only configuration. Entities already may have the properties needed for
> Parameters.
>
> There is an ongoing discussion about Parameters; however it is stalled
> since two weeks, since nobody responded to the latest critics on this
> concept.
>
>


This is (part of ? ) the discussion  Ole is tackling on the RFC page

> OleStreicher <http://wiki.ivoa.net/twiki/bin/view/IVOA/OleStreicher> - 
> 2018-10-22 :Therefore, we propose to homogenize the handling of 
> |Parameter| with the handling of other |Entity| by re-using the 
> |used|, |EntityDescription|, and |UsageDescription| classes also for 
> |Parameter|. Then, |Parameter| is special only by the fact that they 
> directly contain a value, while other |Entity| s would refer to their 
> content by a link.
>
> /-- FrancoisBonnarel 
> <http://wiki.ivoa.net/twiki/bin/view/IVOA/FrancoisBonnarel> - 
> 2018-10-23: No (Data) Entities have values too./ -- FrancoisBonnarel 
> <http://wiki.ivoa.net/twiki/bin/view/IVOA/FrancoisBonnarel> - 2018-10-23
>
> -- OleStreicher 
> <http://wiki.ivoa.net/twiki/bin/view/IVOA/OleStreicher> - 2018-10-24 
> -1 : You did misunderstand this sentence: Our alternative proposal 
> <http://wiki.ivoa.net/internal/IVOA/ProvenanceRFC/ProvenanceDM-alternate.pdf> 
> is to make |Parameter| be a special |Entity| that carries a value, 
> while other (Data) |Entities| refer to the content via link.--
>
> -- MathieuServillat 
> <http://wiki.ivoa.net/twiki/bin/view/IVOA/MathieuServillat> - 
> 2018-10-24 -2 :The limit here is that we should not assume what a 
> general entity will be (how can we?), so we do not describe the 
> content/value of it, just the general category (specialized entity 
> that are commonly manipulated in astronomy) or the type of container 
> (in e.g. the EntityDescription 
> <http://wiki.ivoa.net/twiki/bin/edit/IVOA/EntityDescription?topicparent=IVOA.ProvenanceRFC;nowysiwyg=0> 
> with format, content_type... i.e. how to read it and not what we 
> read). However, configuration information in the form of parameters as 
> an input to an activity are relevevant provenance information (it 
> helps assess the reliability and quality of the activity, see section 
> 2.2.6 of the PR), so for this input parameter, we describe and 
> restrict the expected value of it, so that it becomes queryable in the 
> standard. The content/value of a Data entity is not expected to be 
> queryable using a provenance system, nor to provide relevant 
> provenance information.
>
> -- OleStreicher 
> <http://wiki.ivoa.net/twiki/bin/view/IVOA/OleStreicher> - 2018-10-24 
> -3 The value of a |Parameter| is also stored in the simpler 
> alternative model 
> <http://wiki.ivoa.net/internal/IVOA/ProvenanceRFC/ProvenanceDM-alternate.pdf>. 
> Our model does not restrict the queryability of that value. Also your 
> proposal does not specify that BTW.
>
> OleStreicher <http://wiki.ivoa.net/twiki/bin/view/IVOA/OleStreicher> - 
> 2018-10-22 : This would also make the described hack unnecessary.
>
> /-- FrancoisBonnarel 
> <http://wiki.ivoa.net/twiki/bin/view/IVOA/FrancoisBonnarel> - 
> 2018-10-23 : General statement to refuse this evolution: It is true 
> that Parameter has managed in a special way in the current PR. It is 
> intentional. Why ?/ /The Parameter class is there to tackle what 
> astronomy application users generally call "parameters" of the 
> application./ /Think to SExtractor or HipsGen 
> <http://wiki.ivoa.net/twiki/bin/edit/IVOA/HipsGen?topicparent=IVOA.ProvenanceRFC;nowysiwyg=0>. 
> They have a couple of parameters such as "ANALYSIS _THRESH", 
> "MAG_ZEROPOINT" (SExtractor), fov, skyval, border, publisher (HipsGen 
> <http://wiki.ivoa.net/twiki/bin/edit/IVOA/HipsGen?topicparent=IVOA.ProvenanceRFC;nowysiwyg=0>)./ 
> /They are definitly a different concept than DataEntity 
> <http://wiki.ivoa.net/twiki/bin/edit/IVOA/DataEntity?topicparent=IVOA.ProvenanceRFC;nowysiwyg=0>, 
> which are the things we want to follow with our Provenance model, the 
> things which are used or transformed by the activities./ /So 
> parameters are so strongly bound to activities that they don't have 
> existence outside their bounding to an activity./ /They have a value 
> which is either a number or string, or a reference to external 
> structures (files) or internal entities./ /In the latter case it is 
> possible to use as a parameter value an id of something which has an 
> history in the provenance./ /ActivityDescription and EntityDescription 
> <http://wiki.ivoa.net/twiki/bin/edit/IVOA/EntityDescription?topicparent=IVOA.ProvenanceRFC;nowysiwyg=0> 
> are there to gather all the properties common to activities sharing 
> the same kind of processing and to Entities they use or generate./ 
> /ParameterDescription gathers all the metadata common to Parameters of 
> Activities sharing the same ActivityDescription 
> <http://wiki.ivoa.net/twiki/bin/edit/IVOA/ActivityDescription?topicparent=IVOA.ProvenanceRFC;nowysiwyg=0>./ 
> /They have the same kind of binding to ActivityDescription 
> <http://wiki.ivoa.net/twiki/bin/edit/IVOA/ActivityDescription?topicparent=IVOA.ProvenanceRFC;nowysiwyg=0> 
> than Parameters have to activities./ /The organisation you are 
> proposing, Ole, simply miss the specificity of what is intended by the 
> parameter concept./ /Parameter class has also a strong semantic value 
> and so have ParameterDescription 
> <http://wiki.ivoa.net/twiki/bin/edit/IVOA/ParameterDescription?topicparent=IVOA.ProvenanceRFC;nowysiwyg=0> 
> and hadConfiguration./ /Last point: Parameter is a derivation of 
> Entity and ParameterDescription 
> <http://wiki.ivoa.net/twiki/bin/edit/IVOA/ParameterDescription?topicparent=IVOA.ProvenanceRFC;nowysiwyg=0> 
> a derivation of EntityDescription 
> <http://wiki.ivoa.net/twiki/bin/edit/IVOA/EntityDescription?topicparent=IVOA.ProvenanceRFC;nowysiwyg=0>./ 
> /The benefit of this is that generic W3C 
> <http://wiki.ivoa.net/twiki/bin/edit/IVOA/W3C?topicparent=IVOA.ProvenanceRFC;nowysiwyg=0>-aware 
> software can manage these classes. Parameter is an Entity with a 
> special type= voprov:parameter./ /But this type changes all the 
> "movie" (as we say in French) for behavior and interpretation./
>
> -- OleStreicher 
> <http://wiki.ivoa.net/twiki/bin/view/IVOA/OleStreicher> - 2018-10-26 : 
> For the discussion of (IVOA) parameters see point 6 in section 
> "Several inconsistencies" below.
>
> Independent of this, you did not explain why specifics of |Parameter| 
> require a distinct structure in the model instead of completely 
> integrating them into the normal |Entity| - |Activity| relations. All 
> the requirements you specified for parameters are already fullfilled 
> by the simpler alternative model 
> <http://wiki.ivoa.net/internal/IVOA/ProvenanceRFC/ProvenanceDM-alternate.pdf>. 
> Getting specific configuration parameters can simply done by querying 
> for the |*hadConfiguration*| relation. And specifically for parameters 
> like |ANALYSIS_THRESH|, I don't see why you never want to give them a 
> provenance and follow in our provenance model (Where did this value 
> come from? How was it created? Who entered that value into the 
> pipeline?). Or why you don't want to re-use the same |Parameter| for 
> an |Activity| with a different |ActivityDescription| (f.e. for a 
> bugfixed "HipsGen/1.01" instead of "HipsGen/1.00").
>
> So, parameters are (from the provenance point of view) not so 
> different from other |entities|:
>
>   * They /may/ have provenance information attached, or come without
>     provenance,
>   * They /may/ be restricted to a specific |ActivityDescription|
>     (which is one specific version of an application), or they may be
>     applied to other versions, or even other applications
>
> . -- MathieuServillat 
> <http://wiki.ivoa.net/twiki/bin/view/IVOA/MathieuServillat> - 
> 2018-10-24 : In addition to François' examples, I answer this above, 
> and it is explained why the value of a parameter is relevant 
> provenance information in the PR at the beginning of section 2.2.6
>
As already said the main difference  between Parameter and "Entities 
with prov:type=data" is semantic and not structural. I think if you 
activity is producing a radial velocity or flux at some TimeStamp in a 
time series,  by processing let's say an image or a spectrum, nobody in 
astronomy would like to call this a "parameter".
It is a data  entity  with a single value. and In some use cases 
parameters may be reference to Entities allready known.
So the  distinction you propose between Parameter as Entity with a 
single value and (data?) Entity for datasets and similar things 
introduces confusion and has no added value.  I definetely don't think 
we have to go in this direction.
The "hadConfiguration" relation   is a restriction (simplification) of 
used reserved to relations between Activities and Parameters. So it's 
reinforces the semantic specificity of using Parameters.

 From the UML point of view the Parameter and ParameterDescription 
classes are indeed derived from Entity, but they are REALLY different 
classes which can have different serialisation (and it is indeed the 
case in VOTable for ProvTAP )

It is only in W3C serialisation that we have to express Parameter and 
Parameterdescriptions in terms of w3c entities (basically the 
specification is done by a prov:type=Parameter, etc... in the W3C entity 
serialisation )


Regards
François
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.ivoa.net/pipermail/dm/attachments/20181107/0769455e/attachment.html>