[Ops] Note draft: "Operational Identification"

[Pierre Fernique]

Le 09/03/2021 à 11:17, Mark Taylor a écrit :
> Markus, Pierre, all,
>
> a few comments on comments.
>
> On Fri, 5 Mar 2021, Markus Demleitner wrote:
>
>> I inadvertently replied to Pierre directly (rather than the list)
>> when I commented on his post.  Ahem.  Let me take this opportunity to
>> make some of my points in public:
>>
>> On Thu, Mar 04, 2021 at 05:08:39PM +0000, Mark Taylor wrote:
>>> On Mon, 1 Mar 2021, Pierre Fernique wrote:
>>>> types of op-purpose: validate, monitor, harvest.I think the first two
>>>> will be not so easy to differentiate. In practice, they are usually the
>>>> same tools that perform both functions, and it is unlikely that
>>>> developers will modify the User-agent to suit the use that the user will
>>>> be made of it. Maybe keep only one term: "check" ? Also, the notion of
>>> I agree that there will be grey areas between validation and monitoring,
>>> though e.g. most of the activities that I can recall being presented
>>> in recent Ops IG sessions fall fairly clearly into one or the other
>>> category (e.g. the weather reports carried out by VO-Paris and ESAC,
>>> and taplint are validation activities, while the servicemon tool
>>> presented by Tom Donaldson at the last interop was monitoring).
>>> But it's true that the distinction is not that important for the
>>> purposes that I'd envisage - identifying which queries are
>>> non-science ones.  I'd be inclined to keep the terms "validate"
>>> and "monitor", but I don't have very strong feelings about it,
>>> so if others think "check" would be better, we can change it.
>> I'd have said "monitoring" is a repetitive, timed activity (which
>> would include the weather report checks for me), whereas validation
>> is something more or less initiated by a user (such as the RofR
>> validator).
> ... which isn't the distinction I'd make.  I'd have said that
> validation is testing correctness, while monitoring is testing
> liveness or performance; either could be automated/repetitive
> or human-initiated.
>
> Given that the distinction is evidently a matter of debate,
> maybe Pierre is right, we should collapse these to a single term -
> "check" or "test".
+1 😁
>
>> The reason I believe it makes sense to tell these two apart is that
>> if I have a load spike due to validation I can relax because it'll be
>> rare.  Load spikes due to monitoring would be concern.
> I'd say that the important thing is just to distinguish 'operational'
> from 'science' queries.  In practice if I was looking at these things
> I can imagine just grepping for "(IVOA-" and use the statistics and
> the rest of the User-Agent content to work out what's really going on -
> I don't think we should attempt anything too fine-grained with these
> headers.  However, since I don't run a service that's not something
> I'm actually likely to have to do, so input from those that have
> attempted this or expect to do so would be useful.
>
>>>> "harvest" may be too "VO registry" oriented and very limited in
>>>> practice. I'm afraid it won't be used for other inter-center data
>>>> synchronization operations. It would probably be interesting to enlarge
>>>> its usage by giving some other examples for the use of this
>>>> "op-purpose": Synchronizing data from one server to another server =>
>>>> notably Hipsgen MIRROR could use this term, or Simbad's recurring global
>>>> queries for populating partner DBs. I was also wondering about the
>>> That sounds reasonable, maybe a more neutral term like "mirror" would
>>> be better?  Markus as more of a registry expert than me might want to
>>> comment.
>> In my original response to Pierre I wrote:
>>
>>    A non-Registry example for harvest is GloTS, and that is already
>>    following this proposal (or so I hope); but of course you're right,
>>    other syncing activity should be somehow mentioned, too.  I'm
>>    uncertain whether there's a point to tell these apart but I'm leaning
>>    towards "probably not".
>>
>>    If so, what you're asking for is a different term, right?  Trouble
>>    is, I don't have a good one.  "m2m-transfer" ("machine to
>>    machine")...?  Argl.
>>
>> "mirror" as this other term I don't like too much, as I think it is a
>> different use case than harvest.  I can't think of a major
>> operational reason to tell them apart, though, so having an umbrella
>> term for both would still seem reasonable.
> "copy"?

+2

Concerning the modification of the User-Agent. I think that the simple 
suggestion to update a table on the IVOA site of the AUs used by clients 
as proposed by Mark (perhaps with an implementation date) will allow us 
to adjust the modifications on the computation of statistics according 
to the methods used by each data provider. In any case, for the method 
used by the CDS (agents distributed on the different servers), this will 
be ok.

Cheers
Pierre

>
> --
> Mark Taylor  Astronomical Programmer  Physics, Bristol University, UK
> m.b.taylor at bristol.ac.uk          http://www.star.bristol.ac.uk/~mbt/
> _______________________________________________
> ops mailing list
> ops at ivoa.net
> http://mail.ivoa.net/mailman/listinfo/ops