IVOA Document Standards RFC V1.2

[Robert Hanisch]

On 7/3/09 5:53 AM, "Mark Taylor" <m.b.taylor at bristol.ac.uk> wrote:

> Bob,
> 
> On Thu, 2 Jul 2009, Robert Hanisch wrote:
> 
>> First, let me quote myself in the e-mail message I sent earlier today:
>> 
>> "One might argue that if one or more VO projects invests the efforts into
>> developing the standard itself, it follows that they would put effort into
>> implementations and validators.  But this could put a lot of pressure on
>> schedule and resources of individual projects, and could lead to major
>> delays in the promotion process."
>> 
>> I agree completely with you that doing implementations in parallel with
>> standards development is the best way to go, with the obvious exception that
>> some of the IVOA standards, as you point out, are not really about software
>> implementations.
>> 
>> But making the implementations and the validator mandatory for promotion is
>> where I, and Christophe and Francoise, feel it is necessary to pause.  For
>> example, if right now we were to enforce this requirement on TAP, with the
>> resource constraints we have in several of the major projects, we would be
>> forced to stop the promotion process.  This would be an awful situation for
>> IVOA, I think.  We want prototypes to inform the standards development
>> process, for sure, we SHOULD have them, but from a management perspective I
>> think we have to avoid boxing ourselves into a corner.
> 
> I agree with you that requiring the associated software would slow down
> the promotion process.  However, as I tried to make clear in my previous
> message, I don't believe that the date of publication of the
> Recommendation is what we should be focussing on.
> When a document reaches Recommendation status, yes it looks good on
> Gantt charts, but all it means is we have a bit of paper to wave
> in the air.  Nothing useful has been achieved until effective use of
> that standard is made, which in most cases requires software to be
> written.  I believe that attaining that second goal will actually
> be accelerated rather than decelerated if the software is written
> in tandem with the text (I tried to get you to agree or disagree with
> this statement in my last message, but I still don't know your opinion).
> Doing them in parallel will also lead to a better standard.
> 
> In any case: if you can't find anyone who is prepared to implement
> the standard, there's probably a good reason; either it's too hard
> to do, or it's not something that people need.  In either case
> it's no great achievement to have published it.
I disagree with you on this point, and perhaps that is the crux of the
issue.  Many organizations will not even look at an IVOA standard until it
has been finalized.  They say "why should I bother implementing something
that is just going to change."  Plus, like it or not, just reaching
agreement on the standard IS a major milestone for our development projects.
I don't know what is faster overall ... you are probably right, that when
the standard is accompanied by two or more implementations, it is more
likely to be correct and implementable by others outside of the core
development team.  But we also all work to external expectations (from our
funding agencies), and I have heard many times now comments from people at
data centers saying they are not going to code up an interface because it is
still a WD.  Please keep in mind that many of the implementors of IVOA
standards are not part of the VO development teams.

>> It is also a bit ambiguous just how one proves that implementations are
>> interoperable.  So far we have taken people's word, but IVOA has no means to
>> independently test these assertions.
> 
> Agreed it's ambiguous, and rigorous enforcement is not really feasible.
> I'm prepared to rely on the good will of the authors/implementors,
> supplemented by whatever scrutiny other IVOA members are prepared
> to provide on an ad-hoc basis.
We have no choice at the moment!

>> I would probably agree fully with you if, say, IVOA were a dues-based
>> membership organization, with its own financial resources to build
>> validators, or test those developed by others.  But I see a distinction,
>> albeit a subtle one, between the projects' willingness to contribute to
>> development of a standard and their ability to provide supporting software
>> and validators on a timescale that does not jeopardize the promotion
>> process.
> 
> Again - I believe that the timescale of the promotion process itself
> is a red herring.  That given, I don't see the subtle difference here
> (management is neither my enthusiasm nor my expertise, so I'm quite
> prepared to believe there are issues over my head here) - could you
> elaborate a bit?
As above.  Agreement on these standards is a major milestone.  Read any of
the NVO Quarterly Reports to NSF (all are on the NVO web page, in the
document collection) and you can see how important these are in terms of
deliverables.

> But I'll say this, I think it is somewhat dangerous for the system
> to encourage or allow organisations to shove their oar in when it
> comes to writing requirements, but not when it comes to resourcing
> their implementation.
This is not unique to the IVOA process!  Everyone is eager to tell you what
needs to be done, as long as someone else actually does it!

>> Finally, in any case, we have a revision system that is there to deal with
>> problems and shortcomings found in earlier versions.  If V1.0 of a standard
>> ends up having problems -- unsupportable features, inconsistencies, etc.
>> (and we have had this experience) -- then we fix the problems in V1.x or
>> V2.0.
> 
> My understanding of the basic intention for the versioning system
> is that version X+delta is version X with the additions or changes
> gained from working with version X for a while, as well as
> enhancements which weren't feasible (e.g. from lack of resources)
> on the timescale of version X.  While it can also be used for
> patching up things which were broken or ill-thought-out in the
> first place, it's not very well designed for that purpose, if only
> because of the very long timescales typically involved.
Which will be longer if we make interoperable implementations and validators
mandatory at every step of the process.

>> (and we have had this experience) -- then we fix the problems in V1.x or
> 
> and we'd have had the experience a lot less if software was written
> in tandem with the original versions!
> 
> Mark

Bob