IVOA Document Standards RFC V1.2

[Mark Taylor]

Bob,

On Thu, 2 Jul 2009, Robert Hanisch wrote:

> First, let me quote myself in the e-mail message I sent earlier today:
> 
> "One might argue that if one or more VO projects invests the efforts into
> developing the standard itself, it follows that they would put effort into
> implementations and validators.  But this could put a lot of pressure on
> schedule and resources of individual projects, and could lead to major
> delays in the promotion process."
> 
> I agree completely with you that doing implementations in parallel with
> standards development is the best way to go, with the obvious exception that
> some of the IVOA standards, as you point out, are not really about software
> implementations.
> 
> But making the implementations and the validator mandatory for promotion is
> where I, and Christophe and Francoise, feel it is necessary to pause.  For
> example, if right now we were to enforce this requirement on TAP, with the
> resource constraints we have in several of the major projects, we would be
> forced to stop the promotion process.  This would be an awful situation for
> IVOA, I think.  We want prototypes to inform the standards development
> process, for sure, we SHOULD have them, but from a management perspective I
> think we have to avoid boxing ourselves into a corner.

I agree with you that requiring the associated software would slow down 
the promotion process.  However, as I tried to make clear in my previous
message, I don't believe that the date of publication of the 
Recommendation is what we should be focussing on.
When a document reaches Recommendation status, yes it looks good on
Gantt charts, but all it means is we have a bit of paper to wave
in the air.  Nothing useful has been achieved until effective use of
that standard is made, which in most cases requires software to be
written.  I believe that attaining that second goal will actually
be accelerated rather than decelerated if the software is written
in tandem with the text (I tried to get you to agree or disagree with
this statement in my last message, but I still don't know your opinion).  
Doing them in parallel will also lead to a better standard.

In any case: if you can't find anyone who is prepared to implement
the standard, there's probably a good reason; either it's too hard
to do, or it's not something that people need.  In either case
it's no great achievement to have published it.

> It is also a bit ambiguous just how one proves that implementations are
> interoperable.  So far we have taken people's word, but IVOA has no means to
> independently test these assertions.

Agreed it's ambiguous, and rigorous enforcement is not really feasible.
I'm prepared to rely on the good will of the authors/implementors,
supplemented by whatever scrutiny other IVOA members are prepared
to provide on an ad-hoc basis.

> I would probably agree fully with you if, say, IVOA were a dues-based
> membership organization, with its own financial resources to build
> validators, or test those developed by others.  But I see a distinction,
> albeit a subtle one, between the projects' willingness to contribute to
> development of a standard and their ability to provide supporting software
> and validators on a timescale that does not jeopardize the promotion
> process.

Again - I believe that the timescale of the promotion process itself
is a red herring.  That given, I don't see the subtle difference here 
(management is neither my enthusiasm nor my expertise, so I'm quite 
prepared to believe there are issues over my head here) - could you 
elaborate a bit?

But I'll say this, I think it is somewhat dangerous for the system
to encourage or allow organisations to shove their oar in when it
comes to writing requirements, but not when it comes to resourcing
their implementation.

> Finally, in any case, we have a revision system that is there to deal with
> problems and shortcomings found in earlier versions.  If V1.0 of a standard
> ends up having problems -- unsupportable features, inconsistencies, etc.
> (and we have had this experience) -- then we fix the problems in V1.x or
> V2.0.

My understanding of the basic intention for the versioning system
is that version X+delta is version X with the additions or changes
gained from working with version X for a while, as well as 
enhancements which weren't feasible (e.g. from lack of resources)
on the timescale of version X.  While it can also be used for
patching up things which were broken or ill-thought-out in the
first place, it's not very well designed for that purpose, if only
because of the very long timescales typically involved.

> (and we have had this experience) -- then we fix the problems in V1.x or

and we'd have had the experience a lot less if software was written
in tandem with the original versions!

Mark

-- 
Mark Taylor   Astronomical Programmer   Physics, Bristol University, UK
m.b.taylor at bris.ac.uk +44-117-928-8776 http://www.star.bris.ac.uk/~mbt/