Codebases and the IVOA

[Mark Taylor]

Dear IVOA members,

  [TL;DR: Should there be an IVOA-controlled codebase? answers to apps list]

The IVOA Exec has asked the Applications Working Group to start a
discussion about source code repositories in the context of development
of VO applications and infrastructure.  The background/stimulus for this
was the discussion in the Time Domain focus session at the Heidelberg
interop meeting (May 2013).  Mario Juric (LSST) expressed disappointment
(and hence an impediment to LSST usage of IVOA standards) that there was
no one-stop shop for IVOA-blessed implementations of the various
standards.  His/LSST's main concern was with Python.

For most of the VO standards we do have high-quality implementations
in various languages available from publicly-readable source code
repositories.  However these tend to be controlled locally, at the
national project or institutional level.  This means (a) it is hard
to guarantee continued support in the long term and (b) it is not
clear to third parties which is the "right" one to use.
Since the codebases are developed independently of each other there
is also no guarantee that the the different components will play
well together.

An alternative model is that used by the Astropy Project
(http://www.astropy.org/).  Astropy operates a single github repository
and makes public releases of an integrated set of python packages
to do the kinds of things that astronomers want, including N-d and
tabular data manipulation, I/O formats, units, coordinates, fitting,
cosmology and more.  They even have T-shirts.  Astropy has a wide
base (~60) of contributing developers, and a small team (three)
of coordinators who oversee inclusion of contributed code, builds,
testing, policy, API changes etc.  I am not a regular python user,
but as far as I can see (1) this works pretty well in terms of
producing a high quality product and (2) it takes a lot of work
in terms of project coordination.

There is already some VO content to Astropy itself; the core
includes VOTable I/O, a Cone Search client and (soon) a SAMP hub/client.
The VAO has also registered its PyVO package (Registry and DAL client,
I think) as "affiliated" to Astropy, which counts as step towards
inclusion in the Astropy core.

So the question arises whether the IVOA ought to be doing something
more along the lines of Astropy than it is now.  I'm going to use
the term "codebase" [better suggestions welcome] to cover the
range of options - it would include a central source code repository,
but may also come with procedures and effort for curation activities
such as administration, testing, acceptance requirements,
release management, overall design, policy, etc.

Here is a list of open questions:

 * Is it desirable for the IVOA to operate one or more central codebases?
      - What would they look like in terms of curation?
      - What would be the pros and cons for (a) contributors (b) users?
      - Are there different answers for different types of software:
           client-side/server-side; different languages?
      - Are IVOA members prepared to supply the (significant) effort
           involved in curation?
      - Are contributors willing to cede control to the curators?
      - How would we handle decisions about curation policy and
           who's in charge?

 * Should the IVOA (or parts of it) draw up policy or best practice
   on code development?
      - Encouragement to use particular third-party codebases
           (e.g. Astropy)?
      - Encouragement to use particular technologies (e.g. github)?
      - List of required engineering practices for IVOA endorsement?

 * How should we be interacting with existing codebases?
      - Astropy: should IVOA-wide contributions to AstroPy be via PyVO,
           or separately for individual items?  Or not at all?
      - others?

We will conduct the discussion on the apps at ivoa.net mailing list.
If anyone has opinions on these or other related questions, please
post a response there.  If it looks like a good idea I may post
edited highlights on a wiki page.  Depending on how the discussion
goes, we may programme a session on the topic at the May Interop
meeting.

Mark Taylor (Apps WG chair)
Pierre Fernique (Apps WG vice-chair)

--
Mark Taylor   Astronomical Programmer   Physics, Bristol University, UK
m.b.taylor at bris.ac.uk +44-117-9288776  http://www.star.bris.ac.uk/~mbt/