Codebases and the IVOA

[Markus Demleitner]

Dear Mark, dear Apps WG,

On Fri, Feb 14, 2014 at 03:01:44PM +0000, Mark Taylor wrote:
>   [TL;DR: Should there be an IVOA-controlled codebase? answers to apps list]

TL;DR: There should.  But can it?

> Since the codebases are developed independently of each other there
> is also no guarantee that the the different components will play
> well together.

*That* particular question IMHO is one of validation and hence
shouldn't enter here (except via maintainability, perhaps).

> So the question arises whether the IVOA ought to be doing something
> more along the lines of Astropy than it is now.  I'm going to use
> the term "codebase" [better suggestions welcome] to cover the
> range of options - it would include a central source code repository,
> but may also come with procedures and effort for curation activities
> such as administration, testing, acceptance requirements,
> release management, overall design, policy, etc.

First, whenever I talk to astronomers and astronomy-related computer
people outside the VO, the second-most heard request pretty much goes
in this direction -- "where can I download the VO," more or less.

Finding out what they actually want is quite a bit harder, though.
"Give me a library for SSAP", e.g., is a wish that becomes hazy when
you try to figure out what such a thing should be doing -- most of it
is dealing with metadata that, for legacy services would already be
there in some form -- but an interface abstracting the metadata would
become more unwieldy than just telling people: well, this is what
your VOTable should look like.  Or the alternative: Here's server
software implementing both the metadata management and the network
interface, and discard your old stuff.  Either way, no library.

So, I'm skeptical about too much common code base on the server side
(though I'd obviously love wider participation in DaCHS
development:-).  Here's instead what I believe would be a realistic
and feasible program to offer something on the client side:

There should be "official" VOTable packages at least in C, python,
and java that multiple projects feel responsible for and that are
regularly validated.  For python, that would be astropy's, obviously.
Maybe the Apps WG could capture some volunteer who'd then report on
the current state of the endorsed packages at each interop?

Incidentally, I cannot resist adding that these packages should
really, really include STC handling.

Once VO-DML serialization is specified, similar and compatible
libraries for serializing and deserializing more generic data models
would be the IMHO next-low-hanging fruit, as that will greatly ease
doing useful things with what's in the VOTables and give extra value
on top of what people can already do with FITS tables.  Since that's
coming up and no legacy/tradition code for *this* needs to be taken
into account, maybe this could be a good test case for inter-project
software development?

Basd on this, I'd be planning for clients for our major protocols,
including some abstraction of querying the registry, and fun parts
like unit parsing and conversion.

Much of this exists for python either in astropy or in an affiliated
package, so I guess there, it'd be a matter of encouraging projects
to participate in astropy/pyVO, and that'd be much better than to try
and build infrastructure and community of our own (even though I
stubbornly remain skeptical about using "free" private services).


For Java and C (which, incidentally, I'd really like to see as an
implementation language for its easy embedability -- have it in C,
and you have it in basically any language out there) -- well, I'm not
aware of any community-based efforts for an astronomy toolkit in
these languages that project members could join in.  But again, I'd
much rather think about who would be contributing, who's maintaining,
say, VOTable code in Java right now, and if these groups couldn't
first talk to each other whether to merge their efforts.

Only after this I'd be talking about infrastructure; SVN servers or
publicly pushable git repositories or what have you are cheap, in
comparison to getting people to join up.

Maybe there could be a coder's meetup at the next interop, where
everyone maintaining VOTable code in some language and in some client
has two minutes to say what they're doing, and then we split up
according to who we think could be joining up?

Cheers,

          Markus