Cryptographic authentication of VOEvents

[Norman Gray]

John, hello.

On 2012 Sep 10, at 01:42, John Swinbank wrote:

>> I can't help feeling, however, that the suggested practice of simply signing the serialised XML document, as a binary blob, is a little ... unambitious, and will lead to hassles later.
> 
> I'm a little on the fence about this. On general principle, I agree; however, I've seen few really convincing examples of what those hassles might be. I note the examples in your document ("if you want to do anything with the XML… [or] round-trip [it] into a system which doesn't know about your signature"), and find them somewhat persuasive but not compelling.
> 
> I'd be genuinely interested in a real-world application where regarding the VOEvent as an opaque bitstream to be signed causes serious problems.

I'm a bit of a tourist in this WG, so others will have to add detailed use-cases.  However the fact that you effectively have to abandon the signature as soon as a document enters an XML parser, seems an obvious massive downside.  You have to abandon the signature because it's axiomatic in any XML processing system that the serialisation -- the collection of angle brackets -- doesn't matter, so if it's the serialisation and not the content that you've signed, then you're going right against the grain.  Good engineering goes with the grain.

>> That 'easy' is easy to say (and I and others have I think said it in the past), but it turns out to be pretty easy to demonstrate, too.
>> 
>> See: http://www.astro.gla.ac.uk/users/norman/ivoa/voevent-signing/
> 
> As a relative XML neophyte, I find this really interesting. I do have a dumb question, though: the answer is probably obvious to those better schooled in the ways of XML than I.
> 
> We're told that the complexity of the XML infoset is what makes it hard to canonicalize. However, you suggest that this ESIS model overcomes these problems, presenting a bytestream which can be easily normalized. This sounds great, but: if the ESIS model provides an adequate representation of the document for signing, why hasn't it been adopted for XML-DSig?

An excellent question.  They're solving a somewhat harder, more general, problem, in that they're aiming to preserve everything that an XPath processor would see about a document.  The canonicalisation recommendation <http://www.w3.org/TR/xml-c14n> isn't _deeply_ complicated, but it ends up being more fiddly than one would expect (in particular, I found the stuff about namespace prefixes hard to follow).  Fiddly, here, implies errorprone and fragile, as you suggest, John, in your note.

To avoid the fiddliness, the transformation that I suggest above is a much more aggressive one. It cheerfully discards whitespace between elements, for example, and because it's not trying to transform XML into XML, it has a simpler task (no namespace gymnastics, for example).  Whitespace and namespaces are (in my experience) the two things that are most confusing about processing XML.

To be precise, I should emphasise that the procedure I describe isn't really a _normalisation_, since it doesn't turn XML into XML.  Instead, it's a function which turns XML into a readily-signed blob of bytes.  It's reversible, though, so you can go from that blob back to XML.

(Also, one could imagine an analogous transformation which turned equivalent JSON into the same blob of bytes)

> If, on the other hand, ESIS provides a good representation of a "simple" document like a VOEvent but isn't applicable in the more general case, can we be confident that it will provide an adequate representation of all future VOEvents? Versions 3, 4, … N, etc.

I think we can, on the grounds that the SAX API^W data model, which directly implies this XML-to-blob transformation, has long been regarded as suitable for processing general XML.

>> Implementation note: I've done this in Java because I'm already familiar with the relevant support there.  The normalisation process is inspired by the information made available by the (Java) SAX interface.  However 'SAX' information isn't unique to Java, and the process is not specific to Java, but could be implemented in any other language.  If I recall correctly, Xalan is in C and has a similar SAX-like interface already (I can't check that right now, but almost any language's streaming interface to XML would have something very like SAX).
> 
> For me, implementation issues would be key.
> 
> In particular, I'm worried about barrier-to-entry. Signing a bucket o' bytes has the advantage of simplicity; XML-DSig has the advantage of the W3C behind it, and consequent (presumed, at least) legitimacy and library support. This has neither: we're effectively saying to implementers that they will have to build their own canonicalization system. That might be straightforward to those who already know what they're doing (and I certainly take note of your Java implementation), but it's enough to make me blanche.

Perhaps presentation is key, here.  XML canonicalisation is fiddly but relatively simple.  But it's only a preface to the XML-DSig step, which is the one that looks seriously complicated.

Therefore a better way of describing what I've done here is to avoid the word 'normalisation', and instead describe it as a scheme for turning XML into an easily-signed blob.  Because this scheme handles the <?signature ...?> PI specially, it's easy to pass signatures through the scheme, and to carry signatures around with the XML.

I appreciate your nervousness about the procedure, but I don't blanche, because I think that implementation is pretty easy.

Implementation: this didn't take long to implement, because at heart it's just a custom Java ContentHandler plus the plumbing to handle the GPG subprocess.  What that means is that the ContentHandler receives notifications of the parsed events from the XML parser (eg element start, attribute spotted, character content, ...), and generates one record/line in the blob (it does this on the fly, while passing the notification upstream to the real processor consuming the parse).  On the way, it spots any signature present in the XML.  At the end, it invokes GPG with the blob and any detached signature and verifies or adds a signature as appropriate.  There's no reason why the blob can't be streamed to GPG as it's built up.

The signature, by the way, doesn't need any escaping in this scheme.  You extract the signature directly from the <?signature armor='-----BEGIN....-----'?> PI and pass it to GPG.

I can see no reason why this would be hard to implement in any language which had an analogous API.  The Xerxes-C parser has an API <http://xerces.apache.org/xerces-c/program-sax2-3.html> which directly mimics SAX.  Python has a SAX-like implementation, too <http://docs.python.org/library/xml.sax.html>

All the best,

Norman


-- 
Norman Gray  :  http://nxg.me.uk
SUPA School of Physics and Astronomy, University of Glasgow, UK