Draft CORS guidance for an IVOA JSON protocol

[Russ Allbery]

Markus Demleitner via grid <grid at ivoa.net> writes:
> On Tue, May 28, 2024 at 10:08:09AM -0700, Russ Allbery via grid wrote:

>> Oh, I see.  Hm, I think I would handle that at a different layer,
>> because in my mind the JSON-based protocol and the
>> x-www-form-urlencoded POST protocol are two different protocols that
>> should be advertised separately in the registry, similar to sync and
>> async.

> Oh, but we don't.  There has been attempt to do that in TAP 1.1 drafts,
> but it turned out to be extremely painful to do, and actually
> fundamentally flawed in some respect.  Hence, TAP 1.1 retained the
> registration pattern established in 1.0, i.e., a common interface
> declared with the "base" access URL.  The main thing that's wrong with
> it is that this interface is typed vs:ParamHTTP, which is semantically
> wrong (but in a way that does not hurt us much).

Ah, I think I see my error: I assumed that the information available from
the /capabilities endpoint would also make it into the registry, and it
sounds like that's a bad assumption.

We can do the same thing at the /capabilities level, though, yes?  The
service can indicate which of the two protocols it supports the way that
SODA indicates through /capabilities which of async and sync it supports.
(I realize that people probably don't use the /capabilities endpoint for
SODA very much.)

> Conceivably, having two different *interfaces* may be a good idea, in
> particular because json-POST probably does not fit very well with the
> semantics of vs:ParamHTTP; but I'd much rather think about letting both
> run on the same endpoint for starters and see if it becomes too ugly.

You can try to run them on the same URL with content negotiation, but
you're increasing the complexity a fair bit and my intuition is that a lot
of things could go wrong.  It's conceptually simpler to provide two
different interfaces.

Also, this doesn't create a reusable pattern for handling future wire
protocol changes, which I suspect will require separate interfaces.  I
think the most likely next wire encoding we'll want to move to in order to
keep pace wiht the broader web services community will look more like
gRPC, and if that's correct, reusing the same URL via content negotiation
probably won't be possible.

>> If a given IVOA service advertises (via the registry) the JSON protocol
>> but not the x-www-form-urlencoded POST protocol, then a browser-based
>> client will need to use the JSON protocol.

> *That* is something I would like to avoid, because it would lock out
> legacy clients without a good reason.  As you say, form-posting only is
> a problem for in-browser clients, and I think a solution where only they
> would need to worry about CSRF (and hence shun form-posting) would be
> highly preferable.

I agree that this would be highly preferable, but sadly this is not the
world that we live in.  That solution is not really possible given the web
security model.  A protocol that allows x-www-form-urlencoded simple
requests cannot benefit from any of the CSRF protection work that went
into CORS and has to fall back on the pre-CORS CSRF protections, and the
reliable strategies for CSRF protection for simple POST rely on client
cooperation and thus represent a protocol change.

In other words, it's not true that form-posting is only a problem for
in-browser clients.  Because browsers exist, authenticated simple POST is
inherently vulnerable to confused deputy attacks using browsers.  Form
posting is therefore a problem for any server that provides it, regardless
of what their clients intend to do, unless other CSRF mitigations are in
place.

For now, with the existing IVOA protocols, I think the best fallback CSRF
protection is to require an Authorization header on all POST requests,
which forces them to not be simple requests.  This is still a protocol
change, but it's probably the minimally intrusive one.  It does, however,
break protocol requests driven by an HTML form, so if people want to
support that for services requiring authentication, we'll have to
standardize something else.

> SODA is still a different issue, because it (currently, but largely by
> its nature) is not registry-driven.  Clients will only encounter SODA
> because of information left by the data provider in the VOTables
> returned from a Datalink endpoint (or a Datalink block in a result set,
> but I wish we'd never have specified that).

> There, we currently do not have a way to convey anything but
> form-posting (or GET) access URLs.

Right, this is definitely something that will have to be specified as part
of the protocol transition work.

> Well... it's more that I am convinced we need to keep requiring
> form-posting; it's really not much effort to implement,

It will probably not surprise anyone that I disagree.  :)

> and it will ensure that we don't end up with a VO where some services
> only work with client A but not client B, and others only with client B
> but not client A.

This problem is inherent in a protocol transition, though.  Supporting
every iteration of the protocol forever is another version of the
complexity trap that we're trying to avoid.  There has to be some
mechanism to actually transition and not keep implementing the older
protocol in new services.

> It sounds as if your preferred CSRF mitigation is JSON-posting.  If that
> is so, then let's not confuse the situation by creating a third option.

That is certainly an option; in that case, archives that have
authentication requirements would not support HTML forms as clients.

(Just to be clear, most of my reasons for wanting a JSON protocol are
unrelated to security.  Being able to rely on CORS is a very nice
additional benefit, but since these security threads seem to attract a lot
of attention, I wanted to be clear that I don't see security as the
primary driver.)

-- 
Russ Allbery (eagle at eyrie.org)             <https://www.eyrie.org/~eagle/>