[p3t] x-www-form-urlencoded prohibition

[Joshua Fraustro]

Hi Paul,

Thanks for the write-up. As I'm not a security researcher,  I'll let finer minds than mine expound on security concern details more. I'll note that the two lines referenced in my slide were paraphrasing the excellent document Russ Allbery published on his SODA/UWS implementation experience: https://sqr-063.lsst.io, specifically section 3. For folks' convenience, he cites the Mozilla documentation on HTTP/CORS/simple requests: https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS#simple_requests

I will note generally to this discussion, that the goal of the tiger team was not simply to provide an OpenAPI spec for IVOA services, but also take the opportunity to examine the way we invoke these services, their behaviors, and whether those are consistent with modern HTTP service development practices. The lines we're discussing in my slides, are examples of this kind of examination.

Joshua Fraustro



On 5/23/24, 4:20 PM, "p3t on behalf of Paul Harrison via p3t" <p3t-bounces at ivoa.net <mailto:p3t-bounces at ivoa.net> on behalf of p3t at ivoa.net <mailto:p3t at ivoa.net>> wrote:


External Email - Use Caution


In the recent P3T session, someone (I think Alberto Micol) pointed out that the proposed deprecation of x-www-form-urlencoded POSTs was a big change - and I agree it is probably the most disruptive change to protocols that is being suggested - the OpenAPI definition gets centre stage, but with the exception of clearing up a few ambiguities the current protocols can be described pretty much without change by OpenAPI. I am fully behind machine readable definitions of DMs and interfaces, so the OpenAPI definition is an unqualified “good thing”, and by itself does not really impose any change of web protocols.


It should be noted that the main design principles behind most of the current protocols were


* it should be possible to give someone a URL string that will run the service
* the service should be controllable/runnable using a web browser without javascript.


So html <FORM> POSTS were a natural thing to allow to satisfy the second requirement. The first requirement leads to service protocol definitions that REST purists do not like, and I also have sympathy with this view, and clearly the time to push back against Javascript and JSON has long passed - so again it seems fine to emphasise JSON as a serialisation format going forward.


So how did we get to


"Simple POSTs with x-www-form-urlencoded— no preflight checks— vulnerable to CSRF” being eliminated as bad behaviour.


Once you have JSON, you need JavaScript and you open up a whole extra attack surface for your web services however compared with the pre-javascript era. In that simpler time a user could pretty much trust what was happening if they trusted the address in the web browser and they could see what was happening displayed as graphical elements on the page before them and things only happened when they pressed buttons - Javascript meant that some arbitrary “invisible” interactions with servers could happen without any direct prompting. Security is always a compromise between utility and safety and so the initial reaction to this increased capability was to impose a “same origin policy” on javascript interactions with the servers, so at least the hidden interactions were guaranteed to be with the address that the user could see in the address bar - so again could be trusted. However, people really wanted to be able to talk to other server than the origin server so CORS was invented to punch a hole in the “same origin policy” by the browser doing a pre-flight check which involves asking the service the domains from which it will accept calls. The hole can be quite small and really works best in the sort of micro-services architecture where you have an API on one domain and a UI on another, so the API service can say that it only accepts calls from the UI service. However, in the IVOA context services cannot predict where they might be called from, so the only sensible CORS response is to allow all origins - i.e. adding no extra security - but adding an extra burden on the service provider in responding to CORS properly.


We still have not really decoded what is meant by "Simple POSTs with x-www-form-urlencoded— no preflight checks— vulnerable to CSRF” - lets start at the end with CSRF https://owasp.org/www-community/attacks/csrf <https://owasp.org/www-community/attacks/csrf> - this is an attack that works by tricking the user into performing an action on a site to which they are logged in - typically by having them visit the attackers site which then tricks the user into activating a call back to the original site that they are logged into - There are several mitigation https://cheatsheetseries.owasp.org/cheatsheets/Cross-Site_Request_Forgery_Prevention_Cheat_Sheet.html <https://cheatsheetseries.owasp.org/cheatsheets/Cross-Site_Request_Forgery_Prevention_Cheat_Sheet.html> techniques possible and there is indeed a recommendation to disallow the “simple” request types generated from <form> such as x-www-form-urlencoded because the “pre-flight checks” are not performed - however, we learned above that, in the IVOA service context, even when the pre-flight checks are done, there is no way that CORS can be used to restrict where the services are invoked from, so there is no CSRF mitigation from that alone - additional mitigations are needed. These are typically based around exchanging tokens - the questions are then


1. Do we want to try to standardise on one of the other mitigation techniques?
2. Do we still want to remove the x-www-form-urlencoded possibility as CORS does not provide any CSRF mitigation in our case?


Paul