[p3t] OpenAPI validators

[Joshua Fraustro]

Hi Gregory,

I replied to Paul that afternoon, but it didn’t seem to have bounced off the mailing list correctly. I’m copy-pasting the message here so hopefully everyone gets it. CC’ing also the original p3t mailing list

---

Hi Paul,

I think you’ve raised some very good points regarding the value of identifying a handful of tools the IVOA can informally recommend, or at least highlight, as useful for editors working with OpenAPI documents. As more editors gain experience with OpenAPI in their respective environments, I expect we’ll naturally build up a shared understanding of “this works well if you’re on Ubuntu with editor X” or “this validator integrates nicely with workflow Y.” We’re already seeing the beginnings of that with the work Pat and Dave have been doing.

Where I’m more hesitant is in the IVOA formally prescribing a specific toolchain, or proscribing others, at the standards process level. In practice that tends to create more problems than it solves: tools evolve or fall behind, people work in environments where some tools simply don't work, and we would risk taking on responsibility for maintaining or endorsing tooling that can quickly become outdated. I'm tempted to think of the TeX documents that are published, we don't care so much which of the many TeX distributions folks use (and there are many https://www.tug.org/interest.html#free): what matters is that the output compiles and renders correctly, less so what editor the author uses to create it.

You also noted that many of the tools you surveyed claim support for 3.2. That’s encouraging, but given that 3.2 was only published in September, I’d expect the ecosystem to be in a transitional state for a while. Our uptake curve for even minor IVOA spec updates tends to be much slower than two months, so I’m not surprised we’re seeing uneven support right now. I don't think the IVOA is really any different in this regard. Consider how many now unsupported, or one-off, tools are floating around the IVOA ecosystem for working with our various standards.

Regarding the Calycopis/Execution Broker example: If I recall, Dave mentioned during Interop that he’s using non-standard pre-processing and assembly tooling in that repository. The YAML files there are no doubt intermediate inputs, and not the final OpenAPI document he plans to publish as part of the standard. Given that, I wouldn’t draw strong conclusions from the bundling issue you hit. My assumption is that the final published document will be compiled in some form (perphaps still in separate files with 'proper/working' $ref links) that can be validated by a broad set of OpenAPI tools, regardless of how the pieces are managed internally in his repo.

One area where I do think we might find consensus is on the validation step itself. It may be reasonable for the IVOA to agree on a single validator, perhaps integrated via GitHub Actions or another CI mechanism, that defines the bar for what “valid for publication” means. Editors could work however they prefer internally, but the document they submit would need to pass that agreed-upon validator in CI. It would also be much easier for us at some further date to say, 'This validator has reached end-of-life, we're moving on to this one' for a single tool rather than a suite of them. We’d also want to ensure that reviewers are comfortable with the technical aspects, such as suggesting when an editor should $ref a shared component (e.g., MAXREC) rather than redefining it yet again.

I think Pat and Dave will no doubt have some good feedback on what they've found as reliable sets of tooling.

Kindly,
Joshua


From: Grégory Mantelet <gregory.mantelet at astro.unistra.fr>
Date: Thursday, December 11, 2025 at 4:44 AM
To: Joshua Fraustro <jfraustro at stsci.edu>, Patrick Dowler <pdowler.cadc at gmail.com>, Russ Allbery <eagle at eyrie.org>, Paul Harrison <paul.harrison at manchester.ac.uk>
Cc: "Molinaro, Marco" <marco.molinaro at inaf.it>
Subject: Fwd: [p3t] OpenAPI validators

External Email - Use Caution
Hi Joshua, Pat, Russ,

I feel I am not competent to answer to the request from Paul, as I have not yet
any practical experience with OpenAPI.

Does any of you has some guidance for him about how to use OpenAPI in practice
(especially with the $ref)?

Thank you for your help,
Grégory M.

PS: @Paul, sorry to answer so late...I was very busy since the interop meeting.


-------- Forwarded Message --------
Subject:
[p3t] OpenAPI validators
Date:
Mon, 24 Nov 2025 12:50:34 +0000
From:
Paul Harrison via p3t <p3t at ivoa.net><mailto:p3t at ivoa.net>
Reply-To:
Paul Harrison <paul.harrison at manchester.ac.uk><mailto:paul.harrison at manchester.ac.uk>
To:
Russ Allbery via p3t <P3t at ivoa.net><mailto:P3t at ivoa.net>
CC:
stdproc at ivoa.net<mailto:stdproc at ivoa.net>


Hi,

at the interop at the splinter session about UWS and OpenAPI https://yopad.eu/p/dal-splinter-interop-nov-2025<https://urldefense.com/v3/__https:/yopad.eu/p/dal-splinter-interop-nov-2025__;!!CrWY41Z8OgsX0i-WU-0LuAcUu2o!0RdE6rMwVAV4vhplXXv90T3fvSQyDwu1V0OdhXrUBhuhCxf_5g_Fz_LwPh2rskQT25BkXoyQOL6ia_ehruCluox2HVuRbIbpqHRk$>
I asked about OpenAPI validation tooling and CI, and did not really get an answer - I feel that until we have an agreed-on set
of tools that we are going to use then there is not much point in producing the OpenAPI definitions - I will illustrate what I mean below, but the main point being is that the OpenAPI spec is large and there are potentially some corners that we do not want to use because they are poorly supported in the tooling - This was certainly the case when we started to use XML Schema as a data model/interface definition language, and we banned ourselves from using Substitution groups for instance as they were poorly supported by tooling - we also made several other stylistic choices in how to represent concepts - e.g. there are mainly ComplexType definitions rather than Element definitions at the top level

In searching for tools, an “obvious” place to look is https://tools.openapis.org/categories/description-validators.html<https://urldefense.com/v3/__https:/tools.openapis.org/categories/description-validators.html__;!!CrWY41Z8OgsX0i-WU-0LuAcUu2o!0RdE6rMwVAV4vhplXXv90T3fvSQyDwu1V0OdhXrUBhuhCxf_5g_Fz_LwPh2rskQT25BkXoyQOL6ia_ehruCluox2HVuRbFH7L0gU$> - there are a lot of entries (71 in the description validation section) - not a good sign to me - no obvious “winners” (and probably a lot of 80% “solutions”). However, one filter that might indicate a commitment to producing a solid implementation is if they support the “latest” version (3.1 on this listing - though 3.2 is the recently published latest version in Sept 2025) - This reduces the number of possibilities down to 15, still rather too many to make a choice though - so I thought who is actually supporting 3.2 - and I found this list https://openapi.tools/#description-validators<https://urldefense.com/v3/__https:/openapi.tools/*description-validators__;Iw!!CrWY41Z8OgsX0i-WU-0LuAcUu2o!0RdE6rMwVAV4vhplXXv90T3fvSQyDwu1V0OdhXrUBhuhCxf_5g_Fz_LwPh2rskQT25BkXoyQOL6ia_ehruCluox2HVuRbF7bRXAy$> for which there is only one “winner” https://github.com/speakeasy-api/openapi<https://urldefense.com/v3/__https:/github.com/speakeasy-api/openapi__;!!CrWY41Z8OgsX0i-WU-0LuAcUu2o!0RdE6rMwVAV4vhplXXv90T3fvSQyDwu1V0OdhXrUBhuhCxf_5g_Fz_LwPh2rskQT25BkXoyQOL6ia_ehruCluox2HVuRbNl2FcKU$> - so I decided to do some experiments.

The most comprehensive use of OpenAPI so far that I could find in IVOA related work is https://github.com/ivoa/Calycopis-schema<https://urldefense.com/v3/__https:/github.com/ivoa/Calycopis-schema__;!!CrWY41Z8OgsX0i-WU-0LuAcUu2o!0RdE6rMwVAV4vhplXXv90T3fvSQyDwu1V0OdhXrUBhuhCxf_5g_Fz_LwPh2rskQT25BkXoyQOL6ia_ehruCluox2HVuRbPmHKVtp$>, so I decided to do some experiments.

% openapi spec validate Calycopis-broker.yaml Validating OpenAPI document: Calycopis-broker.yaml
✅ OpenAPI document is valid - 0 errors

Then, because my past experience of OpenAPI has been that most tools work best if everything is in a single document as the $ref does not work the way that most people naively expect, I decided to try the bundle functionality

% openapi spec bundle Calycopis-broker.yaml bundled.yaml
Processing OpenAPI document: Calycopis-broker.yaml
Error: failed to bundle document: failed to bundle item at /paths/~1offersets/post/requestBody/content/application~1json/schema: failed to bundle nested references in /Users/pharriso/Work/ivoa/Calycopis-schema/schema/components.yaml#/components/schemas/OfferSetRequest: failed to bundle item at /allOf/0: failed to resolve external schema reference /Users/pharriso/Work/ivoa/Calycopis-schema/schema/ExecutionComponents: open /Users/pharriso/Work/ivoa/Calycopis-schema/schema/ExecutionComponents: no such file or directory
Usage:
openapi spec bundle [input-file] [output-file] [flags]

Flags:
-h, --help help for bundle
--naming string Naming strategy for bundled components (counter|filepath) (default "filepath")
-w, --write Write bundled document back to input file

Global Flags:
-v, --verbose verbose output

Error: failed to bundle document: failed to bundle item at /paths/~1offersets/post/requestBody/content/application~1json/schema: failed to bundle nested references in /Users/pharriso/Work/ivoa/Calycopis-schema/schema/components.yaml#/components/schemas/OfferSetRequest: failed to bundle item at /allOf/0: failed to resolve external schema reference /Users/pharriso/Work/ivoa/Calycopis-schema/schema/ExecutionComponents: open /Users/pharriso/Work/ivoa/Calycopis-schema/schema/ExecutionComponents: no such file or directory


Oh dear - despite declaring the file valid, the bundle has not worked - hence my call for some guidance.


Paul.






--

p3t mailing list

p3t at ivoa.net<mailto:p3t at ivoa.net>

http://mail.ivoa.net/mailman/listinfo/p3t<https://urldefense.com/v3/__http:/mail.ivoa.net/mailman/listinfo/p3t__;!!CrWY41Z8OgsX0i-WU-0LuAcUu2o!0RdE6rMwVAV4vhplXXv90T3fvSQyDwu1V0OdhXrUBhuhCxf_5g_Fz_LwPh2rskQT25BkXoyQOL6ia_ehruCluox2HVuRbF6OK7X7$>








-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.ivoa.net/pipermail/p3t/attachments/20251211/e05e78ca/attachment-0001.htm>