<html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Aptos;
panose-1:2 11 0 4 2 2 2 2 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
font-size:10.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
span.EmailStyle19
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;
mso-ligatures:none;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style>
</head>
<body lang="EN-AU" link="blue" vlink="purple" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US">Hi Dave,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US">Thanks for raising this. This is a really important point.
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US">We want to make recommendations to the WGs for how to make the protocols easier to implement for modern web tools. Some of this will be tidying up RESTful use to align with industry
best practice. We’ve also said we will investigate OpenAPI as a formal definition of the protocol APIs.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US">Those changes are highly likely to break compatibility with the existing protocols. However, the overall design of the protocols and much of the consensus which has coalesced around
those protocols is still highly valuable and we should not throw that out.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US">Redesigning the protocols from scratch, or coming up with new protocols, is not something we have the resources to do. We have three more monthly meetings plus a two day workshop.
Coming up with an agreed set of best practices and a path forward is already an ambitious goal for this group in that timeframe!<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US">The layout you’ve proposed for the repository looks good to me.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US">Also, I’d be happy to assist with the spring implementation if you want a hand Dave. I’ve used SwaggerCodeGen in the past to generate Java client libraries from OpenAPI specs with
good results.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt">Cheers,<o:p></o:p></span></p>
<p class="MsoNormal"><b><span style="font-size:11.0pt;color:#00A9CE">James Dempsey</span></b><span style="font-size:11.0pt;color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:9.0pt;color:black">Senior Developer </span>
<span style="font-size:9.0pt;color:#00A9CE">|</span><span style="font-size:9.0pt;color:black"> CSIRO <o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><a href="mailto:james.dempsey@csiro.au"><span lang="EN-GB" style="font-size:9.0pt;color:#0563C1">james.dempsey@csiro.au</span></a></span><span style="font-size:9.0pt;color:black">
</span><span style="font-size:9.0pt;color:#00A9CE">|</span><span style="font-size:9.0pt;color:black"> 02 6214 2912<o:p></o:p></span></p>
</div>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<div id="mail-editor-reference-message-container">
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal" style="margin-bottom:12.0pt"><b><span style="font-size:12.0pt;font-family:"Aptos",sans-serif;color:black">From:
</span></b><span style="font-size:12.0pt;font-family:"Aptos",sans-serif;color:black">p3t <p3t-bounces@ivoa.net> on behalf of Dave Morris via p3t <p3t@ivoa.net><br>
<b>Date: </b>Monday, 26 February 2024 at 3:48</span><span style="font-size:12.0pt;font-family:"Arial",sans-serif;color:black"> </span><span style="font-size:12.0pt;font-family:"Aptos",sans-serif;color:black">pm<br>
<b>To: </b>p3t@ivoa.net <p3t@ivoa.net><br>
<b>Subject: </b>[p3t] 'contract first' and OpenAPI<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt">Hiya,<br>
<br>
Can we jump back a bit and just check what the aim of the pt3 team is.<br>
<br>
As I understood it, the aim of the team was to evaluate how to make the <br>
IVOA services compatible with new tools and frameworks like OpenAPI, <br>
FastAPI and Spring. Identify the legacy issues that make it hard to <br>
implement our services using these tools, and make recommendations on <br>
how best to develop the next generation of our services.<br>
<br>
I'm not sure we have the remit to design the next generation cone-search <br>
and UWS here. I'd have though that would be up to the DAL and GWS <br>
working groups to build on the recommendations that come from this team <br>
to help them develop the next generation of their services.<br>
<br>
Anyway, I'd like to leave the details of the cone-search and UWS <br>
improvements for a moment and concentrate on the evaluating the new <br>
tools and processes part. In particular, this email is about how and why <br>
we want to use OpenAPI.<br>
<br>
From what I understand the current contribution from Russ uses Python <br>
and the FastAPI framework to generate the service description. The <br>
service is defined by the Python code and the OpenAPI definition is <br>
reverse engineered from that.<br>
<br>
I'm not clear how this would be applicable in the context of the IVOA, <br>
where we are explicitly designing standards to work across a <br>
heterogeneous environment of different programming languages and <br>
frameworks.<br>
<br>
Where OpenAPI descriptions may be able to help us develop interoperable <br>
specifications is if we work "contract first". Starting with the OpenAPI <br>
definition, included as part of the standard document, and then each <br>
group could use the machine readable definition to generate or validate <br>
their implementations.<br>
<br>
Now that Russ has given us an initial OpenAPI description for <br>
cone-search, I'd like to see if we change direction of the process and <br>
generate our service implementations from the OpenAPI description.<br>
<br>
With this in mind I would like to add a number of directory levels at <br>
the top of the project repository.<br>
<br>
First, I would separate the specification from the implementations, <br>
placing the openapi.json contract in a separate 'ivoa' directory at the <br>
top of the project. Making it clear that this is the interoperable part <br>
that would be provided as part of the IVOA standard.<br>
<br>
cone-search<br>
ivoa<br>
openapi.json<br>
impl<br>
....<br>
<br>
A number of people have commented on the readability and maintainability <br>
of the JSON version of the OpenAPI description. In which case, perhaps <br>
it would be better to start with the YAML version.<br>
<br>
cone-search<br>
ivoa<br>
openapi.yaml<br>
impl<br>
....<br>
<br>
Then, within the implementations, I would add a separation based on <br>
languages and framework.<br>
<br>
cone-search<br>
ivoa<br>
openapi.yaml<br>
impl<br>
python<br>
fastapi<br>
....<br>
java<br>
spring<br>
....<br>
<br>
This enables others to add implementations using their favourite <br>
frameworks and languages.<br>
<br>
I would be willing to volunteer to develop a Java based implementation <br>
using the Spring framework.<br>
<br>
It would be interesting to see if we could get volunteers to have a go <br>
at developing similar implementations using Rust or Go.<br>
<br>
Note that the Python FastAPI implementation here would be the reverse of <br>
the one currently provided by Russ. It would start with the OpenAPI <br>
description and use the FastAPI framework to auto-generate the service <br>
implementation Python classes from the OpenAPI description.<br>
<br>
To make this clearer we could add a further level of distinction between <br>
the forward process, starting with the OpenAPI definition and generating <br>
the implementation classes, and the reverse process, starting with the <br>
implementation classes and auto-generating the OpenAPI definition.<br>
<br>
cone-search<br>
ivoa<br>
openapi.yaml<br>
impl<br>
python<br>
fastapi<br>
forward<br>
# OpenAPI -> auto-generated Python<br>
reverse<br>
# Python -> auto-generated OpenAPI (Russ's <br>
current code)<br>
java<br>
spring<br>
forward<br>
# OpenAPI -> auto-generated Java<br>
reverse<br>
# Java -> auto-generated OpenAPI<br>
<br>
<br>
Once we have at least 2 forward implementations that start "contract <br>
first" from the OpenAPI description and generate the implementation <br>
classes from it, then we would be able to evaluate if including the <br>
machine readable OpenAPI description of the service is a useful addition <br>
that helps in developing interoperable services.<br>
<br>
It is interesting to note that Gregory also has cone-search <br>
implementation that is able to provide OpenAPI and Swagger descriptions <br>
of their service, but so far no one has said that they started with an <br>
OpenAPI description and generated their implementation from it.<br>
<br>
If it turns out that everyone is happier developing their code using <br>
their existing tools, without auto-generating the classes from an <br>
OpenAPI description, then this would suggest that providing an OpenAPI <br>
definition as part of our standards might not be as useful as we first <br>
thought.<br>
<br>
Historical note - the same thing happened with WSDL. Everyone said it <br>
was the "next best thing", but in the end, no one actually used it to <br>
generate code for their services.<br>
<br>
The resulting code was just too fragile.<br>
<br>
Looking forward to the discussion.<br>
<br>
-- Dave<br>
<br>
--------<br>
Dave Morris<br>
Research Software Engineer<br>
Wide Field Astronomy Unit<br>
Institute for Astronomy<br>
University of Edinburgh<br>
--------<br>
AIMetrics: [{"name": "*","contribution": {"value": 0,"units": "%"}}]<br>
--------<br>
-- <br>
p3t mailing list<br>
p3t@ivoa.net<br>
<a href="http://mail.ivoa.net/mailman/listinfo/p3t">http://mail.ivoa.net/mailman/listinfo/p3t</a><o:p></o:p></span></p>
</div>
</div>
</div>
</div>
</body>
</html>