<html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Aptos;
panose-1:2 11 0 4 2 2 2 2 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
font-size:12.0pt;
font-family:"Aptos",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
span.EmailStyle19
{mso-style-type:personal-reply;
font-family:"Aptos",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;
mso-ligatures:none;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style>
</head>
<body lang="EN-AU" link="blue" vlink="purple" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US">Hi Mark,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US">Thank-you for raising this. The question of compatibility is an important one and something we are very keen to explore in phase 1. Examining concrete implementations and being able
to assess the amount of change involved will be a key aspect of the assessment of phase 1 by the P3T team and the whole IVOA community when we report back. We hope to be able to work with the developers of clients in testing these proposals too. With a focus
on prototypes and following the regular processes for changes to standards we are aiming to avoid the ‘</span><span style="font-size:11.0pt">move-fast-and-break-things’ approach.</span><span style="font-size:11.0pt;mso-fareast-language:EN-US"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US">One of the aims of this work is to make it easier for standards to evolve in the future as web technology evolves. Our focus very much includes not just the current change but making
sure we lay the pathways for future evolution of the standards. We feel that having a machine readable definition of the standards will assist with any future evolution. The question of how we might tackle the next evolution is something we have discussed
a lot, and again hope to gain more clarity on in phase 1.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US">The value of validators such as taplint have been mentioned numerous times in the P3T meetings (and in the presentations that motivated the effort). This is an area we hope to be
able to contribute towards, but we understand that off the shelf tools only get us so far. However, tools such as client library generation may well make creation of validators for future standards a bit easier.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US">We would very much appreciate your assistance with prototyping client and validator impacts as part of phase 1. Thank-you for the offer of help there.<o:p></o:p></span></p>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">Cheers,<o:p></o:p></span></p>
<p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#00A9CE">James Dempsey</span></b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:"Calibri",sans-serif;color:black">Senior Developer
</span><span style="font-size:9.0pt;font-family:"Calibri",sans-serif;color:#00A9CE">|</span><span style="font-size:9.0pt;font-family:"Calibri",sans-serif;color:black"> CSIRO <o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"><a href="mailto:james.dempsey@csiro.au"><span lang="EN-GB" style="font-size:9.0pt;color:#0563C1">james.dempsey@csiro.au</span></a></span><span style="font-size:9.0pt;font-family:"Calibri",sans-serif;color:black">
</span><span style="font-size:9.0pt;font-family:"Calibri",sans-serif;color:#00A9CE">|</span><span style="font-size:9.0pt;font-family:"Calibri",sans-serif;color:black"> 02 6214 2912<o:p></o:p></span></p>
</div>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<div id="mail-editor-reference-message-container">
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal" style="margin-bottom:12.0pt"><b><span style="color:black">From:
</span></b><span style="color:black">dal <dal-bounces@ivoa.net> on behalf of Mark Taylor via dal <dal@ivoa.net><br>
<b>Date: </b>Thursday, 23 May 2024 at 6:17</span><span style="font-family:"Arial",sans-serif;color:black"> </span><span style="color:black">AM<br>
<b>To: </b>dal@ivoa.net <dal@ivoa.net><br>
<b>Subject: </b>P3T<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt">Dear colleagues,<br>
<br>
having let the P3T session yesterday sink in I wanted to make a few<br>
comments.<br>
<br>
[TL;DR: I've got reservations, but I'm not going to throw rocks in the way]<br>
<br>
This proposal will lead to some fragmentation of the VO ecosystem.<br>
Up till now we have, with a few exceptions<br>
(e.g. PLASTIC->SAMP, SIA1->SIA2), been extremely conservative<br>
about breaking changes to the standards, e.g. Cone Search responses<br>
are still required to include UCD1s to the exclusion of UCD1+s that<br>
replaced UCD1 in around 2006. The result has been that to a pretty<br>
large extent all the client and server software that has been written<br>
to date continues to interoperate. For long-lived standards such<br>
as TAP and SCS new clients work with old services and old clients<br>
work with new services, though enhancements introduced in later<br>
versions of standards obviously are not available in those cases.<br>
As far as it goes, I'd say that is a pretty good thing.<br>
<br>
The current proposal lobs a cannonball into that approach so that<br>
there is no chance of client/server code written for existing<br>
standards interoperating as it stands with code implementing<br>
the new versions of those standards. Similarly, client/server<br>
code written to target the new versions will not interoperate<br>
with existing implementations. The scope is currently TAP and UWS<br>
but it looks like the thin end of a sizeable wedge;<br>
phase 3 will target other standards, and several comments made<br>
during the presentations seemed to indicate that we can expect<br>
another similar round of breaking changes when OpenAPI falls<br>
out of fashion in a few years time, and so on.<br>
<br>
The no-legacy-code-left-behind vs. move-fast-and-break-things<br>
difference between approaches is reminiscent to me of a comparison<br>
between development of the Java and Python platforms/languages;<br>
v1.1 of the Java application TOPCAT from 2004 runs happily under<br>
Java 21 from 2021, but running older python code on recent<br>
python platforms is typically much more problematic.<br>
Neither approach is absurd; each has its pros and cons.<br>
<br>
Gregory's presentation yesterday acknowledges the issue but I would<br>
say understates the impact a little bit. Even assuming that the<br>
major clients are adapted to work with both old and new APIs,<br>
the existing "Informal client" sector will be unable to talk<br>
to new services, and existing services will be inaccessible<br>
from any new OpenAPI-based clients; Joshua was keen to point<br>
out that such new clients are now going to be easy to develop.<br>
<br>
I can see why the proposed changes appeal to large, well funded service<br>
providers. But there are downsides from the point of view of legacy<br>
services; newly developed clients will not talk to them and the<br>
hurdle to take advantage of incremental changes in the standards<br>
becomes higher. This model for standards development may come<br>
back to bite those projects which are currently well-resourced<br>
if they transition to maintenance-only mode in the future.<br>
<br>
So, I'm wary of this way forward and I think people should be aware<br>
of the impacts we can expect. But, maybe the benefits outweigh<br>
the costs. Judging from the institutional involvement in the P3T<br>
and from most of the audience response at yesterday's session<br>
it looks like the prevailing opinion is that this is the way<br>
we should go, and that it is going to happen.<br>
<br>
I wonder whether there is a middle way: rewrite the standards<br>
in terms of OpenAPI but do what we can to make them describe<br>
existing service behaviour, minimising breaking changes.<br>
There are two things being proposed here, and they are separable<br>
to a degree: firstly providing machine-readable (OpenAPI)<br>
definitions of the services, and secondly transitioning those<br>
services to behave in a more modern way.<br>
Page 8 of Joshua's presentation<br>
(<a href="https://wiki.ivoa.net/internal/IVOA/InterOpMay2024P3T/dal-openapi-tech-overview.pdf">https://wiki.ivoa.net/internal/IVOA/InterOpMay2024P3T/dal-openapi-tech-overview.pdf</a>)<br>
mentions one thing that can't be done in OpenAPI: case-sensitivity<br>
of parameters. The other protocol changes described there are<br>
more about how people would do things given a blank sheet of paper<br>
than simply enabling machine-readable interface definitions.<br>
Use of x-www-form-urlencoded parameters is said to be vulnerable to<br>
CSRF, but there was some discussion on zoom about whether this is<br>
really the case, I haven't looked into the details.<br>
Case sensitivity we could probably fudge if we wanted to<br>
(make a breaking but low-impact change to standards requiring no<br>
modification for most code that uses uppercase anyway).<br>
If the issue is really just about OpenAPI compatibility and<br>
machine-readable interface definitions then perhaps it could be<br>
done with significantly less impact than what's being suggested.<br>
If on the other hand the appetite is really to take the<br>
opportunity to update lots of things without regard to backward<br>
compability, that's a different matter. But we should be clear<br>
about the goals.<br>
<br>
Whatever we do decide to do, I want the VO to work as well as possible<br>
for users, so I'd volunteer to prototype client code at an early<br>
stage in topcat and stilts. I will mention another class of client<br>
that will need to be modified to work in dual mode: validators.<br>
The taplint validator that forms part of stilts is a complex<br>
tool widely used by TAP service deployers to check that their<br>
services are operating according to the specs (I'd be happy for<br>
others to develop a TAP validator, but nobody has), and if it<br>
doesn't work for OpenAPI-based services I predict service<br>
compliance and quality will drop significantly.<br>
It is true that off-the-shelf OpenAPI validators can be used to<br>
do part of the job of service validation in the new picture,<br>
but there's a lot that goes on in taplint that I don't think can be<br>
covered simply on the basis of understanding the OpenAPI contract.<br>
<br>
I think this is going to require a significant amount of development<br>
effort from client authors like me, but how much will depend on the<br>
details of the proposal, so I look forward to the development and<br>
results of the proposed Phase 1.<br>
<br>
Mark<br>
<br>
--<br>
Mark Taylor Astronomical Programmer Physics, Bristol University, UK<br>
m.b.taylor@bristol.ac.uk <a href="https://www.star.bristol.ac.uk/mbt/">https://www.star.bristol.ac.uk/mbt/</a><o:p></o:p></span></p>
</div>
</div>
</div>
</div>
</body>
</html>