[p3t] Fwd: P3T

[Janet Evans]

Hi Folks,

This is the email outlining concerns that we discussed this afternoon.  The email went to DAL.
I’m forwarding it to p3t.

Thanks for the 1/2 hour mtg today … I’ll write up notes.  

-janet


> Begin forwarded message:
> 
> From: Mark Taylor via dal <dal at ivoa.net>
> Subject: P3T
> Date: May 23, 2024 at 6:16:40 AM GMT+10
> To: dal at ivoa.net
> Reply-To: Mark Taylor <m.b.taylor at bristol.ac.uk>
> 
> Dear colleagues,
> 
> having let the P3T session yesterday sink in I wanted to make a few
> comments.
> 
> [TL;DR: I've got reservations, but I'm not going to throw rocks in the way]
> 
> This proposal will lead to some fragmentation of the VO ecosystem.
> Up till now we have, with a few exceptions
> (e.g. PLASTIC->SAMP, SIA1->SIA2), been extremely conservative
> about breaking changes to the standards, e.g. Cone Search responses
> are still required to include UCD1s to the exclusion of UCD1+s that
> replaced UCD1 in around 2006.  The result has been that to a pretty
> large extent all the client and server software that has been written
> to date continues to interoperate.  For long-lived standards such
> as TAP and SCS new clients work with old services and old clients
> work with new services, though enhancements introduced in later
> versions of standards obviously are not available in those cases.
> As far as it goes, I'd say that is a pretty good thing.
> 
> The current proposal lobs a cannonball into that approach so that
> there is no chance of client/server code written for existing
> standards interoperating as it stands with code implementing
> the new versions of those standards.  Similarly, client/server
> code written to target the new versions will not interoperate
> with existing implementations.  The scope is currently TAP and UWS
> but it looks like the thin end of a sizeable wedge;
> phase 3 will target other standards, and several comments made
> during the presentations seemed to indicate that we can expect
> another similar round of breaking changes when OpenAPI falls
> out of fashion in a few years time, and so on.
> 
> The no-legacy-code-left-behind vs. move-fast-and-break-things
> difference between approaches is reminiscent to me of a comparison
> between development of the Java and Python platforms/languages;
> v1.1 of the Java application TOPCAT from 2004 runs happily under
> Java 21 from 2021, but running older python code on recent
> python platforms is typically much more problematic.
> Neither approach is absurd; each has its pros and cons.
> 
> Gregory's presentation yesterday acknowledges the issue but I would
> say understates the impact a little bit.  Even assuming that the
> major clients are adapted to work with both old and new APIs,
> the existing "Informal client" sector will be unable to talk
> to new services, and existing services will be inaccessible
> from any new OpenAPI-based clients; Joshua was keen to point
> out that such new clients are now going to be easy to develop.
> 
> I can see why the proposed changes appeal to large, well funded service
> providers.  But there are downsides from the point of view of legacy
> services; newly developed clients will not talk to them and the
> hurdle to take advantage of incremental changes in the standards
> becomes higher.  This model for standards development may come
> back to bite those projects which are currently well-resourced
> if they transition to maintenance-only mode in the future.
> 
> So, I'm wary of this way forward and I think people should be aware
> of the impacts we can expect.  But, maybe the benefits outweigh
> the costs.  Judging from the institutional involvement in the P3T
> and from most of the audience response at yesterday's session
> it looks like the prevailing opinion is that this is the way
> we should go, and that it is going to happen.
> 
> I wonder whether there is a middle way: rewrite the standards
> in terms of OpenAPI but do what we can to make them describe
> existing service behaviour, minimising breaking changes.
> There are two things being proposed here, and they are separable
> to a degree: firstly providing machine-readable (OpenAPI)
> definitions of the services, and secondly transitioning those
> services to behave in a more modern way.
> Page 8 of Joshua's presentation
> (https://www.google.com/url?q=https://wiki.ivoa.net/internal/IVOA/InterOpMay2024P3T/dal-openapi-tech-overview.pdf&source=gmail-imap&ust=1717013822000000&usg=AOvVaw09NP7wNt6mc6adFMtg4KXX)
> mentions one thing that can't be done in OpenAPI: case-sensitivity
> of parameters.  The other protocol changes described there are
> more about how people would do things given a blank sheet of paper
> than simply enabling machine-readable interface definitions.
> Use of x-www-form-urlencoded parameters is said to be vulnerable to
> CSRF, but there was some discussion on zoom about whether this is
> really the case, I haven't looked into the details.
> Case sensitivity we could probably fudge if we wanted to
> (make a breaking but low-impact change to standards requiring no
> modification for most code that uses uppercase anyway).
> If the issue is really just about OpenAPI compatibility and
> machine-readable interface definitions then perhaps it could be
> done with significantly less impact than what's being suggested.
> If on the other hand the appetite is really to take the
> opportunity to update lots of things without regard to backward
> compability, that's a different matter.  But we should be clear
> about the goals.
> 
> Whatever we do decide to do, I want the VO to work as well as possible
> for users, so I'd volunteer to prototype client code at an early
> stage in topcat and stilts.  I will mention another class of client
> that will need to be modified to work in dual mode: validators.
> The taplint validator that forms part of stilts is a complex
> tool widely used by TAP service deployers to check that their
> services are operating according to the specs (I'd be happy for
> others to develop a TAP validator, but nobody has), and if it
> doesn't work for OpenAPI-based services I predict service
> compliance and quality will drop significantly.
> It is true that off-the-shelf OpenAPI validators can be used to
> do part of the job of service validation in the new picture,
> but there's a lot that goes on in taplint that I don't think can be
> covered simply on the basis of understanding the OpenAPI contract.
> 
> I think this is going to require a significant amount of development
> effort from client authors like me, but how much will depend on the
> details of the proposal, so I look forward to the development and
> results of the proposed Phase 1.
> 
> Mark
> 
> --
> Mark Taylor  Astronomical Programmer  Physics, Bristol University, UK
> m.b.taylor at bristol.ac.uk          https://www.google.com/url?q=https://www.star.bristol.ac.uk/mbt/&source=gmail-imap&ust=1717013822000000&usg=AOvVaw3CwIj5PBWYFMLbE-QzCqin

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.ivoa.net/pipermail/p3t/attachments/20240523/5ec60079/attachment.htm>