ExecutionBroker and prior art.

[Salgado, Jesus]

Hi Paul,

I'm not entirely certain that the functionality of ExecutionBroker aligns precisely with the domain of UWS. ExecutionBroker is designed to assess the complexity of a specific execution on a particular science platform (or node) by considering execution requirements, available resources, permissions, data proximity, and more.

As I've discussed with Dave several times, there is a degree of coupling between the execution interface and ExecutionBroker so the execution interface defined in UWS is partially redefined within ExecutionBroker (Dave could provide more specific details). Due to the time-limited nature of "execution offers," complete decoupling is not feasible in his design. However, this is something I would be in favour of mitigate by a better decoupling if possible.

Rather than adding new functionality to UWS, I prefer to explore a more decoupled approach. This would involve using ExecutionBroker to identify available resources for an execution and employing UWS (potentially with updates in UWS 2.0) as the execution interface to execute the offer. This decoupling would result in clearer interfaces and simpler implementation, as we wouldn't need a monolithic service.

It's true that there is some overlap in the definition of execution parameters and resource data models (e.g., CPUs, GPUs, memory). We might be able to reuse previous work (e.g.  CEA) in this area. In fact, I also advocate for a separate data model that both UWS and ExecutionBroker can utilize for characterization.

Let's discuss this further with Dave.

Cheers,
Jesus


On 19/08/2024, 11:12, "grid on behalf of Paul Harrison via grid" <grid-bounces at ivoa.net <mailto:grid-bounces at ivoa.net> on behalf of grid at ivoa.net <mailto:grid at ivoa.net>> wrote:


Hi,


This has only recently come to my attention partially because ExecutionBroker started out life as executionPlanner and I had thought that it was covering a unique domain in the IVOA - however, since it morphed into ExecutionBroker is has acquired an ExecutionWorker interface, which appears to cover exactly the same domain as UWS. 


It is not clear what has driven the need to create a new ExecutionWorker interface, but it there are some features missing from UWS then I think that it would be better to create a UWS 2.0 rather than this new interface.


As an historical note the whole use case for the ExecutionBroker looks very much like what inspired the Astrogrid common execution architecture (CEA) https://www.ivoa.net/documents/Notes/CEA/CEADesignIVOANote-20050513.html <https://www.ivoa.net/documents/Notes/CEA/CEADesignIVOANote-20050513.html>, and amusingly the there is even a suggestion for something like an execution planner in https://www.ivoa.net/documents/Notes/CEA/CEADesignIVOANote-20050513.html#Future <https://www.ivoa.net/documents/Notes/CEA/CEADesignIVOANote-20050513.html#Future>. Perhaps there are some lessons that can be learned from all that time ago as CEA did undergo the transition from WSDL to REST successfully and so managed to successfully layer the business logic in an implementation independent way.


Paul.

The SKA Observatory is an inter-governmental organisation and the successor of SKA Organisation, a private limited company by guarantee registered in England and Wales with registered number 07881918, with a registered office of Jodrell Bank Observatory, Lower Withington, Macclesfield, Cheshire, England, SK11 9FT.


This message is intended solely for the addressee and may contain confidential information. If you have received this message in error, please inform the sender, and immediately and permanently delete the email. Do not use, copy or disclose the information contained in this message or in any attachment.


This email has been scanned for viruses and malware, and may have been automatically archived, by Mimecast Ltd. Although SKA Observatory and SKA Organisation have taken reasonable precautions to ensure no viruses are present in this email, neither SKA Observatory nor SKA Organisation accept responsibility for any loss or damage sustained as a result of computer viruses and the recipient must ensure that the email (and attachments) are virus free.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.ivoa.net/pipermail/grid/attachments/20240826/5e171928/attachment.htm>