Minutes of 20/09/23 IVOA GWS Meeting - Science Platforms Federated Execution

[Salgado, Jesus]

Sure!
We only have 1.5 hours session in the IVOA interop meeting (probably short taking into account the complexity of the issue) and, also, this is a subject important for many of the ADASS participants so good to have this BoF to have direct discussions.

Thanks for that!
Jesus

[SKAO Horizontal Dividing Rule]

[SKAO Logo]

[SKAO Vertical Dividing Rule]

Jesús Salgado
SKA Regional Centre Architect
jesus.salgado at skao.int<mailto:jesus.salgado at skao.int>
www.skao.int<https://www.skao.int/>
SKA Observatory
Jodrell Bank, Lower Withington,
Macclesfield, SK11 9FT, UK



From: Giuliano Taffoni <giuliano.taffoni at inaf.it>
Date: Friday, 22 September 2023 at 16:32
To: Jesus Salgado <jesus.salgado at skao.int>
Cc: "grid at ivoa.net" <grid at ivoa.net>, src-olive <src-olive at skao.int>, src-orange <src-orange at skao.int>, src-magenta <src-magenta at skao.int>, "Bertocco, Sara" <sara.bertocco at inaf.it>, Dave Morris <dave.morris at metagrid.co.uk>
Subject: Re: Minutes of 20/09/23 IVOA GWS Meeting - Science Platforms Federated Execution

Thanks Jesus, Dave and Sara
I read the minutes, unfortunately I was not able to attend.
I would like to mention that Claudio and I organised a BoF at ADASS on sciplatforms.
It was accepted a couple of days ago.
Maybe we could profit of the BoF (and eventually on one session on interop) to continue the discussion in person in particular for what regards the evolution of the executionplanner :-)
Regards
G.




On 21 Sep 2023, at 12:25, Salgado, Jesus <jesus.salgado at skao.int> wrote:

Dear all,

Thanks for the big attendance and the productive discussion during yesterday’s GWS meeting on Federated Execution across astronomical science platforms kick-off meeting.

Minutes of the meeting are at:
https://wiki.ivoa.net/twiki/bin/view/IVOA/GWSTelecon20230920

It was presented a summary of the issues to be address and Dave Morris presented the status of the Execution Planner, the first standard we want to promote within the team.

As this was a kick-off session, we have identified some points that could be further discussed in the mailing list or during next interop session in Tucson.


  *   Are we replicating work already solved by the Grid community?
Our initial answer is that the grid (or later cloud communities) has some agreed standards that we need to study to introduce concepts but that are not usually software stack agnostics. In the same way we defined UWS (what is a job scheduler when many job schedulers are present in many other contexts), the abstraction layer of the Execution Planner could be implemented with different software stacks below. In any case, of course we need to take attention to other standard ways to solve these problems.


  *   Status of Execution Planner and missing points?
Note is quite complete. It must be addressed the data model to characterise capabilities of the platforms and characterisation of the workflows (in the general sense). This data model could be part of the Execution Planner standard (not a separate document) and, also, it should address data location (this is an open issue)


  *   Why not to create a layer on top of orchestrators? (Most of science platforms already have, e.g. a Kubernetes cluster)
Although it is true that some technologies are quite standard for most of the science platforms, technologies could change, and older platforms could have different set-ups. This is why IVOA protocols are usually software stack agnostics so they could be implemented by all. In any case, it is important to know the level of heterogeneity of the different partners to identify if some things could be simplified assuming that compatible technologies.


  *   Could IVOA endorse or accept S3 as a data transfer protocol?
Not clear answer. Interesting to be discussed in the mailing list.


  *   Next steps?
The more important step is to identify who is willing to help on the prototyping as, in this particular case, the prototyping will clarify a lot about possible missing points on the execution planner definition.

Best Regards,
Jesus Salgado, Sara Bertocco and Dave Morris

<image001.jpg>

<image002.jpg>

<image003.jpg>

Jesús Salgado
SKA Regional Centre Architect
jesus.salgado at skao.int<mailto:jesus.salgado at skao.int>
www.skao.int<https://www.skao.int/>
SKA Observatory
Jodrell Bank, Lower Withington,
Macclesfield, SK11 9FT, UK




The SKA Observatory is an inter-governmental organisation and the successor of SKA Organisation, a private limited company by guarantee registered in England and Wales with registered number 07881918, with a registered office of Jodrell Bank, Lower Withington, Macclesfield, Cheshire, England, SK11 9FT.

This message is intended solely for the addressee and may contain confidential information. If you have received this message in error, please inform the sender, and immediately and permanently delete the email. Do not use, copy or disclose the information contained in this message or in any attachment.

This email has been scanned for viruses and malware, and may have been automatically archived, by Mimecast Ltd. Although SKA Observatory and SKA Organisation have taken reasonable precautions to ensure no viruses are present in this email, neither SKA Observatory nor SKA Organisation accept responsibility for any loss or damage sustained as a result of computer viruses and the recipient must ensure that the email (and attachments) are virus free.

The SKA Observatory is an inter-governmental organisation and the successor of SKA Organisation, a private limited company by guarantee registered in England and Wales with registered number 07881918, with a registered office of Jodrell Bank Observatory, Lower Withington, Macclesfield, Cheshire, England, SK11 9FT.


This message is intended solely for the addressee and may contain confidential information. If you have received this message in error, please inform the sender, and immediately and permanently delete the email. Do not use, copy or disclose the information contained in this message or in any attachment.


This email has been scanned for viruses and malware, and may have been automatically archived, by Mimecast Ltd. Although SKA Observatory and SKA Organisation have taken reasonable precautions to ensure no viruses are present in this email, neither SKA Observatory nor SKA Organisation accept responsibility for any loss or damage sustained as a result of computer viruses and the recipient must ensure that the email (and attachments) are virus free.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.ivoa.net/pipermail/grid/attachments/20230925/d1427bad/attachment-0001.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.jpg
Type: image/jpeg
Size: 2274 bytes
Desc: image001.jpg
URL: <http://mail.ivoa.net/pipermail/grid/attachments/20230925/d1427bad/attachment-0003.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image002.jpg
Type: image/jpeg
Size: 7129 bytes
Desc: image002.jpg
URL: <http://mail.ivoa.net/pipermail/grid/attachments/20230925/d1427bad/attachment-0004.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image003.jpg
Type: image/jpeg
Size: 1650 bytes
Desc: image003.jpg
URL: <http://mail.ivoa.net/pipermail/grid/attachments/20230925/d1427bad/attachment-0005.jpg>