Science Platforms Workshop Overview

[Simon O'Toole]

Hi Brian,

Thanks for sending these around, it looks like a lot of great ground was covered!

Unfortunately I won’t be able to make it to Victoria in May, but should be in Washington (well, Maryland) in October.

Cheers,
Simon


Simon O'Toole
AAO Data Central Project Scientist
Australian Astronomical Observatory
105 Delhi Rd, North Ryde NSW 2113



> On 10 Mar 2018, at 10:55 am, Brian Major <major.brian at gmail.com> wrote:
> 
> Dear grid,
> 
> At the end of February, a "Science Platforms" workshop was held at STScI in Baltimore.  It served as a follow-up to the well attended BoF on Science Platforms at held in October at ADASS in Santiago.  This workshop was a fruitful 2.5 day informal gathering of (predominantly) US-based data centres and projects to share with each other groups' experiences in building cloud-based science platforms.  Thank you to Arfon Smith and the rest of the organizers for putting this on.
> 
> Here are the slides from the presentation:
> 
>     https://stsci.app.box.com/s/65e6suh52b9swr2mg63k6t3qphnnbnrj/folder/47579869370 <https://stsci.app.box.com/s/65e6suh52b9swr2mg63k6t3qphnnbnrj/folder/47579869370>
> 
> Detailed meeting notes of the breakaway sessions can be found on the dedicated GitHub site for the workshop:
> 
>     https://github.com/spacetelescope/science-platforms-workshop <https://github.com/spacetelescope/science-platforms-workshop>
> 
> Some of my observations/notes:
> 
>     - The projects and groups in attendance are converging towards a common set of technologies and architecture to support interactive data analysis.  The list is not surprising:  a notebook environment (Jupyter / JupyterLab), a notebook spawner (Jupyter Hub), a container environment (Docker), a container orchestration environment (Kubernetes).  Data discovery from the notebooks though TAP or a similar query service.  Data is made available through an abstraction layer to distributed file systems and/or network storage.
> 
>     - A reference architecture reflecting these technologies was discussed.
> 
>     - The mechanism for offering and operating batch processing services is less clear.
> 
>     - Kubernetes is emerging as a must-have technology for operating container based platforms.  However, it probably won't be something that users interact with directly.
> 
>     - There were some very good discussions involving the VO:
>         - https://github.com/spacetelescope/science-platforms-workshop/blob/master/notes/VO-Astropy-Integration-Splinter.md <https://github.com/spacetelescope/science-platforms-workshop/blob/master/notes/VO-Astropy-Integration-Splinter.md>
>         - https://github.com/spacetelescope/science-platforms-workshop/blob/master/notes/interoperability-vs-collaboration.md <https://github.com/spacetelescope/science-platforms-workshop/blob/master/notes/interoperability-vs-collaboration.md>
> 
>     - Software is taking a front row seat in astronomy and these platforms are helping in many ways to enable software science reproducibility.
> 
>     - Groups are keen on collaborating, whether it be through open source software and standards.
> 
> At the IVOA meeting to be held in Victoria at the end of May there will be a session dedicated to the topic of Science Platforms.  I encourage you to use this opportunity to share your implementation experiences as the GWS working group looks for ways to integrate these platforms into the astronomy data community.
> 
> Cheers,
> Brian
> 
> 
> Brian Major
> 
> Canadian Astronomy Data Centre
> National Research Council Canada
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.ivoa.net/pipermail/grid/attachments/20180310/64722584/attachment.html>