Science Platforms Workshop Overview

[Brian Major]

Dear grid,

At the end of February, a "Science Platforms" workshop was held at STScI in
Baltimore.  It served as a follow-up to the well attended BoF on Science
Platforms at held in October at ADASS in Santiago.  This workshop was a
fruitful 2.5 day informal gathering of (predominantly) US-based data
centres and projects to share with each other groups' experiences in
building cloud-based science platforms.  Thank you to Arfon Smith and the
rest of the organizers for putting this on.

Here are the slides from the presentation:


https://stsci.app.box.com/s/65e6suh52b9swr2mg63k6t3qphnnbnrj/folder/47579869370

Detailed meeting notes of the breakaway sessions can be found on the
dedicated GitHub site for the workshop:

    https://github.com/spacetelescope/science-platforms-workshop

Some of my observations/notes:

    - The projects and groups in attendance are converging towards a common
set of technologies and architecture to support interactive data analysis.
The list is not surprising:  a notebook environment (Jupyter / JupyterLab),
a notebook spawner (Jupyter Hub), a container environment (Docker), a
container orchestration environment (Kubernetes).  Data discovery from the
notebooks though TAP or a similar query service.  Data is made available
through an abstraction layer to distributed file systems and/or network
storage.

    - A reference architecture reflecting these technologies was discussed.

    - The mechanism for offering and operating batch processing services is
less clear.

    - Kubernetes is emerging as a must-have technology for operating
container based platforms.  However, it probably won't be something that
users interact with directly.

    - There were some very good discussions involving the VO:
        -
https://github.com/spacetelescope/science-platforms-workshop/blob/master/notes/VO-Astropy-Integration-Splinter.md
        -
https://github.com/spacetelescope/science-platforms-workshop/blob/master/notes/interoperability-vs-collaboration.md

    - Software is taking a front row seat in astronomy and these platforms
are helping in many ways to enable software science reproducibility.

    - Groups are keen on collaborating, whether it be through open source
software and standards.

At the IVOA meeting to be held in Victoria at the end of May there will be
a session dedicated to the topic of Science Platforms.  I encourage you to
use this opportunity to share your implementation experiences as the GWS
working group looks for ways to integrate these platforms into the
astronomy data community.

Cheers,
Brian


Brian Major

Canadian Astronomy Data Centre
National Research Council Canada
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.ivoa.net/pipermail/grid/attachments/20180309/fadd5a16/attachment.html>