EP030 Spack: a package manager for supercomputers FLOSS for Science

In episode 30, we interviewed Todd Gamblin from the Lawrence Livermore National Laboratory about the Spack project. We discussed his current research project along with his involvement in Spack. We widely discussed the philosophy of Spack, some usage patterns, its capabilities for managing package management in HPC clusters as well as standalone computers and which operating systems it supports at the moment. Todd shared with us his opinion on the trend for containerized workloads to achieve reproducible science and why it may not be the goal we need to set. He highlighted for us the similarities and differences between EasyBuild and Spack as well as the origin of those differences. We finished the interview with our usual quick questions.

00:00:00 Intro music
00:00:17 Introduction
00:00:36 Introducing Todd Gamblin
00:00:58 His current research topics
00:01:23 Spack as official duties
00:01:43 Spack usage at Lawrence Livermore National Laboratory
00:02:01 Other research projects
00:02:47 Profiling in HPC
00:04:24 His role as leader of software packaging technology for the exascale computing project
00:04:58 One-minute elevator pitch for Spack
00:05:34 Spack's usage philosophy compared to other package managers
00:06:59 Installation from source code or binary?
00:07:28 Spack's usage in the top500 super computers
00:07:49 Geographical distribution of users
00:08:18 Number of packages in the repo and some examples
00:09:05 Managing computer clusters with Spack's automation capabilities
00:11:04 Module files in Spack
00:12:32 Syntax of a Spack package file
00:13:43 Configuration of compiler flags
00:15:00 Importing python libraries in the Spack files
00:15:48 The procedure to submit a package
00:16:27 Review process for new packages
00:17:34 Reasons for rejection of Spack packages
00:18:01 Operating systems supported by Spack
00:18:23 WSL and Spack
00:18:58 Restricting packages to certain hardware and software configurations
00:20:04 Build testing and nightly builds
00:21:28 Working with containers in a Spack environment
00:22:25 Deploying prebuilt containers
00:23:05 About the "universality" of containers
00:24:16 His opinion on containerized applications for reproducible science
00:26:17 Spack's log file to document reproducibility
00:27:13 Reproducing older results
00:28:10 Specifying requirements on compilers
00:30:39 Post-installation verification test
00:31:10 Using Spack on a standalone computer instead of HPC systems
00:32:56 Differences between EasyBuild and Spack
00:34:24 EasyBuild in the top500
00:34:49 Transitionning between EasyBuild and Spack
00:35:38 Other alternatives
00:36:23 Using EasyBuild and Spack on the same system
00:38:36 When did the project start?
00:39:53 External contributions to Spack
00:40:53 How many core developpers?
00:41:30 Organization of the community and governance model
00:43:06 Who decides which package is accepted in the repo?
00:44:38 Spack's choice of software license
00:47:09 Todd's vision about the importance of FLOSS for the openness of science
00:48:13 Possible negative impacts of FLOSS
00:48:58 Most notable recent scientific discovery
00:49:14 Favourite text processing tool
00:49:25 A topic in science about which he recently changed his mind about
00:49:58 Anything else we forgot to ask?
00:50:09 How to contact Todd
00:50:46 Conclusion