HPC Carpentry
Steyn, Jannetta; Hodges, Toby; Reid, Andrew; Rasel, Annajiat Alim; Muite, Benson; O'Cais, Alan; Keller, Trevor; Purwanto, Wirawan
- Publisher
- Zenodo
- Language
- en
Abstract
The HPC Carpentry project aims to develop highly interactive workshop training materials to empower novices to effectively leverage HPC to solve scientific and technical problems in their domains. This talk provides an overview of the project and its history, explores the content and delivery of HPC Carpentry workshops, and summarises the community's plans for the future. Modeled after The Carpentries training programs, the project’s goal is to develop foundational HPC skills and a sense of empowerment, rather than expertise. The workshop setting provides learners with hands-on experience that elicits confidence working with HPC systems and provides sufficient vocabulary to make subsequent self-study more effective.In a major milestone, the steering committee is leading HPC Carpentry through the formal incubation process to become an official lesson program of The Carpentries, a global community teaching essential data and software skills for research. If adopted, HPC Carpentry will join the existing Software, Data, and Library Carpentry programs. This achievement is the product of significant work over the past several years, incorporating valuable materials from many contributors.Our most recent focus has been developing materials for a user workshop. We begin with an introduction to the command-line shell (using Software Carpentry’s Unix Shell lesson), followed by our Introduction to HPC lesson, covering remote access and resource management. We end with a newly developed lesson on HPC workflow management, which walks learners through the execution of a scaling study on an HPC system, emphasizing both the benefits and limitations of the system for domain applications. This workshop program was recently run in full at the Lawrence Livermore National Laboratory.Future plans include building a developer workshop, reconnecting with disparate contributors, and engaging with the broader community through regular open conference calls and outreach.Acknowledgements The work described in the talk was made possible by the contributions of every HPC Carpentry community member. Thank you! The abstract was adapted from a previous conference submission, prepared by Andrew Reid, Trevor Keller, Alan O'Cais, Annajiat Alim Rasel, Wirawan Purwanto, Jane Herriman, and Benson Muite.A recording of this session is available on YouTube: https://youtu.be/r590gM2kZDc
Full text
1/15 HPC Carpentry An Overview of HPC Carpentry’s Activities Jannetta Steyn, Newcastle University, UK Toby Hodges, Andrew Reid, Annajiat Alim Rasel, Benson Muite, Trevor Keller, Wirawan Purwanto September 09, 2025
2/15 HPC means we have choices 1. Hardwood 2. Plywood 3. Chip board
3/15 But that has nothing to do with anything ... 1. Hardwood 2. Plywood 3. Chip board
4/15 Overview 1. What is The Carpentries 2. A Very Brief History of The Carpentries 3. Where does HPC Carpentry fit? 4. Current Status 5. Roadmap and Challenges 6. The mini version 7. Comms Channels
5/15 A Very Brief History of The Carpentries ▶It all started with this guy –> ▶... in 1998 Dr Greg Wilson https://carleton.ca/scs/?p=14196
6/15 This is what Greg said about Software Carpentry We are lab skills for scientific computing. This project started because I was working with physicists and astronomers using first generation and parallel computers. And they would come into my office with their 50,000 lines of Fortran and ask me to make it a zillion times faster. And they had never heard of version control and weren’t really sure why they should be writing functions because nobody every taught them. And it is unfair to look down on people if they are not doing things right if you have never shown them how. https://www.youtube.com/watch?v=FtKO619O5g0
7/15 Where does HPC Carpentry fit? 1. Knowing about version control and functions does not quite solve all problems. 2. Researchers with HPC requirements do not necessarily know how to effectively scale their applications 3. HPC facility operators become frustrated by non-knowledgable users making use of shared resources. 4. Candidate solution: Use Carpentries techniques.
8/15 How did it all begin ▶Earliest commit to hpc-carpentry GitHub 2013 by Compute Canada ▶Peter Steinbach blog post (2017) - HPC in a day ▶CarpentryCon 2018, 2020, 2022 ▶Super Computing BoF 17, 18, 19, 21 ▶Winter 2021, accepted into Carpentries Incubator ▶Begin working towards Lesson Program Incubation in 2021 and formally accepted to the Lesson Program Incubation in June 2023 ▶April 2024, workflow lessons enters testing
9/15 Where are we now? 1. We are doing things In the Carpentries Way ▶two day setting ▶students type along with instructors ▶get feedback 2. Two main takeaways: ▶muscle memory ▶vocabulary 3. Not experts after two days but: ▶know what answer looks like ▶know how to find answers
