Better Scientific Software
a tutorial presented at
The International Conference for High-Performance Computing, Networking, Storage, and Analysis (SC22)
on 1:30 pm - 5:00 pm CST Sunday 13 November 2022
Presenters: David E. Bernholdt (Oak Ridge National Laboratory), Anshu Dubey (Argonne National Laboratory), and Patricia A. Grubel (Los Alamos National Laboratory)
This page provides detailed information specific to the tutorial event above. Expect updates to this page up to, and perhaps shortly after, the date of the tutorial. Pages for other tutorial events can be accessed from the main page of this site.
Quick Links
- Presentation Slides (FigShare)
On this Page
- Description
- Agenda
- Presentation Slides
- Other Software-Related Events at SC22
- Stay in Touch
- Resources from Presentations
- Requested Citation
- Acknowledgements
Description
The computational science and engineering (CSE) community is in the midst of an extremely challenging period created by the confluence of disruptive changes in computing architectures, demand for greater scientific reproducibility, sustainability, and quality, and new opportunities for greatly improved simulation capabilities, especially through coupling physics and scales. Computer architecture advances require new software design and implementation strategies, including significant refactoring of existing code. Reproducibility requires more rigor across the entire software endeavor. Code coupling requires aggregate team interactions including integration of software processes and practices. These challenges demand large investments in scientific software development and improved practices. Focusing on improved developer productivity and software sustainability is both urgent and essential.
This tutorial will provide information and hands-on experience with software practices, processes, and tools explicitly tailored for CSE and HPC. Goals are improving the productivity of those who develop CSE software and increasing the sustainability of software artifacts. We discuss practices that are relevant for projects of all sizes, with emphasis on small teams and teams of teams. Topics include the software processes for (small) teams, including agile processes, collaboration via version control workflows, reproducibility, and scientific software design, refactoring, and testing (including test design strategies and continuous integration).
Agenda
| Time (CDT) | Title | Presenter |
|---|---|---|
| 1:30 PM | Introduction | David E. Bernholdt (ORNL) |
| 1:40 PM | Motivation and Overview of Best Practices in HPC Software Development | David E. Bernholdt (ORNL) |
| 2:10 PM | Scientific Software Design | Anshu Dubey (ANL) |
| 2:30 PM | Collaborative Software Development | Patricia A. Grubel (LANL) |
| 3:00 PM | Break | |
| 3:30 PM | Improving Reproducibility Through Better Software Practices | Patricia A. Grubel (LANL) |
| 4:10 PM | Software Testing and Verification | Anshu Dubey (ANL) |
| 4:55 PM | Summary | David E. Bernholdt (ORNL) |
| 5:00 PM | Adjourn | |
| 5:00 PM | Optional Additional Q&A | All |
| 5:30 PM | Adjourn |
Presentation Slides
The latest version of the slides will always be available at https://doi.org/10.6084/m9.figshare.21384057.
Note that these files may include additional slides that will not be discussed during the tutorial, but questions are welcome.
Other Software-Related Events at SC22
If you’re interested in this tutorial, you might be interested in this list of other software-related events taking place in the SC22 conference.
Stay in Touch
-
After the tutorial please feel free to email questions or feedback to the BSSw tutorial team at bssw-tutorial@lists.mcs.anl.gov.
-
To find out about future events organized by the IDEAS Productivity Project, you can subscribe to our mailing list (usually ~2 messages/month).
-
For monthly updates on the Better Scientific Software site, subscribe to our monthly digest.
Resources from Presentations
Links from the tutorial presentations are listed here for convenience
- Module 1: Introduction
- Module 2: Motivation and Overview of Best Practices in HPC Software Development
- COVID-19 epidemiology saga
- https://doi.org/10.25561/77482
- https://www.nicholaslewis.org/imperial-college-uk-covid-19-numbers-dont-seem-to-add-up/
- https://www.nature.com/articles/d41586-020-01003-6
- https://www.foxnews.com/world/imperial-college-britain-coronavirus-lockdown-buggy-mess-unreliable
- https://www.telegraph.co.uk/technology/2020/05/16/coding-led-lockdown-totally-unreliable-buggy-mess-say-experts/
- https://github.com/mrc-ide/covid-sim/
- https://philbull.wordpress.com/2020/05/10/why-you-can-ignore-reviews-of-scientific-code-by-commercial-software-developers/amp/
- http://doi.org/10.5281/zenodo.3865491
- Best Practices for Scientific Computing
- Good Enough Practices in Scientific Computing
- Linux Foundation Core Infrastructure Initiative (CII) Best Practices Badging Program
- Rate Your Project Assesment Tool
- Progress Tracking Card (PTC) Examples
- Productivity and Sustainability Improvement Planning
- Better Scientific Software (BSSw)
- COVID-19 epidemiology saga
- Module 3: Scientific Software Design
- The Exascale Computing Project (ECP)
- Findings from the ECP Performance Portability Panel Series
- Performance Portability and the Exascale Computing Project
- Kokkos Lecture Series
- Related paper: A Design Proposal for a Next Generation Scientific Software Framework
- Related webinar: Software Design for Longevity with Performance Portability
- Module 4: Collaborative Software Development
- Design Patterns for Git Workflows
- Git Flow (Driessen’s Original Blog)
- GitHub Flow (previously linked to
scottchacon.com/2011/08/31/github-flow.html) - GitLab Flow (previously linked to
docs.gitlab.com/ee/topics/gitlab_flow.html) - Agile Manifesto
- How to code review in a Pull Request
- Investing in Code Reviews for Better Research Software (previously linked to
ideas-productivity.org/events/hpc-best-practices-webinars/#webinar068) - Testing and Code Review Practices in Research Software Development (previously linked to
ideas-productivity.org/events/hpc-best-practices-webinars/#webinar044) - Open Source Initiative
- https://choosealicense.com/
- Introduction to Software Licensing (previously linked to
ideas-productivity.org/events/hpc-best-practices-webinars/#webinar024) - Trilinos
- Open MPI
- FleCSI
- Module 5: Improving Reproducibility Through Better Software Practices
- Motivations and Background:
- Definitions, Guidelines, and Organizations:
- National Science Foundation Data Management Plan Requirements
- Findable, Accessible, Interoperable, Re-usable
- https://gofair.us/
- SC22 Reproducibility Initiative
- ACM Transactions on Mathematical Software (TOMS)
- ACM Artifact Review and Badging
- http://fursin.net/reproducibility.html
- National Information Standards Organization (NISO) on Reproducibility and Badging
- Helpful Tools
- Floating Point Analysis Tools
- Code Ocean (Cloud platforms - publish and reproduce research code and data)
- DOIs and hosting of data, code, documents:
- Other Resources:
- The FAIR Guiding Principles for Scientific Data Management and Stewardship. Mark D. Wilkinson, et al. 2016
- FAIR4RS (previously linked to
www.rd-alliance.org/groups/fair-research-software-fair4rs-wg) - Editorial: ACM TOMS Replicated Computational Results Initiative. Michael A. Heroux. 2015
- Enhancing Reproducibility for Computational Methods
- Simple experiments in reproducibility and technical trust by Mike Heroux and students (work in progress)
- What every scientist should know about floating-point arithmetic. David Goldberg.
- Module 6: Software Testing and Verification
- CI introduction
- Useful resources on testing (formerly linked to
ideas-productivity.org/resources/howtos/) - Lcov (formerly linked to
ltp.sourceforge.net/coverage/lcov.php) - Additional resources:
- Module 7: Summary
Requested Citation
The requested citation the overall tutorial is:
David E. Bernholdt, Anshu Dubey, and Patricia A. Grubel, Better Scientific Software tutorial, in The International Conference for High-Performance Computing, Networking, Storage, and Analysis (SC22), Dallas, Texas, and online, 2022. DOI: 10.6084/m9.figshare.21384057.
Individual modules may be cited as Speaker, Module Title, in Better Scientific Software tutorial…
Acknowledgements
This tutorial is produced by the IDEAS Productivity project.
This work was supported by the U.S. Department of Energy Office of Science, Office of Advanced Scientific Computing Research (ASCR), and by the Exascale Computing Project (17-SC-20-SC), a collaborative effort of the U.S. Department of Energy Office of Science and the National Nuclear Security Administration.