Better Scientific Software
a tutorial presented at
Improving Scientific Software
on 9:00 am - 2:30 pm MDT (UTC-6) Tuesday 18 April 2023
Presenters: David E. Bernholdt (Oak Ridge National Laboratory), Patricia A. Grubel (Los Alamos National Laboratory), and David M. Rogers (Oak Ridge National Laboratory)
This page provides detailed information specific to the tutorial event above. Expect updates to this page up to, and perhaps shortly after, the date of the tutorial. Pages for other tutorial events can be accessed from the main page of this site.
Quick Links
- Playlist (YouTube)
- Presentation Slides (FigShare)
On this Page
- Description
- Agenda
- Presentation Slides
- Stay in Touch
- Resources from Presentations
- Requested Citation
- Acknowledgements
Description
Producing scientific software is a challenge. The high-performance modeling and simulation community, in particular, is dealing with the confluence of disruptive changes in computing architectures and new opportunities (and demands) for greatly improved simulation capabilities, especially through coupling physics and scales. At the same time, computational science and engineering (CSE), as well as other areas of science, are experiencing increasing focus on scientific reproducibility and software quality.
Computer architecture changes require new software design and implementation strategies, including significant refactoring of existing code. Reproducibility demands require more rigor across the entire software endeavor. Code coupling requires aggregate team interactions including integration of software processes and practices. These challenges demand large investments in scientific software development and improved practices. Focusing on improved developer productivity and software sustainability is both urgent and essential.
This half-day tutorial distills multi-project and multi-year experience from members of the IDEAS Productivity project and the creators of the BSSw.io community website. The tutorial will provide information about software practices, processes, and tools explicitly tailored for CSE. Topics to be covered include collaborative software development practices (Agile methodologies and tools, git workflows, peer code review), software design and refactoring, testing and continuous integration, and reproducibility. The material will be mostly at the beginner and intermediate levels. There will also be opportunities to discuss topics raised by the audience.
Agenda
| Time (MDT) | Title | Presenter |
|---|---|---|
| 9:00 AM | Introduction | David E. Bernholdt (ORNL) |
| 9:10 AM | Motivation and Overview of Best Practices in HPC Software Development | David E. Bernholdt (ORNL) |
| 9:35 AM | Scientific Software Design | David E. Bernholdt (ORNL) |
| 10:00 AM | Refactoring Scientific Software | David E. Bernholdt (ORNL) |
| 10:30 AM | Break | |
| 11:00 AM | Collaborative Software Development | Patricia A. Grubel (LANL) |
| 11:40 AM | Software Packaging - Condensed Version | David M. Rogers (ORNL) |
| 12:00 PM | Lunch break | |
| 1:00 PM | Testing Strategies - Condensed Version | David M. Rogers (ORNL) |
| 1:30 PM | Improving Reproducibility Through Better Software Practices | Patricia A. Grubel (LANL) |
| 2:15 PM | Summary | Patricia A. Grubel (LANL) |
| 2:30 PM | Adjourn |
Presentation Slides
The latest version of the slides will always be available at https://doi.org/10.6084/m9.figshare.22179748.
Note that these files may include additional slides that will not be discussed during the tutorial, but questions are welcome.
Stay in Touch
-
After the tutorial please feel free to email questions or feedback to the BSSw tutorial team at bssw-tutorial@lists.mcs.anl.gov.
-
To find out about future events organized by the IDEAS Productivity Project, you can subscribe to our mailing list (usually ~2 messages/month).
-
For monthly updates on the Better Scientific Software site, subscribe to our monthly digest.
Resources from Presentations
Links from the tutorial presentations are listed here for convenience
- Module 1: Introduction
- Module 2: Motivation and Overview of Best Practices in HPC Software Development
- Best Practices for Scientific Computing
- Good Enough Practices in Scientific Computing
- Linux Foundation Core Infrastructure Initiative (CII) Best Practices Badging Program
- Rate Your Project Assesment Tool
- Progress Tracking Card (PTC) Examples
- Productivity and Sustainability Improvement Planning
- Better Scientific Software (BSSw)
- Module 3: Scientific Software Design
- The Exascale Computing Project (ECP)
- Findings from the ECP Performance Portability Panel Series
- Performance Portability and the Exascale Computing Project
- Kokkos Lecture Series
- Related paper: A Design Proposal for a Next Generation Scientific Software Framework
- Related webinar: Software Design for Longevity with Performance Portability
- Module 4: Refactoring Scientific Software
- Module 5: Collaborative Software Development
- Design Patterns for Git Workflows
- Git Flow (Driessen’s Original Blog)
- GitHub Flow (previously linked to
scottchacon.com/2011/08/31/github-flow.html) - GitLab Flow (previously linked to
docs.gitlab.com/ee/topics/gitlab_flow.html) - Agile Manifesto
- How to code review in a Pull Request
- Investing in Code Reviews for Better Research Software (previously linked to
ideas-productivity.org/events/hpc-best-practices-webinars/#webinar068) - Testing and Code Review Practices in Research Software Development (previously linked to
ideas-productivity.org/events/hpc-best-practices-webinars/#webinar044) - Open Source Initiative
- Choose an Open Source License Tool
- Introduction to Software Licensing (previously linked to
ideas-productivity.org/events/hpc-best-practices-webinars/#webinar024) - Trilinos
- Open MPI
- FleCSI
- Module 6: Software Packaging - Condensed Version
- Simple Heat Equation Example
- https://github.com/frobnitzem/lib0
- https://code.ornl.gov/99R/mpi-test
- https://cmake.org/cmake/help/git-stage/manual/cmake-packages.7.html#creating-packages
- https://supercontainers.github.io/sc20-tutorial/
- https://fluid-run.readthedocs.io/en/latest/HowTo/setup_your_repo.html
- https://fastapi.tiangolo.com/deployment/docker/#build-a-docker-image-for-fastapi
- https://supercontainers.github.io/sc20-tutorial/02.docker/index.html
- https://cloud.sylabs.io/builder
- Article on CI team practices
- https://pyscaffold.org/
- https://python-poetry.org/docs/pyproject
- Fortran resources
- https://spack.readthedocs.io
- https://github.com/mpbelhorn/olcf-spack-environments/blob/develop/hosts/frontier/envs/base/spack.yaml
- https://spack.readthedocs.io/en/latest/packaging_guide.html#dependency-specs
- Intermediate example: C++ with spack
- https://spack-tutorial.readthedocs.io/en/latest/tutorial_packaging.html
- https://spack.readthedocs.io/en/latest/spack.util.html#module-spack.util.prefix
- https://spack.readthedocs.io/en/latest/packaging_guide.html#accessing-dependencies
- DCA++
- https://github.com/pyscf/extension-template
- ZFP
- https://github.com/ECP-copa/Cabana
- https://cmake.org/cmake/help/latest/guide/tutorial/index.html
- https://cmake.org/cmake/help/latest/command/add_test.html
- Module 7: Testing Strategies - Condensed Version
- A Holistic Algorithmic Approach to Improving Accuracy, Robustness, and Computational Efficiency for Atmospheric Dynamics
- Useful resources on testing (formerly linked to
ideas-productivity.org/resources/howtos/) - Lcov (formerly linked to
ltp.sourceforge.net/coverage/lcov.php) - CI/CD Introduction
- Joint Center for Satellite Data Assimilation (JEDI) documentation
- https://github.com/CompFUSE/DCA
- https://docs.docker.com/build/ci/
- https://spack.readthedocs.io/en/latest/containers.html
- https://supercontainers.github.io/sc20-tutorial/07.spack/index.html
- https://docs.github.com/en/actions/using-jobs/running-jobs-in-a-container
- Understanding the risk of script injections
- https://github.com/ECP-WarpX/WarpX
- https://cristianadam.eu/20200113/speeding-up-c-plus-plus-github-actions-using-ccache/
- Hints from the front lines
- Good ideas and idioms from across developer spaces
- DCA++: A software framework to solve correlated electron problems with modern quantum cluster methods
- Ginkgo: A Modern Linear Operator Algebra Framework for High Performance Computing
- https://warpx.readthedocs.io/
- An Automated Performance Evaluation Framework for the GINKGO Software Ecosystem
- Hints from the front lines
- Team Experiences Implementing Continuous Integration
- Module 8: Improving Reproducibility Through Better Software Practices
- Toward a Compatible Reproducibility Taxonomy for Computational and Computing Sciences
- Reproducibility and Replicability in Science
- Many Psychology Findings Not As Strong As Claimed
- The War Over Supercooled Water
- Researchers find bug in Python Script may have affected hundreds of studies
- National Science Foundation Data Management Plan Requirements
- Findable, Accessible, Interoperable, Re-usable
- FAIR Data Principles US
- SC23 Reproducibility Initiative
- ACM Transactions on Mathematical Software (TOMS)
- ACM Artifact Review and Badging
- http://fursin.net/reproducibility.html
- National Information Standards Organization (NISO) on Reproducibility and Badging
- Helpful Tools
- Floating Point Analysis Tools
- Code Ocean (Cloud platforms - publish and reproduce research code and data)
- DOIs and hosting of data, code, documents:
- Other Resources:
- The FAIR Guiding Principles for Scientific Data Management and Stewardship. Mark D. Wilkinson, et al. 2016
- FAIR4RS (previously linked to
www.rd-alliance.org/groups/fair-research-software-fair4rs-wg) - Editorial: ACM TOMS Replicated Computational Results Initiative. Michael A. Heroux. 2015
- Enhancing Reproducibility for Computational Methods
- Simple experiments in reproducibility and technical trust by Mike Heroux and students (work in progress)
- What every scientist should know about floating-point arithmetic. David Goldberg.
- Module 9: Summary
Requested Citation
The requested citation the overall tutorial is:
David E. Bernholdt, Patricia A. Grubel, and David M. Rogers, Better Scientific Software tutorial, in Improving Scientific Software, Boulder, Colorado and online, 2023. DOI: 10.6084/m9.figshare.22179748.
Individual modules may be cited as Speaker, Module Title, in Better Scientific Software tutorial…
Acknowledgements
This tutorial is produced by the IDEAS Productivity project.
This work was supported by the U.S. Department of Energy Office of Science, Office of Advanced Scientific Computing Research (ASCR), and by the Exascale Computing Project (17-SC-20-SC), a collaborative effort of the U.S. Department of Energy Office of Science and the National Nuclear Security Administration.