Developing a Testing and Continuous Integration Strategy for your Team
a tutorial presented at
Exascale Computing Project Annual Meeting
on 2:30 pm - 4:00 pm EDT Monday 12 April 2021
Presenters: David E. Bernholdt (Oak Ridge National Laboratory), Patricia A. Grubel (Los Alamos National Laboratory), and James M. Willenbring (Sandia National Laboratories)
This page provides detailed information specific to the tutorial event above. Expect updates to this page up to, and perhaps shortly after, the date of the tutorial. Pages for other tutorial events can be accessed from the main page of this site.
- Recordings (Youtube playlist)
- Presentation Slides (FigShare)
On this Page
- Presentation Slides
- How to Participate
- Related ECP AM Events
- Stay in Touch
- Resources from Presentations
- Requested Citation
A thorough and robust testing regime is central to the productive development, evolution, and maintenance of quality, trustworthy scientific software. Continuous integration, though much discussed, is just one element of such a testing regime. Most project teams feel that they could (and should) do a “better job” of testing. In many cases, designing and implementing a strong testing strategy can seem so daunting that it is hard to know where to start.
In this tutorial, which is aimed at those with beginner to intermediate levels of comfort with testing and continuous integration, we will briefly review the multiple motivations for testing, and the different types of tests that address them. We’ll discuss some strategies for testing complex software systems, and how continuous integration testing fits into the larger picture. Accompanying hands-on activities, available for self-study, will demonstrate how to get started with a very simple level of CI testing.
|2:30pm-2:35pm||00||Introduction||David E. Bernholdt, ORNL|
|2:35pm-2:40pm||01||Motivation and Overview||Patricia A. Grubel, LANL|
|2:40pm-3:00pm||02||Software Testing 1||Patricia A. Grubel, LANL|
|3:00pm-3:25pm||03||Software Testing 2||David E. Bernholdt, ORNL|
|3:25pm-3:55pm||04||Continuous Integration||James M. Willenbring, SNL|
|3:55pm-4:00pm||05||Summary||James M. Willenbring, SNL|
The latest version of the slides will always be available at https://doi.org/10.6084/m9.figshare.14376956.
Note that these files may include additional slides that will not be discussed during the tutorial, but questions are welcome.
How to Participate
Please remember to stay muted except when asking questions, to limit the extraneous noise.
Please use the Zoom chat to ask questions at any time. We’ll try to respond to clarifications promptly. As time permits, we’ll take questions at the ends of presentations through chat (including responding to earlier questions) or verbally.
Related ECP AM Events
You may be interested in other sessions at this year’s ECP Annual Meeting related to the IDEAS project, software process improvement, testing, and continuous integration.
- IDEAS Desk – informal conversations about developer productivity and software sustainability
- Room 125 In the Gather.Town Side Meetings area
- 12:30pm-2:00pm Monday through Friday
- Validation, Verification and Performance Suites, Proxy Apps, and Continuous Integration for Exascale Supercomputers
- 10:00am-11:30am Wednesday
- Advancing Scientific Productivity through Better Scientific Software
- 11:30am-12:30am Wednesday
- Hands-on with Progress Tracking Cards: A Lightweight Method for Improving Your Software Practices
- 1:00pm-3:30pm Wednesday
- Benefiting from ECP CI
- 2:30pm-3:30pm Wednesday
- ECP CI Startup Tutorial
- 2:30pm-6:00pm Friday
Stay in Touch
After the tutorial please feel free to email questions or feedback to the BSSw tutorial team at email@example.com.
If you want to do the hands-on exercises, we’re happy to provide feedback on your pull requests and issues, even after the end of the tutorial.
To find out about future events organized by the IDEAS Productivity Project, you can subscribe to our mailing list (usually ~2 messages/month).
For monthly updates on the Better Scientific Software site, subscribe to our monthly digest.
Resources from Presentations
These are the links included in the tutorial presentations, included here for easier access
- Module 0: Introduction
- Module 1: Motivation and Overview of Best Practices in HPC Software Development
- Module 2: Software Testing 1
- Python Build and Test Framework: pyscaffold.org
- Build-Link-Test CMake Framework: llnl-blt.readthedocs.io
- Static Source Analysis (C++): clang-tidy
- Static Source Analysis (python): flake8 and pylint (updated 2022-03-31 due to dead link)
- Code Coverage Webservices: codecov and coveralls
- Tutorials for code coverage: Online Tutorial, Another example
- Development Practices Survey Article
- Module 3: Software Testing 2
- Useful How-to resources on test and test suites on ideas-productivity.org
- Related Articles: 1, 2
- Module 4: Continuous Integration
- Module 5: Summary
The requested citation the overall tutorial is:
David E. Bernholdt, Patricia A. Grubel, and James M. Willenbring, Developing a Testing and Continuous Integration Strategy for your Team tutorial, in Exascale Computing Project Annual Meeting, online, 2021. DOI: 10.6084/m9.figshare.14376956.
Individual modules may be cited as Speaker, Module Title, in Developing a Testing and Continuous Integration Strategy for your Team tutorial…
This tutorial is produced by the IDEAS Productivity project.
This work was supported by the U.S. Department of Energy Office of Science, Office of Advanced Scientific Computing Research (ASCR), and by the Exascale Computing Project (17-SC-20-SC), a collaborative effort of the U.S. Department of Energy Office of Science and the National Nuclear Security Administration.