Developing a Testing and Continuous Integration Strategy for your Team

a tutorial presented at

Exascale Computing Project Annual Meeting

on 2:30 pm - 4:00 pm EDT Monday 12 April 2021

Presenters: David E. Bernholdt (Oak Ridge National Laboratory), Patricia A. Grubel (Los Alamos National Laboratory), and James M. Willenbring (Sandia National Laboratories)

This page provides detailed information specific to the tutorial event above. Expect updates to this page up to, and perhaps shortly after, the date of the tutorial. Pages for other tutorial events can be accessed from the main page of this site.

Quick Links

On this Page


A thorough and robust testing regime is central to the productive development, evolution, and maintenance of quality, trustworthy scientific software. Continuous integration, though much discussed, is just one element of such a testing regime. Most project teams feel that they could (and should) do a “better job” of testing. In many cases, designing and implementing a strong testing strategy can seem so daunting that it is hard to know where to start.

In this tutorial, which is aimed at those with beginner to intermediate levels of comfort with testing and continuous integration, we will briefly review the multiple motivations for testing, and the different types of tests that address them. We’ll discuss some strategies for testing complex software systems, and how continuous integration testing fits into the larger picture. Accompanying hands-on activities, available for self-study, will demonstrate how to get started with a very simple level of CI testing.


Time (EDT) Module Topic Speaker
2:30pm-2:35pm 00 Introduction David E. Bernholdt, ORNL
2:35pm-2:40pm 01 Motivation and Overview Patricia A. Grubel, LANL
2:40pm-3:00pm 02 Software Testing 1 Patricia A. Grubel, LANL
3:00pm-3:25pm 03 Software Testing 2 David E. Bernholdt, ORNL
3:25pm-3:55pm 04 Continuous Integration James M. Willenbring, SNL
3:55pm-4:00pm 05 Summary James M. Willenbring, SNL

Presentation Slides

The latest version of the slides will always be available at

Note that these files may include additional slides that will not be discussed during the tutorial, but questions are welcome.

How to Participate

You may be interested in other sessions at this year’s ECP Annual Meeting related to the IDEAS project, software process improvement, testing, and continuous integration.

Stay in Touch

Resources from Presentations

These are the links included in the tutorial presentations, included here for easier access

Requested Citation

The requested citation the overall tutorial is:

David E. Bernholdt, Patricia A. Grubel, and James M. Willenbring, Developing a Testing and Continuous Integration Strategy for your Team tutorial, in Exascale Computing Project Annual Meeting, online, 2021. DOI: 10.6084/m9.figshare.14376956.

Individual modules may be cited as Speaker, Module Title, in Developing a Testing and Continuous Integration Strategy for your Team tutorial…


This tutorial is produced by the IDEAS Productivity project.

This work was supported by the U.S. Department of Energy Office of Science, Office of Advanced Scientific Computing Research (ASCR), and by the Exascale Computing Project (17-SC-20-SC), a collaborative effort of the U.S. Department of Energy Office of Science and the National Nuclear Security Administration.