Better Scientific Software

a tutorial presented at

ISC High Performance

on 2:00 pm - 6:00 pm CEST (UTC+2) Sunday 29 May 2022

Presenters: Anshu Dubey (Argonne National Laboratory) and Gregory R. Watson (Oak Ridge National Laboratory)

This page provides detailed information specific to the tutorial event above. Expect updates to this page up to, and perhaps shortly after, the date of the tutorial. Pages for other tutorial events can be accessed from the main page of this site.

Quick Links

Presentation Slides (FigShare)
Hands-On Code Repository (GitHub)

Description

The computational science and engineering (CSE) community is in the midst of an extremely challenging period created by the confluence of disruptive changes in computing architectures, demand for greater scientific reproducibility, and new opportunities for greatly improved simulation capabilities, especially through coupling physics and scales. Computer architecture changes require new software design and implementation strategies, including significant refactoring of existing code. Reproducibility demands require more rigor across the entire software endeavor. Code coupling requires aggregate team interactions including integration of software processes and practices. These challenges demand large investments in scientific software development and improved practices. Focusing on improved developer productivity and software sustainability is both urgent and essential.

This tutorial will provide information and hands-on experience with software practices, processes, and tools explicitly tailored for CSE. Goals are improving the productivity of those who develop CSE software and increasing the sustainability of software artifacts. We discuss practices that are relevant for projects of all sizes, with emphasis on small teams, and on aggregate teams composed of small teams. Topics include software licensing, effective models, tools, and processes for small teams (including agile workflow management), reproducibility, and scientific software testing (including automated testing and continuous integration).

Agenda

Time (CEST)	Module	Title	Presenter
2:00 PM	0	Introduction and Setup	Anshu Dubey (ANL)
2:10 PM	1	Motivation and Overview of Best Practices in HPC Software Development	Anshu Dubey (ANL)
2:30 PM	2	Agile Methodologies	Gregory R. Watson (ORNL)
3:00 PM	3	Git Workflows	Gregory R. Watson (ORNL)
3:30 PM	4	Scientific Software Design	Anshu Dubey (ANL)
4:00 PM		Break
4:30 PM	5	Improving Reproducibility Through Better Software Practices	Gregory R. Watson (ORNL)
5:00 PM	6	Software Testing Introduction	Gregory R. Watson (ORNL)
5:20 PM	7	Continuous Integration	Gregory R. Watson (ORNL)
5:40 PM	8	Summary	Anshu Dubey (ANL)
6:00 PM		Adjourn

Presentation Slides

The latest version of the slides will always be available at https://doi.org/10.6084/m9.figshare.19781752.

Note that these files may include additional slides that will not be discussed during the tutorial, but questions are welcome.

Hands-On Exercises

Introduction

The hands-on exercises for this tutorial are based around a simple numerical model using the one-dimensional heat equation. The example is described briefly in the repository’s README file, and in greater detail in the ATPESC Hands-On lesson. The ATPESC version focuses on the numerical aspects of the model.

But for this tutorial, we’re focused on how to make the software better from a quality perspective, so you don’t need to understand the math to do these exercises.

For the purposes of these hands-on exercises, you should imagine you’ve inherited an early version of the hello-numerical-world software from a colleague who’s left the project, and you’ve been assigned to get it into better shape so that it can be used in the next ATPESC summer school.

The repository you’ll be working with is: bssw-tutorial/hello-numerical-world-2022-05-29-isc.

Note: most of the screenshots will refer to the generic “hello-numerical-world” repository rather than the one specifically for this event.

List of Hands-On Exercises

Note that not every presentation module has exercises (yet).

Setting up the Prerequisites. Setup the accounts needed for these exercises.
Agile Methodologies. You’ll use GitHub issues and project boards to setup a simple “personal kanban” board.
Basic Git for Collaboration. You’ll fork our hello-numerical-world repository, create a feature branch, and make a pull request
Software Testing. You’ll use an example project to try out using test driven development to add new functionality to a project
Continuous Integration. You’ll establish a simple continuous integration workflow and then refine it, adding code coverage assessment

There are also activities associated with a couple of modules that we didn’t have time to cover in this tutorial. You are welcome to try them out too.

Agile Redux. You’ll create epic, story, and task issues for the refactoring task and track them on a kanban board
Refactoring Scientific Software. You’ll perform a small, well-defined refactoring exercise

Stay in Touch

After the tutorial please feel free to email questions or feedback to the BSSw tutorial team at bssw-tutorial@lists.mcs.anl.gov.
If you want to do the hands-on exercises, we’re happy to provide feedback on your pull requests and issues, even after the end of the tutorial.
To find out about future events organized by the IDEAS Productivity Project, you can subscribe to our mailing list (usually ~2 messages/month).
For monthly updates on the Better Scientific Software site, subscribe to our monthly digest.

If you’re interested in this tutorial, you might be interested in this list of other software-related events taking place in the ISC22 conference.

Resources from Presentations

Links from the tutorial presentations are listed here for convenience

Module 0: Introduction and Setup
- IDEAS Productivity Project
- The IDEAS report
- Better Scientific Software site
- IDEAS Productivity mailing list
- BSSw Digest
- RSS Feed
- Inclusive Naming Initiative
- Bssw.io resource on Inclusive Naming
- Write to the tutorial authors
Module 1: Motivation and Overview of Best Practices in HPC Software Development
Module 2: Agile Methodologies
Module 3: Git Workflows
- Atlassian/BitBucket (Comparing Workflows)
- How to code review in a Pull Request
- Git Flow (Driessen’s Original Blog)
  - Git extensions
- GitHub Flow (previously linked to ~~scottchacon.com/2011/08/31/github-flow.html~~)
- GitLab Flow (previously linked to ~~docs.gitlab.com/ee/topics/gitlab_flow.html~~)
- Trilinos
- Open MPI
- FleCSI
Module 4: Scientific Software Design
- The Exascale Computing Project (ECP)
- Findings from the ECP Performance Portability Panel Series
- Performance Portability and the Exascale Computing Project
- Kokkos Lecture Series
- Related paper: A Design Proposal for a Next Generation Scientific Software Framework
- Related webinar: Software Design for Longevity with Performance Portability
Module 5: Improving Reproducibility Through Better Software Practices
- Motivations and Background:
- Definitions, Guidelines, and Organizations:
- Helpful Tools
  - Floating Point Analysis Tools
  - Code Ocean (Cloud platforms - publish and reproduce research code and data)
  - DOIs and hosting of data, code, documents:
    - Zenodo
    - FigShare
- Other Resources:
Module 6: Software Testing Introduction
- Python Build and Test Framework: pyscaffold.org
- Build-Link-Test CMake Framework: llnl-blt.readthedocs.io
- Tutorials for code coverage: Online Tutorial, Another example
- Test drive development example
- CMake Tutorial
- CMake add-test command documentation
- Verification and Validation in Scientific Computing
- Working Effectively with Legacy Code
Module 7: Continuous Integration
- Gitlab CI/CD Concepts
Module 8: Summary

Requested Citation

The requested citation the overall tutorial is:

Anshu Dubey and Gregory R. Watson, Better Scientific Software tutorial, in ISC High Performance, Hamburg, Germany, and online, 2022. DOI: 10.6084/m9.figshare.19781752.

Individual modules may be cited as Speaker, Module Title, in Better Scientific Software tutorial…

Acknowledgements

This tutorial is produced by the IDEAS Productivity project.

This work was supported by the U.S. Department of Energy Office of Science, Office of Advanced Scientific Computing Research (ASCR), and by the Exascale Computing Project (17-SC-20-SC), a collaborative effort of the U.S. Department of Energy Office of Science and the National Nuclear Security Administration.