Software Productivity and Sustainability

a track presented at

Argonne Training Program on Extreme-Scale Computing

on 8:30 am - 9:00 pm CDT (UTC-5) Thursday 11 August 2022

Presenters: David E. Bernholdt (Oak Ridge National Laboratory), Anshu Dubey (Argonne National Laboratory), Todd Gamblin (Lawrence Livermore National Laboratory), Jared O'Neal (Argonne National Laboratory), and Boyana R. Norris (University of Oregon)

This page provides detailed information specific to the tutorial event above. Expect updates to this page up to, and perhaps shortly after, the date of the tutorial. Pages for other tutorial events can be accessed from the main page of this site.

Quick Links

Playlist (YouTube)
Presentation Slides (FigShare)

Description

The BSSw tutorial focuses on issues of developer productivity, software sustainability, and reproducibility in scientific research software, particularly targeting high-performance computers.

Agenda

Time (CDT)	Title	Presenter
8:30 AM	Introduction	David E. Bernholdt (ORNL)
8:35 AM	Motivation and Overview of Best Practices in HPC Software Development	David E. Bernholdt (ORNL)
9:15 AM	Scientific Software Design	Anshu Dubey (ANL)
10:00 AM	Break
10:30 AM	An Introduction to Software Licensing	David E. Bernholdt (ORNL)
11:30 AM	Spack: Package Management for HPC	Todd Gamblin (LLNL)
12:30 PM	Lunch
1:30 PM	Software Testing and Verification	Anshu Dubey (ANL)
2:15 PM	Lab Notebooks for Computational Mathematics, Sciences, & Engineering	Jared O'Neal (ANL)
3:00 PM	Break
3:30 PM	Managing Computational Experiments	Jared O'Neal (ANL) and Anshu Dubey (ANL)
4:15 PM	Understanding Your Software Development	Boyana R. Norris (UO)
5:15 PM	Breakouts: Spack Hands-On or General Discussion	All
6:15 PM	Dinner and After-Dinner Talk
8:15 PM	Optional Evening Discussion Session	All
9:00 PM	Adjourn

Presentation Slides

The latest version of the slides will always be available at https://doi.org/10.6084/m9.figshare.20416215.

Note that these files may include additional slides that will not be discussed during the tutorial, but questions are welcome.

How to Participate

As we will be in-person, please feel free to ask questions at any time.
We will also be monitoring the #track-7-software Slack channel in case you don’t want to interrupt the speaker.
The schedule includes opportunities for extended Q&A/discussion both before and after dinner. This is particularly useful if you’d like to go in-depth on a question or topic, or if you have experiences of your own you’d like to discuss.

Stay in Touch

After the tutorial please feel free to email questions or feedback to the BSSw tutorial team at bssw-tutorial@lists.mcs.anl.gov.
Our usual Better Scientific Software tutorials include some different topics. If you’re interested in more on the topics below, we encourage you to check out the videos and slides from our SC21 tutorial. There are also hands-on exercises for some of these modules, which you’re welcome to try on your own. We’d be happy to comment on issues and pull requests you might file, or respond to emails.
- Git workflows as a tool for collaboration
- Agile methodologies
- Reproducibility
- Continuous integration
- Refactoring scientific software
To find out about future events organized by the IDEAS Productivity Project, you can subscribe to our mailing list (usually ~2 messages/month).
For monthly updates on the Better Scientific Software site, subscribe to our monthly digest.

Resources from Presentations

Links from the tutorial presentations are listed here for convenience

Module 1: Introduction
- IDEAS Productivity Project
- The IDEAS report
- Better Scientific Software site
- IDEAS Productivity mailing list
- BSSw Digest
- RSS Feed
- Write to the tutorial authors
Module 2: Motivation and Overview of Best Practices in HPC Software Development
Module 3: Scientific Software Design
Module 4: An Introduction to Software Licensing
- Free Software Foundation
- Open Source Initiative
- ChooseALicense.com
  - Summary of license features
- Article: Is the Software World Taking Too Much from the Open Source Community?
- License Compatability graphic
- Article: Managing Copyright Information within a Free Software Project
- Software Package Data Exchange (SPDX)
- Creative Commons
- Software Freedom Law Center (SFLC)
- https://en.wikipedia.org/wiki/License_compatibility
- http://contributoragreements.org/
- https://developercertificate.org/
- http://ebb.org/bkuhn/blog/2014/06/09/do-not-need-cla.html
- US DOE ASCR (open source) software policy
- Additional resources from people I trust
  - Neil Chue Hong
  - Todd Gamblin
    - License compatibility resources
      - Open Source Licenses and their Compatibility
      - Which Licenses May Not be Included within Apache Products?
    - (Re-) Licensing considerations of various organizations
      - LLVM Contemplates Relicensing, LLVM Relicensing Effort
      - HEP Software Foundation licensing working group, particularly:
        
        https://hepsoftwarefoundation.org/organization/2017/02/21/licensing.html
        
        https://hepsoftwarefoundation.org/organization/2018/05/09/licensing.html included this Update on Software Licensing
      - EasyBuild: GPLv2 licensing is a big issue, consider relicensing to BSD
      - GEANT Intellectual Property Rights Policy
    - Patents in software licenses
Module 5: Spack: Package Management for HPC
Module 6: Software Testing and Verification
- Useful resources on testing (formerly linked to ~~ideas-productivity.org/resources/howtos/~~)
- Related articles:
  - Methodology for Building Granular Testing in Multicomponent Scientific Software
  - Ongoing verification of a multiphysics community code: FLASH
Module 7: Lab Notebooks for Computational Mathematics, Sciences, & Engineering
Module 8: Managing Computational Experiments
Module 9: Understanding Your Software Development
- Git repository mining: https://github.com/CAT-SDK/GremCat
  - Notebooks: https://github.com/CAT-SDK/GremCat/tree/main/notebooks
- Code analysis:
  - General-purpose: https://github.com/HPCL/code-analysis
  - Project-specific: https://github.com/HPCL/llvm-project/tree/xsdk-uo/clang-tools-extra/clang-tidy/petsc

Requested Citation

The requested citation the overall tutorial is:

David E. Bernholdt, Anshu Dubey, Todd Gamblin, Jared O’Neal, and Boyana R. Norris, Software Productivity and Sustainability track, in Argonne Training Program on Extreme-Scale Computing, St. Charles, Illinois, 2022. DOI: 10.6084/m9.figshare.20416215.

Individual modules may be cited as Speaker, Module Title, in Software Productivity and Sustainability track…

Acknowledgements

This tutorial is produced by the IDEAS Productivity project.

This work was supported by the U.S. Department of Energy Office of Science, Office of Advanced Scientific Computing Research (ASCR), and by the Exascale Computing Project (17-SC-20-SC), a collaborative effort of the U.S. Department of Energy Office of Science and the National Nuclear Security Administration.