Evaluating the Sustainability of Computational Science and Engineering Software: Empirical Observations.
Abstract not provided.
Willenbring, James↗
SEARCH · Engineering Papers
Results for “engineering software”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
An Exploration of the Mentorship Needs of Research Software Engineers.
Abstract not provided.
Milewicz, Reed↗
Building Bridges: Establishing a Dialogue Between Software Engineering Research and Computational Science.
Abstract not provided.
Milewicz, Reed↗
Mentorship of Women in OSS Projects: A Cross-Disciplinary, Integrative Review (in Equity, Diversity, and Inclusion in Software Engineering: Best Practices and Insights)
Explore the source record for details and available documents.
Mariska, Jacobs↗
Seeking Enlightenment: Incorporating Evidence-Based Practice Techniques in a Research Software Engineering Team
Explore the source record for details and available documents.
Milewicz, Reed Morgan↗
RSE-Ops
If DevOps is the intersection of "Developer" and "Operations," then how does this concept map to high performance computing? The answer is RSE-ops, or the intersection of Research Software Engineering and Operations. Research Software Engineers (RSEs) are the individuals writing code for scientific software, and supporting researchers to use codes on high performance computing systems. RSE-ops, then, generally refers to best practices for development and operations of scientific software, which typically happens in a high performance computing environment. A comparison can be made to Cloud Native Devops, which is a similar term that has emerged around Cloud Native Computing. It refers to the application of DevOps practices to the cloud.
97 MATHEMATICS AND COMPUTING↗
Best practices in software development for robust and reproducible geoscientific models based on insights from the Global Carbon Budget's dynamic vegetation models
Computational models play an increasingly vital role in scientific research by enabling the numerical simulation of complex processes. Such models are also fundamental in geosciences. For instance, they offer critical insights into the impacts of global change on the Earth system today and in the future. Beyond their value as research tools, models are also software products and should therefore adhere to certain established software engineering standards. However, scientists are rarely trained as software developers, which can lead to potential deficiencies in software quality like unreadable, inefficient, or erroneous code. The complexity of models, coupled with their integration into broader workflows, also often makes it challenging to reproduce results, evaluate processes, and build upon them. In this paper, we review the state and current practices of the development processes of the state-of-the-art land surface models used by the Global Carbon Budget. We combine the experience of modelers from the respective research groups with the expertise of software engineers from tech companies to outline key principles and tools for improving software quality in research. We explore four main areas: (1) model testing and validation, (2) scientific, technical, and user documentation, (3) version control, continuous integration, and code review, and (4) the portability and reproducibility of workflows. Our review reveals that while modeling communities are incorporating many best practices, significant room for improvement remains in areas such as automated testing, automated documentation, and reproducibility. Therefore, we here identify and promote essential software engineering practices, including numerous examples of practices from within the community that can serve as guidelines for other models and could help streamline processes across the entire community. We conclude with an open-source example implementation of these principles, demonstrating portable and reproducible data flows, a continuous integration setup, and web-based visualizations. This example may serve as a practical resource for model developers, users, and all scientists engaged in scientific programming.
Gregor, Konstantin [Technical Univ. of Munich (Ger↗
WETO Software Stack Best Practices
Wind energy researchers typically share one key characteristic: a passion for increasing wind energy in the global energy mix. The U.S. Department of Energy (DOE) supports this mission in a number of ways including allocating funding directly to various aspects of wind energy research through the Office of Energy Efficiency and Renewable Energy (EERE) via the Wind Energy Technologies Office (WETO). While the traditional output of research is academic publication, software development efforts are increasingly a major focus. Software tools in the research environment allow researchers to describe an idea and quickly increase the scope and scale as they study it further. As a product of research, these tools represent a direct pipeline from researcher to industry practitioners since they are the implementation of ideas described in academic publications. Given this vital role in wind energy research and commercial development, the broad research software portfolio supported by WETO must maintain a minimum level of quality to support the wind energy field in the growing transition to renewable energy. This report outlines a series o f best practices to be adopted by all WETO-supported software projects, as well as expectations that the communities interacting with these projects should have of the developers and tools themselves. Wind energy research software has a unique standing in the field of scientific software. The stakeholders are varied with a subset being: (1) DOE EERE leadership, (2) DOE WETO leadership and program managers, (3) National lab leadership, (4) Associated project principle investigators, (5) Research software engineers, (6) Wind energy researchers in academia (including graduate students, post docs, and national lab staff), (7) Industry researchers and practitioners, (8) Commercial software developers, and (9) The general public interested in wind energy. These software are typically the end-user of other generic software libraries, so the funding cycles are often tied to applied research rather than the development of the software itself. Since the developers are also wind energy researchers, these tools are typically designed in a way that closely resembles the application in which they're used. Additionally, the expertise and incentives for the developers have a high variability, and often neither are aligned with software engineering or computer science. Given the unique environment in which wind energy research software is produced and consumed, it is critical for model owners to understand the context of their software. A framework for developing this understanding is to answer the following questions of a given software project: What is it's purpose? What is its role in the field of wind energy? What is the profile of the expected users? For how long will it be relevant? What is the expected impact? These questions allow model owners to identify the appropriate methods for the design, development, and long term maintenance of their software. Additionally, the answer provide context for future planners to understand why particular decisions were made and discern the consequences of changing course. The information is aggregated from experience within WETO-supported software development groups as well as external organizations and efforts to define the craft of research software engineering. These best practices aim to make the collaborative development process efficient and effective while improving the model understanding across stakeholders. Additionally, the general adoption of a common framework for software quality ensures that the end users of WETO software can trust these tools and accurately understand the risks to workflow integration.
17 WIND ENERGY↗
Building and Sustaining a Community Resource for Best Practices in Scientific Software: The Story of BSSw.io
The development of scientific software—a cornerstone of long-term collaboration and scientific progress—parallels the development of other types of software but still poses distinct challenges, especially in high-performance computing. Although web searches yield numerous resources on software engineering, there is still a scarcity specifically for scientific software development. Here, this article introduces the Better Scientific Software site (https://bssw.io), a platform that hosts a community of researchers, developers, and practitioners who share their experiences and insights on scientific software development. Since 2017, this collaborative hub has gained traction within the scientific computing community, attracting a growing number of readers and contributors eager to share ideas and elevate their software development practices. In sharing the BSSw.io site’s story, we hope to encourage further growth of the BSSw.io community through both readership and contributors, with a long-term goal of fostering culture change by increasing emphasis on best practices in scientific software.
97 MATHEMATICS AND COMPUTING↗
Challenges and Strategies for Testing Automation Practices at Sandia National Laboratories
Sandia National Laboratories is a premier United States national security laboratory which develops science-based technologies in areas such as nuclear deterrence, energy production, and climate change. Computing plays a key role in its diverse missions, and within that environment, Research Software Engineers (RSEs) and other scientific software developers utilize testing automation to ensure quality and maintainability of their work. We conducted a Participatory Action Research study to explore the challenges and strategies for testing automation through the lens of academic literature. Through the experiences collected and comparison with open literature, we identify these challenges in testing automation and then present strategies for mitigation grounded in evidence-based practice and experience reports that other, similar institutions can assess for their automation needs.
97 MATHEMATICS AND COMPUTING↗
Good Practices for High-Quality Scientific Computing
Experimental and observational sciences have developed robust practices for conducting experiments, maintaining their instruments, and record keeping for provenance. Computational science has only recently begun to confront the issue of quality of their instrument, the software, and the credibility of their scientific output. Most of the available literature in software engineering relates to enterprise software. While it can inform practices in scientific software, adjustments are usually needed. From time to time quality conscious practitioners have published collections of best practices for scientific software. Here, this article provides one more such list but with updated suggestions, motivated by the need to keep up with the rapid changes in the computing industry.
97 MATHEMATICS AND COMPUTING↗
In Their Shoes: Persona-Based Approaches to Software Quality Practice Incentivization
Many teams struggle to adapt and right-size software engineering best practices for quality assurance to fit their context. Introducing software quality is not usually framed in a way that motivates teams to take action, thus resulting in it becoming a “check the box for compliance” activity instead of a cultural practice that values software quality and the effort to achieve it. When and how can we provide effective incentives for software teams to adopt and integrate meaningful and enduring software quality practices? Here, we explored this question through a persona-based ideation exercise at the 2021 Collegeville Workshop on Scientific Software in which we created three unique personas that represent different scientific software developer perspectives.
97 MATHEMATICS AND COMPUTING↗
Cyber-Informed Engineering Implementation Guide: Version 1.0 [Slides]
This Implementation Guide describes the principles of Cyber-Informed Engineering (CIE) and outlines questions that engineering teams should consider during each phase of a system's lifecycle to effectively employ these principles. It describes what it means to engineer systems in a cyber-informed way, rather than offering a comprehensive, step-by-step process or procedure for CIE implementation. This guide complements - but does not replace - the application of cybersecurity standards or practices currently in place within an organization. Engineers and technicians that design critical energy infrastructure installations can use this Implementation Guide to integrate the 12 principles of CIE into each phase of the engineering lifecycle, from concept to retirement. The guide is aimed at system or design engineers, rather than software engineers or operational cybersecurity practitioners. The engineers who design, build, operate, and maintain the physical infrastructure are best positioned to leverage a system's engineering design to diminish the severity of cyber attacks or digital technology failures. CIE expands cybersecurity decisions into the engineering space, not by asking engineers to become cyber experts, but by calling on engineers to apply engineering tools and make engineering decisions that improve cybersecurity outcomes. CIE examines the engineering consequences that a sophisticated cyber attacker could achieve and drives engineering changes that may provide deterministic mitigations to limit or eliminate those consequences.
24 POWER TRANSMISSION AND DISTRIBUTION↗
Cyber-Informed Engineering Implementation Guide
This Implementation Guide describes the principles of Cyber-Informed Engineering (CIE) and outlines questions that engineering teams should consider during each phase of a system’s lifecycle to effectively employ these principles. It describes what it means to engineer systems in a cyber-informed way, rather than offering a comprehensive, step-by-step process or procedure for CIE implementation. This guide complements—but does not replace—the application of cybersecurity standards or practices currently in place within an organization. Engineers and technicians that design critical energy infrastructure installations can use this Implementation Guide to integrate the 12 principles of CIE into each phase of the engineering lifecycle, from concept to retirement. The guide is aimed at system or design engineers, rather than software engineers or operational cybersecurity practitioners. The engineers who design, build, operate, and maintain the physical infrastructure are best positioned to leverage a system’s engineering design to diminish the severity of cyber attacks or digital technology failures. CIE expands cybersecurity decisions into the engineering space, not by asking engineers to become cyber experts, but by calling on engineers to apply engineering tools and make engineering decisions that improve cybersecurity outcomes. CIE examines the engineering consequences that a sophisticated cyber attacker could achieve and drives engineering changes that may provide deterministic mitigations to limit or eliminate those consequences.
42 ENGINEERING↗
Teaching Software Sustainability for High Performance Computing at ATPESC
The Argonne Training Program in Extreme Scale Computing (ATPESC) was started by Argonne National Laboratory with the objective of expanding the ranks of better-prepared users of high-performance computing (HPC) machines. One of the unique aspects of the program was inclusion of a track on software engineering and community codes. The inclusion was motivated by the observation that the projects with good software processes were better able to meet their scientific goals. Over the years, with greater awareness of software sustainability issues in the community, the track has evolved into a software productivity and sustainability track. In this paper we present our experience in choosing and disseminating the content related to the topic of software engineering in high performance computing science from the beginning of the program until now. We discuss the motivations and the reception of the tracks. We also document the evolution of the track over the years based on student feedback and also the growth of awareness about software productivity in high performance computing.
Dubey, Anshu↗
Software stewardship and advancement of a high-performance computing scientific application: QMCPACK
Here, we provide an overview of the software engineering efforts and their impact in QMCPACK, a production-level ab-initio Quantum Monte Carlo open-source code targeting high-performance computing (HPC) systems. Aspects included are: (i) strategic expansion of continuous integration (CI) targeting CPUs, using GitHub Actions own runners, and NVIDIA and AMD GPUs used in pre-exascale systems, (ii) incremental reduction of memory leaks using sanitizers, (iii) incorporation of Docker containers for CI and reproducibility, and (iv) refactoring efforts to improve maintainability, testing coverage, and memory lifetime management. We quantify the value of these improvements by providing metrics to illustrate the shift towards a predictive, rather than reactive, maintenance approach. Our goal, in documenting the impact of these efforts on QMCPACK, is to contribute to the body of knowledge on the importance of research software engineering (RSE) for the stewardship and advancement of community HPC codes to enable scientific discovery at scale.
97 MATHEMATICS AND COMPUTING↗
Summer 2024 INL Intern Poster Session Submission - Brian Schumitz
This LRS submission is my poster for the INL Intern Poster Session, Summer 2024. Abstract: The Software Engineering and Cybersecurity Lab (SECL) at Montana State University has developed PIQUE, a system for evaluating software quality. PIQUE's adaptability allows for language-specific static-analysis operations, including a model for assessing cloud microservice ecosystems. These ecosystems often rely on Docker for efficient deployment and management of containerized services. Our research focuses on evaluating the network quality within these microservice ecosystems. To automate this process, we're utilizing Snort, an open-source intrusion detection system renowned for its ability to detect and log network traffic. By leveraging Snort's customizable rules, we aim to construct comprehensive testing methods for measuring and quantifying the network quality based on traffic between Docker containers. This research aims to enhance the overall security and reliability of cloud microservice ecosystems by providing automated and robust quality evaluation mechanisms, ultimately contributing to the advancement of software engineering practices in these environments
97 MATHEMATICS AND COMPUTING↗
LLNL Response to the DOE ASCR RFI, "Stewardship of Software for Scientific and High-Performance Computing"
For decades, Lawrence Livermore National Laboratory (LLNL) has been engaged in significant research, development, and support for software to enable scientific computing and, particularly, the use of high performance computing (HPC) in the NNSA mission space. In particular, the move in the mid-1990’s to simulation as a leading component of stockpile stewardship through the ASCI and the successor ASC programs, as well as the need for reliable data acquisition and control software for the National Ignition Facility, have been important drivers in building expertise in production-quality software development at LLNL. LLNL has also been a leader in the DOE SciDAC FASTMath Institute and the DOE Exascale Computing Project (ECP), both of which have striven to make scientific computing software – in particular, the enabling technologies underpinning simulation capabilities – more widely adopted and sustainable. As such, we believe that our experience can inform the broader goal of software stewardship for scientific and high-performance computing. LLNL strongly supports the formation of a new DOE ASCR program element in software stewardship and sustainment. Historically, DOE ASCR has funded applied mathematics and computer science research that has led to the development of important new capabilities and algorithms that are expressed as artifacts in research software. Such frameworks, libraries, and tools have seldom been directly funded to address the important issues of code maintenance, documentation, robustness, and community building. Software engineering and support have typically been done on the side in support of the ASCR-driven research products. DOE funding priorities have been slow to recognize that good software engineering, the kind that ensures research investments have more adoption and longevity, requires significant resources. Based upon our experiences, we have prepared this response to highlight the concerns and issues we believe to be important as DOE ASCR considers its role in scientific software stewardship. We believe that role is important and will require a significant investment of new funding to legitimately support the technologies past and future DOE ASCR investments have and will produce to facilitate their uptake and adoption in the broader scientific computing community. Following a summary of our involvement in scientific software development, the remainder our response is organized around the nine topics specifically identified in the RFI.
97 MATHEMATICS AND COMPUTING↗