Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “reproducible science”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Science Capsule: Towards Sharing and Reproducibility of Scientific Workflows

Workflows are increasingly processing large volumes of data from scientific instruments, experiments and sensors. These workflows often consist of complex data processing and analysis steps that might include a diverse ecosystem of tools and also often involve human-in-the-loop steps. Sharing and reproducing these workflows with collaborators and the larger community is critical but hard to do without the entire context of the workflow including user notes and execution environment. In this paper, we describe Science Capsule, which is a framework to capture, share, and reproduce scientific workflows. Science Capsule captures, manages and represents both computational and human elements of a workflow. It automatically captures and processes events associated with the execution and data life cycle of workflows, and lets users add other types and forms of scientific artifacts. Science Capsule also allows users to create `workflow snapshots' that keep track of the different versions of a workflow and their lineage, allowing scientists to incrementally share and extend workflows between users. Our results show that Science Capsule is capable of processing and organizing events in near real-time for high-throughput experimental and data analysis workflows without incurring any significant performance overheads.

Ghoshal, Devarshi↗
DOE OSTI2021

The case for free and open source software in research and scholarship

Free and open source software (FOSS) is any computer program released under a licence that grants users rights to run the program for any purpose, to study it, to modify it, and to redistribute it in original or modified form. Here, our aim is to explore the intersection between FOSS and computational reproducibility. We begin by situating FOSS in relation to other ‘open’ initiatives, and specifically open science, open research, and open scholarship. In this context, we argue that anyone who actively contributes to the research process today is a computational researcher, in that they use computers to manage and store information. We then provide a primer to FOSS suitable for anyone concerned with research quality and sustainability—including researchers in any field, as well as support staff, administrators, publishers, funders, and so on. Next, we illustrate how the notions introduced in the primer apply to resources for scientific computing, with reference to the GNU Scientific Library as a case study. We conclude by discussing why the common interpretation of ‘open source’ as ‘open code’ is misplaced, and we use this example to articulate the role of FOSS in research and scholarship today.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Increasing the Reproducibility and Replicability of Supervised AI/ML in the Earth Systems Science by Leveraging Social Science Methods

Artificial intelligence (AI) and machine learning (ML) pose a challenge for achieving science that is both reproducible and replicable. The challenge is compounded in supervised models that depend on manually labeled training data, as they introduce additional decision-making and processes that require thorough documentation and reporting. We address these limitations by providing an approach to hand labeling training data for supervised ML that integrates quantitative content analysis (QCA)—a method from social science research. The QCA approach provides a rigorous and well-documented hand labeling procedure to improve the replicability and reproducibility of supervised ML applications in Earth systems science (ESS), as well as the ability to evaluate them. Specifically, the approach requires (a) the articulation and documentation of the exact decision-making process used for assigning hand labels in a “codebook” and (b) an empirical evaluation of the reliability” of the hand labelers. In this paper, we outline the contributions of QCA to the field, along with an overview of the general approach. We then provide a case study to further demonstrate how this framework has and can be applied when developing supervised ML models for applications in ESS. With this approach, we provide an actionable path forward for addressing ethical considerations and goals outlined by recent AGU work on ML ethics in ESS.

58 GEOSCIENCES↗
DOE OSTI2026

Reproducibility, Replicability, and Research Quality in Homogeneous Catalysis

Researchers from all sectors of homogeneous catalysis convened in response to concerns regarding reproducibility in science to analyze the issue and provide recommendations. In addition to an in- person workshop, the group engaged the broader homogeneous catalysis community through a webinar series and virtually during the workshop. Results of the project affirm that homogeneous catalysis is not in a reproducibility crisis, as evidenced by the field’s current and past contributions to society that have led to economic growth and advances in a range of industries from agriculture to consumer goods to human health. However, it is not uncommon for researchers to encounter obstacles related to reproducibility. Ensuring reproducibility remains the responsibility of the community, both in current work and in training future researchers. This report is intended to engage key stakeholders, including disciplinary societies, publishers, employers, research leaders, and researchers, in practices that maximize reproducible homogeneous catalysis and ensure continued innovation and translatable discoveries. Recommendations made herein are also framed to be applicable beyond homogeneous catalysis, empowering the broader chemical if not scientific community.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2021

Creating a Tools Ecosystem for Cross-Discipline Environmental Data Reuse

Reusing data is difficult even within well-defined science communities and only gets worse when combining data from multiple communities and disciplines. Through the lens of current work on constructing an environmental epidemiological data set from multiple disciplinary sources, we demonstrate the need for a new tool ecosystem to support heterogeneous Big Data science. Extending existing community standards for schemas and/or data formats through human auditing and wrangling of the data is not feasible at scale. This work therefore suggests new approaches for the multi-disciplinary communities to build a shared tool ecosystem for big data. We discuss both the larger context of data wrangling of epidemiological data sets for novel artificial intelligence algorithms and the specific lessons from working with these multi-disciplinary data sets. Adopting a more model-driven, automatable approach promises not only better efficiency but also removes key sources of human-generated errors and promotes reuse and reproducibility of science data.

Logan, Jeremy↗
DOE OSTI2024

A Cast of Thousands: How the IDEAS Productivity Project Has Advanced Software Productivity and Sustainability

Computational and data-enabled science and engineering are revolutionizing advances throughout science and society, at all scales of computing. For example, teams in the U.S. Department of Energy’s Exascale Computing Project have been tackling new frontiers in modeling, simulation, and analysis by exploiting unprecedented exascale computing capabilities—building an advanced software ecosystem that supports next-generation applications and addresses disruptive changes in computer architectures. However, concerns are growing about the productivity of the developers of scientific software. Members of the Interoperable Design of Extreme-scale Application Software project serve as catalysts to address these challenges through fostering software communities, incubating and curating methodologies and resources, and disseminating knowledge to advance developer productivity and software sustainability. This article discusses how these synergistic activities are advancing scientific discovery—mitigating technical risks by building a firmer foundation for reproducible, sustainable science at all scales of computing, from laptops to clusters to exascale and beyond.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

A software environment for effective reliability management for pulsed power design

The reliable design of magnetically insulated transmission lines (MITLs) for very high current pulsed power machines must be accomplished in the future by utilizing a variety of sophisticated modeling tools. The complexity of the models required is high and the number of sub-models and approximations large. The potential for significant analyst error using a single tool is large, with possible reliability issues associated with the plasma modeling tools themselves or the chosen approach by the analyst to solve a given problem. Here, we report on a software infrastructure design that provides a workable framework for building self-consistent models and constraining feedback to limit analyst error. The framework and associated tools aid the development of physical intuition, the development of increasingly sophisticated models, and the comparison of performance results. The work lays the computational foundation for designing state-of-the-art pulsed-power experiments. The design and useful features of this environment are described. We discuss the utility of the Git source code management system and a GitLab interface for use in project management that extends beyond software development tasks.

42 ENGINEERING↗
DOE OSTI2022

Ten simple rules for getting and giving credit for data

This article attempts to summarize current best practices that support the movement towards enabling researchers to cite and receive credit for their data. The authors are a small representation of the people and organizations trying to make this happen, and we acknowledge that it is not possible to capture all efforts behind this endeavor in 10 Simple Rules. We encourage interested readers to dive deeper by providing related resources along the way.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2023

The Quick Rise and Fall of LK-99 as a Room Temperature Superconductor [Slides]

A swift determination of superconductivity vs non-superconductivity of LK-99 demonstrates the importance of reproducibility in science. This has benefited from 4 decades study of high-temperature cuprate mechanism. Holy grail of room-temperature ambient-pressure superconductor remains to be pursued. Caution is needed about DFT calculations. Crucial old experimental data should be referenced. With the existing amount of experimentally discovered superconductors, could data science make a stride?

36 MATERIALS SCIENCE↗
DOE OSTI2026

From Reads to Function Workshop - Milano 2026

The Bicocca Sampling Days (BSDs) model offers a reproducible “citizen science” framework integrating research, education, and public engagement through large-scale microbiome sampling, followed by a workshop of data analysis on select samples. We identified 9 bacterial and archaeal metagenome-assembled genomes from six soil samples across three separate sampling days in two approaches with indidivual sample and replicate co-assembly spanning three unique classes, providing genomic insights into microbial nutrient cycling in these systems.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2026

Data readiness pipeline patterns for scientific AI at scale: Insights from climate, fusion, life sciences, and materials

This article examines how data readiness for AI principles apply to large scientific datasets used to train foundation models. We analyze archetypal workflows across four representative domains—climate, nuclear fusion, life sciences, and materials—to identify common preprocessing patterns and domain‐specific constraints. We introduce a two‐dimensional readiness model that combines canonical preprocessing patterns with a five‐level operational readiness scale, both tailored to high‐performance computing (HPC) environments. This construct helps outline key challenges in transforming large‐scale scientific data into formats suitable for scalable AI training. Together, these dimensions form a conceptual maturity matrix that characterizes scientific data readiness and guides infrastructure development toward standardized, cross‐domain support for scalable and reproducible AI for science. Finally, we evaluate this maturity matrix in the context of case studies including ClimaX (climate), AFLOW (materials), OpenFold (proteomics), and DIII‐D fusion disruption‐prediction workflows, from which we distill lessons learned and provide recommendations to guide practitioners in developing robust AI‐readiness pipelines. Finally, we discuss remaining cross‐cutting challenges that persist across scientific domains.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

How should reproducibility be approached in plastic recycling?

With the growing importance of developing new and improved methodologies for plastic recycling, conducting reproducible research and ensuring that results are transferable across labs are increasingly important. This Voices article reflects on how academia and industry view the path forward for strengthening reproducibility to advance science and enable a circular plastics economy.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2023

UVCGAN: UNet Vision Transformer cycle-consistent GAN for unpaired image-to-image translation

Unpaired image-to-image translation has broad applications in art, design, and scientific simulations. One early breakthrough was CycleGAN that emphasizes one-to-one mappings between two unpaired image domains via generative-adversarial networks (GAN) coupled with the cycle-consistency constraint, while more recent works promote one-to-many mapping to boost diversity of the translated images. Motivated by scientific simulation and one-to-one needs, this work revisits the classic CycleGAN framework and boosts its performance to outperform more contemporary models without relaxing the cycle-consistency constraint. To achieve this, we equip the generator with a Vision Transformer (ViT) and employ necessary training and regularization techniques. Compared to previous best-performing models, our model performs better and retains a strong correlation between the original and translated image. An accompanying ablation study shows that both the gradient penalty and self-supervised pre-training are crucial to the improvement. To promote reproducibility and open science, the source code, hyperparameter configurations, and pre-trained model are available at https: //github.com/LS4GAN/uvcgan.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Data Readiness for Scientific AI at Scale

This paper examines how Data Readiness for AI (DRAI) principles apply to leadership-scale scientific datasets used to train foundation models. We analyze archetypal workflows across four representative domains—climate, nuclear fusion, bio/health, and materials—to identify common preprocessing patterns and domain-specific constraints. We introduce a two-dimensional readiness framework that combines canonical preprocessing patterns with a five-level operational readiness scale, both tailored to high-performance computing (HPC) environments. This framework helps outline key challenges in transforming large-scale scientific data into formats suitable for scalable AI training. Together, these dimensions form a conceptual maturity matrix that characterizes scientific data readiness and guides infrastructure development toward standardized, cross-domain support for scalable and reproducible AI for science.

Brewer, Wes [ORNL] (ORCID:0000000236393956)↗
DOE OSTI2025

gRASPA

GPU Monte Carlo Simulation Code with a taste of RASPA We present enhancements in Monte Carlo simulation speed and functionality within an open-source code, gRASPA, which uses graphical processing units (GPUs) to achieve significant performance improvements compared to serial, CPU implementations of Monte Carlo. The code supports a wide range of Monte Carlo simulations, including canonical ensemble (NVT), grand canonical, NVT Gibbs, Widom test particle insertions, and continuous-fractional component Monte Carlo. Implementation of grand canonical transition matrix Monte Carlo (GC-TMMC) and a novel feature to allow different moves for the different components of metal-organic framework (MOF) structures exemplify the capabilities of gRASPA for precise free energy calculations and enhanced adsorption studies, respectively. The introduction of a High-Throughput Computing (HTC) mode permits many Monte Carlo simulations on a single GPU device for accelerated materials discovery. The code can incorporate machine learning (ML) potentials. The open-source nature of gRASPA promotes reproducibility and openness in science, and users may add features to the code and optimize it for their own purposes. The code is written in CUDA/C++ and SYCL/C++ to support different GPU vendors. The gRASPA code is publicly available at https://github.com/snurr-group/gRASPA.

Li, Zhao [Purdue/Northwestern/Notre Dame Universit↗
DOE OSTI2021

[Re] Drivers of evapotranspiration from boreal wildfires

Computational reproducibility is a difficult challenge across science. I attempted to use R 3.6.1 to reproduce linear model fits, done originally using v2.6.0 for a 2009 paper on the drivers of large-scale forest evapotranspiration after wildfire. Model outputs were largely identical, aside from minor formatting changes, except for one–out of 12 total– regression in which the median residual value changed very slightly (in the sixth decimal place). I suggest that this essentially successful reproducibility is due to the relative simplicity of the script, its use of only base R functions, and R’s historically conservative approach to breaking changes.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Why don't we share data and code? Perceived barriers and benefits to public archiving practices

The biological sciences community is increasingly recognizing the value of open, reproducible and transparent research practices for science and society at large. Despite this recognition, many researchers fail to share their data and code publicly. This pattern may arise from knowledge barriers about how to archive data and code, concerns about its reuse, and misaligned career incentives. Here, we define, categorize and discuss barriers to data and code sharing that are relevant to many research fields. We explore how real and perceived barriers might be overcome or reframed in the light of the benefits relative to costs. By elucidating these barriers and the contexts in which they arise, we can take steps to mitigate them and align our actions with the goals of open science, both as individual scientists and as a scientific community.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
DOE OSTI2026

From Reproducible Edge–Cloud Experimentation to Real-World Practice: The E2Clab Experience

Reproducibility is already difficult in distributed systems; on the computing continuum, it becomes substantially harder. Applications that span sensing devices, edge and fog resources, and cloud platforms must be evaluated across heterogeneous hardware, variable network conditions, cross-layer orchestration decisions, and long-running workflow lifecycles. We use E2Clab as a case study to examine these challenges and their implications for experimental methodology. We explain why reproducible experimentation is harder on the continuum, then revisit E2Clab as an initial response based on explicit modeling of infrastructure, workflow lifecycle, and artifacts. Lastly, we discuss how its evolution toward more realistic application settings can be understood through the lens of Translational Computer Science. We argue that reproducible continuum experimentation requires methods that are rigorous enough for research while remaining adaptable to real-world practice.

42 ENGINEERING↗