Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “scientific reproducibility”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3
All sourcesNASA NTRSDOE OSTI
NASA NTRS2018

Snakes on a Spaceship - An Overview of Python in Heliophysics

Computational analysis has become ubiquitous within the heliophysics community. However, community standards for peer review of codes and analysis have lagged behind these developments. This absence has contributed to the reproducibility crisis, where inadequate analysis descriptions and loss of scientific data have made scientific studies difficult or impossible to replicate. The heliophysics community has responded to this challenge by expressing a desire for a more open, collaborative set of analysis tools. This article summarizes the current state of these efforts and presents an overview of many of the existing Python heliophysics tools. It also outlines the challenges facing community members who are working toward the goal of an open, collaborative, Python heliophysics toolkit and presents guidelines that can ease the transition from individualistic data analysis practices to an accountable, communalistic environment.

Burrell, A.G.↗
DOE OSTI2026

Scientific Core Library Stack (SCLS) v2026

SCLS (Scientific Core Library Stack) is an opinionated build and packaging system for scientific computing libraries developed at Lawrence Berkeley National Laboratory. It produces a coherent, reproducible stack of numerical libraries — including BLAS/LAPACK, MPI, sparse direct and iterative solvers, graph partitioners, and parallel I/O libraries (e.g., PETSc, SLEPc, HDF5, NetCDF, MUMPS, OpenBLAS) — that work together without manual repair by downstream scientific software. From a single recipe-and-flavor model, SCLS produces native RPM packages for RHEL-family Linux, DEB packages for Debian/Ubuntu, direct Unix-style prefix installs for HPC and locked-down environments, and native macOS builds. Multiple build "flavors" (e.g., GCC+OpenBLAS, GCC+MKL, Intel+MKL, debug) coexist in distinct prefixes on the same host. Compared to general-purpose meta-build frameworks, SCLS is deliberately curated rather than infinitely configurable. It enforces deterministic, audit-friendly behavior: explicit build dependencies, no silent feature autodetection, a clear open-source license policy, and rpath-based runtime linkage so installs integrate cleanly with standard package-manager workflows.

Messe, Christian [Lawrence Berkeley National Labor↗
DOE OSTI2024

Democratizing uncertainty quantification

Uncertainty Quantification (UQ) is vital to safety-critical model-based analyses, but the widespread adoption of sophisticated UQ methods is limited by technical complexity. In this paper, we introduce UM-Bridge (the UQ and Modeling Bridge), a high-level abstraction and software protocol that facilitates universal interoperability of UQ software with simulation codes. It breaks down the technical complexity of advanced UQ applications and enables separation of concerns between experts. UM-Bridge democratizes UQ by allowing effective interdisciplinary collaboration, accelerating the development of advanced UQ methods, and making it easy to perform UQ analyses from prototype to High Performance Computing (HPC) scale. In addition, we present a library of ready-to-run UQ benchmark problems, all easily accessible through UM-Bridge. These benchmarks support UQ methodology research, enabling reproducible performance comparisons. We demonstrate UM-Bridge with several scientific applications, harnessing HPC resources even using UQ codes not designed with HPC support.

Benchmarks↗
DOE OSTI2025

HARD: A performance portable radiation hydrodynamics code based on FleCSI framework

Hydrodynamics And Radiation Diffusion (HARD) is an open-source application for high-performance simulations of compressible hydrodynamics with radiation-diffusion coupling. Built on the FleCSI (Bergen et al., 2021 [1]) (Flexible Computational Science Infrastructure) framework, HARD expresses its computational units as tasks whose execution can be orchestrated by multiple back-end runtimes, including Legion (Bauer et al., 2012 [2]), MPI (Forum, 1994 [3]), and HPX (Kaiser et al., 2020 [4]). Node-level parallelism is handled through Kokkos (Edwards et al., 2014 [5]), providing a single-source, portable code base that runs efficiently on laptops, small homogeneous clusters, and the largest heterogeneous supercomputers currently available. To ensure scientific reliability, HARD includes a regression test suite that automatically reproduces canonical verification problems such as the Sod and LeBlanc shock tubes, and the Sedov blast wave, comparing numerical solutions against known analytical results. The project is distributed under an OSI-approved license, hosted on GitHub, and accompanied by reproducible build scripts and continuous integration workflows. This combination of performance portability, verification infrastructure, and community-focused development makes HARD a sustainable platform for advancing radiation hydrodynamics research across multiple domains.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1972

Mariner Mars 1971 data storage subsystem

A digital tape recorder was used on the Mariner Mars 1971 spacecraft to record television and scientific data. Data were recorded at 132 kilobits/sec and were reproduced at one of five available rates (16.2, 8.1, 4.05, 2.05, or 1.0125 kilobits/sec) selected by ground command to be congruous with the spacecraft-to-earth communications link performance. The transport mechanism contained 167 m of 1.2 cm magnetic recording tape. A single motor was used to drive the peripheral drive transport. During development of the design, stick slip problems were encountered. A design life of 2400 tape passes was achieved during the mission.

Grumm, R.↗
NASA NTRS1974

Precise time and time interval (PTTI), an overview

Present applications of precise time and frequency (T/F) technology can be grouped as follows: (1) Communications systems which require T/F for time division multiplexing and for using spread spectrum techniques. (2) Navigation systems which need T/F for position fixing using a timed signal. (3) Scientific-Metrological applications which use T/F as the most precisely reproducible standard of measurement. (4) Astronomical-Space applications which cover a variety of the most demanding applications such as pulsar research, Very Long Baseline Interferometry (VLBI) and laser/radar ranging. In particular, pulsar time-of-arrival measurements require submicrosecond precision over a period of one-half year referred to an extraterrestrial inertial system, and constitute the most stringent requirements for uniform timekeeping to date.

Winkler, G. M. R.↗
NASA NTRS2017

Limiting Data Friction by Reducing Data Download Using Spatiotemporally Aligned Data Organization Through STARE

Current data processing practice limits the volume and variety of relevant geoscience data that can practically be applied to important problems. File archives in centralized data centers are the principal means by which Earth Science data are accessed. This approach, however, requires laborious search, retrieval, and eventual customization/adaptation for the data to be used. Such fractionation makes it even more difficult to share outcomes, i.e. research artifacts and data products, hampering reusability and repeatability, since end users generally have their own research agenda and preferences as well as scarce resources. Thus, while finding and downloading data files from central data centers are already costly for end users working in their own field, using data products from other disciplines rapidly becomes prohibitive. This curtails scientific productivity, limits avenues of study, and endangers quality and reproducibility. The Spatio-Temporal Adaptive Resolution Encoding (STARE) is a unifying scheme that facilitates the indexing, access, and fusion of diverse Earth Science data. STARE implements an innovative encoding of geo-spatiotemporal information, originally developed for aligning datasets with diverse spatiotemporal characteristics in an array database. The spatial component of STARE recursively quadfurcates a root polyhedron, producing a hierarchical scheme for addressing geographic locations and regions. The temporal component of STARE uses conventional date-time units as an indexing hierarchy. The additional encoding of spatial and temporal resolution information in STARE enables comparisons and conditional selections across diverse datasets. Moreover, spatiotemporal set-operations, e.g. union and intersection, are mapped to efficient integer operations with STARE. Applied to existing data models (point, grid, spacecraft swath) and corresponding granules, STARE indexes provide a streamlined description usable as geo-spatiotemporal metadata. When coupled with large scale, distributed hardware and software, STARE-based data access reduces pre-analysis data preparation costs by offering a convenient means to align different datasets spatiotemporally without specialized effort in parallel computing or distributed data management.

Kuo, Kwo-Sen↗
DOE OSTI2026

Examination of Replicate Syntheses of Metal Organic Frameworks as a Window into Reproducibility in Materials Chemistry

Replicate experiments are a useful tool in understanding the repeatability of scientific measurements. In 2019, a systematic search for replicate syntheses of a collection of 130 metal–organic frameworks (MOFs) found that 89% of these materials had no reported replicate syntheses apart from the original publications identifying the material (Agrawal, M. Proc. Natl. Acad. Sci. U.S.A. 2020, 117, 877−88210.1073/pnas.1918484117). A potential weakness of that search was that only 5–11 years had elapsed since the original publication of each material. Here, this analysis is extended to all publications 11–17 years after the original publication. Although this extended time period identifies more repeat syntheses, 83% of the materials still have no reported replicate syntheses. We also consider how appropriately selected Density Functional Theory (DFT) calculations can provide corroboration for the experimentally reported crystal structures. By using data from previous high-throughput DFT studies, corroborating evidence from DFT was available for 17% of the 130 structures for which no replicate syntheses are available. In total, approximately 1/3 of the 130 MOFs have data associated with replicate synthesis experiments and/or directly corroborating DFT calculations.

Sholl, David S. [Oak Ridge National Laboratory (OR↗
DOE OSTI2026

Best practices in software development for robust and reproducible geoscientific models based on insights from the Global Carbon Budget's dynamic vegetation models

Computational models play an increasingly vital role in scientific research by enabling the numerical simulation of complex processes. Such models are also fundamental in geosciences. For instance, they offer critical insights into the impacts of global change on the Earth system today and in the future. Beyond their value as research tools, models are also software products and should therefore adhere to certain established software engineering standards. However, scientists are rarely trained as software developers, which can lead to potential deficiencies in software quality like unreadable, inefficient, or erroneous code. The complexity of models, coupled with their integration into broader workflows, also often makes it challenging to reproduce results, evaluate processes, and build upon them. In this paper, we review the state and current practices of the development processes of the state-of-the-art land surface models used by the Global Carbon Budget. We combine the experience of modelers from the respective research groups with the expertise of software engineers from tech companies to outline key principles and tools for improving software quality in research. We explore four main areas: (1) model testing and validation, (2) scientific, technical, and user documentation, (3) version control, continuous integration, and code review, and (4) the portability and reproducibility of workflows. Our review reveals that while modeling communities are incorporating many best practices, significant room for improvement remains in areas such as automated testing, automated documentation, and reproducibility. Therefore, we here identify and promote essential software engineering practices, including numerous examples of practices from within the community that can serve as guidelines for other models and could help streamline processes across the entire community. We conclude with an open-source example implementation of these principles, demonstrating portable and reproducible data flows, a continuous integration setup, and web-based visualizations. This example may serve as a practical resource for model developers, users, and all scientists engaged in scientific programming.

Gregor, Konstantin [Technical Univ. of Munich (Ger↗
NASA NTRS2012

Figures of Merit for Lunar Simulants

At an earlier SRR the concept for an international standard on Lunar regolith simulants was presented. The international standard, ISO 10788, Lunar Simulants, has recently been published. This paper presents the final content of the standard. Therefore, we are presenting an update of the following: The collection and analysis of lunar samples from 1969 to present has yielded large amounts of data. Published analyses give some idea of the complex nature of the regolith at all scales, rocks, soils and the smaller particulates commonly referred to as dust. Data recently acquired in support of NASA s simulant effort has markedly increased our knowledge and quantitatively demonstrates that complexity. It is anticipated that future analyses will further add to the known complexity. In an effort to communicate among the diverse technical communities performing research on or research using regolith samples and simulants, a set of Figures of Merit (FoM) have been devised. The objective is to allow consistent and concise comparative communication between researchers from multiple organizations and nations engaged in lunar exploration. This paper describes Figures of Merit in a new international standard for Lunar Simulants. The FoM methodology uses scientific understanding of the lunar samples to formulate parameters which are reproducibly quantifiable. Contaminants and impurities in the samples are also addressed.

Slane, Frederick A.↗
DOE OSTI2025

Reproducibility of fixed-node diffusion Monte Carlo across diverse community codes: The case of water–methane dimer

Fixed-node diffusion quantum Monte Carlo (FN-DMC) is a widely trusted many-body method for solving the Schrödinger equation, known for its reliable predictions of material and molecular properties. Furthermore, its excellent scalability with system complexity and near-perfect utilization of computational power make FN-DMC ideally positioned to leverage new advances in computing to address increasingly complex scientific problems. Even though the method is widely used as a computational gold standard, reproducibility across the numerous FN-DMC code implementations has yet to be demonstrated. This difficulty stems from the diverse array of DMC algorithms and trial wave functions, compounded by the method’s inherent stochastic nature. Here, this study represents a community-wide effort to assess the reproducibility of the method, affirming that yes, FN-DMC is reproducible (when handled with care). Using the water–methane dimer as the canonical test case, we compare results from eleven different FN-DMC codes and show that the approximations to treat the non-locality of pseudopotentials are the primary source of the discrepancies between them. In particular, we demonstrate that, for the same choice of determinantal component in the trial wave function, reliable and reproducible predictions can be achieved by employing the T-move, the determinant locality approximation, or the determinant T-move schemes, while the older locality approximation leads to considerable variability in results. These findings demonstrate that, with appropriate choices of algorithmic details, fixed-node DMC is reproducible across diverse community codes—highlighting the maturity and robustness of the method as a tool for open and reliable computational science.

Della Pia, Flaviano [Univ. of Cambridge (United Ki↗
DOE OSTI2026

Data readiness pipeline patterns for scientific AI at scale: Insights from climate, fusion, life sciences, and materials

This article examines how data readiness for AI principles apply to large scientific datasets used to train foundation models. We analyze archetypal workflows across four representative domains—climate, nuclear fusion, life sciences, and materials—to identify common preprocessing patterns and domain‐specific constraints. We introduce a two‐dimensional readiness model that combines canonical preprocessing patterns with a five‐level operational readiness scale, both tailored to high‐performance computing (HPC) environments. This construct helps outline key challenges in transforming large‐scale scientific data into formats suitable for scalable AI training. Together, these dimensions form a conceptual maturity matrix that characterizes scientific data readiness and guides infrastructure development toward standardized, cross‐domain support for scalable and reproducible AI for science. Finally, we evaluate this maturity matrix in the context of case studies including ClimaX (climate), AFLOW (materials), OpenFold (proteomics), and DIII‐D fusion disruption‐prediction workflows, from which we distill lessons learned and provide recommendations to guide practitioners in developing robust AI‐readiness pipelines. Finally, we discuss remaining cross‐cutting challenges that persist across scientific domains.

97 MATHEMATICS AND COMPUTING↗
NASA NTRSDate not supplied

Optimizing Single Nuclei Sequencing of Brain Samples From Space Flown Mice Across Age and Strain

The NASA GeneLab Sample Processing Laboratory offers high-throughput sequencing services to NASA-funded space biology researchers. Space biology studies have specific challenges such as low sample numbers, introducing susceptibility to batch effects from sample handling. These issues are compounded by complex protocols such as single-nuclei isolation and sequencing, which has recently become an attractive methodology for assessing the cellular diversity within spaceflight samples. High quality single-nuclei sequencing requires reproducible protocols to dissociate tissue and generate clean suspension of intact single nuclei. Producing single-nuclei suspension from brain tissue is particularly challenging due to cell type heterogeneity and the myelin sheath that carries over into the nuclei suspension as debris. Current procedures tend to be time consuming and sometimes include steps that can alter gene expression and create cell-type bias. Commercially available nuclei isolation kits, such as the 10X Genomics nuclei isolation kit, offers a streamlined way to process samples for nuclei isolation, thereby minimizing batch effects and enabling reproducibility. In this study, we report on the performance of the 10X Genomics nuclei isolation kit and Chromium Next GEM Single Cell Multiome ATAC + Gene Expression kit to generate sequencing libraries from space-flown mouse brain samples. Single nuclei sequencing was performed on frozen mouse brain tissue from two spaceflight missions, Rodent Research-10 (RR-10) and RR Reference Mission-2 (RRRM-2). RR-10 mice were female B6129SF2/J, euthanized at 18-19 weeks whereas RRRM-2 mice were female C57BL/6NTac, euthanized at 20 or 37 weeks. Sequencing data was processed using standard GeneLab data processing pipelines. We report evaluation of the performance of the 10X Genomics nuclei isolation kit for spaceflight samples from mouse brain, and evaluation of reproducibility across different mouse strains and age groups. We also report preliminary scientific results including cell type inference, cell clustering, and differentially expressed genes and pathways between spaceflight and ground control samples.

RR-10↗
NASA NTRS2017

Embracing Open Source for NASA's Earth Science Data Systems

The overarching purpose of NASAs Earth Science program is to develop a scientific understanding of Earth as a system. Scientific knowledge is most robust and actionable when resulting from transparent, traceable, and reproducible methods. Reproducibility includes open access to the data as well as the software used to arrive at results. Additionally, software that is custom-developed for NASA should be open to the greatest degree possible, to enable re-use across Federal agencies, reduce overall costs to the government, remove barriers to innovation, and promote consistency through the use of uniform standards. Finally, Open Source Software (OSS) practices facilitate collaboration between agencies and the private sector. To best meet these ends, NASAs Earth Science Division promotes the full and open sharing of not only all data, metadata, products, information, documentation, models, images, and research results but also the source code used to generate, manipulate and analyze them. This talk focuses on the challenges to open sourcing NASA developed software within ESD and the growing pains associated with establishing policies running the gamut of tracking issues, properly documenting build processes, engaging the open source community, maintaining internal compliance, and accepting contributions from external sources. This talk also covers the adoption of existing open source technologies and standards to enhance our custom solutions and our contributions back to the community. Finally, we will be introducing the most recent OSS contributions from NASA Earth Science program and promoting these projects for wider community review and adoption.

Earth Science↗
NASA NTRS1978

Management: A continuing literature survey with indexes

This bibliography lists 782 reports, articles, and other documents introduced into the NASA scientific and technical information system in 1977. The citations, and abstracts when available, are reproduced exactly as they appeared originally in IAA and STAR, including the original accession numbers from the respective announcement journals. Topics cover the management of research and development contracts, production, logistics, personnel, safety, reliability and quality control citations. Includes references on: program, project and systems management; management policy, philosophy, tools, and techniques; decisionmaking processes for managers; technology assessment; management of urban problems; and information for managers on Federal resources, expenditures, financing, and budgeting.

Source record↗
NASA NTRS2020

Analysis of Sample Acquisition Dynamics Using Discrete Element Method

The analysis presented in this paper is conducted in the framework of the Ocean Worlds Autonomy Testbed for Exploration Research and Simulation (OceanWATERS) project, currently under development at NASA Ames Research Center. OceanWATERS aims at designing a simulation environment which allows for testing autonomy of scientific lander missions to the icy moons of our solar system. Mainly focused on reproducing the end effector interaction with the inherent terrain, this paper introduces a novel discrete element method (DEM)-based approach to determine forces and torques acting on the lander’s scoop during the sample acquisition process. An accurate force feedback from the terrain on the scoop is required by fault-detection and autonomous decision-making algorithms to identify when the requested torque on the robotic arm’s joints exceeds the maximum available torque. Knowledge of the terrain force feedback significantly helps evaluating the arm’s links structural properties and properly selecting actuators for the joints. Models available in literature constitute a partial representation of the dynamics of the interaction. As an example, Balovnev derived an analytical expression of the vertical and horizontal force acting on a bucket while collecting a sample as a function of its geometry and velocity, soil parameters and reached depth. Although the model represents an adequate approximation of the two force components, it ignores the direction orthogonal to the scoop motion and neglects the torque. This work relies on DEM analysis to compensate for analytical models’ deficiencies and inaccuracies, i. e. provide force and torque 3D vectors, defined in the moving reference (body) frame attached to the scoop, at each instant of the sample collection process. Results from the first presented analysis relate to the specific OceanWATERS sampling strategy, which consists of collecting the sample through five consecutive passes with increasing depth, each pass following the same circularlinear- circular trajectory. Data is collected given a specific scoop design interacting with two types of bulk materials, which may characterize the surface of icy planetary bodies: snow and ice. Although specifically concerned with the OceanWATERS design, this first analysis provides the expected force trends for similar sampling strategies and allows to deduce phenomenological information about the general scooping process. In order to further instruct the community on the use of DEM tools as a solution to the sampling collection problem, two more analyses have been carried out, mainly focused on reducing the DEM computation time, which increases with a decrease in particle size. After running a set of identical simulations, where the only changing parameter is the size of the spherical particle, it is observed that the resulting force trajectories, starting from a given particle size, converge to the true trend. It is deducible that a further decrease in size yields negligible improvements in the accuracy, while it sensibly increases computation time. A final analysis aims at discussing limitations of approximating bulk material particles having a complex shape, e. g. ice fragments, with spheres, by comparing force trends resulting in the two cases for the same simulation scenario.

Catanoso, Damiana↗
NASA NTRS1998

Nucleation and Convection Effects in Protein Crystal Growth

Our work under this grant has significantly contributed to the goals of the NASA supported protein crystallization program. We have achieved the main objectives of the proposed work, as outlined in the original proposal: (1) We have provided important insight into protein nucleation and crystal growth mechanisms to facilitate a rational approach to protein crystallization; (2) We have delineated the factors that currently limit the x-ray diffraction resolution of protein crystals, and their correlation to crystallization conditions; (3) We have developed novel technologies to study and monitor protein crystal nucleation and growth processes, in order to increase the reproducibility and yield of protein crystallization. We have published 17 papers in peer-reviewed scientific journals and books and made more than 15 invited and 9 contributed presentations of our results at international and national scientific meetings.

Vekilow, Peter G.↗
DOE OSTI2025

A standards perspective on genomic data reusability and reproducibility

Genomic and metagenomic sequence data provides an unprecedented ability to re-examine findings, offering a transformative potential for advancing research, developing computational tools, enhancing clinical applications, and fostering scientific collaboration. However, effective and ethical reuse of genomics data is hampered by numerous technical and social challenges. The International Microbiome and Multi’Omics Standards Alliance (IMMSA, https://www.microbialstandards.org/) and the Genomic Standards Consortium (GSC, https://gensc.org) hosted a 5-part seminar series “A Year of Data Reuse” in 2024 to explore challenges and opportunities of data reuse and reproducibility across disparate domains of the genomic sciences. Addressing these challenges will require a multifaceted approach, including common metadata reporting, clear communication, standardized protocols, improved data management infrastructure, ethical guidelines, and collaborative policies that prioritize transparency and accessibility. We offer strategies to enable responsible and technically feasible data reuse, recognition of data reproducibility challenges, and emphasizing the importance of cross-disciplinary efforts in the pursuit of open science and data-driven innovation.

59 BASIC BIOLOGICAL SCIENCES↗