Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Reproducibility”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6
All sourcesNASA NTRSDOE OSTI
DOE OSTI2019

Ongoing solution reproducibility of earth system models as they progress toward exascale computing

We present a methodology for solution reproducibility for the Energy Exascale Earth System Model during its ongoing software infrastructure development to prepare for exascale computers. The nonlinear chaotic nature of climate system simulations precludes traditional model verification approaches since machine precision differences—resulting from code refactoring, changes in software environment, and so on—grow exponentially to a different weather state. Here, we leverage the nature of climate as a statistical description of the atmosphere in order to establish model reproducibility. We evaluate the degree to which two-sample equality of distribution tests can confidently detect the change in climate from minor tuning parameter changes on model output variables in order to establish the level of difference that indicates a new climate. To apply this (baselined test), we target a section of the model’s development cycle wherein no intentional science changes have been applied to its source code. We compare an ensemble of short simulations that were conducted using a verified model configuration against a new ensemble with the same configuration but with the latest software infrastructure (Common Infrastructure for Modeling the Earth, CIME5.0), compiler versions, and software libraries. We also compare these against ensemble simulations conducted using the original version of the software infrastructure (CIME4.0) of the earlier model configuration, but with the latest compilers and software libraries, to test the impact of new compilers and libraries in isolation from additional software infrastructure. The two-sample equality of distribution tests indicates that these ensembles indeed represent the same climate.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Efforts to enhance reproducibility in a human performance research project

Background: Ensuring the validity of results from funded programs is a critical concern for agencies that sponsor biological research. In recent years, the open science movement has sought to promote reproducibility by encouraging sharing not only of finished manuscripts but also of data and code supporting their findings. While these innovations have lent support to third-party efforts to replicate calculations underlying key results in the scientific literature, fields of inquiry where privacy considerations or other sensitivities preclude the broad distribution of raw data or analysis may require a more targeted approach to promote the quality of research output. Methods: We describe efforts oriented toward this goal that were implemented in one human performance research program, Measuring Biological Aptitude, organized by the Defense Advanced Research Project Agency's Biological Technologies Office. Our team implemented a four-pronged independent verification and validation (IV&V) strategy including 1) a centralized data storage and exchange platform, 2) quality assurance and quality control (QA/QC) of data collection, 3) test and evaluation of performer models, and 4) an archival software and data repository. Results: Our IV&V plan was carried out with assistance from both the funding agency and participating teams of researchers. QA/QC of data acquisition aided in process improvement and the flagging of experimental errors. Holdout validation set tests provided an independent gauge of model performance. Conclusions: In circumstances that do not support a fully open approach to scientific criticism, standing up independent teams to cross-check and validate the results generated by primary investigators can be an important tool to promote reproducibility of results.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2017

Reproducibility and Variability of I/O Performance on BG/Q: Lessons Learned from a Data Aggregation Algorithm

Reading and writing data efficiently from different tiers of storage is necessary for most scientific simulations to achieve good performance at scale. Many software solutions have been developed to decrease the I/O bottleneck. One wellknown strategy, in the context of collective I/O operations, is the two-phase I/O scheme. This strategy consists of selecting a subset of processes to aggregate contiguous pieces of data before performing reads/writes. In our previous work, we implemented the two-phase I/O scheme with a MPI-based topology-aware algorithm. Our algorithm showed very good performance at scale compared to the standard I/O libraries such as POSIX I/O and MPI I/O. However, the algorithm had several limitations hindering a satisfying reproducibility of our experiments. In this paper, we extend our work by 1) identifying the obstacles we face to reproduce our experiments and 2) discovering solutions that reduce the unpredictability of our results.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

A standards perspective on genomic data reusability and reproducibility

Genomic and metagenomic sequence data provides an unprecedented ability to re-examine findings, offering a transformative potential for advancing research, developing computational tools, enhancing clinical applications, and fostering scientific collaboration. However, effective and ethical reuse of genomics data is hampered by numerous technical and social challenges. The International Microbiome and Multi’Omics Standards Alliance (IMMSA, https://www.microbialstandards.org/) and the Genomic Standards Consortium (GSC, https://gensc.org) hosted a 5-part seminar series “A Year of Data Reuse” in 2024 to explore challenges and opportunities of data reuse and reproducibility across disparate domains of the genomic sciences. Addressing these challenges will require a multifaceted approach, including common metadata reporting, clear communication, standardized protocols, improved data management infrastructure, ethical guidelines, and collaborative policies that prioritize transparency and accessibility. We offer strategies to enable responsible and technically feasible data reuse, recognition of data reproducibility challenges, and emphasizing the importance of cross-disciplinary efforts in the pursuit of open science and data-driven innovation.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2023

Neural Network-Enhanced Reproducing Kernel Particle Method for Image-Based Multiphysics Damage Modeling of Energy Storage Materials

Energy storage materials undergo significant stresses during charge/discharge cycling, which makes understanding their reliability and durability fundamental in predicting performance and service life. Strong electrochemical-mechanical coupling and highly anisotropic material properties contribute to the formation and propagation of micro-cracking, largely along material interfaces and grain boundaries. With microstructural images supplied by the National Renewable Energy Laboratory (NREL), image-based modeling techniques are used to represent the complex material microstructures that dictate the coupled physics of these systems. Traditional electrochemical-mechanical models rely on mesh-based finite element methods, which can lead to difficulties in capturing crack propagation due to mesh dependency. Additionally, commonly used damage models, such as the continuous damage model and the cohesive zone model, often have steep tradeoffs between discontinuous field accuracy and computational expense. In this work, a neural network-enhanced reproducing kernel particle method (NN-RKPM) [1] is leveraged to accurately capture damage and crack propagation throughout the material by learning the location, orientation, and sharpness of discontinuity while allowing for a coarser nodal distribution than that necessary for capturing sharp solution transitions using traditional mesh-based methods. NN-RKPM is used to inform how crack opening and closure in turn affect the coupled chemical equations and material microstructure. Reference: [1] Baek, J., Chen, J. S., Susuki, K., "Neural Network enhanced Reproducing Kernel Particle Method for Modeling Localizations," International Journal for Numerical Methods in Engineering, Vol. 123, pp 4422-4454, https://doi.org/10.1002/nme.7040, 2022.

damage modeling↗
DOE OSTI2016

Reproducing the observed energy-dependent structure of Earth's electron radiation belts during storm recovery with an event-specific diffusion model

Here, we present dynamic simulations of energy-dependent losses in the radiation belt “slot region” and the formation of the two-belt structure for the quiet days after the 1 March storm. The simulations combine radial diffusion with a realistic scattering model, based data-driven spatially and temporally resolved whistler-mode hiss wave observations from the Van Allen Probes satellites. The simulations reproduce Van Allen Probes observations for all energies and L shells (2–6) including (a) the strong energy dependence to the radiation belt dynamics (b) an energy-dependent outer boundary to the inner zone that extends to higher L shells at lower energies and (c) an “S-shaped” energy-dependent inner boundary to the outer zone that results from the competition between diffusive radial transport and losses. We find that the characteristic energy-dependent structure of the radiation belts and slot region is dynamic and can be formed gradually in ~15 days, although the “S shape” can also be reproduced by assuming equilibrium conditions. The highest-energy electrons (E > 300 keV) of the inner region of the outer belt (L ~ 4–5) also constantly decay, demonstrating that hiss wave scattering affects the outer belt during times of extended plasmasphere. Through these simulations, we explain the full structure in energy and L shell of the belts and the slot formation by hiss scattering during storm recovery. We show the power and complexity of looking dynamically at the effects over all energies and L shells and the need for using data-driven and event-specific conditions.

58 GEOSCIENCES↗
DOE OSTI2022

How Reproducible are Surface Areas Calculated from the BET Equation?

Abstract Porosity and surface area analysis play a prominent role in modern materials science. At the heart of this sits the Brunauer–Emmett–Teller (BET) theory, which has been a remarkably successful contribution to the field of materials science. The BET method was developed in the 1930s for open surfaces but is now the most widely used metric for the estimation of surface areas of micro‐ and mesoporous materials. Despite its widespread use, the calculation of BET surface areas causes a spread in reported areas, resulting in reproducibility problems in both academia and industry. To prove this, for this analysis, 18 already‐measured raw adsorption isotherms were provided to sixty‐one labs, who were asked to calculate the corresponding BET areas. This round‐robin exercise resulted in a wide range of values. Here, the reproducibility of BET area determination from identical isotherms is demonstrated to be a largely ignored issue, raising critical concerns over the reliability of reported BET areas. To solve this major issue, a new computational approach to accurately and systematically determine the BET area of nanoporous materials is developed. The software, called “BET surface identification” (BETSI), expands on the well‐known Rouquerol criteria and makes an unambiguous BET area assignment possible.

36 MATERIALS SCIENCE↗
DOE OSTI2025

Demonstration of the Reproducibility Challenges in the Sintering Behavior of Lithium‐Stuffed Garnets in Scaling up Synthesis

Lithium-stuffed garnets, such as Li 7 La 3 Zr 2 O 12 (LLZO), are promising candidates for next-generation solid-state batteries because of their high room-temperature ionic conductivity and chemical stability against lithium metal anodes, which are crucial for achieving higher energy density. However, realizing LLZO's potential in practical devices requires synthesis methods that can be scaled reliably to large batch sizes for manufacturing. Herein, we investigate the sintering reproducibility of LLZO synthesized at larger scales using ultrasonic spray pyrolysis, a cost-effective and scalable synthesis route. Two 100 g batches of Al-doped LLZO are prepared and their sintering behavior is examined in detail. Both Al-LLZO batches contain over 90 wt.% cubic-phase LLZO, and both batches exhibit room temperature conductivities greater than 1 × 10 −4 S cm −1 at a relative density above 0.8. However, variations in secondary phases and subtle differences in Al content lead to significant differences in densification and microstructure. These results demonstrate that LLZO's sintering behavior is highly sensitive to small changes in secondary phases and Al content, creating reproducibility challenges when moving from laboratory- to manufacturing-scale synthesis.

36 MATERIALS SCIENCE↗
DOE OSTI2019

Sensitivity Analysis of Material Model Parameters to Reproduce Crushing of Composite Tubes

The ability to predict the behavior of composite structures under impact conditions is an important challenge for engineering analysis since it requires modeling beyond the elasticity and into failure initiation and propagation. Crushing of laminate composites is the result of a combination of several and complex failure mechanisms, which are not always possible to reproduce together in a single finite element analysis (FEA). In this paper, models based on lamina-level failure criteria have been used to predict the onset of damage; once failure initiates, the mechanisms of failure propagation require reducing the material properties using several degradation schemes. Among the composite material models available in LS-DYNA, MAT-ENHANCED-COMPOSITE-DAMAGE (MAT-54) has been chosen for modeling the axial crushing of composite tubes with various resistant sections and wall slope, given the reduced experimental input parameters to insert into the card definition. Comparison with experimental data shows that such material model is able to successfully reproduce the crushing behavior, even if the numerical results are strictly correlated with some model parameters which cannot be measured experimentally or are purely mathematical expedients. Therefore, extensive calibration of these parameters by trial-and-error is necessary to achieve successful simulation results. Once done it for a structure made of the same composite material, it is possible to adopt the same card configuration for all the other cases, obtaining good agreement between numerical and experimental results. After, the attempt to model the same geometries using the MAT-LAMINATED-COMPOSITE-FABRIC (MAT-58) card with the same parameter values previously calibrated has been done obtaining results comparable with the experimental data.

36 MATERIALS SCIENCE↗
DOE OSTI2018

Large-Area Graphene Films as Target Surfaces for Highly Reproducible Matrix-Assisted Laser Desorption Ionization Suitable for Quantitative Mass Spectrometry

Due to the known sweet-spot issues that intrinsically arise from inhomogeneous formation of matrix-analyte crystals utilized as samples in matrix-assisted laser desorption ionization (MALDI) mass spectrometry, its reproducibility and thus its applications for quantification have been somewhat limited. In this paper, we report a simple strategy to improve the uniformity of matrix-analyte crystal spots, which we realized by adapting large-area graphene films, i.e., non-inert, interacting surfaces, as target surfaces. In this example, the graphitic surfaces of the graphene films interact with excess matrix molecules during the sample drying process, which induces spontaneous formation of optically uniform MALDI sample crystal layers on the film surfaces. Further, mass spectrometric imaging reveals that the visible uniformity is indeed accompanied by reproducible MALDI ionization over an entire sample spot, which greatly suppresses the appearance of sweet spots. The results of this study confirm that the proposed method achieves good linear responses of ion intensity to the analyte concentration (R{sup 2} > 0.99) with small relative standard deviations (σ < 10%), which is a range applicable for quantitative measurements using MALDI mass spectrometry. .

77 NANOSCIENCE AND NANOTECHNOLOGY↗
DOE OSTI2023

Reproducible benchmark for the SNAP 8 experimental reactor at dry conditions

Here, this work provides fully reproducible benchmark models of the Systems for Nuclear Auxiliary Power (SNAP) 8 Experimental Reactor. Validation benchmarks of the criticality configuration experiments under dry conditions with no coolant are presented. In addition, relevant reactivity worth experiments, and measurement of power distributions are validated and presented here. Discrepancies between modeled and experimental results are at most 110 pcm for fuel and poison worths, and critical configurations that do not manipulate control elements are within 50 pcm. Larger discrepancies found in reflector element worth experiments are due to experimental methodology which was noted as being simplified due to limited calculational capabilities at the time, while discrepancy in the control element manipulated critical configuration is due the absence of structural components for modeling simplification. Power distributions closely follow what was observed in experiment with expected peaking in reflecting elements. All models are well documented with cited references describing and justifying assumptions, simplifications, and adjustments to reproduce the models with any nuclear safety codes; model inputs and outputs are stored in the snapReactors GitHub repository.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
DOE OSTI2026

Reproducible benchmark for the SNAP 8 experimental reactor at operating conditions

This work presents fully reproducible multiphysics benchmark models of the Systems for Nuclear Auxiliary Power (SNAP) 8 Experimental Reactor at operating conditions with coolant flow. Wet experiment (with coolant, at power) validation benchmarks are presented using both deterministic (Serpent-Griffin) and Monte-Carlo (OpenMC-Cardinal) multiphysics frameworks coupled with thermal-hydraulic solvers in MOOSE. Reactivity coefficient measurements including fuel temperature, isothermal temperature, and power coefficients show good agreement with experiments, with discrepancies within experimental uncertainty. Reactivity worth experiments for coolant, samarium, and xenon poisoning are reproduced with differences under 200 pcm. Comparison between Serpent-Griffin and OpenMC-Cardinal frameworks reveal multiphysics coupling introduces positive reactivity effects (100-200 pcm) compared to uniform temperature and density fields at nominal operating conditions. Comparison between Serpent-Griffin and reference Serpent solution shows that power distributions maintain consistent radial and axial peaking behavior. All models, assumptions, thermophysical and thermomechanical properties, and material definitions are thoroughly documented with cited references; model inputs and model generating scripts are stored in the snapReactors GitHub repository.

SNAP↗
DOE OSTI2017

Closed-channel culture system for efficient and reproducible differentiation of human pluripotent stem cells into islet cells

Human pluripotent stem cells (hPSCs) are thought to be a promising cell-source solution for regenerative medicine due to their indefinite proliferative potential and ability to differentiate to functional somatic cells. However, issues remain with regard to achieving reproducible differentiation of cells with the required functionality for realizing human transplantation therapies and with regard to reducing the potential for bacterial or fungal contamination. To meet these needs, we have developed a closed-channel culture device and corresponding control system. Uniformly-sized spheroidal hPSCs aggregates were formed inside wells within a closed-channel and maintained continuously throughout the culture process. Functional islet-like endocrine cell aggregates were reproducibly induced following a 30-day differentiation protocol. Our system shows an easily scalable, novel method for inducing PSC differentiation with both purity and functionality. - Highlights: • A simple, closed-channel-based, semi-automatic culture system is proposed. • Uniform cell aggregate formation and culture is realized in microwell structure. • Functional islet cells are successfully induced following 30-plus-day protocol. • System requires no daily medium replacement and reduces contamination risk.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2026

How should reproducibility be approached in plastic recycling?

With the growing importance of developing new and improved methodologies for plastic recycling, conducting reproducible research and ensuring that results are transferable across labs are increasingly important. This Voices article reflects on how academia and industry view the path forward for strengthening reproducibility to advance science and enable a circular plastics economy.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2018

Repeatability and Reproducibility of Radiomic Features: A Systematic Review

An ever-growing number of predictive models used to inform clinical decision making have included quantitative, computer-extracted imaging biomarkers, or “radiomic features.” Broadly generalizable validity of radiomics-assisted models may be impeded by concerns about reproducibility. We offer a qualitative synthesis of 41 studies that specifically investigated the repeatability and reproducibility of radiomic features, derived from a systematic review of published peer-reviewed literature.

62 RADIOLOGY AND NUCLEAR MEDICINE↗
DOE OSTI2023

Transfer learning as a method to reproduce high-fidelity non-local thermodynamic equilibrium opacities in simulations

Simulations of high-energy density physics often need non-local thermodynamic equilibrium opacity data. These data, however, are expensive to produce at relatively low fidelity. It is even more so at high fidelity such that the opacity calculations can contribute 95 % of the total computation time. This proportion can even reach large proportions. Neural networks can be used to replace the standard calculations of low-fidelity data, and the neural networks can be trained to reproduce artificial, high-fidelity opacity spectra. In this work, it is demonstrated that a novel neural network architecture trained to reproduce high-fidelity krypton spectra through transfer learning can be used in simulations. Further, it is demonstrated that this can be done while achieving a relative per cent error of the peak radiative temperature of the hohlraum of approximately 1 % to 4 % while achieving a 19.4 $\times$ speed up.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2017

Is the Supporting Information the Venue for Reproducibility and Transparency?

Making research data, software, and data processing tools readily available to the public could significantly enhance the impact of scientific publications. Openness and transparency could address reproducibility concerns and accelerate scientific progress.(1, 2) While archives and repositories would be preferable for securing data in the long term, the Supporting Information (SI) already allocates significant space to provide auxiliary files, links, and essential information needed to make scientific findings immediately reproducible as well as data processing protocols and numerical procedures executable. Furthermore, the SI document could ensure that data that might not fit in the tight confines of a journal article lives online even when laboratories move on to other projects, close, and lose track of their data.

Rudshteyn, Benjamin↗
DOE OSTI2016

Effects of thermochemical treatment on CuSbS 2 photovoltaic absorber quality and solar cell reproducibility

CuSbS 2 is a promising nontoxic and earth-abundant photovoltaic absorber that is chemically simpler than the widely studied Cu 2 ZnSnS 4 . However, CuSbS 2 photovoltaic (PV) devices currently have relatively low efficiency and poor reproducibility, often due to suboptimal material quality and insufficient optoelectronic properties. To address these issues, here we develop a thermochemical treatment (TT) for CuSbS 2 thin films, which consists of annealing in Sb 2 S 3 vapor followed by a selective KOH surface chemical etch. The annealed CuSbS 2 films show improved structural quality and optoelectronic properties, such as stronger band-edge photoluminescence and longer photoexcited carrier lifetime. These improvements also lead to more reproducible CuSbS 2 PV devices, with performance currently limited by a large cliff-type interface band offset with CdS contact. Altogether, these results point to the potential avenues to further increase the performance of CuSbS 2 thin film solar cell, and the findings can be transferred to other thin film photovoltaic technologies.

14 SOLAR ENERGY↗