Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Reproducibility”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,423 records · Page 79

A publicly available PyTorch-$\mathrm{ABAQUS}$ $\mathrm{UMAT}$ deep-learning framework for level-set plasticity

Here this paper introduces a publicly available PyTorch-ABAQUS deep-learning framework of a family of plasticity models where the yield surface is implicitly represented by a scalar-valued function. In particular, our focus is to introduce a practical framework that can be deployed for engineering analysis that employs a user-defined material subroutine (UMAT/VUMAT) for ABAQUS, which is written in FORTRAN. To accomplish this task while leveraging the back-propagation learning algorithm to speed up the neural-network training, we introduce an interface code where the weights and biases of the trained neural networks obtained via the PyTorch library can be automatically converted into a generic FORTRAN code that can be a part of the UMAT/VUMAT algorithm. To enable third-party validation, we purposely make all the data sets, source code used to train the neural-network-based constitutive models, and the trained models available in a public repository. Furthermore, the practicality of the workflow is then further tested on a dataset for anisotropic yield function to showcase the extensibility of the proposed framework. A number of representative numerical experiments are used to examine the accuracy, robustness and reproducibility of the results generated by the neural network models.

36 MATERIALS SCIENCE↗

Proliferation of Faulty Materials Data Analysis in the Literature

As a group of subject matter experts in X-ray photoelectron spectroscopy (XPS), we write this letter to raise awareness of the epidemic of poor and incorrect materials data analysis in the Literature. This is a chronic and growing problem with very undesirable consequences. This issue may contribute to what has been identified as a “reproducibility crisis”1,2 that was the subject of a recent U.S. National Academies study. We are using XPS as one example of what we observe to be a systemic issue.

36 MATERIALS SCIENCE↗

P-Mart: Interactive Analysis of Ion Abundance Global Proteomics Data

The use of mass-spectrometry-based techniques for global protein profiling of biomedical or environmental experiments has become a major focus in research centered on biomarker discovery; however, one of the most important issues recently highlighted in the new era of omics data generation is the ability to perform analyses in a robust and reproducible manner. This has been hypothesized to be one of the issues hindering the ability of clinical proteomics to successfully identify clinical diagnostic and prognostic biomarkers of disease. P-Mart (https://pmart.labworks.org) is an interactive web-based software environment that enables domain scientists to perform quality-control processing, statistics, and exploration of large-complex proteomics data sets without requiring statistical programming. P-Mart is developed in a manner that allows researchers to perform analyses via a series of modules, explore the results using interactive visualization, and finalize the analyses with a collection of output files documenting all stages of the analysis and a report to allow reproduction of the analysis.

59 BASIC BIOLOGICAL SCIENCES↗

Snakes on a Spaceship—An Overview of Python in Heliophysics

Computational analysis has become ubiquitous within the heliophysics community. However, community standards for peer review of codes and analysis have lagged behind these developments. This absence has contributed to the reproducibility crisis, where inadequate analysis descriptions and loss of scientific data have made scientific studies difficult or impossible to replicate. The heliophysics community has responded to this challenge by expressing a desire for a more open, collaborative set of analysis tools. Here, this article summarizes the current state of these efforts and presents an overview of many of the existing Python heliophysics tools. It also outlines the challenges facing community members who are working toward the goal of an open, collaborative, Python heliophysics toolkit and presents guidelines that can ease the transition from individualistic data analysis practices to an accountable, communalistic environment.

79 ASTRONOMY AND ASTROPHYSICS↗

Deep transfer operator learning for partial differential equations under conditional shift

Transfer learning enables the transfer of knowledge gained while learning to perform one task (source) to a related but different task (target), hence addressing the expense of data acquisition and labelling, potential computational power limitations and dataset distribution mismatches. Here, we propose a new transfer learning framework for task-specific learning (functional regression in partial differential equations) under conditional shift based on the deep operator network (DeepONet). Task-specific operator learning is accomplished by fine-tuning task-specific layers of the target DeepONet using a hybrid loss function that allows for the matching of individual target samples while also preserving the global properties of the conditional distribution of the target data. Inspired by conditional embedding operator theory, we minimize the statistical distance between labelled target data and the surrogate prediction on unlabelled target data by embedding conditional distributions onto a reproducing kernel Hilbert space. We demonstrate the advantages of our approach for various transfer learning scenarios involving nonlinear partial differential equations under diverse conditions due to shifts in the geometric domain and model dynamics. Our transfer learning framework enables fast and efficient learning of heterogeneous tasks despite considerable differences between the source and target domains.

42 ENGINEERING↗

Weak baselines and reporting biases lead to overoptimism in machine learning for fluid-related partial differential equations

One of the most promising applications of machine learning in computational physics is to accelerate the solution of partial differential equations (PDEs). The key objective of machine-learning-based PDE solvers is to output a sufficiently accurate solution faster than standard numerical methods, which are used as a baseline comparison. Here, we first perform a systematic review of the ML-for-PDE-solving literature. Out of all of the articles that report using ML to solve a fluid-related PDE and claim to outperform a standard numerical method, we determine that 79% (60/76) make a comparison with a weak baseline. Second, we find evidence that reporting biases are widespread, especially outcome reporting and publication biases. We conclude that ML-for-PDE-solving research is overoptimistic: weak baselines lead to overly positive results, while reporting biases lead to under-reporting of negative results. To a large extent, these issues seem to be caused by factors similar to those of past reproducibility crises: researcher degrees of freedom and a bias towards positive results. We call for bottom-up cultural changes to minimize biased reporting as well as top-down structural reforms to reduce perverse incentives for doing so.

97 MATHEMATICS AND COMPUTING↗

A case study: Comparing testers for means and variances in a balanced incomplete block design

In this article, we present a gauge R & R study that is different from the standard setup in several ways. A balanced incomplete block design (BIBD) was used to collect the data because the measurement is destructive. Besides addressing reproducibility, differences in repeatability (i.e., tester variances) were also of interest. Here we use Bayesian methods to analyze the study data to address whether the three tester means and variances are similar. We also consider what happens if the BIBD structure of the data and possibly different tester variances are ignored in analyzing the study data.

42 ENGINEERING↗

Recommendations for harmonized reporting of radiation Dosimetry by adoption of Compatibility in Irradiation Research Protocols Expert Roundtable (CIRPER)

Here, in radiobiology, radiation physics, and related areas, there is growing interest in transitioning from cesium-based irradiators to X-ray technologies. Therefore, it makes it imperative for reproducibility purposes, for researchers to provide specifications regarding device parameters, machine calibration details, and experimental setup when innovative X-ray technology is being employed. Several groups have already identified a glaring lack in completeness of critical information on experimental setups, equipment, and physical parameters in published radiation biology studies. The Alternative Technologies Meta Study Report by Brookhaven National Laboratory (BNL) examined 456 manuscripts published between 2017-2021. Only 64 of the reviewed studies (14.0%) consistently reported methods that could allow for replication of their studies. However, 85% provided information on the manufacturer, the irradiator model and the absorbed radiation dose. Approximately 66% provided information on the energy spectra and the dose rate, but only one third disclosed the filter used. Less than 15% of the reviewed studies reported information on calibration and field geometry, specifically field size and distance from source, which are critical parameters in determining the absorbed radiation dose.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

The case for free and open source software in research and scholarship

Free and open source software (FOSS) is any computer program released under a licence that grants users rights to run the program for any purpose, to study it, to modify it, and to redistribute it in original or modified form. Here, our aim is to explore the intersection between FOSS and computational reproducibility. We begin by situating FOSS in relation to other ‘open’ initiatives, and specifically open science, open research, and open scholarship. In this context, we argue that anyone who actively contributes to the research process today is a computational researcher, in that they use computers to manage and store information. We then provide a primer to FOSS suitable for anyone concerned with research quality and sustainability—including researchers in any field, as well as support staff, administrators, publishers, funders, and so on. Next, we illustrate how the notions introduced in the primer apply to resources for scientific computing, with reference to the GNU Scientific Library as a case study. We conclude by discussing why the common interpretation of ‘open source’ as ‘open code’ is misplaced, and we use this example to articulate the role of FOSS in research and scholarship today.

97 MATHEMATICS AND COMPUTING↗

A three-phase workflow for general and expressive representations of nondeterminism in HPC applications

Nondeterminism is an increasingly entrenched property of high-performance computing (HPC) applications and has recently been shown to seriously hamper debugging and reproducibility efforts. Additionally, tools for addressing the nondeterministic debugging problem have emerged, but they do not provide methods for systematically cataloging the nondeterminism in a given application. We propose a three-phase workflow for representing executions of nondeterministic message passing interface programs as event graphs, quantifying their structural similarity with graph kernels, and applying machine learning techniques to investigate shared properties across applications. We present an empirical study comparing two graph kernels’ suitability for this task and propose future uses of the methodology.

97 MATHEMATICS AND COMPUTING↗

SBML Level 3: an extensible format for the exchange and reuse of biological models

Systems biology has experienced dramatic growth in the number, size, and complexity of computational models. To reproduce simulation results and reuse models, researchers must exchange unambiguous model descriptions. We review the latest edition of the Systems Biology Markup Language (SBML), a format designed for this purpose. A community of modelers and software authors developed SBML Level 3 over the past decade. Its modular form consists of a core suited to representing reaction-based models and packages that extend the core with features suited to other model types including constraint-based models, reaction-diffusion models, logical network models, and rule-based models. The format leverages two decades of SBML and a rich software ecosystem that transformed how systems biologists build and interact with models. More recently, the rise of multiscale models of whole cells and organs, and new data sources such as single-cell measurements and live imaging, has precipitated new ways of integrating data with models. We provide our perspectives on the challenges presented by these developments and how SBML Level 3 provides the foundation needed to support this evolution.

59 BASIC BIOLOGICAL SCIENCES↗

Hysteresis and degaussing of H1 dipole magnets

Hysteresis was observed in the H1 dipole magnets in CBETA SX line. This study is the first step in order to understand and mitigate the effects this has on field reproducibility. Ansys Maxwell 19.2 was used to model the MS1DIP05 magnet with a hysteretic core. The parameter space for a degaussing procedure using the trim coils was explored. It was found that significant degaussing can be obtained with a decaying ringing waveform of magnitudes above 10 A, so long as the frequency ƒ and decay constant w were chosen such that ƒ >> w. The circuit properties of the magnet were simulated and explored using a sinusoidal voltage input. It was found that for low frequencies ƒ << 1 Hz, the magnet behaves as a series LR circuit with R = (28.3 ± 0.5) mΩ and L = (5.7 ± 0.1) mH. At higher frequencies, hysteretic effects become significant, and resistance becomes proportional to the frequency due to hysteretic losses. A drop and saturation in inductance for higher frequencies was observed in the simulation data. Numerical instabilities were found for excitations below small voltages V < 50 mV. Experimental results on degaussing and current corrections were collected with Hall probe measurements. Good qualitative agreement with simulation results was found, but due to B-H curve details quantitative comparisons were inaccurate. Two procedures, MaxOp and MaxNull, were designed to produce repeatable operating field values to within 0.1 G.

43 PARTICLE ACCELERATORS↗

Evaluated Nuclear Data Requirements in Support of Fission and Fusion Applications [Slides]

This presentation touches on thermal scattering cross section and scattering lengths including criticality benchmarks (fission). Secondly, development and completeness of charged ­particle libraries: criticality benchmarks (fission), neutron source (fission/fusion), simulation benchmarks and light elements generation (fusion). Additionally, this presentation details on the covariances (methodology and completeness) with adjustment and optimization of nuclear data libraries (fission/fusion). Finally nuclear data reproducibility and conclusions are discussed.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

LANL MPCL Measurement System Analysis: AMPTEC 630ES Igniter Tester / Ohmmeter

This comprehensive analysis aimed to evaluate the measurement capability of the AMPTEC 630ES Explosive Safety Igniter Tester, a critical tool for organizations engaged in the measurement of High Explosive components and related Electro-Explosive devices. With a history of notable calibration issues involving specific model numbers, the Metrology Program and Calibration Laboratory (MPCL) initiated an extensive Measurement System Analysis. The analysis's primary focus was to thoroughly quantify the various sources of measurement variability intrinsic to the calibration process. It centered on two pivotal aspects: repeatability (the variation observed when the same operator uses the same instrument) and reproducibility (the variation seen when different operators perform measurements). A secondary objective was to determine whether equipment usage and age had any impact on the instrument's performance.

47 OTHER INSTRUMENTATION↗

Dependable classical-quantum computing systems engineering

Increasing evidence suggests quantum computing (QC) complements traditional High-Performance Computing (HPC) by leveraging its unique capabilities, leading to the emergence of a new, hybrid paradigm, QHPC. However, this integration introduces new challenges, with dependability–defined by reproducibility, resiliency, and security and privacy–emerging as a central concern for building trustworthy systems that provide an advantage to the users. This paper proposes a framework for dependable QHPC system design, organized around these three pillars. We identify integration challenges, anticipate roadblocks, and highlight productive synergies across QC, HPC, cloud platforms, and network security. Drawing from both classical computing principles and quantum-specific insights, we present a roadmap for co-design that supports robust hybrid architectures. Our approach offers concrete metrics for assessing dependability, provides design guidance for engineers working at the QC-HPC interface, and surfaces new engineering questions around complexity, scale, and fault tolerance. Ultimately, designing for dependability is key to realizing practical, scalable QHPC systems and accelerating the broader quantum ecosystem capable of translating quantum promises into actual application delivery.

HPC↗

Kernelized approaches to streaming compression of scientific data

In this paper three algorithms are developed for the streaming compression of scientific data. The algorithms presented are reliant on the theory of vector-valued reproducing kernel Hilbert spaces and operator valued kernel. Further, the scientific data is modeled as a snapshot of time dependent vector field F(x, t) over a manifold M and the recovery of the data is framed as a learning problem. These processes are then appropriately modified and ana lyzed for the streaming scenario in which data is generated without the ability to revisit past entries.

97 MATHEMATICS AND COMPUTING↗

Measuring sleep habits without using a diary: the sleep timing questionnaire

STUDY OBJECTIVES: To develop a single-administration instrument yielding equivalent measures of sleep to those obtained from a formal (2-week) sleep diary. DESIGN & SETTING: A single-administration Sleep riming Questionnaire (STQ) is described (and reproduced in the Appendix). Test-retest reliability was examined in 40 subjects who were given the STQ on two occasions separated by less than 1 year. Convergent validity was measured both by comparing STO-derived measures with objective measures derived from wrist actigraphy (n=23) and by comparing STQ-derived measures with other subjective measures derived from a detailed 2-week sleep diary in two nonoverlapping samples (n=101, 93). Correlations of STQ measures with age and momingness-eveningness (chronotype) were also examined. SUBJECTS: The analyses used sample sizes of 40, 23, 101, and 93 (both genders, overall age range 20y-89y). Most subjects were healthy volunteers; some Study 4 subjects were patients (enrolled in research protocols). RESULTS: Test-retest reliability for the STQ was demonstrated for estimates of bedtime (r = 0.705, p < 0.001) and waketime (r = 0.826, p < 0.001). Convergent validity using wrist actigraphy was demonstrated by correlations of 0.592 (p < 0.005) for bedtime, and of 0.769 (p < 0.001) for waketime. Diary studies indicated STQ bedtime and waketime data to be highly correlated (at about 0.8) with those obtained from a formal 2-week sleep diary. The STQ also provided data on estimated sleep latency and wake after sleep onset (WASO), which correlated reliably (at about 0.7) with average nightly ratings of these variables from a 2-week sleep diary. Mean estimated values of sleep latency and WASO from the two instruments were within 1 minute of each other. ST-derived bedtimes and waketimes correlated with both age and chronotype in the expected direction (older subjects earlier, morning types earlier). CONCLUSION: The STQ may be a reliable valid measure of sleep timing that could provide a time-efficient alternative to traditional sleep diaries.

Validation Studies↗

Development of a cw-laser-based cavity-ringdown sensor aboard a spacecraft for trace air constituents

The progress in the development of a sensor for the detection of trace air constituents to monitor spacecraft air quality is reported. A continuous-wave (cw), external-cavity tunable diode laser centered at 1.55 micrometers is used to pump an optical cavity absorption cell in cw-cavity ringdown spectroscopy (cw-CRDS). Preliminary results are presented that demonstrate the sensitivity, selectivity and reproducibility of this method. Detection limits of 2.0 ppm for CO, 2.5 ppm for CO2, 1.8 ppm for H2O, 19.4 ppb for NH3, 7.9 ppb for HCN and 4.0 ppb for C2H2 are calculated.

NASA Discipline Life Sciences Technologies↗