Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “reproducible science”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4
All sourcesNASA NTRSDOE OSTI
NASA NTRS2016

Persistent Identifiers Implementation in EOSDIS

This presentation provides the motivation for and status of implementation of persistent identifiers in NASA's Earth Observation System Data and Information System (EOSDIS). The motivation is provided from the point of view of long-term preservation of datasets such that a number of questions raised by current and future users can be answered easily and precisely. A number of artifacts need to be preserved along with datasets to make this possible, especially when the authors of datasets are no longer available to address users questions. The artifacts and datasets need to be uniquely and persistently identified and linked with each other for full traceability, understandability and scientific reproducibility. Current work in the Earth Science Data and Information System (ESDIS) Project and the Distributed Active Archive Centers (DAACs) in assigning Digital Object Identifiers (DOI) is discussed as well as challenges that remain to be addressed in the future.

persistent identifiers↗
NASA NTRS1994

Vibration Isolation Technology (VIT) ATD Project

A fundamental advantage for performing material processing and fluid physics experiments in an orbital environment is the reduction in gravity driven phenomena. However, experience with manned spacecraft such as the Space Transportation System (STS) has demonstrated a dynamic acceleration environment far from being characterized as a 'microgravity' platform. Vibrations and transient disturbances from crew motions, thruster firings, rotating machinery etc. can have detrimental effects on many proposed microgravity science experiments. These same disturbances are also to be expected on the future space station. The Microgravity Science and Applications Division (MSAD) of the Office of Life and Microgravity Sciences and Applications (OLMSA), NASA Headquarters recognized the need for addressing this fundamental issue. As a result an Advanced Technology Development (ATD) project was initiated in the area of Vibration Isolation Technology (VIT) to develop methodologies for meeting future microgravity science needs. The objective of the Vibration Isolation Technology ATD project was to provide technology for the isolation of microgravity science experiments by developing methods to maintain a predictable, well defined, well characterized, and reproducible low-gravity environment, consistent with the needs of the microgravity science community. Included implicitly in this objective was the goal of advising the science community and hardware developers of the fundamental need to address the importance of maintaining, and how to maintain, a microgravity environment. This document will summarize the accomplishments of the VIT ATD which is now completed. There were three specific thrusts involved in the ATD effort. An analytical effort was performed at the Marshall Space Flight Center to define the sensitivity of selected experiments to residual and dynamic accelerations. This effort was redirected about half way through the ATD focusing specifically on the sensitivity of protein crystals to a realistic orbital environment. The other two thrusts of the ATD were performed at the Lewis Research Center. The first was to develop technology in the area of reactionless mechanisms and robotics to support the eventual development of robotics for servicing microgravity science experiments. This activity was completed in 1990. The second was to develop vibration isolation and damping technology providing protection for sensitive science experiments. In conjunction with the this activity, two workshops were held. The results of these were summarized and are included in this report.

Lubomski, Joseph F.↗
DOE OSTI2025

Breaking the reproducibility barrier with standardized protocols for plant–microbiome research

Inter-laboratory replicability is crucial yet challenging in microbiome research. Leveraging microbiomes to promote soil health and plant growth requires understanding underlying molecular mechanisms using reproducible experimental systems. In a global collaborative effort involving five laboratories, we aimed to help advance reproducibility in microbiome studies by testing our ability to replicate synthetic community assembly experiments. Our study compared fabricated ecosystems constructed using two different synthetic bacterial communities, the model grass Brachypodium distachyon, and sterile EcoFAB 2.0 devices. All participating laboratories observed consistent inoculum-dependent changes in plant phenotype, root exudate composition, and final bacterial community structure, where Paraburkholderia sp. OAS925 could dramatically shift microbiome composition. Comparative genomics and exudate utilization linked the pH-dependent colonization ability of Paraburkholderia, which was further confirmed with motility assays. The study provides detailed protocols, benchmarking datasets, and best practices to help advance replicable science and inform future multi-laboratory reproducibility studies.

Novak, Vlastimil↗
DOE OSTI2026

Using AI to Reproduce Neutrino Cross Section Analysis - Prototyping the Neutrino Discovery Platform

The Neutrino Discovery Platform (NDP) aims to accelerate DUNE-era science by making the neutrino program's existing datasets analyzable through fast, reproducible, and auditable workflows. We report a working version of two of its layers, data curation and agentic orchestration, built and tested end to end on MINERvA open data. The guiding lesson throughout is that a cross section is a measurement, and not just a plotted shape, only if it carries a defensible systematic-uncertainty budget, a trustworthy unfolding, and a reproducible record. Using a single medium-energy playlist pair from the MINERvA open-data release (about $2.05\times10^{17}$ protons on target of data), we first reproduced the shapes of two published charged-current inclusive $\nu_\mu$ measurements through a complete extraction ladder: selection, background subtraction, D'Agostini unfolding, efficiency correction, and flux normalization. These shape-level reproductions ran and tracked the published results, but they lacked the systematic-uncertainty machinery that defines a MINERvA cross section. To supply it, we vendored and built the MINERvA Analysis Toolkit and developed a many-universe systematic-uncertainty tool that produces a portable covariance artifact, a parallel event-loop runner, and a per-run auditability harness. Validated against a published covariance release, the toolchain reproduces the released statistical, flux, and muon-energy-scale terms and shows that they account for roughly 63\% of the total variance, with the remainder unreleased. Using this same infrastructure, we then performed a measurement of our own design, the hadronic recoil-energy distribution of low-energy ($E_\nu<2.5$~GeV) charged-current inclusive events, and found data/simulation shape agreement of $\chi^2/\mathrm{ndf}=1.26$. Together these results show that the platform supports original physics and not only reproductions.

Breaux, Auto [Tulane U. (main)]↗
DOE OSTI2021

Resiliency in numerical algorithm design for extreme scale simulations

Here this work is based on the seminar titled ‘Resiliency in Numerical Algorithm Design for Extreme Scale Simulations’ held March 1–6, 2020, at Schloss Dagstuhl, that was attended by all the authors. Advanced supercomputing is characterized by very high computation speeds at the cost of involving an enormous amount of resources and costs. A typical large-scale computation running for 48 h on a system consuming 20 MW, as predicted for exascale systems, would consume a million kWh, corresponding to about 100k Euro in energy cost for executing 10 23 floating-point operations. It is clearly unacceptable to lose the whole computation if any of the several million parallel processes fails during the execution. Moreover, if a single operation suffers from a bit-flip error, should the whole computation be declared invalid? What about the notion of reproducibility itself: should this core paradigm of science be revised and refined for results that are obtained by large-scale simulation? Naive versions of conventional resilience techniques will not scale to the exascale regime: with a main memory footprint of tens of Petabytes, synchronously writing checkpoint data all the way to background storage at frequent intervals will create intolerable overheads in runtime and energy consumption. Forecasts show that the mean time between failures could be lower than the time to recover from such a checkpoint, so that large calculations at scale might not make any progress if robust alternatives are not investigated. More advanced resilience techniques must be devised. The key may lie in exploiting both advanced system features as well as specific application knowledge. Research will face two essential questions: (1) what are the reliability requirements for a particular computation and (2) how do we best design the algorithms and software to meet these requirements? While the analysis of use cases can help understand the particular reliability requirements, the construction of remedies is currently wide open. One avenue would be to refine and improve on system- or application-level checkpointing and rollback strategies in the case an error is detected. Developers might use fault notification interfaces and flexible runtime systems to respond to node failures in an application-dependent fashion. Novel numerical algorithms or more stochastic computational approaches may be required to meet accuracy requirements in the face of undetectable soft errors. These ideas constituted an essential topic of the seminar. The goal of this Dagstuhl Seminar was to bring together a diverse group of scientists with expertise in exascale computing to discuss novel ways to make applications resilient against detected and undetected faults. In particular, participants explored the role that algorithms and applications play in the holistic approach needed to tackle this challenge. This article gathers a broad range of perspectives on the role of algorithms, applications and systems in achieving resilience for extreme scale simulations. The ultimate goal is to spark novel ideas and encourage the development of concrete solutions for achieving such resilience holistically.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2025

IROS 2023 Workshop Report: Draft Guidelines on Manufacturing Procedures, Test Methods and Reporting for Soft Robotics

Soft roboticists are facing challenges with reproducibility, which prevents researchers from making holistic comparisons to prior work, impedes full understanding of results, and forces the need to “reinvent the wheel,” delaying fundamental advances. Reproducibility of results is key to advancing science as well as achieving technology transfer from research laboratories to industrial applications. Recently, a discussion-based workshop dedicated to the topic, “Developing Standard Testing and Reporting Guidelines for Soft Robotics,” was held at IROS 2023 in Detroit, MI. The purpose of this document is to record a set of recommendations and voluntary draft guidelines for soft roboticists concerning fabrication/manufacturing, test procedures, and reporting, which were collectively developed at the workshop. Together as a community, we hope to improve the reporting standards of soft robotics and drive the field as a whole toward more rigorous research practices.

43 PARTICLE ACCELERATORS↗
DOE OSTI2021

Reproducibility Practice in High-Performance Computing: Community Survey Results

The integrity of science and engineering research is grounded in assumptions of rigor and transparency on the part of those engaging in such research. HPC community effort to strengthen rigor and transparency take the form of reproducibility efforts. In a recent survey of the SC conference community, we collected information about the SC Reproducibility Initiative activities. Here, we present the survey results in this paper. Results show that the reproducibility initiative activities have contributed to higher levels of awareness on the part of SC conference technical program participants, and hint at contributing to greater scientific impact for the published papers of the SC conference series. Stringent point-of-manuscript-submission verification is problematic for reasons we point out, as are inherent difficulties of computational reproducibility in HPC. Future efforts should better decouple the community educational goals from goals that specifically strengthen a research work's potential for long-term impact through reuse 5-10 years down the road.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Explaining and predicting human behavior and social dynamics in simulated virtual worlds: reproducibility, generalizability, and robustness of causal discovery methods

Ground Truth program was designed to evaluate social science modeling approaches using simulation test beds with ground truth intentionally and systematically embedded to understand and model complex Human Domain systems and their dynamics Lazer et al. (Science 369:1060–1062, 2020). Our multidisciplinary team of data scientists, statisticians, experts in Artificial Intelligence (AI) and visual analytics had a unique role on the program to investigate accuracy, reproducibility, generalizability, and robustness of the state-of-the-art (SOTA) causal structure learning approaches applied to fully observed and sampled simulated data across virtual worlds. In addition, we analyzed the feasibility of using machine learning models to predict future social behavior with and without causal knowledge explicitly embedded. In this paper, we first present our causal modeling approach to discover the causal structure of four virtual worlds produced by the simulation teams—Urban Life, Financial Governance, Disaster and Geopolitical Conflict. Our approach adapts the state-of-the-art causal discovery (including ensemble models), machine learning, data analytics, and visualization techniques to allow a human-machine team to reverse-engineer the true causal relations from sampled and fully observed data. We next present our reproducibility analysis of two research methods team’s performance using a range of causal discovery models applied to both sampled and fully observed data, and analyze their effectiveness and limitations. We further investigate the generalizability and robustness to sampling of the SOTA causal discovery approaches on additional simulated datasets with known ground truth. Our results reveal the limitations of existing causal modeling approaches when applied to large-scale, noisy, high-dimensional data with unobserved variables and unknown relationships between them. We show that the SOTA causal models explored in our experiments are not designed to take advantage from vasts amounts of data and have difficulty recovering ground truth when latent confounders are present; they do not generalize well across simulation scenarios and are not robust to sampling; they are vulnerable to data and modeling assumptions, and therefore, the results are hard to reproduce. Finally, when we outline lessons learned and provide recommendations to improve models for causal discovery and prediction of human social behavior from observational data, we highlight the importance of learning data to knowledge representations or transformations to improve causal discovery and describe the benefit of causal feature selection for predictive and prescriptive modeling.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Planetary normal mode computation: Parallel algorithms, performance and reproducibility

This report is an extension of work entitled “Computing planetary interior normal modes with a highly parallel polynomial filtering eigensolver.” by Shi et al., [1] originally presented at the SC18 conference. A highly parallel polynomial filtered eigensolver was developed and exploited to calculate the planetary normal modes. The proposed method is ideally suited for computing interior eigenpairs for large-scale eigenvalue problems as it greatly enhances memory and computational efficiency. In this article, the second-order finite element method is used to further improve the accuracy as only the first-order finite element method was deployed in the previous work. The parallel algorithm, its parallel performance up to 20k processors, and the great computational accuracy are illustrated. The reproducibility of the previous work was successfully performed on the Student Cluster Competition at the SC19 conference by several participant teams using a completely different Mars-model dataset on different clusters. Both weak and strong scaling performances of the reproducibility by the participant teams were impressive and encouraging. The analysis and reflection of their results are demonstrated and future direction is discussed.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Interlaboratory Reproducibility of Contour Method Data in a High Strength Aluminum Alloy

The contour method for residual stress measurement has seen significant development, but an experimental reproducibility study utilizing physical samples has not been published. A double-blind reproducibly study is reported, having scope beginning with EDM cutting and ending with residual stress calculation. A reinforced I-beam sample geometry is identified for its unique residual stress profile when extracted from residual stress bearing quenched aluminum bar (7050-T74). Contour measurements are prescribed on a midplane of symmetry with dimensions 24.0 mm by 50.0 mm. Fourteen identically prepared samples are fabricated from a single long bar with well characterized and uniform residual stress. Five samples throughout the bar are identified for planning measurements to validate sample uniformity and overall suitability of the residual stress field. The planning measurements employ a range of techniques: contour method, neutron diffraction, and hole-drilling. Eight samples are distributed to an international group of participants to execute their standard measurement practice. A double-blind process is followed to provide anonymity. Results are provided by eight participants: six being self-similar and two being quite different, the latter set aside as outliers. An average residual stress field is established from non-outlying results and the spatial distribution of reproducibility standard deviation is determined. The average stress field ranges from -60 to 70 MPa and the reproducibility standard deviation averages 8.1 MPa on the measurement plane. The average reproducibility standard deviation is about 3 × larger for points within 1.0 mm of plane boundaries (17.6 MPa) than for the remaining points (6.1 MPa). Reproducibility standard deviation (among different labs) for contour method residual stress measurement is found to be very similar to repeatability standard deviation (in a single lab) reported in prior work. The reproducibility observed here, for the entire measurement process, is also similar to that found in a prior reproducibility study limited to contour method data analysis.

36 MATERIALS SCIENCE↗
DOE OSTI2022

Open Reproducible Electron Microscopy Data Analysis

Electron microscopy (EM) is a cornerstone technique in the materials and biological sciences capable of imaging structures at nano- to atomic-scale resolution. Advances in technologies mean that one acquires datasets at increasing data rates and sizes. These advancements present enormous opportunities for researchers to understand complex systems. However, processing the resulting large-scale, complex data in a reproducible and shareable way is a real challenge for researchers. The building, managing, and maintaining complex workflows in a reproducible manner requires extensive knowledge in several areas outside the researchers’ core skill sets, such as software engineering, data science, and high-performance computing (HPC). Our work demonstrates an innovative approach to solving these problems, enabling reproducible EM data analysis through container encapsulated pipelines. Using modern container technologies, we encapsulate processing elements and connect them using shared memory. We expose user-friendly, advanced algorithms and tools to allow end users to utilize without expert programming skills. The platform enables reproducible, scalable, shareable pipelines for the analysis and visualization of EM data. Focusing on interoperability, we leverage the DOE and other agencies’ existing investments to provide a powerful software platform for EM data analysis.

Harris, Christopher↗
DOE OSTI2023

nmrrr : A Reproducible Workflow for Binning and Visualizing NMR Spectra From Environmental Samples

Nuclear magnetic resonance (NMR) spectroscopy is a useful tool for detection and identification of molecular structural information, with increasing applications in environmental sciences. NMR instrument outputs are however heterogeneous and require extensive post-processing, creating barriers to their use and application by non-specialists. Here, we report on a new open-source R package, nmrrr, that processes and visualizes spectral data obtained from one-dimensional solution-state and solid-state NMR experiments; the package also performs relevant calculations commonly applied in natural organic matter communities, such as computing the relative abundance of various functional groups. We document the package's installation, dependencies, and functions; and provide a standard workflow for processing NMR data. This package is currently available on CRAN and GitHub, and community contributions are welcome.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Integrating HPC, AI, and Workflows for Scientific Data Analysis: Report from Dagstuhl Seminar 23352

The Dagstuhl Seminar 23352, titled “Integrating HPC, AI, and Workflows for Scientific Data Analysis,” held from August 27 to September 1, 2023, was a significant event focusing on the synergy between High-Performance Computing (HPC), Artificial Intelligence (AI), and scientific workflow technologies. The seminar recognized that modern Big Data analysis in science rests on three pillars: workflow technologies for reproducibility and steering, AI and Machine Learning (ML) for versatile analysis, and HPC for handling large data sets. These elements, while crucial, have traditionally been researched separately, leading to gaps in their integration. The seminar aimed to bridge these gaps, acknowledging the challenges and opportunities at the intersection of these technologies. The event highlighted the complex interplay between HPC, workflows, and ML, noting how ML has increasingly been integrated into scientific workflows, thereby enhancing resource demands and bringing new requirements to HPC architectures, like support for GPUs and iterative computations. The seminar also addressed the challenges in adapting HPC for large-scale ML tasks, including in areas like deep learning, and the need for workflow systems to evolve to leverage ML in data analysis fully. Moreover, the seminar explored how ML could optimize scientific workflow systems and HPC operations, such as through improved scheduling and fault tolerance. A key focus was on identifying prestigious use cases of ML in HPC and understanding their unique, unmet requirements. The stochastic nature of ML and its impact on the reproducibility of data analysis on HPC systems was also a topic of discussion.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS2009

Electroactive Polymer (EAP) Actuators for Future Humanlike Robots

Human-like robots are increasingly becoming an engineering reality thanks to recent technology advances. These robots, which are inspired greatly by science fiction, were originated from the desire to reproduce the human appearance, functions and intelligence and they may become our household appliance or even companion. The development of such robots is greatly supported by emerging biologically inspired technologies. Potentially, electroactive polymer (EAP) materials are offering actuation capabilities that allow emulating the action of our natural muscles for making such machines perform lifelike. There are many technical issues related to making such robots including the need for EAP materials that can operate as effective actuators. Beside the technology challenges these robots also raise concerns that need to be addressed prior to forming super capable robots. These include the need to prevent accidents, deliberate harm, or their use in crimes. In this paper, the potential EAP actuators and the challenges that these robots may pose will be reviewed.

biomimetics↗
DOE OSTI2024

Plant science history manuscript GitHub repo

This archive contains codes, notebooks, and other information for reproducing the study that links to this file. The original files are in the plant_sci_hist Github repository which will likely be updated when the code base is adopted for other projects. Thus, this archive is provided so a frozen version used for publishing the plant science history manuscript that references this file.

Botany/trend/methods↗
DOE OSTI2022

Improving the performance of PM6 donor polymer by random ternary copolymerization of BDD and DTBT segments

Developing high-performance wide bandgap polymer donors to match the rapidly growing non-fullerene small molecule acceptors (NF-SMAs) is of great importance to further improve the photovoltaic performances of organic solar cells (OSCs). PM6 and D18 are two leading wide bandgap donor polymers, but their performances are highly sensitive to the molecular weights, which have distinct impact on the morphology and thereby the charge generation and recombination processes. Here, in this work, we reported the random ternary copolymerization of PM6 and D18 segments to develop random terpolymers with finely tuned photoelectronic properties and optimized blend morphology for high-performance OSCs. The resulting random terpolymers (PMD-5, PMD-10, PMD-15 and PMD-20) exhibited finely tuned absorptions, energy levels and blend morphology. The optimal morphology of PMD-15:L8-BO contributed to the fast exciton diffusion and dissociation, efficient charge extraction with suppressed recombination. Thus, the power conversion efficiency was improved from 17.22% of the control PM6 devices and 17.47% of the control D18 devices to 18.12% of PMD-15 based devices, which is the top performance of random terpolymers. Moreover, the PMD-15 exhibited much higher batch-to-batch reproducibility at a broad molecular weight range than that of PM6 and D18, showing promising for practical applications.

36 MATERIALS SCIENCE↗
NASA NTRSDate not supplied

Open Science Practices at the Community Coordinated Modeling Center

Open Science is defined as “the principle and practice of making research products and processes available to all, while respecting diverse cultures, maintaining security and privacy, and fostering collaborations, reproducibility, and equity” by Federal Agencies. The CCMC has been practicing open science based on FAIR (Findable, Accessible, Interoperable and Reusable) principle by providing access to the state-of -the art space science and space weather models to users around the world through various simulation services such as Runs-on-Request, Instant Runs, Real time runs on iSWA system. The CCMC also provides a wide range of tools and framework to help users easily utilize modeled data. One of the tools is the official NASA open-sourced software called Kamodo. Kamodo allows users to work with complex space weather models and data with little or no coding experience. Additionally, to support transparent model validation efforts, the CCMC is providing an integrated and flexible framework called CAMEL. CAMEL allows users to seamlessly compare model outputs with observational data sets. Currently, we are working on a user-friendly database of the papers and research that used CCMC services, so that the future users will have open access to previously performed research by other users and its details. In this presentation, we will show the open tools and resources provided by the CCMC. Furthermore, we will share our new efforts to support open data and open science results.

Ja Soon Shim↗
DOE OSTI2024

Towards FAIR Workflows for Federated Experimental Sciences

A de-centralized, peer-to-peer AI metadata framework is demonstrated which can enable end-to-end metadata & lineage tracking for distributed Machine Learning pipelines spanning edge, High Performance Computing, and cloud environments. With a specific example of end-to-end microscopy algorithm and datasets, the proposed method shows how to enable reproducibility, audit trail, provenance of metadata artifacts. The emerging needs of automation in experimental sciences, ML-centric workflows, and FAIR metadata management across federated compute environments is addressed.

machine learning↗