Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Trustworthiness”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

213 records · Page 12
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Data Curation for Machine Learning Applied to Geothermal Power Plant Operational Data for GOOML: Geothermal Operational Optimization with Machine Learning: Preprint

Geothermal Operational Optimization with Machine Learning (GOOML) is a transferable and extensible component-based geothermal asset modeling framework that considers complex steamfield relationships and identifies optimization prospects using a data-driven approach to physics-guided, data-centric machine learning. This framework has been used to develop digital twins that provide steamfield operators with operational environments to analyze and understand historical and forecasted power production, explore new steamfield configuration possibilities, and seek optimal asset management in real world applications. To create, test, and apply the GOOML framework, diverse time-series datasets spanning multiple years were sourced from various geothermal power plant components within several complex real-world geothermal operations. These operations are based in the United States and New Zealand and include a variety of technologies, end-uses and configurations, collectively covering nearly all relevant operating conditions for modern geothermal fields. Datasets were acquired from multiple sources to ensure that machine learning experiments generalized properly to various operating conditions. It was found that the data varied in quality, format, and completeness. To ensure consistency between the various datasets, a standardized data curation process was developed to reliably streamline data preparation. This paper will discuss best practices as learned from the GOOML data curation process which takes the following steps: 1) acquisition of large quantities of data from power plant operators, 2) digestion of data to gain an initial understanding of what is included, 3) data transformation, which includes converting the data into a standardized machine-readable format so that they can be visualized, quality checked, and cleaned, 4) quality assurance and quality control, involving identification of significant data gaps and apparent anomalies through mapping of data features to real world componentry via the GOOML historical model, followed by discussion with modelers and power plant operators to identify additional data needs and to resolve issues, 5) use in machine learning algorithms, and 6) repetition of steps one through five until all data needs are met and data are deemed suitable for producing trustworthy modeling results which may be disseminated, ideally along with the curated dataset. This iterative process is focused on improving the quality of the data rather than tuning machine learning model parameters and supports a shift towards data-centric AI as a means to improving real-world applicability of geothermal machine learning projects.

access↗
DOE OSTI2022

Application of Orthogonal Defect Classification for Software Reliability Analysis

The modernization of existing and new nuclear power plants with digital instrumentation and control systems (DI&C) is a recent and highly trending topic. However, there lacks strong consensus on best-estimate reliability methodologies by both the United States (U.S.) Nuclear Regulatory Commission (NRC) and the industry. This has resulted in hesitation for further modernization projects until a more unified methodology is realized. In this work, we develop an approach called Orthogonal-defect Classification for Assessing Software Reliability (ORCAS) to quantify probabilities of various software failure modes in a DI&C system. The method utilizes accepted industry methodologies for software quality assurance that are also verified by experimental or mathematical formulations. In essence, the approach combines a semantic failure classification model with a reliability growth model to predict (and quantify) the potential failure modes of a DI&C software system. The semantic classification model is used to address the question: How do latent defects in software contribute to different software failure root causes? The use of reliability growth models is then used to address the question: Given the connection between latent defects and software failure root causes, how can we quantify the reliability of the software? A case study was conducted on a representative I&C platform (ChibiOS) running a smart sensor acquisition software developed by Virginia Commonwealth University (VCU). The testing and evidence collection guidance in ORCAS was applied, and defects were uncovered in the software. Qualitative evidence, such as condition coverage, was used to gauge the completeness and trustworthiness of the assessment while quantitative evidence was used to determine the software failure probabilities. The reliability of the software was then estimated and compared to existing operational data of the sensor device. It is demonstrated that by using ORCAS, a semantic reasoning framework can be developed to justify if the software is reliable (or unreliable) while still leveraging the strength of the existing methods.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2022

Application of Orthogonal Defect Classification for Software Reliability Analysis

The modernization of existing and new nuclear power plants with digital instrumentation and control systems (DI&C) is a recent and highly trending topic. However, there lacks strong consensus on best-estimate reliability methodologies by both the United States (U.S.) Nuclear Regulatory Commission (NRC) and the industry. This has resulted in hesitation for further modernization projects until a more unified methodology is realized. In this work, we develop an approach called Orthogonal-defect Classification for Assessing Software Reliability (ORCAS) to quantify probabilities of various software failure modes in a DI&C system. The method utilizes accepted industry methodologies for software quality assurance that are also verified by experimental or mathematical formulations. In essence, the approach combines a semantic failure classification model with a reliability growth model to predict (and quantify) the potential failure modes of a DI&C software system. The semantic classification model is used to address the question: How do latent defects in software contribute to different software failure root causes? The use of reliability growth models is then used to address the question: Given the connection between latent defects and software failure root causes, how can we quantify the reliability of the software? A case study was conducted on a representative I&C platform (ChibiOS) running a smart sensor acquisition software developed by Virginia Commonwealth University (VCU). The testing and evidence collection guidance in ORCAS was applied, and defects were uncovered in the software. Qualitative evidence, such as condition coverage, was used to gauge the completeness and trustworthiness of the assessment while quantitative evidence was used to determine the software failure probabilities. The reliability of the software was then estimated and compared to existing operational data of the sensor device. It is demonstrated that by using ORCAS, a semantic reasoning framework can be developed to justify software reliability (or unreliability) while still leveraging the strength of the existing methods.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2022

Smart sensor for online situational awareness in power grids

Waveforms in power grids typically reveal a certain pattern with specific features and peculiarities driven by the system operating conditions, internal and external uncertainties, etc. This prompts an observation of different types of waveforms at the measurement points (substations). An innovative next-generation smart sensor technology includes a measurement unit embedded with sophisticated analytics for power grid online surveillance and situational awareness. The smart sensor brings additional levels of smartness into the existing phasor measurement units (PMUs) and intelligent electronic devices (IEDs). It unlocks the full potential of advanced signal processing and machine learning for online power grid monitoring in a distributed paradigm. Within the smart sensor are several interconnected units for signal acquisition, feature extraction, machine learning-based event detection, and a suite of multiple measurement algorithms where the best-fit algorithm is selected in real-time based on the detected operating condition. Embedding such analytics within the sensors and closer to where the data is generated, the distributed intelligence mechanism mitigates the potential risks to communication failures and latencies, as well as malicious cyber threats, which would otherwise compromise the trustworthiness of the end-use applications in distant control centers. The smart sensor achieves a promising classification accuracy on multiple classes of prevailing conditions in the power grid and accordingly improves the measurement quality across the power grid.

Dehghanian, Payman↗
DOE OSTI2021

On the Stochastic Stability of Deep Markov Models

Deep Markov models (DMM) are generative models which are scalable and expressive generalization of Markov models for representation, learning, and inference problems. DMMs using deep neural networks to parametrize the transition of Markov probability distributions have recently been shown to provide more expressiveness in modeling sequential data and dynamical system responses. However, the fundamental stochastic stability guarantees of such models have not been thoroughly investigated. In this paper, we present a rigorous analytical method to prove the necessary and sufficient conditions of DMM's stochastic stability. This task is achieved by spectral analysis of the efficiently computed Jacobians of probabilistic maps modeled by deep neural networks. We make theoretical connections between the eigenvalues of neural network's weights and the different activation function types used on the stability and overall dynamic behavior of DMMs with Gaussian distributions. We empirically substantiate our theoretical results on stochastic stability and eigenvalue spectra via several numerical experiments. Formal stability guarantees of DMMs can substantially improve their robustness and trustworthiness, necessary for reliable use in safety-critical real-world applications.

Drgona, Jan↗
DOE OSTI2023

Opportunities and Challenges for Remote Microreactor Operations

The nuclear industry is developing new advanced reactor technologies, and many companies are embracing this advancement by pursuing the development of microreactors. The term microreactor generally refers to a nuclear reactor with an operating power of 20 MW or less. The power range of microreactors makes them appealing for many use cases, such as powering remote communities, mining sites, and military bases. Most of the microreactor designs being pursued will incorporate remote facility operations into the final product. However, no framework has yet been developed to determine what remote operations systems require for reliable, resilient, and secure operation of a microreactor. This work identifies the challenges unique to remote operations and monitoring for microreactors specifically regarding instrumentation and control, communication methods, regulatory requirements, and operational policies. The types of commands and sensor measurements that must be transmitted between the facilities as well as methods for verifying the trustworthiness of these signals are assessed. This work evaluates the security, reliability, and performance requirements that must be met when considering the selection of communication hardware and protocols for use in remote operations. Also, an assessment was performed to study how remote operations fit within current regulatory requirements and what may need to be updated in regulatory policy to allow for remote operation. Finally, the operational contingencies unique to remote operations that must be in place for response to abnormal events are identified. This paper details these challenges and research opportunities to provide a foundation for the design of remote operation systems.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2023

Uncertainty Quantified Machine Learning for Street Level Flooding Predictions in Norfolk, Virginia

Everyday citizens, emergency responders, and critical infrastructure can be dramatically affected by street flooding caused by the increasing tidal levels and major storm events magnified by climate change. Low-level flooding, such as nuisance flooding, continues to increase in frequency, especially in cities like Norfolk, Virginia, which can expect nearly 200 flooding events by 2050 [1]. Recently, machine learning (ML) models have been leveraged to produce real-time predictions based on local weather and geographic conditions. However, ML models are known to produce unusual results when presented with data that varies from their training set. For decision-makers to determine the trustworthiness of the model's predictions, ML models need to quantify their prediction uncertainty. This study applies Deep Quantile Regression (DQR) to a previously published, LSTM-based model for hourly water depth predictions [2], and analyzes its out-of-distribution (OOD) performance.

Goldenberg, Steven↗
DOE OSTI2024

Dynamic Modeling and Characterization of Nuclear-grade Graphite

Idaho National Labs serves as the spearhead for many innovative energy solutions to the world's energy crisis. One such solution is the INL's Microreactor which is designed to deploy to extreme/remote environments where other sources of power are either unavailable or unreliable. In order to best design these energy solutions for their operational environments, it is crucial to understand how the design, components, and materials will respond to the environmental conditions. One key material in these innovative designs is a nuclear-grade graphite known as PCEA. This study examines the behavior of PCEA graphite under dynamic loading, similar to that which may occur in extreme environments. The objective is to characterize the dynamic behavior and produce an accurate, reliable constitutive model suitable for use in simulation tools such as INL's MOOSE. Graphite specimens were tested using a Split Hopkinson Pressure Bar (SHPB) to administer the dynamic compressive load. The SHPB was charged at various pressures to produce a range of strain rates on the material in compression. Data was acquired via strain gauges on the SHPB setup, from which stress, strain, and time data were collected. Analysis revealed the stress-strain behavior of the material as well as insights into the material behavior's relationship to strain rate. Further work must continue to characterize the various other dynamic behaviors of the material which will combine to create a substantially trustworthy constitutive model for this grade of nuclear-grade graphite. Ultimately, this will allow for realistic simulation of the material in reactor designs, allowing for prediction of design weaknesses and leading to improved designs for increased resilience, security, and reliability.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2025

CHEMREASONER: Heuristic Search over a Large Language Model’s Knowledge Space using Quantum-Chemical Feedback

The discovery of new catalysts is essential for the design of new and more efficient chemical processes in order to transition to a sustainable future. We introduce an AI-guided computational screening framework unifying linguistic reasoning with quantum-chemistry based feedback from 3D atomistic representations. Our approach formulates catalyst discovery as an uncertain environment where an agent actively searches for highly effective catalysts via the iterative combination of large language model (LLM)-derived hypotheses and atomistic graph neural network (GNN)-derived feedback. Identified catalysts in intermediate search steps undergo structural evaluation based on spatial orientation, reaction pathways, and stability. Scoring functions based on adsorption energies and barriers steer the exploration in the LLM's knowledge space toward energetically favorable, high-efficiency catalysts. We introduce planning methods that automatically guide the exploration without human input, providing competitive performance against expert-enumerated chemical descriptor-based implementations. By integrating language-guided reasoning with computational chemistry feedback, our work pioneers AI-accelerated, trustworthy catalyst discovery.

artificial intelligence↗
DOE OSTI2025

Using a Large Language Model for Accurate Technical Language Generation in the Predictive Maintenance of Circulating Water Systems in Nuclear Power Plants

Machine learning (ML) methods for predictive maintenance (PdM) are emerging as effective proactive strategies for diagnosing equipment degradation and enabling effective decision-making. However, explainability and trustworthiness of artificial intelligence are two salient challenges that need to be addressed for wider deployment of these technologies in nuclear power plants (NPPs). Large language models (LLMs) offer a unique approach to tackle these challenges by explaining PdM, work orders, diagnosis results, and ML algorithms to users, who may not be familiar with ML and PdM in general. Moreover, by dynamically retrieving relevant information from technical documents and evaluating factuality of LLM generation, the accuracy and relevance of LLM generations can be improved. This work demonstrates using LLMs to explain the causes and consequences of circulating water system failures based on multiyear NPP work orders. This work tests the capability of multimodal LLM approaches in explaining the differences in the circulating water system from both the Salem and Hope Creek NPPs using both text and image resources. This work also demonstrates the use of multimodal LLMs in describing the diagnosis tab of a predictive maintenance software named VIsualization for PrEdictive maintenance Recommendation (VIPER) to users.

22 - GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2026

REFSafE: A RAG-Enabled Framework for Predictive Risk Analysis and Automated Safety Report Generation in Mission-Critical Environments

Operational safety in mission-critical environments requires AI systems that are accurate, interpretable, and resistant to hallucination. We present an agentic Retrieval-Augmented Generation (RAG) framework, REFSafe, for grounded hazard analysis and automated safety report generation. The system integrates Large Language Models (LLMs) with structured operational data, historical incident repositories, policy documents, and external authoritative sources. Through iterative agentic reasoning, the framework retrieves, verifies, and synthesizes evidence prior to generation, enforcing citation-backed outputs with explicit source attribution (documents, links, and prior events) to ensure traceability and trust. To mitigate hallucinations and unsupported claims, all risk assessments and forecasts are constrained to retrieved evidence, with confidence signals derived from retrieval relevance and source consistency. A transparent pipeline enables subject matter experts (SMEs) to validate predictions, and provide structured feedback, forming a continuous performance calibration loop. Preliminary deployment demonstrates improved reliability in hazard detection and safety/vulnerability report generation. This work advances trustworthy, evidence-grounded AI for predictive safety intelligence in mission-critical operations.

Das, Sanjay [ORNL] (ORCID:0009000542591915)↗
DOE OSTI2026

Toward the Neutrino Discovery Platform: An Auditable, Uncertainty-Bearing Toolchain for MINERvA Open-Data Cross-Section Analysis

The Neutrino Discovery Platform (NDP) aims to accelerate DUNE-era science by making the neutrino program's existing datasets analyzable through fast, reproducible, and auditable workflows. We report a working version of two of its layers, data curation and agentic orchestration, built and tested end to end on MINERvA open data. The guiding lesson throughout is that a cross section is a measurement, and not just a plotted shape, only if it carries a defensible systematic-uncertainty budget, a trustworthy unfolding, and a reproducible record. Using a single medium-energy playlist pair from the MINERvA open-data release (about $2.05\times10^{17}$ protons on target of data), we first reproduced the shapes of two published charged-current inclusive $\nu_\mu$ measurements through a complete extraction ladder: selection, background subtraction, D'Agostini unfolding, efficiency correction, and flux normalization. These shape-level reproductions ran and tracked the published results, but they lacked the systematic-uncertainty machinery that defines a MINERvA cross section. To supply it, we vendored and built the MINERvA Analysis Toolkit and developed a many-universe systematic-uncertainty tool that produces a portable covariance artifact, a parallel event-loop runner, and a per-run auditability harness. Validated against a published covariance release, the toolchain reproduces the released statistical, flux, and muon-energy-scale terms and shows that they account for roughly 63\% of the total variance, with the remainder unreleased. Using this same infrastructure, we then performed a measurement of our own design, the hadronic recoil-energy distribution of low-energy ($E_\nu<2.5$~GeV) charged-current inclusive events, and found data/simulation shape agreement of $\chi^2/\mathrm{ndf}=1.26$. Together these results show that the platform supports original physics and not only reproductions.

Breaux, Auto [Tulane U. (main)]↗
DOE OSTI2025

Applications of explainable artificial intelligence in renewable energy research

Researchers in renewable energy are applying deep learning (DL) to a variety of problems from diverse renewable energy domains, such as biofuels, wind, solar, power systems, buildings, vehicles, and transportation systems. Improvements in accuracy may be demonstrated using DL in laboratory settings. However, the lack of interpretability of DL models poses a practical limitation to their utility in advancing scientific knowledge and in the deployment of DL models in safety-critical energy systems. In this article, we discuss explainable artificial intelligence (XAI) as one pathway toward more interpretable DL models. We explore a brief timeline of U.S. national laboratory interest in XAI, an overview and taxonomy of methods in the field of XAI, and a selection of applications across renewable energy research domains. We conclude by highlighting pivotal areas where XAI can accelerate innovation in artificial intelligence for renewable energy research and other essential future directions.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Uncertainty propagation in feed-forward neural network models

We develop new uncertainty propagation methods for feed-forward neural network architectures with leaky ReLU activation functions subject to random perturbations in the input vectors. In particular, we derive analytical expressions for the probability density function (PDF) of the neural network output and its statistical moments as a function of the input uncertainty and the parameters of the network, i.e., weights and biases. A key finding is that an appropriate linearization of the leaky ReLU activation function yields accurate statistical results even for large perturbations in the input vectors. This can be attributed to the way information propagates through the network. We also propose new analytically tractable Gaussian copula surrogate models to approximate the full joint PDF of the neural network output. To validate our theoretical results, we conduct Monte Carlo simulations and a thorough error analysis on a multi-layer neural network representing a nonlinear integro-differential operator between two polynomial function spaces. Our findings demonstrate excellent agreement between the theoretical predictions and Monte Carlo simulations.

MLP networks↗
DOE OSTI2026

What Is the Agent Doing? Visualizing Agentic AI Querying Workflows

We explore how visualizations can help users understand what an AI agent is doing as it builds and runs queries over data. As part of the LinkQ system, a natural language interface for querying knowledge graphs with a large language model (LLM), we designed two complementary views: A State Diagram that shows where the agent is within a larger workflow, and a Live Action Display that gives real-time updates about the agent's current task. In a study with 14 practitioners, we found that these visuals helped participants build stronger mental models of the agent's behavior while also increasing their confidence in the system. However, we also observed that users sometimes trusted incorrect outputs simply because the agent appeared to be doing the "right" thing. Our findings point to both the value and risk of visualizing agent behavior in interactive AI systems.

97 MATHEMATICS AND COMPUTING↗