Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Machine Learning Models”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Development of a machine learning model for polyethylene pyrolysis using a detailed reaction mechanism

Waste plastics have recently received significant attention as the issue of waste generation continues to increase. Thermal conversion processes, such as pyrolysis and gasification, are attractive potential technologies for utilizing waste plastics and reducing overall waste generation. Efficient utilization of plastics requires a detailed understanding of the conversion process such as pyrolysis and gasification. However, a mechanistic understanding of these processes lead to large and complex kinetic schemes that are not suited for large-scale and long-time simulation methods. Currently, most modeling approaches for pyrolysis and gasification rely on globally lumped, simplified kinetic schemes that provide results that are classified by their product type and not individual species, which limit the level of fidelity achieved via modeling. A machine learning (ML) model has been developed for the primary reactions of high-density polyethylene (HDPE) in an attempt to increase computational efficiency while still maintaining a high level of detail and accuracy. The ML model is trained on a detailed reaction mechanism containing 42 total species and 737 chemical reactions. A DeepONet branch and trunk architecture was adopted to train the model using time-steps relevant to computational fluid dynamics simulations. The ML used physics-informed loss functions to ensure mass conservation. The surrogate model has been deployed in simple MFiX CFD simulations, single particle and an experimental drop tube reactor, and has shown promising performance compared to the original scheme.

Houston, Ross↗
DOE OSTI2022

Fusion RF Modeling Machine Learning (FusionML_RF) v1.0

FusionML_RF consists of multiple codes and trained machine learning (ML) models that perform low-cost output modeling from the Genray-CQL3D. Three machine learning techniques (multilayer perceptron, random forest, and Gaussian process) provide fast surrogate models for lower hybrid current drive (LHCD) simulations. For example, completing a single GENRAY/CQL3D simulation without radial diffusion of fast electrons requires several minutes of wall-clock time. On the other hand, these ML models achieve ~ms of inference time with high accuracy across the input parameter space. This software collection consists of multiple components. (1) codes that use ML methods and precomputed Genray-CQL3D simulation output to build regression models that enable approximate computations of Genray-CLQ3D outputs from arbitrary but physically meaningful input parameters (surrogate modeling); (2) three trained models created by the team, using a database of 16,000+ GENRAY/CQL3D simulations, to study the performance of ML models for surrogate modeling; (3) codes that load the trained models and simulation data, and then compute mean squared error between the models' predictions and the ground truth of simulation output data. This collection is being made available in conjunction with a scientific publication about the work to promote reusability and provide an artifact of the scientific work.

Bai, Zhe↗
DOE OSTI2022

Predicting Future Laboratory Fault Friction Through Deep Learning Transformer Models

Machine learning models using seismic emissions as input can predict instantaneous fault characteristics such as displacement and friction in laboratory experiments, and slow slip in Earth. Here, we address whether the seismic/acoustic emission (AE) from laboratory experiments contains information about future frictional behavior. The approach uses a convolutional encoder-decoder containing a transformer model in the latent space, similar to models used for natural language processing. We test the model limits using progressively larger AE input time windows and progressively larger output friction time windows. The results demonstrate that very near-term friction predictions are indeed contained in the AE signal, and predictions are progressively worse farther into the future. The future predictions by the model of impending failure in the near-term are remarkably robust. This first effort predicting future fault frictional behavior with machine learning will aid in guiding efforts for applications in Earth.

58 GEOSCIENCES↗
DOE OSTI2023

Dynamic Model Agnostic Reliability Evaluation of Machine-Learning Models Integrated in Instrumentation & Control Systems

In recent years, the field of machine learning (ML), specifically neural networks, has grown significantly and has spurred research in its applicability to digital instrumentation and control systems (DI&C). While ML models have shown promise in operational contexts, the trustworthiness of using such algorithms has not been adequately assessed. Failures of ML integrated systems are not well understood, and the lack of comprehensive risk modeling can degrade the trustworthiness in these systems. In recent reports by the National Institute for Standards and Technology (NIST) [1] and the Nuclear Regulatory Commission (NRC) [2], they indicate that trustworthiness in ML is a critical barrier and will play a vital role in the safe, accountable, and secure operation of intelligent systems. Thus, in this work, we demonstrate a dynamic model-agnostic method to quantify the relative reliability of AI/ML predictions by incorporating out-of-distribution (OOD) detection on the training dataset. It is well documented that most ML algorithms excel at interpolation (or near-interpolation) tasks but experience significant performance degradation at extrapolation. The method, referenced as the Laplacian distributed decay for reliability (LADDR), determines the difference between the operational and training datasets which can used to the relative reliability of AI/ML predictions. LADDR is then demonstrated on a feedforward neural network based digital twin used for the prediction of safety significant factors during a loss-of-flow transient. LADDR is used to demonstrate how training data can be used as evidence to support the relative reliability of ML/AI predictions enhancing the overall trustworthiness of the system.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2022

Calibrating hypersonic turbulence flow models with the HIFiRE-1 experiment using data-driven machine-learned models.

In this paper we study the efficacy of combining machine-learning methods with projection-based model reduction techniques for creating data-driven surrogate models of computationally expensive, high-fidelity physics models. Such surrogate models are essential for many-query applications e.g., engineering design optimization and parameter estimation, where it is necessary to invoke the high-fidelity model sequentially, many times. Surrogate models are usually constructed for individual scalar quantities. However there are scenarios where a spatially varying field needs to be modeled as a function of the model’s input parameters. Here we develop a method to do so, using projections to represent spatial variability while a machine-learned model captures the dependence of the model’s response on the inputs. The method is demonstrated on modeling the heat flux and pressure on the surface of the HIFiRE-1 geometry in a Mach 7.16 turbulent flow. The surrogate model is then used to perform Bayesian estimation of freestream conditions and parameters of the SST (Shear Stress Transport) turbulence model embedded in the high-fidelity (Reynolds-Averaged Navier–Stokes) flow simulator, using shock-tunnel data. The paper provides the first-ever Bayesian calibration of a turbulence model for complex hypersonic turbulent flows. We find that the primary issues in estimating the SST model parameters are the limited information content of the heat flux and pressure measurements and the large model-form error encountered in a certain part of the flow.

42 ENGINEERING↗
DOE OSTI2023

Protecting and Defending against Autonomous Control Systems and Digital Twin Cyber Attacks: Response Strategy for Hyperparameter attacks of Digital Twin Machine Learning Models in Nuclear Power Plants (Final)

Navigating through the complex tapestry of technological advancements, "Response Strategy for Hyperparameter attacks of Digital Twin Machine Learning Model in Nuclear Power Plants" stands at the intersection of cybersecurity and nuclear power plant operations, embarking on a journey through the intricacies of securing digital twins against malicious cyber activities. As nuclear power plants progressively integrate digital twin technology and machine learning models to optimize operations and ensure system reliability, they inadvertently expose themselves to a new spectrum of vulnerabilities, notably in the realm of hyperparameter attacks. Hyperparameters, integral in machine learning model tuning and optimal performance of digital twins, have emerged as a target for adversaries aiming to destabilize the predictive capabilities and therefore, the operational accuracy of these digital entities within critical infrastructures like nuclear plants. This paper, therefore, meticulously threads the needle through the development of a robust response strategy, poised to shield these digital reflections against calculated hyperparameter manipulations, ensuring that the digital twin can effectively and securely function as a reliable proxy for its physical counterpart. The ensuing sections delve into the orchestrated maelstrom of multi-rate time-changing intelligent coordinated hyperparameter attacks and the implementation of event-triggered predictive control, laying down a structured, predictive, and responsive framework that safeguards the nexus where the digital and physical realms of nuclear power plants coalesce. The operational integrity of digital twins in nuclear power plants depends critically on the security of machine learning hyperparameters. This study makes two different contributions. First, a decision-based idea known as a multi-rate time changing intelligent coordinated hyperparameter attack is put forth. In this attack, many hyperparameters are repeatedly changed using both random and intelligent optimal techniques by the attacker. These assaults introduce varied rates at different attack steps, compromise various amounts of hyperparameters, and improve stealth and flexibility. Second, a technique is developed for event triggered predictive control to rapidly respond to potential hyperparameter attacks. This control integrates a sliding window framework, retaining a history of previous data points and employing linear regression to predict the next data point from the current dataset. The control gain K is determined using the Lyapunov-Krasovskii method, and subsequently, an action is developed. Finally, the outcome of the simulation demonstrates the viability of the proposed method for defending nuclear power plant digital twins from hyperparameter attacks.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2026

Expanding the Domain of Applicability of Machine Learning Models with Limited Data for Drug Property Prediction

Accurate machine learning models for predicting small molecule interactions with biological targets are essential for therapeutic discovery, biothreat response, and computational drug design, but their performance is often limited for understudied targets with sparse experimental data. To address this challenge, we developed and evaluated methods to improve molecular property prediction under low-data conditions, using the NimA-related kinase (NEK) family as a proof-of-concept. This work focused on two complementary goals within the ATOM Modeling PipeLine (AMPL) and the Generative Molecular Design (GMD) loop: expanding model applicability through transfer learning, representation learning, feature scaling, sampling strategies, and active-learning-inspired compound selection; and enabling efficient virtual screening to prioritize compounds that balance predicted activity, design objectives, and synthetic accessibility.

organic↗
NASA NTRS2023

Diagnosis of Antarctic Blowing Snow Properties Using MERRA-2 Reanalysis with a Machine Learning Model

This paper presents the work on using a machine learning model to diagnose Antarctic blowing snow (BLSN) properties with the Modern Era Retrospective analysis for Research and Applications v2 (MERRA-2) data. We adopt the random forest classifier for BLSN identification and the random forest regressor for BLSN optical depth and height diagnosis. BLSN properties observed from the Cloud-Aerosol Lidar and Infrared Pathfinder Satellite Observation (CALIPSO) are used as the truth for training the model. Using MERRA-2 fields such as snow age, surface elevation and pressure, temperature, specific humidity, and temperature gradient at the 2m level, and wind speed at the 10m level as input, reasonable results are achieved. Hourly blowing snow property diagnostics are generated with the trained model. Using the year 2010 as an example, it is shown that the Antarctic BLSN frequency is much higher over East than West Antarctica. High frequency months are from April to September, during which BLSN frequency exceeds 20% over East Antarctica. For May 2010, the BLSN snow frequency in the region is as high as 37%. Due to the suppression by strong surface-based inversions, larger values of BLSN height and optical depth are usually limited to the coastal regions, wherein the strength of surface-based inversions is weaker.

Antarctic↗
DOE OSTI2022

Machine learning models for rat multigeneration reproductive toxicity prediction

Reproductive toxicity is one of the prominent endpoints in the risk assessment of environmental and industrial chemicals. Due to the complexity of the reproductive system, traditional reproductive toxicity testing in animals, especially guideline multigeneration reproductive toxicity studies, take a long time and are expensive. Therefore, machine learning, as a promising alternative approach, should be considered when evaluating the reproductive toxicity of chemicals. We curated rat multigeneration reproductive toxicity testing data of 275 chemicals from ToxRefDB (Toxicity Reference Database) and developed predictive models using seven machine learning algorithms (decision tree, decision forest, random forest, k-nearest neighbors, support vector machine, linear discriminant analysis, and logistic regression). A consensus model was built based on the seven individual models. An external validation set was curated from the COSMOS database and the literature. The performances of individual and consensus models were evaluated using 500 iterations of 5-fold cross-validations and the external validation data set. The balanced accuracy of the models ranged from 58% to 65% in the 5-fold cross-validations and 45%–61% in the external validations. Prediction confidence analysis was conducted to provide additional information for more appropriate applications of the developed models. The impact of our findings is in increasing confidence in machine learning models. We demonstrate the importance of using consensus models for harnessing the benefits of multiple machine learning models (i.e., using redundant systems to check validity of outcomes). While we continue to build upon the models to better characterize weak toxicants, there is current utility in saving resources by being able to screen out strong reproductive toxicants before investing in vivo testing. The modeling approach (machine learning models) is offered for assessing the rat multigeneration reproductive toxicity of chemicals. Our results suggest that machine learning may be a promising alternative approach to evaluate the potential reproductive toxicity of chemicals.

consensus model↗
DOE OSTI2023

An Explainable Machine-Learning Model for Compensatory Reserve Measurement: Methods for Feature Selection and the Effects of Subject Variability

Tracking vital signs accurately is critical for triaging a patient and ensuring timely therapeutic intervention. The patient’s status is often clouded by compensatory mechanisms that can mask injury severity. The compensatory reserve measurement (CRM) is a triaging tool derived from an arterial waveform that has been shown to allow for earlier detection of hemorrhagic shock. However, the deep-learning artificial neural networks developed for its estimation do not explain how specific arterial waveform elements lead to predicting CRM due to the large number of parameters needed to tune these models. Alternatively, we investigate how classical machine-learning models driven by specific features extracted from the arterial waveform can be used to estimate CRM. More than 50 features were extracted from human arterial blood pressure data sets collected during simulated hypovolemic shock resulting from exposure to progressive levels of lower body negative pressure. A bagged decision tree design using the ten most significant features was selected as optimal for CRM estimation. This resulted in an average root mean squared error in all test data of 0.171, similar to the error for a deep-learning CRM algorithm at 0.159. By separating the dataset into sub-groups based on the severity of simulated hypovolemic shock withstood, large subject variability was observed, and the key features identified for these sub-groups differed. This methodology could allow for the identification of unique features and machine-learning models to differentiate individuals with good compensatory mechanisms against hypovolemia from those that might be poor compensators, leading to improved triage of trauma patients and ultimately enhancing military and emergency medicine.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2024

Learning together: Towards foundation models for machine learning interatomic potentials with meta-learning

Abstract The development of machine learning models has led to an abundance of datasets containing quantum mechanical (QM) calculations for molecular and material systems. However, traditional training methods for machine learning models are unable to leverage the plethora of data available as they require that each dataset be generated using the same QM method. Taking machine learning interatomic potentials (MLIPs) as an example, we show that meta-learning techniques, a recent advancement from the machine learning community, can be used to fit multiple levels of QM theory in the same training process. Meta-learning changes the training procedure to learn a representation that can be easily re-trained to new tasks with small amounts of data. We then demonstrate that meta-learning enables simultaneously training to multiple large organic molecule datasets. As a proof of concept, we examine the performance of a MLIP refit to a small drug-like molecule and show that pre-training potentials to multiple levels of theory with meta-learning improves performance. This difference in performance can be seen both in the reduced error and in the improved smoothness of the potential energy surface produced. We therefore show that meta-learning can utilize existing datasets with inconsistent QM levels of theory to produce models that are better at specializing to new datasets. This opens new routes for creating pre-trained, foundation models for interatomic potentials.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2025

Balancing Trade-offs: Adaptive Differential Privacy in Interpretable Machine Learning Models

In the advancing field of machine learning, balancing accuracy, interpretability, and privacy represents a significant challenge. The problem is exacerbated by the widespread deployment of pre-trained models locally in diverse applications, which could lead to various amounts of privacy leakage. Conventional Differential Privacy strategies, in which uniform noises are applied to model gradients, guarantee data privacy at the expense of accuracy and interpretability. This paper introduces a Feature-Sensitive Adaptive Differential Privacy (FADP) framework with a unique noise-adding strategy. Noises are adaptively added based on feature importance clustering, where important features are considered for interpretability. By employing a unique masking technique, FADP selectively preserves crucial features with minimal noise interference, maintaining accuracy while enhancing interpretability. The FADP framework addresses the limitations of traditional DP methods by preserving critical channels and improving interpretability — a vital requirement in machine learning applications that demand transparency in model decisions. Through comprehensive testing, FADP is shown to balance the trade-offs among accuracy, privacy, and interpretability, marking a substantial advancement in the field of privacy-preserving machine learning.

Farhad Riya, Farhin [University of Tennessee, Knox↗
DOE OSTI2023

Machine learning model explanation apparatus and methods

Explanation apparatus and methods are described. In one aspect, an explanation apparatus includes processing circuity configured to access a source instance which has been classified by a machine learning model; create associations of the source instance with a plurality of training instances; and process the associations of the source instance and the training instances to identify a first subset of the training instances which have less relevance to the classification decision of the source instance by the machine learning model compared with a second subset of the training instances; and an interface configured to communicate information to a user, and wherein the processing circuitry is configured to control the user interface to communicate the second subset of the training instances to the user as evidence to explain the classification of the source instance by the machine learning model.

Arendt, Dustin↗
DOE OSTI2023

Dynamic Model Agnostic Reliability Evaluation of Machine-Learning Models Integrated in Instrumentation & Control Systems

In recent years, the field of data-driven neural network-based machine learning (ML) algorithms has grown significantly and spurred research in its applicability to instrumentation and control systems. While they are promising in operational contexts, the trustworthiness of such algorithms is not adequately assessed. Failures of ML-integrated systems are poorly understood; the lack of comprehensive risk modeling can degrade the trustworthiness of these systems. In recent reports by the National Institute for Standards and Technology, trustworthiness in ML is a critical barrier to adoption and will play a vital role in intelligent systems' safe and accountable operation. Thus, in this work, we demonstrate a real-time model-agnostic method to evaluate the relative reliability of ML predictions by incorporating out-of-distribution detection on the training dataset. It is well documented that ML algorithms excel at interpolation (or near-interpolation) tasks but significantly degrade at extrapolation. This occurs when new samples are "far" from training samples. The method, referred to as the Laplacian distributed decay for reliability (LADDR), determines the difference between the operational and training datasets, which is used to calculate a prediction's relative reliability. LADDR is demonstrated on a feedforward neural network-based model used to predict safety significant factors during different loss-of-flow transients. LADDR is intended as a "data supervisor" and determines the appropriateness of well-trained ML models in the context of operational conditions. Ultimately, LADDR illustrates how training data can be used as evidence to support the trustworthiness of ML predictions when utilized for conventional interpolation tasks.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Physics-informed machine-learning model of temperature evolution under solid phase processes

We model temperature dynamics during Shear Assisted Proccess Extrusion (ShAPE), a solid phase process that plasticizes feedstock with a rotating tool and subsequently extrudes it into a consolidated tube, rod, or wire. Control of temperature is critical during ShAPE processing to avoid liquefaction, ensure smooth extrusion, and develop desired material properties in the extruded products. Accurate modeling of the complicated thermo-mechanical feedbacks between process inputs, material temperature, and heat generation presents a significant barrier to predictive modeling and process design. In particular, connecting micro-structural scale mechanisms of heat generation to macro-scale predictions of temperature can become computationally intractable. In this work we use a neural network (NN) model of heat generation to bridge this gap, by combining it with a simplified model of the temperature dynamics due to conduction and convection to capture the macro scale evolution of temperature. We inform the construction of the NN heat generation model using crystal plasticity simulations at the micro-structural scale to model the effects of process inputs on generation of heat. We achieved close fits of the temperature dynamics model to a diverse experimental data-set. Further, the relationships learned by the NN model between process inputs and heat generation showed qualitative agreement with those predicted by crystal plasticity simulations.

36 MATERIALS SCIENCE↗
DOE OSTI2026

A hybrid Penman-Monteith and machine learning model for simulating evapotranspiration and its components

Integrating physical processes with machine learning has advanced evapotranspiration (ET) simulation, yet most hybrid models fail to partition total ET into its components: soil evaporation (E) and vegetation transpiration (T). This study introduces Residual Neural Network–Penman–Monteith (RNN-PM), a novel hybrid dual-source ET model designed to overcome this limitation. The model synergizes the physically-based Penman–Monteith framework with three specialized residual neural networks trained to estimate key conductance parameters (canopy conductance, soil surface conductance, and aerodynamic conductance). Furthermore this explicit parameterization allows for the direct partitioning of total ET. Validation at National Ecological Observatory Network (NEON) flux sites using high-frequency partitioned E and T shows that RNN-PM reliably reproduces ET and the transpiration fraction (T/ET). For ET, the model achieves an average Kling–Gupta efficiency (KGE) of 0.89 and a root-mean-square error (RMSE) of 0.55 mm/day; for T/ET, the KGE is 0.87 with an RMSE of 0.06. Furthermore, RNN-PM demonstrates robust generalization, accurately simulating ET and its components well beyond the initial training dataset, even under extreme climatic conditions. This study extended the analysis by comparing the RNN-PM model with seven established dual-source ET models. The results indicate that RNN-PM outperforms both conventional machine learning models and purely physical process-based models in simulating ET components in most cases. Among the purely physical process-based dual-source models, those based on surface temperature decomposition showed improved performance as the leaf area index (LAI) decreased when evaluated against high-frequency ET component datasets. In contrast, the performance of conductance-based dual-source models declined with decreasing LAI. Although purely machine learning-based models can produce relatively accurate simulations of ET components, they often exhibit limited generalization capability, an issue that the RNN-PM model effectively overcomes. Ultimately, the RNN-PM model represents a significant advance in simulating ET components, offering a novel and scalable approach for improving the representation of land–atmosphere interactions in Earth system models.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Performance Evaluation of Gray-box and Machine Learning Models of a Thermal Energy Storage System with Active Insulation

An interior partition wall integrated with active thermal storage and a dynamic insulation system was built and then installed in an office building in Oak Ridge, Tennessee, TN. This smart wall, termed the Empower Wall, was equipped with embedded pipes in the building envelope core component and an additional pipe network enclosing rigid insulation to switch on and off the active insulation dynamically. The performance of the wall's contribution to cooling load reduction under different parameters has been investigated in previous publications. Aiming to be deployed into model predictive control and other optimization methods, simplified and reliable models for the developed wall and the room accommodating it are required. They are needed to characterize the properties and thermal response of both Empower Wall and building envelope, which form an essential component for accurate indoor temperature or cooling/heating demand prediction. In this study, simplified gray-box and regression models as well as machine learning model were developed and the performance of them were compared and analyzed.

Cui, Borui↗