Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Machine Learning Models”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,333 records · Page 74
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

Exploring model complexity in machine learned potentials for simulated properties

Abstract Machine learning (ML) enables the development of interatomic potentials with the accuracy of first principles methods while retaining the speed and parallel efficiency of empirical potentials. While ML potentials traditionally use atom-centered descriptors as inputs, different models such as linear regression and neural networks map descriptors to atomic energies and forces. This begs the question: what is the improvement in accuracy due to model complexity irrespective of descriptors? We curate three datasets to investigate this question in terms of ab initio energy and force errors: (1) solid and liquid silicon, (2) gallium nitride, and (3) the superionic conductor Li $$_{10}$$ 10 Ge(PS $$_{6}$$ 6 ) $$_{2}$$ 2 (LGPS). We further investigate how these errors affect simulated properties and verify if the improvement in fitting errors corresponds to measurable improvement in property prediction. By assessing different models, we observe correlations between fitting quantity (e.g. atomic force) error and simulated property error with respect to ab initio values. Graphical abstract

Rohskopf, A. (ORCID:0000000227128296)↗
DOE OSTI2021

Application of Machine Learning Techniques to an Agent-Based Model of Pantoea

Agent-based modeling (ABM) is a powerful simulation technique which describes a complex dynamic system based on its interacting constituent entities. While the flexibility of ABM enables broad application, the complexity of real-world models demands intensive computing resources and computational time; however, a metamodel may be constructed to gain insight at less computational expense. Here, we developed a model in NetLogo to describe the growth of a microbial population consisting of Pantoea . We applied 13 parameters that defined the model and actively changed seven of the parameters to modulate the evolution of the population curve in response to these changes. We efficiently performed more than 3,000 simulations using a Python wrapper, NL4Py . Upon evaluation of the correlation between the active parameters and outputs by random forest regression, we found that the parameters which define the depth of medium and glucose concentration affect the population curves significantly. Subsequently, we constructed a metamodel, a dense neural network, to predict the simulation outputs from the active parameters and found that it achieves high prediction accuracy, reaching an R 2 coefficient of determination value up to 0.92. Our approach of using a combination of ABM with random forest regression and neural network reduces the number of required ABM simulations. The simplified and refined metamodels may provide insights into the complex dynamic system before their transition to more sophisticated models that run on high-performance computing systems. The ultimate goal is to build a bridge between simulation and experiment, allowing model validation by comparing the simulated data to experimental data in microbiology.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2021

Prediction of Solar Irradiance and Photovoltaic Solar Energy Product Based on Cloud Coverage Estimation Using Machine Learning Methods

Cloud cover estimation from images taken by sky-facing cameras can be an important input for analyzing current weather conditions and estimating photovoltaic power generation. The constant change in position, shape, and density of clouds, however, makes the development of a robust computational method for cloud cover estimation challenging. Accurately determining the edge of clouds and hence the separation between clouds and clear sky is difficult and often impossible. Toward determining cloud cover for estimating photovoltaic output, we propose using machine learning methods for cloud segmentation. We compare several methods including a classical regression model, deep learning methods, and boosting methods that combine results from the other machine learning models. To train each of the machine learning models with various sky conditions, we supplemented the existing Singapore whole sky imaging segmentation database with hazy and overcast images collected by a camera-equipped Waggle sensor node. We found that the U-Net architecture, one of the deep neural networks we utilized, segmented cloud pixels most accurately. However, the accuracy of segmenting cloud pixels did not guarantee high accuracy of estimating solar irradiance. We confirmed that the cloud cover ratio is directly related to solar irradiance. Additionally, we confirmed that solar irradiance and solar power output are closely related; hence, by predicting solar irradiance, we can estimate solar power output. This study demonstrates that sky-facing cameras with machine learning methods can be used to estimate solar power output. This ground-based approach provides an inexpensive way to understand solar irradiance and estimate production from photovoltaic solar facilities.

14 SOLAR ENERGY↗
DOE OSTI2023

Efficient Reliability Analysis using Generalized Multifidelity Modeling and Explainable Active Learning

To assess the reliability of critical technologies like nuclear plants and infrastructure systems and improve the robustness of design, engineers have to quantify the uncertainties surrounding the system behavior accurately. However, the complexity of the problem can make standard reliability analysis algorithms prohibitively expensive, primarily due to the high computational cost of estimating the system response at each iteration. This cost can be greatly reduced by using multi-fidelity modeling and machine learning to build a surrogate model to replace the expensive response function. We propose a general and robust method for building surrogates from multiple Low Fidelity (LF) models coupled with machine learning to retain accuracy. Our framework first constructs “Corrected Low Fidelity models” (CLFs) by coupling a High Fidelity (HF) model inferred Gaussian Process correction term with each of the LF models. It then uses the correction terms to assign model probabilities to each of these CLFs in an explainable way before using them to assemble the final surrogate. No assumptions are made about the type of the LF models or their correlation with the HF model. The proposed surrogate modeling framework is used within the subset simulation algorithm (a variance-reduced MCMC-based reliability analysis algorithm) for enhanced efficiency. Additionally, an active learning step is added to the algorithm to adaptively decide when the surrogate is not sufficiently accurate, at which point the HF model is called and used to refine the surrogate. Through a frame buckling example, our method is shown to be highly efficient at reducing the expensive HF model calls while accurately estimating the failure probability.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Hierarchical Model Reduction Driven by Machine Learning for Parametric Advection-Diffusion-Reaction Problems in the Presence of Noisy Data

Abstract We propose a new approach to generate a reliable reduced model for a parametric elliptic problem, in the presence of noisy data. The reference model reduction procedure is the directional HiPOD method, which combines Hierarchical Model reduction with a standard Proper Orthogonal Decomposition, according to an offline/online paradigm. In this paper we show that directional HiPOD looses in terms of accuracy when problem data are affected by noise. This is due to the interpolation driving the online phase, since it replicates, by definition, the noise trend. To overcome this limit, we replace interpolation with Machine Learning fitting models which better discriminate relevant physical features in the data from irrelevant unstructured noise. The numerical assessment, although preliminary, confirms the potentialities of the new approach.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Dataset for 'Stream Temperature Predictions for River Basin Management in the Pacific Northwest and Mid-Atlantic Regions Using Machine Learning', Water 2022

This data package presents forcing data, model code, and model output for classical machine learning models that predict monthly stream water temperature as presented in the manuscript ‘Stream Temperature Predictions for River Basin Management in the Pacific Northwest and Mid-Atlantic Regions Using Machine Learning’, Water (Weierbach et al., 2022). Specifically, for input forcing datasets we include two files each generated using the BASIN-3D data integration tool (Varadharajan et al., 2022) for stations in the Pacific Northwest and Mid Atlantic Hydrologic regions. Model code (written in python with the use of jupyter notebooks) includes codes for data preprocessing, training Multiple Linear Regression, Support Vector Regression, and Extreme Gradient Boosted Tree models, and additional notebooks for analysis of model output. We include specific model output files which represent modeling configurations presented in the manuscript also presented in an hdf5 format. Together, these data make up the workflow for predictions across three scenarios (single station, regional, and predictions in unmonitored basins) presented in the manuscript and allow for reproducibility of modeling procedures.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

A machine learning aided interpretable model for rupture strength prediction in Fe-based martensitic and austenitic alloys

Abstract The class of 9–12% Cr ferritic-martensitic alloys (FMA) and austenitic stainless steels have received considerable attention due to their numerous applications in high temperature power generation industries. To design high strength steels with prolonged service life requires a thorough understanding of the long-term properties, e.g., creep rupture strength, rupture life, etc., as a function of the chemical composition and processing parameters that govern the microstructural characteristics. In this article, the creep rupture strength of both 9–12% Cr FMA and austenitic stainless steel has been parameterized using curated experimental datasets with a gradient boosting machine. The trained model has been cross validated against unseen test data and achieved high predictive performance in terms of correlation coefficient ( $$R^{2} > 0.98 $$ R 2 > 0.98 for 9–12% Cr FMA and $$R^{2} > 0.95 $$ R 2 > 0.95 for austenitic stainless steel) thus bypassing the need for additional comprehensive tensile test campaigns or physical theoretical calculations. Furthermore, the feature importance has been computed using the Shapley value analysis to understand the complex interplay of different features.

36 MATERIALS SCIENCE↗
DOE OSTI2022

Using machine learning for particle track identification in the CLAS12 detector

Particle track reconstruction is the most computationally intensive process in nuclear physics experiments. Traditional algorithms use a combinatorial approach that exhaustively tests track measurements ("hits") to identify those that form an actual particle trajectory. In this article, we describe the development of four machine learning (ML) models that assist the tracking algorithm by identifying valid track candidates from the measurements in drift chambers. Several types of machine learning models were tested, including: Convolutional Neural Networks (CNN), Multi-Layer Perceptrons (MLP), Extremely Randomized Trees (ERT) and Recurrent Neural Networks (RNN). As a result of this work, an MLP network classifier was implemented as part of the CLAS12 reconstruction software to provide the tracking code with recommended track candidates. The resulting software achieved accuracy of greater than 99% and resulted in an end-to-end speedup of 35% compared to existing algorithms.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2024

Efficient Anomaly Detection Driven By Different Machine Learning Architectures And Models

The rapid growth and ubiquitous adoption of the internet and cyber-physical systems (CPS) have fundamentally transformed modern communication, work, and human-system interactions. While networks now form the backbone of critical digital ecosystems, enabling seamless data transmission across diverse, interconnected systems, this increased connectivity also expands the attack surface, making real-time detection of network intrusions and anomalies a pressing challenge. Detecting unusual activities within network infrastructure requires advanced data traffic analysis to differentiate between legitimate and malicious interactions. Traditional approaches to network anomaly detectionâ??such as rule-based and signature-based systemsâ??often depend on predefined patterns to identify known anomalies, limiting their effectiveness against emerging, stealthy, or previously unseen threats. These conventional methods suffer from high false alarm rates and fail to adapt to the ever-evolving nature of network traffic, particularly in large-scale, decentralized environments where data volume, velocity, and variety are constantly increasing. This dissertation presents artificial intelligence (AI)-driven approaches to anomaly detection that leverage graphics processing unit (GPU)-enabled high-performance computing (HPC) platforms for processing massive network traffic data and monitoring the components of cyber-physical systems (CPS) for potentially hazardous conditions. The research advances several key contributions: (1) Designing efficient machine learning techniques for CPS condition monitoring and anomaly detection; (2) enabling federated learning (FL) frameworks that enable distributed detection while preserving data privacy and system resilience; (3) exploring graph-based methodologies combining graph neural networks (GNN) and graph machine learning (ML) approaches for the Internet of Things (IoT) and automotive network security, and (4) performing distributed edge computing optimizations that integrate FL with scalable technologies for reduced communication overhead. Through extensive experiments, these methodologies demonstrate that complex anomaly detection and condition monitoring tasks can be achieved while balancing computational efficiency and detection accuracy through fine-grained network information processing. The frameworks developed in this research establish a robust foundation for network anomaly detection, providing scalable, adaptive, and privacy-preserving solutions for safeguarding CPS and IoT networks in an increasingly interconnected digital landscape. The practical implications of these research findings are significant, as they can inform the development of next-generation network security systems and contribute to the protection of critical infrastructure against sophisticated cyber attacks.

Marfo, William↗
DOE OSTI2023

Machine learning-based surrogate models and transfer learning for derivative free optimization of HT-PEM fuel cells

Widespread adoption of high-temperature polymer electrolyte membrane electrochemical systems, such as fuel cells (HT-PEMFCs), requires models and computational tools for accurate optimization and guiding new materials for enhancing performance and durability. In this contribution, knowledge-based modelling and data-driven modelling are combined using Few-Shot Learning and implementing an Automated Machine Learning framework for the generation of Machine Learning-based surrogate models. Applicability of the resulting model for derivative-free optimization is demonstrated. Additionally, a way of considering extrapolation in the optimization task is presented. Results show that although extrapolation is needed to achieve better solutions during optimization, it can be monitored and managed. As a result, tuning the electrode ionomer binder's properties, such as ionic conductivity, in the fuel cell represents a promising pathway for improving HT-PEMFC performance.

08 HYDROGEN↗
DOE OSTI2022

Machine learning-enabled model-data integration for predicting subsurface water storage

Subsurface water storage (SWS) is a key variable of the climate system and a storage component for precipitation and radiation anomalies, inducing persistence in the climate system. It plays a critical role in climate-change projections and can mitigate the impacts of climate change on ecosystems. However, because of the difficult accessibility of the underground, hydrologic properties and dynamics of SWS are poorly known. Direct observations of SWS are limited, and accurate incorporation of SWS dynamics into Earth system land models remains challenging. We propose a machine learning-enabled model-data integration framework to improve the SWS prediction at local to conus scales in a changing climate by leveraging all the available observation and simulation resources, as well as to inform the model development and guide the observation collection. The accurate prediction will enable an optimal decision of water management and land use and improve the ecosystem's resilience to the climate change.

Lu, Dan↗
DOE OSTI2024

Machine learning-guided design of direct methanol fuel cells with a platinum group metal-free cathode

Direct methanol fuel cells (DMFCs) offer a promising solution for clean electricity generation, particularly in small electronics and remote auxiliary power units. However, optimizing their efficiency and performance is challenging due to the complex interactions between various factors. Here, we present a novel approach that integrates experiments with machine learning to model and predict the performance of these fuel cells using atomically dispersed platinum group metal (PGM)-free catalysts at the cathode. Further, our machine learning models, trained on diverse input parameters, allow for the comprehensive optimization of DMFC performance prior to fabrication and testing. Through extensive experimental validation, we demonstrate that this data-driven approach accurately predicts key performance metrics, such as maximum power output and polarization curves. By combining our models with interpretable game-theory methods, we provide deep insights into the factors governing fuel cell performance, ultimately paving the way for the design of scalable and efficient DMFC technologies.

25 ENERGY STORAGE↗
DOE OSTI2023

Global fire modelling and control attributions based on the ensemble machine learning and satellite observations

Contemporary fire dynamics is one of the most complex and least understood land surface phenomena. Global fire controls related to climate, vegetation, and anthropogenic activity are usually intertwined, and difficult to disentangle in a quantitative way. Here, we leveraged an ensemble of five machine learning (ML) models and multiple satellite-based observations to conduct global fire modeling for three fire metrics (burned area, fire number, and fire size), and quantified driving mechanisms underlying annual fire changes in a spatially resolved manner for the period 2003–2019. Ensemble learning is a meta-approach that combines multiple ML predictions to improve accuracy, robustness, and generalization performance. We found that the optimized ensemble ML well reproduced annual dynamics of global burned area (R 2 = 0.90, P < 0.001), total fire numbers (R 2 = 0.86, P < 0.001), and averaged fire size (R 2 = 0.70, P < 0.001). Additionally, the ensemble ML captured key spatial patterns of multi-year mean magnitudes, annual variabilities, anomalies, and trends for different fire metrics. Our ML-based fire attributions further highlighted the dominant role of enhanced anthropogenic activity in reducing global burned area (–1.9 Mha/yr, P < 0.01), followed by climate control (–1.3 Mha/yr, P < 0.01) and insignificant positive vegetation control (0.4 Mha/yr, P = 0.60). Spatially, climate dominated a much larger burned area (53.7%) than human (23.4%) or vegetation control (22.9%); however, the counteracting effects from regional wetting and drying trends weakened the net climate impacts on global burned area. The fire number and fire size exhibited similar spatial control patterns with burned area; globally, however, fire number tended to be more affected by climate while fire size more influenced by human activities. Overall, our study confirmed the feasibility and efficiency of ensemble ML in global fire modeling and subsequent control attributions, providing a better understanding of contemporary fire regimes and contributing to robust fire projections in a changing environment.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

Predicting impurity spectral functions using machine learning

The Anderson Impurity Model (AIM) is a canonical model of quantum many-body physics. Here we investigate whether machine learning models, both neural networks (NN) and kernel ridge regression (KRR), can accurately predict the AIM spectral function in all of its regimes, from empty orbital, to mixed valence, to Kondo. To tackle this question, we construct two large spectral databases containing approximately 410 000 and 600 000 spectral functions of the single-channel impurity problem. We show that the NN models can accurately predict the AIM spectral function in all of its regimes, with pointwise mean absolute errors down to 0.003 in normalized units. We find that the trained NN models outperform models based on KRR and enjoy a speedup on the order of 10 5 over traditional AIM solvers. Finally, the required size of the training set of our model can be significantly reduced using farthest point sampling in the AIM parameter space, which is important for generalizing our method to more complicated multichannel impurity problems of relevance to predicting the properties of real materials.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
DOE OSTI2022

Combinatorial Evaluation of Physical Feature Engineering, Classical Machine Learning, and Deep Learning Models for Synchrophasor Data at Scale

A major objective of the project was to train and evaluate the effectiveness of multiple event and anomaly detection, identification and classification deep temporal learning models for processing of real-time phasor measurement unit (PMU) data streams. A vast dataset, consisting of two years of phasor measurements from all three U.S. Interconnections, was curated and released by the Department of Energy (DOE) through Pacific Northwest National Laboratory (PNNL). The dataset also included an event log that provided event times and types (e.g. generator trips, line trips, planned service events, transformer operations, etc.). Our analysis of this dataset addressed six (6) of the eleven (11) research priorities identified in Funding Opportunity Announcement (FOA) DE-FOA-0001861 “Big Data Analysis of Synchrophasor Data” (FOA 1861). Rather than being limited to pre-determined specific algorithms, this project relied on the uniquely structured, highly performant underlying time series database capabilities of the PredictiveGrid platform to assess the vast dataset utilizing a wide variety of algorithms.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2024

Improving the reliability of machine learned potentials for modeling inhomogeneous liquids

The atomic-scale response of inhomogeneous fluids at interfaces and surrounding solute particles plays a critical role in governing chemical, electrochemical, and biological processes. Classical molecular dynamics simulations have been applied extensively to simulate the response of fluids to inhomogeneities directly, but are limited by the accuracy of the underlying interatomic potentials. Here, we use neural network potentials (NNPs) trained to ab initio simulations to accurately predict the inhomogeneous responses of two distinct fluids: liquid water and molten NaCl. Although NNPs can be readily trained to model complex bulk systems across a range of state points, we show that to appropriately model a fluid's response at an interface, relevant inhomogeneous configurations must be included in the training data. In order to sufficiently sample appropriate configurations of such inhomogeneous fluids, we develop protocols based on molecular dynamics simulations in the presence of external potentials. We demonstrate that NNPs trained on inhomogeneous fluid configurations can more accurately predict several key properties of fluids—including the density response, surface tension and size-dependent cavitation free energies—for liquid water and molten NaCl, compared to both empirical interatomic potentials and NNPs that are not trained on such inhomogeneous configurations. This work therefore provides a first demonstration and framework to extract the response of inhomogeneous fluids from first principles for classical density-functional treatment of fluids free from empirical potentials.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2022

Earth and Space Science Informatics Perspectives on Integrated, Coordinated, Open, Networked (ICON) Science

Abstract This article is composed of three independent commentaries about the state of Integrated, Coordinated, Open, Networked (ICON) principles (Goldman, et al., 2021b, https://doi.org/10.1029/2021EO153180 ) in Earth and Space Science Informatics (ESSI) and includes discussion on the opportunities and challenges of adopting them. Each commentary focuses on a different topic: (Section 2) Global collaboration, cyberinfrastructure, and data sharing; (Section 3) Machine learning for multiscale modeling; (Section 4) Aerial and satellite remote sensing for advancing Earth system model development by integrating field and ancillary data. ESSI addresses data management practices, computation and analysis, and hardware and software infrastructure. Our role in ICON science therefore involves collaborative work to assess, design, implement, and promote practices and tools that enable effective data management, discovery, integration, and reuse for interdisciplinary work in Earth and space science disciplines. Networks of diverse people with expertise across Earth, space, and data science disciplines are essential for efficient and ethical exchanges of findable, accessible, interoperable, and reusable (FAIR) research products and practices. Our challenge is then to coordinate the development of standards, curation practices, and tools that enable integrating and reusing multiple data types, software, multi‐scale models, and machine learning approaches across disciplines in a way that is as open and/or FAIR as ethically possible. This is a major endeavor that could greatly increase the pace and potential of interdisciplinary scientific discovery.

58 GEOSCIENCES↗
DOE OSTI2026

Machine-learning force-field models for dynamical simulations of metallic magnets

We review recent advances in machine-learning (ML) force-field methods for Landau–Lifshitz–Gilbert simulations of itinerant electron magnets, focusing on their scalability and transferability. Built on the principle of locality, a deep neural-network model is developed to efficiently and accurately predict electron-mediated forces governing spin dynamics. Symmetry-aware descriptors constructed through a group-theoretical approach ensure rigorous incorporation of both lattice and spin-rotation symmetries. The framework is demonstrated using the prototypical s-d exchange model widely employed in spintronics. ML-enabled large-scale simulations reveal novel nonequilibrium phenomena, including anomalous coarsening of tetrahedral spin order on the triangular lattice and the freezing of phase-separation dynamics in lightly hole-doped, strong-coupling square-lattice systems. These results establish ML force-field frameworks as scalable, accurate, and versatile tools for modeling nonequilibrium spin dynamics in itinerant magnets.

Artificial neural networks↗