Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Machine Learning Models”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,459 records · Page 81
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

Ising-CF: A Pathbreaking Collaborative Filtering Method Through Efficient Ising Machine Learning

Due to Ising models’ strong expressivity and Ising machines’ unique computational power, it is highly desired if Ising-based learning can be used in real-world applications. Unfortunately, the challenges in learning Ising models and gaps between the practical accuracy of Ising machines and the theoretical accuracy of Ising models impede the realization of Ising machines’ potential. Hence, we propose an Ising Machine Learning framework, Ising-CF, for collaborative filtering, a widely-used recommendation method. Specifically, Ising-CF uses Linear Neural Networks with Besag’s pseudo-likelihood and voltage polarization for fast, accurate Ising model learning and an Ising-specific logarithmic quantization for ns-level Ising machine inference with near-theoretical accuracy, 7.3% over SOTA.

Liu, Zhuo↗
DOE OSTI2023

MLMOD: Machine Learning Methods for Data-Driven Modeling in LAMMPS

MLMOD is a software package for incorporating machine learning approaches and models into simulations of microscale mechanics and molecular dynamics in LAMMPS. Recent machine learning approaches provide promising data-driven approaches for learning representations for system behaviors from experimental data and high fidelity simulations. The package facilitates learning and using data-driven models for (i) dynamics of the system at larger spatial-temporal scales (ii) interactions between system components, (iii) features yielding coarser degrees of freedom, and (iv) features for new quantities of interest characterizing system behaviors. MLMOD provides hooks in LAMMPS for (i) modeling dynamics and time-step integration, (ii) modeling interactions, and (iii) computing quantities of interest characterizing system states. The package allows for use of machine learning methods with general model classes including Neural Networks, Gaussian Process Regression, Kernel Models, and other approaches. Here we discuss our prototype C++/Python package, aims, and example usage. For related papers, examples, updates, and additional information see https://github.com/atzberg/mlmod and http://atzberger.org/.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Machine-Learned Force Field Modeling of Metal Organic Frameworks for CO2 Direct Air Capture

To cope with legacy greenhouse gas emissions and to achieve net-zero emissions by 2050, the U.S. Department of Energy (DOE) is funding efforts to develop direct air capture (DAC), a method for removing CO2 directly from air. Metal organic frameworks (MOFs) have been well studied as DAC sorbent materials due to their tunable structural and compositional properties. Thermodynamic simulations using force fields are often used to provide predictions of a material’s performance in many separations. However, these force fields often make assumptions about bonds and the physics of the adsorption process. A new class of force fields called machine-learned force fields (MLFFs) use machine learning to form quantitative relationships between a material’s chemical structure and the forces and energies predicted by more accurate quantum mechanical calculations, such as dispersion-corrected density functional theory (DFT). In this work, models were developed to achieve DFT-level accuracy for the forces and energies associated with MOF flexibility and CO2 adsorption using MLFFs. These methods were parametrized based on thousands of DFT calculations of CO2 in flexible MOFs and used to predict MOF structural properties as well as CO2 adsorption in several MOFs.

Findley, John↗
NASA NTRSDate not supplied

Machine Learning Airport Surface Model

Future needs of the National Airspace System require decision support tools to adopt a service-oriented architecture in alignment with the FAA’s vision for an Info-Centric NAS. To achieve this, many existing systems will need to undergo a digital transformation from a monolithic decision support tool to a service-oriented architecture where individual services are exposed through well defined Application Programming Interfaces (APIs). To enable this transformation, NASA has developed the Digital Information Platform as a cloud based foundation for development of aviation services with a special focus towards Artificial Intelligence and Machine Learning (ML) services. This paper describes the work required for the transformation of NASA’s legacy surface management system to a real-time ML based decision support system deployed in the cloud. Details of the Machine Learning Operations (MLOps) infrastructure and best practices are described which enabled the end-toend lifecycle management of ML within an integrated software system. Validation results are provided from an operational field evaluation where performance was benchmarked against the legacy approach.

Jeremy Coupe↗
NASA NTRSDate not supplied

DELTA: An Open-Source Framework to Simplify Machine Learning with Satellite Imagery

DELTA (Deep Earth Learning, Tools, and Analysis) is an open-source framework developed at NASA to simplify running and training machine learning (ML) models on satellite imagery. Users new to machine learning can run existing ML models on satellite imagery with minimal setup and configuration. For experienced ML users, DELTA helps simplify data engineering, preprocessing steps, and reduces the need for boilerplate code that needs written to make satellite imagery datasets palatable for machine learning. This lets data scientists focus on model development while DELTA handles the imagery manipulation. This presentation will demonstrate DELTA’s functionality and share some examples from an active project using it for flood mapping using imagery from multiple satellite sources.

deep learning↗
DOE OSTI2023

Exploring model complexity in machine learned potentials for simulated properties

Abstract Machine learning (ML) enables the development of interatomic potentials with the accuracy of first principles methods while retaining the speed and parallel efficiency of empirical potentials. While ML potentials traditionally use atom-centered descriptors as inputs, different models such as linear regression and neural networks map descriptors to atomic energies and forces. This begs the question: what is the improvement in accuracy due to model complexity irrespective of descriptors? We curate three datasets to investigate this question in terms of ab initio energy and force errors: (1) solid and liquid silicon, (2) gallium nitride, and (3) the superionic conductor Li $$_{10}$$ 10 Ge(PS $$_{6}$$ 6 ) $$_{2}$$ 2 (LGPS). We further investigate how these errors affect simulated properties and verify if the improvement in fitting errors corresponds to measurable improvement in property prediction. By assessing different models, we observe correlations between fitting quantity (e.g. atomic force) error and simulated property error with respect to ab initio values. Graphical abstract

Rohskopf, A. (ORCID:0000000227128296)↗
DOE OSTI2021

Application of Machine Learning Techniques to an Agent-Based Model of Pantoea

Agent-based modeling (ABM) is a powerful simulation technique which describes a complex dynamic system based on its interacting constituent entities. While the flexibility of ABM enables broad application, the complexity of real-world models demands intensive computing resources and computational time; however, a metamodel may be constructed to gain insight at less computational expense. Here, we developed a model in NetLogo to describe the growth of a microbial population consisting of Pantoea . We applied 13 parameters that defined the model and actively changed seven of the parameters to modulate the evolution of the population curve in response to these changes. We efficiently performed more than 3,000 simulations using a Python wrapper, NL4Py . Upon evaluation of the correlation between the active parameters and outputs by random forest regression, we found that the parameters which define the depth of medium and glucose concentration affect the population curves significantly. Subsequently, we constructed a metamodel, a dense neural network, to predict the simulation outputs from the active parameters and found that it achieves high prediction accuracy, reaching an R 2 coefficient of determination value up to 0.92. Our approach of using a combination of ABM with random forest regression and neural network reduces the number of required ABM simulations. The simplified and refined metamodels may provide insights into the complex dynamic system before their transition to more sophisticated models that run on high-performance computing systems. The ultimate goal is to build a bridge between simulation and experiment, allowing model validation by comparing the simulated data to experimental data in microbiology.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2021

Prediction of Solar Irradiance and Photovoltaic Solar Energy Product Based on Cloud Coverage Estimation Using Machine Learning Methods

Cloud cover estimation from images taken by sky-facing cameras can be an important input for analyzing current weather conditions and estimating photovoltaic power generation. The constant change in position, shape, and density of clouds, however, makes the development of a robust computational method for cloud cover estimation challenging. Accurately determining the edge of clouds and hence the separation between clouds and clear sky is difficult and often impossible. Toward determining cloud cover for estimating photovoltaic output, we propose using machine learning methods for cloud segmentation. We compare several methods including a classical regression model, deep learning methods, and boosting methods that combine results from the other machine learning models. To train each of the machine learning models with various sky conditions, we supplemented the existing Singapore whole sky imaging segmentation database with hazy and overcast images collected by a camera-equipped Waggle sensor node. We found that the U-Net architecture, one of the deep neural networks we utilized, segmented cloud pixels most accurately. However, the accuracy of segmenting cloud pixels did not guarantee high accuracy of estimating solar irradiance. We confirmed that the cloud cover ratio is directly related to solar irradiance. Additionally, we confirmed that solar irradiance and solar power output are closely related; hence, by predicting solar irradiance, we can estimate solar power output. This study demonstrates that sky-facing cameras with machine learning methods can be used to estimate solar power output. This ground-based approach provides an inexpensive way to understand solar irradiance and estimate production from photovoltaic solar facilities.

14 SOLAR ENERGY↗
DOE OSTI2023

Efficient Reliability Analysis using Generalized Multifidelity Modeling and Explainable Active Learning

To assess the reliability of critical technologies like nuclear plants and infrastructure systems and improve the robustness of design, engineers have to quantify the uncertainties surrounding the system behavior accurately. However, the complexity of the problem can make standard reliability analysis algorithms prohibitively expensive, primarily due to the high computational cost of estimating the system response at each iteration. This cost can be greatly reduced by using multi-fidelity modeling and machine learning to build a surrogate model to replace the expensive response function. We propose a general and robust method for building surrogates from multiple Low Fidelity (LF) models coupled with machine learning to retain accuracy. Our framework first constructs “Corrected Low Fidelity models” (CLFs) by coupling a High Fidelity (HF) model inferred Gaussian Process correction term with each of the LF models. It then uses the correction terms to assign model probabilities to each of these CLFs in an explainable way before using them to assemble the final surrogate. No assumptions are made about the type of the LF models or their correlation with the HF model. The proposed surrogate modeling framework is used within the subset simulation algorithm (a variance-reduced MCMC-based reliability analysis algorithm) for enhanced efficiency. Additionally, an active learning step is added to the algorithm to adaptively decide when the surrogate is not sufficiently accurate, at which point the HF model is called and used to refine the surrogate. Through a frame buckling example, our method is shown to be highly efficient at reducing the expensive HF model calls while accurately estimating the failure probability.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Hierarchical Model Reduction Driven by Machine Learning for Parametric Advection-Diffusion-Reaction Problems in the Presence of Noisy Data

Abstract We propose a new approach to generate a reliable reduced model for a parametric elliptic problem, in the presence of noisy data. The reference model reduction procedure is the directional HiPOD method, which combines Hierarchical Model reduction with a standard Proper Orthogonal Decomposition, according to an offline/online paradigm. In this paper we show that directional HiPOD looses in terms of accuracy when problem data are affected by noise. This is due to the interpolation driving the online phase, since it replicates, by definition, the noise trend. To overcome this limit, we replace interpolation with Machine Learning fitting models which better discriminate relevant physical features in the data from irrelevant unstructured noise. The numerical assessment, although preliminary, confirms the potentialities of the new approach.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Dataset for 'Stream Temperature Predictions for River Basin Management in the Pacific Northwest and Mid-Atlantic Regions Using Machine Learning', Water 2022

This data package presents forcing data, model code, and model output for classical machine learning models that predict monthly stream water temperature as presented in the manuscript ‘Stream Temperature Predictions for River Basin Management in the Pacific Northwest and Mid-Atlantic Regions Using Machine Learning’, Water (Weierbach et al., 2022). Specifically, for input forcing datasets we include two files each generated using the BASIN-3D data integration tool (Varadharajan et al., 2022) for stations in the Pacific Northwest and Mid Atlantic Hydrologic regions. Model code (written in python with the use of jupyter notebooks) includes codes for data preprocessing, training Multiple Linear Regression, Support Vector Regression, and Extreme Gradient Boosted Tree models, and additional notebooks for analysis of model output. We include specific model output files which represent modeling configurations presented in the manuscript also presented in an hdf5 format. Together, these data make up the workflow for predictions across three scenarios (single station, regional, and predictions in unmonitored basins) presented in the manuscript and allow for reproducibility of modeling procedures.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

A machine learning aided interpretable model for rupture strength prediction in Fe-based martensitic and austenitic alloys

Abstract The class of 9–12% Cr ferritic-martensitic alloys (FMA) and austenitic stainless steels have received considerable attention due to their numerous applications in high temperature power generation industries. To design high strength steels with prolonged service life requires a thorough understanding of the long-term properties, e.g., creep rupture strength, rupture life, etc., as a function of the chemical composition and processing parameters that govern the microstructural characteristics. In this article, the creep rupture strength of both 9–12% Cr FMA and austenitic stainless steel has been parameterized using curated experimental datasets with a gradient boosting machine. The trained model has been cross validated against unseen test data and achieved high predictive performance in terms of correlation coefficient ( $$R^{2} > 0.98 $$ R 2 > 0.98 for 9–12% Cr FMA and $$R^{2} > 0.95 $$ R 2 > 0.95 for austenitic stainless steel) thus bypassing the need for additional comprehensive tensile test campaigns or physical theoretical calculations. Furthermore, the feature importance has been computed using the Shapley value analysis to understand the complex interplay of different features.

36 MATERIALS SCIENCE↗
DOE OSTI2022

Using machine learning for particle track identification in the CLAS12 detector

Particle track reconstruction is the most computationally intensive process in nuclear physics experiments. Traditional algorithms use a combinatorial approach that exhaustively tests track measurements ("hits") to identify those that form an actual particle trajectory. In this article, we describe the development of four machine learning (ML) models that assist the tracking algorithm by identifying valid track candidates from the measurements in drift chambers. Several types of machine learning models were tested, including: Convolutional Neural Networks (CNN), Multi-Layer Perceptrons (MLP), Extremely Randomized Trees (ERT) and Recurrent Neural Networks (RNN). As a result of this work, an MLP network classifier was implemented as part of the CLAS12 reconstruction software to provide the tracking code with recommended track candidates. The resulting software achieved accuracy of greater than 99% and resulted in an end-to-end speedup of 35% compared to existing algorithms.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2024

Efficient Anomaly Detection Driven By Different Machine Learning Architectures And Models

The rapid growth and ubiquitous adoption of the internet and cyber-physical systems (CPS) have fundamentally transformed modern communication, work, and human-system interactions. While networks now form the backbone of critical digital ecosystems, enabling seamless data transmission across diverse, interconnected systems, this increased connectivity also expands the attack surface, making real-time detection of network intrusions and anomalies a pressing challenge. Detecting unusual activities within network infrastructure requires advanced data traffic analysis to differentiate between legitimate and malicious interactions. Traditional approaches to network anomaly detectionâ??such as rule-based and signature-based systemsâ??often depend on predefined patterns to identify known anomalies, limiting their effectiveness against emerging, stealthy, or previously unseen threats. These conventional methods suffer from high false alarm rates and fail to adapt to the ever-evolving nature of network traffic, particularly in large-scale, decentralized environments where data volume, velocity, and variety are constantly increasing. This dissertation presents artificial intelligence (AI)-driven approaches to anomaly detection that leverage graphics processing unit (GPU)-enabled high-performance computing (HPC) platforms for processing massive network traffic data and monitoring the components of cyber-physical systems (CPS) for potentially hazardous conditions. The research advances several key contributions: (1) Designing efficient machine learning techniques for CPS condition monitoring and anomaly detection; (2) enabling federated learning (FL) frameworks that enable distributed detection while preserving data privacy and system resilience; (3) exploring graph-based methodologies combining graph neural networks (GNN) and graph machine learning (ML) approaches for the Internet of Things (IoT) and automotive network security, and (4) performing distributed edge computing optimizations that integrate FL with scalable technologies for reduced communication overhead. Through extensive experiments, these methodologies demonstrate that complex anomaly detection and condition monitoring tasks can be achieved while balancing computational efficiency and detection accuracy through fine-grained network information processing. The frameworks developed in this research establish a robust foundation for network anomaly detection, providing scalable, adaptive, and privacy-preserving solutions for safeguarding CPS and IoT networks in an increasingly interconnected digital landscape. The practical implications of these research findings are significant, as they can inform the development of next-generation network security systems and contribute to the protection of critical infrastructure against sophisticated cyber attacks.

Marfo, William↗
NASA NTRSDate not supplied

DELTA: An Open-Source Framework to Simplify Machine Learning with Satellite Imagery

DELTA (Deep Earth Learning, Tools, and Analysis) is an open-source framework developed at NASA to simplify running and training machine learning (ML) models on satellite imagery. Users new to machine learning can run existing ML models on satellite imagery with minimal setup and configuration. For experienced ML users, DELTA helps simplify data engineering, preprocessing steps, and reduces the need for boilerplate code that needs written to make satellite imagery datasets palatable for machine learning. This lets data scientists focus on model development while DELTA handles the imagery manipulation. This presentation will demonstrate DELTA’s functionality and share some examples from an active project using it for flood mapping using imagery from multiple satellite sources

Michael von Pohle↗
DOE OSTI2023

Machine learning-based surrogate models and transfer learning for derivative free optimization of HT-PEM fuel cells

Widespread adoption of high-temperature polymer electrolyte membrane electrochemical systems, such as fuel cells (HT-PEMFCs), requires models and computational tools for accurate optimization and guiding new materials for enhancing performance and durability. In this contribution, knowledge-based modelling and data-driven modelling are combined using Few-Shot Learning and implementing an Automated Machine Learning framework for the generation of Machine Learning-based surrogate models. Applicability of the resulting model for derivative-free optimization is demonstrated. Additionally, a way of considering extrapolation in the optimization task is presented. Results show that although extrapolation is needed to achieve better solutions during optimization, it can be monitored and managed. As a result, tuning the electrode ionomer binder's properties, such as ionic conductivity, in the fuel cell represents a promising pathway for improving HT-PEMFC performance.

08 HYDROGEN↗
DOE OSTI2022

Machine learning-enabled model-data integration for predicting subsurface water storage

Subsurface water storage (SWS) is a key variable of the climate system and a storage component for precipitation and radiation anomalies, inducing persistence in the climate system. It plays a critical role in climate-change projections and can mitigate the impacts of climate change on ecosystems. However, because of the difficult accessibility of the underground, hydrologic properties and dynamics of SWS are poorly known. Direct observations of SWS are limited, and accurate incorporation of SWS dynamics into Earth system land models remains challenging. We propose a machine learning-enabled model-data integration framework to improve the SWS prediction at local to conus scales in a changing climate by leveraging all the available observation and simulation resources, as well as to inform the model development and guide the observation collection. The accurate prediction will enable an optimal decision of water management and land use and improve the ecosystem's resilience to the climate change.

Lu, Dan↗
NASA NTRSDate not supplied

Prediction of High-Latitude Ionospheric Electrodynamics Using the Machine Learning Based Auroral Ionospheric Electrodynamics Model

We introduce a new framework for Machine-Learning (ML) based Auroral Ionosphere Model (ML-AIM). ML-AIM solves a current continuity equation by utilizing the ML model of Field Aligned Currents (FACs) of Kunduri et al., 2020 (https://doi.org/10.1029/2020JA027908), the FAC-derived aurora conductance model of Robinson et al., 2020 (https://doi.org/10.1029/2020JA028008), and the solar irradiance conductance model of Moen & Brekke (1993). The ML-AIM inputs are 60min time histories of solar wind plasma, interplanetary magnetic fields (IMF), and geomagnetic indices, and its outputs are ionospheric electric potential, electric fields, Pederson/Hall currents, and Joule Heating. We conduct two ML-AIM simulations for a weak geomagnetic activity on 14 May 2013 and a geomagnetic storm on 7-8 September 2017. ML-AIM produces reasonable ionospheric potential patterns such as two cell convection patterns and the enhancement of electric potentials during active times. The cross polar cap potential drop from ML-AIM is also comparable to the ones from the Weimer 2005 model, Super Dual Auroral Radar Network (SuperDARN), and Defense Meteorological Satellite Program (DMSP) F17 observations. ML-AIM is unique in a sense that it predicts ionospheric responses to the time-varying solar wind and geomagnetic conditions, while other traditional empirical model like Weimer 2005 is designed to provide static ionospheric conditions under steady solar wind/IMF conditions. In future, ML-AIM will include ML-based models of aurora precipitation and ionospheric conductance, improving its performance during active times.

H. K. Connor↗