Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Machine Learning Models”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,783 records · Page 99
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

A Novel Method to Train Classification Models for Structure Detection in In Situ Spacecraft Data

We present a method for creating spacecraft-like data which can be used to train Machine Learning (ML) models to detect and classify structures in in situ spacecraft data. First, we use the Grad-Shafranov equation to numerically solve for several magnetohydrostatic equilibria which are variations on a known analytic equilibrium. These equilibria are then used as the initial conditions for Particle-In-Cell simulations in which the structures of interest are observed and labeled. We then take one-dimensional slices through the simulations to replicate what a spacecraft collecting data from the simulation would observe. This sliced data then can be used as training data for the initial training of ML models intended for use on spacecraft data. We demonstrate the method applied to the problem of detecting small-scale plasmoids in the magnetotail, which is important for understanding complex magnetotail reconnection dynamics. The simple 1D classifier we train is able to detect more than 70% of the plasmoid points in the data set but also produces a large number of false positives. Our further work on this example problem is detailed, and further potential uses of the method are discussed.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2024

Acceleration of Graph Neural Network-Based Prediction Models in Chemistry via Co-Design Optimization on Intelligence Processing Units

Atomic structure prediction and associated property calculations are the bedrock of chemical physics. Since high-fidelity ab initio modeling techniques for computing the structure and properties can be prohibitively expensive, this motivates the development of machine-learning (ML) models that make these predictions more efficiently. Training graph neural networks over large atomistic databases introduces unique computational challenges such as the need to process millions of small graphs with variable size and support communication patterns that are distinct from learning over large graphs such as social networks. We demonstrate a novel hardware-software co-design approach to scale up the training of atomistic graph neural networks (GNN) for structure and property prediction. First, to eliminate redundant computation and memory associated with alternative padding techniques and to improve throughput via minimizing communication, we formulate the effective coalescing of the batches of variable-size atomistic graphs as the bin packing problem and introduce a hardware-agnostic algorithm to pack these batches. In addition, we propose hardware-specific optimizations including a planner and vectorization for the gather-scatter operations targeted for Graphcore’s Intelligence Processing Unit (IPU), as well as model-specific optimizations such as merged communication collectives and optimized softplus. Putting these all together, we demonstrate the effectiveness of the proposed co-design approach by providing an implementation of a well-established atomistic GNN on the Graphcore IPUs. We evaluate the training performance on multiple atomistic graph databases with varying degrees of graph counts, sizes and sparsity. Here, we demonstrate that such a co-design approach can reduce the training time of atomistic GNNs and can improve the performance by up to 1.5× compared to the baseline implementation of the model on the IPUs. Additionally, we compare our IPU implementation with a Nvidia GPU-based implementation and show that our atomistic GNN implementation on the IPUs can run 1.8× faster on average compared to the execution time on the GPUs.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2021

Bridging Multiscale Processes in Earth System Models with Physics-Guided Hierarchical Machine Learning

Focal Area(s): The focal area of this whitepaper is, “Predictive modeling through the use of AI techniques and AI-derived model components; the use of AI and other tools to design a prediction system comprising of a hierarchy of models (e.g., AI driven model/component/parameterization selection).” The ideas and frameworks described herein are deemed site agnostic. As a use case, this group will initially focus on the coupling between land processes, surface/subsurface hydrological processes, coastal processes, and human activities in the U.S. Gulf states.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Machine-learning-assisted deciphering of microstructural effects on ionic transport in composite materials: A case study of Li 7 La 3 Zr 2 O 12 -LiCoO 2

The effective diffusivity of ionic species in multiphase materials is critical for the design and function of composite materials for electrochemical energy storage. In practice, effective diffusivity depends sensitively not only on the intrinsic diffusivities of constituting materials but also on their topological arrangement; nevertheless, these coupled contributions are oversimplified in most analytical models. Here, we combine atomistically informed mesoscale modeling and machine learning (ML) analysis to unravel how such features affect effective diffusivity in two-phase composites. Using the Li 7 La 3 Zr 2 O 12 -LiCoO 2 composite solid-state battery cathode as a model system, we compute effective diffusivity for 600 distinct dense polycrystalline microstructures with different topological configurations of grains, grain boundaries, and heterointerfaces. We verify that in addition to atomic-scale variabilities, microstructural feature diversity can significantly impact effective transport properties. Across the ensemble of test microstructures, this often results in bimodal distributions of effective diffusivity that encompass two qualitatively distinct operating mechanisms, which we identify via flux analysis. An ML approach reveals that the most critical determining factors for effective diffusivity are the connectivity of bulk phases and their heterointerfaces. The role of ionic mobility at the heterointerfaces is also discussed. These insights highlight the combined importance of microstructure and interface engineering in tuning the transport properties of ionic species in composite materials. In conclusion, our framework can also be extended for understanding generic microstructure-property relationships in other complex multiphase materials.

25 ENERGY STORAGE↗
DOE OSTI2023

Frost prediction using machine learning and deep neural network models

This study describes accurate, computationally efficient models that can be implemented for practical use in predicting frost events for point-scale agricultural applications. Frost damage in agriculture is a costly burden to farmers and global food security alike. Timely prediction of frost events is important to reduce the cost of agricultural frost damage and traditional numerical weather forecasts are often inaccurate at the field-scale in complex terrain. In this paper, we developed machine learning (ML) algorithms for the prediction of such frost events near Alcalde, NM at the point-scale. ML algorithms investigated include deep neural network, convolution neural networks, and random forest models at lead-times of 6–48 h. Our results show promising accuracy (6-h prediction RMSE = 1.53–1.72°C) for use in frost and minimum temperature prediction applications. Seasonal differences in model predictions resulted in a slight negative bias during Spring and Summer months and a positive bias in Fall and Winter months. Additionally, we tested the model transferability by continuing training and testing using data from sensors at a nearby farm. We calculated the feature importance of the random forest models and were able to determine which parameters provided the models with the most useful information for predictions. We determined that soil temperature is a key parameter in longer term predictions (>24 h), while other temperature related parameters provide the majority of information for shorter term predictions. The model error compared favorable to previous ML based frost studies and outperformed the physically based High Resolution Rapid Refresh forecasting system making our ML-models attractive for deployment toward real-time monitoring of frost events and damage at commercial farming operations.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Teaching AI when to care about gender

Natural Language Processing (NLP) is a branch of Artificial Intelligence (AI) concerned with solving language tasks by modeling large amounts of textual data. Some NLP techniques use word embeddings which are semantic models where machine learning (ML) is used to learn to cluster semantically related words by learning about word co-occurrences in the original training text. Unfortunately, these models tend to reflect or even exaggerate biases that are present in the training corpus. Here we describe the Word Embedding Navigator (WEN), which is a tool for exploring word embedding models. We examine a specific potential use case for this tool: interactive discovery and neutralization of gender bias in word embedding models, and compare this human-in-the-loop approach to reducing bias in word embeddings with a debiasing post-processing technique.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Advancements on Multi-Fidelity Random Fourier Neural Networks: Application to Hurricane Modeling for Wind Energy

Multi-fidelity approaches are emerging as effective strategies in computational science to handle otherwise intractable tasks like Uncertainty Quantification (UQ), training of Machine Learning (ML) models, and optimization, for expensive high-fidelity applications in which the amount of available simulations or data is limited. The main idea is simple: large datasets generated for low-fidelity approximations of the problem at hand are fused with a much sparser dataset for the target (high-fidelity) system. In this paper, we build on our recent success in designing random Fourier Neural Networks (rFNNs) [1] to target problems arising in wind energy applications and in particular problems of interest for hurricane modeling. In this context, data for the high-fidelity models are limited and lower fidelity alternatives are needed. In this work, we introduce a novel multi-fidelity training approach for our rFNNs and demonstrate its use on a simple verification problem and on a hurricane modeling problem in which high-fidelity data are generated via Large-Eddy Simulations (LES), while low-fidelity data are given by a mesoscale model. Initial results demonstrate how the multi-fidelity training approach can improve the quality of the resulting surrogate.

Fourier Neural Networks↗
NASA NTRS2021

Automated Data Accountability for Missions in Mars Rover Data

As the Mars Curiosity Rover transmits data to the JPL Ground Data System (GDS), it frequently observes data loss and corruption, requiring re-transmits from the rover and Ground Data System Analysts (GDSA) to monitor the downlink process. As new missions are launched, the GDSA team redistributes analysts to these new missions, causing shortages in previous missions. The GDSA team can significantly benefit from the automation and optimization of the downlink process of telemetry data. In fact, there is a need for a better understanding of why the data is corrupted, so that the GDSA team can best determine the root cause of the issues in the GDS. This paper presents machine learning and deep learning based approaches to automate and optimize the detection of data loss. We first created a pipeline to automatically accumulate data from the telemetry databases (MAROS, Telemetry Data Storage, and GDS Elastic Search Database) in the downlink process. With our newly created datasets, we perform feature selection to supplement the GDSA understanding of the downlink process and provide supplemental analysis on the importance of different features. We implement various machine learning and deep learning based models, including support vector machines, ensemble methods, and deep neural networks and evaluate their accuracies in identifying whether a downlink process is complete or incomplete. We utilize fast hyperparameter optimization methods that allow our models to quickly be re-trained, allowing them to quickly be tuned and optimized on daily incoming data in real time. This hyperparameter optimization also allows our methods to be quickly integrated into other JPL missions. Our results show that our best-performing machine learning and deep learning based models outperform the existing GDSA detection software by 6 accuracy points and can aid analysts by providing insights into the data accountability problem. Since these various machine learning and deep learning approaches vary significantly in interpretability, we provide a discussion on the tradeoffs between their performance and trustworthiness in helping detect issues in data transmission.

Divsalar, Dariush↗
NASA NTRSDate not supplied

Tracking the Hunga Tonga-Hunga Ha’apai Eruption Stratospheric Aerosol and Trace Gas Plumes Using Machine Learning

On January 15, 2022, the Hunga Tonga-Hunga Ha’apai (hereafter, Hunga Tonga) submarine volcano had an explosive eruption that thrusted ash, gases, and water vapor through the troposphere into the stratosphere and mesosphere. Previous studies manually tracked the aerosol and trace gas plumes over time across different positions in the southern hemisphere. Using data retrieved from low earth orbiting satellite instruments (e.g., OMPS, OMI, and CALIPSO), this research demonstrates how open-source machine learning (ML) models, like Meta’s Segment Anything Model (SAM), with prompt engineering can perform automatic plume tracking following the Hunga Tonga eruption. This extensible methodology, and modular data processing and modeling pipeline using NASA Earthdata and Openscapes, establishes a framework for systematically and rapidly studying extreme events, including volcanic eruptions and large-scale wildfires. By combining advanced machine learning techniques, such as SAM’s zero-shot learning, with large volumes of remote sensing data, this work demonstrates how AI and open science can accelerate research and generate actionable results. The tools and technologies presented here can help translate earth science to action from NASA’s current and future Earth observing satellite missions (e.g., the Atmosphere Observing System (AOS)), and assist researchers and stakeholders in understanding, mapping, and responding to natural disasters and extreme events in a changing world.

David M. Giles↗
DOE OSTI2021

Probabilistic simulation of quantum circuits using a deep-learning architecture

The fundamental question of how to best simulate quantum systems using conventional computational resources lies at the forefront of condensed matter and quantum computation. It impacts both our understanding of quantum materials and our ability to emulate quantum circuits. Here we present an exact formulation of quantum dynamics via factorized generalized measurements which maps quantum states to probability distributions with the advantage that local unitary dynamics and quantum channels map to local quasistochastic matrices. This representation provides a general framework for using state-of-the-art probabilistic models in machine learning for the simulation of quantum many-body dynamics. Using this framework, we have developed a practical algorithm to simulate quantum circuits using an attention network based on a powerful neural network ansatz responsible for the most recent breakthroughs in natural language processing. We demonstrate our approach by simulating circuits that build Greenberger-Horne-Zeilinger and linear graph states of up to 60 qubits, as well as a variational quantum eigensolver circuit for preparing the ground state of the transverse field Ising model on several system sizes. Our methodology constitutes a modern machine learning approach to the simulation of quantum physics with applicability both to quantum circuits as well as other quantum many-body systems.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2020

A non-cooperative meta-modeling game for automated third-party calibrating, validating and falsifying constitutive laws with parallelized adversarial attacks

The evaluation of constitutive models, especially for high-risk and high-regret engineering applications, requires efficient and rigorous third-party calibration, validation and falsification. While there are numerous efforts to develop paradigms and standard procedures to validate models, difficulties may arise due to the sequential, manual, and often biased nature of the commonly adopted calibration and validation processes, thus slowing down data collections, hampering the progress towards discovering new physics, increasing expenses and possibly leading to misinterpretations of the credibility and application ranges of proposed models. This work attempts to introduce concepts from game theory and machine learning techniques to overcome many of these existing difficulties. Here, we introduce an automated meta-modeling game where two competing AI agents systematically generate experimental data to calibrate a given constitutive model and to explore its weakness such that the experiment design and model robustness can be improved through competitions. The two agents automatically search for the Nash equilibrium of the meta-modeling game in an adversarial reinforcement learning framework without human intervention. In particular, a protagonist agent seeks to find the more effective ways to generate data for model calibrations, while an adversary agent tries to find the most devastating test scenarios that expose the weaknesses of the constitutive model calibrated by the protagonist. By capturing all possible design options of the laboratory experiments into a single decision tree, we recast the design of experiments as a game of combinatorial moves that can be resolved through deep reinforcement learning by the two competing players. Our adversarial framework emulates idealized scientific collaborations and competitions among researchers to achieve a better understanding of the application range of the learned material laws and prevent misinterpretations caused by conventional AI-based third-party validation. Numerical examples are given to demonstrate the wide applicability of the proposed meta-modeling game with adversarial attacks on both human-crafted constitutive models and machine learning models.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Ontologies for Intelligent Data Science

As anyone even vaguely aware of current technology can tell you, machine learning (ML) and artificial intelligence (AI) have made exceptional breakthroughs in recent years. Generative artificial intelligence (GAI) emerged circa 2022 dominated by Large Language Models (LLMs) and generative tools for images emerged at about the same time.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
DOE OSTI2021

Modeling household online shopping demand in the U.S.: a machine learning approach and comparative investigation between 2009 and 2017

Despite the rapid growth of online shopping and research interest in the relationship between online and in-store shopping, national-level modeling and investigation of the demand for online shopping with a prediction focus remain limited in the literature. Here, this paper differs from prior work and leverages two recent releases of the U.S. National Household Travel Survey (NHTS) data for 2009 and 2017 to develop machine learning (ML) models, specifically gradient boosting machine (GBM), for predicting household-level online shopping purchases. The NHTS data allow for not only conducting nationwide investigation but also at the level of households, which is more appropriate than at the individual level given the connected consumption and shopping needs of members in a household. We follow a systematic procedure for model development including employing Recursive Feature Elimination algorithm to select input variables (features) in order to reduce the risk of model overfitting and increase model explainability. Among several ML models, GBM is found to yield the best prediction accuracy. Extensive post-modeling investigation is conducted in a comparative manner between 2009 and 2017, including quantifying the importance of each input variable in predicting online shopping demand, and characterizing value-dependent relationships between demand and the input variables. In doing so, two latest advances in machine learning techniques, namely Shapley value-based feature importance and Accumulated Local Effects plots, are adopted to overcome inherent drawbacks of the popular techniques in current ML modeling. The modeling and investigation are performed at the national level, with a number of findings obtained. The models developed and insights gained can be used for online shopping-related freight demand generation and may also be considered for evaluating the potential impact of relevant policies on online shopping demand.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2025

Enhancing stability, magnetic anisotropy, and coercivity of manganese aluminum: Machine learning, ab initio , and micromagnetic modeling

The binary manganese aluminum (MnAl) alloy with L⁢1 0 crystal structure is a promising rare earth (RE) element-free permanent magnetic material because of its exceptional magnetic properties. However, experimentally synthesizing it in a stable bulk form is extremely challenging. Here, in this study, an alternative method of stabilizing the material, a pathway for experimental synthesis and validation, is proposed and theoretically verified. This is done by partially substituting Mn and Al sites with Fe and Ni and identifying its enhanced phase stability, saturation magnetization density, magnetic anisotropy, and coercivity from density functional theory (DFT), machine learning (ML) crystal graph convolution neural network (CGCNN), and micro-magnetic modeling. When considering a fixed 50% Ni, the magnetic anisotropy increases with the increasing Fe content but decreases the formation energy. The calculated formation energies, elastic constants, and phonon frequencies demonstrate that the binary and quaternary compositions are stable. Most importantly, in 50% Fe and Ni-substituted-equiatomic phase, magnetic anisotropy constants and saturation magnetization density increase by 56% and 23% as compared to the MnAl. Further, the coercivity of the equiatomic phase predicted with micro-magnetic modeling is higher by 17% than the parent compound.

Bhandari, Churna [Ames National Laboratory, and Io↗
DOE OSTI2022

Deep Generative Models for Materials Discovery and Machine Learning-Accelerated Innovation

Machine learning and artificial intelligence (AI/ML) methods are beginning to have significant impact in chemistry and condensed matter physics. For example, deep learning methods have demonstrated new capabilities for high-throughput virtual screening, and global optimization approaches for inverse design of materials. Recently, a relatively new branch of AI/ML, deep generative models (GMs), provide additional promise as they encode material structure and/or properties into a latent space, and through exploration and manipulation of the latent space can generate new materials. These approaches learn representations of a material structure and its corresponding chemistry or physics to accelerate materials discovery, which differs from traditional AI/ML methods that use statistical and combinatorial screening of existing materials via distinct structure-property relationships. However, application of GMs to inorganic materials has been notably harder than organic molecules because inorganic structure is often more complex to encode. In this work we review recent innovations that have enabled GMs to accelerate inorganic materials discovery. We focus on different representations of material structure, their impact on inverse design strategies using variational autoencoders or generative adversarial networks, and highlight the potential of these approaches for discovering materials with targeted properties needed for technological innovation.

36 MATERIALS SCIENCE↗
NASA NTRSDate not supplied

Dust Machine Learning Probability and Assessment

- NASA SPoRT introduced the "Dust RGB" via NASA satellites to demonstrate GOES-R ABI capabilities and then evaluated the impact in operations (Fuell et al. 2016) - The Dust RGB allows for continued dust detection at night, but the cooling ground surface limits the effectiveness as night progresses. - SPoRT has developed a 'Machine Learning' (ML) model using a physically-based approach which can correctly label 85% of dust pixels and 99% of no-dust pixels

Dust↗
DOE OSTI2021

Machine Learning Tools Set for Natural Gas Fuel Cell System Design

This study is focusing on leveraging the system design tools set for the next-generation solid oxide fuel cell (SOFC) based natural gas fuel cell (NGFC) system. Conventionally, system design and optimization of NGFC systems rely heavily on traditional reduced order model (ROM) techniques and designers’ experience level. For overcoming the technical barriers of system design, multiple multi-physics models and machine learning (ML) tools have been utilized to automate the conceptual design process and enhance the reliability of solutions for the NGFC system. The proposed tools set includes a physics-informed ML tool for automated ROM construction that leverages advances in deep neural networks to significantly reduce ROM prediction error for the NGFC power island compared to traditional approaches. The constructed physics-informed ML ROM can be used in system design, and optimization tools set Institute for the Design of Advanced Energy Systems (IDAES) Process Systems Engineering (PSE) framework. The tools set also provides a user-friendly graphic user interface built within Jupyter Notebooks, and the complete tools set is open-source public available.

Wang, Dewei↗