Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “machine learning models”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 559 records · Page 31
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Understanding Twinning and Deformation in High Entropy Alloys

A combination of high strength and high ductility has been observed in multi-principal element alloys due to twin formation attributed to low stacking fault energy (SFE). In the pursuit of low SFE alloys, a key bottleneck is the lack of understanding of the composition–SFE cor- relations that would guide tailoring SFE via alloy composition. Using density functional theory (DFT), we show that dopant radius, which have been postulated as a key descriptor for SFE in dilute alloys, does not fully explain SFE trends across different host metals. Instead, charge density is a much more central descriptor. It allows us to (1) explain contrasting SFE trends in Ni and Cu host metals due to various dopants in dilute concentrations, (2) explain the large SFE variations observed in the literature even within a given alloy composition due to the nearest neighbor environments in “model” concentrated alloys, and (3) develop a machine learning model that can be used to predict SFEs in multi-elemental alloys. This model opens a possibility to use charge density as a descriptor for predicting SFE in alloys. Furthermore, a descriptor-less machine learning (ML) model based only on charge density images extracted from density functional theory (DFT) is developed to predict stacking fault energies (SFE) in concentrated alloys. The model is based on convolutional neural networks (CNNs) as one of the promising ML techniques for dealing with complex images and data. Identification of correct descriptors is a key bottleneck to develop ML models for predicting materials properties. Often, in most ML models, textbook physical descriptors such as atomic radius, valence charge and electronegativity are used as descriptors which have limitations because these properties change in concentrated alloys when multiple elements are mixed to form a solid solution. We illustrate that, within the scope of DFT, the search for descriptors can be circumvented by electronic charge density, which is the backbone of the Kohn-Sham DFT and describes the system completely. The performance of our model is demonstrated by predicting SFE of concentrated alloys with an RMSE and R2 of 6.18 mJ/m2 and 0.87, respectively, validating the accuracy of the proposed approach.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Assessment of fine-tuned large language models for real-world chemistry and material science applications

The current generation of large language models (LLMs) has limited chemical knowledge. Recently, it has been shown that these LLMs can learn and predict chemical properties through fine-tuning. Using natural language to train machine learning models opens doors to a wider chemical audience, as field-specific featurization techniques can be omitted. In this work, we explore the potential and limitations of this approach. We studied the performance of fine-tuning three open-source LLMs (GPT-J-6B, Llama-3.1-8B, and Mistral-7B) for a range of different chemical questions. We benchmark their performances against “traditional” machine learning models and find that, in most cases, the fine-tuning approach is superior for a simple classification problem. Depending on the size of the dataset and the type of questions, we also successfully address more sophisticated problems. The most important conclusions of this work are that, for all datasets considered, their conversion into an LLM fine-tuning training set is straightforward and that fine-tuning with even relatively small datasets leads to predictive models. These results suggest that the systematic use of LLMs to guide experiments and simulations will be a powerful technique in any research study, significantly reducing unnecessary experiments or computations.

Van Herck, Joren↗
DOE OSTI2024

A semi-supervised learning method to produce explainable radioisotope proportion estimates for NaI-based synthetic and measured gamma spectra

Quantifying the radioactive sources present in gamma spectra is an ever-present and growing national security mission and a time-consuming process for human analysts. While machine learning models exist that are trained to estimate radioisotope proportions in gamma spectra, few address the eventual need to provide explanatory outputs beyond the estimation task. In this work, we develop two machine learning models for a NaI detector measurements: one to perform the estimation task, and the other to characterize the first model’s ability to provide reasonable estimates. To ensure the first model exhibits a behavior that can be characterized by the second model, the first model is trained using a custom, semi-supervised loss function which constrains proportion estimates to be explainable in terms of a spectral reconstruction. The second auxiliary model is an out-of-distribution detection function (a type of meta-model) leveraging the proportion estimates of the first model to identify when a spectrum is sufficiently unique from the training domain and thus is out-of-scope for the model. In demonstrating the efficacy of this approach, we encourage the use of meta-models to better explain ML outputs used in radiation detection and increase trust.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
NASA NTRS2021

A Machine Learning Approach to Predict Aircraft Landing Times using Mediated Predictions from Existing Systems

We developed a novel approach for predicting the landing time of airborne flights in real-time operations. The first step predicts a landing time by using mediation rules to select from among physics-based predictions (relying on the expected flight trajectory) already available in real time in the Federal Aviation Administration System Wide Information Management system data feeds. The second step uses a machine learning model built upon the mediated predictions. The model is trained to predict the error in the mediated prediction, using features describing the current state of an airborne flight. These features are calculated in real time from a relatively small number of data elements that are readily available for airborne flights. Initial results based on five months of data at six large airports demonstrate that incorporating a machine learning model on top of the mediated physics-based prediction can lead to substantial additional improvements in prediction quality.

Machine learning↗
NASA NTRS2021

A Machine Learning Approach to Predict Aircraft Landing Times using Mediated Predictions from Existing Systems

We developed a novel approach for predicting the landing time of airborne flights in real-time operations. The first step predicts a landing time by using mediation rules to select from among physics-based predictions (relying on the expected flight trajectory) already available in real time in the Federal Aviation Administration System Wide Information Management system data feeds. The second step uses a machine learning model built upon the mediated predictions. The model is trained to predict the error in the mediated prediction, using features describing the current state of an airborne flight. These features are calculated in real time from a relatively small number of data elements that are readily available for airborne flights. Initial results based on five months of data at six large airports demonstrate that incorporating a machine learning model on top of the mediated physics-based prediction can lead to substantial additional improvements in prediction quality.

Machine learning↗
DOE OSTI2020

Exploiting Machine Learning to Efficiently Predict Multidimensional Optical Spectra in Complex Environments

The excited-state dynamics of chromophores in complex environments determine a range of vital biological and energy capture processes. Time-resolved, multidimensional optical spectroscopies provide a key tool to investigate these processes. Although theory has the potential to decode these spectra in terms of the electronic and atomistic dynamics, the need for large numbers of excited-state electronic structure calculations severely limits first-principles predictions of multidimensional optical spectra for chromophores in the condensed phase. In this work, we leverage the locality of chromophore excitations to develop machine learning models to predict the excited-state energy gap of chromophores in complex environments for efficiently constructing linear and multidimensional optical spectra. By analyzing the performance of these models, which span a hierarchy of physical approximations, across a range of chromophore–environment interaction strengths, we provide strategies for the construction of machine learning models that greatly accelerate the calculation of multidimensional optical spectra from first principles.

36 MATERIALS SCIENCE↗
DOE OSTI2020

Hindsight logging for model training

In modern Machine Learning, model training is an iterative, experimental process that can consume enormous computation resources and developer time. To aid in that process, experienced model developers log and visualize program variables during training runs. Exhaustive logging of all variables is infeasible, so developers are left to choose between slowing down training via extensive conservative logging, or letting training run fast via minimalist optimistic logging that may omit key information. As a compromise, optimistic logging can be accompanied by program checkpoints; this allows developers to add log statements post-hoc, and "replay" desired log statements from checkpoint---a process we refer to as hindsight logging. Unfortunately, hindsight logging raises tricky problems in data management and software engineering. Done poorly, hindsight logging can waste resources and generate technical debt embodied in multiple variants of training code. In this paper, we present methodologies for efficient and effective logging practices for model training, with a focus on techniques for hindsight logging. Our goal is for experienced model developers to learn and adopt these practices. To make this easier, we provide an open-source suite of tools for Fast Low-Overhead Recovery (flor) that embodies our design across three tasks: (i) efficient background logging in Python, (ii) adaptive periodic checkpointing, and (iii) an instrumentation library that codifies hindsight logging for efficient and automatic record-replay of model-training. Model developers can use each flor tool separately as they see fit, or they can use flor in hands-free mode, entrusting it to instrument their code end-to-end for efficient record-replay. Our solutions leverage techniques from physiological transaction logs and recovery in database systems. Evaluations on modern ML benchmarks demonstrate that flor can produce fast checkpointing with small user-specifiable overheads (e.g. 7%), and still provide hindsight log replay times orders of magnitude faster than restarting training from scratch.

Computer Science↗
DOE OSTI2020

Detection of topological materials with machine learning

Databases compiled using ab initio and symmetry-based calculations now contain tens of thousands of topological insulators and topological semimetals. This makes the application of modern machine learning methods to topological materials possible. Using gradient boosted trees, we show how to construct a machine learning model which can predict the topology of a given existent material with an accuracy of 90%. Such predictions are orders of magnitude faster than actual ab initio calculations. In this work, we use machine learning models to probe how different material properties affect topological features. Notably, we observe that topology is mostly determined by the “coarse-grained” chemical composition and crystal symmetry and depends little on the particular positions of atoms in the crystal lattice. We identify the sources of our model's errors and we discuss approaches to overcome them.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
NASA NTRSDate not supplied

Off-Nominal Event Analysis in Autonomous Flights Based on Explainable Artificial Intelligence

A key objective in the Urban Air Mobility program at NASA is to intelligently perform an autonomous flight in a complex urban environment under all weather conditions with guaranteed levels of safety. To accomplish this, the mission manager (central decision-making module) of the vehicle needs to make informed decisions between various Courses of Action (CoA) based on its' interpretation of the inputs it receives. If an off-nominal event is detected either based on the amalgamation of sensor data or the use of machine learning models, the mission manager may greatly benefit from identification of the input features that most likely contributed to that specific event. Such an understanding is usually not possible to obtain from the classical machine learning models (deep learning) due to the inherent black box like structure. However, this understanding is achieved using eXplainable Artificial Intelligence (XAI) models that provide a human interpretable rationale for the predictions made. This work presents a game theory inspired XAI model for the off-nominal assessment of autonomous flights. The proposed approach based on Shapley values is model agnostic, provides local as well as global explanation and satisfies the four axioms (efficiency, symmetry, dummy, additivity) to achieve fair contribution. The versatility of the approach is first demonstrated on a simulated dataset in which the significance of each input to flight phase prediction is clearly identified. Subsequently, data from simulated flight trajectories are fed into the model which reveal the input features that most likely contributed to a rotor failure event thereby empowering the mission manager to take the appropriate CoA.

autonomy↗
NASA NTRSDate not supplied

Off-Nominal Event Analysis in Autonomous Flights Based on Explainable Artificial Intelligence

A key objective in the Urban Air Mobility program at NASA is to intelligently perform an autonomous flight in a complex urban environment under all weather conditions with guaranteed levels of safety. To accomplish this, the mission manager (central decision-making module) of the vehicle needs to make informed decisions between various Courses of Action (CoA) based on its' interpretation of the inputs it receives. If an off-nominal event is detected either based on the amalgamation of sensor data or the use of machine learning models, the mission manager may greatly benefit from identification of the input features that most likely contributed to that specific event. Such an understanding is usually not possible to obtain from the classical machine learning models (deep learning) due to the inherent black box like structure. However, this understanding is achieved using eXplainable Artificial Intelligence (XAI) models that provide a human interpretable rationale for the predictions made. This work presents a game theory inspired XAI model for the off-nominal assessment of autonomous flights. The proposed approach based on Shapley values is model agnostic, provides local as well as global explanation and satisfies the four axioms (efficiency, symmetry, dummy, additivity) to achieve fair contribution. The versatility of the approach is first demonstrated on a simulated dataset in which the significance of each input to flight phase prediction is clearly identified. Subsequently, data from simulated flight trajectories are fed into the model which reveal the input features that most likely contributed to a rotor failure event thereby empowering the mission manager to take the appropriate CoA.

autonomy↗
DOE OSTI2024

LandScan Global 2023: Silver Edition

For a quarter of a century, the LandScan Global (LSG) project has annually released a global, high-resolution gridded population dataset representing the ambient or unwarned population at a 30 arcsecond resolution. LSG supports a range of applications such as emergency management, disaster response, and human health and security for understanding populations at risk. The 2023 release of LSG, the LandScan Silver Edition, represents a major methodological leap forward while also leveraging previous knowledge—the previous year was the baseline for the current annual update carrying forward valuable knowledge of the built environment for the past quarter century—to train the machine learning models. Compared with annual releases over the past 24years, multiple advancements were made to different aspects of the methodology to achieve reproducibility, transparency, and consistent global propagation of solutions to modeling or population distribution issues identified during the review process. These novel changes include incorporation of the latest available geospatial inputs across the globe, machine learning models instead of manual modifications, population feature importance analysis, open-source solutions vs. proprietary software, generation of multiple global versions, analytic validations, and human-in-the-loop revisions to produce the final version. Additionally, algorithms—such as anomaly detection—were introduced to quickly identify areas of focus to develop a new and robust systematic review. Significant changes in modeled population distributions were observed between the 2022 and 2023 releases, largely attributable to improvements in data and methods and discussed thoroughly within this report. In summation, the LandScan Silver Edition leverages the best of the past quarter century of LSG legacy knowledge and continues a tradition of applying cutting-edge enhancements to serve as a new benchmark for accurate, actionable gridded population data

Lebakula, Viswadeep↗
DOE OSTI2025

Temporal Forecasting of Distributed Temperature Sensing in a Thermal Hydraulic System With Machine Learning and Statistical Models

We benchmark performance of long-short term memory (LSTM) network machine learning model and autoregressive integrated moving average (ARIMA) statistical model in temporal forecasting of distributed temperature sensing (DTS). Data in this study consists of fluid temperature transient measured with two co-located Rayleigh scattering fiber optic sensors (FOS) in a forced convection mixing zone of a thermal tee. We treat each gauge of a FOS as an independent temperature sensor. We first study prediction of DTS time series using Vanilla LSTM and ARIMA models trained on prior history of the same FOS that is used for testing. The results yield maximum absolute percentage error (MaxAPE) and root mean squared percentage error (RMSPE) of 1.58% and 0.06% for ARIMA, and 3.14% and 0.44% for LSTM, respectively. Next, we investigate zero-shot forecasting (ZSF) with LSTM and ARIMA trained on history of the co-located FOS only, which is advantageous when limited training data is available. The ZSF MaxAPE and RMSPE values for ARIMA are comparable to those of the Vanilla use case, while the error values for LSTM increase. We show that in ZSF, performance of LSTM network can be improved by training on most correlated gauges between the two FOS, which are identified by calculating the Pearson correlation coefficient. The improved ZSF MaxAPE and RMSPE for LSTM are 4.4% and 0.33%, respectively. Performance of ZSF LSTM can be further enhanced through transfer learning (TL), where LSTM is re-trained on a subset of the FOS that is the target of forecasting. We show that LSTM pre-trained on correlated dataset and re-trained on 30% of testing target dataset achieves MaxAPE and RMSPE values of 2.32% and 0.28%, respectively.

ARIMA↗
DOE OSTI2019

Quantifying Climate and Catchment Control on Hydrological Drought in the Continental United States

The evolution of hydrological drought events is a result of complex (nonlinear) interactions between climate and catchment processes. To investigate such nonlinear relationship, we integrated a machine learning modeling framework based on the random forest (RF) algorithms with an interpretation framework to quantify the role of climate and catchment controls on hydrological drought. More particularly, our framework interprets a built RF machine-learning model to identify dominant variables and visualize their functional dependence and interaction effects on hydrological drought characteristics utilizing concepts of minimal depth, interactive depth, and partial dependence. We test our proposed modeling framework based on a set of 652 continental United States catchments with minimal human interference for a period of 1979–2010. Application of this framework indicated presence of three distinct drought regimes, which includes, Regime 1: droughts with longer duration, less frequent and lesser intensity; Regime 2: droughts with moderate duration, moderate frequency, and moderate intensity; and Regime 3: droughts with shorter duration, more frequent, and more intense. RF algorithm was able to accurately model the drought characteristics (intensity, duration, and number of events) for all the three drought regimes as a function of selected variables. It was observed that the type of dominant variables as well as their nonlinear functional relationship with hydrological droughts characteristics can vary between three selected regimes. Finally, our interpretation framework indicated that catchment characteristics have a significant role in controlling the hydrologic drought for catchments (regime 1), whereas both climate and catchment characteristics control hydrological drought in regimes 2 and 3.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Training models using forces computed by stochastic electronic structure methods

Abstract Quantum Monte Carlo (QMC) can play a very important role in generating accurate data needed for constructing potential energy surfaces. We argue that QMC has advantages in terms of a smaller systematic bias and an ability to cover phase space more completely. The stochastic noise can ease the training of the machine learning model. We discuss how stochastic errors affect the generation of effective models by analyzing the errors within a linear least squares procedure, finding that there is an advantage to having many relatively imprecise data points for constructing models. We then analyze the effect of noise on a model of many-body silicon finding that noise in some situations improves the resulting model. We then study the effect of QMC noise on two machine learning models of dense hydrogen used in a recent study of its phase diagram. The noise enables us to estimate the errors in the model. We conclude with a discussion of future research problems.

Ceperley, David M. (ORCID:0000000150826271)↗
DOE OSTI2026

From atomistic models to machine learning: Predictive design of nanocarbons under extreme conditions

The formation of technologically valuable nanocarbon structures under extreme conditions, such as those produced during high-explosive detonations, remains poorly understood but holds significant potential for the development of controlled synthesis pathways. While detonation shockwaves provide the high-pressure, high-temperature environment required for nanodiamond formation, subsequent cooling and decompression dictate whether the diamond phase is preserved or transformed into other nanocarbon structures. Here, in this study, we employ GPU-accelerated reactive molecular dynamics (ReaxFF) simulations to investigate the graphitization and structural remodeling of detonation nanodiamond under nonlinear quench and pressure-release trajectories. We further investigate how the initial nanodiamond morphology; cuboctahedral, octahedral, or hexagonal prism influences the resulting transformation products. Evolution of nanostructure, allotrope (via simulated x-ray diffraction), carbon hybridization, and ring statistics are tracked during a two-stage quench from 5000 K to 60 GPa. Rapid cooling combined with slow decompression optimizes cubic diamond retention, whereas slow cooling with rapid pressure release promotes surface-to-core graphitization, producing concentric sp 2 -hybridized layers and hollowed inner shells. Octahedral nanodiamonds evolve into carbon nano-onions, initially forming bucky diamonds that progressively transform into fully sp 2 -hybridized structures, while hexagonal prisms preferentially form parallel-stacked graphite layers resembling carbon dots. Transient hexagonal diamond (lonsdaleite) emerges as an interfacial phase, suggesting potential reversibility in the shock-induced graphite-to-diamond transformation pathway transformation route. To extend predictive capabilities, we trained machine learning (ML) regressors on over 10 5 node-hours of molecular dynamics (MD) trajectories. A multilayer perceptron (MLP) model reliably predicts the number of graphitized layers from temperature–pressure trajectories with a coefficient of determination (R 2 ) exceeding 0.90. This high predictive fidelity enables efficient, high-throughput mapping of the synthesis parameter space for optimized graphitization outcomes. Collectively, morphological control combined with optimized quench–decompression conditions promote the selective synthesis of nanocarbon allotropes. This work establishes a data-driven framework for the rational, a priori design of carbon nanomaterials for applications in energy storage, sensing, and biomedicine.

Detonation nanodiamond remodeling↗
NASA NTRS2021

A Chlorophyll-a Algorithm for Landsat-8 Based on Mixture Density Networks

Retrieval of aquatic biogeochemical variables, such as the near-surface concentration of chlorophyll-a (Chla) in inland and coastal waters via remote observations, has long been regarded as a challenging task. This manuscript applies Mixture Density Networks (MDN) that use the visible spectral bands available by the Operational Land Imager (OLI) aboard Landsat-8 to estimate Chla. We utilize a database of co-located in situ radiometric and Chla measurements (N = 4,354), referred to as Type A data, to train and test an MDN model (MDN(A)). This algorithm’s performance, having been proven for other satellite missions, is further evaluated against other widely used machine learning models (e.g., support vector machines), as well as other domain-specific solutions (OC3), and shown to offer significant advancements in the field. Our performance assessment using a held-out test data set suggests that a 49% (median) accuracy with near-zero bias can be achieved via the MDN(A) model, offering improvements of 20 to 100% in retrievals with respect to other models. The sensitivity of the MDN(A) model and benchmarking methods to uncertainties from atmospheric correction (AC) methods, is further quantified through a semi-global matchup dataset (N = 3,337), referred to as Type B data. To tackle the increased uncertainties, alternative MDN models (MDN(B)) are developed through various features of the Type B data (e.g., Rayleigh-corrected reflectance spectra ρ(s)). Using held-out data, along with spatial and temporal analyses, we demonstrate that these alternative models show promise in enhancing the retrieval accuracy adversely influenced by the AC process. Results lend support for the adoption of MDN(B) models for regional and potentially global processing of OLI imagery, until a more robust AC method is developed. Index Terms—Chlorophyll-a, coastal water, inland water, Landsat-8, machine learning, ocean color, aquatic remote sensing.

Brandon Smith↗
DOE OSTI2023

GRAPH CONVOLUTIONAL NEURAL NETWORK (GCNN)FOR POLYMER PROPERTY PREDICTION

SF-22-121 Graph convolutional neural networks (GCNN) are powerful machine learning models that apply a message-passing algorithm on adjacent nodes in a graph. Due to the natural representation of molecules via discrete graphs (where nodes encode atoms and edges encode bonds), GCNNs are an intuitive machine learning model chemical space informatics, including the prediction of polymer properties. Here, molecules are represented in feature and adjacency matrices where they are directly used as inputs to a GCNN.

FEINSTEIN, JEREMY↗
DOE OSTI2020

Toward fusion plasma scenario planning for NSTX-U using machine-learning-accelerated models

One of the most promising devices for realizing power production through nuclear fusion is the tokamak. To maximize performance, it is preferable that tokamak reactors achieve advanced operating scenarios characterized by good plasma confinement, improved magnetohydrodynamic (MHD) stability, and a largely non-inductively driven plasma current. Such scenarios could enable steady-state reactor operation with high \emph{fusion gain} --- the ratio of produced fusion power to the external power provided through the plasma boundary. Precise and robust control of the evolution of the plasma boundary shape as well as the spatial distribution of the plasma current, density, temperature, and rotation will be essential to achieving and maintaining such scenarios. The complexity of the evolution of tokamak plasmas, arising due to nonlinearities and coupling between various parameters, motivates the use of model-based control algorithms that can account for the system dynamics. In this work, a learning-based accelerated model trained on data from the National Spherical Torus Experiment Upgrade (NSTX-U) is employed to develop planning and control strategies for regulating the density and temperature profile evolution around desired trajectories. The proposed model combines empirical scaling laws developed across multiple devices with neural networks trained on empirical data from NSTX-U and a database of first-principles-based computationally intensive simulations. The reduced execution time of the accelerated model will enable practical application of optimization algorithms and reinforcement learning approaches for scenario planning and control development. An initial demonstration of applying optimization approaches to the learning-based model is presented, including a strategy for mitigating the effect of leaving the finite validity range of the accelerated model. The approach shows promise for actuator planning between experiments and in real-time.

machine learning↗