Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “learning theory”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 127 records · Page 7
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Machine Learning Predictions of Transition Probabilities in Atomic Spectra

Forward modeling of optical spectra with absolute radiometric intensities requires knowledge of the individual transition probabilities for every transition in the spectrum. In many cases, these transition probabilities, or Einstein A-coefficients, quickly become practically impossible to obtain through either theoretical or experimental methods. Complicated electronic orbitals with higher order effects will reduce the accuracy of theoretical models. Experimental measurements can be prohibitively expensive and are rarely comprehensive due to physical constraints and sheer volume of required measurements. Due to these limitations, spectral predictions for many element transitions are not attainable. In this work, we investigate the efficacy of using machine learning models, specifically fully connected neural networks (FCNN), to predict Einstein A-coefficients using data from the NIST Atomic Spectra Database. For simple elements where closed form quantum calculations are possible, the data-driven modeling workflow performs well but can still have lower precision than theoretical calculations. For more complicated nuclei, deep learning emerged more comparable to theoretical predictions, such as Hartree–Fock. Unlike experiment or theory, the deep learning approach scales favorably with the number of transitions in a spectrum, especially if the transition probabilities are distributed across a wide range of values. It is also capable of being trained on both theoretical and experimental values simultaneously. In addition, the model performance improves when training on multiple elements prior to testing. The scalability of the machine learning approach makes it a potentially promising technique for estimating transition probabilities in previously inaccessible regions of the spectral and thermal domains on a significantly reduced timeline.

74 ATOMIC AND MOLECULAR PHYSICS↗
DOE OSTI2024

Prediction of O and OH Adsorption on Transition Metal Oxide Surfaces from Bulk Descriptors

In the search for stable and active catalysts, density functional theory and machine learning (ML) based models can accelerate the screening of materials. While stability is conveniently addressed on the bulk level of computation, the modelling of catalytic activity requires expensive surface simulations. Here, in this work, we develop models for the surface adsorption energy of O and OH intermediates across a consistent and extensive dataset of pure transition metal oxide surfaces. We show that adsorption energies across metal oxidation states of +2 to +6 are well captured from the metal-oxygen bond strength extracted from the bulk level calculation. Specifically, we calculate the integrated crystal orbital Hamiltonian population (ICOHP) of the metal-oxygen bond in the bulk oxide and employ a simple normalization scheme to obtain a strong correlation with adsorption energetics. By combining our ICOHP descriptor with non DFT features in a Gaussian Process regression (GPR) model, we achieve high model accuracy with mean absolute errors of 0.166 and 0.219 eV for OH and O adsorption, respectively. By targeting the O-OH adsorption energy difference with our GPR model, we predict the the oxygen evolution reaction (OER) activity from bulk descriptors only. Furthermore, we utilize the strong correlation between the COHP and metal oxygen bond lengths to rapidly predict adsorption energetics and catalytic activity from the optimized bulk geometry. Our approach can enable an efficient search for active catalysts by eliminating the need for surface calculations in the initial screening phase.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2025

Materials Learning Algorithms (MALA): Scalable machine learning for electronic structure calculations in large-scale atomistic simulations

We present the Materials Learning Algorithms (MALA) package, a scalable machine learning framework designed to accelerate density functional theory (DFT) calculations suitable for large-scale atomistic simulations. Using local descriptors of the atomic environment, MALA models efficiently predict key electronic observables, including local density of states, electronic density, density of states, and total energy. The package integrates data sampling, model training and scalable inference into a unified library, while ensuring compatibility with standard DFT and molecular dynamics codes. We demonstrate MALA's capabilities with examples including boron clusters, aluminum across its solid-liquid phase boundary, and predicting the electronic structure of a stacking fault in a large beryllium slab. Scaling analyses reveal MALA's computational efficiency and identify bottlenecks for future optimization. With its ability to model electronic structures at scales far beyond standard DFT, MALA is well suited for modeling complex material systems, making it a versatile tool for advanced materials research.

Density functional theory↗
DOE OSTI2021

The Rise of Neural Networks for Materials and Chemical Dynamics

Machine learning (ML) is quickly becoming a premier tool for modeling chemical processes and materials. ML-based force fields, trained on large data sets of high-quality electron structure calculations, are particularly attractive due their unique combination of computational efficiency and physical accuracy. This Perspective summarizes some recent advances in the development of neural network-based interatomic potentials. Designing high-quality training data sets is crucial to overall model accuracy. One strategy is active learning, in which new data are automatically collected for atomic configurations that produce large ML uncertainties. Another strategy is to use the highest levels of quantum theory possible. Transfer learning allows training to a data set of mixed fidelity. A model initially trained to a large data set of density functional theory calculations can be significantly improved by retraining to a relatively small data set of expensive coupled cluster theory calculations. These advances are exemplified by applications to molecules and materials.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2020

Spectroscopic Probe Molecule Selection Using Quantum Theory, First-Principles Calculations, and Machine Learning

Probe molecule vibrational spectra have a long history of being used to characterize materials including metals, oxides, metal-organic frameworks, and even human proteins. Furthermore, recent advances in machine learning have enabled computationally generated spectra to aid in detailed characterization of complex surfaces with probe molecules. Despite widespread use of probe molecules, the science of probe molecule selection is underdeveloped. Here, we develop physical concepts, including orbital interaction energy and the energy overlap integral, to explain and predict the ability of probe molecules to discriminate structural descriptors. We resolve the crystal orbital overlap population (COOP) to specific molecular orbitals and quantify their bonding character, which directly influences vibrational frequencies. Using only a single adsorbate calculation from density function theory (DFT), we compute the interaction energy of individual adsorbate molecular orbitals with adsorption site atomic orbitals across many different sites. Combining the molecular orbital resolved COOP and changes in orbital interaction energy enables probe molecule selection for improved discrimination of various sites. We demonstrate these concepts by comparing the predicted effectiveness of carbon monoxide (CO), nitric oxide (NO), and ethylene (C 2 H 4 ) to probe Pt adsorption sites. Finally, using a previously developed machine learning framework, we show that models trained on hundreds of thousand C 2 H 4 spectra, computed from DFT, which regress surface binding-type and generalized coordination number (GCN), outperform those trained using CO and NO spectra. Lastly, a python package, pDOS_overlap, for implementing the electron density based analysis on any combination of adsorbates and materials, is also made available.

77 NANOSCIENCE AND NANOTECHNOLOGY↗
DOE OSTI2025

Exploring the energy landscape of RBMs: reciprocal space insights into bosons, hierarchical learning and symmetry breaking

Deep generative models have become ubiquitous due to their ability to learn and sample from complex distributions. Despite the proliferation of various frameworks, the relationships among these models remain largely unexplored, a gap that hinders the development of a unified theory of AI learning. In this work, we address two central challenges: clarifying the connections between different deep generative models and deepening our understanding of their learning mechanisms. We focus on Restricted Boltzmann Machines (RBMs), a class of generative models known for their universal approximation capabilities for discrete distributions. By introducing a reciprocal space formulation for RBMs, we reveal a connection between these models, diffusion processes, and systems of coupled bosons. Our analysis shows that at initialization, the RBM operates at a saddle point, where the local curvature is determined by the singular values of the weight matrix, whose distribution follows the Marc̆enko-Pastur law and exhibits rotational symmetry. During training, this rotational symmetry is broken due to hierarchical learning, where different degrees of freedom progressively capture features at multiple levels of abstraction. This leads to a symmetry breaking in the energy landscape, reminiscent of Landau’s theory. This symmetry breaking in the energy landscape is characterized by the singular values and the weight matrix eigenvector matrix. We derive the corresponding free energy in a mean-field approximation. We show that in the limit of infinite size RBM, the reciprocal variables are Gaussian distributed. Our findings indicate that in this regime, there will be some modes for which the diffusion process will not converge to the Boltzmann distribution. To illustrate our results, we trained replicas of RBMs with different hidden layer sizes using the MNIST dataset. Our findings not only bridge the gap between disparate generative frameworks but also shed light on the fundamental processes underpinning learning in deep generative models.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Machine learning prediction and experimental verification of Pt-modified nitride catalysts for ethanol reforming with reduced precious metal loading

Ethanol is the smallest molecule containing C–O, C–C, C–H, and O–H bonds present in biomass-derived oxygenates. The development of inexpensive and selective catalysts for ethanol reforming is important towards the renewable generation of hydrogen from biomass. Transition metal nitrides (TMN) are interesting catalyst support materials that can effectively reduce precious metal loading for the catalysis of ethanol and other oxygenates. Herein theoretical and experimental methods were used to probe platinum-modified molybdenum nitride (Pt/Mo 2 N) surfaces for ethanol reforming. Computations using density-functional theory and machine learning predicted monolayer Pt/Mo 2 N to be highly active and selective for ethanol reforming. Temperature-programmed desorption (TPD) experiments verified that ethanol primarily underwent decomposition on Mo 2 N, and the reaction pathway shifted to reforming on Pt/Mo 2 N surfaces. Additionally, high-resolution electron energy loss spectroscopy (HREELS) results further indicated that while Mo2N decomposed the ethoxy intermediate by cleaving C–C, C–O, and C–H bonds, Pt-modification preserved the C–O bond, resulting in ethanol reforming.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
NASA NTRS1992

There is No Free Lunch: Tradeoffs in the Utility of Learned Knowledge

With the recent introduction of learning in integrated systems, there is a need to measure the utility of learned knowledge for these more complex systems. A difficulty arrises when there are multiple, possibly conflicting, utility metrics to be measured. In this paper, we present schemes which trade off conflicting utility metrics in order to achieve some global performance objectives. In particular, we present a case study of a multi-strategy machine learning system, mutual theory refinement, which refines world models for an integrated reactive system, the Entropy Reduction Engine. We provide experimental results on the utility of learned knowledge in two conflicting metrics - improved accuracy and degraded efficiency. We then demonstrate two ways to trade off these metrics. In each, some learned knowledge is either approximated or dynamically 'forgotten' so as to improve efficiency while degrading accuracy only slightly.

Kedar, Smadar T.↗
NASA NTRS1992

Indirect decentralized learning control

The new field of learning control develops controllers that learn to improve their performance at executing a given task, based on experience performing this specific task. In a previous work, the authors presented a theory of indirect learning control based on use of indirect adaptive control concepts employing simultaneous identification and control. This paper develops improved indirect learning control algorithms, and studies the use of such controllers in decentralized systems. The original motivation of the learning control field was learning in robots doing repetitive tasks such as on an assembly line. This paper starts with decentralized discrete time systems, and progresses to the robot application, modeling the robot as a time varying linear system in the neighborhood of the nominal trajectory, and using the usual robot controllers that are decentralized, treating each link as if it is independent of any coupling with other links. The basic result of the paper is to show that stability of the indirect learning controllers for all subsystems when the coupling between subsystems is turned off, assures convergence to zero tracking error of the decentralized indirect learning control of the coupled system, provided that the sample time in the digital learning controller is sufficiently short.

Longman, Richard W.↗
DOE OSTI2022

On the specificity between mapping of initial and final states of the magneto Rayleigh-Taylor instability

In this LDRD we investigated the application of machine learning methods to understand dimensionality reduction and evolution of the Rayleigh-Taylor instability (RTI). As part of the project, we undertook a significant literature review to understand current analytical theory and machine learning based methods to treat evolution of this instability. We note that we chose to refocus on assessing the hydrodynamic RTI as opposed to the magneto-Rayleigh-Taylor instability originally proposed. This choice enabled utilizing a wealth of analytic test cases and working with relatively fast running open-source simulations of single-mode RTI. This greatly facilitated external collaboration with URA summer fellowship student, Theodore Broeren. In this project we studied the application of methods from dynamical systems learning and traditional regression methods to recover behavior of RTI ranging from the fully nonlinear to weakly nonlinear (wNL) regimes. Here we report on two of the tested methods SINDy and a more traditional regression-based approach inspired by analytic wNL theory with which we had the most success. We conclude with a discussion of potential future extensions to this work that may improve our understanding from both theoretical and phenomenological perspectives.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2024

Phase Selection Rules of Multi‐Principal Element Alloys

Abstract Computational prediction of phase stability of multi‐principal element alloys (MPEAs) holds a lot of promise for rapid exploration of the enormous design space and autonomous discovery of superior structural and functional properties. Regardless of many plausible works that rely on phenomenological theory and machine learning, precise prediction is still limited by insufficient data and the lack of interpretability of some machine learning algorithms, e.g., convolutional neural network. In this work, a comprehensive approach is presented, encompassing the development of a complete dataset that contains 72 387 density functional theory calculations, as well as a predictive global phenomenological descriptor. The phase selection descriptor, based on atomic electronegativity and valence electron concentration, significantly outperforms the widely used valence electron concentration, excelling in both accuracy (with an f1 score of 63% compared to 47%) and its ability to predict the HCP phase (0.48 recall compared to 0). The comprehensive data mining on the global design space of 61 425 quaternary MPEAs made from 28 possible metals, together with the phenomenological theory and physical interpretation, will set up a solid computational science foundation for data‐driven exploration of MPEAs.

Chemistry↗
DOE OSTI2024

Selective consolidation of learning and memory via recall-gated plasticity

In a variety of species and behavioral contexts, learning and memory formation recruits two neural systems, with initial plasticity in one system being consolidated into the other over time. Moreover, consolidation is known to be selective; that is, some experiences are more likely to be consolidated into long-term memory than others. Here, we propose and analyze a model that captures common computational principles underlying such phenomena. The key component of this model is a mechanism by which a long-term learning and memory system prioritizes the storage of synaptic changes that are consistent with prior updates to the short-term system. This mechanism, which we refer to as recall-gated consolidation, has the effect of shielding long-term memory from spurious synaptic changes, enabling it to focus on reliable signals in the environment. We describe neural circuit implementations of this model for different types of learning problems, including supervised learning, reinforcement learning, and autoassociative memory storage. These implementations involve synaptic plasticity rules modulated by factors such as prediction accuracy, decision confidence, or familiarity. We then develop an analytical theory of the learning and memory performance of the model, in comparison to alternatives relying only on synapse-local consolidation mechanisms. We find that recall-gated consolidation provides significant advantages, substantially amplifying the signal-to-noise ratio with which memories can be stored in noisy environments. We show that recall-gated consolidation gives rise to a number of phenomena that are present in behavioral learning paradigms, including spaced learning effects, task-dependent rates of consolidation, and differing neural representations in short- and long-term pathways.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2024

Selective consolidation of learning and memory via recall-gated plasticity

In a variety of species and behavioral contexts, learning and memory formation recruits two neural systems, with initial plasticity in one system being consolidated into the other over time. Moreover, consolidation is known to be selective; that is, some experiences are more likely to be consolidated into long-term memory than others. Here, we propose and analyze a model that captures common computational principles underlying such phenomena. The key component of this model is a mechanism by which a long-term learning and memory system prioritizes the storage of synaptic changes that are consistent with prior updates to the short-term system. This mechanism, which we refer to as recall-gated consolidation, has the effect of shielding long-term memory from spurious synaptic changes, enabling it to focus on reliable signals in the environment. We describe neural circuit implementations of this model for different types of learning problems, including supervised learning, reinforcement learning, and autoassociative memory storage. These implementations involve synaptic plasticity rules modulated by factors such as prediction accuracy, decision confidence, or familiarity. We then develop an analytical theory of the learning and memory performance of the model, in comparison to alternatives relying only on synapse-local consolidation mechanisms. We find that recall-gated consolidation provides significant advantages, substantially amplifying the signal-to-noise ratio with which memories can be stored in noisy environments. We show that recall-gated consolidation gives rise to a number of phenomena that are present in behavioral learning paradigms, including spaced learning effects, task-dependent rates of consolidation, and differing neural representations in short- and long-term pathways.

Lindsey, Jack W. (ORCID:0000000309307327)↗
DOE OSTI2022

From predicting to learning dissipation from pair correlations of active liquids

Active systems, which are driven out of equilibrium by local non-conservative forces, can adopt unique behaviors and configurations. Towards designing such materials, an important challenge is to precisely connect the static structure of active systems to the dissipation of energy induced by the local driving. Here, we use tools from liquid-state theories and machine learning to take on these challenges. We first demonstrate analytically for an isotropic active matter system that dissipation and pair correlations are closely related when driving forces behave like an active temperature. We then extend a nonequilibrium mean-field framework for predicting these pair correlations which, unlike most existing approaches, is applicable even for strongly interacting particles and far from equilibrium, to predict dissipation in these systems. Based on this theory, we reveal analytically a robust relation between dissipation and structure which holds even as the system approaches a nonequilibrium phase transition. Finally, we construct a neural network which maps static configurations of particles to their dissipation rate without any prior knowledge of the underlying dynamics. Furthermore, our results open novel perspectives on the interplay between dissipation and organization out-of-equilibrium.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2024

Defect Production and Microstructural Feature Impact for Radiation Damage in Additively Manufactured 316 Stainless Steel

This milestone presents multi-scale modeling research results for additively manufactured 316 stainless steel. A combination of phase field, cluster dynamics, molecular dynamics, and density functional theory with machine learning is used, allowing for predictions of radiation-driven microstructural evolution in additively manufactured 316 stainless steel over a range of temperatures, damage rates, neutron spectra, and microstructures, and supporting the development of combined ion and neutron irradiation for material qualification. Informed by ion irradiation and neutron irradiation results across the Advanced Materials and Manufacturing Technologies program, we investigate the unique aspects of radiation-driven microstructure evolution in additively manufactured 316 stainless steel. In particular, we focus on understanding the impact of carbon concentration (varying, for example, between the 316L and 316H standards) on void formation; radiation-induced segregation at dislocation cells and grain boundaries; and the evolution of dislocation loops and network dislocation populations. We find that the unique characteristics of the additively manufactured microstructures must be accounted for in understanding the evolution of dislocation populations under thermal and irradiation conditions, such as the variation in sink strengths arising due to the variation in dislocation density. We also find that the radiation-induced segregation of Cr and Ni to grain boundaries and cell walls differs due to the differences in their defect sink biases. We also find that increasing the Ni content can slow vacancy diffusion, which may provide a mechanism for the observed reduction in transient swelling rate for austenitic Fe-Cr-Ni alloys with increasing Ni content. In addition, ion irradiations have shown that increasing carbon content in 316 SS results in a larger population of smaller voids, suggesting reduced vacancy diffusion. Our results show that the carbon content of additively manufactured 316 SS has a significant impact on the migration rate of defect clusters. The presence of carbon atoms results in carbon-vacancy trapping, significantly reducing the diffusion rate of vacancies. Carbon atoms may also be trapped near the surface of a void, which may reduce void growth by trapping vacancies that diffuse toward the void.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗