Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Learning Theory”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 973 records · Page 54

Identifying and Documenting Expert Knowledge, A Practical Study of Design Patterns

Globalization, powered by digitization, is increasing technology growth and knowledge transfer rates to levels not seen in the previous 3500 years, obscuring absolute truth and accelerating rates of innovation and production. Competing,or remaining competitive, in this global marketplace requires a learning organization adopt a method of capture, retention, and reuse for demonstrated tacit knowledge to accelerate the development of increasingly complex systems of high quality at a reasonable cost. The architectural theory of Patterns and Pattern Language is a validated methodology for mining tacit domain knowledge from a proven system. This work applies architectural theory to a multi-year development experiment and captures exposed knowledgeas a design patternin a model based systems engineering tool, demonstrating applicability ofdigital engineering initiativesin thedescription and reuse of expert design knowledge.By creating and archiving model based expressions of expert knowledge,a learning organization canimprovepractical decision making and avoid uninformed concept phase decision making

systems engineering↗

Effects of Aluminum and Molybdenum on the Phase Stability of Iron-Chromium Alloys: A First-Principles Study

The interaction between solute atoms is critical to the thermodynamic behavior of Fe-Cr alloys, but the effects of non-dilute Al and Mo on the Fe-Cr phase stability and vacancy formation energy are not clearly understood. In this study, density functional theory, cluster expansion, and Monte Carlo simulation are used to predict the effects of ternary solute elements on the thermodynamic properties in multicomponent Fe-Cr alloys. The machine learning regression approach is applied to train and construct energy models that accurately describe the complex chemical interactions. The computational outcomes include the prediction of the partial ternary phase diagram, mixing enthalpy, and vacancy formation energy for different compositions. The phase boundary calculation predicts a pronounced change of Cr solubility in bcc Fe by the addition of Al and the rejection of Al atoms from a’ precipitates. The mixing enthalpy calculation shows that Mo may also reduce the Cr solubility in bcc Fe. Additionally, the simulations show that non-dilute Cr decreases the vacancy formation energy in bcc Fe, while adding Al results in a less significant effect. Finally, the results demonstrate important applications of using machine learning energy models to study model or commercial alloys with multicomponent solute species and point defects.

36 MATERIALS SCIENCE↗

Infinite neural network quantum states: entanglement and training dynamics

We study infinite limits of neural network quantum states (∞-NNQS), which exhibit representation power through ensemble statistics, and also tractable gradient descent dynamics. Ensemble averages of entanglement entropies are expressed in terms of neural network correlators, and architectures that exhibit volume-law entanglement are presented. The analytic calculations of entanglement entropy bound are tractable because the ensemble statistics are simplified in the Gaussian process limit. A general framework is developed for studying the gradient descent dynamics of neural network quantum states (NNQS), using a quantum state neural tangent kernel (QS-NTK). For ∞-NNQS the training dynamics is simplified, since the QS-NTK becomes deterministic and constant. An analytic solution is derived for quantum state supervised learning, which allows an ∞-NNQS to recover any target wavefunction. Numerical experiments on finite and infinite NNQS in the transverse field Ising model and Fermi Hubbard model demonstrate excellent agreement with theory. ∞-NNQS opens up new opportunities for studying entanglement and training dynamics in other physics applications, such as in finding ground states.

quantum state supervised learning↗

High-throughput search for magnetic topological materials using spin-orbit spillage, machine learning, and experiments

Magnetic topological insulators and semi-metals have a variety of properties that make them attractive for applications including spintronics and quantum computation. Here, we use systematic high-throughput density functional theory calculations to identify magnetic topological materials from the ≈ 40000 three-dimensional materials in the JARVIS-DFT database. First, we screen materials with net magnetic moment > 0.5 μB and spin-orbit spillage > 0.25, resulting in 25 insulating and 564 metallic candidates. The spillage acts as a signature of spin-orbit induced band-inversion. Then, we carry out calculations of Wannier charge centers, Chern numbers, anomalous Hall conductivities, surface bandstructures, and Fermi-surfaces to determine interesting topological characteristics of the screened compounds. We also train machine learning models for predicting the spillage, bandgaps, and magnetic moments of new compounds, to further accelerate the screening process. We experimentally synthesize and characterize a few candidate materials to support our theoretical predictions.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

In-Silico Analysis of High Refractive Index Materials Through Principles of Materials Design

The intent of the paper is to use specific principles of Materials Design that were developed and applied in the electronics industry for enabling understanding and design of improved high refractive index materials. Further, by combining first-principle based ab-initio, semiempirical interatomic potential methods, and machine learning approaches in conjunction with experimental data, we identified specific determinants of high refractive index materials, which can be critically applied for informing materials design and accelerating discovery. Specifically, it was demonstrated that chalcogenides and perovskites as bulk materials can exhibit higher refractive indices with appropriate engineering of specific aspects of the materials.

36 MATERIALS SCIENCE↗

Generative deep-learning reveals collective variables of Fermionic systems

Complex processes of fermionic systems ranging from protein folding to nuclear fission often follow a low-dimensional reaction path parametrized in terms of a few collective variables. In nuclear theory, variables related to the shape of the nuclear density in a mean-field picture are key to describing the large amplitude collective motion of the neutrons and protons. Exploring the adiabatic energy landscape spanned by these degrees of freedom reveals the possible reaction channels while simulating the dynamics in this reduced space yields their respective probabilities. Unfortunately, this theoretical framework breaks down whenever the systems encounters a quantum phase transition with respect to the collective variables. Here, in this study, we introduce a novel generative deep-learning algorithm designed to build reaction paths that ensure that the many-fermion wave function stays differentiable with respect to the collective variables. This approach is applicable to any fermionic system described by a coherent state. We use the case of potential energy curves in the 16 O nucleus within the Hartree-Fock theory to illustrate its main features.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Nature of molybdenum carbide surfaces for catalytic hydrogen dissociation using machine-learned potentials: an ensemble-averaged perspective

Molybdenum carbides with an electronic structure similar to noble metals have gained attention as a promising low-cost catalyst for biomass valorization and the hydrogen evolution reaction. However, our fundamental understanding of the catalyst surface and how different phases of these catalysts behave at varying reaction conditions is limited to ground state density functional theory calculations as ab initio molecular dynamics (AIMD) is computationally prohibitive at relevant length and time scales. Here, in this work, we train a multi-atomic cluster expansion (MACE) machine-learned interatomic potentials (MLIP) to study hydrogen dissociation and dynamics over Mo, δ-MoC, α-Mo 2 C, and β-Mo 2 C surfaces at varying temperatures and hydrogen partial pressures. Our simulations identify unique and different molecular and atomic hydrogen adsorption sites on different surfaces that do not depend on the temperature. At low hydrogen pressures, the surface coverage is monolayer, which transitions to two-layer adsorption at higher pressures. We find that atomic hydrogen diffusion and recombinations are preferred over molybdenum atom hollow sites, while the diffusion over carbon-terminated facets was negligible, signifying particularly strong C–H interactions. In contrast, molecular hydrogen adsorption occurs mostly atop Mo or the bridging sites. At a comparable hydrogen loading, β-Mo 2 C (001) is the most active surface for hydrogen dissociation reaction. This work provides insights into the dynamic nature of the hydrogen dissociation chemistry and the diversity of hydrogen adsorption sites on molybdenum carbides.

08 HYDROGEN↗

Gauge/Gravity Duality

Gauge theories, which describe the particle interactions, are well understood, while quantum gravity leads to many puzzles. Remarkably, in recent years we have learned that these are actually dual, the same system written in different variables. On the one hand, this provides our most precise description of quantum gravity, resolves some long-standing paradoxes, and points to new principles. On the other, it gives a new perspective on strong interactions, with surprising connections to other areas of physics. I describe these ideas, and discuss current and future directions.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Optimization Methods for Large-Scale Machine Learning

Here this paper provides a review and commentary on the past, present, and future of numerical optimization algorithms in the context of machine learning applications. Through case studies on text classification and the training of deep neural networks, we discuss how optimization problems arise in machine learning and what makes them challenging. A major theme of our study is that large-scale machine learning represents a distinctive setting in which the stochastic gradient (SG) method has traditionally played a central role while conventional gradient-based nonlinear optimization techniques typically falter. Based on this viewpoint, we present a comprehensive theory of a straightforward, yet versatile SG algorithm, discuss its practical behavior, and highlight opportunities for designing algorithms with improved performance. This leads to a discussion about the next generation of optimization methods for large-scale machine learning, including an investigation of two main streams of research on techniques that diminish noise in the stochastic directions and methods that make use of second-order derivative approximations.

97 MATHEMATICS AND COMPUTING↗

Elucidating the temperature and density dependence of silver chloride hydration numbers in high-temperature water vapor: A first-principles molecular simulation study

Hydration numbers of metal complexes in low-density aqueous solutions are required for developing geochemical models for ore-forming metals and for designing supercritical desalination processes. In this study, we investigate the temperature and density dependence of the hydration numbers of silver chloride at 623 K, 673 K, and 713 K and densities of 10–100 kg/m 3 . Experimental estimates of the hydration number in the literature for AgCl at these conditions are inconclusive and possibly contradictory as to the temperature and density dependence. First-principles molecular simulation presents an attractive alternative to experimental measurements. Specifically, recent work shows that machine-learning-accelerated nested Monte Carlo simulations provide reliable estimates for the hydration numbers of CuCl at 623 K from 10 to 100 kg/m 3 . Using the same technique, we find a monotonic temperature dependence, with the hydration number decreasing slightly with increasing temperature. In addition, the simulation-predicted hydration numbers steadily increase with increasing density. These temperature and density trends are in agreement with certain experimental data sets. Therefore, this work demonstrates how first-principles Monte Carlo simulations assist in resolving discrepancies between experimental data sets. Our simulation results also correctly predict that the hydration number, and thus also the solubility, of AgCl is lower than that of CuCl under the same conditions. Furthermore, the bond length and angle formed between the water complex and AgCl differ from those for CuCl, consistent with the lower solubility of AgCl.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Example on how to (intelligently) augment the nuclear-data pipeline with machine learning [Slides]

The presentation discusses how machine learning has helped the Los Alamos National Laboratory (LANL) nuclear-data pipeline. It also discusses the strengths of machine learning as it finds trends in large amounts of data where human brains are overwhelmed and that this information may be crucial to improve our nuclear data. It does stress, however, that machine learning is no "silver bullet" and that it is critical to feed it expert knowledge and use physics intuition to interpret the results. The presentation discusses the need to develop infrastructure and tools to provide data in an easily readable and unambiguously interpretable format (e.g., EXFOR format), to develop experimental data and theory to solve physics questions, and that statisticians and nuclear-data experts must be brought together to correctly interpret the results. The presentation concludes by stating that machine learning is a great tool and that LANL needs to use the algorithms along with developing physics data, tools and infrastructure.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

When Do Extended Physics-Informed Neural Networks (XPINNs) Improve Generalization?

Physics-informed neural networks (PINNs) have become a popular choice for solving high-dimensional partial differential equations (PDEs) due to their excellent approximation power and generalization ability. Recently, extended PINNs (XPINNs) based on domain decomposition methods have attracted considerable attention due to their effectiveness in modeling multiscale and multiphysics problems and their parallelization. However, theoretical understanding of their convergence and generalization properties remains unexplored. In this study, we take an initial step towards understanding how and when XPINNs outperform PINNs. Specifically, for general multilayer PINNs and XPINNs, we first provide a prior generalization bound via the complexity of the target functions in the PDE problem and a posterior generalization bound via the posterior matrix norms of the networks after optimization. Moreover, based on our bounds, we analyze the conditions under which XPINNs improve generalization. Concretely, our theory shows that the key building block of XPINN, namely, the domain decomposition, introduces a tradeoff for generalization. On the one hand, XPINNs decompose the complex PDE solution into several simple parts, which decreases the complexity needed to learn each part and boosts generalization. On the other hand, decomposition leads to less training data being available in each subdomain, and hence such a model is typically prone to overfitting and may become less generalizable. Empirically, we choose five PDEs to show when XPINNs perform better than, similar to, or worse than PINNs, hence demonstrating and justifying our new theory.

97 MATHEMATICS AND COMPUTING↗

Theoretical Prediction of Thermal Expansion Anisotropy for Y 2 Si 2 O 7 Environmental Barrier Coatings Using a Deep Neural Network Potential and Comparison to Experiment

Environmental barrier coatings (EBCs) are an enabling technology for silicon carbide (SiC)-based ceramic matrix composites (CMCs) in extreme environments such as gas turbine engines. However, the development of new coating systems is hindered by the large design space and difficulty in predicting the properties for these materials. Density Functional Theory (DFT) has successfully been used to model and predict some thermodynamic and thermo-mechanical properties of high-temperature ceramics for EBCs, although these calculations are challenging due to their high computational costs. In this work, we use machine learning to train a deep neural network potential (DNP) for Y 2 Si 2 O 7 , which is then applied to calculate the thermodynamic and thermo-mechanical properties at near-DFT accuracy much faster and using less computational resources than DFT. We use this DNP to predict the phonon-based thermodynamic properties of Y 2 Si 2 O 7 with good agreement to DFT and experiments. We also utilize the DNP to calculate the anisotropic, lattice direction-dependent coefficients of thermal expansion (CTEs) for Y 2 Si 2 O 7 . Molecular dynamics trajectories using the DNP correctly demonstrate the accurate prediction of the anisotropy of the CTE in good agreement with the diffraction experiments. In the future, this DNP could be applied to accelerate additional property calculations for Y 2 Si 2 O 7 compared to DFT or experiments.

36 MATERIALS SCIENCE↗

Naturalistic Decision Making in Power Grid Operations: Implications for Dispatcher Training and Usability Testing

The focus of the present study is on improved training approaches to accelerate learning and improved methods for analyzing effectiveness of tools within a high-fidelity power grid simulated environment. A theory-based model has been developed to document and understand the mental processes that an expert power system operator uses when making critical decisions. The theoretical foundation for the method is based on the concepts of situation awareness, the methods of cognitive task analysis, and the naturalistic decision making (NDM) approach of Recognition Primed Decision Making. The method has been systematically explored and refined as part of a capability demonstration of a high-fidelity real-time power system simulator under normal and emergency conditions. To examine NDM processes, we analyzed transcripts of operator-to-operator conversations during the simulated scenario to reveal and assess NDM-based performance criteria. The results of the analysis indicate that the proposed framework can be used constructively to map or assess the Situation Awareness Level of the operators at each point in the scenario. We can also identify the mental models and mental simulations that the operators employ at different points in the scenario. This report documents the method, describes elements of the model, and provides appendices that document the simulation scenario and the associated mental models used by operators in the scenario.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Ab Initio Modeling of Aqueous Methanol Mixtures at DFT-SCAN Level Using Machine Learning Interatomic Potentials

Abstract Methanol–water mixtures find use in many applications, particularly catalytic energy conversion processes. Their importance has motivated numerous computational studies, most of which employed molecular dynamics based on classical force fields. These enable simulations of large systems on long time scales but do not reliably describe reactive dynamics involving bond breaking and bond formation. In contrast, ab initio molecular dynamics (AIMD) based on density functional theory (DFT) is generally more reliable for such applications but has a high computational cost, which discourages systematic studies of alcohol-water mixtures. To remedy this, we trained a machine learning interatomic potential capable of probing the properties of aqueous methanol mixtures at the DFT level using the SCAN functional. Our results show that SCAN qualitatively reproduces multiple key experimental features arising from the amphiphilic nature of methanol, including density, diffusion coefficients, X-ray structure factors, and Kirkwood–Buff integrals. We also find that structural correlations between water molecules are somewhat overestimated, leading to a stronger preferential association than that predicted by experiments. However, increasing the temperature by 30 K mitigates this effect and also recovers the correct mobilities of both methanol and water. These results indicate that SCAN provides an accurate description of methanol–water mixtures, making it a reliable choice for investigating the reactive dynamics in such systems.

Park, Sanghyun J. [Princeton University , , , ,]↗

Multi-fidelity machine-learning with uncertainty quantification and Bayesian optimization for materials design: Application to ternary random alloys

We present a scale-bridging approach based on a multi-fidelity (MF) machine-learning (ML) framework leveraging Gaussian processes (GP) to fuse atomistic computational model predictions across multiple levels of fidelity. Through the posterior variance of the MFGP, our framework naturally enables uncertainty quantification, providing estimates of confidence in the predictions. We used density functional theory as high-fidelity prediction, while a ML interatomic potential is used as low-fidelity prediction. Practical materials’ design efficiency is demonstrated by reproducing the ternary composition dependence of a quantity of interest (bulk modulus) across the full aluminum–niobium–titanium ternary random alloy composition space. The MFGP is then coupled to a Bayesian optimization procedure, and the computational efficiency of this approach is demonstrated by performing an on-the-fly search for the global optimum of bulk modulus in the ternary composition space. The framework presented in this manuscript is the first application of MFGP to atomistic materials simulations fusing predictions between density functional theory and classical interatomic potential calculations.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Learning to predict the cosmological structure formation

Matter evolved under the influence of gravity from minuscule density fluctuations. Nonperturbative structure formed hierarchically over all scales and developed non-Gaussian features in the Universe, known as the cosmic web. To fully understand the structure formation of the Universe is one of the holy grails of modern astrophysics. Astrophysicists survey large volumes of the Universe and use a large ensemble of computer simulations to compare with the observed data to extract the full information of our own Universe. However, to evolve billions of particles over billions of years, even with the simplest physics, is a daunting task. We build a deep neural network, the Deep Density Displacement Model ( D 3 M ), which learns from a set of prerun numerical simulations, to predict the nonlinear large-scale structure of the Universe with the Zel’dovich Approximation (ZA), an analytical approximation based on perturbation theory, as the input. Our extensive analysis demonstrates that D 3 M outperforms the second-order perturbation theory (2LPT), the commonly used fast-approximate simulation method, in predicting cosmic structure in the nonlinear regime. We also show that D 3 M is able to accurately extrapolate far beyond its training data and predict structure formation for significantly different cosmological parameters. Our study proves that deep learning is a practical and accurate alternative to approximate 3D simulations of the gravitational structure formation of the Universe.

79 ASTRONOMY AND ASTROPHYSICS↗

A Real-Time Big Data Control-Theoretical Framework for Cyber-Physical-Human Systems [Book Chapter]

It is known that cyber-physical-human systems naturally arise from interdependent infrastructure systems and smart connected communities. Such applications require ubiquitous information sensing and processing, intelligent machine-to-machine communication for a seamless coordination, as well as intelligent interactions between humans and machines. This chapter presents a control-theoretical framework to model heterogeneous physical dynamic systems, information and communication, as well as cooperative controls and/or distributed optimization of such interconnected systems. It is shown that efficient analytical and computational algorithms can be modularly designed and hierarchically implemented to operate and optimize cyber-physical-human systems, first to quantify individually the input–output relationship of nonlinear dynamic behaviors of every physical subsystem, then to coordinate locally both cyber-physical interactions of neighboring agents as well as physical-human interactions, and finally to dynamically model and optimize the overall networked system. The hierarchical structure makes the overall optimization and control problem scalable and solvable. Moreover, the three levels integrate individual designs and optimization, distributed cooperative optimization, and decision-making through real-time, data-driven, model-based learning and control. Specifically, one of the contributions of the chapter is to demonstrate how the combination of dissipativity theory and cooperative control serves as a natural framework and promising tools to analyze, optimize, and control such large-scale system. Application to digital grid is investigated as an illustrative example.

97 MATHEMATICS AND COMPUTING↗