Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Learning Theory”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 721 records · Page 40

Neural network potential from bispectrum components: A case study on crystalline silicon

In this article, we present a systematic study on developing machine learning force fields (MLFFs) for crystalline silicon. While the main-stream approach of fitting a MLFF is to use a small and localized training set from molecular dynamics simulations, it is unlikely to cover the global features of the potential energy surface. Additionally, to remedy this issue, we used randomly generated symmetrical crystal structures to train a more general Si-MLFF. Furthermore, we performed substantial benchmarks among different choices of material descriptors and regression techniques on two different sets of silicon data. Our results show that neural network potential fitting with bispectrum coefficients as descriptors is a feasible method for obtaining accurate and transferable MLFFs.

36 MATERIALS SCIENCE↗

Machine Learning Thermodynamics And Kinetics of Defects For Accelerated Materials Discovery

Atomistic defects play a pivotal role in functional and structural materials’ performance across a myriad of technology applications. Quantitative prediction of the thermodynamics and kinetics of defect formation and migration, respectively, typically requires accurate but expensive first-principles approaches, such as density functional theory (DFT). Their computational expense limits the throughput needed to perform high-throughput materials discovery/screening exercises or to perform materials modeling tasks relying on extensive sampling techniques. Therefore, in this Sandia National Laboratories Laboratory Directed Research and Development (LDRD) project (Project #229366), we developed a variety of machine learning techniques, trained on density functional theory calculations, to accelerate the discovery and modeling of materials in which vacancy and interstitial defects primarily dictate material performance. These include applications such as metal oxides for water-splitting or mixed ionic-electronic conduction, metal hydrides for hydrogen storage, and transition metal dichalcogenides for electronics, and the approaches developed herein can further be applied to many other domains that similarly depend on materials’ thermodynamic and kinetic defect properties for their desired functionality.

36 MATERIALS SCIENCE↗

Evidence of chaotic pattern in solar flux through a reproducible sequence of period-doubling-type bifurcations

Presented here is a preliminary study of the limits to solar flux intensity prediction, and of whether the general lack of predictability in the solar flux arises from the nonlinear chaotic nature of the Sun's physical activity. Statistical analysis of a chaotic signal can extract only its most gross features, and detailed physical models fail, since even the simplest equations of motion for a nonlinear system can exhibit chaotic behavior. A recent theory by Feigenbaum suggests that nonlinear systems that can be led into chaotic behavior through a sequence of period-doubling bifurcations will exhibit a universal behavior. As the control parameter is increased, the bifurcation points occur in such a way that a proper ratio of these will approach the universal Feigenbaum number. Experimental evidence supporting the applicability of the Feigenbaum scenario to solar flux data is sparse. However, given the hypothesis that the Sun's convection zones are similar to a Rayleigh-Bernard mechanism, we can learn a great deal from the remarkable agreement observed between the prediction by theory (period doubling - a universal route to chaos) and the amplitude decrease of the signal's regular subharmonics. The authors show that period-doubling-type bifurcation is a possible route to a chaotic pattern of solar flux that is distinguishable from the logarithm of its power spectral density. This conclusion is the first positive step toward a reformulation of solar flux by a nonlinear chaotic approach. The ultimate goal of this research is to be able to predict an estimate of the upper and lower bounds for solar flux within its predictable zones. Naturally, it is an important task to identify the time horizons beyond which predictability becomes incompatible with computability.

Ashrafi, S.↗

Evidence of chaotic pattern in solar flux through a reproducible sequence of period-doubling-type bifurcations

A preliminary study of the limits to solar flux intensity prediction, and of whether the general lack of predictability in the solar flux arises from the nonlinear chaotic nature of the Sun's physical activity is presented. Statistical analysis of a chaotic signal can extract only its most gross features, and detailed physical models fail, since even the simplest equations of motion for a nonlinear system can exhibit chaotic behavior. A recent theory by Feigenbaum suggests that nonlinear systems that can be led into chaotic behavior through a sequence of period-doubling bifurcations will exhibit a universal behavior. As the control parameter is increased, the bifurcation points occur in such a way that a proper ratio of these will approach the universal Feigenbaum number. Experimental evidence supporting the applicability of the Feigenbaum scenario to solar flux data is sparse. However, given the hypothesis that the Sun's convection zones are similar to a Rayleigh-Bernard mechanism, we can learn a great deal from the remarkable agreement observed between the prediction by theory (period doubling - a universal route to chaos) and the amplitude decrease of the signal's regular subharmonics. It is shown that period-doubling-type bifurcation is a possible route to a chaotic pattern of solar flux that is distinguishable from the logarithm of its power spectral density. This conclusion is the first positive step toward a reformulation of solar flux by a nonlinear chaotic approach. The ultimate goal of this research is to be able to predict an estimate of the upper and lower bounds for solar flux within its predictable zones. Naturally, it is an important task to identify the time horizons beyond which predictability becomes incompatible with computability.

Ashrafi, S.↗

Revisiting the ODE Method for Recursive Algorithms: Fast Convergence Using Quasi Stochastic Approximation

Several decades ago, Profs. Sean Meyn and Lei Guo were postdoctoral fellows at ANU, where they shared interest in recursive algorithms. It seems fitting to celebrate Lei Guo's 60th birthday with a review of the ODE Method and its recent evolution. The method has been regarded as a technique for algorithm analysis. It is argued that this viewpoint is backwards: The original stochastic approximation method was surely motivated by an ODE, and tools for analysis came much later (based on establishing robustness of Euler approximations). The paper presents a brief survey of recent research in machine learning that shows the power of algorithm design in continuous time, following by careful approximation to obtain a practical recursive algorithm. While these methods are usually presented in a stochastic setting, this is not a prerequisite. In fact, recent theory shows that rates of convergence can be dramatically accelerated by applying techniques inspired by quasi Monte-Carlo. Subject to conditions, the optimal rate of convergence can be obtained by applying the averaging technique of Polyak and Ruppert. The conditions are not universal, but theory suggests alternatives to achieve acceleration. The theory is illustrated with applications to gradient-free optimization, and policy gradient algorithms for reinforcement learning.

learning and adaptive systems in artificial intell↗

Towards AI-assisted neutrino flavor theory design

Particle physics theories, such as those which explain neutrino flavor mixing, arise from a vast landscape of model-building possibilities. A model’s construction typically relies on the intuition of theorists. It also requires considerable effort to identify appropriate symmetry groups, assign field representations, and extract predictions for comparison with experimental data. We develop Autonomous Model Builder (AMBer), a framework in which a reinforcement learning agent interacts with a streamlined physics software pipeline to search these spaces efficiently. AMBer selects symmetry groups, particle content, and group representation assignments to construct models while minimizing the number of free parameters introduced. We validate our approach in well-studied regions of theory space and extend the exploration to a previously unexamined symmetry group. While demonstrated in the context of neutrino flavor theories, this approach of reinforcement learning with physics software feedback may be extended to other theoretical model-building problems in the future.

Baretz, Jason Benjamin↗

Recent developments in the PySCF program package

PySCF is a Python-based general-purpose electronic structure platform that supports first-principles simulations of molecules and solids as well as accelerates the development of new methodology and complex computational workflows. Here, we explain the design and philosophy behind PySCF that enables it to meet these twin objectives. With several case studies, we show how users can easily implement their own methods using PySCF as a development environment. We then summarize the capabilities of PySCF for molecular and solid-state simulations. Finally, we describe the growing ecosystem of projects that use PySCF across the domains of quantum chemistry, materials science, machine learning, and quantum information science.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Surface Segregation Studies in Ternary Noble Metal Alloys: Comparing DFT and Machine Learning with Experimental Data

Abstract Surface segregation, whereby the surface composition of an alloy differs systematically from the bulk, has historically been hard to study, because it requires experimental and modeling methods that span alloy composition space. In this work, we study surface segregation in catalytically relevant noble and platinum‐group metal alloys with a focus on three ternary systems: AgAuCu, AuCuPd, and CuPdPt. We develop a data set of 2478 fcc slabs with those compositions including all three low‐index crystallographic orientations relaxed with Density Functional Theory using the PBEsol functional with D3 dispersion corrections. We fine‐tune a machine learning model on this data and use the model in a series of 1800 Monte Carlo simulations spanning ternary composition space for each surface orientation and ternary chemical system. The results of these simulations are validated against prior experimental surface segregation data collected using composition spread alloy films for AgAuCu and AuCuPd. Our findings reveal that simulations conducted using the (110) orientation most closely match experimentally observed surface segregation trends, and while predicted trends qualitatively match observation, biases in the PBEsol functional limit numeric accuracy. This study advances understanding of surface segregation and the utility of computational studies and highlights the need for further improvements in simulation accuracy.

Chemistry↗

Reformulating Non-Monotonic Theories for Inference and Updating

We aim to help build programs that do large-scale, expressive non-monotonic reasoning (NMR): especially, 'learning agents' that store, and revise, a body of conclusions while continually acquiring new, possibly defeasible, premise beliefs. Currently available procedures for forward inference and belief revision are exhaustive, and thus impractical: they compute the entire non-monotonic theory, then re-compute from scratch upon updating with new axioms. These methods are thus badly intractable. In most theories of interest, even backward reasoning is combinatoric (at least NP-hard). Here, we give theoretical results for prioritized circumscription that show how to reformulate default theories so as to make forward inference be selective, as well as concurrent; and to restrict belief revision to a part of the theory. We elaborate a detailed divide-and-conquer strategy. We develop concepts of structure in NM theories, by showing how to reformulate them in a particular fashion: to be conjunctively decomposed into a collection of smaller 'part' theories. We identify two well-behaved special cases that are easily recognized in terms of syntactic properties: disjoint appearances of predicates, and disjoint appearances of individuals (terms). As part of this, we also definitionally reformulate the global axioms, one by one, in addition to applying decomposition. We identify a broad class of prioritized default theories, generalizing default inheritance, for which our results especially bear fruit. For this asocially monadic class, decomposition permits reasoning to be localized to individuals (ground terms), and reduced to propositional. Our reformulation methods are implementable in polynomial time, and apply to several other NM formalisms beyond circumscription.

Grosof, Benjamin N.↗

Machine-learned impurity level prediction for semiconductors: the example of Cd-based chalcogenides

The ability to predict the likelihood of impurity incorporation and their electronic energy levels in semiconductors is crucial for controlling its conductivity, and thus the semiconductor's performance in solar cells, photodiodes, and optoelectronics. The difficulty and expense of experimental and computational determination of impurity levels makes a data-driven machine learning approach appropriate. In this work, we show that a density functional theory-generated dataset of impurities in Cd-based chalcogenides CdTe, CdSe, and CdS can lead to accurate and generalizable predictive models of defect properties. By converting any semiconductor + impurity system into a set of numerical descriptors, regression models are developed for the impurity formation enthalpy and charge transition levels. These regression models can subsequently predict impurity properties in mixed anion CdX compounds (where X is a combination of Te, Se and S) fairly accurately, proving that although trained only on the end points, they are applicable to intermediate compositions. We make machine-learned predictions of the Fermi-level-dependent formation energies of hundreds of possible impurities in 5 chalcogenide compounds, and we suggest a list of impurities which can shift the equilibrium Fermi level in the semiconductor as determined by the dominant intrinsic defects. Machine learning predictions for the dominating impurities compare well with DFT predictions, revealing the power of machine-learned models in the quick screening of impurities likely to affect the optoelectronic behavior of semiconductors.

36 MATERIALS SCIENCE↗

Scientific machine learning for closure models in multiscale problems: A review

Here, closure problems are omnipresent when simulating multiscale systems, where some quantities and processes cannot be fully prescribed despite their effects on the simulation's accuracy. Recently, scientific machine learning approaches have been proposed as a way to tackle the closure problem, combining traditional (physics-based) modeling with data-driven (machine-learned) techniques, typically through enriching differential equations with neural networks. This paper reviews the different reduced model forms, distinguished by the degree to which they include known physics, and the different objectives of a priori and a posteriori learning. The importance of adhering to physical laws (such as symmetries and conservation laws) in choosing the reduced model form and choosing the learning method is discussed. The effect of spatial and temporal discretization and recent trends toward discretization-invariant models are reviewed. In addition, we make the connections between closure problems and several other research disciplines: inverse problems, Mori-Zwanzig theory, and multi-fidelity methods. In conclusion, much progress has been made with scientific machine learning approaches for solving closure problems, but many challenges remain. In particular, the generalizability and interpretability of learned models is a major issue that needs to be addressed further.

97 MATHEMATICS AND COMPUTING↗

Evaluation of global terrestrial evapotranspiration using state-of-the-art approaches in remote sensing, machine learning and land surface modeling

Abstract. Evapotranspiration (ET) is critical in linking global water, carbon and energy cycles. However, direct measurement of global terrestrial ET is not feasible. Here, we first reviewed the basic theory and state-of-the-art approaches for estimating global terrestrial ET, including remote-sensing-based physical models, machine-learning algorithms and land surface models (LSMs). We then utilized 4 remote-sensing-based physical models, 2 machine-learning algorithms and 14 LSMs to analyze the spatial and temporal variations in global terrestrial ET. The results showed that the ensemble means of annual global terrestrial ET estimated by these three categories of approaches agreed well, with values ranging from 589.6 mm yr-1 (6.56×104 km3 yr-1) to 617.1 mm yr-1 (6.87×10 4 km3 yr -1 ). For the period from 1982 to 2011, both the ensembles of remote-sensing-based physical models and machine-learning algorithms suggested increasing trends in global terrestrial ET (0.62 mm yr -2 with a significance level of p<0.05 and 0.38 mm yr -2 with a significance level of p<0.05, respectively). In contrast, the ensemble mean of the LSMs showed no statistically significant change (0.23 mm yr -2 , p>0.05), although many of the individual LSMs reproduced an increasing trend. Nevertheless, all 20 models used in this study showed that anthropogenic Earth greening had a positive role in increasing terrestrial ET. The concurrent small interannual variability, i.e., relative stability, found in all estimates of global terrestrial ET, suggests that a potential planetary boundary exists in regulating global terrestrial ET, with the value of this boundary being around 600 mm yr -1 . Uncertainties among approaches were identified in specific regions, particularly in the Amazon Basin and arid/semiarid regions. Improvements in parameterizing water stress and canopy dynamics, the utilization of new available satellite retrievals and deep-learning methods, and model–data fusion will advance our predictive understanding of global terrestrial ET.

58 GEOSCIENCES↗

Machine learning guided discovery of superconducting calcium borocarbides

Pursuit of superconductivity in light-element systems at ambient pressure is of great experimental and theoretical interest. In this work, we combine a machine learning (ML) method with first-principles calculations to efficiently search for the energetically favorable ternary Ca-B-C compounds. Three layered borocarbides (stable CaBC 5 and metastable Ca 2 BC 11 and CaB 3 C 3 ) are predicted to be phonon-mediated superconductors at ambient pressure. The stable CaBC 5 and the low-energy metastable Ca 2 BC 11 (with formation energy only 9.5 meV/atom above the convex hull) have a superconducting T c of 5.2 and 8.9 K, respectively. While the hexagonal CaB 3 C 3 possesses a T c of 26.1 K, it is metastable with formation energy of 153 meV/atom above the convex hull. The ML-guided approach opens up a way for greatly accelerating the discovery of new high-T c superconductors.

36 MATERIALS SCIENCE↗

The role of information structures in game-theoretic multi-agent learning

Multi-agent learning (MAL) studies how agents learn to behave optimally and adaptively from their experience when interacting with other agents in dynamic environments. The outcome of a MAL process is jointly determined by all agents’ decision-making. Hence, each agent needs to think strategically about others’ sequential moves, when planning future actions. The strategic interactions among agents makes MAL go beyond the direct extension of single-agent learning to multiple agents. With the strategic thinking, each agent aims to build a subjective model of others decision-making using its observations. Such modeling is directly influenced by agents’ perception during the learning process, which is called the information structure of the agent’s learning. As it determines the input to MAL processes, information structures play a significant role in the learning mechanisms of the agents. This review creates a taxonomy of MAL and establishes a unified and systematic way to understand MAL from the perspective of information structures. Here we define three fundamental components of MAL: the information structure (i.e., what the agent can observe), the belief generation (i.e., how the agent forms a belief about others based on the observations), as well as the policy generation (i.e., how the agent generates its policy based on its belief). In addition, this taxonomy enables the classification of a wide range of state-of-the-art algorithms into four categories based on the belief-generation mechanisms of the opponents, including stationary, conjectured, calibrated, and sophisticated opponents. We introduce Value of Information (VoI) as a metric to quantify the impact of different information structures on MAL. Finally, we discuss the strengths and limitations of algorithms from different categories and point to promising avenues of future research.

97 MATHEMATICS AND COMPUTING↗

Gaussian approximation potentials for accurate thermal properties of two-dimensional materials

Two-dimensional materials (2DMs) continue to attract a lot of attention, particularly for their extreme flexibility and superior thermal properties. Molecular dynamics simulations are among the most powerful methods for computing these properties, but their reliability depends on the accuracy of interatomic interactions. While first principles approaches provide the most accurate description of interatomic forces, they are computationally expensive. In contrast, classical force fields are computationally efficient, but have limited accuracy in interatomic force description. Machine learning interatomic potentials, such as Gaussian Approximation Potentials, trained on density functional theory (DFT) calculations offer a compromise by providing both accurate estimation and computational efficiency. Here, in this work, we present a systematic procedure to develop Gaussian approximation potentials for selected 2DMs, graphene, buckled silicene, and h-XN (X = B, Al, and Ga, as binary compounds) structures. We validate our approach through calculations that require various levels of accuracy in interatomic interactions. The calculated phonon dispersion curves and lattice thermal conductivity, obtained through harmonic and anharmonic force constants (including fourth order) are in excellent agreement with DFT results. HIPHIVE calculations, in which the generated GAP potentials were used to compute higher-order force constants instead of DFT, demonstrated the first-principles level accuracy of the potentials for interatomic force description. Molecular dynamics simulations based on phonon density of states calculations, which agree closely with DFT-based calculations, also show the success of the generated potentials in high-temperature simulations.

2D Materials↗

Directly Imaged Giant Planets: What Do We Hope to Learn?

As we move into an era when GPI and SPHERE are (hopefully) discovering and characterizing new young giant planets, it is worthwhile to step back and review our science goals for young giant planets. Of course for individual planets we ideally would hope to measure mass, radius, atmospheric composition, temperature, and cloud properties, but how do these characteristics fit into our broader understanding of planetary system origin and evolution theories? In my presentation I will review both the specifics of what we hope to learn from newly discovered young worlds as well as how these characteristics inform our broader understanding of giant planets and planetary systems. Finally I will consider the limitations realistic datasets will place on our ability to understand newly discovered planets, illustrating with data from any new such worlds that are available by the conference date.

temperature↗

Best practices in catalysis: A perspective

Catalysis, from its roots in petrochemical refining and conversion, has emerged as a transdisciplinary field that now encompasses synthesis of materials and molecules that enable applications in energy conversion and storage, environmental remediation, medicine, plastics, and fertilizer production, among numerous others. A handful of disciplines can claim relevance and success over such an extended period of time and continue to claim a preeminent role in defining the state-of-the-art in science and technology. Syncretic and rapid advancements in formulation and spectroscopic characterization of materials and molecules useful as catalysts, high-level density functional and molecular orbital theory calculations, and a strong foundation in concepts of physical chemistry, thermodynamics, and chemical kinetics offer new and abundant opportunities at the present day for addressing the grand challenge of controlling chemical transformations using catalysis. Here, we examine what we have learned of concepts that underpin heterogeneous catalysis but more importantly, how we learned to archive our knowledge in context of a set of best practices and standards that have emerged in the course of our learnings—ones we seek to highlight herein. Our perspective emphasizes concepts in synthesis, characterization, kinetics, and theory, because these four elements combined enable description of molecular acts that happen on surfaces and how fast they occur. In revisiting best practices in heterogeneous catalysis in these sub-fields and in authorship and peer review we aspire to augment clarity, reproducibility, and rigor in the science and practice of catalysis.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Machine learning prediction of the mechanical properties of refractory multicomponent alloys based on a dataset of phase and first principles simulation

In this work, a dataset including structural and mechanical properties of refractory multicomponent alloys was developed by fusing computations of phase diagram (CALPHAD) and density functional theory (DFT). The refractory multicomponent alloys, also named refractory complex concentrated alloys (CCAs) which contain 2–5 types of refractory elements were constructed based on Special Quasi-random Structure (SQS). The phase of alloys was predicted using CALPHAD and the mechanical property of alloys with stable and single body-centered cubic (BCC) at high temperature (over 1,500°C) was investigated using DFT-based simulation. As a result, a dataset with 393 refractory alloys and 12 features, including volume, melting temperature, density, energy, elastic constants, mechanical moduli, and hardness, were produced. To test the capability of the dataset on supporting machine learning (ML) study to investigate the property of CCAs, CALPHAD, and DFT calculations were compared with principal components analysis (PCA) technique and rule of mixture (ROM), respectively. It is demonstrated that the CALPHAD and DFT results are more in line with experimental observations for the alloy phase, structural and mechanical properties. Furthermore, the data were utilized to train a verity of ML models to predict the performance of certain CCAs with advanced mechanical properties, highlighting the usefulness of the dataset for ML technique on CCA property prediction.

36 MATERIALS SCIENCE↗