Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Learning Theory”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,189 records · Page 66

Code for the manuscript titled "Blackout Diffusion: Generative Diffusion Models in Discrete-State Spaces"

We would like to disclose two scripts, written in Jupyter notebook, in which we implement the "Blackout Diffusion Process" described in the manuscript "Blackout Diffusion: Generative Diffusion Models in Discrete-State Spaces" (LA-UR-23-20509), to be submitted to the International Conference of Machine Learning (ICML). The abstract of the manuscript is append below. == Typical generative diffusion models rely on a Gaussian diffusion process for training the backward transformations, which can then be used generate samples from Gaussian noise. However, real world data often takes place in discrete-state spaces, which occur in many scientific applications. Here, we develop a theoretical formulation for arbitrary discrete-state Markov processes in the forward diffusion process. We relate the theory to the existing continuous-state Gaussian diffusion and identify the corresponding reverse-time stochastic process and score function in the continuous-time setting, and the reverse-time mapping in the discrete-time setting. As an example of this framework, we introduce "Blackout Diffusion", which learns to produce samples from an empty image instead of from noise. Numerical experiments on the CIFAR-10 dataset confirm the feasibility of generative diffusion modeling in a discrete space. Generalizing from specific (Gaussian) forward processes to a more general framework also sheds light on how to interpret generative diffusion models and their mathematical structure, which we comment on.

Lin, Yen Ting↗

Real-time dynamics of the Schwinger model as an open quantum system with Neural Density Operators

Ab-initio simulations of multiple heavy quarks propagating in a Quark-Gluon Plasma are computationally difficult to perform due to the large dimension of the space of density matrices. This work develops machine learning algorithms to overcome this difficulty by approximating exact quantum states with neural network parametrisations, specifically Neural Density Operators. As a proof of principle demonstration in a QCD-like theory, the approach is applied to solve the Lindblad master equation in the 1 + 1d lattice Schwinger Model as an open quantum system. Neural Density Operators enable the study of in-medium dynamics on large lattice volumes, where multiple-string interactions and their effects on string-breaking and recombination phenomena can be studied. Thermal properties of the system at equilibrium can also be probed with these methods by variationally constructing the steady state of the Lindblad master equation. Scaling of this approach with system size is studied, and numerical demonstrations on up to 32 spatial lattice sites and with up to 3 interacting strings are performed.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Quantifying the Relevance of Long-Range Forces for Crystal Nucleation in Water

Understanding nucleation from aqueous solutions is of fundamental importance in a multitude of fields, ranging from materials science to biophysics. The complex solvent-mediated interactions in aqueous solutions hamper the development of a simple physical picture, elucidating the roles of different interactions in nucleation processes. In this study, we make use of three complementary techniques to disentangle the role played by short- and long-range interactions in solvent-mediated nucleation. Specifically, the first approach we utilize is the local molecular field (LMF) theory to renormalize long-range Coulomb electrostatics. Second, we use well-tempered metadynamics to speed up rare events governed by short-range interactions. Third, the deep learning-based State Predictive Information Bottleneck approach is employed in analyzing the reaction coordinate of the nucleation processes obtained from the LMF treatment coupled with well-tempered metadynamics. We find that the two-step nucleation mechanism can largely be captured by the short-range interactions, while the long-range interactions further contribute to the stability of the primary crystal state under ambient conditions. Furthermore, by analyzing the reaction coordinate obtained from the combined LMF-metadynamics treatment, we discern the fluctuations on different time scales, highlighting the need for long-range interactions when accounting for metastability.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Constructing High-Dimensional Neural Network Potential Energy Surfaces for Gas–Surface Scattering and Reactions

While the ab initio molecular dynamics (AIMD) approach to gas–surface interaction has been instrumental in exploring important issues such as energy transfer and reactivity, it is only amenable to short-time events and a limited number of trajectories because of the on-the-fly nature of the density functional theory (DFT) calculations. In this work, we report a high-dimensional global reactive potential energy surface (PES) constructed with high fidelity from judiciously placed DFT points, using a machine learning method; and it is orders-of-magnitude more efficient than AIMD in dynamical calculations and can be employed in various simulations without performing additional electronic structure calculations. Importantly, the surface atoms are included in such a PES, which provides a unique platform for studying energy transfer and scattering/reaction of the impinging molecule on the solid surface on an equal footing.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Quantification of flexoelectricity in PbTiO 3 /SrTiO 3 superlattice polar vortices using machine learning and phase-field modeling

Flexoelectricity refers to electric polarization generated by heterogeneous mechanical strains, namely strain gradients, in materials of arbitrary crystal symmetries. Despite more than 50 years of work on this effect, an accurate identification of its coupling strength remains an experimental challenge for most materials, which impedes its wide recognition. Here, we show the presence of flexoelectricity in the recently discovered polar vortices in PbTiO 3 /SrTiO 3 superlattices based on a combination of machine-learning analysis of the atomic-scale electron microscopy imaging data and phenomenological phase-field modeling. By scrutinizing the influence of flexocoupling on the global vortex structure, we match theory and experiment using computer vision methodologies to determine the flexoelectric coefficients for PbTiO 3 and SrTiO 3 . Here, our findings highlight the inherent, nontrivial role of flexoelectricity in the generation of emergent complex polarization morphologies and demonstrate a viable approach to delineating this effect, conducive to the deeper exploration of both topics.

36 MATERIALS SCIENCE↗

Group-equivariant autoencoder for identifying spontaneously broken symmetries

We introduce the group-equivariant autoencoder (GE autoencoder), a deep neural network (DNN) method that locates phase boundaries by determining which symmetries of the Hamiltonian have spontaneously broken at each temperature. We use group theory to deduce which symmetries of the system remain intact in all phases, and then use this information to constrain the parameters of the GE autoencoder such that the encoder learns an order parameter invariant to these “never-broken” symmetries. This procedure produces a dramatic reduction in the number of free parameters such that the GE-autoencoder size is independent of the system size. We include symmetry regularization terms in the loss function of the GE autoencoder so that the learned order parameter is also equivariant to the remaining symmetries of the system. By examining the group representation by which the learned order parameter transforms, we are then able to extract information about the associated spontaneous symmetry breaking. We test the GE autoencoder on the 2D classical ferromagnetic and antiferromagnetic Ising models, finding that the GE autoencoder (1) accurately determines which symmetries have spontaneously broken at each temperature; (2) estimates the critical temperature in the thermodynamic limit with greater accuracy, robustness, and time efficiency than a symmetry-agnostic baseline autoencoder; and (3) detects the presence of an external symmetry-breaking magnetic field with greater sensitivity than the baseline method. Lastly, we describe various key implementation details, including a quadratic-programming-based method for extracting the critical temperature estimate from trained autoencoders and calculations of the DNN initialization and learning rate settings required for fair model comparisons.

42 ENGINEERING↗

Crystal Graph Convolutional Neural Networks for an Accurate and Interpretable Prediction of Material Properties

The use of machine learning methods for accelerating the design of crystalline materials usually requires manually constructed feature vectors or complex transformation of atom coordinates to input the crystal structure, which either constrains the model to certain crystal types or makes it difficult to provide chemical insights. Here, we develop a crystal graph convolutional neural networks framework to directly learn material properties from the connection of atoms in the crystal, providing a universal and interpretable representation of crystalline materials. Our method provides a highly accurate prediction of density functional theory calculated properties for eight different properties of crystals with various structure types and compositions after being trained with 10 4 data points. Further, our framework is interpretable because one can extract the contributions from local chemical environments to global properties. Using an example of perovskites, we show how this information can be utilized to discover empirical rules for materials design.

36 MATERIALS SCIENCE↗

Deep Koopman Neural Network for Analyzing High-Energy-Density Simulations of Electrical Wire Explosions

Megaampere-scale electrical wire experiments (EWEs) provide a platform for studying magnetohydrodynamic (MHD) instability growth in magneto-inertial fusion (MIF) devices. Even when nonlinear simulations of these experiments can digitally reproduce much of the experimentally observed instability growth, interpreting the results and understanding mode growth and evolution can be non-trivial. As a first step toward providing better interpretation of these simulation features, this work investigates the use of a deep neural network that uses Koopman operator theory to analyze the dynamics of pulsed-power-driven explosions of EWEs. This deep neural network is trained on 1-D resistive MHD simulations of EWEs. This neural network learns to transform the nonlinear data into a lower-dimensional representation where the time dynamics are linear. Layers of this neural network are shown to learn features of the simulations, including the locations of shock waves and different physical regimes of the simulation. Using the learned features, the network can compress a time state of the simulation consisting of 5120 data point into a 36-parameter lower-dimensional latent space embedding. Furthermore, these embeddings are shown to be clustered in the latent space by initial radius and time state.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

The contribution of theory and experiments to conservation in fragmented landscapes

The clearing and fragmentation of terrestrial ecosystems is commonly acknowledged as a major cause of the decline of biodiversity. These and other predicted responses to habitat fragmentation are derived from theory, which ecologists have tested with empirical approaches ranging from observations to experimental manipulations. These empirical approaches have also identified areas of theory in need of additional development. For example, experimental studies of fragmentation have provided insights such as the key role played by the matrix surrounding fragments, the importance of edge effects, and the impacts of corridors linking fragments with one another. Much less clear, however, is the extent to which these theoretical and empirical studies – while advancing our conceptual understanding of ecological responses to landscape change – help guide management and conservation practice. We review lessons learned from landscape-scale fragmentation experiments and observational studies, present the results of a survey of fragmentation and conservation experts which probed for links and mismatches between fragmentation studies and conservation practice, and discuss how future studies can contribute to conservation practice. Our survey showed that respondents consider fragmentation theory and empirical studies and their findings important for guiding conservation and management practices. The survey also identified that there are disconnects between what is typically studied by fragmentation ecologists and factors that are central to the practice of biodiversity conservation, notably, community-based human dimensions (e.g. economic, social, health issues), policy and governance, ecosystem services, eco-evolutionary responses of species, and interaction of multiple threats to biodiversity and ecosystem processes. We discuss how these disconnects can present opportunities for experiments to continue to provide valuable recommendations for conservation practice in fragmented landscapes.

60 APPLIED LIFE SCIENCES↗

Machine-Learned Force Field Modeling of Metal Organic Frameworks for CO2 Direct Air Capture

To cope with legacy greenhouse gas emissions and to achieve net-zero emissions by 2050, the U.S. Department of Energy (DOE) is funding efforts to develop direct air capture (DAC), a method for removing CO2 directly from air. Metal organic frameworks (MOFs) have been well studied as DAC sorbent materials due to their tunable structural and compositional properties. Thermodynamic simulations using force fields are often used to provide predictions of a material’s performance in many separations. However, these force fields often make assumptions about bonds and the physics of the adsorption process. A new class of force fields called machine-learned force fields (MLFFs) use machine learning to form quantitative relationships between a material’s chemical structure and the forces and energies predicted by more accurate quantum mechanical calculations, such as dispersion-corrected density functional theory (DFT). In this work, models were developed to achieve DFT-level accuracy for the forces and energies associated with MOF flexibility and CO2 adsorption using MLFFs. These methods were parametrized based on thousands of DFT calculations of CO2 in flexible MOFs and used to predict MOF structural properties as well as CO2 adsorption in several MOFs.

Findley, John↗

Machine-Learned Force Field Modeling of Metal Organic Frameworks for CO2 Direct Air Capture

To cope with legacy greenhouse gas emissions and to achieve net-zero emissions by 2050, the U.S. Department of Energy (DOE) is funding efforts to develop direct air capture (DAC), a method for removing CO2 directly from air. Metal organic frameworks (MOFs) have been studied as DAC sorbent materials because of their structural and chemical diversity. Thermodynamic calculations using classical force fields are often used to screen MOFs for their performance in separations such as CO2 capture. Machine-learned force fields (MLFFs) can use machine learning to form quantitative relationships between a material’s chemical structure and the forces and energies predicted by more accurate quantum mechanical calculations, such as dispersion-corrected density functional theory (DFT). These descriptions of forces and energies can be used to improve the accuracy of adsorption calculations. In this work, MLFF models were developed for MOFs to achieve DFT-level accuracy for the forces and energies associated with MOF flexibility and CO2 adsorption. These methods were parametrized based on thousands of DFT calculations of CO2 in flexible MOFs and used to predict MOF structural properties as well as CO2 adsorption properties.

Findley, John↗

Machine Learned Force Field Modeling of Metal Organic Frameworks for CO2 Direct Air Capture

Direct air capture (DAC) is a method for removing CO2 directly from air. Metal organic frameworks (MOFs) have been studied as DAC sorbent materials because of their structural and chemical diversity. Thermodynamic calculations using classical force fields are often used to evaluate MOFs for their performance in separations such as CO2 capture. Machine-learned force fields (MLFFs) can use machine learning to form quantitative relationships between a material’s chemical structure and the forces and energies predicted by more accurate quantum mechanical calculations, such as dispersion-corrected density functional theory (DFT). These descriptions of forces and energies can be used to improve the accuracy of adsorption calculations. In this work, classical models were used to pre-screen MOFs for CO2 capture. DFT calculations were then used to examine the adsorption mechanism. Next, MLFF models were developed for MOFs to achieve DFT-level accuracy for the forces and energies associated with MOF flexibility and CO2 adsorption. These methods were parametrized based on thousands of DFT calculations of CO2 in flexible MOFs and used to predict MOF structural properties as well as CO2 adsorption properties.

Findley, John↗

Symmetry as Bias: Rediscovering Special Relativity

This paper describes a rational reconstruction of Einstein's discovery of special relativity, validated through an implementation: the Erlanger program. Einstein's discovery of special relativity revolutionized both the content of physics and the research strategy used by theoretical physicists. This research strategy entails a mutual bootstrapping process between a hypothesis space for biases, defined through different postulated symmetries of the universe, and a hypothesis space for physical theories. The invariance principle mutually constrains these two spaces. The invariance principle enables detecting when an evolving physical theory becomes inconsistent with its bias, and also when the biases for theories describing different phenomena are inconsistent. Structural properties of the invariance principle facilitate generating a new bias when an inconsistency is detected. After a new bias is generated. this principle facilitates reformulating the old, inconsistent theory by treating the latter as a limiting approximation. The structural properties of the invariance principle can be suitably generalized to other types of biases to enable primal-dual learning.

Lowry, Michael R.↗

The Magnetic Roots of Enhanced Coronal Heating

Building a description of the sites of enhanced heating in the solar atmosphere has been a primary interest of the solar group at MSFC. For locations within and between active regions, as well as in the quiet Sun, this work shows that extended magnetic structures are heated from active compact magnetic structures at their feet. This talk will review what we have learned about the magnetic characteristics of these active footpoint sites, and the relationships of the intensity variations in the low-lying and higher structures, in the context of current theories and observations of magnetic reconnection and jets. One puzzling result is the poor correlation of intensity variations in the low and high loops.

Porter, J. G.↗

Distributed Control with Collective Intelligence

We consider systems of interacting reinforcement learning (RL) algorithms that do not work at cross purposes , in that their collective behavior maximizes a global utility function. We call such systems COllective INtelligences (COINs). We present the theory of designing COINs. Then we present experiments validating that theory in the context of two distributed control problems: We show that COINs perform near-optimally in a difficult variant of Arthur's bar problem [Arthur] (and in particular avoid the tragedy of the commons for that problem), and we also illustrate optimal performance in the master-slave problem.

Wolpert, David H.↗

Allosterically coupled conformational dynamics in solution prepare the sterol transfer protein StarD4 to release its cargo upon interaction with target membranes

Complex mechanisms regulate the cellular distribution of cholesterol, a critical component of eukaryote membranes involved in regulation of membrane protein functions directly and through the physiochemical properties of membranes. StarD4, a member of the steroidogenic acute regulator-related lipid-transfer (StART) domain (StARD)-containing protein family, is a highly efficient sterol-specific transfer protein involved in cholesterol homeostasis. Its mechanism of cargo loading and release remains unknown despite recent insights into the key role of phosphatidylinositol phosphates in modulating its interactions with target membranes. We have used large-scale atomistic Molecular dynamics (MD) simulations to study how the dynamics of cholesterol bound to the StarD4 protein can affect interaction with target membranes, and cargo delivery. We identify the two major cholesterol (CHL) binding modes in the hydrophobic pocket of StarD4, one near S136&S147 (the Ser-mode), and another closer to the putative release gate located near W171, R92&Y117 (the Trp-mode). We show that conformational changes of StarD4 associated directly with the transition between these binding modes facilitate the opening of the gate. To understand the dynamics of this connection we apply a machine-learning algorithm for the detection of rare events in MD trajectories (RED), which reveals the structural motifs involved in the opening of a front gate and a back corridor in the StarD4 structure occurring together with the spontaneous transition of CHL from the Ser-mode of binding to the Trp-mode. Further analysis of MD trajectory data with the information-theory based NbIT method reveals the allosteric network connecting the CHL binding site to the functionally important structural components of the gate and corridor. Mutations of residues in the allosteric network are shown to affect the performance of the allosteric connection. These findings outline an allosteric mechanism which prepares the CHL-bound StarD4 to release and deliver the cargo when it is bound to the target membrane.

59 BASIC BIOLOGICAL SCIENCES↗

Data-driven, multi-moment fluid modeling of Landau damping

Deriving governing equations of complex physical systems based on first principles can be quite challenging when there are certain unknown terms and hidden physical mechanisms in the systems. In this work, we apply a deep learning architecture to learn fluid partial differential equations (PDEs) of a plasma system based on the data acquired from a fully kinetic model. Here, the learned multi-moment fluid PDEs are demonstrated to incorporate kinetic effect such as Landau damping. Based on the learned fluid closure, the data-driven, multi-moment fluid modeling can well reproduce all the physical quantities derived from the fully kinetic model. The calculated damping rate of Landau damping is consistent with both the fully kinetic simulation and the linear theory. The data-driven fluid modeling of PDEs for complex physical systems may be applied to improve the fluid closure and reduce the computational cost of multi-scale modeling of global systems.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗