Neural network assisted analysis of bimetallic nanocatalysts using X-ray absorption near edge structure spectroscopy
Explore the source record for details and available documents.
99 GENERAL AND MISCELLANEOUS↗
SEARCH · Engineering Papers
Results for “neural”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Deep Kronecker neural networks: A general framework for neural networks with adaptive activation functions
Here we propose a new type of neural networks, Kronecker neural networks (KNNs), that form a general framework for neural networks with adaptive activation functions. KNNs employ the Kronecker product, which provides an efficient way of constructing a very wide network while keeping the number of parameters low. Our theoretical analysis reveals that under suitable conditions, KNNs induce a faster decay of the loss than that by the feed-forward networks. This is also empirically verified through a set of computational examples. Furthermore, under certain technical assumptions, we establish global convergence of gradient descent for KNNs. As a specific case, we propose the Rowdy activation function that is designed to get rid of any saturation region by injecting sinusoidal fluctuations, which include trainable parameters. The proposed Rowdy activation function can be employed in any neural network architecture like feed-forward neural networks, Recurrent neural networks, Convolutional neural networks etc. The effectiveness of KNNs with Rowdy activation is demonstrated through various computational experiments including function approximation using feed-forward neural networks, solution inference of partial differential equations using the physics-informed neural networks, and standard deep learning benchmark problems using convolutional and fully-connected neural networks.
97 MATHEMATICS AND COMPUTING↗
Learning generative neural networks with physics knowledge
Deep generative neural networks have enabled modeling complex distributions, but incorporating physics knowledge into the neural networks is still challenging and is at the core of current physics-based machine learning research. To this end, we propose a physics generative neural network (PhysGNN), a new class of generative neural networks for learning unknown distributions in a physical system described by partial differential equations (PDE). PhysGNN couples PDE systems with generative neural networks. It is a fully differentiable model that allows back-propagation of gradients through both numerical PDE solvers and generative neural networks, and is trained by minimizing the discrete Wasserstein distance between generated and observed probability distributions of the PDE outputs using the stochastic gradient descent method. Moreover, PhysGNN does not require adversarial training like standard generative neural networks, which offers better stability than adversarial training. We show that PhysGNN can learn complex distributions in stochastic inverse problems, where conventional methods such as maximum likelihood estimation and momentum matching methods may be inapplicable when little knowledge is known about the form of unknown distributions or the physical model is too complex. Furthermore, our method allows physics-based generative neural network training for learning complex distributions in the context of differential equations.
97 MATHEMATICS AND COMPUTING↗
Modeling the AC Power Flow Equations with Optimally Compact Neural Networks: Application to Unit Commitment
Nonlinear power flow constraints render a variety of power system optimization problems computationally intractable. Emerging research shows, however, that the nonlinear AC power flow equations can be successfully modeled using neural networks. These neural networks can be exactly transformed into mixed integer linear programs and embedded inside challenging optimization problems, thus replacing nonlinearities that are intractable for many applications with tractable piecewise linear approximations. Such approaches, though, suffer from an explosion of the number of binary variables needed to represent the neural network. Accordingly, this paper develops a technique for training an "optimally compact'' neural network, i.e., one that can represent the power flow equations with a sufficiently high degree of accuracy while still maintaining a tractable number of binary variables. We demonstrate the use of this neural network as an approximator of the nonlinear power flow equations by embedding it in the AC unit commitment problem, transforming the problem from a mixed integer nonlinear program into a more manageable mixed integer linear program. We use the 14-, 57-, and 89-bus networks as test cases and compare the AC-feasibility of commitment decisions resulting from the neural network, DC, and linearized power flow approximations. Our results show that the neural network model outperforms both the DC and linearized power flow approximations when embedded in the unit commitment problem. The neural network formulation most often selects a feasible unit commitment schedule, and furthermore, it only s
AC power flow↗
B-PINNs: Bayesian Physics-informal Neural Networks for Forward and Inverse PDE Problems with Noisy Data
We propose a Bayesian physics-informed neural network (B-PINN) to solve both forward and inverse nonlinear problems described by partial differential equations (PDEs) and noisy data. In this Bayesian framework, the Bayesian neural network (BNN) combined with a PINN for PDEs serves as the prior while the Hamiltonian Monte Carlo (HMC) or the variational inference (VI) could serve as an estimator of the posterior. B-PINNs make use of both physical laws and scattered noisy measurements to provide predictions and quantify the aleatoric uncertainty arising from the noisy data in the Bayesian framework. Compared with PINNs, in addition to uncertainty quantification, B-PINNs obtain more accurate predictions in scenarios with large noise due to their capability of avoiding overfitting. We conduct a systematic comparison between the two different approaches for the B-PINNs posterior estimation (i.e., HMC or VI), along with dropout used for quantifying uncertainty in deep neural networks. Our experiments show that HMC is more suitable than VI with mean field Gaussian approximation for the B-PINNs posterior estimation, while dropout employed in PINNs can hardly provide accurate predictions with reasonable uncertainty. Finally, we replace the BNN in the prior with a truncated Karhunen-Loève (KL) expansion combined with HMC or a deep normalizing flow (DNF) model as posterior estimators. The KL is as accurate as BNN and much faster but this framework cannot be easily extended to high-dimensional problems unlike the BNN based framework.
Non linear PDEs, Noisy data, Bysian physics inform↗
Reducing Communication in Graph Neural Network Training
Graph Neural Networks (GNNs) are powerful and flexible neural networks that use the naturally sparse connectivity information of the data. GNNs represent this connectivity as sparse matrices, which have lower arithmetic intensity and thus higher communication costs compared to dense matrices, making GNNs harder to scale to high concurrencies than convolutional or fully-connected neural networks. Here, we introduce a family of parallel algorithms for training GNNs and show that they can asymptotically reduce communication compared to previous parallel GNN training methods. We implement these algorithms, which are based on 1D, 1. 5D, 2D, and 3D sparse-dense matrix multiplication, using torch.distributed on GPU-equipped clusters. Our algorithms optimize communication across the full GNN training pipeline. We train GNNs on over a hundred GPUs on multiple datasets, including a protein network with over a billion edges.
97 MATHEMATICS AND COMPUTING↗
A systematic approach to generating accurate neural network potentials: the case of carbon
Abstract Availability of affordable and widely applicable interatomic potentials is the key needed to unlock the riches of modern materials modeling. Artificial neural network-based approaches for generating potentials are promising; however, neural network training requires large amounts of data, sampled adequately from an often unknown potential energy surface. Here we propose a self-consistent approach that is based on crystal structure prediction formalism and is guided by unsupervised data analysis, to construct an accurate, inexpensive, and transferable artificial neural network potential. Using this approach, we construct an interatomic potential for carbon and demonstrate its ability to reproduce first principles results on elastic and vibrational properties for diamond, graphite, and graphene, as well as energy ordering and structural properties of a wide range of crystalline and amorphous phases.
Chemistry↗
Quantum-inspired weight-constrained neural network: Reducing variable numbers by 100× compared to standard neural networks
Although quantum machine learning has shown great promise, the practical application of quantum computers remains constrained in the noisy intermediate-scale quantum era. To take advantage of quantum machine learning, we investigate the underlying mathematical principles of these quantum models and find that the quantum neural network with amplitude encoding is equivalent to a weight-constrained neural network. Motivated by this discovery, we develop a classical weight-constrained neural network. We find that this approach can reduce the number of variables in a classical neural network by a factor of 135 while preserving its accuracy. In addition, we develop a dropout method to enhance the robustness of quantum machine learning models, which are highly susceptible to adversarial attacks. This technique can also be applied to improve the adversarial robustness of the classical weight-constrained neural network, which is essential for industry applications, such as self-driving vehicles. Our work offers an approach to reduce the complexity of large classical neural networks, addressing a critical challenge in machine learning.
quantum algorithms & computation↗
Accelerating Hamiltonian Monte Carlo for Bayesian inference in neural networks and neural operators
Hamiltonian Monte Carlo (HMC) is a powerful and accurate method to sample from the posterior distribution in Bayesian inference. However, HMC techniques are computationally demanding for Bayesian neural networks due to the high dimensionality of the network’s parameter space and the non-convexity of their posterior distributions. Therefore, various approximation techniques, such as variational inference (VI) or stochastic gradient MCMC, are often employed to infer the posterior distribution of the network parameters. Such approximations introduce inaccuracies in the inferred distributions, resulting in unreliable uncertainty estimates. In this work, we propose a hybrid approach that combines inexpensive VI and accurate HMC methods to efficiently and accurately quantify uncertainties in neural networks and neural operators. The proposed approach leverages an initial VI training on the full network. We examine the influence of individual parameters on the prediction uncertainty, which shows that a large proportion of the parameters do not contribute substantially to uncertainty in the network predictions. This information is then used to significantly reduce the dimension of the parameter space, and HMC is performed only for the subset of network parameters that strongly influence prediction uncertainties. This yields a framework for accelerating the full batch HMC for posterior inference in neural networks. We demonstrate the efficiency and accuracy of the proposed framework on deep neural networks and operator networks, showing that inference can be performed for large networks with tens to hundreds of thousands of parameters. Finally, we show that this method can effectively learn surrogates for complex physical systems by modeling the operator that maps from upstream conditions to wall-pressure data on a cone in hypersonic flow.
Bayesian inference↗
Variational Monte Carlo Calculations of A ≤ 4 Nuclei with an Artificial Neural-Network Correlator Ansatz
Here, the complexity of many-body quantum wave functions is a central aspect of several fields of physics and chemistry where nonperturbative interactions are prominent. Artificial neural networks (ANNs) have proven to be a flexible tool to approximate quantum many-body states in condensed matter and chemistry problems. In this work we introduce a neural-network quantum state ansatz to model the ground-state wave function of light nuclei, and approximately solve the nuclear many-body Schrodinger equation. Using efficient stochastic sampling and optimization schemes, our approach extends pioneering applications of ANNs in the field, which present exponentially scaling algorithmic complexity. We compute the binding energies and point-nucleon densities of A ≤ 4 nuclei as emerging from a leading-order pionless effective field theory Hamiltonian. We successfully benchmark the ANN wave function against more conventional parametrizations based on two- and three-body Jastrow functions, and virtually exact Green's function Monte Carlo results.
73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
ℤ2 × ℤ2 Equivariant Quantum Neural Networks: Benchmarking against Classical Neural Networks
This paper presents a comparative analysis of the performance of Equivariant Quantum Neural Networks (EQNNs) and Quantum Neural Networks (QNNs), juxtaposed against their classical counterparts: Equivariant Neural Networks (ENNs) and Deep Neural Networks (DNNs). We evaluate the performance of each network with three two-dimensional toy examples for a binary classification task, focusing on model complexity (measured by the number of parameters) and the size of the training dataset. Our results show that the Z2×Z2 EQNN and the QNN provide superior performance for smaller parameter sets and modest training data samples.
Dong, Zhongtian (ORCID:0000000210003454)↗
Stability Analysis of Substituted Cobaltocenium [Bis(cyclopentadienyl)cobalt(III)] Employing Chemistry-Informed Neural Networks
Cobaltocenium derivatives are promising components of the anion exchange membranes due to their excellent thermal and alkaline stability under the operating conditions of a fuel cell. Here we present an efficient modeling approach of assessing the chemical stability of substituted cobaltocenium CoCp 2 + based on the computed electronic structure enhanced by machine learning techniques. Within the aqueous environment, the positive charge of the metal cation is balanced by the hydroxide anion through formation of the CoCp 2 + OH¯ complexes, whose dissociation is studied within the implicit solvent employing density functional theory. The data set of about 118 species based on 42 substituent groups characterized by a range of electron- donating (ED) and electron-withdrawing (EW) properties is constructed and analyzed. Given 12 carefully chosen chemistry-informed descriptors of the complexes and relevant fragments, the stability of the complexes is found to strongly correlate with the energies of the highest occupied and lowest unoccupied molecular orbitals, modulated by a switching function of the Hirshfeld charge. The latter is used as a measure of the electron withdrawing-donating character of the substituents. Based on this observation from the conventional regression analysis, two fully connected, feed-forward neural network (FNN) models with different unit structures, called the chemistry-informed (CINN) and the quadratic (QNN) neural networks, are developed. Both models predict the bond dissociation energies of the cobaltocenium complexes with mean relative errors less than 5.40% and average absolute errors less than 0.94 kcal/mol. The results show the potential of QNN to efficiently capture more complex relationships. Here, the concept of incorporating the domain (chemical) knowledge/insight into the neural network structure paves the way to applications of machine learning techniques with small data sets, ultimately leading to better predictive models compared to the conventional regression analysis.
36 MATERIALS SCIENCE↗
Neural entropy-stable conservative flux form neural networks for learning hyperbolic conservation laws
We propose a neural entropy-stable conservative flux form neural network (NESCFN) for learning hyperbolic conservation laws and their associated entropy functions directly from solution trajectories, without requiring any predefined numerical discretization. While recent neural network architectures have successfully integrated classical numerical principles into learned models, most rely on prior knowledge of the governing equations or assume a fixed discretization. Our approach removes this dependency by embedding entropy-stable design principles into the learning process itself, enabling the discovery of physically consistent dynamics in a fully data-driven setting. By jointly learning both the flux function and a corresponding entropy, NESCFN promotes conservation and entropy dissipation, which is critical for long-term stability and fidelity in the system of hyperbolic conservation laws. Furthermore, numerical results demonstrate that the method achieves stability and conservation over extended time horizons and accurately captures shock propagation speeds, even without oracle access to future-time solution profiles in the training data.
Conservative flux form↗
Neural Network Water Model Based on the MB-Pol Many-Body Potential
The MB-pol many-body potential accurately predicts many properties of water, including cluster, liquid phase, and vapor–liquid equilibrium properties, but its high computational cost can make applying it in large-scale simulations quite challenging. Here, in order to address this limitation, we developed a “deep potential” neural network (DPMD) model based on the MB-pol potential for water. We find that a DPMD model trained on mostly liquid configurations yields a good description of the bulk liquid phase but severely underpredicts vapor–liquid coexistence densities. By contrast, adding cluster configurations to the neural network training set leads to a good agreement for the vapor coexistence densities. Liquid phase densities under supercooled conditions are also represented well, even though they were not included in the training set. These results confirm that neural network models can combine accuracy and transferability if sufficient attention is given to the construction of a representative training set for the target system.
37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
Complex imaging of phase domains by deep neural networks
Explore the source record for details and available documents.
Wu, Longlong↗
3-D components of a biological neural network visualized in computer generated imagery. II - Macular neural network organization
Computer-assisted reconstructions of small parts of the macular neural network show how the nerve terminals and receptive fields are organized in 3-dimensional space. This biological neural network is anatomically organized for parallel distributed processing of information. Processing appears to be more complex than in computer-based neural network, because spatiotemporal factors figure into synaptic weighting. Serial reconstruction data show anatomical arrangements which suggest that (1) assemblies of cells analyze and distribute information with inbuilt redundancy, to improve reliability; (2) feedforward/feedback loops provide the capacity for presynaptic modulation of output during processing; (3) constrained randomness in connectivities contributes to adaptability; and (4) local variations in network complexity permit differing analyses of incoming signals to take place simultaneously. The last inference suggests that there may be segregation of information flow to central stations subserving particular functions.
Ross, Muriel D.↗