Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “tensor networks”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 235 records · Page 13
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Accelerating Binarized Neural Networks via Bit-Tensor-Cores in Turing GPUs

Despite foreseeing tremendous speedups over conventional deep neural networks, the performance advantage of binarized neural networks (BNNs) has merely been showcased on general-purpose processors such as CPUs and GPUs. In fact, due to being unable to leverage bit-level-parallelism with a word-based architecture, GPUs have been criticized for extremely low utilization (1%) when executing BNNs. Consequently, the latest tensorcores in NVIDIA Turing GPUs start to experimentally support bit computation. In this work, we look into this brand new bit computation capability and characterize its unique features. We show that the stride of memory access can significantly affect performance delivery and a data-format co-design is highly desired to support the tensorcores for achieving superior performance than existing software solutions without tensorcores. We realize the tensorcore-accelerated BNN design, particularly the major functions for fully-connect and convolution layers — bit matrix multiplication and bit convolution. Evaluations on two NVIDIA Turing GPUs show that, with ResNet-18, our BTC-BNN design can process ImageNet at a rate of 5.6K images per second, 77% faster than state-of-the-art. Our BNN approach is released on https://github.com/pnnl/TCBNN.

Li, Ang↗
DOE OSTI2020

Generic, Sparse Tensor Core for Neural Networks

Sparse neural network attracts more attention for model compression, fast execution, and power reduction. The state-of-the-art designed sparse tensor core for structured and static sparsity, which did not support well for generic or dynamic sparsity. We design a sparse tensor core to support generic sparsity pruning with a novel hybrid and blocked sparse matrix storage format, HB-ELL, which saves computation and storage while keeping the most significant elements, as well as supporting dynamic sparsity for data flow in neural networks. We achieve better performance with preliminary results than the state-of- the-art on an NVIDIA GPU simulator.

Wu, Xiaolong↗
DOE OSTI2022

Entanglement perspective on the quantum approximate optimization algorithm

Many quantum algorithms seek to output a specific bitstring solving the problem of interest—or a few if the solution is degenerate. It is the case for the quantum approximate optimization algorithm (QAOA) in the limit of large circuit depth, which aims to solve quadratic unconstrained binary optimization problems. Hence, the expected final state for these algorithms is either a product state or a low-entangled superposition involving a few bitstrings. What happens in between the initial N -qubit product state | 0 〉 ⊗ N and the final one regarding entanglement? Here, we consider the QAOA algorithm for solving the paradigmatic MaxCut problem on different types of graphs. We study the entanglement growth and spread resulting from randomized and optimized QAOA circuits and find that there is a volume-law entanglement barrier between the initial and final states. We also investigate the entanglement spectrum in connection with random matrix theory. In addition, we compare the entanglement production with a quantum annealing protocol aiming to solve the same MaxCut problems. Finally, we discuss the implications of our results for the simulation of QAOA circuits with tensor network-based methods relying on low-entanglement for efficiency, such as matrix product states.

Dupont, Maxime↗
DOE OSTI2023

Lightweight and effective tensor sensitivity for atomistic neural networks

Atomistic machine learning focuses on the creation of models that obey fundamental symmetries of atomistic configurations, such as permutation, translation, and rotation invariances. In many of these schemes, translation and rotation invariance are achieved by building on scalar invariants, e.g., distances between atom pairs. There is growing interest in molecular representations that work internally with higher rank rotational tensors, e.g., vector displacements between atoms, and tensor products thereof. Here, we present a framework for extending the Hierarchically Interacting Particle Neural Network (HIP-NN) with Tensor Sensitivity information (HIP-NN-TS) from each local atomic environment. Crucially, the method employs a weight tying strategy that allows direct incorporation of many-body information while adding very few model parameters. We show that HIP-NN-TS is more accurate than HIP-NN, with negligible increase in parameter count, for several datasets and network sizes. As the dataset becomes more complex, tensor sensitivities provide greater improvements to model accuracy. In particular, HIP-NN-TS achieves a record mean absolute error of 0.927 $\frac{\text{kcal}}{\text{mol}}$ for conformational energy variation on the challenging COMP6 benchmark, which includes a broad set of organic molecules. We also compare the computational performance of HIP-NN-TS to HIP-NN and other models in the literature.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2023

Variational Neural-Network Ansatz for Continuum Quantum Field Theory

Physicists dating back to Feynman have lamented the difficulties of applying the variational principle to quantum field theories. In nonrelativistic quantum field theories, the challenge is to parametrize and optimize over the infinitely many n-particle wave functions comprising the state’s Fock-space representation. Here we approach this problem by introducing neural-network quantum field states, a deep learning ansatz that enables application of the variational principle to nonrelativistic quantum field theories in the continuum. Our ansatz uses the Deep Sets neural network architecture to simultaneously parametrize all of the n-particle wave functions comprising a quantum field state. We employ our ansatz to approximate ground states of various field theories, including an inhomogeneous system and a system with long-range interactions, thus demonstrating a powerful new tool for probing quantum field theories.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2022

Quantum Perturbation Theory Using Tensor Cores and a Deep Neural Network

In this work, time-independent quantum response calculations are performed using Tensor cores. This is achieved by mapping density matrix perturbation theory onto the computational structure of a deep neural network. The main computational cost of each deep layer is dominated by tensor contractions, i.e., dense matrix–matrix multiplications, in mixed-precision arithmetics, which achieves close to peak performance. Quantum response calculations are demonstrated and analyzed using self-consistent charge density-functional tight-binding theory as well as coupled-perturbed Hartree–Fock theory. For linear response calculations, a novel parameter-free convergence criterion is presented that is well-suited for numerically noisy low-precision floating point operations and we demonstrate a peak performance of almost 200 Tflops using the Tensor cores of two Nvidia A100 GPUs.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2023

Machine Learning Full NMR Chemical Shift Tensors of Silicon Oxides with Equivariant Graph Neural Networks

The nuclear magnetic resonance (NMR) chemical shift tensor is a highly sensitive probe of the electronic structure of an atom and furthermore its local structure. Recently, machine learning has been applied to NMR in the prediction of isotropic chemical shifts from a structure. Current machine learning models, however, often ignore the full chemical shift tensor for the easier-to-predict isotropic chemical shift, effectively ignoring a multitude of structural information available in the NMR chemical shift tensor. Here we use an equivariant graph neural network (GNN) to predict full 29 Si chemical shift tensors in silicate materials. The equivariant GNN model predicts full tensors to a mean absolute error of 1.05 ppm and is able to accurately determine the magnitude, anisotropy, and tensor orientation in a diverse set of silicon oxide local structures. When compared with other models, the equivariant GNN model outperforms the state-of-the-art machine learning models by 53%. The equivariant GNN model also outperforms historic analytical models by 57% for isotropic chemical shift and 91% for anisotropy. The software is available as a simple-to-use open-source repository, allowing similar models to be created and trained with ease.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2023

Toward Global Regional Seismic Moment Tensor Inversion with Three-Dimensional Earth Models for Nuclear Explosion Monitoring with Sparse Networks: Demonstration of Reciprocity for Strain Greens Tensor Database Simulation with Salvus

Seismic source characterization is an essential function of global nuclear explosion monitoring (NEM). While large events (roughly with moment magnitude, M w , greater than 5.0) can often be easily detected, located and identified with high signal-to-noise ratios at teleseismic distances (> 20°), trends in NEM research require confident source characterization at much lower magnitudes (say down to 3.0) and exploitation of sparse observations (from only a few stations) at regional distance (< 20°). Regional distance waveform inversion to characterize sources is now widely used and effective (e.g. Ford et al., 2009; Alvizuri and Tape, 2018; Alvizuri et al., 2018; Chiang et al., 2018; Ford et al., 2022). These methods obtain the magnitude, depth and seismic moment tensor, which represents the forces that excited the observed seismic waves (slip on an earthquake fault, explosion, collapse or a combination of various forces). Common to many problems in seismology, the isolation of the source 2 properties requires removal of path propagation effects that waves experience while traveling through the three-dimensional (3D) Earth (the structure exists due to different rock types, material properties, temperature and tectonic processes).

58 GEOSCIENCES↗
DOE OSTI2022

Towards Compact Neural Networks via End-to-End Training: A Bayesian Tensor Approach with Automatic Rank Determination

Post-training model compression can reduce the inference costs of deep neural networks, but uncompressed training still consumes enormous hardware resources and energy. To enable low-energy training on edge devices, it is highly desirable to directly train a compact neural network from scratch with a low memory cost. Low-rank tensor decomposition is an effective approach to reduce the memory and computing costs of large neural networks. However, directly training low-rank tensorized neural networks is a very challenging task because it is hard to determine a proper tensor rank a priori, and the tensor rank controls both model complexity and accuracy. Here, this paper presents a novel end-to-end framework for low-rank tensorized training. We first develop a Bayesian model that supports various low-rank tensor formats (e.g., CANDECOMP/PARAFAC, Tucker, tensor-train, and tensor-train matrix) and reduces neural network parameters with automatic rank determination during training. Then we develop a customized Bayesian solver to train large-scale tensorized neural networks. Our training methods shows orders-of-magnitude parameter reduction and little accuracy loss (or even better accuracy) in the experiments. On a very large deep learning recommendation system with over 4.2 ×10 9 model parameters, our method can reduce the parameter number to 1.6 ×10 5 automatically in the training process (i.e., by 2.6 ×10 4 times) while achieving almost the same accuracy. Code is available at https://github.com/colehawkins/bayesian-tensor-rank-determination.

compact neural networks↗
DOE OSTI2025

Direct interpolative construction of the discrete Fourier transform as a matrix product operator

The quantum Fourier transform (QFT), which can be viewed as a reindexing of the discrete Fourier transform (DFT), has been shown to be compressible as a low-rank matrix product operator (MPO) or quantized tensor train (QTT) operator. However, the original proof of this fact does not furnish a construction of the MPO with a guaranteed error bound. Meanwhile, the existing practical construction of this MPO, based on the compression of a quantum circuit, is not as efficient as possible. We present a simple closed-form construction of the QFT MPO using the interpolative decomposition, with guaranteed near-optimal compression error for a given rank. This construction can speed up the application of the QFT and the DFT, respectively, in quantum circuit simulations and QTT applications. We also connect our interpolative construction to the approximate quantum Fourier transform (AQFT) by demonstrating that the AQFT can be viewed as an MPO constructed using a different interpolation scheme.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Variational quantum reinforcement learning via evolutionary optimization

Abstract Recent advances in classical reinforcement learning (RL) and quantum computation point to a promising direction for performing RL on a quantum computer. However, potential applications in quantum RL are limited by the number of qubits available in modern quantum devices. Here, we present two frameworks for deep quantum RL tasks using gradient-free evolutionary optimization. First, we apply the amplitude encoding scheme to the Cart-Pole problem, where we demonstrate the quantum advantage of parameter saving using amplitude encoding. Second, we propose a hybrid framework where the quantum RL agents are equipped with a hybrid tensor network-variational quantum circuit (TN-VQC) architecture to handle inputs of dimensions exceeding the number of qubits. This allows us to perform quantum RL in the MiniGrid environment with 147-dimensional inputs. The hybrid TN-VQC architecture provides a natural way to perform efficient compression of the input dimension, enabling further quantum RL applications on noisy intermediate-scale quantum devices.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Ground-state-based model reduction with unitary circuits

Here, we present a method to numerically obtain low-energy effective models based on a unitary transformation of the ground state. The algorithm finds a unitary circuit that transforms the ground state of the original model to a projected wavefunction with only the low-energy degrees of freedom. The effective model can then be derived using the unitary transformation encoded in the circuit. We test our method on the one-dimensional and two-dimensional square-lattice Hubbard model at half-filling, and obtain more accurate effective spin models than the standard perturbative approach.

Hubbard model↗
DOE OSTI2025

Spontaneous magnon decays from nonrelativistic time-reversal symmetry breaking in altermagnets

Quasiparticles are central to condensed matter physics, but their stability can be undermined by quantum many-body interactions. Magnons, i.e., quasiparticles in quantum magnets, are particularly intriguing because their properties are governed by both real and spin space. While crystal symmetries may be low, spin interactions often remain approximately isotropic, limiting spontaneous magnon decay. Textbook wisdom holds that collinear Heisenberg magnets follow a dichotomy: ferromagnets host stable magnons, while antiferromagnetic magnons may decay depending on dispersion curvature. Up to now, relativistic spin-orbit coupling and noncollinear order that connect spin space to real space were shown to introduce more complex magnon instability mechanisms. Here, we show that even in nonrelativistic isotropic collinear systems, this conventional dichotomy is disrupted in altermagnets. Altermagnets, a newly identified class of collinear magnets, exhibit compensated spin order with nonrelativistic time-reversal symmetry breaking and even-parity band splitting. Using kinematic analysis, nonlinear spin-wave theory, and quantum simulations, we reveal that even weak band splitting opens a decay phase space, driving quasiparticle breakdown. Additionally, 𝑑-wave altermagnets form a rare “island of stability” at the Brillouin-zone center. Furthermore, our findings establish a quasiparticle stability trichotomy in collinear Heisenberg magnets and position altermagnets as a promising platform for unconventional spin dynamics.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
DOE OSTI2023

Variational quantum simulation of the critical Ising model with symmetry averaging

Here we investigate the use of deep multiscale entanglement renormalization ansatz (DMERA) circuits as a variational ansatz. We use the exactly solvable one-dimensional critical transverse-field Ising model as a test bed. Numerically exact simulation of the quantum circuit ansatz can in this case be carried out to hundreds of qubits by exploiting efficient classical algorithms for simulating matchgate circuits. We find that, for this system, the DMERA strongly outperforms a standard quantum approximate optimization algorithm (QAOA)–style ansatz, and that a major source of systematic error in correlation functions approximated using the DMERA is the breaking of the translational and Kramers-Wannier symmetries of the transverse-field Ising model. We are able to reduce this error by up to four orders of magnitude by symmetry averaging, without incurring additional cost in qubits or circuit depth. Here, we propose that this technique for mitigating systematic error could be applied to noisy intermediate-scale quantum (NISQ) simulations of physical systems with other symmetries.

1-dimensional spin chains↗
DOE OSTI2020

Time-dependent variational principle for mixed matrix product states in the thermodynamic limit

Here, we describe a time evolution algorithm for quantum spin chains whose Hamiltonians are composed of an infinite uniform left and right bulk part, and an arbitrary finite region in between. The left and right bulk parts are allowed to be different from each other. The algorithm is based on the time-dependent variational principle (TDVP) of matrix product states. It is inversion-free and very simple to adapt from an existing TDVP code for finite systems. The importance of working in the projective Hilbert space is highlighted. We study the quantum Ising model as a benchmark and an illustrative example. The spread of information after a local quench is studied in both the ballistic and the diffusive case. We also offer a derivation of TDVP directly from symplectic geometry.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
DOE OSTI2020

Efficient simulation of moiré materials using the density matrix renormalization group

Here we present an infinite density-matrix renormalization group (DMRG) study of an interacting continuum model of twisted bilayer graphene (tBLG) near the magic angle. Because of the long-range Coulomb interaction and the large number of orbital degrees of freedom, tBLG is difficult to study with standard DMRG techniques - even constructing and storing the Hamiltonian already poses a major challenge. To overcome these difficulties, we use a recently developed compression procedure to obtain a matrix product operator representation of the interacting tBLG Hamiltonian which we show is both efficient and accurate even when including the spin, valley, and orbital degrees of freedom. To benchmark our approach, we focus mainly on the spinless, single-valley version of the problem where, at half filling, we find that the ground state is a nematic semimetal. Remarkably, we find that the ground state is essentially a k-space Slater determinant, so that Hartree-Fock and DMRG give virtually identical results for this problem. Our results show that the effects of long-range interactions in magic angle graphene can be efficiently simulated with DMRG and open up a new route for numerically studying strong correlation physics in spinful, two-valley tBLG, and other moiré materials in future work.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
DOE OSTI2021

Study of spin symmetry in the doped t-J model using infinite projected entangled pair states

We study the two-dimensional t-J model on a square lattice using infinite projected entangled pair states (iPEPS). At small doping, multiple orders, such as antiferromagnetic order, stripe order and superconducting order, are intertwined or compete with each other. We demonstrate the role of spin symmetry at small doping by either imposing SU(2) spin symmetry or its U(1) subgroup in the iPEPS ansatz, thereby excluding or allowing spontaneous spin-symmetry breaking, respectively, in the thermodynamic limit. From a detailed comparison of our simulations, we provide evidence that stripe order is pinned by long-range antiferromagnetic order. We also find SU(2) iPEPS, enforcing a spin-singlet state, yields a uniform charge distribution and favors d-wave singlet pairing.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
DOE OSTI2021

Visualizing quasiparticles from quantum entanglement for general one-dimensional phases

In this work, we present a quantum information framework for the entanglement behavior of the low-energy quasiparticle (QP) excitations in various quantum phases in one-dimensional (1D) systems. We first establish an exact correspondence between the correlation matrix and the QP entanglement Hamiltonian for free fermions and find an extended in-gap state in the QP entanglement Hamiltonian as a consequence of the position uncertainty of the QP. A more general understanding of such an in-gap state can be extended to a Kramers theorem for the QP entanglement Hamiltonian, which also applies to strongly interacting systems. Further, we present a set of ubiquitous entanglement spectrum features, dubbed entanglement fragmentation, conditional mutual information, and measurement-induced nonlocal entanglement for QPs in 1D symmetry protected topological phases. Our result thus provides another framework to identify different phases of matter in terms of their QP entanglement.

1-dimensional spin chains↗