Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Tensor”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Nonnegative canonical tensor decomposition with linear constraints: nnCANDELINC

Abstract There is an emerging interest for tensor factorization applications in big‐data analytics and machine learning. To speed up the factorization of extra‐large datasets, organized in multidimensional arrays (also known as tensors), easy to compute compression‐based tensor representations, such as, Tucker and tensor train formats, are used to approximate the initial large‐tensor. Further, tensor factorization is used to extract latent features that can facilitate discoveries of new mechanisms and signatures hidden in the data, where the explainability of the latent features is of principal importance. Nonnegative tensor factorization extracts latent features that are naturally sparse and parts of the data, which makes them easily interpretable. However, to take into account available domain knowledge and subject matter expertise, often additional constraints need to be imposed, which lead us to canonical decomposition with linear constraints (CANDELINC), a canonical polyadic decomposition with rank deficient factors. In CANDELINC, Tucker compression is used as a preprocessing step, which lead to a larger residual error but to more explainable latent features. Here, we propose a nonnegative CANDELINC (nnCANDELINC) accomplished via a specific nonnegative Tucker decomposition; we refer to as minimal or canonical nonnegative Tucker. We derive several results required to understand the specificity of nnCANDELINC, focusing on the difficulties of preserving the nonnegative rank of a tensor to its Tucker core and comparing the real valued to nonnegative case. Finally, we demonstrate nnCANDELINC performance on synthetic and real‐world examples.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Tensor force role in β decays analyzed within the Gogny-interaction shell model

The half-life of the famous C 14 β decay is anomalously long, with different mechanisms: the tensor force, cross-shell mixing, and three-body forces, proposed to explain the cancellations that lead to a small transition matrix element. In this study, we revisit and analyze the role of the tensor force for the β decay of C 14 as well as of neighboring isotopes. We add a tensor force to the Gogny interaction, and derive an effective Hamiltonian for shell-model calculations. The calculations were carried out in a p – s d model space to investigate cross-shell effects. Furthermore, we decompose the wave functions according to the total orbital angular momentum L in order to analyze the effects of the tensor force and cross-shell mixing. The inclusion of the tensor force significantly improves the shell-model calculations of the β -decay properties of carbon isotopes. In particular, the anomalously slow β decay of C 14 can be explained by the isospin T = 0 part of the tensor force, which changes the components of N 14 with the orbital angular momentum L = 0 , 1 , and results in a dramatic suppression of the Gamow-Teller transition strength. At the same time, the description of other nearby β decays are improved. Decomposition of wave function into L components illuminates how the tensor force modifies nuclear wave functions, in particular suppression of β -decay matrix elements. Cross-shell mixing also has a visible impact on the β -decay strength. Inclusion of the tensor force does not seem to significantly change, however, binding energies of the nuclei within the phenomenological interaction.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2018

Optimizing Tensor Contractions in CCSD(T) for Efficient Execution on GPUs

Tensor contractions are higher dimensional analogs of matrix multiplications, used in many computational contexts such as high order models in quantum chemistry, deep learning, nite element methods etc. In contrast to the wide availability of high-performance libraries for matrix multiplication on GPUs, the same is not true for tensor contractions. In this paper, we address the optimization of a set of symmetrized tensor contractions that form the computational bottleneck in the CCSD(T) coupled-cluster method in computational chemistry suites like NWChem. Some of the challenges in optimizing tensor contractions that arise in practice from the variety of dimensionalities and shapes for tensors include effective mapping of the high-dimensional iteration space to threads, choice of data buffering in shared-memory and registers, and tile sizes for multi-level tiling. Furthermore, in the case of symmetrized tensor contractions in CCSD(T), it is also a challenge to fuse contractions to reduce data movement cost by exploiting reuse of intermediate tensors. In this paper, we developed an efficient GPU implementation of the tensor contractions in CCSD(T) using shared-memory buffering, register tiling, loop fusion and register transpose. Experimental results demonstrate significant improvement over the current state-of-the-art.

Kim, Jinsung↗
DOE OSTI2020

GentenMPI: Distributed Memory Sparse Tensor Decomposition

GentenMPl is a toolkit of sparse canonical polyadic (CP) tensor decomposition algorithms that is designed to run effectively on distributed-memory high-performance computers. Its use of distributed-memory parallelism enables it to efficiently decompose tensors that are too large for a single compute node's memory. GentenMPl leverages Sandia's decades-long investment in the Trilinos solver framework for much of its parallel-computation capability. Trilinos contains numerical algorithms and linear algebra classes that have been optimized for parallel simulation of complex physical phenomena. This work applies these tools to the data science problem of sparse tensor decomposition. In this report, we describe the use of Trilinos in GentenMPl, extensions needed for sparse tensor decomposition, and implementations of the CP-ALS (CP via alternating least squares) and GCP-SGD (generalized CP via stochastic gradient descent) sparse tensor decomposition algorithms. We show that GentenMPl can decompose sparse tensors of extreme size, e.g., a 12.6-terabyte tensor on 8192 computer cores. We demonstrate that the Trilinos backbone provides good strong and weak scaling of the tensor decomposition algorithms.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Improving Runtime Performance of Tensor Computations using Rust From Python

In this work, we investigate improving the runtime performance of key computational kernels in the Python Tensor Toolbox (pyttb), a package for analyzing tensor data across a wide variety of applications. Recent runtime performance improvements have been demonstrated using Rust, a compiled language, from Python via extension modules leveraging the Python C API—e.g., web applications, data parsing, data validation, etc. Using this same approach, we study the runtime performance of key tensor kernels of increasing complexity, from simple kernels involving sums of products over data accessed through single and nested loops to more advanced tensor multiplication kernels that are key in low-rank tensor decomposition and tensor regression algorithms. In numerical experiments involving synthetically generated tensor data of various sizes and these tensor kernels, we demonstrate consistent improvements in runtime performance when using Rust from Python over 1) using Python alone, 2) using Python and the Numba just-in-time Python compiler (for loop-based kernels), and 3) using the NumPy Python package for scientific computing (for pyttb kernels).

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Tucker-1 Boolean Tensor Factorization with Quantum Annealers

Quantum annealers are an emerging computational architecture that have the potential to address some challenging computational issues that will be left unresolved as we approach the end of the Moore's Law era of computing. D-Wave quantum annealers are designed to solve a challenging set of problems - quadratic unconstrained binary optimization problems. This makes them a natural fit for solving problems with binary or Boolean variables. Here, we explore the use of a quantum annealer to solve Boolean tensor factorization. The goal of Boolean tensor factorization is to represent a high-dimensional tensor filled with Boolean values as a product of Boolean matrices and a Boolean core tensor. We show that a particular Boolean tensor factorization problem (called Tucker-1 factorization) can be decomposed into a sequence of quadratic unconstrained binary optimization problems that can be solved with a D-Wave 2000Q quantum annealer. While quantum annealers specifically and quantum computers in general are at a fairly early stage in their development, they are currently capable of solving these Boolean tensor factorization problems. Importantly, our results show that for fairly small tensors, we are frequently able to obtain an accurate (sometimes exact) factorization using quantum annealing.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2016

Analysis of structural correlations in a model binary 3D liquid through the eigenvalues and eigenvectors of the atomic stress tensors

It is possible to associate with every atom or molecule in a liquid its own atomic stress tensor. These atomic stress tensors can be used to describe liquids’ structures and to investigate the connection between structural and dynamic properties. In particular, atomic stresses allow to address atomic scale correlations relevant to the Green-Kubo expression for viscosity. Previously correlations between the atomic stresses of different atoms were studied using the Cartesian representation of the stress tensors or the representation based on spherical harmonics. In this paper we address structural correlations in a 3D model binary liquid using the eigenvalues and eigenvectors of the atomic stress tensors. This approach allows to interpret correlations relevant to the Green-Kubo expression for viscosity in a simple geometric way. On decrease of temperature the changes in the relevant stress correlation function between different atoms are significantly more pronounced than the changes in the pair density function. We demonstrate that this behaviour originates from the orientational correlations between the eigenvectors of the atomic stress tensors. We also found correlations between the eigenvalues of the same atomic stress tensor. For the studied system, with purely repulsive interactions between the particles, the eigenvalues of every atomic stress tensor are positive and they can be ordered: λ{sub 1} ≥ λ{sub 2} ≥ λ{sub 3} ≥ 0. We found that, for the particles of a given type, the probability distributions of the ratios (λ{sub 2}/λ{sub 1}) and (λ{sub 3}/λ{sub 2}) are essentially identical to each other in the liquids state. We also found that λ{sub 2} tends to be equal to the geometric average of λ{sub 1} and λ{sub 3}. In our view, correlations between the eigenvalues may represent “the Poisson ratio effect” at the atomic scale.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2024

QuTree: A tree tensor network package

Here we present QuTree, a C++ library for tree tensor network approaches. QuTree provides class structures for tensors, tensor trees, and related linear algebra functions that facilitate the fast development of tree tensor network approaches such as the multilayer multiconfigurational time-dependent Hartree approach or the density matrix renormalization group approach and its various extensions. We investigate the efficiency of relevant tensor and tensor network operations and show that the overhead for managing the network structure is negligible, even in cases with a million leaves and small tensors. QuTree focuses on providing simple, high-level routines while retaining easy access to the backend to facilitate novel developments. We demonstrate the capabilities of the package by computing the eigenstates of coupled harmonic oscillator Hamiltonians and performing random circuit simulations on a virtual quantum computer.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2022

ExaTN: Scalable GPU-Accelerated High-Performance Processing of General Tensor Networks at Exascale

We present ExaTN (Exascale Tensor Networks), a scalable GPU-accelerated C++ library which can express and process tensor networks on shared- as well as distributed-memory high-performance computing platforms, including those equipped with GPU accelerators. Specifically, ExaTN provides the ability to build, transform, and numerically evaluate tensor networks with arbitrary graph structures and complexity. It also provides algorithmic primitives for the optimization of tensor factors inside a given tensor network in order to find an extremum of a chosen tensor network functional, which is one of the key numerical procedures in quantum many-body theory and quantum-inspired machine learning. Numerical primitives exposed by ExaTN provide the foundation for composing rather complex tensor network algorithms. We enumerate multiple application domains which can benefit from the capabilities of our library, including condensed matter physics, quantum chemistry, quantum circuit simulations, as well as quantum and classical machine learning, for some of which we provide preliminary demonstrations and performance benchmarks just to emphasize a broad utility of our library.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2016

Maxwell–Dirac stress–energy tensor in terms of Fierz bilinear currents

We analyse the stress–energy tensor for the self-coupled Maxwell–Dirac system in the bilinear current formalism, using two independent approaches. The first method used is that attributed to Belinfante: starting from the spinor form of the action, the well-known canonical stress–energy tensor is augmented, by extending the Noether symmetry current to include contributions from the Lorentz group, to a manifestly symmetric form. This form admits a transcription to bilinear current form. The second method used is the variational derivation based on the covariant coupling to general relativity. The starting point here at the outset is the transcription of the action using, as independent field variables, both the bilinear currents, together with a gauge invariant vector field (a proxy for the electromagnetic vector potential). A central feature of the two constructions is that they both involve the mapping of the Dirac contribution to the stress–energy from the spinor fields to the equivalent set of bilinear tensor currents, through the use of appropriate Fierz identities. Although this mapping is done at quite different stages, nonetheless we find that the two forms of the bilinear stress–energy tensor agree. Finally, as an application, we consider the reduction of the obtained stress–energy tensor in bilinear form, under the assumption of spherical symmetry. -- Highlights: •Maxwell–Dirac stress–energy tensor derived in manifestly gauge invariant bilinear form. •Dirac spinor Belinfante tensor transcribed to bilinear fields via Fierz mapping. •Variational stress–energy obtained via bilinearized action, in contrast to Belinfante case. •Independent derivations via the Belinfante and variational methods agree, as required. •Spherical symmetry reduction given as a working example for wider applications.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2021

Quantum-Based Molecular Dynamics Simulations Using Tensor Cores

Tensor cores, along with tensor processing units, represent a new form of hardware acceleration specifically designed for deep neural network calculations in artificial intelligence applications. Tensor cores provide extraordinary computational speed and energy efficiency but with the caveat that they were designed for tensor contractions (matrix–matrix multiplications) using only low-precision floating-point operations. Despite this perceived limitation, we demonstrate how tensor cores can be applied with high efficiency to the challenging and numerically sensitive problem of quantum-based Born–Oppenheimer molecular dynamics, which requires highly accurate electronic structure optimizations and conservative force evaluations. The interatomic forces are calculated on-the-fly from an electronic structure that is obtained from a generalized deep neural network, where the computational structure naturally takes advantage of the exceptional processing power of the tensor cores and allows for high performance in excess of 100 Tflops on a single Nvidia A100 GPU. Stable molecular dynamics trajectories are generated using the framework of extended Lagrangian Born–Oppenheimer molecular dynamics, which combines computational efficiency with long-term stability, even when using approximate charge relaxations and force evaluations that are limited in accuracy by the numerically noisy conditions caused by the low-precision tensor core floating-point operations. A canonical ensemble simulation scheme is also presented, where the additional numerical noise in the calculated forces is absorbed into a Langevin-like dynamics.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2018

Tensor network method for reversible classical computation

We develop a tensor network technique that can solve universal reversible classical computational problems, formulated as vertex models on a square lattice [Nat. Commun. 8, 15303 (2017)]. By encoding the truth table of each vertex constraint in a tensor, the total number of solutions compatible with partial inputs and outputs at the boundary can be represented as the full contraction of a tensor network. We introduce an iterative compression-decimation (ICD) scheme that performs this contraction efficiently. The ICD algorithm first propagates local constraints to longer ranges via repeated contraction-decomposition sweeps over all lattice bonds, thus achieving compression on a given length scale. It then decimates the lattice via coarse-graining tensor contractions. Repeated iterations of these two steps gradually collapse the tensor network and ultimately yield the exact tensor trace for large systems, without the need for manual control of tensor dimensions. Our protocol allows us to obtain the exact number of solutions for computations where a naive enumeration would take astronomically long times.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
DOE OSTI2019

An Efficient Mixed-Mode Representation of Sparse Tensors

The Compressed Sparse Fiber (CSF) representation for sparse tensors is a generalization of the Compressed Sparse Row (CSR) format for sparse matrices. For a tensor with d modes, typical tensor methods such as CANDECOMP/PARAFAC decomposition (CPD) require a sequence of d tensor computations, where efficient memory access with respect to different modes is required for each of them.The straightforward solution is to use d distinct representations of the tensor, with each one being efficient for one of the d computations. However, a d-fold space overhead is often unacceptable in practice, especially with memory-constrained GPUs. In this paper, we present a mixed-mode tensor representation that partitions the tensor’s nonzero elements into disjoint sections, each of which is compressed to create fibers along a different mode. Experimental results on the latest generation of GPU device demonstrate that better performance can be achieved by using the mixed-mode representation, while utilizing only a small fraction of the space required to keep d distinct CSF representations.

Nisa, Israt↗
DOE OSTI2025

The Poisson tensor completion non-parametric differential entropy estimator

We introduce the Poisson tensor completion (PTC) estimator, a non-parametric differential entropy estimator. The PTC estimator leverages inter-sample relationships to compute a low-rank Poisson tensor decomposition of the frequency histogram. Our crucial observation is that the histogram bins are an instance of a space partitioning of counts and thus can be identified with a spatial Poisson process. The Poisson tensor decomposition leads to a completion of the intensity measure over all bins—including those containing few to no samples—and leads to our proposed PTC differential entropy estimator. A Poisson tensor decomposition models the underlying distribution of the count data and guarantees non-negative estimated values and so can be safely used directly in entropy estimation. Our estimator is the first tensor-based estimator that exploits the underlying spatial Poisson process related to the histogram explicitly when estimating the probability density with low-rank tensor decompositions for the purpose of tensor completion. Furthermore, we demonstrate that our PTC estimator is a substantial improvement over standard histogram-based estimators for sub-Gaussian probability distributions because of the concentration of norm phenomenon.

42 ENGINEERING↗
DOE OSTI2025

A Latent-Variable Formulation of the Poisson Canonical Polyadic Tensor Model: Maximum Likelihood Estimation and Fisher Information

We establish parameter inference for the Poisson canonical polyadic (PCP) tensor model through a latent-variable formulation. Our approach exploits the observation that any random PCP tensor can be derived by marginalizing an unobservable random tensor of one dimension larger. The loglikelihood of this larger dimensional tensor, referred to as the “complete” loglikelihood, is comprised of multiple rank one PCP loglikelihoods. Using this methodology, we first derive maximum likelihood estimators for the PCP model and demonstrate that several existing algorithms for fitting non-negative matrix and tensor factorizations are Expectation-Maximization algorithms. Next, we derive the observed and expected Fisher information matrices for the PCP model. The Fisher information provides us crucial insights into the well-posedness of the tensor model, such as the role that tensor rank plays in identifiability and indeterminacy. For the special case of rank one PCP models, we demonstrate that these results are greatly simplified.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1992

Visualization of second order tensor fields and matrix data

We present a study of the visualization of 3-D second order tensor fields and matrix data. The general problem of visualizing unsymmetric real or complex Hermitian second order tensor fields can be reduced to the simultaneous visualization of a real and symmetric second order tensor field and a real vector field. As opposed to the discrete iconic techniques commonly used in multivariate data visualization, the emphasis is on exploiting the mathematical properties of tensor fields in order to facilitate their visualization and to produce a continuous representation of the data. We focus on interactively sensing and exploring real and symmetric second order tensor data by generalizing the vector notion of streamline to the tensor concept of hyperstreamline. We stress the importance of a structural analysis of the data field analogous to the techniques of vector field topology extraction in order to obtain a unique and objective representation of second order tensor fields.

Delmarcelle, Thierry↗
DOE OSTI2018

A continuous analogue of the tensor-train decomposition

We develop new approximation algorithms and data structures for representing and computing with multivariate functions using the functional tensor-train (FT), a continuous extension of the tensor-train (TT) decomposition. The FT represents functions using a tensor-train ansatz by replacing the three-dimensional TT cores with univariate matrix-valued functions. The main contribution of this paper is a framework to compute the FT that employs adaptive approximations of univariate fibers, and that is not connected to any tensorized discretization. The algorithm can be coupled with any univariate linear or nonlinear approximation procedure. We demonstrate that this approach can generate multivariate function approximations that are several orders of magnitude more accurate, for the same cost, than those based on the conventional approach of compressing the coefficient tensor of a tensor-product basis. Our approach is in the spirit of other continuous computation packages such as Chebfun, and yields an algorithm which requires the computation of “continuous” matrix factorizations such as the LU and QR decompositions of vector-valued functions. To support these developments, we describe continuous versions of an approximate maximum-volume cross approximation algorithm and of a rounding algorithm that re-approximates an FT by one of lower ranks. We demonstrate that our technique improves accuracy and robustness, compared to TT and quantics-TT approaches with fixed parameterizations, of high-dimensional integration, differentiation, and approximation of functions with local features such as discontinuities and other nonlinearities.

42 ENGINEERING↗
DOE OSTI2021

Bayesian tensorized neural networks with automatic rank selection

Tensor decomposition is an effective approach to compress over-parameterized neural networks and to enable their deployment on resource-constrained hardware platforms. However, directly applying tensor compression in the training process is a challenging task due to the difficulty of choosing a proper tensor rank. In order to address this challenge, this paper proposes a low-rank Bayesian tensorized neural network. Our Bayesian method performs automatic model compression via an adaptive tensor rank determination. We also present approaches for posterior density calculation and maximum a posteriori (MAP) estimation for the end-to-end training of our tensorized neural network. Here, we provide experimental validation on a two-layer fully connected neural network, a 6-layer CNN and a 110-layer residual neural network where our work produces 7.4x to 137x more compact neural networks directly from the training while achieving high prediction accuracy.

Low-rank tensor↗