Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Tensor Contraction Expressions”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

Performance Optimization of Tensor Contraction Expressions for Many Body Methods in Quantum Chemistry

Complex tensor contraction expressions arise in accurate electronic structure models in quantum chemistry, such as the coupled cluster method. This paper addresses two complementary aspects of performance optimization of such tensor contraction expressions. Transformations using algebraic properties of commutativity and associativity can be used to significantly decrease the number of arithmetic operations required for evaluation of these expressions. The identification of common subexpressions among a set of tensor contraction expressions can result in a reduction of the total number of operations required to evaluate the tensor contractions. The first part of the paper describes an effective algorithm for operation minimization with common subexpression identification and demonstrates its effectiveness on tensor contraction expressions for coupled cluster equations. The second part of the paper highlights the importance of data layout transformation in the optimization of tensor contraction computations on modern processors. A number of considerations such as minimization of cache misses and utilization of multimedia vector instructions are discussed. A library for efficient index permutation of multi-dimensional tensors is described and experimental performance data is provided that demonstrates its effectiveness.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

Performance Optimization of Tensor Contraction Expressions for Many Body Methods in Quantum Chemistry

Complex tensor contraction expressions arise in accurate electronic structure models in quantum chemistry, such as the coupled cluster method. This paper addresses two complementary aspects of performance optimization of such tensor contraction expressions. Transformations using algebraic properties of commutativity and associativity can be used to significantly decrease the number of arithmetic operations required for evaluation of these expressions. The identification of common subexpressions among a set of tensor contraction expressions can result in a reduction of the total number of operations required to evaluate the tensor contractions. The first part of the paper describes an effective algorithm for operation minimization with common subexpression identification and demonstrates its effectiveness on tensor contraction expressions for coupled cluster equations. The second part of the paper highlights the importance of data layout transformation in the optimization of tensor contraction computations on modern processors. A number of considerations, such as minimization of cache misses and utilization of multimedia vector instructions, are discussed. A library for efficient index permutation of multidimensional tensors is described, and experimental performance data is provided that demonstrates its effectiveness.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

A Framework for Load Balancing of Tensor Contraction Expressions via Dynamic Task Partitioning

In this paper, we introduce the Dynamic Load-balanced Tensor Contractions (DLTC), a domain-specific library for efficient task parallel execution of tensor contraction expressions, a class of computation encountered in quantum chemistry and physics. Our framework decomposes each contraction into smaller unit of tasks, represented by an abstraction referred to as iterators. We exploit an extra level of parallelism by having tasks across independent contractions executed concurrently through a dynamic load balancing run- time. We demonstrate the improved performance, scalability, and flexibility for the computation of tensor contraction expressions on parallel computers using examples from coupled cluster methods.

Tensor contraction, domain-specific language, dyna↗

Optimizing Tensor Contraction Expressions for Hybrid CPU-GPU Execution

Tensor contractions are generalized multidimensional matrix multiplication operations that widely occur in quantum chemistry. Efficient execution of tensor contractions on Graphics Processing Units (GPUs) requires several challenges to be addressed, including index permutation and small dimension-sizes reducing thread block utilization. Moreover, to apply the same optimizations to various expressions, we need a code generation tool. In this paper, we present our approach to automatically generate CUDA code to execute tensor contractions on GPUs, including management of data movement between CPU and GPU. To evaluate our tool, GPU-enabled code is generated for the most expensive contractions in CCSD(T), a key coupled cluster method, and incorporated into NWChem, a popular computational chemistry suite. For this method, we demonstrate speedup over a factor of 8.4 using one GPU (instead of one core per node) and over 2.6 when utilizing the entire system using hybrid CPU+GPU solution with 2 GPUs and 5 cores (instead of 7 cores per node). Finally, we analyze the implementation behavior on future GPU systems.

CUDA↗

Acceleration of Streamed Tensor Contraction Expressions on GPGPU-based Clusters

Tensor contractions are generalized multidimensional matrix multiplication operations that widely occur in quantum chemistry. Efficient execution of tensor contractions on GPUs requires tackling several challenges to be addressed, including index permutation and small dimension-sizes reducing thread block utilization. In this paper, we present our approach to automatically generate CUDA code to execute tensor contractions on GPUs, including management of data movement between CPU and GPU. GPU-enabled code is generated for the most expensive contractions in CCSD(T) and incorporated into NWChem, a popular computational chemistry suite. We demonstrate speedup over a factor of 8.4 using one core per node and over 2.6 when utilizing the entire system using hybrid CPU+GPU solution with 2 GPUs and 5 cores. We finally analyze the behavior of the application on future GPU systems.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Practical Loop Transformations for Tensor Contraction Expressions on Multi-Level Memory Hierarchies

Modern architectures are characterized by deeper levels of memory hierarchy, often explicitly addressable. Optimizing applications for such architectures requires careful management of the data movement across all these levels. In this paper, we focus on the problem of mapping tensor contractions to memory hierarchies with more than two levels, specifically addressing placement of memory allocation and data movement statements, choice of loop fusions, and tile size selection. Existing algorithms to find an integrated solution to this problem even for two-level memory hierarchies has been shown to be expensive. We improve upon this work by focusing on the first-order cost components, simplifying the analysis required and reducing the number of candidates to be evaluated. We have evaluated our framework on a cluster of GPUs. Using five tensor contraction expressions, we show that fusion at multiple levels improves performance, and our framework is effective in determining protable transformations.

99 GENERAL AND MISCELLANEOUS↗

Empirical Performance Model-Driven Data Layout Optimization and Library Call Selection for Tensor Contraction Expressions

Empirical optimizers like ATLAS have been very effective in optimizing computational kernels in libraries. The best choice of parameters such as tile size and degree of loop unrolling is determined by executing different versions of the computation. In contrast, optimizing compilers use a model-driven approach to program transformation. While the model-driven approach of optimizing compilers is generally orders of magnitude faster than ATLAS-like library generators, its effectiveness can be limited by the accuracy of the performance models used. In this paper, we describe an approach where a class of computations is modeled in terms of constituent operations that are empirically measured, thereby allowing modeling of the overall execution time. The performance model with empirically determined cost components is used to perform data layout optimization together with the selection of library calls and layout transformations in the context of the Tensor Contraction Engine, a compiler for a high-level domain-specific language for expressing computational models in quantum chemistry. The effectiveness of the approach is demonstrated through experimental measurements on representative computations from quantum chemistry.

99 GENERAL AND MISCELLANEOUS↗

Parameterized Micro-benchmarking: An Auto-tuning Approach for Complex Applications

Auto-tuning has emerged as an important practical method for creating highly optimized implementations of key computational kernels and applications. However, the growing complexity of architectures and applications is creating new challenges for auto-tuning. Complex applications can involve a prohibitively large search space that precludes empirical auto-tuning. Similarly, architectures are becoming increasingly complicated, making it hard to model performance. In this paper, we focus on the challenge to auto-tuning presented by applications with a large number of kernels and kernel instantiations. While these kernels may share a somewhat similar pattern, they differ considerably in problem sizes and the exact computation performed. We propose and evaluate a new approach to auto-tuning which we refer to as parameterized micro-benchmarking. It is an alternative to the two existing classes of approaches to auto-tuning: analytical model-based and empirical search-based. Particularly, we argue that the former may not be able to capture all the architectural features that impact performance, whereas the latter might be too expensive for an application that has several different kernels. In our approach, different expressions in the application, different possible implementations of each expression, and the key architectural features, are used to derive a simple micro-benchmark and a small parameter space. This allows us to learn the most significant features of the architecture that can impact the choice of implementation for each kernel. We have evaluated our approach in the context of GPU implementations of tensor contraction expressions encountered in excited state calculations in quantum chemistry. We have focused on two aspects of GPUs that affect tensor contraction execution: memory access patterns and kernel consolidation. Using our parameterized micro-benchmarking approach, we obtain a speedup of up to 2 over the version that used default optimizations, but no auto-tuning. We demonstrate that observations made from microbenchmarks match the behavior seen from real expressions. In the process, we make important observations about the memory hierarchy of two of the most recent NVIDIA GPUs, which can be used in other optimization frameworks as well.

auto-tuning↗

A Brief Summary on Formalizing Parallel Tensor Distributions, Redistributions, and Algorithm Derivations

Large-scale datasets in computational chemistry typically require distributed-memory parallel methods to perform a special operation known as tensor contraction. Tensors are multidimensional arrays, and a tensor contraction is akin to matrix multiplication with special types of permutations. Creating an efficient algorithm and optimized implementation in this domain is complex, tedious, and error-prone. To address this, we develop a notation to express data distributions so that we can apply use automated methods to find optimized implementations for tensor contractions. We consider the spin-adapted coupled cluster singles and doubles method from computational chemistry and use our methodology to produce an efficient implementation. Experiments per- formed on the IBM Blue Gene/Q and Cray XC30 demonstrate impact both improved performance and reduced memory consumption.

97 MATHEMATICS AND COMPUTING↗

torch-einshard v1.0

torch-einshard is a Python library for describing local and distributed PyTorch tensor computations with compact, einsum-like notation. Its expressions name logical axes, specify how they are sharded across a PyTorch DeviceMesh, and represent partial reductions. The library automatically performs contractions, permutations, reshaping, splitting, gathering, reduction, reduce-scatter, and repartitioning while preserving autograd. Additional features include sharding-aware FFTs, tensor rolls, halo exchange, sliding windows, 1D–3D convolutions, uneven-shard handling, parameter initialization and gradient management, and cost-based execution planning. It is designed for scientific machine learning and large-model workloads, including tensor-, sequence-, and spatial-parallel MLPs, attention, convolutions, and spectral operations. Compared with manually combining torch.einsum and distributed collectives, torch-einshard expresses both the mathematical operation and data placement in one readable formula. This reduces boilerplate and synchronization errors, keeps forward and backward communication consistent, and allows the library to select optimized collective strategies without changing model code.

Morozov, Dmitriy [Lawrence Berkeley National Labor↗

Efficiently evaluating loop integrals in the EFTofLSS using QFT integrals with massive propagators

We develop a new way to analytically calculate loop integrals in the Effective Field Theory of Large Scale-Structure. Previous available methods show severe limitations beyond the one-loop power spectrum due to analytical challenges and computational and memory costs. Our new method is based on fitting the linear power spectrum with cosmology-independent functions that resemble integer powers of quantum field theory massive propagators with complex masses. A remarkably small number of them is sufficient to reach enough accuracy. Similarly to former approaches, the cosmology dependence is encoded in the coordinate vector of the expansion of the linear power spectrum in our basis. We first produce cosmology-independent tensors where each entry is the loop integral evaluated on a given combination of basis vectors. For each cosmology, the evaluation of a loop integral amounts to contracting this tensor with the coordinate vector of the linear power spectrum. The 3-dimensional loop integrals for our basis functions can be evaluated using techniques familiar to particle physics, such as recursion relations and Feynman parametrization. We apply our formalism to evaluate the one-loop bispectrum of galaxies in redshift space. The final analytical expressions are quite simple and can be evaluated with little computational and memory cost. We show that the same expressions resolve the integration of all one-loop N-point function in the EFTofLSS. This method, which is originally presented here, has already been applied in the first one-loop bispectrum analysis of the BOSS data to constraint ΛCDM parameters and primordial non-Gaussianities [1, 2].

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Semi-inclusive charged-current neutrino-nucleus reactions

The general, universal formalism for semi-inclusive charged-current (anti)neutrino-nucleus reactions is given for studies of any hadronic system, namely, either nuclei or the nucleon itself. The detailed developments are presented with the former in mind and are further specialized to cases where the final-state charged lepton and an ejected nucleon are presumed to be detected. General kinematics for such processes are summarized and then explicit expressions are developed for the leptonic and hadronic tensors involved and for the corresponding responses according to the usual charge, longitudinal and transverse projections, keeping finite the masses of all particles involved. In the case of the hadronic responses, general symmetry principles are invoked to determine which contributions can occur. As a result, the general leptonic-hadronic tensor contraction is given as well as the cross section for the process.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Relativistic distorted-wave analysis of quasielastic proton-nucleus scattering

A relativistic distorted-wave impulse approximation formalism is presented for the calculation of quasielastic proton-nucleus scattering. It is shown that the double differential cross section may be written as a contraction between the hadronic tensor (describing the projectile and ejectile) and the polarization tensor (describing the nuclear target) and that this mathematical structure also holds for the case where distortions are included. The eikonal approximation is used to introduce distortions in the wave functions, and the nuclear response is described using a Fermi gas model. The highly oscillatory nine-dimensional integrand contained in the expression for the double differential cross section is computed using a novel technique based on combining traditional Gaussian integration methods with the powerful fitting functions in the matlab programming language. This work has successfully calculated the distorted-wave quasielastic differential cross section for proton-nucleus scattering within a fully relativistic framework. It is found that the distortions lead to a reduction in the double differential cross section and have a negligible effect on the computed spin observables.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Pairwise connected tensor network representation of path integrals

It has been recently shown how the tensorial nature of real-time path integrals (PIs) involving the Feynman-Vernon influence functional can be utilized with matrix product states, taking advantage of the finite length of the bath-induced memory. Tensor networks (TNs) promise to provide a unified language to express the structure of a PI. A generalized TN specifically incorporating the pairwise interaction structure of the influence functional and its invariance with respect to the average forward-backward position or the sojourn value in the form of the blip representation is derived and implemented. This pairwise connected TNPI (PC-TNPI) is illustrated through applications to typical spin-boson problems and explorations of the differences caused by the exact form of the spectral density. The storage and performance scalings are reported, showing the compactness of the representation and the efficiency of the contraction process. Finally, taking advantage of the compressed representation, the viability of using PC-TNPI for simulating multistate problems is demonstrated. The PC-TNPI structure can be shown to yield other TN algorithms currently in use. Consequently, it should be possible to use it as a starting point for deriving other optimized procedures.

36 MATERIALS SCIENCE↗

Observational signatures of a non-singular bouncing cosmology

We study a cosmological scenario in which inflation is preceded by a bounce. In this scenario, the primordial singularity, one of the major shortcomings of inflation, is replaced by a non-singular bounce, prior to which the universe undergoes a phase of contraction. Our starting point is the bouncing cosmology investigated in Falciano et al. (2008), which we complete by a detailed study of the transfer of cosmological perturbations through the bounce and a discussion of possible observational effects of bouncing cosmologies. We focus on a symmetric bounce and compute the evolution of cosmological perturbations during the contracting, bouncing and inflationary phases. We derive an expression for the Mukhanov-Sasaki perturbation variable at the onset of the inflationary phase that follows the bounce. Rather than being in the Bunch-Davies vacuum, it is found to be in an excited state that depends on the time scale of the bounce. We then show that this induces oscillations superimposed on the nearly scale-invariant primordial spectra for scalar and tensor perturbations. We discuss the effects of these oscillations in the cosmic microwave background and in the matter power spectrum. We propose a new way to indirectly measure the spatial curvature energy density parameter Ω{sub K} in the context of this model.

79 ASTRONOMY AND ASTROPHYSICS↗

ExaTENSOR

ExaTENSOR is a software math library for performing basic numerical tensor algebra operations on distributed heterogeneous HPC platforms. The library provides the following basic numerical tensor algebra primitives: tensor contraction, tensor product, tensor addition, tensor scaling by a scalar, etc. ExaTENSOR supports dense, block-sparse and hierarchical block-sparse tensors stored and processed across many heterogeneous HPC nodes (node equipped with NVIDIA GPU and Intel Xeon Phi have been considered so far). The architecture of ExaTENSOR is based on the concept of domain-specific virtual processor (DSVP), that is, an intermediate software layer capable of processing domain-specific instructions, in this case, numerical tensor algebra instructions. In this way, a separation of the domain-specific algorithm expression and hardware-agnostic algorithm execution is achieved, thus ensuring portability of the applications which use ExaTENSOR as a numerical backend.

Liakh, Dmytro↗

NWChem: scalable parallel computational chemistry

NWChem is a general purpose computational chemistry code specifically designed to run on distributed memory parallel computers. The core functionality of the code focuses on molecular dynamics, Hartree-Fock and density functional theory methods for both plane-wave basis sets as well as Gaussian basis sets, tensor contraction engine based coupled cluster capabilities and combined quantum mechanics/molecular mechanics descriptions. It was realized from the beginning that scalable implementations of these methods required a programming paradigm inherently different from what message passing approaches could offer. In response a global address space library, the Global Array Toolkit, was developed. The programming model it offers is based on using predominantly one-sided communication. This model underpins most of the functionality in NWChem and the power of it is exemplified by the fact that the code scales to tens of thousands of processors. In this paper the core capabilities of NWChem are described as well as their implementation to achieve an efficient computational chemistry code with high parallel scalability. NWChem is a modern, open source, computational chemistry code1 specifically designed for large scale parallel applications2. To meet the challenges of developing efficient, scalable and portable programs of this nature a particular code design was adopted. This code design involved two main features. First of all, the code is build up in a modular fashion so that a large variety of functionality can be integrated easily. Secondly, to facilitate writing complex parallel algorithms the Global Array toolkit was developed. This toolkit allows one to write parallel applications in a shared memory like approach, but offers additional mechanisms to exploit data locality to lower communication overheads. This framework has proven to be very successful in computational chemistry but is applicable to any engineering domain. Within the context created by the features above NWChem has grown into a general purpose computational chemistry code that supports a wide variety of energy expressions and capabilities to calculate properties based there upon. The main energy expressions are classical mechanics force fields, Hartree-Fock and DFT both for finite systems and condensed phase systems, coupled cluster, as well as QM/MM. For most energy expressions single point calculations, geometry optimizations, excited states, and other properties are available. Below we briefly discuss each of the main energy expressions and the critical points involved in scalable implementations thereof.

99 GENERAL AND MISCELLANEOUS↗