Thread parallelism in Trilinos' sparse linear algebra interfaces & linear solvers.
Abstract not provided.
Hoemmen, Mark Frederick↗
SEARCH · Engineering Papers
Results for “linear algebra”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Bringing randomized algorithms to mainstream numerical linear algebra
Numerical linear algebra (NLA) underpins huge swaths of computational science and engineering. For scientists and engineers to make the most of the DOE’s computing resources, it is essential that they have access to high-performance implementations of algorithms with best-in-class scalability and reliability. Despite this, prevailing NLA libraries have little to no support for breakthrough algorithms from the field of randomized numerical linear algebra (RandNLA) that have been developed over the past twenty years. The goal of this LDRD was to break a log-jam that had prevented broad adoption of RandNLA. Our work had two thrusts. The first was to develop RandBLAS: a trustworthy and high-performance C++ library for randomized dimension reduction (an operation widely known as sketching). The second was the development of a novel randomized algorithm for computing a challenging type of matrix decomposition known as Householder QR with column pivoting (Householder QRCP). In this one-year late-start LDRD we successfully delivered RandBLAS 1.0 and new CPU and GPU codes for Householder QRCP. RandBLAS has extensive documentation at https://randblas.readthedocs.io/en/stable/. Papers on RandBLAS and and our high-performance QRCP codes are forthcoming.
97 MATHEMATICS AND COMPUTING↗
HALA: Handy Accelerated Linear Algebra
Accelerated linear algebra libraries often come with C-style of interfaces or even older Fortran77 standards, which makes it difficult to use within generic template programming environment of modern C++. For example, the BLAS and derived standards such as cuBLAS do not accept overloads and a different functions have to be called for each input type, thus it is very challenging to write a single template that handles multiple precision modes and works well with different C++ containers. HALA offers a series of templates that wrap around the BLAS/cuBLAS and other similar methods, automatically infer the relevant types and call the appropriate back-end. The templates work out-of-the-box with all C++ standard vector-like containers and can be easily extended (with template specializations) to handle user-provided containers. The HALA API is also overloaded to handle both GPU and CPU cases with a single "engine" class, thus a single high-level algorithms can utilize both CPU or GPU backends, i.e., the HALA API is unified and handles the variations between cuBLAS and BLAS. In addition to BLAS and cuBLAS, wrapper templates are provided for subset of LAPACK, cuSparse, and Cholmod methods. Work is in progress to complete as many of these standards as possible. HALA also includes a module for low-level extended register vectorization for complex number arithmetic and several simple solvers that build on the basic linear algebra capabilities. A single unified C++ API will benefit fast prototyping and easily writing code that is portable across GPU/CPU platforms as well as precision types and user containers. Note that HALA does not provide any accelerated algorithms, it simply serves as a "handy" front-end to other standards.
Stoyanov, MiroslavK.↗
Linear-algebraic bath transformation for simulating complex open quantum systems
In studying open quantum systems, the environment is often approximated as a collection of non-interacting harmonic oscillators, a configuration also known as the star-bath model. It is also well known that the star-bath can be transformed into a nearest-neighbor interacting chain of oscillators. The chain-bath model has been widely used in renormalization group approaches. The transformation can be obtained by recursion relations or orthogonal polynomials. Based on a simple linear algebraic approach, we propose a bath partition strategy to reduce the system-bath coupling strength. As a result, the non-interacting star-bath is transformed into a set of weakly coupled multiple parallel chains. Furthermore, the transformed bath model allows complex problems to be practically implemented on quantum simulators, and it can also be employed in various numerical simulations of open quantum dynamics.
71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
Kokkos/Qthreads Task Parallel Approach to Linear Algebra Based Graph Analytics.
Abstract not provided.
Wolf, Michael↗
Scaling Up Data-Parallel Analytics Platforms: Linear Algebraic Operation Cases
Linear algebraic operations such as matrix manipulations form the kernel of many machine learning and other crucial algorithms. Scaling up as well as scaling out such algorithms are key to supporting large scale data analysis that require efficient processing over millions of data samples. To this end, we present, ARION, a hardware acceleration based approach for scaling-up individual tasks of Spark, a popular data-parallel analytics platform. We support both linear algebraic operations of between two dense matrices, and between sparse and dense matrices in distributed environments. ARION provides a flexible control of acceleration according to matrix density, along with efficient scheduling based on runtime resource utilization. We demonstrate the benefit of our approach for general matrix multiplication operations over large matrices with up to four billion elements by using Gramian matrix computation that is commonly used in machine learning. Experiments show that our approach achieves more than 2× and 1.5× end-to-end performance speedups for dense and sparse matrices, respectively, and up to 57.04× faster computation compared to MLlib, a state of the art Spark-based implementation. This work is sponsored in part by the NSF under the grants: CNS-1565314, CNS-1405697, and CNS-1615411. The manuscript has been authored by UT-Battelle, LLC under Contract No. DE-AC05-00OR22725 with the U.S. Department of Energy. The United States Government retains and the publisher, by accepting the article for publication, acknowledges that the United States Government retains a non-exclusive, paid-up, irrevocable, world-wide license to publish or reproduce the published form of this manuscript, or allow others to do so, for United States Government purposes. The Department of Energy will provide public access to these results of federally sponsored research in accordance with the DOE Public Access Plan (http://energy.gov/downloads/doe-public-access-plan). This research used resources of the Oak Ridge Leadership Computing Facility at the Oak Ridge National Laboratory, which is supported by the Office of Science of the U.S. Department of Energy under Contract No. DE-AC05-00OR22725.
Xu, Luna↗
A Task-Based Linear Algebra Building Blocks Approach for Scalable Graph Analytics.
Abstract not provided.
Wolf, Michael↗
Conjugacy Analysis of the Finite-Difference Equations of the Method of Characteristics on the Basis of a Linear-Algebraic Formulation
A linear-algebraic form of the equations of the method of characteristics, which is used to approximate the neutron transport equation, is obtained. It is shown on the basis of the obtained linear-algebraic form that the discrete form of the conjugate equation differs from the algebraically discrete problem constructed by linear-algebraic transformations of the discrete form of the normal problem. The reason for the discrepancy lies in the approximation of the volumes of the spatial cells in covering the working region by a network of characteristics. It is shown by means of test calculations that when the network of characteristics is refined the solution of the conjugate transport equation converges to the solution of the algebraically conjugate problem.
97 MATHEMATICS AND COMPUTING↗
Evolution of the SLATE linear algebra library
SLATE (Software for Linear Algebra Targeting Exascale) is a distributed, dense linear algebra library targeting both CPU-only and GPU-accelerated systems, developed over the course of the Exascale Computing Project (ECP). While it began with several documents setting out its initial design, significant design changes occurred throughout its development. In some cases, these were anticipated: an early version used a simple consistency flag that was later replaced with a full-featured consistency protocol. In other cases, performance limitations and software and hardware changes prompted a redesign. Sequential communication tasks were parallelized; host-to-host MPI calls were replaced with GPU device-to-device MPI calls; more advanced algorithms such as Communication Avoiding LU and the Random Butterfly Transform (RBT) were introduced. Early choices that turned out to be cumbersome, error prone, or inflexible have been replaced with simpler, more intuitive, or more flexible designs. Applications have been a driving force, prompting a lighter weight queue class, nonuniform tile sizes, and more flexible MPI process grids. Of paramount importance has been building a portable library that works across several different GPU architectures – AMD, Intel, and NVIDIA – while keeping a clean and maintainable codebase. Here we explore the evolving design choices and their effects, both in terms of performance and software sustainability.
Gates, Mark↗
Enabling Efficient Sparse Computations using Linear Algebra Aware Compilers
This project developed the LAPIS compiler framework, built on the Multilevel Intermediate Representation (MLIR), to optimize sparse linear algebra operations and support performance portability across diverse architectures. The main innovation of LAPIS is the Kokkos dialect, which allows for lowering codes from a high productivity language to different architectures in an elegant way. The dialect also allows the conversion of lower-level MLIR code to C++ Kokkos code, facilitating the integration of scientific machine learning (SciML) models into applications. To extend LAPIS for distributed memory architectures, a new partition dialect was created to manage the distribution of sparse tensors and express communication patterns for sparse linear algebra operations. This dialect also supports the distributed execution of operators and includes algorithmic optimizations to minimize communication to improve performance. The project also demonstrates that MLIR can enable effective linear algebra-level optimizations, improving performance on different GPUs for both sparse and dense linear algebra kernels. Key applications of LAPIS include sparse linear algebra and graph kernels, TenSQL, a relational database management solution built on GraphBLAS, and the development of subgraph isomorphism and monomorphism kernels, showcasing performance portability. In summary, the LAPIS framework supports productivity, performance, portability, and distributed memory execution, while also enabling linear algebra-level optimizations that are challenging in traditional programming languages, with successful applications ranging from simple sparse linear algebra to complex graph kernels.
97 MATHEMATICS AND COMPUTING↗
A survey of numerical linear algebra methods utilizing mixed-precision arithmetic
The efficient utilization of mixed-precision numerical linear algebra algorithms can offer attractive acceleration to scientific computing applications. Especially with the hardware integration of low-precision special-function units designed for machine learning applications, the traditional numerical algorithms community urgently needs to reconsider the floating point formats used in the distinct operations to efficiently leverage the available compute power. In this study, we provide a comprehensive survey of mixed-precision numerical linear algebra routines, including the underlying concepts, theoretical background, and experimental results for both dense and sparse linear algebra problems.
97 MATHEMATICS AND COMPUTING↗
LAPIS: Linear Algebra Performance for Intermediate Subprograms
SAND2025-11594O LAPIS (Linear Algebra Performance for Intermediate Subprograms) is a compiler infrastructure for linear algebra that targets both high productivity and performance portability. It is based on the open-source MLIR package from the LLVM project. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525.
Kelley, Brian↗
Constant depth, near constant depth, and subcubic size threshold circuits for linear algebraic calculations
A method of increasing an efficiency at which a plurality of threshold gates arranged as neuromorphic hardware is able to perform a linear algebraic calculation having a dominant size of N. The computer-implemented method includes using the plurality of threshold gates to perform the linear algebraic calculation in a manner that is simultaneously efficient and at a near constant depth. “Efficient” is defined as a calculation algorithm that uses fewer of the plurality of threshold gates than a naïve algorithm. The naïve algorithm is a straightforward algorithm for solving the linear algebraic calculation. “Constant depth” is defined as an algorithm that has an execution time that is independent of a size of an input to the linear algebraic calculation. The near constant depth comprises a computing depth equal to or between O(log(log(N)) and the constant depth.
97 MATHEMATICS AND COMPUTING↗
Quantum many-body linear algebra, Hamiltonian moments, and a coupled-cluster inspired framework
Here, we propose a general strategy to develop quantum many-body approximations of primitives in linear algebra algorithms. As a practical example, we introduce a coupled-cluster inspired framework to produce approximate Hamiltonian moments and demonstrate its application in various linear algebra algorithms for ground state estimation. Through numerical examples, we illustrate the difference between the ground-state energies arising from quantum many-body linear algebra and those from the analogous many-body perturbation theory. Our results support the general idea of designing quantum many-body approximations outside of perturbation theory, providing a route to new algorithms and approximations.
Algorithms and data structure↗
Heterogenous Acceleration for Linear Algebra in Multi-coprocessor Environments
We present an efficient and scalable programming model for the development of linear algebra in heterogeneous multi-coprocessor environments. The model incorporates some of the current best design and implementation practices for the heterogeneous acceleration of dense linear algebra (DLA). Examples are given as the basis for solving linear systems' algorithms - the LU, QR, and Cholesky factorizations. To generate the extreme level of parallelism needed for the efficient use of coprocessors, algorithms of interest are redesigned and then split into well-chosen computational tasks. The tasks execution is scheduled over the computational components of a hybrid system of multi-core CPUs and coprocessors using a light-weight runtime system. The use of lightweight runtime systems keeps scheduling overhead low, while enabling the expression of parallelism through otherwise sequential code. This simplifies the development efforts and allows the exploration of the unique strengths of the various hardware components.
Luszczek, Piotr R↗
Replicated Computational Results (RCR) Report for “Adaptive Precision Block-Jacobi for High Performance Preconditioning in the Ginkgo Linear Algebra Software”
The article by Flegar et al. titled “Adaptive Precision Block-Jacobi for High Performance Preconditioning in the Ginkgo Linear Algebra Software” presents a novel, practical implementation of an adaptive precision block-Jacobi preconditioner. Performance results using state-of-the-art GPU architectures for the block-Jacobi preconditioner generation and application demonstrate the practical usability of the method, compared to a traditional full-precision block-Jacobi preconditioner. A production-ready implementation is provided in the Ginkgo numerical linear algebra library. In this report, the Ginkgo library is reinstalled and performance results are generated to perform a comparison to the original results when using Ginkgo’s Conjugate Gradient solver with either the full or the adaptive precision block-Jacobi preconditioner for a suite of test problems on an NVIDIA GPU accelerator. After completing this process, the published results are deemed reproducible.
97 MATHEMATICS AND COMPUTING↗
FatMan vs. LittleBoy: Scaling Up Linear Algebraic Operations in Scale-Out Data Platforms
Linear algebraic operations such as matrix manipulations form the kernel of many machine learning and other crucial algorithms. Scaling up as well as scaling out such algorithms are highly desirable to enable efficient processing over millions of data points. To this end, we present a matrix manipulation approach to effectively scale-up each node in a scale-out data parallel platform such as Apache Spark. Specifically, we enable hardware acceleration for matrix multiplications in a distributed Spark setup without user intervention. Our approach supports both dense and sparse distributed matrices, and provides flexible control of acceleration by matrix density. We demonstrate the benefit of our approach for generalized matrix multiplication operations over large matrices with up to four billion elements. To connect the effectiveness of our approach with machine learning applications, we performed Gramian matrix computation via generalized matrix multiplications. Our experiments show that our approach achieves more than 2× performance speed-up, and up to 96.1% computation improvement, compared to a state of the art Spark MLlib for dense matrices.
97 MATHEMATICS AND COMPUTING↗
FatMan vs. LittleBoy: Scaling up Linear Algebraic Operations in Scale-out Data Platforms
Linear algebraic operations such as matrix manipulations form the kernel of many machine learning and other crucial algorithms. Scaling up as well as scaling out such algorithms are highly desirable to enable efficient processing over millions of data points. To this end, we present a matrix manipulation approach to effectively scale-up each node in a scale-out data parallel platform such as Apache Spark. Specifically, we enable hardware acceleration for matrix multiplications in a distributed Spark setup without user intervention. Our approach supports both dense and sparse distributed matrices, and provides flexible control of acceleration by matrix density. We demonstrate the benefit of our approach for generalized matrix multiplication operations over large matrices with up to four billion elements. To connect the effectiveness of our approach with machine learning applications, we performed Gramian matrix computation via generalized matrix multiplications. Our experiments show that our approach achieves more than 2x performance speed-up, and up to 96.1% computation improvement, compared to a state of the art Spark MLlib for dense matrices.
Lim, Seung-Hwan↗