Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “matrix-vector”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
NASA NTRS1983

Direct and implicit optical matrix-vector algorithms

New direct and implicit algorithms for optical matrix-vector and systolic array processors are considered. Direct rather than indirect algorithms to solve linear systems and implicit rather than explicit solutions to solve second-order partial differential equations are discussed. In many cases, such approaches more properly utilize the advantageous features of optical systolic array processors. The matrix-decomposition operation (rather than solution of the simplified matrix-vector equation that results) is recognized as the computationally burdensome aspect of such problems that should be computed on an optical system. The Householder QR matrix-decomposition algorithm is considered as a specific example of a direct solution. Extensions to eigenvalue computation and formation of matrices of special structure are also noted.

Casasent, D.↗
DOE OSTI2025

A Study of Performance Portability of Low-bit Fused Matrix-Vector Multiplication Kernels in SYCL

Understanding the causes of performance gaps between a portable programming model and a vendor-specific programming model is important for improving performance portability. This paper studies performance portability of low-bit fused general matrix-vector multiplication kernels in SYCL on vendors’ graphics processing units (GPUs). This work introduces the use case, explains the kernel implementations in detail, evaluates the performance of the CUDA, HIP, and SYCL kernels on datacenter, desktop, and laptop GPUs, and investigates the causes of performance gaps. The results show that loop unrolling, kernel dispatch overhead, and sum reduction contribute to the gaps.

Jin, Zheming [ORNL] (ORCID:000000027197780X)↗
NASA NTRS1986

Twos-complement data processing form improved encoded matrix-vector processors

A new method for handling bipolar data by twos-complement representation is detailed. This technique requires fewer bits, uses simpler optical processor devices (fewer channels), and provides a higher processing rate and throughput. It is directly extendable to more complex matrix operations because of its data flow property and requires only a modest increase in the complexity of the digital support system.

Taylor, B. K.↗
DOE OSTI2026

Quantum Time-Space Tradeoffs for Matrix Problems

We consider the time and space required for quantum computers to solve a wide variety of problems involving matrices, many of which have only been analyzed classically in prior work. Our main results show that for a range of linear algebra problems—including matrix-vector product, matrix inversion, matrix multiplication and powering—existing classical time-space tradeoffs, several of which are tight for every space bound, also apply to quantum algorithms with at most a constant factor loss. For example, for almost all fixed matrices 𝐴, including the discrete Fourier transform matrix, we prove that quantum circuits with at most 𝑇 input queries and 𝑆 qubits of memory require 𝑇 = Ω⁢(𝑛 2 /𝑆) to compute matrix-vector product 𝐴⁢𝑥 for 𝑥 ∈{0,1 𝑛 . We similarly prove that matrix multiplication for 𝑛 ×𝑛 binary matrices requires 𝑇 = Ω⁢(𝑛 3 /$\sqrt{𝑆}$). Because many of our lower bounds are matched by deterministic algorithms with the same time and space complexity, our results show that quantum computers cannot provide any asymptotic advantage for these problems with any space bound. We obtain matching lower bounds for the stronger notion of quantum cumulative memory complexity—the sum of the space per layer of a circuit. We also consider Boolean (i.e., AND-OR) matrix multiplication and matrix-vector products, improving the previous quantum time-space tradeoff lower bounds for 𝑛 × 𝑛 Boolean matrix multiplication to 𝑇 = Ω⁢(𝑛 2.5 /𝑆 1/4 ) from 𝑇 = Ω⁢(𝑛 2.5 /𝑆 1/2 ). Our improved lower bound for Boolean matrix multiplication is based on a new coloring argument that extracts more from the strong direct product theorem that was the basis for prior work. To obtain our tight lower bounds for linear algebra problems, we require much stronger bounds than strong direct product theorems. We obtain these bounds by adding a new bucketing method to the quantum recording-query technique of Zhandry that lets us apply classical arguments to upper bound the success probability of quantum circuits.

lower bounds↗
NASA NTRS1983

LU and Cholesky decomposition on an optical systolic array processor

Direct solutions of matrix-vector equations on an optical systolic array processor are considered. The solutions are discussed and a parallel algorithm for LU matrix decomposition that is very attractive for an optical realization is formulated. It is noted that when direct techniques are used, it is preferable to realize the matrix decomposition on an optical system and to utilize a digital processor for the solution of the simplified resultant matrix-vector problem. One method of realizing LU matrix decomposition on a new frequency-multiplexed optical systolic array matrix-matrix processor is described. A simple method for extending the process of LU decomposition to Cholesky decomposition on the optical processor is discussed.

Casasent, D.↗
NASA NTRS1983

Optical systolic array processor using residue arithmetic

The use of residue arithmetic to increase the accuracy and reduce the dynamic range requirements of optical matrix-vector processors is evaluated. It is determined that matrix-vector operations and iterative algorithms can be performed totally in residue notation. A new parallel residue quantizer circuit is developed which significantly improves the performance of the systolic array feedback processor. Results are presented of a computer simulation of this system used to solve a set of three simultaneous equations.

Jackson, J.↗
NASA NTRS1984

A high-accuracy optical linear algebra processor for finite element applications

Optical linear processors are computationally efficient computers for solving matrix-matrix and matrix-vector oriented problems. Optical system errors limit their dynamic range to 30-40 dB, which limits their accuray to 9-12 bits. Large problems, such as the finite element problem in structural mechanics (with tens or hundreds of thousands of variables) which can exploit the speed of optical processors, require the 32 bit accuracy obtainable from digital machines. To obtain this required 32 bit accuracy with an optical processor, the data can be digitally encoded, thereby reducing the dynamic range requirements of the optical system (i.e., decreasing the effect of optical errors on the data) while providing increased accuracy. This report describes a new digitally encoded optical linear algebra processor architecture for solving finite element and banded matrix-vector problems. A linear static plate bending case study is described which quantities the processor requirements. Multiplication by digital convolution is explained, and the digitally encoded optical processor architecture is advanced.

Casasent, D.↗
DOE OSTI2024

A Performance and Energy Study of GPU-Resident Preconditioners for Conjugate Gradient Solvers: In the Context of Existing and Novel Approaches

Optimizing a particular subprogram out of the set of Basic (sparse) Linear Algebra Subprograms (BLAS) for a given architecture is a common topic of research. In applications, however, these BLAS functions rarely appear in isolation; usually, many of them are used together, in various combinations and with varying inputs. As the need to solve a large, sparse linear system is ubiquitous throughout HPC applications, linear solvers constitute a realistic, sufficiently complex and well-defined representative use case for composite BLAS routines. To this end, based on a representative set of matrices drawn from a diverse set of fields, we present a framework to study, from the performance and energy perspective, the efficacy of GPU- resident parallel Conjugate Gradient (CG) linear solver with different preconditioner options, including Gauss-Seidel, Jacobi, and incomplete Cholesky. We also propose a novel GPU-based preconditioner, in which the triangular solves are approximated by an iterative process. The development of this preconditioner was motivated by solving large graph Laplacian linear systems, for which the existing preconditioners either perform slow on GPU-based platforms or are not applicable. We compare the performance of these preconditioners on different hardware accelerator architectures, i.e., AMD MI250X, MI100, Nvidia A100, V100, and Jetson. Our experiments reveal performance trade-offs and provide information on how to select the best strategy for the given linear system, dictated by its properties, and the platform of interest. We demonstrate the application of our novel preconditioner for solving CG and graph Laplacian systems. Overall, the framework can be utilized as a benchmark to guide informed decisions in choosing a specific preconditioner, i.e., whether it is better to rely on the performance of a triangular solver or on the performance of sparse matrix-vector product. Finally, by considering power consumption to solve the linear systems, we report the energy footprint for the solvers.

Preconditioned Conjugate Gradient, GPUs, iterative↗
DOE OSTI2025

Analog In-Memory Computing for the Synthetic Aperture Radar Polar Format Algorithm

As the utility of synthetic aperture radar (SAR) systems increases in autonomous vehicles, satellites, and other power- and space-constrained edge applications, there is a growing need for processors that can form SAR images at low power. In recent years, analog in-memory compute (AIMC) has shown immense promise for accelerating neural networks and other matrix-vector multiplication (MVM) heavy workloads at the edge. Here, in this work, we examine how the polar format algorithm (PFA), a popular SAR image formation algorithm, can be mapped to these AIMC systems. The PFA maps readily onto analog MVMs because it primarily consists of two linear operations: interpolation of frequency-domain data to a Cartesian grid, followed by a 2-D Fourier transform. This work presents two approaches to map the interpolation operation onto MVMs in analog hardware: a chirp transform and a modified form of sinc interpolation. These mappings introduce algorithmic errors, and their effect on the quality of SAR image formation is examined, both quantitatively and qualitatively. In addition, the impact of errors introduced by the analog hardware is explored to determine which approach is optimal under varying assumptions about the underlying analog memory devices and circuits.

Analog computing↗
DOE OSTI2025

Fast and Scalable FFT-Based GPU-Accelerated Algorithms for Block-Triangular Toeplitz Matrices with Application to Linear Inverse Problems Governed by Autonomous Dynamical Systems

In this work, we present an efficient and scalable algorithm for performing matrix-vector multiplications (matvecs) for block Toeplitz matrices. Such matrices, which are shift-invariant with respect to their blocks, arise in the context of solving inverse problems governed by autonomous systems, and time-invariant systems in particular. In this article, we consider inverse problems that infer unknown parameters from observational data of a linear time-invariant dynamical system given in the form of partial differential equations (PDEs). Matrix-free Newton-conjugate-gradient methods are often the gold standard for solving these inverse problems, but they require numerous actions of the Hessian on a vector. Matrix-free adjoint-based Hessian matvecs require solution of a pair of linearized forward/adjoint PDE solves per Hessian action, which may be prohibitive for large-scale inverse problems. Time invariance of the forward PDE problem leads to a block Toeplitz structure of the discretized parameter-to-observable (p2o) map defining the mapping from inputs (parameters) to outputs (observables) of the PDEs. This block Toeplitz structure enables us to exploit two key properties: (1) compact storage of the p2o map and its adjoint, and (2) efficient fast Fourier transform–based Hessian matvecs. The proposed algorithm is mapped onto large multi-GPU clusters and achieves more than 80% of peak bandwidth on NVIDIA A100 GPUs. Excellent weak scaling is shown for up to 48 A100 GPUs. For the targeted problems, the implementation executes Hessian matvecs within fractions of a second, which is orders of magnitude faster than can be achieved by conventional matrix-free Hessian matvecs via forward/adjoint PDE solves.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

A Low-Rank QTT-based Finite Element Method for Elasticity Problems

We present an efficient and robust numerical algorithm for solving the linear elasticity problem that combines the Quantized Tensor Train format and a domain partitioning strategy. This approach makes it possible to solve the linear elasticity problem on a computational domain that is more general than a square. By integrating Z-ordering and subdomain concatenation, our method substantially decreases memory usage and achieves a notable reduction in rank compared to established Finite Element implementations like the FEniCS platform. This efficiency is maintained while still guaranteeing exponential convergence with respect to the number of degrees of freedom. This performance gain, however, requires a fundamental rethinking of how core finite element operations are implemented. This includes changes to mesh discretization, node and degree of freedom ordering, stiffness matrix and internal nodal force assembly, and the execution of algebraic matrix-vector operations. In this work, we discuss all these aspects in detail and assess the method’s performance in the numerical approximation of three representative test cases.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1983

Preconditioners for the spectral multigrid method

The systems of algebraic equations which arise from spectral discretizations of elliptic equations are full and direct solutions of them are rarely feasible. Iterative methods are an attractive alternative because Fourier transform techniques enable the discrete matrix-vector products to be computed with nearly the same efficiency as is possible for corresponding but sparse finite difference discretizations. For realistic Dirichlet problems preconditioning is essential for acceptable convergence rates. A brief description of Chebyshev spectral approximations and spectral multigrid methods for elliptic problems is given. A survey of preconditioners for Dirichlet problems based on second-order finite difference methods is made. New preconditioning techniques based on higher order finite differences and on the spectral matrix itself are presented. The preconditioners are analyzed in terms of their spectra and numerical examples are presented.

Phillips, T. N.↗
NASA NTRS1983

Triangular system solutions on an optical systolic processor

It is noted that the simplified system of equations presented by Casasent and Ghosh (1983) can also be solved optically. One realization of this is described, with attention given to data flow and speed for the case of a lower triangular system of equations. The solution of an upper triangular system of equations can be handled in analogous fashion. The problem requires a matrix-vector processor. Only one carrier frequency input to the acoustooptic cell and only one output detector are used. To multiply a matrix by a vector in this system, the time-multiplexed vector elements are fed into the acoustooptic cell.

Ghosh, A.↗
NASA NTRS1983

Advanced acousto-optic signal processors

The basic acousto-optic signal processing architectures (spectrum analyzer, space-integrating, time-integrating, and triple product processor) systems and algorithms such as the chirp-Z transform are reviewed. New acousto-optic data processing systems and applications that utilze these basic architectures and new ones are described. These include a matched spatial filter acousto-optic processor, two new hybrid time and space-integrating systems, a triple product processor, and four new matrix-vector iterative feedback systems.

Casasent, D.↗
NASA NTRS1984

Acoustooptic linear algebra processors - Architectures, algorithms, and applications

Architectures, algorithms, and applications for systolic processors are described with attention to the realization of parallel algorithms on various optical systolic array processors. Systolic processors for matrices with special structure and matrices of general structure, and the realization of matrix-vector, matrix-matrix, and triple-matrix products and such architectures are described. Parallel algorithms for direct and indirect solutions to systems of linear algebraic equations and their implementation on optical systolic processors are detailed with attention to the pipelining and flow of data and operations. Parallel algorithms and their optical realization for LU and QR matrix decomposition are specifically detailed. These represent the fundamental operations necessary in the implementation of least squares, eigenvalue, and SVD solutions. Specific applications (e.g., the solution of partial differential equations, adaptive noise cancellation, and optimal control) are described to typify the use of matrix processors in modern advanced signal processing.

Casasent, D.↗
NASA NTRS1987

A generalized procedure for constructing an upwind based TVD scheme

A generalized formulation for constructing second- and higher-order accurate TVD (total variation diminishing) schemes is presented. A given scheme is made TVD by limiting antidiffusive flux differences with some linear functions, so-called limiters. The general idea of the formulation and its mathematical proof of Harten's TVD conditions is shown by applying the Lax-Wendroff method to scalar nonlinear equations and a constant-coefficient system of conservation laws. For the system of equations, several definitions are derived for the argument used in the limiter function and present their performance in numerical experiments. The formulation is extended to the nonlinear system. It is demonstrated that the present procedure can easily convert existing central or upwind, and second- or higher-order differencing schemes to preserve monotonicity and yield physically admissible solutions. The formulation is simple mathematically as well as numerically; both matrix-vector multiplication and Riemann solver are avoided. Although the notion of TVD is based on the initial value problem, application to the steady Euler equations of the formulation is also made.

Liou, Meng-Sing↗
NASA NTRS1986

Preconditioners for the spectral multigrid method

The systems of algebraic equations which arise from spectral discretizations of elliptic equations are full and direct solutions of them are rarely feasible. Iterative methods are an attractive alternative because Fourier transform techniques enable the discrete matrix-vector products to be computed with nearly the same efficiency as is possible for corresponding but sparse finite difference discretizations. For realistic Dirichlet problem preconditioning is essential for acceptable convergence rates. A brief description of Chebyshev spectral approximations and spectral multigrid methods for elliptic problems is given. A survey of preconditioners for Dirichlet problems based on second-order finite difference methods is made. New preconditioning techniques based on higher order finite differences and on the spectral matrix itself are presented. The preconditioners are analyzed in terms of their spectra and numerical examples are presented.

Phillips, T. N.↗
NASA NTRS1987

A generalized procedure for constructing an upwind-based TVD scheme

A generalized formulation for constructing second- and higher-order accurate TVD (total variation diminishing) schemes is presented. A given scheme is made TVD by limiting antidiffusive flux differences with some nonlinear functions, so-called limiters. The general idea of the formulation and its mathematical proof of Harten's TVD conditions is shown by applying the Lax-Wendroff method to a scalar nonlinear equation and constant-coefficient system of conservation laws. For the system of equations, several definitions are derived for the argument used in the limiter function and present their performance to numerical experiments. Then the formulation is formally extended to the nonlinear system of equations. It is demonstrated that use of the present procedure allows easy conversion of existing central or upwind, and second- or higher-order differencing schemes so as to preserve monotonicity and to yield physically admissible solutions. The formulation is simple mathematically as well as numerically; neither matrix-vector multiplication nor Riemann solver is required. Roughly twice as much computational effort is needed as compared to conventional scheme. Although the notion of TVD is based on the initial value problem, application to the steady Euler equations of the formulation is also made. Numerical examples including various ranges of problems show both time- and spatial-accuracy in comparison with exact solutions.

Liou, Meng-Sing↗