Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “matrix multiplication”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

Mixed-Precision S/DGEMM Using the TF32 and TF64 Frameworks on Low-Precision AI Tensor Cores

Using NVIDIA graphics processing units (GPUs) equipped with Tensor Cores has enabled the significant acceleration of general matrix multiplication (GEMM) for applications in machine learning (ML) and artificial intelligence (AI) and in high-performance computing (HPC) generally. The use of such power-efficient, specialized accelerators can provide a performance increase between 8 × and 20 ×, albeit with a loss in precision. However, a high level of precision is required in many large scientific and HPC applications, and computing in single or double precision is still necessary for many of these applications to maintain accuracy. Fortunately, mixed-precision methods can be employed to maintain a higher level of numerical precision while also taking advantage of the performance increases from computing with lower-precision AI cores. With this in mind, we extend the state of the art by using NVIDIA’s new TF32 framework. This new framework not only burdens some constraints of the previous frameworks, such as costly 32 16-bit castings but also provides an equivalent precision and performance by using a much simpler approach. We also propose a new framework called TF64 that attempts double-precision arithmetic with low-precision Tensor Cores. Although this framework does not exist yet, we validated the correctness of this idea and achieved an equivalent of 64-bit precision on 32-bit hardware.

Valero Lara, Pedro↗
DOE OSTI2024

FTTN: Feature-Targeted Testing for Numerical Properties of NVIDIA & AMD Matrix Accelerators

FTTN is a test suite to evaluate the numerical behaviors of matrix accelerators of GPUs (NVIDIA Tensor Cores and AMD Matrix Cores) in a quick and simple setting. Matrix accelerators are heavily used in today's computationally intense applications to speed up matrix multiplications. This test suite provides a comprehensive study on the numerical behaviors of these accelerators, including support for subnormals, rounding modes, extra precision bits and FMA features. Is there

Laguna Peralta, Ignacio↗
DOE OSTI2024

A Provably Accurate Randomized Sampling Algorithm for Logistic Regression

In statistics and machine learning, logistic regression is a widely-used supervised learning technique primarily employed for binary classification tasks. When the number of observations greatly exceeds the number of predictor variables, we present a simple, randomized sampling-based algorithm for logistic regression problem that guarantees high-quality approximations to both the estimated probabilities and the overall discrepancy of the model. Our analysis builds upon two simple structural conditions that boil down to randomized matrix multiplication, a fundamental and well-understood primitive of randomized numerical linear algebra. We analyze the properties of estimated probabilities of logistic regression when leverage scores are used to sample observations, and prove that accurate approximations can be achieved with a sample whose size is much smaller than the total number of observations. To further validate our theoretical findings, we conduct comprehensive empirical evaluations. Overall, our work sheds light on the potential of using randomized sampling approaches to efficiently approximate the estimated probabilities in logistic regression, offering a practical and computationally efficient solution for large-scale datasets.

Chowdhury, Agniva↗
DOE OSTI2025

Accelerating Floating-Point Computations with Intel AMX

Intel AMX is a built-in component of recent Intel CPU architectures, first supported by the Intel Sapphire Rapids in 2023, that enables efficient dense matrix multiplications using mixed precision with low-precision data types. The popularity of mixed-precision algorithms has grown recently, primarily due to their use on GPUs to enhance the efficiency of HPC applications, particularly for the training of large language models. The availability of mixed precision on CPUs represents a cost-effective solution for applications where high speed is not critical. This report shows how to use the Intel AMX accelerator through examples in C++ and Python. The examples will focus on mixed-precision floating-point operations obtained by the use of bfloat16 (or BF16) to accelerate code in single precision. We employ a bottom-up methodology, starting from specific register instructions (TMUL operation) to higher-level applications in libraries such as Intel MKL, PyTorch, and TensorFlow, ensuring a comprehensive understanding of the accelerator's potential. Additionally, we provide insights into the expected performance gains when leveraging the accelerator on the Kestrel HPC machine at the National Renewable Energy Laboratory.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Energy-Efficient Neuromorphic Architectures for Nuclear Radiation Detection Applications

A comprehensive analysis and simulation of two memristor-based neuromorphic architectures for nuclear radiation detection is presented. Both scalable architectures retrofit a locally competitive algorithm to solve overcomplete sparse approximation problems by harnessing memristor crossbar execution of vector–matrix multiplications. The proposed systems demonstrate excellent accuracy and throughput while consuming minimal energy for radionuclide detection. To ensure that the simulation results of our proposed hardware are realistic, the memristor parameters are chosen from our own fabricated memristor devices. Based on these results, we conclude that memristor-based computing is the preeminent technology for a radiation detection platform.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
NASA NTRS1978

Pattern classification using Charge Transfer Devices

The potential uses of Charge Transfer Devices (CTDs) in pattern classification operations are explored. The needs for a hardware-based pattern classifier are established, and a matrix multiplication subsystem based upon a sum of products CTD is presented. An evaluation process for sum of products devices (particularly analog-analog correlators) is developed, and the feasibility of employing a particular device in a pattern classifier is determined. Finally, the possible impact of future trends in technology is considered.

Snyder, W. E.↗
NASA NTRS1978

The Karhunen-Loeve, discrete cosine, and related transforms obtained via the Hadamard transform

A general class of even/odd transforms is presented that includes the Karhunen-Loeve transform, the discrete cosine transform, the Walsh-Hadamard transform, and other familiar transforms. The more complex even/odd transforms can be computed by combining a simpler even/odd transform with a sparse matrix multiplication. A theoretical performance measure is computed for some even/odd transforms, and two image compression experiments are reported.

Jones, H. W.↗
NASA NTRS1979

Satellite pattern classification using charge transfer devices

The potential uses of Charge Transfer Devices (CTDs) in pattern classification operations are explored. The needs for a hardware-based pattern classifier are established, and a matrix multiplication subsystem based upon a sum-of-products CTD is presented. Applications of the subsystem to the classification of multi-modal Gaussian distributions in general and to LANDSAT data processing in particular are discussed. Finally, the potential impact of this technology on satellite data processing methodologies is discussed.

Snyder, W. E.↗
NASA NTRS1980

Pattern classification using charge transfer devices

The feasibility of using charge transfer devices in the classification of multispectral imagery was investigated by evaluating particular devices to determine their suitability in matrix multiplication subsystem of a pattern classifier and by designing a protype of such a system. Particular attention was given to analog-analog correlator devices which consist of two tapped delay lines, chip multipliers, and a summed output. The design for the classifier and a printed circuit layout for the analog boards were completed and the boards were fabricated. A test j:g for the board was built and checkout was begun.

Source record↗
NASA NTRS1981

Transportation systems evaluation methodology development and applications, phase 3

Transportation systems or proposed changes in current systems are evaluated. Four principal evaluation criteria are incorporated in the process, operating performance characteristics as viewed by potential users, decisions based on the perceived impacts of the system, estimating what is required to reduce the system to practice; and predicting the ability of the concept to attract financial support. A series of matrix multiplications in which the various matrices represent evaluations in a logical sequence of the various discrete steps in a management decision process is used. One or more alternatives are compared with the current situation, and the result provides a numerical rating which determines the desirability of each alternative relative to the norm and to each other. The steps in the decision process are isolated so that contributions of each to the final result are readily analyzed. The ability to protect against bias on the part of the evaluators, and the fact that system parameters which are basically qualitative in nature can be easily included are advantageous.

Kuhlthau, A. R.↗
NASA NTRS1987

Integrated Optical Processor

Images processed rapidly in rugged, compact device. Conceptual integrated optical image processor performs complicated analog calculations, including Fourier transforms, convolutions, correlations, and matrix multiplications. Compact and rugged on account of monolithic structure, processor faster than equivalent all-electronic devices because calculations require only time for light to travel across device; processing speed limited by electronic input and output circuits.

Stephens, William E.↗
NASA NTRS1985

Supercomputing on massively parallel bit-serial architectures

Research on the Goodyear Massively Parallel Processor (MPP) suggests that high-level parallel languages are practical and can be designed with powerful new semantics that allow algorithms to be efficiently mapped to the real machines. For the MPP these semantics include parallel/associative array selection for both dense and sparse matrices, variable precision arithmetic to trade accuracy for speed, micro-pipelined train broadcast, and conditional branching at the processing element (PE) control unit level. The preliminary design of a FORTRAN-like parallel language for the MPP has been completed and is being used to write programs to perform sparse matrix array selection, min/max search, matrix multiplication, Gaussian elimination on single bit arrays and other generic algorithms. A description is given of the MPP design. Features of the system and its operation are illustrated in the form of charts and diagrams.

Iobst, Ken↗
NASA NTRS1987

Array architectures for iterative algorithms

Regular mesh-connected arrays are shown to be isomorphic to a class of so-called regular iterative algorithms. For a wide variety of problems it is shown how to obtain appropriate iterative algorithms and then how to translate these algorithms into arrays in a systematic fashion. Several 'systolic' arrays presented in the literature are shown to be specific cases of the variety of architectures that can be derived by the techniques presented here. These include arrays for Fourier Transform, Matrix Multiplication, and Sorting.

Jagadish, Hosagrahar V.↗
NASA NTRS1987

A hypermatrix formulation for subspace iteration

The computational efficiency of subspace iteration is addressed relative to the data structures adopted for the very large and generally sparse coefficient matrices. The frequent triangulations and matrix multiplications demand that access to the terms in the coefficient matrices be unbiased. Reliance on virtual memory (paging) operating systems with no special considerations for localized data access is not adequate. Specific data structures must be designed that accommodate the needs of the numerical algorithm yet eliminate unnecessary paging. An implementation of the subspace iteration method using hypermatrix data structures is presented. Use of hypermatrices is shown to provide unbiased and localized data access. The various modifications to the conventional formulation are described and an example problem illustrates the potential benefits of the hypermatrix formulation. Possibilities for adapting hypermatrix data structures to new supercomputer architectures are discussed.

Schmidt, Richard J.↗
NASA NTRS1989

A Transient Response Method for Linear Coupled Substructures

A method is presented for determining the transient response of a discrete coordinate model of a linear structural system composed of substructures. The method is applicable to systems consisting of any number of substructures, both determinate and indeterminate interface boundaries, and any topological arrangement of the substructures. The method is simple to implement from a computational point of view because the equations of motion of each of the substructures are solved independently, and the interface boundary compatibility conditions are enforced at each integration time step by a matrix multiplication. The method is demonstrated for a structural system consisting of two beam segments and acted upon by a time dependent force. The numerical results from the demonstration problem validates the accuracy of the method. The application of this method to structural systems with changing interface boundary conditions between substructures is discussed.

Admire, J. R.↗
NASA NTRS1990

Comparisons of one- and two- interface methods for tunnel wall interference calculation

Tunnel wall interference in two-dimensional subsonic flow can be analytically calculated on the basis of the one- or two-interface method. The application of numerical integration by means of matrix multiplication to both methods has been found to significantly reduce CPU time in on-line operation. An investigation of the sensitivity of these methods to random velocity-measurement disturbances shows the one-interface method to be less sensitive to measurement errors, and therefore to be preferable for practical applications.

Lo, C. F.↗
NASA NTRS1991

Fuzzy associative memories

Mappings between fuzzy cubes are discussed. This level of abstraction provides a surprising and fruitful alternative to the propositional and predicate-calculas reasoning techniques used in expert systems. It allows one to reason with sets instead of propositions. Discussed here are fuzzy and neural function estimators, neural vs. fuzzy representation of structured knowledge, fuzzy vector-matrix multiplication, and fuzzy associative memory (FAM) system architecture.

Kosko, Bart↗