Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Sparse Coding”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

symPACK: A GPU-Capable Fan-Out Sparse Cholesky Solver

Sparse symmetric positive definite systems of equations are ubiquitous in scientific workloads and applications. Parallel sparse Cholesky factorization is the method of choice for solving such linear systems. Therefore, the development of parallel sparse Cholesky codes that can efficiently run on today’s large-scale heterogeneous distributed-memory platforms is of vital importance. Modern supercomputers offer nodes that contain a mix of CPUs and GPUs. To fully utilize the computing power of these nodes, scientific codes must be adapted to offload expensive computations to GPUs. We present symPACK, a GPU-capable parallel sparse Cholesky solver that uses one-sided communication primitives and remote procedure calls provided by the UPC++ library. We also utilize the UPC++ "memory kinds" feature to enable efficient communication of GPU-resident data. We show that on a number of large problems, symPACK outperforms comparable state-of-the-art GPU-capable Cholesky factorization codes by up to 14x on the NERSC Perlmutter supercomputer.

Bellavita, Julian↗
DOE OSTI2024

Diversity of visual inputs to Kenyon cells of the Drosophila mushroom body

The arthropod mushroom body is well-studied as an expansion layer representing olfactory stimuli and linking them to contingent events. However, 8% of mushroom body Kenyon cells in Drosophila melanogaster receive predominantly visual input, and their function remains unclear. Here, we identify inputs to visual Kenyon cells using the FlyWire adult whole-brain connectome. Input repertoires are similar across hemispheres and connectomes with certain inputs highly overrepresented. Many visual neurons presynaptic to Kenyon cells have large receptive fields, while interneuron inputs receive spatially restricted signals that may be tuned to specific visual features. Individual visual Kenyon cells randomly sample sparse inputs from combinations of visual channels, including multiple optic lobe neuropils. These connectivity patterns suggest that visual coding in the mushroom body, like olfactory coding, is sparse, distributed, and combinatorial. However, the specific input repertoire to the smaller population of visual Kenyon cells suggests a constrained encoding of visual stimuli.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2022

Performance of an Astrophysical Radiation Hydrodynamics Code under Scalable Vector Extension Optimization

We present results of a performance study of an astrophysical radiation hydrodynamics code, V2D, on the Arm-based A64FX processor developed by Fujitsu. The code solves sparse linear systems, a task for which the A64FX architecture should be well suited. Here, we performed the performance analysis study on Ookami, an Apollo 80 platform utilizing the A64FX processor. We explored several compilers and performance anal-ysis packages and found the code did not perform as expected under scalable vector extension optimization, suggesting that a “deeper dive” into analyzing the code is worthwhile. However, a simple driver program that exercised basic sparse linear algebra routines used by V2D did show significant speedup with the use of the scalable vector extension optimization. We present the initial results from the study which used V2D on a relatively simple test problem that emphasized the repeated solution of sparse linear systems.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2018

On the Information in Spike Timing: Neural Codes Derived from Polychronous Groups

There is growing evidence regarding the importance of spike timing in neural information processing, with even a small number of spikes carrying information, but computational models lag significantly behind those for rate coding. Experimental evidence on neuronal behavior is consistent with the dynamical and state dependent behavior provided by recurrent connections. This motivates the minimalistic abstraction investigated in this paper, aimed at providing insight into information encoding in spike timing via recurrent connections. We employ information-theoretic techniques for a simple reservoir model which encodes input spatiotemporal patterns into a sparse neural code, translating the polychronous groups introduced by Izhike-vich into codewords on which we can perform standard vector operations. Finally, we show that the distance properties of the code are similar to those for (optimal) random codes. In particular, the code meets benchmarks associated with both linear classification and capacity, with the latter scaling exponentially with reservoir size.

computational modeling↗
DOE OSTI2016

The Improved Simple Corner Balance Method and Efforts to Enhance its Computational Performance

The diamond difference (DD) spatial discretization has long been a work horse of the neutron transport community. For nearly as long as DD has been used in neutron transport, it has been observed that DD produces undamped, non-physical angular fluxes in spatially under resolved problems and can fail in the thick diffusion limit. A long list of alternative spatial discretizations, all with better physics properties: accuracy, robustness, and/or preservation of the thick diffusion limit, have been proposed as alternatives to DD. Unfortunately, these alternative spatial discretizations also require significantly more floating point operations (FLOPs) per unknown compared to DD. Historically, neutron transport problems of interest have been able to be resolved sufficiently that DD is accurate enough and FLOPs have been the computational performance bottleneck controlling time to solution. Thus, in practice DD has been and remains a preferred spatial discretization for neutron transport simulations. Trends in computing architecture are leading to a new paradigm where data movement, not FLOPs will control time to solution. This presents an opportunity to improve neutron transport physics by reconsidering more FLOP intensive spatial discretization than DD without incurring a time to solution penalty. In this work, we consider the improved simple corner balance (ISCB) spatial discretization, a unique alternative to DD that has the same memory footprint as DD, and detail efforts to improve its computational performance. The ISCB spatial discretization is a more FLOP intensive spatial discretization than DD with the same memory footprint as DD. Though designed to maximize FLOP intensity, third party libraries such as LAPACK are not tuned for the unique, frequent, small linear algebra needs of neutron transport problems. While hand coding a sparse Gauss elimination is likely to be faster than a dense matrix solve (CM vs. HGE), were LAPACK suitable for small linear algebra problems, we would not have seen a performance increase with decreased numbers of LAPACK calls for the Schur complement solve (SC4 vs. SC1). Significant effort has reduced the relative compute time of ISCB to DD, but on current computer architectures DD is still faster than ISCB. Future work comparing the overall efficiency of ISCB to DD may reveal that while ISCB is slower than DD on current computers, ISCB is more accurate than DD by a factor greater than the compute time slowdown. As new computer architectures become available, we will test our hypothesis that neutron transport simulations will be bandwidth limited, ideally rendering the additional FLOPs required by ISCB to be 'free' [in terms of time to solution]. (authors)

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2021

SBIR Phase I Final Report, TACO: Distributed and Heterogeneous Sparse Compiler

Tensor algebra is a powerful tool for computing, but writing optimized codes that operate on sparse tensors can be very complex. This project enables a Tensor Algebra Compiler (TACO) that simplifies this task from man-years to man-days and extends TACO to support complex and large distributed systems. This report details the hypotheses, approaches used, and findings in this project.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2019

Adaptive Sparse Grid Discretization (ASGarD)

This code solves the Vlasov-Poisson equations in 6-dimensions without approximation. It uses sparse grid discretization to solve the high-dimensional PDEs. The code functions as a discontinuous Galerkin finite element solver. Advantages/benefits include no approximations, supports high-dimensionality, leadership-scale capability, and implicit time advance.

Hahn, Steven [Oak Ridge National Laboratory (ORNL)↗
DOE OSTI2024

Milestone 49 Report: Batched Sparse LA Phase 5 Implementation

Batched sparse linear algebra operations in general, and solvers in particular, have become the major algorithmic development activity and foremost performance engineering effort in the numerical software libraries work on modern hardware with accelerators such as GPUs. Many applications, ECP and non-ECP alike, require simultaneous solutions of many small linear systems of equations that are structurally sparse in one form or another. In order to move towards high hardware utilization levels, it is important to provide these applications with appropriate interface designs to be both functionally efficient and performance portable and give full access to the appropriate batched sparse solvers running on modern hardware accelerators prevalent across DOE supercomputing sites since the inception of ECP. To this end, we present here a summary of recent advances on the interface designs in use by HPC software libraries supporting batched sparse linear algebra and the development of sparse batched kernel codes for solvers and preconditioners. We also address the potential interoperability opportunities to keep the corresponding software portable between the major hardware accelerators from AMD, Intel, and NVIDIA, while maintaining the appropriate disclosure levels conforming to the active NDA agreements. The presented interface specifications include a mix of batched band, sparse iterative, and sparse direct solvers with their accompanying functionality that is already required by the application codes or we anticipated to be needed in the near future. This report summarizes progress in Kokkos Kernels and the xSDK libraries MAGMA, Ginkgo, hypre, PETSc, and SuperLU.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Batched Sparse Linear Algebra (Final Report for Subcontract B648960)

This report finalizes design specifications for developing batched kernels for small tensor operations for unassembled matrix-free iterative solvers, batched solvers for partially assembled operators, and batched solvers with support for various sparse formats. The outcome of the project milestones is a set of interfaces to Batched Sparse LA solvers running on hardware accelerators for use in ECP Libraries and Applications. It is part of the development of sparse batched kernels, solvers/preconditioners as well as creating interoperability in xSDK libraries with sparse and dense batched functions to benefit ECP applications. The participants included representatives from ECP libraries (not limited to the xSDK project), applications, and vendors (AMD, Intel, and NVIDIA). Batched sparse linear algebra solvers form the new frontier for algorithmic development and performance engineering. Many applications (ECP and non-ECP alike) require simultaneous solutions of small linear systems of equations that are structurally sparse. To move towards high hardware utilization, it is important to provide these applications with appropriate interfaces to efficient batched sparse solvers running on modern hardware accelerators. We present interface designs in use by HPC software libraries supporting batched sparse linear algebra and the development of sparse batched kernel codes for solvers and preconditioners. We also address the potential interoperability opportunities to keep the software portable between the major hardware accelerators from AMD, Intel, and NVIDIA. The presented interface specifications includes batched band, sparse iterative, and sparse direct solvers. This report summarizes progress in Kokkos Kernels and the xSDK libraries MAGMA, Ginkgo, hypre, SUNDIALS, and SuperLU_dist.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

A graphics processing unit accelerated sparse direct solver and preconditioner with block low rank compression

We present the GPU implementation efforts and challenges of the sparse solver package STRUMPACK. The code is made publicly available on github with a permissive BSD license. STRUMPACK implements an approximate multifrontal solver, a sparse LU factorization which makes use of compression methods to accelerate time to solution and reduce memory usage. Multiple compression schemes based on rank-structured and hierarchical matrix approximations are supported, including hierarchically semi-separable, hierarchically off-diagonal butterfly, and block low rank. Here, in this paper, we present the GPU implementation of the block low rank (BLR) compression method within a multifrontal solver. Our GPU implementation relies on highly optimized vendor libraries such as cuBLAS and cuSOLVER for NVIDIA GPUs, rocBLAS and rocSOLVER for AMD GPUs and the Intel oneAPI Math Kernel Library (oneMKL) for Intel GPUs. Additionally, we rely on external open source libraries such as SLATE (Software for Linear Algebra Targeting Exascale), MAGMA (Matrix Algebra on GPU and Multi-core Architectures), and KBLAS (KAUST BLAS). SLATE is used as a GPU-capable ScaLAPACK replacement. From MAGMA we use variable sized batched dense linear algebra operations such as GEMM, TRSM and LU with partial pivoting. KBLAS provides efficient (batched) low rank matrix compression for NVIDIA GPUs using an adaptive randomized sampling scheme. The resulting sparse solver and preconditioner runs on NVIDIA, AMD and Intel GPUs. Interfaces are available from PETSc, Trilinos and MFEM, or the solver can be used directly in user code. We report results for a range of benchmark applications, using the Perlmutter system from NERSC, Frontier from ORNL, and Aurora from ALCF. For a high frequency wave equation on a regular mesh, using 32 Perlmutter compute nodes, the factorization phase of the exact GPU solver is about 6.5× faster compared to the CPU-only solver. The BLR-enabled GPU solver is about 13.8× faster than the CPU exact solver. For a collection of SuiteSparse matrices, the STRUMPACK exact factorization on a single GPU is on average 1.9× faster than NVIDIA’s cuDSS solver.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Epistatic Net allows the sparse spectral regularization of deep neural networks for inferring fitness functions

Abstract Despite recent advances in high-throughput combinatorial mutagenesis assays, the number of labeled sequences available to predict molecular functions has remained small for the vastness of the sequence space combined with the ruggedness of many fitness functions. While deep neural networks (DNNs) can capture high-order epistatic interactions among the mutational sites, they tend to overfit to the small number of labeled sequences available for training. Here, we developed Epistatic Net (EN), a method for spectral regularization of DNNs that exploits evidence that epistatic interactions in many fitness functions are sparse. We built a scalable extension of EN, usable for larger sequences, which enables spectral regularization using fast sparse recovery algorithms informed by coding theory. Results on several biological landscapes show that EN consistently improves the prediction accuracy of DNNs and enables them to outperform competing models which assume other priors. EN estimates the higher-order epistatic interactions of DNNs trained on massive sequence spaces-a computational problem that otherwise takes years to solve.

Aghazadeh, Amirali (ORCID:0000000302230873)↗
DOE OSTI2019

Geometric Mapping of Tasks to Processors on Parallel Computers with Mesh or Torus Networks

We present a new method for reducing parallel applications' communication time by mapping their MPI tasks to processors in a way that lowers the distance messages travel and the amount of congestion in the network. Assuming geometric proximity among the tasks is a good approximation of their communication interdependence, we use a geometric partitioning algorithm to order both the tasks and the processors, assigning task parts to the corresponding processor parts. In this way, interdependent tasks are assigned to “nearby” cores in the network. We also present a number of algorithmic optimizations that exploit specific features of the network or application to further improve the quality of the mapping. We specifically address the case of sparse node allocation, where the nodes assigned to a job are not necessarily located in a contiguous block nor within close proximity to each other in the network. Furthermore, our methods generalize to contiguous allocations as well, and results are shown for both contiguous and non-contiguous allocations. We show that, for the structured finite difference mini-application MiniGhost, our mapping methods reduced communication time up to 75% relative to MiniGhost's default mapping on 128K cores of a Cray XK7 with sparse allocation. For the atmospheric modeling code E3SM/HOMME, our methods reduced communication time up to 31% on 16K cores of an IBM BlueGene/Q with contiguous allocation.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2018

Exploiting Geometric Partitioning in Task Mapping for Parallel Computes

We present a new method for mapping applications' MPI tasks to cores of a parallel computer such that applications' communication time is reduced. We address the case of sparse node allocation, where the nodes assigned to a job are not necessarily located in a contiguous block nor within close proximity to each other in the network, although our methods generalize to contiguous allocations as well. The goal is to assign tasks to cores so that interdependent tasks are performed by "nearby' cores, thus lowering the distance messages must travel, the amount of congestion in the network, and the overall cost of communication. Our new method applies a geometric partitioning algorithm to both the tasks and the processors, and assigns task parts to the corresponding processor parts. We also present a number of algorithmic optimizations that exploit specific features of the network or application. We show that, for the structured finite difference mini-application MiniGhost, our mapping methods reduced communication time up to 75% relative to MiniGhost's default mapping on 128K cores of a Cray XK7 with sparse allocation. For the atmospheric modeling code E3SM/HOMME, our methods reduced communication time up to 31% on 32K cores of an IBM BlueGene/Q with contiguous allocation.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Joint Channel Equalization and Symbol Detection for IoT Devices in Severe Multipath Channels

Internet of Things (IoT) devices often transmit very short message blocks without strong error correction coding or pilots for channel equalization. When the transmitted signals encounter a severe multipath channel, the receiver is often unable to equalize the resulting inter-symbol interference (ISI) via traditional methods, leading to a high retransmission rate. This paper proposes a joint channel equalization and symbol detection scheme for this scenario by utilizing the preamble sequence as a training pilot for equalization and the relatively weak cyclic redundancy check (CRC) 8-Dallas code for error correction. By assuming a sparse channel impulse response, the proposed joint channel equalization and symbol detection scheme formulates an l1 norm constrained optimization problem and solves it by a saddle-point algorithm. The proposed algorithm is applied to a set of real-world underwater animal tracking IoT data, and the results show improvement of correct message detection rate from 21:5% to 41:6%.

Jing, Shusen↗
DOE OSTI2018

Compact video content representation for video coding using low multi-linear tensor rank approximation with dynamic core tensor order

Tensor is a multi-way representation of an entity that is widely used in many applications of mathematics and physics. The wide range of tensor algebra operations attract the researchers of graphical and vision field to identify unique methods for various applications such as segmentation, compression, indexing and so on. In this paper, low multi-linear rank approximation of a tensor is used for decomposition so as to perform compression on a video that is represented as a 4D tensor. Thus, the encoder has blocks for (1) multilinear rank approximation method so as to aid decomposition, (2) sparsity removal of residual data to reduce the required memory storage to a small level, (3) quantization on the sparse less residual data and (4) LZ77 dictionary coding for the video. The decoder uses a tensor reconstruction block along with residual error correction block that reduces loss/error in the reconstructed video. The size of core tensor is an important factor in rank approximation. It is identified at run time depending on the video contents. This size is determined adaptively using Tikhonov’s regularization method. The best value of core tensor size is identified with the corner of L-curve which preserves the video contents from heavy loss/error. Experimental results of the proposed work on comparison with H.264/AVC compression method indicate that the proposed method achieves up to 10% improvement in compression ratio with the least amount of loss and a maximum signal to noise ratio of 0.14. The proposed system gives better compression ratio than H.265 for videos with more number of frames without compromising quality.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Spray: Sparse Reductions of Arrays in OPEN MP

We present SPRAY, an open-source header-only C++ library for sparse reductions of arrays. SPRAY is meant for applications in which a large array is collaboratively updated by multiple threads using an associative and commutative operation such as +=. Especially when each thread accesses only parts of the array, SPRAY can perform significantly better than OPENMP's built-in reduction clause or atomic updates, while also using less memory than the former. SPRAY provides both an easy-to-use interface that can serve as a drop-in replacement for OPENMP reductions and a selection of reducer objects that accumulate the final result in different thread-safe ways. We demonstrate SPRAY through multiple test cases including the LULESH shock hydrodynamics code and a transpose-matrix-vector multiplication for sparse matrices stored in CSR format. SPRAY reductions outperform built-in OPENMP reductions consistently, in some cases improving run time and memory overhead by 20X, and even beating domain-specific approaches such as Intel MKI, by over 2X in some cases. Furthermore, SPRAY reductions have a minimal impact on the code base, requiring only a few lines of source code changes. Once in place, SPRAY reduction schemes can be switched easily, allowing performance portability and tuning opportunities by separating performance-critical implementation details from application code.

OpenMP↗