Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “structured sparsity”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Online Voltage Event Detection Using Synchrophasor Data with Structured Sparsity-Inducing Norms

This paper develops an accurate and computationally efficient data-driven framework to detect voltage events from PMU data streams. It develops an innovative Proximal Bilateral Random Projection (PBRP) algorithm to quickly decompose the PMU data matrix into a low-rank matrix, a row-sparse event-pattern matrix and a noise matrix. Here, the row-sparse pattern matrix significantly distinguishes events from normal behavior. These matrices are then fed into a clustering algorithm to separate voltage events from normal operating conditions. Large-scale numerical study results on real-world PMU data show that the proposed algorithm is computationally more efficient and achieves higher F scores than state-of-the-art benchmarks.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2024

Spike-and-Slab Shrinkage Priors for Structurally Sparse Bayesian Neural Networks

Network complexity and computational efficiency have become increasingly significant aspects of deep learning. Sparse deep learning addresses these challenges by recovering a sparse representation of the underlying target function by reducing heavily overparameterized deep neural networks. Specifically, deep neural architectures compressed via structured sparsity (e.g., node sparsity) provide low-latency inference, higher data throughput, and reduced energy consumption. In this article, we explore two well-established shrinkage techniques, Lasso and Horseshoe, for model compression in Bayesian neural networks (BNNs). To this end, we propose structurally sparse BNNs, which systematically prune excessive nodes with the following: 1) spike-and-slab group Lasso (SS-GL) and 2) SS group Horseshoe (SS-GHS) priors, and develop computationally tractable variational inference, including continuous relaxation of Bernoulli variables. We establish the contraction rates of the variational posterior of our proposed models as a function of the network topology, layerwise node cardinalities, and bounds on the network weights. Furthermore, we empirically demonstrate the competitive performance of our models compared with the baseline models in prediction accuracy, model compression, and inference latency.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1990

Highly parallel sparse Cholesky factorization

Several fine grained parallel algorithms were developed and compared to compute the Cholesky factorization of a sparse matrix. The experimental implementations are on the Connection Machine, a distributed memory SIMD machine whose programming model conceptually supplies one processor per data element. In contrast to special purpose algorithms in which the matrix structure conforms to the connection structure of the machine, the focus is on matrices with arbitrary sparsity structure. The most promising algorithm is one whose inner loop performs several dense factorizations simultaneously on a 2-D grid of processors. Virtually any massively parallel dense factorization algorithm can be used as the key subroutine. The sparse code attains execution rates comparable to those of the dense subroutine. Although at present architectural limitations prevent the dense factorization from realizing its potential efficiency, it is concluded that a regular data parallel architecture can be used efficiently to solve arbitrarily structured sparse problems. A performance model is also presented and it is used to analyze the algorithms.

Gilbert, John R.↗
NASA NTRS1998

Algorithms for Efficient Computation of Transfer Functions for Large Order Flexible Systems

An efficient and robust computational scheme is given for the calculation of the frequency response function of a large order, flexible system implemented with a linear, time invariant control system. Advantage is taken of the highly structured sparsity of the system matrix of the plant based on a model of the structure using normal mode coordinates. The computational time per frequency point of the new computational scheme is a linear function of system size, a significant improvement over traditional, still-matrix techniques whose computational times per frequency point range from quadratic to cubic functions of system size. This permits the practical frequency domain analysis of systems of much larger order than by traditional, full-matrix techniques. Formulations are given for both open- and closed-loop systems. Numerical examples are presented showing the advantages of the present formulation over traditional approaches, both in speed and in accuracy. Using a model with 703 structural modes, the present method was up to two orders of magnitude faster than a traditional method. The present method generally showed good to excellent accuracy throughout the range of test frequencies, while traditional methods gave adequate accuracy for lower frequencies, but generally deteriorated in performance at higher frequencies with worst case errors being many orders of magnitude times the correct values.

Maghami, Peiman G.↗
NASA NTRS1997

Efficient Computation of Closed-loop Frequency Response for Large Order Flexible Systems

An efficient and robust computational scheme is given for the calculation of the frequency response function of a large order, flexible system implemented with a linear, time invariant control system. Advantage is taken of the highly structured sparsity of the system matrix of the plant based on a model of the structure using normal mode coordinates. The computational time per frequency point of the new computational scheme is a linear function of system size, a significant improvement over traditional, full-matrix techniques whose computational times per frequency point range from quadratic to cubic functions of system size. This permits the practical frequency domain analysis of systems of much larger order than by traditional, full-matrix techniques. Formulations are given for both open and closed loop loop systems. Numerical examples are presented showing the advantages of the present formulation over traditional approaches, both in speed and in accuracy. Using a model with 703 structural modes, a speed-up of almost two orders of magnitude was observed while accuracy improved by up to 5 decimal places.

Maghami, Peiman G.↗
DOE OSTI2025

Dissipative ground state preparation in ab initio electronic structure theory

Dissipative engineering is a powerful tool for quantum state preparation, and has drawn significant attention in quantum algorithms and quantum many-body physics in recent years. In this work, we introduce a novel approach using the Lindblad dynamics to efficiently prepare the ground state for general ab initio electronic structure problems on quantum computers, without variational parameters. These problems often involve Hamiltonians that lack geometric locality or sparsity structures, which we address by proposing two generic types of jump operators for the Lindblad dynamics. Type-I jump operators break the particle number symmetry and should be simulated in the Fock space. Type-II jump operators preserves the particle number symmetry and can be simulated more efficiently in the full configuration interaction space. For both types of jump operators, we prove that in a simplified Hartree-Fock framework, the spectral gap of our Lindbladian is lower bounded by a universal constant. For physical observables such as energy and reduced density matrices, the convergence rate of our Lindblad dynamics with Type-I jump operators remains universal, while the convergence rate with Type-II jump operators only depends on coarse grained information such as the number of orbitals and the number of electrons. To validate our approach, we employ a Monte Carlo trajectory-based algorithm for simulating the Lindblad dynamics for full ab initio Hamiltonians, demonstrating its effectiveness on molecular systems amenable to exact wavefunction treatment.

Quantum chemistry↗
DOE OSTI2026

Semi-Analytical Hierarchical Bayesian Inference of Nonlinear Model Structure in Stochastic Dynamics: Applied to Compartmental Models of Infectious Diseases

A Bayesian computational framework for parsimonious inference in stochastic nonlinear dynamical systems is presented. This framework enables the concurrent estimation of system states, time-varying parameters, time-invariant parameters, and the optimal sparsity structure of the model parameters. Because differential equation-based models are often simplified mechanistic or phenomenological representations, robust inference from noisy measurement data requires explicit treatment of model error and uncertainty. Model error and time-varying parameters can be represented as random processes, enabling inference while making minimal assumptions about the underlying sources of discrepancy and variability. Adopting stochastic differential equation representations affords the model significant flexibility, but can also render it susceptible to overfitting during statistical inversion, where the inferred model may track noise rather than the underlying signal. To alleviate the effects of overfitting and to enable the discovery of the optimal sparse representation of the time-invariant parameters, a Bayesian sparse learning algorithm is embedded within the framework. This sparse learning framework adopts an approximate hierarchical Bayesian setting defined by a series of semi-analytical expressions. The model structure inference framework is validated using a stochastic compartmental model for tracking and forecasting active cases of an infectious disease. Compartmental models describe population-level infectious disease dynamics through interactions among population fractions grouped by disease state. Mathematically, such models consist of a system of coupled ordinary differential equations. This example adopts an expressive compartmental model that includes multiple possible interactions between disease states, motivated by early uncertainty surrounding COVID-19 reinfection dynamics and their implications for long-term epidemic forecasting. The sparse learning exercise permits the inference of a priori unknown epidemiological dynamics from simulated public health data, discovering the nested compartmental model that optimizes the trade-off between average data-fit and model complexity. It is shown that inducing sparsity among the model parameters eliminates redundant interactions between compartments, equivalently revealing the optimal coupling structure between differential equations.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Jacobian sparsity detection using Bloom filters

Determining Jacobian sparsity structure is an important step in the efficient computation of sparse Jacobians. We introduce a new method for determining Jacobian sparsity patterns by combining bit vector probing with Bloom filters. In conclusion, we further refine Bloom filter probing by combining it with hierarchical probing to yield a highly effective strategy for Jacobian sparsity pattern determination.

Bloom filter↗
DOE OSTI2020

Multifrontal Non-negative Matrix Factorization

Non-negative matrix factorization (Nmf) is an important tool in high-performance large scale data analytics with applications ranging from community detection, recommender system, feature detection and linear and non-linear unmixing. While traditional Nmf works well when the data set is relatively dense, however, it may not extract sufficient structure when the data is extremely sparse. Specifically, traditional Nmf fails to exploit the structured sparsity of the large and sparse data sets resulting in dense factors. We propose a new algorithm for performing Nmf on sparse data that we call multifrontal Nmf (Mf-Nmf) since it borrows several ideas from the multifrontal method for unconstrained factorization (e.g. LU and QR). We also present an efficient shared memory parallel implementation of Mf-Nmf and discuss its performance and scalability. We conduct several experiments on synthetic and realworld datasets and demonstrate the usefulness of the algorithm by comparing it against standard baselines. We obtain a speedup of 1.2x to 19.5x on 24 cores with an average speed up of 10.3x across all the real world datasets.

Sao, Piyush↗
DOE OSTI2026

PowerMappeR: Power-Optimized Mapping of SNNs onto ReRAM Crossbars coupled via Packet-Switched NoCs

Many recent efforts in developing hardware-accelerated spiking neural networks (SNNs) are characterized by deep co-design between algorithms, architectures, and devices. Architectural advances overcome device constraints by coupling together many small resistive-RAM (ReRAM) crossbars via a network-on-chip (NoC) for neuromorphic component operation. Concurrently, improved SNN training methods increase accuracy and structural sparsity in networks despite growing problem sizes. Finally, compilers leverage these attributes to minimize area and inter-crossbar communication while mapping large SNNs to sophisticated architectures. However, for compiler-driven co-design to realize increasingly complex and profitable optimizations, a compile-time view of power consumption is critical. We present PowerMappeR to express and optimize over mapping-, architecture-, and device-specific power consumption information. By modeling the dynamic power of well-established components, we develop an integer linear programming (ILP)-based, encoding-agnostic, parametric power estimation model. Using this model, we demonstrate practical improvements in area and inter-crossbar communication by 0%–9.5% and 1.4%–5.1%, respectively. We also limit hotspot formation during optimization, achieving comparable or better results in targeted metrics with up to 96.4%–97.1% restriction of hotspot magnitude. Finally, we introduce profile-guided formulations to reduce worst-case and expected-case hotspot magnitude by 40.7%–69.5% and 40.6%–56.3%, respectively. Optimizing worst-case hotspot magnitude incidentally improves expected-case magnitude by 10.85%–33.45%. Reciprocally, optimizing expected-case magnitude incidentally improves worst-case magnitude by 4.33%–39.87%. Validation against hardware simulators confirms that PowerMappeR can decrease dynamic power consumption by 12.6%–27.3%.

Pohl, Devin [ORNL] (ORCID:0009000040149027)↗
DOE OSTI2025

Mapping Spiking Neural Networks to Heterogeneous Crossbar Architectures using Integer Linear Programming

Advances in novel hardware devices and architectures allow Spiking Neural Network (SNN) evaluation using ultra-low power, mixed-signal, memristor crossbar arrays. As individual network sizes quickly scale beyond the dimensional capabilities of single crossbars, networks must be mapped onto multiple crossbars. Crossbar sizes within modern Memristor Crossbar Architectures (MCAs) are determined predominately not by device technology but by network topology; more, smaller crossbars consume less area thanks to the high structural sparsity found in larger, brain-inspired SNNs. Motivated by continuing increases in SNN sparsity due to improvements in training methods, we propose utilizing heterogeneous crossbar sizes to further reduce area consumption. This approach was previously unachievable as prior compiler studies only explored solutions targeting homogeneous MCAs. Our work improves on the state-of-the-art by providing Integer Linear Programming (ILP) formulations supporting arbitrarily heterogeneous architectures. By modeling axonal interactions between neurons, our methods produce better mappings while removing inhibitive a priori knowledge requirements. We first show a 16.7-27.6% reduction in area consumption for square-crossbar homogeneous architectures. Then, we demonstrate 66.9-72.7% further reduction when using a reasonable configuration of heterogeneous crossbar dimensions. Next, we present a new optimization formulation capable of minimizing the number of inter-crossbar routes. When applied to solutions already near-optimal in area, an 11.9-26.4% routing reduction is observed without impacting area consumption. Finally, we present a profile-guided optimization capable of minimizing the number of runtime spikes between crossbars. Compared to the best-area-then-route optimized solutions, we observe a further 0.5-14.8% inter-crossbar spike reduction while requiring 1–3 orders of magnitude less solver time.

Pohl, Devin [ORNL] (ORCID:0009000040149027)↗
NASA NTRS1978

The Lanczos algorithm with selective orthogonalization

A new stable and efficient implementation of the Lanczos algorithm is presented. The algorithm is a powerful method for finding a few eigenvalues and eigenvectors at one or both ends of the spectrum of a symmetric matrix A. The algorithm is particularly effective if A is large and sparse in that the only way in which A enters the calculation is through a subroutine which computes Av for any vector v. Thus the user is free to take advantage of any sparsity structure in A and A need not even be represented as a matrix et al.

Parlett, B. N.↗
NASA NTRS1994

Tensor-GMRES method for large sparse systems of nonlinear equations

This paper introduces a tensor-Krylov method, the tensor-GMRES method, for large sparse systems of nonlinear equations. This method is a coupling of tensor model formation and solution techniques for nonlinear equations with Krylov subspace projection techniques for unsymmetric systems of linear equations. Traditional tensor methods for nonlinear equations are based on a quadratic model of the nonlinear function, a standard linear model augmented by a simple second order term. These methods are shown to be significantly more efficient than standard methods both on nonsingular problems and on problems where the Jacobian matrix at the solution is singular. A major disadvantage of the traditional tensor methods is that the solution of the tensor model requires the factorization of the Jacobian matrix, which may not be suitable for problems where the Jacobian matrix is large and has a 'bad' sparsity structure for an efficient factorization. We overcome this difficulty by forming and solving the tensor model using an extension of a Newton-GMRES scheme. Like traditional tensor methods, we show that the new tensor method has significant computational advantages over the analogous Newton counterpart. Consistent with Krylov subspace based methods, the new tensor method does not depend on the factorization of the Jacobian matrix. As a matter of fact, the Jacobian matrix is never needed explicitly.

Feng, Dan↗
DOE OSTI2020

Approximate Inverse Chain Preconditioner: Iteration Count Case Study for Spectral Support Solvers

As the growing availability of computational power slows, there has been an increasing reliance on algorithmic advances. However, faster algorithms alone will not necessarily bridge the gap in allowing computational scientists to study problems at the edge of scientific discovery in the next several decades. Often, it is necessary to simplify or precondition solvers to accelerate the study of large systems of linear equations commonly seen in a number of scientific fields. Preconditioning a problem to increase efficiency is often seen as the best approach; yet, preconditioners which are fast, smart, and efficient do not always exist. Following the progress of [1], we present a new preconditioner for symmetric diagonally dominant (SDD) systems of linear equations. These systems are common in certain PDEs, network science, and supervised learning among others. Based on spectral support graph theory, this new preconditioner builds off of the work of [2], computing and applying a V-cycle chain of approximate inverse matrices. This preconditioner approach is both algebraic in nature as well as hierarchically-constrained depending on the condition number of the system to be solved. Due to its generation of an Approximate Inverse Chain of matrices, we refer to this as the AIC preconditioner. We further accelerate the AIC preconditioner by utilizing precomputations to simplify setup and multiplications in the con-text of an iterative Krylov-subspace solver. While these iterative solvers can greatly reduce solution time, the number of iterations can grow large quickly in the absence of good preconditioners. Initial results for the AIC preconditioner have shown a very large reduction in iteration counts for SDD systems as compared to standard preconditioners such as Incomplete Cholesky (ICC) and Multigrid (MG). We further show significant reduction in iteration counts against the more advanced Combinatorial Multigrid (CMG) preconditioner. We have further developed no-fill sparsification techniques to ensure that the computational cost of applying the AIC preconditioner does not grow prohibitively large as the depth of the V-cycle grows for systems with larger condition numbers. Our numerical results have shown that these sparsifiers maintain the sparsity structure of our system while also displaying significant reductions in iteration counts.1 2

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Approximate Inverse Chain Preconditioner: Iteration Count Case Study for Spectral Support Solvers

As the growing availability of computational power slows, there has been an increasing reliance on algorithmic advances. However, faster algorithms alone will not necessarily bridge the gap in allowing computational scientists to study problems at the edge of scientific discovery in the next several decades. Often, it is necessary to simplify or precondition solvers to accelerate the study of large systems of linear equations commonly seen in a number of scientific fields. Preconditioning a problem to increase efficiency is often seen as the best approach; yet, preconditioners which are fast, smart, and efficient do not always exist. Following the progress of [1], we present a new preconditioner for symmetric diagonally dominant (SDD) systems of linear equations. These systems are common in certain PDEs, network science, and supervised learning among others. Based on spectral support graph theory, this new preconditioner builds off of the work of [2], computing and applying a V-cycle chain of approximate inverse matrices. This preconditioner approach is both algebraic in nature as well as hierarchically-constrained depending on the condition number of the system to be solved. Due to its generation of an Approximate Inverse Chain of matrices, we refer to this as the AIC preconditioner. We further accelerate the AIC preconditioner by utilizing precomputations to simplify setup and multiplications in the con-text of an iterative Krylov-subspace solver. While these iterative solvers can greatly reduce solution time, the number of iterations can grow large quickly in the absence of good preconditioners. Initial results for the AIC preconditioner have shown a very large reduction in iteration counts for SDD systems as compared to standard preconditioners such as Incomplete Cholesky (ICC) and Multigrid (MG). We further show significant reduction in iteration counts against the more advanced Combinatorial Multigrid (CMG) preconditioner. We have further developed no-fill sparsification techniques to ensure that the computational cost of applying the AIC preconditioner does not grow prohibitively large as the depth of the V-cycle grows for systems with larger condition numbers. Our numerical results have shown that these sparsifiers maintain the sparsity structure of our system while also displaying significant reductions in iteration counts.1 2

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Data-Driven Distributed Algorithms for Estimating Eigenvalues and Eigenvectors of Interconnected Dynamical Systems

Here, the paper presents data-driven algorithms to estimate in a distributed manner the eigenvalues, right and left eigenvectors of an unknown linear (or linearized) interconnected dynamic system. In particular, the proposed algorithms do not require the identification of the system model in advance before performing the estimation. As a first step, we consider interconnected dynamical system with distinct eigenvalues. The proposed strategy first estimates the eigenvalues using the well-known Prony method. The right and left eigenvectors are then estimated by solving distributively a set of linear equations. One important feature of the proposed algorithms is that the topology of communication network used to perform the distributed estimation can be chosen arbitrarily, given that it is connected, and is also independent of the structure or sparsity of the system (state) matrix. The proposed distributed algorithms are demonstrated via a numerical example.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Utilizing distributed acoustic sensing and ocean bottom fiber optic cables for submarine structural characterization

The sparsity of permanent seismic instrumentation in marine environments often limits the availability of subsea information on geohazards, including active fault systems, in both time and space. One sensing resource that provides observational access to the seafloor environment are existing networks of ocean bottom fiber optic cables; these cables, coupled to modern distributed acoustic sensing (DAS) systems, can provide dense arrays of broadband seismic observations capable of recording both seismic events and the ambient noise wavefield. Here, we report a marine DAS application which demonstrates the strength and limitation of this new technique on submarine structural characterization. Based on ambient noise DAS records on a 20 km section of a fiber optic cable offshore of Moss Landing, CA, in Monterey Bay, we extract Scholte waves from DAS ambient noise records using interferometry techniques and invert the resulting multimodal dispersion curves to recover a high resolution 2D shear-wave velocity image of the near seafloor sediments. We show for the first time that the migration of coherently scattered Scholte waves observed on DAS records can provide an approach for resolving sharp lateral contrasts in subsurface properties, particularly shallow faults and depositional features near the seafloor. Our results provide improved constraints on shallow submarine features in Monterey Bay, including fault zones and paleo-channel deposits, thus highlighting one of many possible geophysical uses of the marine cable network.

47 OTHER INSTRUMENTATION↗
NASA NTRS1982

Minimization versus homotopy algorithms

The relative merits and demerits of the minimization techniques are assessed using globally convergent quasi-Newton algorithms on the one hand and the homotopy algorithms on the other hand for the solution of problems of nonlinear structural analysis. Like the homotopy algorithms, the globally convergent quasi-Newton algorithms are equally suited for the solution of the nonlinear equations of structural analysis directly without having to pose the problem as an equivalent minimization problem. In the close neighborhood of the limit and bifurcation points quasi-Newton algorithms experience difficulties. Homotopy algorithms are robust for practically all types of nonlinear problems but are computationally not as cost effective since they provide an extremely accurate prediction of the response by calculating it as a large number of points. Globally convergent algorithms can perform well with very approximate Hessians, while homotopy algorithms require extremely accurate Hessians. While quasi-Newton algorithms can be very easily structured to exploit sparsity and symmetry, homotopy algorithms are not presently so structured and would require special modifications for exploitation of such features without sacrificing robustness and global convergence.

Kamat, M. P.↗