SEARCH · Engineering Papers

Results for “sparse factorization”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 145 records · Page 8

All sources NASA NTRS DOE OSTI

DOE OSTI2017

Acquisition of STEM Images by Adaptive Compressive Sensing

Compressive Sensing (CS) allows a signal to be sparsely measured first and accurately recovered later in software [1]. In scanning transmission electron microscopy (STEM), it is possible to compress an image spatially by reducing the number of measured pixels, which decreases electron dose and increases sensing speed [2,3,4]. The two requirements for CS to work are: (1) sparsity of basis coefficients and (2) incoherence of the sensing system and the representation system. However, when pixels are missing from the image, it is difficult to have an incoherent sensing matrix. Nevertheless, dictionary learning techniques such as Beta-Process Factor Analysis (BPFA) [5] are able to simultaneously discover a basis and the sparse coefficients in the case of missing pixels. On top of CS, we would like to apply active learning [6,7] to further reduce the proportion of pixels being measured, while maintaining image reconstruction quality. Suppose we initially sample 10% of random pixels. We wish to select the next 1% of pixels that are most useful in recovering the image. Now, we have 11% of pixels, and we want to decide the next 1% of “most informative” pixels. Active learning methods are online and sequential in nature. Our goal is to adaptively discover the best sensing mask during acquisition using feedback about the structures in the image. In the end, we hope to recover a high quality reconstruction with a dose reduction relative to the non-adaptive (random) sensing scheme. In doing this, we try three metrics applied to the partial reconstructions for selecting the new set of pixels: (1) variance, (2) Kullback-Leibler (KL) divergence using a Radial Basis Function (RBF) kernel, and (3) entropy. Figs. 1 and 2 display the comparison of Peak Signal-to-Noise (PSNR) using these three different active learning methods at different percentages of sampled pixels. At 20% level, all the three active learning methods underperform the original CS without active learning. However, they all beat the original CS as more of the “most informative” pixels are sampled. One can also argue that CS equipped with active learning requires less sampled pixels to achieve the same value of PSNR than CS with pixels randomly sampled, since all the three PSNR curves with active learning grow at a faster pace than that without active learning. For this particular STEM image, by observing the reconstructed images and the sensing masks, we find that while the method based on RBF kernel acquires samples more uniformly, the one on entropy samples more areas of significant change, thus less uniformly. The KL-divergence method performs the best in terms of reconstruction error (PSNR) for this example [8].

Xie, Weiyi↗

DOE OSTI2017

Standard grids for high-precision integration of modern density functionals: SG-2 and SG-3

Density-functional approximations developed in the past decade necessitate the use of quadrature grids that are far more dense than those required to integrate older generations of functionals. This category of difficult-to-integrate functionals includes meta-generalized gradient approximations, which depend on orbital gradients and/or the Laplacian of the density, as well as functionals based on B97 and the popular “Minnesota” class of functionals, each of which contain complicated and/or oscillatory expressions for the exchange inhomogeneity factor. Following a strategy introduced previously by Gill and co-workers to develop the relatively sparse “SG-0” and “SG-1” standard quadrature grids, we introduce two higher-quality grids that we designate SG-2 and SG-3, obtained by systematically “pruning” medium- and high-quality atom-centered grids. The pruning procedure affords computational speedups approaching a factor of two for hybrid functionals applied to systems of ~100 atoms, without significant loss of accuracy. The grid dependence of several popular density functionals is characterized for various properties.

74 ATOMIC AND MOLECULAR PHYSICS↗

DOE OSTI2019

A communication-avoiding 3D sparse triangular solver

We present a novel distributed memory algorithm to improve the strong scalability of the solution of a sparse triangular system. This operation appears in the solve phase of direct methods for solving general sparse linear systems, Ax = b. Our 3D sparse triangular solver employs several techniques, including a 3D MPI process grid, elimination tree parallelism, and data replication, all of which reduce the per-process communication when combined. We present analytical models to understand the communication cost of our algorithm and show that our 3D sparse triangular solver can reduce the per-process communication volume asymptotically by a factor of O(n1/4) and O(n1/6) for problems arising from the finite element discretizations of 2D "planar" and 3D "non-planar" PDEs, respectively. We implement our algorithm for use in SuperLU_DIST3D, using a hybrid MPI+OpenMP programming model. Our 3D triangular solve algorithm, when run on 12k cores of Cray XC30, outperforms the current state-of-the-art 2D algorithm by 7.2x for planar and 2.7x for the non-planar sparse matrices, respectively.

Sao, Piyush↗

DOE OSTI2021

Simulations of plasmas and fluids using anti-symmetric models

ALMA (Anti-symmetric, Large-Moment, Accelerated) is a fast, flexible, and scalable toolkit designed to solve hyperbolic conservation law systems in hybrid supercomputers. Here this manuscript describes the theoretical background and implementation of ALMA, which uses the anti-symmetric formulation of fluids to obtain simple, robust, and easily paralellizable code. Practical GPU acceleration is realized on entire applications with an overall gain factor of 2 to 4. ALMA also provides a parallel, GPU accelerated sparse solver based on geometric multigrid, capable of diagonalizing linear systems with 239 unknowns. Here we demonstrate ALMA's scaling and performance in petascale supercomputers and use standard fluid models to verify the overall approach with canonical benchmark problems.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

DOE OSTI2019

Monitoring the Urban Heat Island Effect and the Efficacy of Future Countermeasures

To relate fine-scale spatial air-temperature variations in local urban heat islands and urban cool islands—increases and decreases in outside air temperature—within a large urban-climate archipelago to variations in land-use and land-cover properties in the Los Angeles Basin, the research team sought to (a) use fine-resolution meso-urban climate models to identify areas of urban heat and cool islands, select sites for fixed weather monitoring, and choose routes for mobile observations; (b) relate observed intraurban temperature variations to land use and land cover and surface physical properties; and (c) calibrate/validate the climate models. The research team assessed urban temperature variations via simulations and observations, including mobile transects, mesonet, dense networks of personal weather stations, and sparse but more accurate research-grade monitors. To identify the causative factors of the urban heat and cool islands at the neighborhood scale, the research team collected detailed urban morphometric and land use and land cover datasets, such as 1-meter (3.3 foot) resolution roof albedo (solar reflectance) and tree canopy cover. The research team used the observationvalidated model to finalize the transect routes and site the stationary monitors. This study provides the first observational evidence from analysis of high-spatial-density weather stations that increases in roof albedo at neighborhood scale are associated with reductions in near-surface air temperature. This finding was corroborated with the analysis from mobile transect measurements and correlation of observed air temperature with neighborhood-scale albedo and vegetation. This correlation revealed a cooling effect from areawide increase in albedo or canopy cover or both. The calibrated meteorological model accurately identified the localized urban heat and cool islands observed in this study. Interested stakeholders/researchers can use the same models and calibration/validation methodology to characterize within-city microclimate variations elsewhere in California, and can apply them to analyze the benefits from using urban heat island countermeasures. This project report is an extended and more detailed version of a related report prepared for California’s Fourth Climate Change Assessment.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

NASA NTRS2012

A First Order Wavefront Estimation Algorithm for P1640 Calibrator

P1640 calibrator is a wavefront sensor working with the P1640 coronagraph and the Palomar 3000 actuator adaptive optics system (P3K) at the Palomar 200 inch Hale telescope. It measures the wavefront by interfering post-coronagraph light with a reference beam formed by low-pass filtering the blocked light from the coronagraph focal plane mask. The P1640 instrument has a similar architecture to the Gemini Planet Imager (GPI) and its performance is currently limited by the quasi-static speckles due to non-common path wavefront errors, which comes from the non-common path for the light to arrive at the AO wavefront sensor and the coronagraph mask. By measuring the wavefront after the coronagraph mask, the non-common path wavefront error can be estimated and corrected by feeding back the error signal to the deformable mirror (DM) of the P3K AO system. Here, we present a first order wavefront estimation algorithm and an instrument calibration scheme used in experiments done recently at Palomar observatory. We calibrate the P1640 calibrator by measuring its responses to poking DM actuators with a sparse checkerboard pattern at different amplitudes. The calibration yields a complex normalization factor for wavefront estimation and establishes the registration of the DM actuators at the pupil camera of the P1640 calibrator, necessary for wavefront correction. Improvement of imaging quality after feeding back the wavefront correction to the AO system demonstrated the efficacy of the algorithm.

Wavefront sensing↗

DOE OSTI2018

Assessing robustness of factor ranking for supersaturated designs

Supersaturated designs can potentially be a beneficial tool for efficiently exploring a large number of factors with a moderately sized design. However, because more factors are being considered than there are runs, the stability of the identified factors depends heavily on effect sparsity and the lack of highly influential observations. A helpful tool for the analysis of supersaturated designs is least absolute shrinkage and selection operation (LASSO), which is useful when the effects of many explanatory variables are sparse in a high-dimensional dataset. To understand the impact of individual observations on the selected factors, the LASSO influence plot was created. This work describes an application of this plot and its variants that can be used to identify influential points, increase understanding of the impact of individual observations on model parameters, and the robustness of results in analyses with supersaturated designs. These graphical methods can serve as a complement to other regression diagnostics techniques in the LASSO regression setting.

3-D LASSO influence plots↗

DOE OSTI2026

GPU-Accelerated Analytic Simulation of Sparse Ionization Signal Formation in Pixelated Projection Detector

This paper presents a GPU-accelerated simulation package, TRED, for next-generation neutrino detectors with pixelated charge readout, leveraging community-driven software ecosystems to ensure adaptability and extensibility. We introduce two generic contributions: (i) an effective-charge representation based on Gaussian quadrature rules, in which the linear- interpolation factors for the field response inside each voxel are absorbed into the effective charge, and (ii) a sparse, block- binned tensor representation that enables efficient FFT-based computation of induced signals on readout electrodes for sparsely activated detector volumes. The former captures structure inside a voxel without dense sampling, while the latter achieves low memory usage and scalable runtime, as demonstrated in bench- mark studies. The underlying data representation is applicable to large-scale detectors and to other computational problems involving sparse activity.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

DOE OSTI2026

Sparse non-Markovian Noise Modeling of Transmon-Based Multi-Qubit Operations

The influence of noise on quantum dynamics is one of the main factors preventing current quantum processors from performing accurate quantum computations. Sufficient noise characterization and modeling can provide key insights into the effect of noise on quantum algorithms and inform the design of targeted error protection protocols. However, constructing effective noise models that are sparse in model parameters, yet predictive can be challenging. In this work, we present an approach for effective noise modeling of multi-qubit operations on transmon-based devices. Through a comprehensive characterization of seven devices offered by the IBM Quantum Platform, we show that the model can capture and predict a wide range of single- and two-qubit behaviors, including non-Markovian effects resulting from spatiotemporally correlated noise sources. The model’s predictive power is further highlighted through multi-qubit dynamical decoupling demonstrations and an implementation of the variational quantum eigensolver. As a training proxy for the hardware, we show that the model can predict expectation values within a relative error of 0.5%; this is a sevenfold improvement over default hardware noise models. Through these demonstrations, we highlight key error sources in superconducting qubits and illustrate the utility of reduced noise models for predicting hardware dynamics.

open quantum systems & decoherence↗

DOE OSTI2019

A Novel Sparse Compositional Technique Reveals Microbial Perturbations

ABSTRACT The central aims of many host or environmental microbiome studies are to elucidate factors associated with microbial community compositions and to relate microbial features to outcomes. However, these aims are often complicated by difficulties stemming from high-dimensionality, non-normality, sparsity, and the compositional nature of microbiome data sets. A key tool in microbiome analysis is beta diversity, defined by the distances between microbial samples. Many different distance metrics have been proposed, all with varying discriminatory power on data with differing characteristics. Here, we propose a compositional beta diversity metric rooted in a centered log-ratio transformation and matrix completion called robust Aitchison PCA. We demonstrate the benefits of compositional transformations upstream of beta diversity calculations through simulations. Additionally, we demonstrate improved effect size, classification accuracy, and robustness to sequencing depth over the current methods on several decreased sample subsets of real microbiome data sets. Finally, we highlight the ability of this new beta diversity metric to retain the feature loadings linked to sample ordinations revealing salient intercommunity niche feature importance. IMPORTANCE By accounting for the sparse compositional nature of microbiome data sets, robust Aitchison PCA can yield high discriminatory power and salient feature ranking between microbial niches. The software to perform this analysis is available under an open-source license and can be obtained at https://github.com/biocore/DEICODE ; additionally, a QIIME 2 plugin is provided to perform this analysis at https://library.qiime2.org/plugins/q2-deicode .

59 BASIC BIOLOGICAL SCIENCES↗

DOE OSTI2015

Solving a Large Scale Thermal Radiation Problem Using an Interoperable Executive Library Framework on Petascale Supercomputers

We present a novel methodology to compute the transient thermal condi- tion of a set of objects in an open space environment. The governing energy equa- tion and the convective energy transfer are solved by the sparse iterative solvers. The average radiating energy on a set of surfaces is represented by a linear system of the radiosity equations, which is factorized by an out-of-core parallel Cholesky decomposition solver. The coupling and interplay of the direct radiosity solver us- ing GPUs and the CPU-based sparse solver are handled by a light weight software integrator called Interoperable Executive Library (IEL). IEL manages the distribu- tion of data and memory, coordinates communication among parallel processes, and also directs execution of the set of loosely coupled physics tasks as warranted by the thermal condition of the simulated object and its surrounding environment.

Wong, Kwai↗

NASA NTRS1995

A General Purpose Sparse Matrix Parallel Solvers Package

A general purpose solver package for constructing and solving a range of sparse linear systems arising from discretization of PDEs on unstructured meshes is developed. Once the sparse symmetric complex matrix is constructed, it can be solved by either a preconditioned bi-conjugate gradient solver, a two-stage Cholesky LDLT factorization solver, or a hybrid solver combining the above two methods.

solver sparse matrix solvers package PDE↗

DOE OSTI2022

A Novel Fault Diagnosis Approach for Rolling Bearing Based on CWT and Adaptive Sparse Representation

Extraction and enhancement of weak impulse signature is the key of rolling bearing fault prognostics in which case the features are often weak and covered by noise. Tunable Q-factor wavelet transform (TQWT), as an emerging wavelet construction theory developed in a frequency domain explicitly, has the advantages of matching with the specific oscillation behavior of signal components. In this article, an adaptive sparse representation (ASR) method is proposed, which integrates the sparse code shrinkage (SCS) and parameter optimization into TQWT. However, direct application of ASR is difficult to extract fault signatures at the early stage or low-speed operation due to weak fault symptoms and background noise. A novel fault diagnosis strategy based on continuous wavelet transform (CWT) and ASR is investigated. CWT owns significant advantages on multiscale subdivision and weak signal detection. The results of simulated and experimental vibration signal analyses verify the effectiveness of the proposed method in accurately extracting weak impulse features from the noise environment.

Yuan, Xing↗

DOE OSTI2016

HPC-NMF: A High-Performance Parallel Algorithm for Nonnegative Matrix Factorization

NMF is a useful tool for many applications in different domains such as topic modeling in text mining, background separation in video analysis, and community detection in social networks. Despite its popularity in the data mining community, there is a lack of efficient distributed algorithms to solve the problem for big data sets. We propose a high-performance distributed-memory parallel algorithm that computes the factorization by iteratively solving alternating non-negative least squares (NLS) subproblems for $\WW$ and $\HH$. It maintains the data and factor matrices in memory (distributed across processors), uses MPI for interprocessor communication, and, in the dense case, provably minimizes communication costs (under mild assumptions). As opposed to previous implementation, our algorithm is also flexible: It performs well for both dense and sparse matrices, and allows the user to choose any one of the multiple algorithms for solving the updates to low rank factors $\WW$ and $\HH$ within the alternating iterations.

Kannan, Ramakrishnan↗

DOE OSTI2023

Distributed out-of-memory NMF on CPU/GPU architectures

We propose an efficient distributed out-of-memory implementation of the non-negative matrix factorization (NMF) algorithm for heterogeneous high-performance-computing systems. The proposed implementation is based on prior work on NMFk, which can perform automatic model selection and extract latent variables and patterns from data. In this work, we extend NMFk by adding support for dense and sparse matrix operation on multi-node, multi-GPU systems. The resulting algorithm is optimized for out-of-memory problems where the memory required to factorize a given matrix is greater than the available GPU memory. Memory complexity is reduced by batching/tiling strategies, and sparse and dense matrix operations are significantly accelerated with GPU cores (or tensor cores when available). Input/output latency associated with batch copies between host and device is hidden using CUDA streams to overlap data transfers and compute asynchronously, and latency associated with collective communications (both intra-node and inter-node) is reduced using optimized NVIDIA Collective Communication Library (NCCL) based communicators. Benchmark results show significant improvement, from 32X to 76x speedup, with the new implementation using GPUs over the CPU-based NMFk. Good weak scaling was demonstrated on up to 4096 multi-GPU cluster nodes with approximately 25,000 GPUs when decomposing a dense 340 Terabyte-size matrix and an 11 Exabyte-size sparse matrix of density 10 -6 .

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2017

Low-Rank Correction Methods for Algebraic Domain Decomposition Preconditioners

This study presents a parallel preconditioning method for distributed sparse linear systems, based on an approximate inverse of the original matrix, that adopts a general framework of distributed sparse matrices and exploits domain decomposition (DD) and low-rank corrections. The DD approach decouples the matrix and, once inverted, a low-rank approximation is applied by exploiting the Sherman--Morrison--Woodbury formula, which yields two variants of the preconditioning methods. The low-rank expansion is computed by the Lanczos procedure with reorthogonalizations. Numerical experiments indicate that, when combined with Krylov subspace accelerators, this preconditioner can be efficient and robust for solving symmetric sparse linear systems. Comparisons with pARMS, a DD-based parallel incomplete LU (ILU) preconditioning method, are presented for solving Poisson's equation and linear elasticity problems.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2017

High-Resolution 2D Lg and Pg Attenuation Models in the Basin and Range Region with Implications for Frequency-Dependent Q

Here, we develop high–resolution, laterally varying attenuation models for the regional crustal phases of Pg and Lg in the area surrounding the Basin and Range Province in the western United States. The models are part of the characterization effort for the Source Physics Experiment (SPE), a series of chemical explosions at the Nevada National Security Site designed to improve our understanding of explosion source phenomenology. To aid in SPE modeling efforts, we focus on improving our ability to accurately predict amplitudes in a set of narrow frequency bands ranging from 0.5 to 16.0 Hz. To explore constraints at higher frequencies where data become more sparse, we test the robustness of the empirically observed power–law relationship between quality factor Q and frequency (Q=Q 0 f γ ). Our methodology uses a staged approach to consider attenuation, physics–based source terms, site terms, and geometrical spreading contributions to amplitude measurements. Tomographic inversion results indicate that the frequency dependence is a reasonable assumption as attenuation varies laterally for this region through all frequency bands considered. Our 2D Pg and Lg attenuation models correlate with underlying physiographic provinces, with the highest Q located in the Sierra Nevada Mountains and the Colorado plateau. Compared to a best–fitting 1D model for the region, the 2D model provides an 81% variance reduction overall for Lg residuals and a 75% reduction for Pg . These detailed attenuation maps at high frequencies will facilitate further study of local and regional distance P/S amplitude discriminants that are typically used to distinguish between earthquakes and underground explosions.

58 GEOSCIENCES↗

NASA NTRS1995

A General Purpose Sparse Matrix Parallel Solvers Package

solver sparse matrix solvers package PDE↗