Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Sparsity”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6

Half-closed discontinuous Galerkin discretisations

Here we introduce the concept of half-closed nodes for nodal discontinuous Galerkin (DG) discretisations. Unlike more commonly used closed nodes in DG, where on every element nodes are placed on all of its boundaries, half-closed nodes only require nodes to be placed on a subset of the element's boundaries. The effect of using different nodes on DG operator sparsity is studied and we find in particular for there to be no difference in the sparsity pattern of the Laplace operator whether closed or half-closed nodes are used. On quadrilateral/hexahedral elements we use the Gauss-Radau points as the half-closed nodes of choice, which we demonstrate is able to speed up DG operator assembly in addition to leverage previously known superconvergence results. We also discuss in this work some linear solver techniques commonly used for Finite Element or discontinuous Galerkin methods such as static condensation and block-based methods, and how they can be applied to half-closed DG discretisations.

97 MATHEMATICS AND COMPUTING↗

Spike-and-Slab Shrinkage Priors for Structurally Sparse Bayesian Neural Networks

Network complexity and computational efficiency have become increasingly significant aspects of deep learning. Sparse deep learning addresses these challenges by recovering a sparse representation of the underlying target function by reducing heavily overparameterized deep neural networks. Specifically, deep neural architectures compressed via structured sparsity (e.g., node sparsity) provide low-latency inference, higher data throughput, and reduced energy consumption. In this article, we explore two well-established shrinkage techniques, Lasso and Horseshoe, for model compression in Bayesian neural networks (BNNs). To this end, we propose structurally sparse BNNs, which systematically prune excessive nodes with the following: 1) spike-and-slab group Lasso (SS-GL) and 2) SS group Horseshoe (SS-GHS) priors, and develop computationally tractable variational inference, including continuous relaxation of Bernoulli variables. We establish the contraction rates of the variational posterior of our proposed models as a function of the network topology, layerwise node cardinalities, and bounds on the network weights. Furthermore, we empirically demonstrate the competitive performance of our models compared with the baseline models in prediction accuracy, model compression, and inference latency.

97 MATHEMATICS AND COMPUTING↗

Compactly‐Supported Nonstationary Kernels for Computing Exact Gaussian Processes on Big Data

The Gaussian process (GP) is a widely used method for analyzing large-scale data sets, including spatio-temporal measurements of nonlinear processes that are now commonplace in the environmental sciences. Traditional implementations of GPs involve stationary kernels (also termed covariance functions) that limit their flexibility, and exact methods for inference that prevent application to data sets with more than about 10,000 points. Modern approaches to address stationarity assumptions generally fail to accommodate large data sets, while all attempts to address scalability focus on approximating the Gaussian likelihood, which can involve subjectivity and lead to inaccuracies. In this work, we explicitly derive an alternative kernel that can discover and encode both sparsity and nonstationarity. We embed the kernel within a fully Bayesian GP model and leverage high-performance computing resources to enable the analysis of massive data sets. We demonstrate the favorable performance of our novel kernel relative to existing exact and approximate GP methods across a variety of synthetic data examples. Furthermore, we conduct space–time prediction based on more than 1 million measurements of daily maximum temperature and verify that our results outperform state-of-the-art methods in the Earth sciences. More broadly, having access to exact GPs that use ultra-scalable, sparsity-discovering, nonstationary kernels allows GP methods to truly compete with a wide variety of machine learning methods.

Gaussian processes↗

Out-of-time-order correlators and Lyapunov exponents in sparse SYK

We use a combination of analytical and numerical methods to study out-of-time order correlators (OTOCs) in the sparse Sachdev-Ye-Kitaev (SYK) model. We find that at a given order of N, the standard result for the q-local, all-to-all SYK, obtained through the sum over ladder diagrams, is corrected by a series in the sparsity parameter, k. We present an algorithm to sum the diagrams at any given order of 1/(kq) n . We also study OTOCs numerically as a function of the sparsity parameter and determine the Lyapunov exponent. We find that numerical stability when extracting the Lyapunov exponent requires averaging over a massive number of realizations. This trade-off between the efficiency of the sparse model and consistent behavior at finite N becomes more significant for larger values of N.

2D gravity↗

A Galerkin method with two-dimensional Haar basis functions for the computation of the Karhunen–Loève expansion

We study the numerical approximation of a homogeneous Fredholm integral equation of second kind associated with the Karhunen–Loève expansion of Gaussian random fields. We develop, validate, and discuss an algorithm based on the Galerkin method with two-dimensional Haar wavelets as basis functions. The shape functions are constructed from the orthogonal decomposition of tensor-product spaces of one-dimensional Haar functions, and a recursive algorithm is employed to compute the matrix of the discrete eigenvalue system without the explicit calculation of integrals, allowing the implementation of a fast and efficient algorithm that provides considerable reduction in CPU time, when compared with classical Galerkin methods. Numerical experiments confirm the convergence rate of the method and assess the approximation error and the sparsity of the eigenvalue system when the wavelet expansion is truncated. We illustrate the numerical solution of a diffusion problem with random input data with the present method. In this problem, accuracy was retained after dropping the coefficients below a threshold value that was numerically determined. A similar method with scaling functions rather than wavelet functions does not need a discrete wavelet transform and leads to eigenvalue systems with better conditioning but lower sparsity.

97 MATHEMATICS AND COMPUTING↗

Improved natural convection heat transfer correlations for reactor cavity cooling systems of high-temperature gas-cooled reactors: From computational fluid dynamics to Pronghorn

The Reactor Cavity Cooling System (RCCS) is a common reactor safety system in High Temperature Gas Cooled Reactors (HTGR) that removes heat from the Reactor Pressure Vessel (RPV) by radiation ($\sim 80\%$) and natural convection ($\sim 20\%$). For simulation of accident scenarios of HTGRs, intermediate fidelity and system codes models must be employed for limiting the models' execution time. While accurate quantification of the radiative heat transfer is available in these models, quantification of natural convection must rely on correlations of questionable accuracy for the Nusselt number. Commonly used correlations are based in experiments performed at low Rayleigh numbers and/or using isothermal walls in simplified geometries. Here, this work improves on the accuracy of natural convection heat transfer correlations in support for HTGR designs. These correlations include both local and average Nusselt numbers as a function of the global Rayleigh number, the local Rayleigh number, and the temperature profile at the hot wall of the RCCS. In the absence of dedicated experiments and the difficulty of performing high-fidelity simulations at realistic Rayleigh numbers, the data to fit the correlations are generated with Computational Fluid Dynamics (CFD) using Reynolds Averaged Navier-Stokes (RANS) models. First, a careful selection of the RANS turbulence model is performed by comparing the results obtained with different RANS turbulence models against high fidelity simulations of natural convection at $Ra \ 1 \times 10^{11}$ in a rectangular cavity. Next, the selected model is used to perform simulations of an HTGR cavity at different high Rayleigh numbers $\in [6.1 \times 10^{11},2.9 \times 10^{13}]$ to encompass several HTGR designs, assuming an isothermal RPV wall. The results obtained are used to fit a correlation for the average and space-varying Nusselt number as a function of the global and local Rayleigh numbers via a sparsity-promoting least-squares method. The selected RANS model is then used to perform simulations of a PBMR 400 HTGR cavity with the temperature profiles at the RPV wall obtained during a PLOFC transient. We use the results obtained to fit a temperature-dependent correction to the space-varying Nusselt number with the sparsity-promoting least-squares method. The results obtained in this work, enable system-level codes, such as Pronghorn, to perform higher-fidelity simulations of the heat exchange process in the RCCS while still maintaining a low computational cost.

42 ENGINEERING↗

Data-Driven Distributed Algorithms for Estimating Eigenvalues and Eigenvectors of Interconnected Dynamical Systems

Here, the paper presents data-driven algorithms to estimate in a distributed manner the eigenvalues, right and left eigenvectors of an unknown linear (or linearized) interconnected dynamic system. In particular, the proposed algorithms do not require the identification of the system model in advance before performing the estimation. As a first step, we consider interconnected dynamical system with distinct eigenvalues. The proposed strategy first estimates the eigenvalues using the well-known Prony method. The right and left eigenvectors are then estimated by solving distributively a set of linear equations. One important feature of the proposed algorithms is that the topology of communication network used to perform the distributed estimation can be chosen arbitrarily, given that it is connected, and is also independent of the structure or sparsity of the system (state) matrix. The proposed distributed algorithms are demonstrated via a numerical example.

97 MATHEMATICS AND COMPUTING↗

A Markov chain Monte Carlo (MCMC) Bayesian inference approach to analyze apparent activation barriers and reaction orders from microreactor data

Statistical analysis of steady-state catalytic kinetic data is often limited by data sparsity due to the slow pace at which the data is collected. Data sparsity and limitations in statistical analysis make it difficult to differentiate between mechanistic models and catalytic sites. A Bayesian inference tool is reported for catalysis researchers to estimate error in the determination of reaction orders from steady state microreactor data. The benefits of a Bayesian inference approach are discussed, as an alternative to the more common frequentist approach. The approach incorporates prior knowledge of the system and the data collected to form an error estimate on reaction orders. We investigated the effects of three distinct data treatments—individual fitting of trials, pooled analysis, and constrained regression methods—on the precision and uncertainty of reaction order determinations. To assess the robustness of our findings, we conducted sensitivity analyses to evaluate the influence of Bayesian parameters on uncertainty estimation. Additionally, we utilized synthetic data to illustrate how data quality impacts the precision of uncertainty assessments. We show Bayesian analysis can obtain a more precise estimation of error with a sparse data set than a frequentist analysis. Finally, this work provides strong evidence that the adoption of Bayesian analysis of kinetic data may help researchers make more precise arguments as to the strength of their evidence for a particular mechanistic hypothesis, or in comparing across different catalysts.

42 ENGINEERING↗

A heterogeneous stochastic FEM framework for elliptic PDEs

We introduce a new concept of sparsity for the stochastic elliptic operator −div(a(x,ω)∇(⋅)), which reflects the compactness of its inverse operator in the stochastic direction and allows for spatially heterogeneous stochastic structure. This new concept of sparsity motivates a heterogeneous stochastic finite element method (HSFEM) framework for linear elliptic equations, which discretizes the equations using the heterogeneous coupling of spatial basis with local stochastic basis to exploit the local stochastic structure of the solution space. We also provide a sampling method to construct the local stochastic basis for this framework using the randomized range finding techniques. The resulting HSFEM involves two stages and suits the multi-query setting: in the offline stage, the local stochastic structure of the solution space is identified; in the online stage, the equation can be efficiently solved for multiple forcing functions. An online error estimation and correction procedure through Monte Carlo sampling is given. Numerical results for several problems with high dimensional stochastic input are presented to demonstrate the efficiency of the HSFEM in the online stage.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Sparse-grid, reduced-basis Bayesian inversion: Nonaffine-parametric nonlinear equations

We extend the reduced basis (RB) accelerated Bayesian inversion methods for affine-parametric, linear operator equations which are considered in [16,17] to non-affine, nonlinear parametric operator equations. We generalize the analysis of sparsity of parametric forward solution maps in [20] and of Bayesian inversion in [48,49] to the fully discrete setting, including Petrov–Galerkin high-fidelity (“HiFi”) discretization of the forward maps. We develop adaptive, stochastic collocation based reduction methods for the efficient computation of reduced bases on the parametric solution manifold. The nonaffinity and nonlinearity with respect to (w.r.t.) the distributed, uncertain parameters and the unknown solution is collocated; specifically, by the so-called Empirical Interpolation Method (EIM). For the corresponding Bayesian inversion problems, computational efficiency is enhanced in two ways: first, expectations w.r.t. the posterior are computed by adaptive quadratures with dimension-independent convergence rates proposed in [49]; the present work generalizes [49] to account for the impact of the PG discretization in the forward maps on the convergence rates of the Quantities of Interest (QoI for short). Second, we propose to perform the Bayesian estimation only w.r.t. a parsimonious, RB approximation of the posterior density. Based on the approximation results in [49], the infinite-dimensional parametric, deterministic forward map and operator admit N-term RB and EIM approximations which converge at rates which depend only on the sparsity of the parametric forward map. In several numerical experiments, the proposed algorithms exhibit dimension-independent convergence rates which equal, at least, the currently known rate estimates for N-term approximation. We propose to accelerate Bayesian estimation by first offline construction of reduced basis surrogates of the Bayesian posterior density. The parsimonious surrogates can then be employed for online data assimilation and for Bayesian estimation. They also open a perspective for optimal experimental design.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Vector tomography for reconstructing electric fields with non-zero divergence in bounded domains

In vector tomography (VT), the aim is to reconstruct an unknown multi-dimensional vector field using line integral data. In the case of a 2-dimensional VT, two types of line integral data are usually required. These data correspond to integration of the parallel and perpendicular projection of the vector field along the integration lines and are called the longitudinal and transverse measurements, respectively. In most cases, however, the transverse measurements cannot be physically acquired. Therefore, the VT methods are typically used to reconstruct divergence-free (or source-free) velocity and flow fields that can be reconstructed solely from the longitudinal measurements. In this paper, we show how vector fields with non-zero divergence in a bounded domain can also be reconstructed from the longitudinal measurements without the need of explicitly evaluating the transverse measurements. To the best of our knowledge, VT has not previously been used for this purpose. In particular, we study low-frequency, time-harmonic electric fields generated by dipole sources in convex bounded domains which arise, for example, in electroencephalography (EEG) source imaging. We explain in detail the theoretical background, the derivation of the electric field inverse problem and the numerical approximation of the line integrals. We show that fields with non-zero divergence can be reconstructed from the longitudinal measurements with the help of two sparsity constraints that are constructed from the transverse measurements and the vector Laplace operator. As a comparison to EEG source imaging, we note that VT does not require mathematical modeling of the sources. By numerical simulations, we show that the pattern of the electric field can be correctly estimated using VT and the location of the source activity can be determined accurately from the reconstructed magnitudes of the field. - Highlights: • Vector tomography is used to reconstruct electric fields generated by dipole sources. • Inverse solutions are based on longitudinal and transverse line integral measurements. • Transverse line integral measurements are used as a sparsity constraint. • Numerical procedure to approximate the line integrals is described in detail. • Patterns of the studied electric fields are correctly estimated.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

GenMod: A generative modeling approach for spectral representation of PDEs with random inputs

Here, we propose a method for quantifying uncertainty in high-dimensional PDE systems with random parameters, where the number of solution evaluations is small. Parametric PDE solutions are often approximated using a spectral decomposition based on polynomial chaos expansions. For the class of systems we consider (i.e., high dimensional with limited solution evaluations) the coefficients are given by an underdetermined linear system in a regression formulation. This implies additional assumptions, such as sparsity of the coefficient vector, are needed to approximate the solution. Here, we present an approach where we assume the coefficients are close to the range of a generative model that maps from a low to a high dimensional space of coefficients. Our approach is inspired be recent work examining how generative models can be used for compressed sensing in systems with random Gaussian measurement matrices. Using results from PDE theory on coefficient decay rates, we construct an explicit generative model that predicts the polynomial chaos coefficient magnitudes. The algorithm we developed to find the coefficients, which we call GenMod, is composed of two main steps. First, we predict the coefficient signs using Orthogonal Matching Pursuit. Then, we assume the coefficients are within a sparse deviation from the range of a sign-adjusted generative model. This allows us to find the coefficients by solving a nonconvex optimization problem, over the input space of the generative model and the space of sparse vectors. We obtain theoretical recovery results for a Lipschitz continuous generative model and for a more specific generative model, based on coefficient decay rate bounds. We examine three high-dimensional problems and show that, for all three examples, the generative model approach outperforms sparsity promoting methods at small sample sizes.

97 MATHEMATICS AND COMPUTING↗

Multi-head physics-informed neural networks for learning functional priors and uncertainty quantification

In numerous applications, the integration of prior knowledge and historical information is essential, particularly for tasks requiring the solution of ordinary or partial differential equations (ODEs/PDEs) in data-sparse or noisy environments. For instance, achieving accurate solutions to time-dependent PDEs with limited initial condition measurements necessitates an effective strategy for embedding prior knowledge. Hard-parameter sharing architectures in neural networks (NNs) have demonstrated success in both traditional and scientific machine learning domains, facilitating the learning of informative representations. Here, in this study, we introduce a novel, yet efficient, method to enhance physics-informed neural networks (PINNs) by incorporating a multi-head structure that enables the learning of functional priors from both empirical data and governing physical laws. This prior information can then be used to address data sparsity and high-level noise in solving ODE/PDE problems with uncertainty quantification (UQ). The approach, termed Multi-Head PINN (MH-PINN), consists of a shared body NN and multiple head NNs, each corresponding to an individual PINN instance. Our framework for functional prior learning is carried out in two stages: (1) training the MH-PINNs to develop a shared body NN alongside multiple head NNs, and (2) employing these trained head NNs to estimate a prior distribution through a normalizing flow-based density estimator. The learned functional prior can then be applied as a regularization mechanism in deterministic contexts or as an informative prior within a Bayesian inference framework, aiding in the resolution of subsequent ODE/PDE tasks. We evaluate the efficacy of MH-PINNs across five benchmark problems, including a high-dimensional parametric PDE, all characterized by data sparsity or substantial noise levels. Our findings reveal that MH-PINNs deliver accurate solutions and robust UQ, demonstrating adaptability across a range of complex and challenging scenarios.

Bayesian inference↗

Variable rate neural compression for sparse detector data

Particle colliders produce data at extraordinary rates, posing major challenges for transmission and storage. High-throughput compression algorithms are therefore essential. In the sPHENIX experiment taking data at the Relativistic Heavy Ion Collider, a time projection chamber records three-dimensional (3D) particle trajectories that are highly sparse, making conventional learning-free lossy compression ineffective. Convolutional neural networks have surpassed traditional methods in compression ratio and accuracy. However, they fail to exploit sparsity for efficiency. To address these gaps, we present BCAE-VS, a bicephalous convolutional autoencoder with variable compression ratio for sparse data, which adapts compression to input complexity through key-point identification and sparse convolution. BCAE-VS achieves higher accuracy and compression ratios than prior neural approaches while being orders of magnitude smaller. Moreover, its throughput increases with sparsity—a property not observed in other methods. Although it was developed for collider experiments, BCAE-VS readily extends to other sparse data domains, such as light detection and ranging (LiDAR) sensing and 3D microscopy.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Dynamic signal recovery in distribution grids using compressive lossy measurements

Distribution system state estimation requires reliable aggregation of the measured data. However, the large volume of the measured data imposes a significant stress on the underlying communication infrastructure. With the challenges associated with measurement availability, current distribution systems are typically unobservable. To cope with the unobservability issue, compressive sensing theory allows us to recover system state information from a small number of measurements provided the states of the distribution system exhibit sparsity. In this paper, we evaluate the robustness of an updated Kalman filtered modified compressive sensing (KF-ModCS) technique that dynamically estimates the grid states using a small fraction of measured data. In practice, measurements used for sparsity based state estimation may also be intermittent due to communication network induced losses. Further, to understand the effect of packet losses on KF-ModCS, we provide an upper bound for the expected variances of the state estimation error for a given rate of information loss. This upper bound is further improved if the support set of the sparse signal that characterizes the state dynamics does not change over time and/or the reduced model is observable. Simulations based on two practical data sets collected from actual customers in a distribution grid validate the theoretical results.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Linear Scaling Self-Consistent Field Calculations with Millions of Atoms in the Condensed Phase

In this work, the applicability and performance of a linear scaling algorithm is investigated for three-dimensional condensed phase systems. A simple but robust approach based on the matrix sign function is employed together with a thresholding matrix multiplication that does not require a prescribed sparsity pattern. Semiempirical methods and density functional theory have been tested. We demonstrate that self-consistent calculations with 1 million atoms are feasible for simple systems. With this approach, the computational cost of the calculation depends strongly on basis set quality. In the current implementation, high quality calculations for dense systems are limited to a few hundred thousand atoms. We report on the sparsities of the involved matrices as obtained at convergence and for intermediate iterations. We investigate how determining the chemical potential impacts the computational cost for very large systems.

Chemistry↗

Observed Relationships Between Cloud Droplet Effective Radius and Biogenic Gas Concentrations in Summertime Marine Stratocumulus Over the Eastern North Atlantic

Abstract Biogenic gases are a prominent component of the summertime marine boundary layer (MBL) over the Eastern North Atlantic. One of these gases, dimethyl sulfide (DMS), can produce sulfate cloud condensation nuclei (CCN) that, in theory, can brighten clouds through photolysis, and produces a reaction product, methane sulfonic acid (MSA). It is also possible that DMS can interact with sea‐salt or other marine aerosols changing their CCN activation spectrum, which could also modify cloud microphysical structure. Data collected aboard the G1 aircraft during the Aerosol Cloud Experiment Eastern North Atlantic (ACE‐ENA) in well‐mixed and decoupled marine boundary layers (MBLs) were used to examine relationships between the cloud droplet effective radii, , and the concentrations of DMS and MSA in constant cloud liquid water content (LWC) bins. A weak but statistically significant negative correlation was observed between CCN concentration and in most LWC bins, regardless of the source of the CCN, while a weak but statistically significant positive correlation between and DMS was observed. No correlation between the cloud droplet number concentration and DMS was found. The presence of MSA indicated that DMS‐to‐sulfate photolysis was likely occurring, but data sparsity prevented a statistically significant conclusion regarding the relationship between MSA and . Data sparsity in decoupled conditions also prevented statistically significant conclusions. To properly address biogenic gas impacts on cloud microphysics, it is recommended that aircraft data be supplemented by long‐term biogenic gas measurements at the surface in marine locations with appropriate remote and in‐situ cloud sensing capabilities, and the analysis limited to well‐mixed MBL's.

54 ENVIRONMENTAL SCIENCES↗