Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Sparsity”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 217 records · Page 12

Nonlinear regularization techniques for seismic tomography

The effects of several nonlinear regularization techniques are discussed in the framework of 3D seismic tomography. Traditional, linear, l{sub 2} penalties are compared to so-called sparsity promoting l{sub 1} and l{sub 0} penalties, and a total variation penalty. Which of these algorithms is judged optimal depends on the specific requirements of the scientific experiment. If the correct reproduction of model amplitudes is important, classical damping towards a smooth model using an l{sub 2} norm works almost as well as minimizing the total variation but is much more efficient. If gradients (edges of anomalies) should be resolved with a minimum of distortion, we prefer l{sub 1} damping of Daubechies-4 wavelet coefficients. It has the additional advantage of yielding a noiseless reconstruction, contrary to simple l{sub 2} minimization ('Tikhonov regularization') which should be avoided. In some of our examples, the l{sub 0} method produced notable artifacts. In addition we show how nonlinear l{sub 1} methods for finding sparse models can be competitive in speed with the widely used l{sub 2} methods, certainly under noisy conditions, so that there is no need to shun l{sub 1} penalizations.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Adaptive sparse polynomial chaos expansion based on least angle regression

Polynomial chaos (PC) expansions are used in stochastic finite element analysis to represent the random model response by a set of coefficients in a suitable (so-called polynomial chaos) basis. The number of terms to be computed grows dramatically with the size of the input random vector, which makes the computational cost of classical solution schemes (may it be intrusive (i.e. of Galerkin type) or non intrusive) unaffordable when the deterministic finite element model is expensive to evaluate. To address such problems, the paper describes a non intrusive method that builds a sparse PC expansion. First, an original strategy for truncating the PC expansions, based on hyperbolic index sets, is proposed. Then an adaptive algorithm based on least angle regression (LAR) is devised for automatically detecting the significant coefficients of the PC expansion. Beside the sparsity of the basis, the experimental design used at each step of the algorithm is systematically complemented in order to avoid the overfitting phenomenon. The accuracy of the PC metamodel is checked using an estimate inspired by statistical learning theory, namely the corrected leave-one-out error. As a consequence, a rather small number of PC terms are eventually retained (sparse representation), which may be obtained at a reduced computational cost compared to the classical 'full' PC approximation. The convergence of the algorithm is shown on an analytical function. Then the method is illustrated on three stochastic finite element problems. The first model features 10 input random variables, whereas the two others involve an input random field, which is discretized into 38 and 30 - 500 random variables, respectively.

97 MATHEMATICS AND COMPUTING↗

A non-adapted sparse approximation of PDEs with stochastic inputs

We propose a method for the approximation of solutions of PDEs with stochastic coefficients based on the direct, i.e., non-adapted, sampling of solutions. This sampling can be done by using any legacy code for the deterministic problem as a black box. The method converges in probability (with probabilistic error bounds) as a consequence of sparsity and a concentration of measure phenomenon on the empirical correlation between samples. We show that the method is well suited for truly high-dimensional problems.

97 MATHEMATICS AND COMPUTING↗

Reweighted ℓ{sub 1} minimization method for stochastic elliptic differential equations

We consider elliptic stochastic partial differential equations (SPDEs) with random coefficients and solve them by expanding the solution using generalized polynomial chaos (gPC). Under some mild conditions on the coefficients, the solution is “sparse” in the random space, i.e., only a small number of gPC basis makes considerable contribution to the solution. To exploit this sparsity, we employ reweighted l{sub 1} minimization to recover the coefficients of the gPC expansion. We also combine this method with random sampling points based on the Chebyshev probability measure to further increase the accuracy of the recovery of the gPC coefficients. We first present a one-dimensional test to demonstrate the main idea, and then we consider 14 and 40 dimensional elliptic SPDEs to demonstrate the significant improvement of this method over the standard l{sub 1} minimization method. For moderately high dimensional (∼10) problems, the combination of Chebyshev measure with reweighted l{sub 1} minimization performs well while for higher dimensional problems, reweighted l{sub 1} only is sufficient. The proposed approach is especially suitable for problems for which the deterministic solver is very expensive since it reuses the sampling results and exploits all the information available from limited sources.

97 MATHEMATICS AND COMPUTING↗

Expectation propagation for nonlinear inverse problems – with an application to electrical impedance tomography

In this paper, we study a fast approximate inference method based on expectation propagation for exploring the posterior probability distribution arising from the Bayesian formulation of nonlinear inverse problems. It is capable of efficiently delivering reliable estimates of the posterior mean and covariance, thereby providing an inverse solution together with quantified uncertainties. Some theoretical properties of the iterative algorithm are discussed, and the efficient implementation for an important class of problems of projection type is described. The method is illustrated with one typical nonlinear inverse problem, electrical impedance tomography with complete electrode model, under sparsity constraints. Numerical results for real experimental data are presented, and compared with that by Markov chain Monte Carlo. The results indicate that the method is accurate and computationally very efficient.

97 MATHEMATICS AND COMPUTING↗

Wavelets as basis functions to represent the coarse-graining potential in multiscale coarse graining approach

In this paper, we apply Multiresolution Analysis (MRA) to develop sparse but accurate representations for the Multiscale Coarse-Graining (MSCG) approximation to the many-body potential of mean force. We rigorously framed the MSCG method into MRA so that all the instruments of this theory become available together with a multitude of new basis functions, namely the wavelets. The coarse-grained (CG) force field is hierarchically decomposed at different resolution levels enabling to choose the most appropriate wavelet family for each physical interaction without requiring an a priori knowledge of the details localization. The representation of the CG potential in this new efficient orthonormal basis leads to a compression of the signal information in few large expansion coefficients. The multiresolution property of the wavelet transform allows to isolate and remove the noise from the CG force-field reconstruction by thresholding the basis function coefficients from each frequency band independently. We discuss the implementation of our wavelet-based MSCG approach and demonstrate its accuracy using two different condensed-phase systems, i.e. liquid water and methanol. Simulations of liquid argon have also been performed using a one-to-one mapping between atomistic and CG sites. The latter model allows to verify the accuracy of the method and to test different choices of wavelet families. Furthermore, the results of the computer simulations show that the efficiency and sparsity of the representation of the CG force field can be traced back to the mathematical properties of the chosen family of wavelets. This result is in agreement with what is known from the theory of multiresolution analysis of signals.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

The Curvature-Augmented Closest Point method with vesicle inextensibility application

Here, the Closest Point method, initially developed by Ruuth and Merriman, allows for the numerical solution of surface partial differential equations without the need for a parameterization of the surface itself. Surface quantities are embedded into the surrounding domain by assigning each value at a given spatial location to the corresponding value at the closest point on the surface. This embedding allows for surface derivatives to be replaced by their Cartesian counterparts (e.g. ∇ s =∇). This equivalence is only valid on the surface, and thus, interpolation is used to enforce what is known as the side condition away from the surface. To improve upon the method, this work derives an operator embedding that incorporates curvature information, making it valid in a neighborhood of the surface. With this, direct enforcement of the side condition is no longer needed. Comparisons in R 2 and R 3 show that the resulting Curvature-Augmented Closest Point method has better accuracy and requires less memory, through increased matrix sparsity, than the Closest Point method, while maintaining similar matrix condition numbers. To demonstrate the utility of the method in a physical application, simulations of inextensible, bi-lipid vesicles evolving toward equilibrium shapes are also included.

97 MATHEMATICS AND COMPUTING↗

Gradient-based optimization for regression in the functional tensor-train format

Predictive analysis of complex computational models, such as uncertainty quantification (UQ), must often rely on using an existing database of simulation runs. In this paper we consider the task of performing low-multilinear-rank regression on such a database. Specifically we develop and analyze an efficient gradient computation that enables gradient-based optimization procedures, including stochastic gradient descent and quasi-Newton methods, for learning the parameters of a functional tensor-train (FT). We compare our algorithms with 22 other nonparametric and parametric regression methods on 10 real-world data sets and show that for many physical systems, exploiting low-rank structure facilitates efficient construction of surrogate models. Here, we use a number of synthetic functions to build insight into behavior of our algorithms, including the rank adaptation and group-sparsity regularization procedures that we developed to reduce overfitting. Finally we conclude the paper by building a surrogate of a physical model of a propulsion plant on a naval vessel.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Compressive sensing adaptation for polynomial chaos expansions

Basis adaptation in Homogeneous Chaos spaces rely on a suitable rotation of the underlying Gaussian germ. Several rotations have been proposed in the literature resulting in adaptations with different convergence properties. Here, we present a new adaptation mechanism that builds on compressive sensing algorithms, resulting in a reduced polynomial chaos approximation with optimal sparsity. The developed adaptation algorithm consists of a two-step optimization procedure that computes the optimal coefficients and the input projection matrix of a low dimensional chaos expansion with respect to an optimally rotated basis. Finally, we demonstrate the attractive features of our algorithm through several numerical examples including the application on Large-Eddy Simulation (LES) calculations of turbulent combustion in a HIFiRE scramjet engine.

42 ENGINEERING↗

Bayesian sparse learning with preconditioned stochastic gradient MCMC and its applications

Deep neural networks have been successfully employed in an extensive variety of research areas, including solving partial differential equations. Despite its significant success, there are some challenges in effectively training DNN, such as avoiding overfitting in over-parameterized DNNs and accelerating the optimization in DNNs with pathological curvature. Here, we propose a Bayesian type sparse deep learning algorithm. The algorithm utilizes a set of spike-and-slab priors for the parameters in the deep neural network. The hierarchical Bayesian mixture will be trained using an adaptive empirical method. That is, one will alternatively sample from the posterior using preconditioned stochastic gradient Langevin Dynamics (PSGLD), and optimize the latent variables via stochastic approximation. The sparsity of the network is achieved while optimizing the hyperparameters with adaptive searching and penalizing. A popular SG-MCMC approach is Stochastic gradient Langevin dynamics (SGLD). However, considering the complex geometry in the model parameter space in nonconvex learning, updating parameters using a universal step size in each component as in SGLD may cause slow mixing. To address this issue, we apply a computationally manageable preconditioner in the updating rule, which provides a step-size parameter to adapt to local geometric properties. Moreover, by smoothly optimizing the hyperparameter in the preconditioning matrix, our proposed algorithm ensures a decreasing bias, which is introduced by ignoring the correction term in the preconditioned SGLD. According to the existing theoretical framework, we show that the proposed algorithm can asymptotically converge to the correct distribution with a controllable bias under mild conditions. Numerical tests are performed on both synthetic regression problems and learning solutions of elliptic PDE, which demonstrate the accuracy and efficiency of the present work.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

An adaptive Hessian approximated stochastic gradient MCMC method

Bayesian approaches have been successfully integrated into training deep neural networks. One popular family is stochastic gradient Markov chain Monte Carlo methods (SG-MCMC), which have gained increasing interest due to their ability to handle large datasets and the potential to avoid overfitting. Although standard SG-MCMC methods have shown great performance in a variety of problems, they may be inefficient when the random variables in the target posterior densities have scale differences or are highly correlated. Here, we present an adaptive Hessian approximated stochastic gradient MCMC method to incorporate local geometric information while sampling from the posterior. The idea is to apply stochastic approximation (SA) to sequentially update a preconditioning matrix at each iteration. The preconditioner possesses second-order information and can guide the random walk of a sampler efficiently. Instead of computing and saving the full Hessian of the log posterior, we use limited memory of the samples and their stochastic gradients to approximate the inverse Hessian-vector multiplication in the updating formula. Moreover, by smoothly optimizing the preconditioning matrix via SA, our proposed algorithm can asymptotically converge to the target distribution with a controllable bias under mild conditions. To reduce the training and testing computational burden, we adopt a magnitude-based weight pruning method to enforce the sparsity of the network. Our method is user-friendly and demonstrates better learning results compared to standard SG-MCMC updating rules. The approximation of inverse Hessian alleviates storage and computational complexities for large dimensional models. Numerical experiments are performed on several problems, including sampling from 2D correlated distribution, synthetic regression problems, and learning the numerical solutions of heterogeneous elliptic PDE. The numerical results demonstrate great improvement in both the convergence rate and accuracy.

97 MATHEMATICS AND COMPUTING↗

WeakIdent: Weak formulation for identifying differential equation using narrow-fit and trimming

Data-driven identification of differential equations is an interesting but challenging problem, especially when the given data are corrupted by noise. When the governing differential equation is a linear combination of various differential terms, the identification problem can be formulated as solving a linear system, with the feature matrix consisting of linear and nonlinear terms multiplied by a coefficient vector. This product is equal to the time derivative term, and thus generates dynamical behaviors. The goal is to identify the correct terms that form the equation to capture the dynamics of the given data. We propose a general and robust framework to recover differential equations using a weak formulation with two new mechanisms, narrow-fit and trimming, for both ordinary and partial differential equations (ODEs and PDEs). The weak formulation facilitates an efficient and robust way to handle noise, and two new mechanisms, narrow-fit and trimming, improve the coefficient support and value recoveries respectively. For each sparsity level, Subspace Pursuit is utilized to find an initial set of support from the large dictionary. Then, we focus on highly dynamic regions (rows of the feature matrix), and error normalize the feature matrix in the narrow-fit step. The support is further updated via trimming the terms that contribute the least. Finally, the support set of features with the smallest Cross-Validation error is chosen as the result. A comprehensive set of numerical experiments are presented for both systems of ODEs and PDEs with various noise levels. The proposed method gives a robust recovery of the coefficients, and a significant denoising effect which can handle up to 100% noise-to-signal ratio for some equations. We compare the proposed method with several state-of-the-art algorithms for the recovery of differential equations.

97 MATHEMATICS AND COMPUTING↗

Comparison of four learning-based methods for predicting groundwater redox status

Knowing the location where groundwater denitrification occurs, or by proxy the groundwater redox status (oxic, mixed, and anoxic), is valuable information for assessing and managing potential agricultural land-use impacts on freshwater quality. We compare the efficacy of supervised (Linear Discriminant Analysis LDA; Boosted Regression Trees, BRT; and Random Forest, RF) and unsupervised (Modified Self-Organizing Map, MSOM) learning-based methods to predict groundwater redox status in the agriculturally dominated Tasman, Waikato, and Wellington regions of New Zealand. Thresholds applied to regional groundwater-quality samples provide redox status variables and learn heuristics constrained by these variables and applied to spatial factors (climate, elevation, geologic, hydrology soils, and well depth) identify optimal sets of regional predictor variables. A split-sample approach is used to train and test the learning methods ability to predict redox status using the optimal predictor variables. Overall, the supervised methods demonstrate a prediction bias toward oxic conditions and inability to perform statistically well when using independent regional data; for example, consider kappa statistics for BRT (Tasman: 0.42, Waikato: 0.38, Wellington: 0.17), RF (Tasman: 0.42, Waikato: 0.47, Wellington: 0.17 and LDA (Tasman: 0.46, Waikato: 0.32, Wellington: 0.17). By contrast, the unsupervised method performs statistically well when predicting oxic, mixed, and anoxic conditions and corresponding depths when using independent regional data; for example, consider MSOM kappa statistics for Tasman: 0.78, Waikato: 0.80, Wellington: 0.76. The unsupervised learning method provides the added benefits of being (1) able to combine predictions into 3D regional anoxic probability plots for interpreting the spatial influence of paleosols and groundwater flowpaths on redox status, and (2) readily extended to map 3D redox status across New Zealand and other countries despite data bias and sparsity.

machine learning algorithms, Random Forest, booste↗

A deep learning-based workflow for fast prediction of 3D state variables in geological carbon storage: A dimension reduction approach

Deep learning (DL) models are extensively used as surrogate models for high-fidelity simulations of multiphase fluid flow in porous media at large scales, enabling fast forecasts of the spatial–temporal evolution of three-dimensional (3D) state variables in geological carbon storage (GCS). However, training these models in high-dimensional space remains computationally demanding and prone to overfitting because of limited training data. This paper presents a novel workflow to address these challenges by integrating dimension reduction (DR) methods. Here, the proposed workflow employed pre-trained DR models to extract the latent variables of geological models and state variables and utilized the multi-layer perceptron (MLP) for constructing mapping functions between the input and output variables in latent spaces. Subsequently, the pre-trained reconstruction models converted the MLP-predicted latent state variables to their original high-dimensional form. Furthermore, we proposed a novel strategy for the DR and reconstruction of 3D saturation fields to account for the unique data characteristics of sparsity, nonuniformity, and discontinuity. The proposed strategy applied PCA and inverse PCA for 2D average saturation fields and developed a DL-based 3D reconstruction model, leveraging three 2D average saturation fields as input to produce a 3D saturation field as output. The pre-training of DR and reconstruction models and training of MLP models were conducted on 84 Gulf of Mexico (GoM) simulations and evaluated on 12 testing simulations. Each simulation contained 720 monthly time steps, with the first 360 months as the injection period and the rest as the post-injection period. The proposed workflow, incorporating DR and DL models, accurately predicts the normalized 3D pressure fields, achieving mean square error (MSE) of 2.92 × 10 -7 compared to the ground truth obtained from a full-physics simulator. Furthermore, the proposed strategy outperformed PCA and convolutional autoencoder (CAE) models on 3D saturation fields, resulting in minor workflow prediction errors with an MSE of 2.93 × 10 -5 . The results suggest the proposed workflow provides sufficient predictive fidelity across temporal and spatial scales, and enables a speedup of 160 times compared to the full-physics simulator, facilitating improved decision-making and risk assessment for large-scale GCS management in real-time scenarios.

3D reconstruction model↗

Inference of phase field fracture models

The phase field approach to modeling fracture uses a diffuse damage field to represent cracks. This representation mollifies singularities that arise in computations with sharp interface models and some of the resultant difficulties in the mathematical and numerical treatment of fracture. Phase field fracture models have proven effective at representing crack propagation, branching, and merging. Specific formulations, beginning with brittle fracture, have also been shown to converge to classical solutions. Extensions to cover the range of material failure, including ductile and cohesive fracture, lead to an array of possible models. There exists a large body of literature focusing on this class of models and on the impact of model form on the predicted crack evolution. However, there have not been systematic studies into how optimal models may be chosen. Here, we take a first step in this direction by developing formal methods for identification of the best parsimonious model of phase field fracture given full-field data on the damage and deformation fields. We consider some of the main models that have been used for the degradation of elastic response due to damage and its propagation. Our approach builds upon Variational System Identification (VSI), a weak form variant of the Sparse Identification of Nonlinear Dynamics (SINDy). Furthermore, in this first communication we focus on synthetically generated data but we also consider central issues associated with the use of experimental full-field data, such as data sparsity and noise.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Parallel algorithms for finding connected components using linear algebra

Finding connected components is one of the most widely used operations on a graph. Optimal serial algorithms for the problem have been known for half a century, and many competing parallel algorithms have been proposed over the last several decades under various different models of parallel computation. This paper presents a class of parallel connected-component algorithms designed using linear-algebraic primitives. These algorithms are based on a PRAM algorithm by Shiloach and Vishkin and can be designed using standard GraphBLAS operations. Here, we demonstrate two algorithms of this class, one named LACC for Linear Algebraic Connected Components, and the other named FastSV which can be regarded as LACC’s simplification. With the support of the highly-scalable Combinatorial BLAS library, LACC and FastSV outperform the previous state-of-the-art algorithm by a factor of up to 12x for small to medium scale graphs. For large graphs with more than 50B edges, LACC and FastSV scale to 4K nodes (262K cores) of a Cray XC40 supercomputer and outperform previous algorithms by a significant margin. This remarkable performance is accomplished by (1) exploiting sparsity that was not present in the original PRAM algorithm formulation, (2) using high-performance primitives of Combinatorial BLAS, and (3) identifying hot spots and optimizing them away by exploiting algorithmic insights.

97 MATHEMATICS AND COMPUTING↗

A scalable design of experiments framework for optimal sensor placement

Here, we present a scalable design of an experiments framework for sensor placement in systems described by partial differential equations (PDEs). In particular, we aim to compute optimal sensor locations by minimizing the uncertainty of parameters estimated from Bayesian inverse problems and where the system. The resulting problem is a computationally intractable mixed-integer nonlinear program constrained by PDEs. We approach this problem with two heuristics used in compressed sensing and optimal control literature: a sparsity-inducing approach and a sum-up rounding approach. We also investigate metrics to guide the design of experiments (the total flow variance and the A-optimal design criterion) and analyze the effect of different noise structures (white and colored). Using an application in natural gas pipelines, we conclude that the sum-up rounding approach approach gives the best results and produces shrinking gaps with increasing mesh resolution. We also observe that convergence for the white noise measurement error case is slower than for the colored noise case. For A-optimal design, the solution is close to a uniform distribution of sensors along the pipeline while for the flow variance design the distribution is unstructured.

42 ENGINEERING↗

CryoFold: Determining protein structures and data-guided ensembles from cryo-EM density maps

Cryoelectron microscopy requires molecular modeling for refinement of structures. Ensemble models arrive at low free-energy molecular structures, but are computationally expensive and limited to resolving only small proteins. Here, we introduce CryoFold, a pipeline of molecular dynamics simulations that determines ensembles of protein structures by integrating density data of varying sparsity at 3–5 Å resolution with sequence information and coarse-grained topological knowledge of the protein folds. We present six examples, folding proteins between 72 and 2,000 residues, including large membrane and multi-domain systems, and results from two Electron Microscopy Data Bank (EMDB) competitions. Driven by data from a single state, CryoFold discovers ensembles of common low-energy models together with rare low-probability structures that capture the equilibrium distribution of proteins constrained by the density maps. Many of these conformations are experimentally validated and functionally relevant. We arrive at a set of best practices for data-guided protein folding that are controlled using a Python graphical user interface (GUI).

59 BASIC BIOLOGICAL SCIENCES↗