Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Hessian approximation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5

Sensitivity of magnetic islands in permanent magnet stellarators using the gradient and Hessian methods

Stellarator plasmas are known to be very sensitive to perturbations in the magnetic field. The permanent magnet stellarator was in part developed as a solution to high machining tolerances placed on the shape properties of electromagnetic coils in traditional stellarators. However, as a consequence of this high sensitivity to the field structure, sensitivities of permanent magnet stellarator plasmas to perturbations of permanent magnet properties must necessarily be well-understood. The gradient and Hessian matrix methods have been previously demonstrated to be useful sensitivity analysis methods for modular coils. We apply these two methods to the study of island width sensitivities in both the MUSE and PM4STELL permanent magnet stellarator projects. These sensitivity methods were used to determine the relative impacts of permanent magnet parameter perturbations on island widths in the vacuum field approximation of both stellarator equilibria. The square of resonant magnetic field perturbation is used here as a proxy for island width. In particular, gradients of magnetizations of individual magnets were examined in MUSE, as well as gradients of magnet group displacements informed by device design. Three different forms of permanent magnet magnetization perturbations are investigated for MUSE, and the flux surface response to perturbations is demonstrated. The Hessian matrix method is applied to PM4STELL, illustrating the sensitivity of dominant island widths to displacements of toroidal wedge structures. These methods allow for selective direction of experimental resources toward regions of heightened sensitivity, while constraints on less impactful permanent magnet parameters can be relaxed.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Nonlinear sparse Bayesian learning for physics-based models

This paper addresses the issue of overfitting while calibrating unknown parameters of over-parameterized physics-based models with noisy and incomplete observations. Here, a semi-analytical Bayesian framework of nonlinear sparse Bayesian learning (NSBL) is proposed to identify sparsity among model parameters during Bayesian inversion. NSBL offers significant advantages over machine learning algorithm of sparse Bayesian learning (SBL) for physics-based models, such as 1) the likelihood function or the posterior parameter distribution is not required to be Gaussian, and 2) prior parameter knowledge is incorporated into sparse learning (i.e. not all parameters are treated as questionable). NSBL employs the concept of automatic relevance determination (ARD) to facilitate sparsity among questionable parameters through parameterized prior distributions. The analytical tractability of NSBL is enabled by employing Gaussian ARD priors and by building a Gaussian mixture-model approximation of the posterior parameter distribution that excludes the contribution of ARD priors. Subsequently, type-II maximum likelihood is executed using Newton's method whereby the evidence and its gradient and Hessian information are computed in a semi-analytical fashion. We show numerically and analytically that SBL is a special case of NSBL for linear regression models. Subsequently, a linear regression example involving multimodality in both parameter posterior pdf and model evidence is considered to demonstrate the performance of NSBL in cases where SBL is inapplicable. Next, NSBL is applied to identify sparsity among the damping coefficients of a mass-spring-damper model of a shear building frame. These numerical studies demonstrate the robustness and efficiency of NSBL in alleviating overfitting during Bayesian inversion of nonlinear physics-based models.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Approximate Bayesian Model Inversion for PDEs with Heterogeneous and State-Dependent Coefficients

We present two approximate Bayesian inference methods for parameter estimation in partial differential equation (PDE) models with space-dependent and state-dependent parameters. We demonstrate that these methods provide accurate and cost-effective alternatives to Markov Chain Monte Carlo simulation. We assume a parameterized Gaussian prior on the unknown functions, and approximate the posterior density by a parameterized multivariate Gaussian density. The parameters of the prior and posterior are estimated from sparse observations of the PDE model's states and the unknown functions themselves by maximizing the evidence lower bound (ELBO), a lower bound on the log marginal likelihood of the observations. The first method, Laplace-EM, employs the expectation maximization algorithm to maximize the ELBO, with a Laplace approximation of the posterior on the E-step, and minimization of a Kullback-Leibler divergence on the M-step. The second method, DSVI-EB, employs the doubly stochastic variational inference (DSVI) algorithm, in which the ELBO is maximized via gradient-based stochastic optimization, with nosiy gradients computed via simple Monte Carlo sampling and Gaussian backpropagation. We apply these methods to identifying diffusion coefficients in linear and nonlinear diffusion equations, and we find that both methods provide accurate estimates of posterior densities and the hyperparameters of Gaussian priors. While the Laplace-EM method is more accurate, it requires computing Hessians of the physics model. The DSVI-EB method is found to be less accurate but only requires gradients of the physics model.

approximate Bayesian inference, model inversion, v↗

A fast mollified impulse method for biomolecular atomistic simulations

Classical integration methods for molecular dynamics are inherently limited due to resonance phenomena occurring at certain time-step sizes. The mollified impulse method can partially avoid this problem by using appropriate filters based on averaging or projection techniques. However, existing filters are computationally expensive and tedious in implementation since they require either analytical Hessians or they need to solve nonlinear systems from constraints. In this work we follow a different approach based on corotation for the construction of a new filter for (flexible) biomolecular simulations. The main advantages of the proposed filter are its excellent stability properties and ease of implementation in standard softwares without Hessians or solving constraint systems. By simulating multiple realistic examples such as peptide, protein, ice equilibrium and ice–ice friction, the new filter is shown to speed up the computations of long-range interactions by approximately 20%. The proposed filtered integrators allow step sizes as large as 10 fs while keeping the energy drift less than 1% on a 50 ps simulation.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Optimized structure and electronic band gap of monolayer GeSe from quantum Monte Carlo methods

Here, we have used highly accurate quantum Monte Carlo methods to determine the chemical structure and electronic band gaps of monolayer GeSe. Two-dimensional (2D) monolayer GeSe has received a great deal of attention due to its unique thermoelectric, electronic, and optoelectronic properties with a wide range of potential applications. Density functional theory (DFT) methods have usually been applied to obtain optical and structural properties of bulk and 2D GeSe. For the monolayer, DFT typically yields a larger band-gap energy than for bulk GeSe but cannot conclusively determine if the monolayer has a direct or indirect gap. Moreover, the DFT-optimized lattice parameters and atomic coordinates for monolayer GeSe depend strongly on the choice of approximation for the exchange-correlation functional, which makes the ideal structure-and its electronic properties-unclear. In order to obtain accurate lattice parameters and atomic coordinates for the monolayer, we use a surrogate Hessian-based parallel line search within diffusion Monte Carlo to fully optimize the GeSe monolayer structure. The DMC-optimized structure is different from those obtained using DFT, as are calculated band gaps. The potential energy surface has a shallow minimum at the optimal structure. This, combined with the sensitivity of the electronic structure to strain, suggests that the optical properties of monolayer GeSe are highly tunable by strain.

36 MATERIALS SCIENCE↗

Three-dimensional super line-localization in low signal-to-noise microscope images via prior-apprised unsupervised learning (PAUL)

Biological processes such as processive enzyme turnover and intracellular cargo tracking involve the dynamic motion of a small "article" along a curvilinear biopolymer track. To understand these processes that occur across multiple length and time scales, one must acquire both the trajectory of the particle and the position of the track along which it moves, possibly by combining high-resolution single-particle tracking with conventional microscopy. Yet, usually there is a significant resolution mismatch between these modalities: while the tracked particle is localized with a precision of 10 nm, the image of the surroundings is limited by optical difraction, with 200 nm lateral and 500 nm axial resolutions. Compared to the particle's trajectory, the surrounding curvilinear structure appears as a blurred and noisy image. This disparity in the spatial resolutions of the particle trajectory and the surrounding curvilinear structure image makes data reconstruction, as well as interpretation, particularly challenging. Analysis is further complicated when the curvilinear structures are oriented arbitrarily in 3D space. Here, we present a prior-apprised unsupervised learning (PAUL) approach to extract information from 3D images where the underlying features resemble a curved line such as a filament or microtubule. This three-stage framework starts with a Hessian-based feature enhancement, which is followed by feature registration, where local line segments are detected on repetitively sampled subimage tiles. In the final stage, statistical learning, segments are clustered based on their geometric relationships. Principal curves are then approximated from each segment group via statistical tools including principal component analysis, bootstrap and kernel transformation. This procedure is characterized on simulated images, where sub-voxel medium deviations from true curves have been achieved. The 3D PAUL approach has also been implemented for successful line localization in experimental 3D images of gold nanowires obtained using a multifocal microscope. Lastly, this work not only bridges the resolution gap between two microscopy modalities, but also allows us to conduct 3D super line-localization imaging experiments, without using super-resolution techniques.

42 ENGINEERING↗

Capturing Deuteration Effects in a Molecular Mechanics Force Field: Deuterated THF and the THF–Water Miscibility Gap

Deuteration is a common chemical modification used in conjunction with experiments such as neutron scattering, NMR, and Fourier-transform infrared for the study of molecular systems. Under the Born–Oppenheimer (BO) approximation, while the underlying potential energy surface remains unchanged by isotopic substitutions, isotopic substitution still alters intramolecular vibrations, which in turn may alter intermolecular interactions. Molecular mechanics (MM) force fields used in classical molecular dynamics simulations are assumed to represent local approximations of the BO potential energy surfaces, and hence, MD simulations using simple isotopic mass substitutions should capture BO-compatible isotope effects. However, standard MM force-field parameterizations do not directly fit to the local harmonic quantum mechanical (QM) Hessian that describes the BO surface, but rather to QM normal-modes and/or mass-dependent internal-coordinate derived distortion energies. In this study, using tetrahydrofuran (THF)–water mixtures as our model system, we show that not only does a simple mass-substitution approach fail to capture an experimentally characterized deuteration effect (the loss of the closed-loop miscibility gap associated with the complete deuteration of THF) but also it is necessary to generate new MM force-field parameters that correctly describe isotopic dependent vibrations to capture the experimental deuteration effect. We show that the origin of this failure is a result of using mass-dependent features to fit the THF MM force field, which unintentionally biases the bonded terms of the force field to represent only the isotopologue used during the original force-field parameterization. In addition, we make use of our isotopologue-corrected force field for D 8 THF to examine the molecular origins of the isotope-dependent loss of the THF–water miscibility gap.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Uncertainty estimates for the Bayes Inference Engine, (BIE)

In the fall 2007 meeting of the BIB users group, two approaches to making uncertainty estimates were presented. Ken Hanson asserted that if the BFGS optimizer was used, the inverse Hessian matrix was the same as the covariance matrix representing parameter uncertainties. John Pang presented preliminary results of a Monte Carlo method called Randomized Maximum Likelihood (RML). The BFGS/Hessian matrix approach may be applied to the region of the 'ideal model' Approximately 250 parameters describing the object density patches that are varied to match an image of 1,000,000 pixels. I cast this in terms of least squares analysis, as it is much better understood. This not as large a conceptual jump as some suppose because many of the functional blocks in the BIB are taken directly from existing least squares programs. If a Gaussian (normal) probability density function is assumed for both the observation and parameter errors, the Bayesian and least squares result should be identical.

97 MATHEMATICS AND COMPUTING↗

Sequential limiting in continuous and discontinuous Galerkin methods for the Euler equations

We present a new predictor-corrector approach to enforcing local maximum principles in piecewise-linear finite element schemes for the compressible Euler equations. The new element-based limiting strategy is suitable for continuous and discontinuous Galerkin methods alike. In contrast to synchronized limiting techniques for systems of conservation laws, we constrain the density, momentum, and total energy in a sequential manner which guarantees positivity preservation for the pressure and internal energy. After the density limiting step, the total energy and momentum gradients are adjusted to incorporate the irreversible effect of density changes. Antidiffusive corrections to bounds-compatible low-order approximations are limited to satisfy inequality constraints for the specific total and kinetic energy. An accuracy-preserving smoothness indicator is introduced to gradually adjust lower bounds for the element-based correction factors. The employed smoothness criterion is based on a Hessian determinant test for the density. Finally, a numerical study is performed for test problems with smooth and discontinuous solutions.

97 MATHEMATICS AND COMPUTING↗

An inexact regularized Newton framework with a worst-case iteration complexity of $ {\mathscr O}(\varepsilon^{-3/2}) $ for nonconvex optimization

Abstract An algorithm for solving smooth nonconvex optimization problems is proposed that, in the worst-case, takes ${\mathscr O}(\varepsilon ^{-3/2})$ iterations to drive the norm of the gradient of the objective function below a prescribed positive real number $\varepsilon $ and can take ${\mathscr O}(\varepsilon ^{-3})$ iterations to drive the leftmost eigenvalue of the Hessian of the objective above $-\varepsilon $. The proposed algorithm is a general framework that covers a wide range of techniques including quadratically and cubically regularized Newton methods, such as the Adaptive Regularization using Cubics (arc) method and the recently proposed Trust-Region Algorithm with Contractions and Expansions (trace). The generality of our method is achieved through the introduction of generic conditions that each trial step is required to satisfy, which in particular allows for inexact regularized Newton steps to be used. These conditions center around a new subproblem that can be approximately solved to obtain trial steps that satisfy the conditions. A new instance of the framework, distinct from arc and trace, is described that may be viewed as a hybrid between quadratically and cubically regularized Newton methods. Numerical results demonstrate that our hybrid algorithm outperforms a cubically regularized Newton method.

Mathematics↗

A log-barrier Newton-CG method for bound constrained optimization with complexity guarantees

Abstract We describe an algorithm based on a logarithmic barrier function, Newton’s method and linear conjugate gradients that seeks an approximate minimizer of a smooth function over the non-negative orthant. We develop a bound on the complexity of the approach, stated in terms of the required accuracy and the cost of a single gradient evaluation of the objective function and/or a matrix-vector multiplication involving the Hessian of the objective. The approach can be implemented without explicit calculation or storage of the Hessian.

O’Neill, Michael↗

hIPPYlib-MUQ: A Bayesian Inference Software Framework for Integration of Data with Complex Predictive Models under Uncertainty

Bayesian inference provides a systematic framework for integration of data with mathematical models to quantify the uncertainty in the solution of the inverse problem. However, the solution of Bayesian inverse problems governed by complex forward models described by partial differential equations (PDEs) remains prohibitive with black-box Markov chain Monte Carlo (MCMC) methods. We present hIPPYlib-MUQ, an extensible and scalable software framework that contains implementations of state-of-the art algorithms aimed to overcome the challenges of high-dimensional, PDE-constrained Bayesian inverse problems. These algorithms accelerate MCMC sampling by exploiting the geometry and intrinsic low-dimensionality of parameter space via derivative information and low rank approximation. The software integrates two complementary open-source software packages, hIPPYlib and MUQ. hIPPYlib solves PDE-constrained inverse problems using automatically-generated adjoint-based derivatives, but it lacks full Bayesian capabilities. MUQ provides a spectrum of powerful Bayesian inversion models and algorithms, but expects forward models to come equipped with gradients and Hessians to permit large-scale solution. By combining these two complementary libraries, we created a robust, scalable, and efficient software framework that realizes the benefits of each and allows us to tackle complex large-scale Bayesian inverse problems across a broad spectrum of scientific and engineering disciplines. To illustrate the capabilities of hIPPYlib-MUQ, we present a comparison of a number of MCMC methods available in the integrated software on several high-dimensional Bayesian inverse problems. These include problems characterized by both linear and nonlinear PDEs, various noise models, and different parameter dimensions. The results demonstrate that large (~ 50×) speedups over conventional black box and gradient-based MCMC algorithms can be obtained by exploiting Hessian information (from the log-posterior), underscoring the power of the integrated hIPPYlib-MUQ framework.

97 MATHEMATICS AND COMPUTING↗

A Framework for Distributed Approximation of Moments with Higher-Order Derivatives through Automatic Differentiation

We present a framework for the distributed approximation of moments, enabling the evaluation of the uncertainty in a dynamical system. The first and second moment, mean, and variance are computed with up to third-order Taylor series expansion. The required derivatives for the expansion are generated automatically by automatic differentiation and propagated through an implicit time stepper. The computational kernels are the accumulation of the derivatives (Jacobian, Hessian, tensor) and the covariance matrix. We apply distributed parallelism to the Hessian or third-order tensor, and the user merely has to provide a function for the differential equation, thus achieving similar ease of use as Monte Carlo-based methods. We demonstrate our approach using with benchmarks on Theta, a KNL-based system at the Argonne Leadership Computing Facility.

Automatic Differentiation↗

On the Derivation of Quasi-Newton Formulas for Optimization in Function Spaces

Newton’s method is usually preferred when solving optimization problems due to its superior convergence properties compared to gradient-based or derivative-free optimization algorithms. However, deriving and computing second-order derivatives needed by Newton’s method often is not trivial and, in some cases, not possible. In such cases quasi-Newton algorithms are a great alternative. In this paper, we provide a new derivation of well-known quasi-Newton formulas in an infinite-dimensional Hilbert space setting. Furthermore, it is known that quasi-Newton update formulas are solutions to certain variational problems over the space of symmetric matrices. In this paper, we formulate similar variational problems over the space of bounded symmetric operators in Hilbert spaces. By changing the constraints of the variational problem we obtain updates (for the Hessian and Hessian inverse) not only for the Broyden-Fletcher-Goldfarb-Shanno (BFGS) quasi-Newton method but also for Davidon–Fletcher–Powell (DFP), Symmetric Rank One (SR1), and Powell-Symmetric-Broyden (PSB). In addition, for an inverse problem governed by a partial differential equation (PDE), we derive DFP and BFGS “structured” secant formulas that explicitly use the derivative of the regularization and only approximates the second derivative of the misfit term. We show numerical results that demonstrate the desired mesh-independence property and superior performance of the resulting quasi-Newton methods.

97 MATHEMATICS AND COMPUTING↗

Computer-aided pulmonary image analysis in small animal models

Purpose: To develop an automated pulmonary image analysis framework for infectious lung diseases in small animal models. Methods: The authors describe a novel pathological lung and airway segmentation method for small animals. The proposed framework includes identification of abnormal imaging patterns pertaining to infectious lung diseases. First, the authors’ system estimates an expected lung volume by utilizing a regression function between total lung capacity and approximated rib cage volume. A significant difference between the expected lung volume and the initial lung segmentation indicates the presence of severe pathology, and invokes a machine learning based abnormal imaging pattern detection system next. The final stage of the proposed framework is the automatic extraction of airway tree for which new affinity relationships within the fuzzy connectedness image segmentation framework are proposed by combining Hessian and gray-scale morphological reconstruction filters. Results: 133 CT scans were collected from four different studies encompassing a wide spectrum of pulmonary abnormalities pertaining to two commonly used small animal models (ferret and rabbit). Sensitivity and specificity were greater than 90% for pathological lung segmentation (average dice similarity coefficient > 0.9). While qualitative visual assessments of airway tree extraction were performed by the participating expert radiologists, for quantitative evaluation the authors validated the proposed airway extraction method by using publicly available EXACT’09 data set. Conclusions: The authors developed a comprehensive computer-aided pulmonary image analysis framework for preclinical research applications. The proposed framework consists of automatic pathological lung segmentation and accurate airway tree extraction. The framework has high sensitivity and specificity; therefore, it can contribute advances in preclinical research in pulmonary diseases.

60 APPLIED LIFE SCIENCES↗

A moving mesh finite difference method for equilibrium radiation diffusion equations

An efficient moving mesh finite difference method is developed for the numerical solution of equilibrium radiation diffusion equations in two dimensions. The method is based on the moving mesh partial differential equation approach and moves the mesh continuously in time using a system of meshing partial differential equations. The mesh adaptation is controlled through a Hessian-based monitor function and the so-called equidistribution and alignment principles. Several challenging issues in the numerical solution are addressed. Particularly, the radiation diffusion coefficient depends on the energy density highly nonlinearly. This nonlinearity is treated using a predictor–corrector and lagged diffusion strategy. Moreover, the nonnegativity of the energy density is maintained using a cutoff method which has been known in literature to retain the accuracy and convergence order of finite difference approximation for parabolic equations. Numerical examples with multi-material, multiple spot concentration situations are presented. Numerical results show that the method works well for radiation diffusion equations and can produce numerical solutions of good accuracy. It is also shown that a two-level mesh movement strategy can significantly improve the efficiency of the computation.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Variance-Reduced Accelerated First-Order Methods: Central Limit Theorems and Confidence Statements

In this paper, we consider a strongly convex stochastic optimization problem and propose three classes of variable sample-size stochastic first-order methods: (i) the standard stochastic gradient descent method, (ii) its accelerated variant, and (iii) the stochastic heavy-ball method. In each scheme, the exact gradients are approximated by averaging across an increasing batch size of sampled gradients. We prove that when the sample size increases at a geometric rate, the generated estimates converge in mean to the optimal solution at an analogous geometric rate for schemes (i)–(iii). Based on this result, we provide central limit statements, whereby it is shown that the rescaled estimation errors converge in distribution to a normal distribution with the associated covariance matrix dependent on the Hessian matrix, the covariance of the gradient noise, and the step length. If the sample size increases at a polynomial rate, we show that the estimation errors decay at a corresponding polynomial rate and establish the associated central limit theorems (CLTs). Under certain conditions, we discuss how both the algorithms and the associated limit theorems may be extended to constrained and nonsmooth regimes. As a result, we provide an avenue to construct confidence regions for the optimal solution based on the established CLTs and test the theoretical findings on a stochastic parameter estimation problem.

Lei, Jinlong↗

Interparameter trade-off quantification and reduction in isotropic-elastic full-waveform inversion: synthetic experiments and Hussar land data set application

The problem of inverting for multiple physical parameters in the subsurface using seismic full-waveform inversion (FWI) is complicated by interparameter trade-off arising from inherent ambiguities between different physical parameters. Parameter resolution is often characterized using scattering radiation patterns, but these neglect some important aspects of interparameter trade-off. More general analysis and mitigation of interparameter trade-off in isotropic-elastic FWI is possible through judiciously chosen multiparameter Hessian matrix-vector products. We show that products of multiparameter Hessian off-diagonal blocks with model perturbation vectors, referred to as interparameter contamination kernels , are central to the approach. We then apply the multiparameter Hessian to various vectors designed to provide information regarding the strengths and characteristics of interparameter contamination, both locally and within the whole volume. With numerical experiments, we observe that S-wave velocity perturbations introduce strong contaminations into density and phase-reversed contaminations into P-wave velocity, but themselves experience only limited contaminations from other parameters. Based on these findings, we introduce a novel strategy to mitigate the influence of interparameter trade-off with approximate contamination kernels. Furthermore, we recommend that the local spatial and interparameter trade-off of the inverted models be quantified using extended multiparameter point spread functions (EMPSFs) obtained with pre-conditioned conjugate-gradient algorithm. Compared to traditional point spread functions, the EMPSFs appear to provide more accurate measurements for resolution analysis, by de-blurring the estimations, scaling magnitudes and mitigating interparameter contamination. Approximate eigenvalue volumes constructed with stochastic probing approach are proposed to evaluate the resolution of the inverted models within the whole model. With a synthetic Marmousi model example and a land seismic field data set from Hussar, Alberta, Canada, we confirm that the new inversion strategy suppresses the interparameter contamination effectively and provides more reliable density estimations in isotropic-elastic FWI as compared to standard simultaneous inversion approach.

58 GEOSCIENCES↗