Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Hessian approximation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4

Designing stellarator coils by a modified Newton method using FOCUS

To find the optimal coils for stellarators, nonlinear optimization algorithms are applied in existing coil design codes. However, none of these codes have used the information from the second-order derivatives. In this paper, we present a modified Newton method in the recently developed code FOCUS. The Hessian matrix is calculated with analytically derived equations. Its inverse is approximated by a modified Cholesky factorization and applied in the iterative scheme of a classical Newton method. Using this method, FOCUS is able to recover the W7-X modular coils starting from a simple initial guess. Results demonstrate significant advantages.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Interpolating moving least-squares methods for fitting potential energy surfaces : using classical trajectories to explore configuration space.

We develop two approaches for growing a fitted potential energy surface (PES) by the interpolating moving least-squares (IMLS) technique using classical trajectories. We illustrate both approaches by calculating nitrous acid (HONO) cis {yields} trans isomerization trajectories under the control of ab initio forces from low-level HF/cc-pVDZ electronic structure calculations. In this illustrative example, as few as 300 ab initio energy/gradient calculations are required to converge the isomerization rate constant at a fixed energy to {approx}10%. Neither approach requires any preliminary electronic structure calculations or initial approximate representation of the PES (beyond information required for trajectory initial conditions). Hessians are not required. Both approaches rely on the fitting error estimation properties of IMLS fits. The first approach, called IMLS-accelerated direct dynamics, propagates individual trajectories directly with no preliminary exploratory trajectories. The PES is grown 'on the fly' with the computation of new ab initio data only when a fitting error estimate exceeds a prescribed tight tolerance. The second approach, called dynamics-driven IMLS fitting, uses relatively inexpensive exploratory trajectories to both determine and fit the dynamically accessible configuration space. Once exploratory trajectories no longer find configurations with fitting error estimates higher than the designated accuracy, the IMLS fit is considered to be complete and usable in classical trajectory calculations or other applications.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

RegularizedOptimization.jl: A Julia framework for regularized and nonsmooth optimization

RegularizedOptimization.jl is a Julia package that implements families of quadratic regularization and trust-region methods for solving the nonsmooth optimization problem $^{\textrm{minimize}}_{𝑥∈ℝ^𝑛}$ 𝑓(𝑥) + ℎ(𝑥) subject to 𝑐(𝑥) = 0, (1) where 𝑓 ∶ ℝ 𝑛 → ℝ and 𝑐 ∶ ℝ 𝑛 → ℝ 𝑚 are continuously differentiable, and ℎ ∶ ℝ 𝑛 → ℝ∪{+∞} is lower semi-continuous. The nonsmooth objective ℎ can be a regularizer, such as a sparsity inducing penalty, model simple constraints, such as 𝑥 belonging to a simple convex set, or can be a combination of both. All 𝑓, ℎ, and 𝑐 can be nonconvex. RegularizedOptimization.jl provides a modular and extensible framework for solving (1), and developing novel solvers. Currently, the following solvers are implemented: • Trust-region solvers TR and TRDH (Aravkin et al., 2022; Leconte & Orban, 2025) • Quadratic regularization solvers R2, R2DH and R2N (Aravkin et al., 2022; Diouane, Habiboullah, et al., 2024) • Levenberg-Marquardt solvers LM and LMTR (Aravkin et al., 2024) used when 𝑓 is a least-squares residual. • Augmented Lagrangian solver AL (De Marchi et al., 2023). All solvers rely on first derivatives of 𝑓 and 𝑐, and optionally on their second derivatives in the form of Hessian-vector products. If second derivatives are not available, quasi-Newton approximations can be used. In addition, the proximal mapping of the nonsmooth part ℎ, or adequate models thereof, must be evaluated. At each iteration, a step is computed by solving a subproblem of the form (1) inexactly, in which 𝑓, ℎ, and 𝑐 are replaced with appropriate models around the current iterate. The solvers R2, R2DH, and TRDH are particularly well suited to solve the subproblems, though they are general enough to solve (1). All solvers are allocation-free, so re-solves incur no additional allocations. To illustrate our claim of extensibility, a first version of the AL solver was implemented by an external contributor. Furthermore, a nonsmooth penalty approach, described in Diouane, Gollier, et al. (2024), is currently being developed, that relies on the library to efficiently solve the subproblems.

Gollier, Maxence [Polytechnique Montréal, QC (Cana↗

Accuracy of finite-difference harmonic frequencies in density functional theory

Analytic Hessians are often viewed as essential for the calculation of accurate harmonic frequencies, but the implementation of analytic second derivatives is nontrivial and solution of the requisite coupled-perturbed equations engenders a sizable memory footprint for large systems, given that these equations are not required for energy and gradient calculations in density functional theory. Here, we benchmark the alternative approach to harmonic frequencies based on finite differences of analytic first derivatives, a procedure that is amenable to large-scale parallelization. Not only for absolute frequencies but also for isotopic and conformer-dependent frequency shifts in flexible molecules, we find that the finite-difference approach exhibits mean errors < 0.1 cm -1 as compared to results based on an analytic Hessian. For very small frequencies corresponding to nonbonded vibrations in noncovalent complexes (for which the harmonic approximation is questionable anyway), the finite-difference error can be larger, but even in these cases the errors can be reduced below 0.1 cm -1 by judicious choice of the displacement step size and a higher-order finite-difference approach. Finally, the surprising accuracy and robustness of the finite-difference results suggests that availability of the analytic Hessian is not so important in today's era of commodity processors that are readily available in large numbers.

74 ATOMIC AND MOLECULAR PHYSICS↗

A quantitative quantum-chemical analysis tool for the distribution of mechanical force in molecules

The promising field of mechanochemistry suffers from a general lack of understanding of the distribution and propagation of force in a stretched molecule, which limits its applicability up to the present day. In this article, we introduce the JEDI (Judgement of Energy DIstribution) analysis, which is the first quantum chemical method that provides a quantitative understanding of the distribution of mechanical stress energy among all degrees of freedom in a molecule. The method is carried out on the basis of static or dynamic calculations under the influence of an external force and makes use of a Hessian matrix in redundant internal coordinates (bond lengths, bond angles, and dihedral angles), so that all relevant degrees of freedom of a molecule are included and mechanochemical processes can be interpreted in a chemically intuitive way. The JEDI method is characterized by its modest computational effort, with the calculation of the Hessian being the rate-determining step, and delivers, except for the harmonic approximation, exact ab initio results. We apply the JEDI analysis to several example molecules in both static quantum chemical calculations and Born-Oppenheimer Molecular Dynamics simulations in which molecules are subject to an external force, thus studying not only the distribution and the propagation of strain in mechanically deformed systems, but also gaining valuable insights into the mechanochemically induced isomerization of trans-3,4-dimethylcyclobutene to trans,trans-2,4-hexadiene. The JEDI analysis can potentially be used in the discussion of sonochemical reactions, molecular motors, mechanophores, and photoswitches as well as in the development of molecular force probes.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Scalable and efficient algorithms for the propagation of uncertainty from data through inference to prediction for large-scale problems, with application to flow of the Antarctic ice sheet

The majority of research on efficient and scalable algorithms in computational science and engineering has focused on the forward problem : given parameter inputs, solve the governing equations to determine output quantities of interest. In contrast, in this paper, we consider the broader question: given a (large-scale) model containing uncertain parameters, (possibly) noisy observational data, and a prediction quantity of interest, how do we construct efficient and scalable algorithms to (1) infer the model parameters from the data (the deterministic inverse problem ), (2) quantify the uncertainty in the inferred parameters (the Bayesian inference problem), and (3) propagate the resulting uncertain parameters through the model to issue predictions with quantified uncertainties (the forward uncertainty propagation problem )? We present efficient and scalable algorithms for this end-to-end, data-to-prediction process under the Gaussian approximation and in the context of modeling the flow of the Antarctic ice sheet and its effect on loss of grounded ice to the ocean. The ice is modeled as a viscous, incompressible, creeping, shear-thinning fluid. The observational data come from satellite measurements of surface ice flow velocity, and the uncertain parameter field to be inferred is the basal sliding parameter, represented by a heterogeneous coefficient in a Robin boundary condition at the base of the ice sheet. The prediction quantity of interest is the present-day ice mass flux from the Antarctic continent to the ocean. We show that the work required for executing this data-to-prediction process—measured in number of forward (and adjoint) ice sheet model solves—is independent of the state dimension, parameter dimension, data dimension, and the number of processor cores. The key to achieving this dimension independence is to exploit the fact that, despite their large size, the observational data typically provide only sparse information on model parameters. This property can be exploited to construct a low rank approximation of the linearized parameter-to-observable map via randomized SVD methods and adjoint-based actions of Hessians of the data misfit functional.

54 ENVIRONMENTAL SCIENCES↗

Stability analysis of an approximate scheme for moving horizon estimation.

We analyze the stability properties of an approximate algorithm for moving horizon estimation (MHE). The strategy provides instantaneous state estimates and is thus suitable for large-scale feedback control. In particular, we study the interplay between numerical approximation errors and the convergence of the estimator error. In addition, we establish connections between the numerical properties of the Hessian of the MHE problem and traditional observability definitions. We demonstrate the developments through a simulation case study.

99 GENERAL AND MISCELLANEOUS↗

Scalable Matrix-Free Adaptive Product-Convolution Approximation for Locally Translation-Invariant Operators

We present an adaptive grid matrix-free operator approximation scheme based on a "product-convolution" interpolation of convolution operators. This scheme is appropriate for operators that are locally translation-invariant, even if these operators are high rank or full rank. Such operators arise in Schur complement methods for solving partial differential equations (PDEs), as Hessians in PDE-constrained optimization and inverse problems, as integral operators, as covariance operators, and as Dirichlet-to-Neumann maps. Constructing the approximation requires computing the impulse responses of the operator to point sources centered on nodes in an adaptively refined grid of sample points. A randomized a posteriori error estimator drives the adaptivity. Once constructed, the approximation can be efficiently applied to vectors using the fast Fourier transform. The approximation can be efficiently converted to hierarchical matrix (H-matrix) format, then inverted or factorized using scalable H-matrix arithmetic. The quality of the approximation degrades gracefully as fewer sample points are used, allowing cheap lower quality approximations to be used as preconditioners. Furthermore, this yields an automated method to construct preconditioners for locally translation-invariant Schur complements. We directly address issues related to boundaries and prove that our scheme eliminates boundary artifacts. We test the scheme on a spatially varying blurring kernel, on the nonlocal component of an interface Schur complement for the Poisson operator, and on the data misfit Hessian for an advection dominated advection-diffusion inverse problem. Numerical results show that the scheme outperforms existing methods.

97 MATHEMATICS AND COMPUTING↗

A new smoothing function to introduce long-range electrostatic effects in QM/MM calculations

A new method to account for long range electrostatic contributions is proposed and implemented for quantum mechanics/molecular mechanics long range electrostatic correction (QM/MM-LREC) calculations. This method involves the use of the minimum image convention under periodic boundary conditions and a new smoothing function for energies and forces at the cutoff boundary for the Coulomb interactions. Compared to conventional QM/MM calculations without long-range electrostatic corrections, the new method effectively includes effects on the MM environment in the primary image from its replicas in the neighborhood. QM/MM-LREC offers three useful features including the avoidance of calculations in reciprocal space (k-space), with the concomitant avoidance of having to reproduce (analytically or approximately) the QM charge density in k-space, and the straightforward availability of analytical Hessians. The new method is tested and compared with results from smooth particle mesh Ewald (PME) for three systems including a box of neat water, a double proton transfer reaction, and the geometry optimization of the critical point structures for the rate limiting step of the DNA dealkylase AlkB. As with other smoothing or shifting functions, relatively large cutoffs are necessary to achieve comparable accuracy with PME. For the double-proton transfer reaction, the use of a 22 Å cutoff shows a close reaction energy profile and geometries of stationary structures with QM/MM-LREC compared to conventional QM/MM with no truncation. Geometry optimization of stationary structures for the hydrogen abstraction step by AlkB shows some differences between QM/MM-LREC and the conventional QM/MM. These differences underscore the necessity of the inclusion of the long-range electrostatic contribution.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Importance of imposing gauge invariance in time-dependent density functional theory calculations with meta-generalized gradient approximations

It has been known for more than a decade that the gauge variance of the kinetic energy density τ leads to additional terms in the magnetic orbital rotation Hessian used in linear-response time-dependent density functional theory (TDDFT), affecting excitation energies obtained with τ-dependent exchange–correlation functionals. While previous investigations found that a correction scheme based on the paramagnetic current density has a small effect on benchmark results, we report more pronounced effects here, in particular, for the popular M06-2X functional and for some other meta-generalized gradient approximations (mGGAs). In the first part of this communication, this is shown by a reassessment of a set of five Ni(II) complexes for which a previous benchmark study that did not impose gauge invariance has found surprisingly large errors for excitation energies obtained with M06-2X. These errors are more than halved by restoring gauge invariance. The variable importance of imposing gauge invariance for different mGGA-based functionals can be rationalized by the derivative of the mGGA exchange energy integrand with respect to τ. In the second part, a large set of valence excitations in small main-group molecules is analyzed. For M06-2X, several selected n → π* and π→π$^{*}_{⊥}$ excitations are heavily gauge-dependent with average changes of –0.17 and –0.28 eV, respectively, while π→π$^{*}_{∥}$ excitations are marginally affected (–0.04 eV). Similar patterns, but of the opposite signs, are found for SCAN0. Here, the results suggest that reevaluation of previous gauge variant TDDFT results based on M06-2X and other mGGA functionals is warranted.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

An Adaptive Derivative-based Method for Function Approximation

To alleviate the high computational cost of large-scale multi-physics simulations to study the relationships between the model parameters and the outputs of interest, response surfaces are often used in place of the exact functional relationships. This report explores a method for response surface construction using adaptive sampling guided by derivative information at each selected sample point. This method is especially suitable for applications that can readily provide added information such as gradients and Hessian with respect to the input parameters under study. When higher order terms (third and above) in the Taylor series are negligible, the approximation error for this method can be controlled. We present details of the adaptive algorithm and numerical results on a few test problems.

99 GENERAL AND MISCELLANEOUS↗

Advances in dual algorithms and convex approximation methods

A new algorithm for solving the duals of separable convex optimization problems is presented. The algorithm is based on an active set strategy in conjunction with a variable metric method. This first order algorithm is more reliable than Newton's method used in DUAL-2 because it does not break down when the Hessian matrix becomes singular or nearly singular. A perturbation technique is introduced in order to remove the nondifferentiability of the dual function which arises when linear constraints are present in the approximate problem.

Smaoui, H.↗

SCF Framework, HF Stability, and RPA Correlation for Jordan–Wigner-Transformed Spin Hamiltonians on Arbitrary Coupling Topologies

Mapping spins to fermions via the Jordan–Wigner (JW) transformation can render mean-field (Hartree–Fock, HF) descriptions effective for strongly correlated spin systems. As established in recent work, the application of such approaches is not limited by the nonlocal structure of JW strings or by site ordering because string operators can be absorbed into Thouless rotations of a Slater determinant, and the variational optimization of a unitary Lie-algebraic similarity transformation removes any ordering dependence. Leveraging these ideas, we develop a self-consistent field (SCF) scheme that expresses the mean-field energy as a functional of the single-particle density matrix, providing an alternative to gradient-based optimization of Thouless parameters. We derive the analytical orbital Hessian to diagnose HF stability and compute the ground-state correlation energy through the random-phase approximation (RPA). Benchmark results for the XXZ and J 1 –J 2 model on one- and two-dimensional lattices demonstrate that RPA significantly improves mean-field accuracy.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Electronically nonadiabatic wave packet propagation using frozen Gaussian scattering

We present an approach, which allows to employ the adiabatic wave packet propagation technique and semiclassical theory to treat the nonadiabatic processes by using trajectory hopping. The approach developed generates a bunch of hopping trajectories and gives all additional information to incorporate the effect of nonadiabatic coupling into the wave packet dynamics. This provides an interface between a general adiabatic frozen Gaussian wave packet propagation method and the trajectory surface hopping technique. The basic idea suggested in [A. D. Kondorskiy and H. Nakamura, J. Chem. Phys. 120, 8937 (2004)] is revisited and complemented in the present work by the elaboration of efficient numerical algorithms. We combine our approach with the adiabatic Herman-Kluk frozen Gaussian approximation. The efficiency and accuracy of the resulting method is demonstrated by applying it to popular benchmark model systems including three Tully’s models and 24D model of pyrazine. It is shown that photoabsorption spectrum is successfully reproduced by using a few hundreds of trajectories. We employ the compact finite difference Hessian update scheme to consider feasibility of the ab initio “on-the-fly” simulations. It is found that this technique allows us to obtain the reliable final results using several Hessian matrix calculations per trajectory.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Inexact Newton-CG algorithms with complexity guarantees

Abstract We consider variants of a recently developed Newton-CG algorithm for nonconvex problems (Royer, C. W. & Wright, S. J. (2018) Complexity analysis of second-order line-search algorithms for smooth nonconvex optimization. SIAM J. Optim., 28, 1448–1477) in which inexact estimates of the gradient and the Hessian information are used for various steps. Under certain conditions on the inexactness measures, we derive iteration complexity bounds for achieving $\epsilon $-approximate second-order optimality that match best-known lower bounds. Our inexactness condition on the gradient is adaptive, allowing for crude accuracy in regions with large gradients. We describe two variants of our approach, one in which the step size along the computed search direction is chosen adaptively, and another in which the step size is pre-defined. To obtain second-order optimality, our algorithms will make use of a negative curvature direction on some steps. These directions can be obtained, with high probability, using the randomized Lanczos algorithm. In this sense, all of our results hold with high probability over the run of the algorithm. We evaluate the performance of our proposed algorithms empirically on several machine learning models. Our approach is a first attempt to introduce inexact Hessian and/or gradient information into the Newton-CG algorithm of Royer & Wright (2018, Complexity analysis of second-order line-search algorithms for smooth nonconvex optimization. SIAM J. Optim., 28, 1448–1477).

Mathematics↗

An investigation of Newton-Sketch and subsampled Newton methods

Sketching, a dimensionality reduction technique, has received much attention in the statistics community. In this paper, we study sketching in the context of Newton's method for solving finite-sum optimization problems in which the number of variables and data points are both large. In this work, we study two forms of sketching that perform dimensionality reduction in data space: Hessian subsampling and randomized Hadamard transformations. Each has its own advantages, and their relative tradeoffs have not been investigated in the optimization literature. Additionally, our study focuses on practical versions of the two methods in which the resulting linear systems of equations are solved approximately, at every iteration, using an iterative solver. The advantages of using the conjugate gradient method vs. a stochastic gradient iteration are revealed through a set of numerical experiments, and a complexity analysis of the Hessian subsampling method is presented.

97 MATHEMATICS AND COMPUTING↗

The general theory of secondary weak gravitational lensing

Weak gravitational lensing is normally assumed to have only two principle effects: a magnification of a source and a distortion of the sources shape in the form of a shear. However, further distortions are actually present owing to changes in the gravitational field across the scale of the ray bundle of light propagating to us, resulting in the familiar arcs in lensed images. This is normally called the flexion, and is approximated by Taylor expanding the shear and magnification across the image plane. However, the physical origin of this effect arises from higher-order corrections in the geodesic deviation equation governing the gravitational force between neighbouring geodesics— so involves derivatives of the Riemann tensor. We show that integrating the second-order geodesic deviation equation results in a 'Hessian map' for gravitational lensing, which is a higher-order addition to the Jacobi map. We derive the general form of the Hessian map in an arbitrary spacetime paying particular attention to the separate effects of local Ricci versus non-local Weyl curvature. We then specialise to the case of a perturbed FLRW model, and give the general form of the Hessian for the first time. This has a host of new contributions which could in principle be used as tests for modified gravity.

79 ASTRONOMY AND ASTROPHYSICS↗

A Numerical Comparison of Barrier and Modified Barrier Methods for Large-Scale Bound-Constrained Optimization

When a classical barrier method is applied to the solution of a nonlinear programming problem with inequality constraints, the Hessian matrix of the barrier function becomes increasingly ill-conditioned as the solution is approached. As a result, it may be desirable to consider alternative numerical algorithms. We compare the performance of two methods motivated by barrier functions. The first is a stabilized form of the classical barrier method, where a numerically stable approximation to the Newton direction is used when the barrier parameter is small. The second is a modified barrier method where a barrier function is applied to a shifted form of the problem, and the resulting barrier terms are scaled by estimates of the optimal Lagrange multipliers. The condition number of the Hessian matrix of the resulting modified barrier function remains bounded as the solution to the constrained optimization problem is approached. Both of these techniques can be used in the context of a truncated-Newton method, and hence can be applied to large problems, as well as on parallel computers. In this paper, both techniques are applied to problems with bound constraints and we compare their practical behavior.

Nash, Stephen G.↗