Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Newton Method”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Bound Constrained Partial DifferentialEquation Inverse Problem Solution by theSemi-Smooth Newton Method

We present the mathematical derivation, software implementation details, and computational results for a semi-smooth Newton method applied to two inverse problems governed by partial differential equations with bound constraints. The two problems share mathematical structural similarities to density-based topology optimization problems. The semi-smooth Newton method provides a mesh independent solution computation for the two test problems. A key step is that the complementarity part of the necessary optimality conditions are reformulated with the use of a complementarity functionφsuch that the complementarity conditions are satisfied if and only if a zero of a nonsmooth function has been obtained. The modular finite element package MFEM is utilized for the software implementation. In addition we constructed a matrix-free Operator to enable the use of efficient Krylov subspace IterativeSolver of MFEM for the solution of our two target problems.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Advanced Newton methods for geodynamical models of Stokes flow with viscoplastic rheologies

Strain localization and resulting plasticity and failure play an important role in the evolution of the lithosphere. These phenomena are commonly modeled by Stokes flows with viscoplastic rheologies. The nonlinearities of these rheologies make the numerical solution of the resulting systems challenging, and iterative methods often converge slowly or not at all. Yet accurate solutions are critical for representing the physics. Moreover, for some rheology laws, aspects of solvability are still unknown. We study a basic but representative viscoplastic rheology law. The law involves a yield stress that is independent of the dynamic pressure, referred to as von Mises yield criterion. Two commonly used variants, perfect/ideal and composite viscoplasticity, are compared. We derive both variants from energy minimization principles, and we use this perspective to argue when solutions are unique. We propose a new stress-velocity Newton solution algorithm that treats the stress as an independent variable during the Newton linearization but requires solution only of Stokes systems that are of the usual velocity-pressure form. To study different solution algorithms, we implement 2-D and 3-D finite element discretizations, and we generate Stokes problems with up to 7 orders of magnitude viscosity contrasts, in which compression or tension results in significant nonlinear localization effects. Comparing the performance of the proposed Newton method with the standard Newton method and the Picard fixed-point method, we observe a significant reduction in the number of iterations and improved stability with respect to problem nonlinearity, mesh refinement, and the polynomial order of the discretization.

Rudi, Johann↗
DOE OSTI2026

A Stochastic Quasi-Newton Method in the Absence of Common Random Numbers

We present Q-SASS, a quasi-Newton method for unconstrained stochastic optimization that does not rely on common random numbers. Most existing quasi-Newton approaches leverage common random numbers to construct second-order updates. However, motivated by challenges in variational quantum algorithms—where such coordination is not possible—we consider the setting in which function values and gradients are accessible only through noisy probabilistic zeroth- and first-order oracles, and no common random numbers can be exploited. We derive high-probability tail bounds on the iteration complexity of our algorithm for nonconvex, convex, and strongly convex (more generally, those satisfying the PL condition) objective functions. Finally, we demonstrate the empirical benefits of our quasi-Newton updating scheme on both synthetic and quantum chemistry problems.

Complexity bound↗
DOE OSTI2023

Parameter identification for symmetrical Prandtl-Ishlinskii hysteresis model using Gauss-Newton method

The Prandtl-Ishlinskii (PI) hysteresis model has been used to describe nonlinear hysteretic behaviors. In this paper, the PI model is first simplified to accommodate the symmetrical behavior of ferromagnetic hysteresis. A parameter identification method is then developed to extract the parameters of the symmetrical PI model from experimental data. Based on the Gauss-Newton method, regularization technique, and line search method, the model parameters can be retrieved efficiently from the measured data. Numerical examples are presented to demonstrate the performance of the proposed method.

Yan, Su↗
DOE OSTI2023

Adaptive sampling quasi-Newton methods for zeroth-order stochastic optimization

Here, we consider unconstrained stochastic optimization problems with no available gradient information. Such problems arise in settings from derivative-free simulation optimization to reinforcement learning. We propose an adaptive sampling quasi-Newton method where we estimate the gradients using finite differences of stochastic function evaluations within a common random number framework. We develop modified versions of a norm test and an inner product quasi-Newton test to control the sample sizes used in the stochastic approximations and provide global convergence results to the neighborhood of a locally optimal solution. We present numerical experiments on simulation optimization problems to illustrate the performance of the proposed algorithm. When compared with classical zeroth-order stochastic gradient methods, we observe that our strategies of adapting the sample sizes significantly improve performance in terms of the number of stochastic function evaluations required.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

An inexact semismooth Newton method with application to adaptive randomized sketching for dynamic optimization

In many applications, one can only access the inexact gradients and inexact hessian times vector products. Thus it is essential to consider algorithms that can handle such inexact quantities with a guaranteed convergence to solution. An inexact adaptive and provably convergent semismooth Newton method is considered to solve constrained optimization problems. In particular, dynamic optimization problems, which are known to be highly expensive, are the focus. A memory efficient semismooth Newton algorithm is introduced for these problems. The source of efficiency and inexactness is the randomized matrix sketching. Further, applications to optimization problems constrained by partial differential equations are also considered.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

An investigation of Newton-Sketch and subsampled Newton methods

Sketching, a dimensionality reduction technique, has received much attention in the statistics community. In this paper, we study sketching in the context of Newton's method for solving finite-sum optimization problems in which the number of variables and data points are both large. In this work, we study two forms of sketching that perform dimensionality reduction in data space: Hessian subsampling and randomized Hadamard transformations. Each has its own advantages, and their relative tradeoffs have not been investigated in the optimization literature. Additionally, our study focuses on practical versions of the two methods in which the resulting linear systems of equations are solved approximately, at every iteration, using an iterative solver. The advantages of using the conjugate gradient method vs. a stochastic gradient iteration are revealed through a set of numerical experiments, and a complexity analysis of the Hessian subsampling method is presented.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

A Scalable Interior‐Point Gauss–Newton Method for PDE‐Constrained Optimization With Bound Constraints

Here, we present a scalable approach to solve a class of partial differential equation (PDE)‐constrained optimization problems with bound constraints. This approach utilizes a robust full‐space interior‐point (IP)‐Gauss–Newton optimization method. To cope with the poorly‐conditioned IP‐Gauss–Newton saddle‐point linear systems that need to be solved approximately, once per optimization step, we propose two spectrally related preconditioners. These preconditioners leverage the limited informativeness of data in regularized PDE‐constrained optimization problems. A block Gauss–Seidel preconditioner is proposed for the GMRES‐based solution of the IP‐Gauss–Newton linear systems. It is shown, for a large‐class of PDE‐ and bound‐constrained optimization problems, that the spectrum of the block Gauss–Seidel preconditioned IP‐Gauss–Newton matrix is asymptotically independent of discretization and is not impacted by the ill‐conditioning that notoriously plagues interior‐point methods. We exploit symmetry of the IP‐Gauss–Newton linear systems and propose a regularization and log‐barrier Hessian preconditioner for the preconditioned conjugate gradient (PCG)‐based solution of the equivalent IP‐Gauss–Newton–Schur complement linear systems. The eigenvalues of the block Gauss–Seidel preconditioned IP‐Gauss–Newton matrix, that are not equal to one, are identical to the eigenvalues of the regularization and log‐barrier Hessian preconditioned Schur complement matrix. The scalability of the approach is demonstrated on two example problems. The numerical solution of these optimization problems is shown to require a discretization independent number of IP‐Gauss–Newton linear solves. Furthermore, the linear systems are solved in a discretization and IP ill‐conditioning independent number of preconditioned Krylov subspace iterations. The parallel scalability of the preconditioner, achieved via algebraic multigrid component solvers when applicable, and the aforementioned algorithmic scalability permits a parallel scalable means to compute solutions of a large class of PDE‐ and bound‐constrained problems.

PDE-constrained optimization↗
DOE OSTI2023

Higher-Order Corrections to Optimisers based on Newton's Method

The Newton, Gauss–Newton and Levenberg–Marquardt methods all use the first derivative of a vector function (the Jacobian) to minimise its sum of squares. When the Jacobian matrix is ill-conditioned, the function varies much faster in some directions than others and the space of possible improvement in sum of squares becomes a long narrow ellipsoid in the linear model. This means that even a small amount of nonlinearity in the problem parameters can cause a proposed point far down the long axis of the ellipsoid to fall outside of the actual curved valley of improved values, even though it is quite nearby. This paper presents a differential equation that ‘follows’ these valleys, based on the technique of geodesic acceleration, which itself provides a 2 nd order improvement to the Levenberg–Marquardt iteration step. Higher derivatives of this equation are computed that allow n th order improvements to the optimisation methods to be derived. These higher-order accelerated methods up to 4 th order are tested numerically and shown to provide substantial reduction of both number of steps and computation time.

43 PARTICLE ACCELERATORS↗
DOE OSTI2020

Solving Coupled Cluster Equations by the Newton Krylov Method

We describe using the Newton Krylov method to solve the coupled cluster equation. The method uses a Krylov iterative method to compute the Newton correction to the approximate coupled cluster amplitude. The multiplication of the Jacobian with a vector, which is required in each step of a Krylov iterative method such as the Generalized Minimum Residual (GMRES) method, is carried out through a finite difference approximation, and requires an additional residual evaluation. The overall cost of the method is determined by the sum of the inner Krylov and outer Newton iterations. We discuss the termination criterion used for the inner iteration and show how to apply pre-conditioners to accelerate convergence. We will also examine the use of regularization technique to improve the stability of convergence and compare the method with the widely used direct inversion of iterative subspace (DIIS) methods through numerical examples.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2023

Solution and sensitivity analysis of nonlinear equations using a hypercomplex-variable Newton-Raphson method

Here, the classical Newton-Raphson (NR) method for solving nonlinear equations is enhanced in two ways through the use of hypercomplex variables and algebra. In particular, i) the Jacobian is computed in a highly accurate and automated way, and ii) the derivative of the solution to the nonlinear equations is computed with respect to any parameter contained within the system of equations. These advances provide two significant enhancements in that it is straightforward to provide an accurate Jacobian and to construct a reduced order model (ROM) of arbitrary order with respect to any parameter of the system. The ROM can then be used to approximate the solution for other parameter values without requiring additional solutions of the nonlinear equations. Several case studies are presented including 1D and 2D academic examples with fully functioning Python code provided. Additionally, a case of study of the catenary of an elastic cable subject to its own weight and a vertical point load. Derivatives up to 10th order were computed with respect to material, loading, and geometrical parameters. The derivatives were used to generate reduced order models of the cable deformation and reaction forces at its ends with respect to multiple input parameters. Results show that from a single hypercomplex evaluation of the cable under a single vertical point load, it is possible to generate an accurate reduced order model capable of predicting the cable deformation with 1.5 times the load in the opposite direction and with 3.5 times the load in the same direction without resolving the system of equations.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Enhancing ACPF Analysis: Integrating Newton-Raphson Method with Gradient Descent and Computational Graphs

This paper presents a new method for enhancing Alternating Current Power Flow (ACPF) analysis. The method integrates the Newton-Raphson (NR) method with Enhanced-Gradient Descent (GD) and computational graphs. The integration of renewable energy sources in power systems introduces variability and unpredictability, and this method addresses these challenges. It leverages the robustness of NR for accurate approximations and the flexibility of GD for handling variable conditions, all without requiring Jacobian matrix inversion. Furthermore, computational graphs provide a structured and visual framework that simplifies and systematizes the application of these methods. The goal of this fusion is to overcome the limitations of traditional ACPF methods and improve the resilience, adaptability, and efficiency of modern power grid analyses. We validate the effectiveness of our advanced algorithm through comprehensive testing on established IEEE benchmark systems. Furthermore, our findings demonstrate that our approach not only speeds up the convergence process but also ensures consistent performance across diverse system states, representing a significant advancement in power flow computation.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2020

A log-barrier Newton-CG method for bound constrained optimization with complexity guarantees

Abstract We describe an algorithm based on a logarithmic barrier function, Newton’s method and linear conjugate gradients that seeks an approximate minimizer of a smooth function over the non-negative orthant. We develop a bound on the complexity of the approach, stated in terms of the required accuracy and the cost of a single gradient evaluation of the objective function and/or a matrix-vector multiplication involving the Hessian of the objective. The approach can be implemented without explicit calculation or storage of the Hessian.

O’Neill, Michael↗
DOE OSTI2026

Implementing a unified solver for nonlinearly constrained optimization

SQP and interior-point methods (also referred to as Lagrange-Newton methods) typically share key algorithmic components, such as strategies for computing descent directions and mechanisms that promote global convergence. Building on this insight, we introduce a unifying framework with eight building blocks that abstracts the workflows of Lagrange-Newton methods. We then present Uno, a modular C++ solver that implements our unifying framework and allows the automatic combination of a wide range of strategies with no programming effort from the user. Uno is meant to (1) organize mathematical optimization strategies into a coherent hierarchy; (2) offer a wide range of efficient and robust methods that can be compared for a given instance; (3) enable researchers to experiment with novel optimization strategies; and (4) reduce the cost of development and maintenance of multiple optimization solvers. Uno’s software design allows user to compose new customized solvers for emerging optimization areas such as robust optimization or optimization problems with complementarity constraints, while building on reliable nonlinear optimization techniques. We demonstrate that Uno is highly competitive against state-of-the-art solvers filterSQP, IPOPT, SNOPT, MINOS, LANCELOT, LOQO, and CONOPT on a subset of 429 small problems from the CUTE collection. Uno is available as open-source software under the MIT license at https://github.com/cvanaret/Uno and via its C, Julia, Python, Fortran, and AMPL interfaces.

97 MATHEMATICS AND COMPUTING↗