Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “trust regions”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2026

Adaptive Sampling Trust Region Method for Bi-fidelity Simulation Optimization [SWR-25-166]

Adaptive Sampling Trust Region Method for Bi-fidelity Simulation Optimization aims to demonstrate the effect of adaptive sampling-based bi-fidelity stochastic trust region method (ASTRO-BFDF). ASTRO-BFDF, derived from a derivative-free adaptive sampling trust-region optimization (ASTRO-DF) (Shashaani et al. 2018, Ha and Shashaani 2023), intended to efficiently solve the bi-fidelity simulation optimization.

Mueller, Juliane [National Laboratory of the Rocki↗
DOE OSTI2025

Efficient proximal subproblem solvers for a nonsmooth trust-region method

In [R. J. Baraldi and D. P. Kouri, Mathematical Programming, (2022), pp. 1-40], we introduced an inexact trust-region algorithm for minimizing the sum of a smooth nonconvex and nonsmooth convex function. The principle expense of this method is in computing a trial iterate that satisfies the so-called fraction of Cauchy decrease condition—a bound that ensures the trial iterate produces sufficient decrease of the subproblem model. In this paper, we expound on various proximal trust-region subproblem solvers that generalize traditional trust-region methods for smooth unconstrained and convex-constrained problems. We introduce a simplified spectral proximal gradient solver, a truncated nonlinear conjugate gradient solver, and a dogleg method. Finally, we compare algorithm performance on examples from data science and PDE-constrained optimization.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

A randomized sketching trust-region secant method for low-memory dynamic optimization

The numerical solution of dynamic optimization problems is often limited by the memory required to store the state trajectory, which is used to evaluate the objective function and its derivatives. Recently, [R. Muthukumar et al., SIAM Journal on Optimization 31(2), pp. 1242–1275 (2021)] introduced a trust-region method for dynamic optimization that employs randomized sketching to compress the state trajectory, resulting in inexact derivative computations. By adaptively learning the sketch rank, the trust-region algorithm achieves rigorous convergence guarantees. Here, we extend this approach to use secant Hessian approximations. Due to the randomness introduced by the sketch, the traditional secant update formulae can produce poor Hessian approximations. In particular, the difference of two gradients, computed from two different sketches, may be inconsistent. To overcome this, we employ a sketched approximation of the Hessian application, in lieu of computing the gradient difference. We numerically demonstrate the improved stability of this approach on an example from PDE-constrained optimization.

dynamic optimization↗
DOE OSTI2025

Least H 2 norm updating of quadratic interpolation models for derivative-free trust-region algorithms

One particular class of derivative-free optimization algorithms is trust-region algorithms based on quadratic models given by the under-determined interpolation. Different techniques in updating the quadratic model from iteration to iteration will give different interpolation models. We propose a new way to update the quadratic model by minimizing the $H^{2}$ norm of the difference between neighboring quadratic models. The motivation for applying the $H^{2}$ norm is given. The theoretical properties of our new updating technique are also presented. We propose the projection in the sense of $H^{2}$ norm and the interpolation error analysis of our model function. We obtain the coefficients of the quadratic model function using the Karush–Kuhn–Tucker (KKT) conditions. Numerical results show the advantages of our model on the test set considered, and the derivative-free algorithms based on our least $H^{2}$ norm updating quadratic model functions can solve test problems with fewer function evaluations than the algorithm based on the least Frobenius norm updating model and the other compared methods.

derivative-free optimization↗
DOE OSTI2026

Adaptive Sampling-Based Bi-Fidelity Stochastic Trust Region Method for Stochastic Derivative-Free Optimization

Bi-fidelity stochastic optimization has gained increasing attention as an efficient approach to reduce computational costs by leveraging a low-fidelity (LF) model to optimize an expensive high-fidelity (HF) objective. In this paper, we propose ASTRO-BFDF, an adaptive sampling trust-region method specifically designed for unconstrained bi-fidelity stochastic derivative-free optimization problems. In ASTRO-BFDF, the LF function serves two purposes: (i) to identify better iterates for the HF function when the optimization process indicates a high correlation between them and (ii) to reduce the variance of the HF function estimates using bi-fidelity Monte Carlo (BFMC). The algorithm dynamically determines sample sizes while adaptively choosing between crude Monte Carlo and BFMC to balance the trade-off between optimization and sampling errors. We prove that the iterates generated by ASTRO-BFDF converge to a first-order stationary point almost surely. Additionally, we demonstrate the effectiveness of the proposed algorithm through numerical experiments on synthetic benchmarks and simulation optimization problems involving discrete event systems.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

A unified funnel restoration SQP algorithm

We consider nonlinearly constrained optimization problems and discuss a generic double-loop framework consisting of basic algorithmic ingredients that unifies a broad range of nonlinear optimization solvers. This framework has been implemented in the open-source solver Uno, a Swiss Army knife-like C++ optimization framework that unifies many nonlinearly constrained nonconvex optimization solvers. We illustrate the framework with a sequential quadratic programming (SQP) algorithm that maintains an acceptable upper bound on the constraint violation, called a funnel, that is monotonically decreased to control the feasibility of the iterates. Infeasible quadratic subproblems are handled by a feasibility restoration strategy. Globalization is controlled by a line search or a trust-region method. We prove global convergence of the trust-region funnel SQP method, building on known results from filter methods. We implement the algorithm in Uno, and we provide extensive test results for the trust-region line-search funnel SQP on small CUTEst instances.

Kiessling, David [Katholieke Univ. Leuven, Heverle↗
DOE OSTI2024

Two-Stage Estimation and Variance Modeling for Latency-Constrained Variational Quantum Algorithms

The quantum approximate optimization algorithm (QAOA) has enjoyed increasing attention in noisy, intermediate-scale quantum computing with its application to combinatorial optimization problems. QAOA has the potential to demonstrate a quantum advantage for NP-hard combinatorial optimization problems. As a hybrid quantum-classical algorithm, the classical component of QAOA resembles a simulation optimization problem in which the simulation outcomes are attainable only through a quantum computer. The simulation that derives from QAOA exhibits two unique features that can have a substantial impact on the optimization process: (i) the variance of the stochastic objective values typically decreases in proportion to the optimality gap, and (ii) querying samples from a quantum computer introduces an additional latency overhead. In this paper, we introduce a novel stochastic trust-region method derived from a derivative-free, adaptive sampling trust-region optimization method intended to efficiently solve the classical optimization problem in QAOA by explicitly taking into account the two mentioned characteristics. The key idea behind the proposed algorithm involves constructing two separate local models in each iteration: a model of the objective function and a model of the variance of the objective function. Exploiting the variance model allows us to restrict the number of communications with the quantum computer and also helps navigate the nonconvex objective landscapes typical in QAOA optimization problems. In conclusion, we numerically demonstrate the superiority of our proposed algorithm using the SimOpt library and Qiskit when we consider a metric of computational burden that explicitly accounts for communication costs.

Derivative-free Optimization↗
DOE OSTI2025

Machine learning models for PDE constrained optimization

Partial differential equation (PDE)-constrained optimization problems arise in a variety of scientific and engineering applications, such as topology optimization, electrodynamics, fluid dynamics, and structural dynamics. However, these problems are often challenging and computationally expensive to solve, due to the need to solve the PDEs within the optimization loop. One approach to reducing the computational cost of these methods while providing convergence guarantees is through inexact trust region methods; this method uses lower fidelity solutions of the PDE at early stages of the optimization and adjusts the required accuracy of inexact PDE solvers as the optimization progresses. In this work, we explore the use of machine learning based surrogate models with these inexact trust region methods. We first demonstrate the potential of this approach by using Gaussian processes as the surrogate model and test this on a simple PDE-constrained optimization problem. We then document explorations into improving the computational costs of evolutional deep neural network / neural Galerkin methods, with the eventual goal of using these methods with the inexact trust region algorithms. We are able to speed up these approaches, albeit at the cost of lower accuracy.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1989

Numerical optimization in Hilbert space using inexact function and gradient evaluations

Trust region algorithms provide a robust iterative technique for solving non-convex unstrained optimization problems, but in many instances it is prohibitively expensive to compute high accuracy function and gradient values for the method. Of particular interest are inverse and parameter estimation problems, since function and gradient evaluations involve numerically solving large systems of differential equations. A global convergence theory is presented for trust region algorithms in which neither function nor gradient values are known exactly. The theory is formulated in a Hilbert space setting so that it can be applied to variational problems as well as the finite dimensional problems normally seen in trust region literature. The conditions concerning allowable error are remarkably relaxed: relative errors in the gradient error condition is automatically satisfied if the error is orthogonal to the gradient approximation. A technique for estimating gradient error and improving the approximation is also presented.

Carter, Richard G.↗
NASA NTRS2012

Recursive Branching Simulated Annealing Algorithm

This innovation is a variation of a simulated-annealing optimization algorithm that uses a recursive-branching structure to parallelize the search of a parameter space for the globally optimal solution to an objective. The algorithm has been demonstrated to be more effective at searching a parameter space than traditional simulated-annealing methods for a particular problem of interest, and it can readily be applied to a wide variety of optimization problems, including those with a parameter space having both discrete-value parameters (combinatorial) and continuous-variable parameters. It can take the place of a conventional simulated- annealing, Monte-Carlo, or random- walk algorithm. In a conventional simulated-annealing (SA) algorithm, a starting configuration is randomly selected within the parameter space. The algorithm randomly selects another configuration from the parameter space and evaluates the objective function for that configuration. If the objective function value is better than the previous value, the new configuration is adopted as the new point of interest in the parameter space. If the objective function value is worse than the previous value, the new configuration may be adopted, with a probability determined by a temperature parameter, used in analogy to annealing in metals. As the optimization continues, the region of the parameter space from which new configurations can be selected shrinks, and in conjunction with lowering the annealing temperature (and thus lowering the probability for adopting configurations in parameter space with worse objective functions), the algorithm can converge on the globally optimal configuration. The Recursive Branching Simulated Annealing (RBSA) algorithm shares some features with the SA algorithm, notably including the basic principles that a starting configuration is randomly selected from within the parameter space, the algorithm tests other configurations with the goal of finding the globally optimal solution, and the region from which new configurations can be selected shrinks as the search continues. The key difference between these algorithms is that in the SA algorithm, a single path, or trajectory, is taken in parameter space, from the starting point to the globally optimal solution, while in the RBSA algorithm, many trajectories are taken; by exploring multiple regions of the parameter space simultaneously, the algorithm has been shown to converge on the globally optimal solution about an order of magnitude faster than when using conventional algorithms. Novel features of the RBSA algorithm include: 1. More efficient searching of the parameter space due to the branching structure, in which multiple random configurations are generated and multiple promising regions of the parameter space are explored; 2. The implementation of a trust region for each parameter in the parameter space, which provides a natural way of enforcing upper- and lower-bound constraints on the parameters; and 3. The optional use of a constrained gradient- search optimization, performed on the continuous variables around each branch s configuration in parameter space to improve search efficiency by allowing for fast fine-tuning of the continuous variables within the trust region at that configuration point.

Bolcar, Matthew↗
DOE OSTI2025

Domain Decomposition for Integer Optimal Control with Total Variation Regularization

Total variation integer optimal control problems admit solutions and necessary optimality conditions via geometric variational analysis. In spite of the existence of said solutions, algorithms which solve the discretized objective suffer from high numerical cost associated with the combinatorial nature of integer programming. Hence, such methods are often limited to small and medium-sized problems. We propose a globally convergent, coordinate descent–inspired algorithm that allows tractable subproblem solutions restricted to a partition of the domain. Our decomposition method solves relatively small trust-region subproblems that modify the control variable on a subdomain only. Given nontrivial subdomain overlap, we prove that a global first-order necessary optimality condition is equivalent to a first-order necessary optimality condition per subdomain. We additionally show that a sufficient decrease is achieved on a single subdomain by way of a trust-region subproblem solver using geometric measure–theoretic arguments, which we integrate with a greedy patch selection to prove convergence of our algorithm. In conclusion, we demonstrate the practicality of our algorithm on a benchmark large-scale, PDE-constrained integer optimal control problem and find that our method is faster than the state of the art.

domain decomposition↗
NASA NTRS1983

A new algorithm for constrained nonlinear least-squares problems, part 1

A Gauss-Newton algorithm is presented for solving nonlinear least squares problems. The problem statement may include simple bounds or more general constraints on the unknowns. The algorithm uses a trust region that allows the objective function to increase with logic for retreating to best values. The computations for the linear problem are done using a least squares system solver that allows for simple bounds and linear constraints. The trust region limits are defined by a box around the current point. In its current form the algorithm is effective only for problems with small residuals, linear constraints and dense Jacobian matrices. Results on a set of test problems are encouraging.

Hanson, R. J.↗
NASA NTRS1994

Multilevel algorithms for nonlinear optimization

Multidisciplinary design optimization (MDO) gives rise to nonlinear optimization problems characterized by a large number of constraints that naturally occur in blocks. We propose a class of multilevel optimization methods motivated by the structure and number of constraints and by the expense of the derivative computations for MDO. The algorithms are an extension to the nonlinear programming problem of the successful class of local Brown-Brent algorithms for nonlinear equations. Our extensions allow the user to partition constraints into arbitrary blocks to fit the application, and they separately process each block and the objective function, restricted to certain subspaces. The methods use trust regions as a globalization strategy, and they have been shown to be globally convergent under reasonable assumptions. The multilevel algorithms can be applied to all classes of MDO formulations. Multilevel algorithms for solving nonlinear systems of equations are a special case of the multilevel optimization methods. In this case, they can be viewed as a trust-region globalization of the Brown-Brent class.

Alexandrov, Natalia↗
DOE OSTI2026

RegularizedOptimization.jl: A Julia framework for regularized and nonsmooth optimization

RegularizedOptimization.jl is a Julia package that implements families of quadratic regularization and trust-region methods for solving the nonsmooth optimization problem $^{\textrm{minimize}}_{𝑥∈ℝ^𝑛}$ 𝑓(𝑥) + ℎ(𝑥) subject to 𝑐(𝑥) = 0, (1) where 𝑓 ∶ ℝ 𝑛 → ℝ and 𝑐 ∶ ℝ 𝑛 → ℝ 𝑚 are continuously differentiable, and ℎ ∶ ℝ 𝑛 → ℝ∪{+∞} is lower semi-continuous. The nonsmooth objective ℎ can be a regularizer, such as a sparsity inducing penalty, model simple constraints, such as 𝑥 belonging to a simple convex set, or can be a combination of both. All 𝑓, ℎ, and 𝑐 can be nonconvex. RegularizedOptimization.jl provides a modular and extensible framework for solving (1), and developing novel solvers. Currently, the following solvers are implemented: • Trust-region solvers TR and TRDH (Aravkin et al., 2022; Leconte & Orban, 2025) • Quadratic regularization solvers R2, R2DH and R2N (Aravkin et al., 2022; Diouane, Habiboullah, et al., 2024) • Levenberg-Marquardt solvers LM and LMTR (Aravkin et al., 2024) used when 𝑓 is a least-squares residual. • Augmented Lagrangian solver AL (De Marchi et al., 2023). All solvers rely on first derivatives of 𝑓 and 𝑐, and optionally on their second derivatives in the form of Hessian-vector products. If second derivatives are not available, quasi-Newton approximations can be used. In addition, the proximal mapping of the nonsmooth part ℎ, or adequate models thereof, must be evaluated. At each iteration, a step is computed by solving a subproblem of the form (1) inexactly, in which 𝑓, ℎ, and 𝑐 are replaced with appropriate models around the current iterate. The solvers R2, R2DH, and TRDH are particularly well suited to solve the subproblems, though they are general enough to solve (1). All solvers are allocation-free, so re-solves incur no additional allocations. To illustrate our claim of extensibility, a first version of the AL solver was implemented by an external contributor. Furthermore, a nonsmooth penalty approach, described in Diouane, Gollier, et al. (2024), is currently being developed, that relies on the library to efficiently solve the subproblems.

Gollier, Maxence [Polytechnique Montréal, QC (Cana↗