Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Sequential quadratic programming”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

16 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

An adaptive stochastic sequential quadratic programming with differentiable exact augmented lagrangians

In this study, we consider solving nonlinear optimization problems with a stochastic objective and deterministic equality constraints. We assume for the objective that its evaluation, gradient, and Hessian are inaccessible, while one can compute their stochastic estimates by, for example, subsampling. We propose a stochastic algorithm based on sequential quadratic programming (SQP) that uses a differentiable exact augmented Lagrangian as the merit function. To motivate our algorithm design, we first revisit and simplify an old SQP method Lucidi developed for solving deterministic problems, which serves as the skeleton of our stochastic algorithm. Based on the simplified deterministic algorithm, we then propose a non-adaptive SQP for dealing with stochastic objective, where the gradient and Hessian are replaced by stochastic estimates but the stepsizes are deterministic and prespecified. Finally, we incorporate a recent stochastic line search procedure Paquette and Scheinberg into the non-adaptive stochastic SQP to adaptively select the random stepsizes, which leads to an adaptive stochastic SQP. The global "almost sure" convergence for both non-adaptive and adaptive SQP methods is established. Numerical experiments on nonlinear problems in CUTEst test set demonstrate the superiority of the adaptive algorithm.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

A Sequential Quadratic Programming Algorithm for Nonsmooth Problems with Upper- \({\boldsymbol{\mathcal{C}^2}}\) Objective

An optimization algorithm for nonsmooth nonconvex constrained optimization problems with upper- \({\boldsymbol{\mathcal{C}^2}}\) objective functions is proposed and analyzed. Upper- \({\boldsymbol{\mathcal{C}^2}}\) is a weakly concave property that exists in difference of convex (DC) functions and arises naturally in many applications, particularly certain classes of solutions to parametric optimization problems e.g., recourse of stochastic programming and projection onto closed sets. The algorithm can be viewed as an extension of sequential quadratic programming (SQP) to nonsmooth problems with upper- \({\boldsymbol{\mathcal{C}^2}}\) objectives or a simplified bundle method. It is globally convergent with bounded algorithm parameters that are updated with a trust-region criterion. The algorithm handles general smooth constraints through linearization and uses a line search to ensure progress. The potential inconsistencies from the linearization of the constraints are addressed through a penalty method. In conclusion, the capabilities of the algorithm are demonstrated by solving both simple upper- \({\boldsymbol{\mathcal{C}^2}}\) problems and a real-world optimal power flow problem used in current power grid industry practices.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

A Fast Temporal Decomposition Procedure for Long-Horizon Nonlinear Dynamic Programming

We propose a fast temporal decomposition procedure for solving long-horizon nonlinear dynamic programs. The core of the procedure is sequential quadratic programming (SQP) that utilizes a differentiable exact augmented Lagrangian as the merit function. Within each SQP iteration, we approximately solve the Newton system using an overlapping temporal decomposition strategy. We show that the approximate search direction is still a descent direction of the augmented Lagrangian provided the overlap size and penalty parameters are suitably chosen, which allows us to establish the global convergence. Moreover, we show that a unit step size is accepted locally for the approximate search direction and further establish a uniform, local linear convergence over stages. This local convergence rate matches the rate of the recent Schwarz scheme (Na et al. 2022). However, the Schwarz scheme has to solve nonlinear subproblems to optimality in each iteration, whereas we only perform a single Newton step instead. Numerical experiments validate our theories and demonstrate the superiority of our method.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

On the Convergence of Overlapping Schwarz Decomposition for Nonlinear Optimal Control

Here, we study the convergence properties of an overlapping Schwarz decomposition algorithm for solving nonlinear optimal control problems (OCPs). The algorithm decomposes the time domain into a set of overlapping subdomains, and solves all subproblems defined over subdomains in parallel. The convergence is attained by updating primal-dual information at the boundaries of overlapping subdomains. We show that the algorithm exhibits local linear convergence, and that the convergence rate improves exponentially with the overlap size. We also establish global convergence results for a general quadratic programming, which enables the application of the Schwarz scheme inside second-order optimization algorithms (e.g., sequential quadratic programming). The theoretical foundation of our convergence analysis is a sensitivity result of nonlinear OCPs, which we call "exponential decay of sensitivity" (EDS). Intuitively, EDS states that the impact of perturbations at domain boundaries (i.e., initial and terminal time) on the solution decays exponentially as one moves into the domain. Here, we expand a previous analysis available in the literature by showing that EDS holds for both primal and dual solutions of nonlinear OCPs, under uniform second-order sufficient condition, controllability condition, and boundedness condition. We conduct experiments with a quadrotor motion planning problem and a partial differential equations (PDE) control problem to validate our theory, and show that the approach is significantly more efficient than alternating direction method of multipliers and as efficient as the centralized interior-point solver.

42 ENGINEERING↗
DOE OSTI2022

A nonsmooth nonconvex optimization algorithm for two-stage optimization problems

An optimization algorithm for a group of nonsmooth nonconvex problems inspired by two-stage stochastic programming problems is proposed. The main challenges for these problems include (1) the problems lack the popular lower-type properties such as prox-regularity assumed in many nonsmooth nonconvex optimization algorithms, (2) the objective can not be analytically expressed and (3) the evaluation of function values and subgradients are computationally expensive. To address these challenges, this report first examines the properties that exist in many two-stage problems, specifically upper-C 2 objectives. Then, we show that quadratic penalty method for securityconstrained alternating current optimal power flow (SCACOPF) contingency problems can make the contingency solution functions upper-C 2 . Based on these observations, a simplified bundle algorithm that bears similarity to sequential quadratic programming (SQP) method is proposed. It is more efficient in implementation and computation compared to conventional bundle methods. Global convergence analysis of the algorithm is presented under novel and reasonable assumptions. The proposed algorithm therefore fills the gap of theoretical convergence for smoothed SCACOPF problems. The inconsistency that might arise in our treatment of the constraints are addressed through a penalty algorithm whose convergence analysis is also provided. Finally, theoretical capabilities and numerical performance of the algorithm are demonstrated through numerical examples.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

A unified funnel restoration SQP algorithm

We consider nonlinearly constrained optimization problems and discuss a generic double-loop framework consisting of basic algorithmic ingredients that unifies a broad range of nonlinear optimization solvers. This framework has been implemented in the open-source solver Uno, a Swiss Army knife-like C++ optimization framework that unifies many nonlinearly constrained nonconvex optimization solvers. We illustrate the framework with a sequential quadratic programming (SQP) algorithm that maintains an acceptable upper bound on the constraint violation, called a funnel, that is monotonically decreased to control the feasibility of the iterates. Infeasible quadratic subproblems are handled by a feasibility restoration strategy. Globalization is controlled by a line search or a trust-region method. We prove global convergence of the trust-region funnel SQP method, building on known results from filter methods. We implement the algorithm in Uno, and we provide extensive test results for the trust-region line-search funnel SQP on small CUTEst instances.

Kiessling, David [Katholieke Univ. Leuven, Heverle↗
DOE OSTI2026

Using Filter Methods to Guide Convergence for ADMM, with Applications to Nonnegative Matrix Factorization Problems

Nonconvex, nonlinear optimization problems arise naturally in parameter fitting and machine learning. While augmented Lagrangian methods have demonstrated robust convergence for classes of these problems, their convergence for block updates has been relatively unexplored outside of the context of the alternating direction method of multipliers (ADMM). ADMM has seen extensive use in these applications, but may exhibit uncertain convergence behavior in many practical nonconvex settings, and struggles with general nonlinear constraints. In contrast, filter methods have proved effective in enforcing convergence for sequential quadratic programming methods and interior point methods with feasibility criteria. We develop an ADMM-filter method for highly nonlinear and nonconvex problems. Here, we show convergence under mild assumptions for several types of coordinate descent schemes, and demonstrate our algorithm on nonnegative matrix factorization and completion problems in imaging and chemical spectrum analysis.

Nonconvex optimization↗
DOE OSTI2024

A fast and accurate domain decomposition nonlinear manifold reduced order model

Here, this paper integrates nonlinear-manifold reduced order models (NM-ROMs) with domain decomposition (DD). NM ROMs approximate the full order model (FOM) state in a nonlinear-manifold by training a shallow, sparse autoencoder using FOM snapshot data. These NM-ROMs can be advantageous over linear-subspace ROMs (LS-ROMs) for problems with slowly decaying Kolmogorov n-width. However, the number of NM-ROM parameters that need to be trained scales with the size of the FOM. Moreover, for “extreme-scale” problems, the storage of high-dimensional FOM snapshots alone can make ROM training expensive. To alleviate the training cost, this paper applies DD to the FOM, computes NM-ROMs on each subdomain, and couples them to obtain a global NM-ROM. This approach has several advantages: Subdomain NM-ROMs can be trained in parallel, involve fewer parameters to be trained than global NM-ROMs, require smaller subdomain FOM dimensional training data, and can be tailored to subdomain specific features of the FOM. The shallow, sparse architecture of the autoencoder used in each subdomain NM-ROM allows application of hyper-reduction (HR), reducing the complexity caused by nonlinearity and yielding computational speedup of the NM-ROM. This paper provides the first application of NM-ROM (with HR) to a DD problem. In particular, this paper details an algebraic DD reformulation of the FOM, training a NM-ROM with HR for each sub domain, and a sequential quadratic programming (SQP) solver to evaluate the coupled global NM-ROM. Theoretical convergence results for the SQP method and a priori and a posteriori error estimates for the DD NM-ROM with HR are provided. The proposed DD NM-ROM with HR approach is numerically compared to a DD LS-ROM with HR on the 2D steady-state Burgers’ equation, showing an order of magnitude improvement in accuracy of the proposed DD NM-ROM over the DD LS-ROM.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Fast model-based scenario optimization in NSTX-U enabled by analytic gradient computation

Model-based optimization offers a systematic approach to advanced scenario planning. In this case, the feedforward-control inputs (actuator trajectories) that are needed to attain and sustain a desired scenario are obtained by solving a nonlinear constrained optimization problem. This class of problems generally minimize a cost function that measures the difference between desired and actual plasma states. Several numerical optimization algorithms, such as sequential quadratic programming, require repeated calculation of the cost function gradients with respect to the input trajectories. Calculating these gradients numerically can be computationally intensive, increasing the time needed to solve the feedforward-control optimization problem. Here, this work introduces a method to analytically calculate these cost function gradients from the current profile evolution model. This can significantly reduce the computational time and allow for fast feedforward-control optimization, which would eventually enable optimal scenario planning between discharges. The performance of the feedforward optimizer with analytical gradients is compared to a traditional optimization algorithm based on numerical gradients for different NSTX-U scenarios. The plasma dynamics in the optimization algorithm are simulated using the Control Oriented Transport SIMulator (COTSIM). Results of the work show that analytical gradients consistently reduce the computation time while achieving trajectories that are comparable to those obtained by traditional optimization algorithms based on numerical gradients.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2022

Implicit shock tracking for unsteady flows by the method of lines

Here, a recently developed high-order implicit shock tracking (HOIST) framework for resolving discontinuous solutions of inviscid, steady conservation laws is extended to the unsteady case. Central to the framework is an optimization problem which simultaneously computes a discontinuity-aligned mesh and the corresponding high-order approximation to the flow, which provides nonlinear stabilization and a high-order approximation to the solution. This work extends the implicit shock tracking framework to the case of unsteady conservation laws using a method of lines discretization via a diagonally implicit Runge-Kutta method by “solving a steady problem at each timestep”. We formulate and solve an optimization problem that produces a feature-aligned mesh and solution at each Runge-Kutta stage of each timestep, and advance this solution in time by standard Runge-Kutta update formulas. A Rankine-Hugoniot based prediction of the shock location together with a high-order, untangling mesh smoothing procedure provides a high-quality initial guess for the optimization problem at each time, which results in rapid convergence of the sequential quadratic programing (SQP) optimization solver. This method is shown to deliver highly accurate solutions on coarse, high-order discretizations without nonlinear stabilization and recover the design accuracy of the Runge-Kutta scheme. We demonstrate this framework on a series of inviscid, unsteady conservation laws in both one- and two- dimensions. We also verify that our method is able to recover the design order of accuracy of our time integrator in the presence of a strong discontinuity.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2023

ALESQP: An Augmented Lagrangian Equality-Constrained SQP Method for Optimization with General Constraints

Here we present a new algorithm for infinite-dimensional optimization with general constraints, called ALESQP. In short, ALESQP is an augmented Lagrangian method that penalizes inequality constraints and solves equality-constrained nonlinear optimization subproblems at every iteration. The subproblems are solved using a matrix-free trust-region sequential quadratic programming (SQP) method that takes advantage of iterative, i.e., inexact linear solvers, and is suitable for large-scale applications. A key feature of ALESQP is a constraint decomposition strategy that allows it to exploit problem-specific variable scalings and inner products. We analyze convergence of ALESQP under different assumptions. We show that strong accumulation points are stationary. Consequently, in finite dimensions ALESQP converges to a stationary point. In infinite dimensions we establish that weak accumulation points are feasible in many practical situations. Under additional assumptions we show that weak accumulation points are stationary. We present several infinite-dimensional examples where ALESQP shows remarkable discretization-independent performance in all of its iterative components, requiring a modest number of iterations to meet constraint tolerances at the level of machine precision. Also, we demonstrate a fully matrix-free solution of an infinite-dimensional problem with nonlinear inequality constraints.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Domain-decomposition nonlinear manifold reduced order model

This software combines nonlinear-manifold reduced order models (NM-ROMs) with domain decomposition (DD) techniques. NM-ROMs, which utilize a shallow, sparse autoencoder trained with full order model (FOM) snapshot data, approximate the FOM state on a nonlinear manifold. These models offer advantages over linear-subspace ROMs (LS-ROMs) particularly in scenarios with slowly decaying Kolmogorov n-width. However, the training of NM-ROMs involves a number of parameters that scale with the size of the FOM, and storing high-dimensional FOM snapshots can significantly increase the cost of ROM training for extreme-scale problems. To mitigate these costs, the software employs DD to partition the FOM into smaller subdomains, computes NM-ROMs for each, and then integrates these to form a global NM-ROM. This strategy offers multiple benefits: it enables parallel training of subdomain NM-ROMs, reduces the number of parameters needed, decreases the dimensional requirements of subdomain FOM training data, and allows for customization to the unique characteristics of each FOM subdomain. The use of a shallow, sparse autoencoder architecture in each subdomain NM-ROM facilitates the application of hyper-reduction (HR), simplifying the nonlinear complexities and enhancing computational speed. This software marks the inaugural application of NM-ROM combined with HR to a DD problem. It features an algebraic DD reformulation of the FOM, training of NM-ROMs with HR for each subdomain, and employs a sequential quadratic programming (SQP) solver for the evaluation of the coupled global NMROM. The effectiveness of the DD NM-ROM with HR is numerically demonstrated on the 2D steady-state Burgers' equation, showing an order of magnitude improvement in accuracy over the DD LS-ROM with HR.

Diaz, AlejandroN↗
DOE OSTI2024

On optimizing the sensor spacing for pressure measurements on wind turbine airfoils

This research article presents a robust approach to optimizing the layout of pressure sensors around an airfoil. A genetic algorithm and a sequential quadratic programming algorithm are employed to derive a sensor layout best suited to represent the expected pressure distribution and, thus, the lift force. The fact that both optimization routines converge to almost identical sensor layouts suggests that an optimum exists and is reached. By comparing against a cosine-spaced sensor layout, it is demonstrated that the underlying pressure distribution can be captured more accurately with the presented layout optimization approach. Conversely, a 39 %–55 % reduction in the number of sensors compared to cosine spacing is achievable without loss in lift prediction accuracy. Given these benefits, an optimized sensor layout improves the data quality, reduces unnecessary equipment and saves cost in experimental setups. While the optimization routine is demonstrated based on the generic example of the IEA 15 MW reference wind turbine, it is suitable for a wide range of applications requiring pressure measurements around airfoils.

17 WIND ENERGY↗
DOE OSTI2026

Implementing a unified solver for nonlinearly constrained optimization

SQP and interior-point methods (also referred to as Lagrange-Newton methods) typically share key algorithmic components, such as strategies for computing descent directions and mechanisms that promote global convergence. Building on this insight, we introduce a unifying framework with eight building blocks that abstracts the workflows of Lagrange-Newton methods. We then present Uno, a modular C++ solver that implements our unifying framework and allows the automatic combination of a wide range of strategies with no programming effort from the user. Uno is meant to (1) organize mathematical optimization strategies into a coherent hierarchy; (2) offer a wide range of efficient and robust methods that can be compared for a given instance; (3) enable researchers to experiment with novel optimization strategies; and (4) reduce the cost of development and maintenance of multiple optimization solvers. Uno’s software design allows user to compose new customized solvers for emerging optimization areas such as robust optimization or optimization problems with complementarity constraints, while building on reliable nonlinear optimization techniques. We demonstrate that Uno is highly competitive against state-of-the-art solvers filterSQP, IPOPT, SNOPT, MINOS, LANCELOT, LOQO, and CONOPT on a subset of 429 small problems from the CUTE collection. Uno is available as open-source software under the MIT license at https://github.com/cvanaret/Uno and via its C, Julia, Python, Fortran, and AMPL interfaces.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Sequential Linearization Method for Bound-Constrained Mathematical Programs with Complementarity Constraints

Here, we propose an algorithm for solving bound-constrained mathematical programs with complementarity constraints on the variables. Each iteration of the algorithm involves solving a linear program with complementarity constraints in order to obtain an estimate of the active set. The algorithm enforces descent on the objective function to promote global convergence to B-stationary points. We provide a convergence analysis and preliminary numerical results on a range of test problems. We also study the effect of fixing the active constraints in a bound-constrained quadratic program that can be solved on each iteration in order to obtain fast convergence.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Cybersecurity Enhancement for Multi-Infeed High-Voltage DC Systems

Composed of multiple two-terminal high-voltage DC (HVDC) transmission systems, a multi-infeed HVDC (MIDC) system exchanges massive power among multiple asynchronous AC systems. However, as an intrinsically cyber-physical system, an MIDC system could suffer from cyber-attacks, leading to massive power mismatches in multiple AC systems, and resulting in catastrophic consequences. Since the sequential responses of an MIDC system and interconnected AC systems are in different timescales, this paper first establishes a two-timescale model to evaluate the sequential impacts caused by cyber-attacks. Then, an event-triggered cyber-defense strategy is proposed to enhance the cybersecurity of an MIDC system by mitigating multiple non-simultaneous cyber-attacks. Whenever new cyber-attack events occur, the proposed cyber-defense strategy, which is mathematically modeled as a mixed-integer quadratic programming problem, is executed on-line and updated in an event-triggered manner. Here, simulation results on an MIDC system demonstrate that the low-cost and almost blind cyber-attacks can cause severe frequency deviations, and the proposed strategy can mitigate multi-cyber-attacks effectively.

24 POWER TRANSMISSION AND DISTRIBUTION↗