Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Nonconvex optimization”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5

Structural optimization with dynamic behavior constraints

The minimum weight optimum design of damped linearly elastic structural systems subjected to periodic loading with behavior constraints on maximum deflections and side constraints on design variables is addressed. Attention is focused on the two major impediments to an optimal solution: (1) the time parametric nature of the behavior constraints; and (2) the severe nonconvexity of the design space. A solution method based on upper bound approximations for the behavior constraints and an innovative mathematical programming scheme for seeking the optimal frequency subspace is set forth. Numerical results for several test problems illustrate the effectiveness of the method reported.

Mills-Curran, W. C.↗

Extended convex hull-based distributed optimal energy flow of integrated electricity-gas systems

Integrated electricity and gas systems are constructed to facilitate the gas-fired generation, and the distributed operation of these integrated systems have received much attention due to the increased emphasis on data security and privacy between different agencies. This paper proposes an extended convex hull based method to address optimal energy flow problems for the integrated electricity and gas systems in a distributed manner. First, a multi-block electricity-gas system model is constructed by dividing the whole system into N blocks considering both physical and regional differences. This multi-block model is then convexified by replacing the nonconvex gas transmission equation with the extended convex hull-based constraints. The Jacobi-Proximal alternating direction method of multipliers algorithm is adopted to solve the convexified model and minimize its operation cost. Finally, the feasibility of the optimal solution for the convexified model is checked, and a sufficient condition is developed. If the sufficient condition is satisfied, the optimal solution for the original nonconvex problem can be recovered from that for the convexified problem. Simulation results demonstrate that the proposed method is tractable and effective in obtaining feasible optimal solutions for multi-block optimal energy flow problems.

24 POWER TRANSMISSION AND DISTRIBUTION↗

A Hierarchical Volt-var Optimization with Discrete Variables in Unbalanced Distribution Systems

This paper proposes a framework to determine the optimal active and reactive power dispatch of distributed photovoltaic (PV) generation, switched capacitors, and voltage regulators in multi-phase unbalanced distribution systems. The objectives of the optimal dispatch are minimization of the energy loss, PV real power curtailment, and switching operations of capacitors and voltage regulators, in addition to elimination of voltage violation and reverse power flow. The optimization problem is formulated in rectangular coordinates as a nonlinear, nonconvex problem with discrete variables. A hierarchical twostage framework is proposed to effectively handle those discrete variables and reduce the computational time compared to the unified approach in which all variables are solved simultaneously. The efficacy of the proposed approach and the accuracy of the obtained numerical solution is validated using the unbalanced multi-phase IEEE 34-bus with 15-minute load and PV data.

Nguyen, Quan H.↗

On the Training and Generalization of Deep Operator Networks

Here, we present a novel training method for deep operator networks (DeepONets), one of the most popular neural network models for operators. DeepONets are constructed by two subnetworks, namely the branch and trunk networks. Typically, the two subnetworks are trained simultaneously, which amounts to solving a complex optimization problem in a high dimensional space. In addition, the nonconvex and nonlinear nature makes training very challenging. To tackle such a challenge, we propose a two-step training method that trains the trunk network first and then sequentially trains the branch network. The core mechanism is motivated by the divide-and-conquer paradigm and is the decomposition of the entire complex training task into two subtasks with reduced complexity. Therein the Gram–Schmidt orthonormalization process is introduced which significantly improves stability and generalization ability. On the theoretical side, we establish a generalization error estimate in terms of the number of training data, the width of DeepONets, and the number of input and output sensors. Numerical examples are presented to demonstrate the effectiveness of the two-step training method, including Darcy flow in heterogeneous porous media.

deep operator networks↗

Learning Optimal Solutions for Extremely Fast AC Optimal Power Flow

We develop, in this paper, a machine learning approach to optimize the real-time operation of electric power grids. In particular, we learn feasible solutions to the AC optimal power flow (OPF) problem with negligible optimality gaps. The AC OPF problem aims at identifying optimal operational conditions of the power grids that minimize power losses and/or generation costs. Due to the computational challenges with solving this nonconvex problem, many efforts have focused on linearizing or approximating the problem in order to solve the AC OPF on faster timescales. However, many of these approximations can be fairly poor representations of the actual system state and still require solving an optimization problem, which can be time consuming for large networks. In this work, we learn a mapping between the system loading and optimal generation values, enabling us to find near-optimal and feasible AC OPF solutions. This allows us to bypass solving the traditionally nonconvex AC OPF problem, resulting in a significant decrease in computational burden for grid operators.

machine learning↗

Learning Optimal Solutions for Extremely Fast AC Optimal Power Flow: Preprint

We develop, in this paper, a machine learning approach to optimize the real-time operation of electric power grids. In particular, we learn feasible solutions to the AC optimal power flow (OPF) problem with negligible optimality gaps. The AC OPF problem aims at identifying optimal operational conditions of the power grids that minimize power losses and/or generation costs. Due to the computational challenges with solving this nonconvex problem, many efforts have focused on linearizing or approximating the problem in order to solve the AC OPF on faster timescales. However, many of these approximations can be fairly poor representations of the actual system state and still require solving an optimization problem, which can be time consuming for large networks. In this work, we learn a mapping between the system loading and optimal generation values, enabling us to find near-optimal and feasible AC OPF solutions. This allows us to bypass solving the traditionally nonconvex AC OPF problem, resulting in a significant decrease in computational burden for grid operators.

machine learning↗

Assessing the Optimality of LinDist3Flow for Optimal Tap Selection of Step Voltage Regulators in Unbalanced Distribution Networks

The adoption of distributed energy resources such as photovoltaics (PVs) has increased dramatically during the previous decade. The increased penetration of PVs into distribution networks (DNs) can cause voltage fluctuations that have to be mitigated. One of the key utility assets employed to this end are step-voltage regulators (SVRs). It is desirable to include tap selection of SVRs in optimal power flow (OPF) routines, a task that turns out to be challenging because the resultant OPF problem is nonconvex with added complexities stemming from accurate SVR modeling. While several convex relaxations based on semi-definite programming (SDP) have been presented in the literature for optimal tap selection, SDP based schemes do not scale well and are challenging to implement in large-scale planning or operational frameworks. This paper deals with the optimal tap selection (OPTS) problem for wye-connected SVRs using linear approximations of power flow equations. Specifically, the LinDist3Flow model is adopted and the effective SVR ratio is assumed to be continuous- enabling the formulation of a problem called LinDist3Flow-OPTS, which amounts to a linear program. The scalability and optimality gap of LinDist3Flow-OPTS are evaluated with respect to existing SDP-based and nonlinear programming techniques for optimal tap selection in three standard feeders, namely, the IEEE 13-bus, 123-bus, and 8500-node DNs. For all DNs considered, LinDist3Flow-OPTS achieves an optimality gap of approximately 1% or less while significantly lowering the computational burden.

linear approximations↗

Assessing the Optimality of LinDist3Flow for Optimal Tap Selection of Step Voltage Regulators in Unbalanced Distribution Networks: Preprint

The adoption of distributed energy resources such as photovoltaics (PVs) has increased dramatically during the previous decade. The increased penetration of PVs into distribution networks (DNs) can cause voltage fluctuations that have to be mitigated. One of the key utility assets employed to this end are step-voltage regulators (SVRs). It is desirable to include tap selection of SVRs in optimal power flow (OPF) routines, a task that turns out to be challenging because the resultant OPF problem is nonconvex with added complexities stemming from accurate SVR modeling. While several convex relaxations based on semi-definite programming (SDP) have been presented in the literature for optimal tap selection, SDP based schemes do not scale well and are challenging to implement in large-scale planning or operational frameworks. This paper deals with the optimal tap selection (OPTS) problem for wye-connected SVRs using linear approximations of power flow equations. Specifically, the LinDist3Flow model is adopted and the effective SVR ratio is assumed to be continuous–enabling the formulation of a problem called LinDist3Flow-OPTS, which amounts to a linear program. The scalability and optimality gap of LinDist3Flow-OPTS are evaluated with respect to existing SDP-based and nonlinear programming techniques for optimal tap selection in three standard feeders, namely, the IEEE 13-bus, 123-bus, and 8500-node DNs. For all DNs considered, LinDist3Flow-OPTS achieves an optimality gap of approximately 1% or less while significantly lowering the computational burden.

linear approximations↗

Convergence of sum-up rounding schemes for cloaking problems governed by the Helmholtz equation

In this work, we consider the problem of designing a cloak for waves described by the Helmholtz equation from an integer programming point of view. The problem can be modeled as a PDE-constrained optimization problem with integer-valued control inputs that are distributed in the computational domain. A first-discretize-then-optimize approach results in a large-scale mixed-integer nonlinear program that is in general intractable because of the large number of integer variables that arise from the discretization of the domain. Instead, we propose an efficient algorithm that is able to approximate the local infima of the underlying nonconvex infinite-dimensional problem arbitrarily close without the need to solve the discretized finite-dimensional integer programs to optimality. We optimize only the continuous relaxations of the approximations for local minima and then apply the sum-up rounding methodology to obtain integer-valued controls. If the solutions of the discretized continuous relaxations converge to a local minimizer of the continuous relaxation, then the resulting discrete-valued control sequence converges weakly \(^*\) in \(L^\infty\) to the same local minimizer. These approximation properties follow under suitable refinements of the involved discretization grids. Our results use familiar concepts arising from the analytical properties of the underlying PDE and complement previous results, derived from a topology optimization point of view.

97 MATHEMATICS AND COMPUTING↗

Classical and neo-classical cruise-dash optimization

Cruise-dash flight performance is analyzed in the context of singular perturbations. Attention is given to the problem of determining an atmospheric flight path between given end points which minimizes a linear combination of time and fuel. It is shown that nonconvexity in the fuel-flow vs. airspeed graph has important consequences in optimum-cruise problems with time restrictions. Certain velocity regions are nonoptimal for cruise-dash and optimal cruise-dash sometimes requires time-shared operation between two altitude-airspeed points. Calculations are presented illustrating the occurrence of time-shared operation between two altitude-airspeed combinations for optimal cruise-dash.

Cliff, E. M.↗

Algebraic Multigrid with Filtering: An Efficient Preconditioner for Interior Point Methods in Large-Scale Contact Mechanics Optimization

Large-scale contact mechanics simulations are crucial in many engineering fields such as structural design and manufacturing. In the frictionless case, contact can be modeled by minimizing an energy functional; however, these problems are often nonlinear, nonconvex, and increasingly difficult to solve as mesh resolution increases. In this work, we employ a Newton-based interior-point (IP) filter line-search method, an effective approach for large-scale constrained optimization. While this method converges rapidly, each iteration requires solving a large saddle-point linear system that becomes ill-conditioned as the optimization process converges, largely due to IP treatment of the contact constraints. Such ill-conditioning can hinder solver scalability and increase iteration counts with mesh refinement. Here, to address this, we introduce a novel preconditioner, algebraic multigrid with filtering (AMGF), tailored to the Schur complement of the saddle-point system. Building on the classical AMG solver, commonly used for elasticity, we augment it with a specialized subspace correction that filters near null space components introduced by contact interface constraints. Through theoretical analysis and numerical experiments on a range of linear and nonlinear contact problems, we demonstrate that the proposed solver achieves mesh independent convergence and maintains robustness against the ill-conditioning that notoriously plagues IP methods. These results indicate that AMGF makes contact mechanics simulations more tractable and broadens the applicability of Newton-based IP methods in challenging engineering scenarios. More broadly, AMGF is well suited for problems, optimization or otherwise, where solver performance is limited by a low-dimensional subspace, such as those arising from localized constraints, interface conditions, or model heterogeneities. This makes the method widely applicable beyond contact mechanics and constrained optimization.

Mathematics and Computing↗

Convergence analysis for a nonlocal gradient descent method via directional Gaussian smoothing

We analyze the convergence of a nonlocal gradient descent method for minimizing a class of high-dimensional non-convex functions, where a directional Gaussian smoothing (DGS) is proposed to define the nonlocal gradient (also referred to as the DGS gradient). The method was first proposed in [Zhang et al., Enabling long-range exploration in minimization of multimodal functions, UAI 2021], in which multiple numerical experiments showed that replacing the traditional local gradient with the DGS gradient can help the optimizers escape local minima more easily and significantly improve their performance. However, a rigorous theory for the efficiency of the method on nonconvex landscape is lacking. In this work, we investigate the scenario where the objective function is composed of a convex function, perturbed by deterministic oscillating noise. We provide a convergence theory under which the iterates exponentially converge to a tightened neighborhood of the solution, whose size is characterized by the noise wavelength. Here, we also establish a correlation between the optimal values of the Gaussian smoothing radius and the noise wavelength, thus justifying the advantage of using moderate or large smoothing radii with the method. Furthermore, if the noise level decays to zero when approaching the global minimum, we prove that DGS-based optimization converges to the exact global minimum with linear rates, similarly to standard gradient-based methods in optimizing convex functions. Several numerical experiments are provided to confirm our theory and illustrate the superiority of the approach over those based on the local gradient.

Tran, Hoang [Oak Ridge National Laboratory (ORNL),↗

A Two-Stage Decomposition Approach for AC Optimal Power Flow

The alternating current optimal power flow (AC-OPF) problem is critical to power system operations and planning, but it is generally hard to solve due to its nonconvex and large-scale nature. Furthermore, this paper proposes a scalable decomposition approach in which the power network is decomposed into a master network and a number of subnetworks, where each network has its own AC-OPF subproblem. This formulates a two-stage optimization problem and requires only a small amount of communication between the master and subnetworks. The key contribution is a smoothing technique that renders the response of a subnetwork differentiable with respect to the input from the master problem, utilizing properties of the barrier problem formulation that naturally arises when subproblems are solved by a primal-dual interior-point algorithm. Consequently, existing efficient nonlinear programming solvers can be used for both the master problem and the subproblems. The advantage of this framework is that speedup can be obtained by processing the subnetworks in parallel, and it has convergence guarantees under reasonable assumptions. The formulation is readily extended to instances with stochastic subnetwork loads. Numerical results show favorable performance and illustrate the scalability of the algorithm which is able to solve instances with more than 11 million buses.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Lax-Oleinik-Type Formulas and Efficient Algorithms for Certain High-Dimensional Optimal Control Problems

Two of the main challenges in optimal control are solving problems with state-dependent running costs and developing efficient numerical solvers that are computationally tractable in high dimension. In this paper, we provide analytical solutions to certain optimal control problems whose running cost depends on the state variable and with constraints on the control. We also provide Lax-Oleinik-type representation formulas for the corresponding Hamilton-Jacobi partial differential equations with state-dependent Hamiltonians. Additionally, we present an efficient, grid-free numerical solver based on our representation formulas, which is shown to scale linearly with the state dimension, and thus, to overcome the curse of dimensionality. Using existing optimization methods and the min-plus technique, we extend our numerical solvers to address more general classes of convex and nonconvex initial costs. We demonstrate the capabilities of our numerical solvers using implementations on a central processing unit (CPU) and a field-programmable gate array (FPGA). In several cases, our FPGA implementation obtains over a 10 times speedup compared to the CPU, which demonstrates the promising performance boosts FPGAs can achieve. Furthermore, our numerical results show that our solvers have the potential to serve as a building block for solving broader classes of high-dimensional optimal control problems in real-time.

97 MATHEMATICS AND COMPUTING↗

Optimal symmetric flight studies

Several topics in optimal symmetric flight of airbreathing vehicles are examined. In one study, an approximation scheme designed for onboard real-time energy management of climb-dash is developed and calculations for a high-performance aircraft presented. In another, a vehicle model intermediate in complexity between energy and point-mass models is explored and some quirks in optimal flight characteristics peculiar to the model uncovered. In yet another study, energy-modelling procedures are re-examined with a view to stretching the range of validity of zeroth-order approximation by special choice of state variables. In a final study, time-fuel tradeoffs in cruise-dash are examined for the consequences of nonconvexities appearing in the classical steady cruise-dash model. Two appendices provide retrospective looks at two early publications on energy modelling and related optimal control theory.

Weston, A. R.↗

Exponential Decay in the Sensitivity Analysis of Nonlinear Dynamic Programming

In this report, we study the sensitivity of discrete-time dynamic programs with nonlinear dynamics and objective to perturbations in the initial conditions and reference parameters. Under uniform controllability and boundedness assumptions for the problem data, we prove that the directional derivative of the optimal state and control at time $k$, $x^*_k$, and $u^*_k$, with respect to the reference signal at time $i$, $d_i$, will have exponential decay in terms of $|k-i|$ with a decay rate $\rho$ independent of the temporal horizon length. The key technical step is to prove that a version of the convexification approach proposed by Verschueren et al. can be applied to the KKT conditions and results in a convex quadratic program with uniformly bounded data. In turn, Riccati techniques can be further employed to obtain the sensitivity result, borne from the observation that the directional derivatives are solutions of quadratic programs with structure similar to the KKT conditions themselves. We validate our findings with numerical experiments on a small nonlinear, nonconvex, dynamic program.

97 MATHEMATICS AND COMPUTING↗

Bayesian sparse learning with preconditioned stochastic gradient MCMC and its applications

Deep neural networks have been successfully employed in an extensive variety of research areas, including solving partial differential equations. Despite its significant success, there are some challenges in effectively training DNN, such as avoiding overfitting in over-parameterized DNNs and accelerating the optimization in DNNs with pathological curvature. Here, we propose a Bayesian type sparse deep learning algorithm. The algorithm utilizes a set of spike-and-slab priors for the parameters in the deep neural network. The hierarchical Bayesian mixture will be trained using an adaptive empirical method. That is, one will alternatively sample from the posterior using preconditioned stochastic gradient Langevin Dynamics (PSGLD), and optimize the latent variables via stochastic approximation. The sparsity of the network is achieved while optimizing the hyperparameters with adaptive searching and penalizing. A popular SG-MCMC approach is Stochastic gradient Langevin dynamics (SGLD). However, considering the complex geometry in the model parameter space in nonconvex learning, updating parameters using a universal step size in each component as in SGLD may cause slow mixing. To address this issue, we apply a computationally manageable preconditioner in the updating rule, which provides a step-size parameter to adapt to local geometric properties. Moreover, by smoothly optimizing the hyperparameter in the preconditioning matrix, our proposed algorithm ensures a decreasing bias, which is introduced by ignoring the correction term in the preconditioned SGLD. According to the existing theoretical framework, we show that the proposed algorithm can asymptotically converge to the correct distribution with a controllable bias under mild conditions. Numerical tests are performed on both synthetic regression problems and learning solutions of elliptic PDE, which demonstrate the accuracy and efficiency of the present work.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Mixed-Strategy Chance Constrained Optimal Control

This paper presents a novel chance constrained optimal control (CCOC) algorithm that chooses a control action probabilistically. A CCOC problem is to find a control input that minimizes the expected cost while guaranteeing that the probability of violating a set of constraints is below a user-specified threshold. We show that a probabilistic control approach, which we refer to as a mixed control strategy, enables us to obtain a cost that is better than what deterministic control strategies can achieve when the CCOC problem is nonconvex. The resulting mixed-strategy CCOC problem turns out to be a convexification of the original nonconvex CCOC problem. Furthermore, we also show that a mixed control strategy only needs to "mix" up to two deterministic control actions in order to achieve optimality. Building upon an iterative dual optimization, the proposed algorithm quickly converges to the optimal mixed control strategy with a user-specified tolerance.

Ono, Masahiro↗