Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “nonconvex”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records

An inexact regularized Newton framework with a worst-case iteration complexity of $ {\mathscr O}(\varepsilon^{-3/2}) $ for nonconvex optimization

Abstract An algorithm for solving smooth nonconvex optimization problems is proposed that, in the worst-case, takes ${\mathscr O}(\varepsilon ^{-3/2})$ iterations to drive the norm of the gradient of the objective function below a prescribed positive real number $\varepsilon $ and can take ${\mathscr O}(\varepsilon ^{-3})$ iterations to drive the leftmost eigenvalue of the Hessian of the objective above $-\varepsilon $. The proposed algorithm is a general framework that covers a wide range of techniques including quadratically and cubically regularized Newton methods, such as the Adaptive Regularization using Cubics (arc) method and the recently proposed Trust-Region Algorithm with Contractions and Expansions (trace). The generality of our method is achieved through the introduction of generic conditions that each trial step is required to satisfy, which in particular allows for inexact regularized Newton steps to be used. These conditions center around a new subproblem that can be approximately solved to obtain trial steps that satisfy the conditions. A new instance of the framework, distinct from arc and trace, is described that may be viewed as a hybrid between quadratically and cubically regularized Newton methods. Numerical results demonstrate that our hybrid algorithm outperforms a cubically regularized Newton method.

Mathematics↗

Running Primal-Dual Gradient Method for Time-Varying Nonconvex Problems

This paper focuses on a time-varying constrained nonconvex optimization problem, and considers the synthesis and analysis of online regularized primal-dual gradient methods to track a Karush-Kuhn-Tucker (KKT) trajectory. The proposed regularized primal-dual gradient method is implemented in a running fashion, in the sense that the underlying optimization problem changes during the execution of the algorithms. In order to study its performance, we first derive its continuous-time limit as a system of differential inclusions. We then study sufficient conditions for tracking a KKT trajectory, and also derive asymptotic bounds for the tracking error (as a function of the time-variability of a KKT trajectory). Further, we provide a set of sufficient conditions for the KKT trajectories not to bifurcate or merge, and also investigate the optimal choice of the parameters of the algorithm. Illustrative numerical results for a time-varying nonconvex problem are provided.

differential inclusion↗

A nonsmooth nonconvex optimization algorithm for two-stage optimization problems

An optimization algorithm for a group of nonsmooth nonconvex problems inspired by two-stage stochastic programming problems is proposed. The main challenges for these problems include (1) the problems lack the popular lower-type properties such as prox-regularity assumed in many nonsmooth nonconvex optimization algorithms, (2) the objective can not be analytically expressed and (3) the evaluation of function values and subgradients are computationally expensive. To address these challenges, this report first examines the properties that exist in many two-stage problems, specifically upper-C 2 objectives. Then, we show that quadratic penalty method for securityconstrained alternating current optimal power flow (SCACOPF) contingency problems can make the contingency solution functions upper-C 2 . Based on these observations, a simplified bundle algorithm that bears similarity to sequential quadratic programming (SQP) method is proposed. It is more efficient in implementation and computation compared to conventional bundle methods. Global convergence analysis of the algorithm is presented under novel and reasonable assumptions. The proposed algorithm therefore fills the gap of theoretical convergence for smoothed SCACOPF problems. The inconsistency that might arise in our treatment of the constraints are addressed through a penalty algorithm whose convergence analysis is also provided. Finally, theoretical capabilities and numerical performance of the algorithm are demonstrated through numerical examples.

97 MATHEMATICS AND COMPUTING↗

Multistart algorithm for identifying all optima of nonconvex stochastic functions

Here, we propose a multistart algorithm to identify all local minima of a constrained, nonconvex stochastic optimization problem. The algorithm uniformly samples points in the domain and then starts a local stochastic optimization run from any point that is the "probabilistically best" point in its neighborhood. Under certain conditions, our algorithm is shown to asymptotically identify all local optima with high probability; this holds even though our algorithm is shown to almost surely start only finitely many local stochastic optimization runs. We demonstrate the performance of an implementation of our algorithm on nonconvex stochastic optimization problems, including identifying optimal variational parameters for the quantum approximate optimization algorithm.

97 MATHEMATICS AND COMPUTING↗

Benchmarking ADMM in Nonconvex NLPs

Here, we study connections between the alternating direction method of multipliers (ADMM), the classical method of multipliers (MM), and progressive hedging (PH). The connections are used to derive benchmark metrics and strategies to monitor and accelerate convergence and to help explain why ADMM and PH are capable of solving complex nonconvex NLPs. Specifically, we observe that ADMM is an inexact version of MM and approaches its performance when multiple coordination steps are performed. In addition, we use the observation that PH is a specialization of ADMM and borrow Lyapunov function and primal-dual feasibility metrics used in ADMM to explain why PH is capable of solving nonconvex NLPs. This analysis also highlights that specialized PH schemes can be derived to tackle a wider range of stochastic programs and even other problem classes. Our exposition is tutorial in nature and seeks to to motivate algorithmic improvements and new decomposition strategies.

42 ENGINEERING↗

A Feedback-Based Regularized Primal-Dual Gradient Method for Time-Varying Nonconvex Optimization: Preprint

This paper considers time-varying nonconvex optimization problems, utilized to model optimal operational trajectories of systems governed by possibly nonlinear physical or logical models. Algorithms for tracking a Karush-Kuhn-Tucker point are synthesized, based on a regularized primal-dual gradient method. In particular, the paper proposes a feedback-based primal-dual gradient algorithm, where analytical models for system state or constraints are replaced with actual measurements. When cost and constraints functions are twice continuously differentiable, conditions for the proposed algorithms to have bounded tracking error are derived, and a discussion of their practical implications is provided. Illustrative numerical simulations are presented for an application in power systems.

24 POWER TRANSMISSION AND DISTRIBUTION↗

A Bilevel Approach for Identifying the Worst Contingencies for Nonconvex Alternating Current Power Systems

We address the bilevel optimization problem of identifying the most critical attacks to an alternating current (AC) power flow network. The upper-level binary maximization problem consists of choosing an attack that is treated as a parameter in the lower-level defender minimization problem. Instances of the lower-level global minimization problem by themselves are NP-hard due to the nonconvex AC power flow constraints, and bilevel solution approaches commonly apply a convex relaxation or approximation to allow for tractable bilevel reformulations at the cost of underestimating some power system vulnerabilities. Our main contribution is to provide an alternative branch-and-bound algorithm whose upper bounding mechanism (in a maximization context) is based on a reformulation that avoids relaxation of the AC power flow constraints in the lower-level defender problem. Lower bounding is provided with semidefinite programming (SDP) relaxed solutions to the lower-level problem. We establish finite termination with guarantees of either a globally optimal solution to the original bilevel problem, or a globally optimal solution to the SDP-relaxed bilevel problem which is included in a vetted list of upper-level attack solutions, at least one of which is a globally optimal solution to the bilevel problem. We demonstrate through computational experiments applied to IEEE case instances both the relevance of our contribution, and the effectiveness of our contributed algorithm for identifying power system vulnerabilities without resorting to convex relaxations of the lower-level problem. We conclude with a discussion of future extensions and improvements.

97 MATHEMATICS AND COMPUTING↗

Convergence Properties of the Regularized Newton Method for the Unconstrained Nonconvex Optimization

The regularized Newton method (RNM) is one of the efficient solution methods for the unconstrained convex optimization. It is well-known that the RNM has good convergence properties as compared to the steepest descent method and the pure Newton's method. For example, Li, Fukushima, Qi and Yamashita showed that the RNM has a quadratic rate of convergence under the local error bound condition. Recently, Polyak showed that the global complexity bound of the RNM, which is the first iteration k such that -parallel {nabla}f(x{sub k})-parallel {<=}{epsilon}, is O({epsilon}{sup -4}), where f is the objective function and {epsilon} is a given positive constant. In this paper, we consider a RNM extended to the unconstrained 'nonconvex' optimization. We show that the extended RNM (E-RNM) has the following properties. (a) The E-RNM has a global convergence property under appropriate conditions. (b) The global complexity bound of the E-RNM is O({epsilon}{sup -2}) if {nabla}{sup 2}f is Lipschitz continuous on a certain compact set. (c) The E-RNM has a superlinear rate of convergence under the local error bound condition.

97 MATHEMATICS AND COMPUTING↗

An adaptive, multivariate partitioning algorithm for global optimization of nonconvex programs

We develop an adaptive, multivariate partitioning algorithm for solving nonconvex, Mixed-Integer Nonlinear Programs (MINLPs) with polynomial functions to global optimality. In particular, we present an iterative algorithm that exploits piecewise, convex relaxation approaches via disjunctive formulations to solve MINLPs that is different than conventional spatial branch-and-bound approaches. The algorithm partitions the domains of variables in an adaptive and non-uniform manner at every iteration to focus on productive areas of the search space. Furthermore, domain reduction techniques based on sequential, optimization-based bound-tightening and piecewise relaxation techniques, as a part of a presolve step, are integrated into the main algorithm. Finally, we demonstrate the effectiveness of the algorithm on well-known benchmark problems (including Pooling and Blending instances) from MINLPLib and compare our algorithm with state-of-the-art global optimization solvers. With our novel approach, we solve several large-scale instances, some of which are not solvable by state-of-the-art solvers. We also succeed in reducing the best known optimality gap for a hard, generalized pooling problem instance.

97 MATHEMATICS AND COMPUTING↗

Nonconvex regularization for sparse neural networks

Convex ℓ 1 regularization using an infinite dictionary of neurons has been suggested for constructing neural networks with desired approximation guarantees, but can be affected by an arbitrary amount of over-parametrization. This can lead to a loss of sparsity and result in networks with too many active neurons for the given data, in particular if the number of data samples is large. As a remedy, in this paper, a nonconvex regularization method is investigated in the context of shallow ReLU networks: We prove that in contrast to the convex approach, any resulting (locally optimal) network is finite even in the presence of infinite data (i.e., if the data distribution is known and the limiting case of infinite samples is considered). Moreover, here we show that approximation guarantees and existing bounds on the network size for finite data are maintained.

97 MATHEMATICS AND COMPUTING↗

Final Report---Optimization Under Nonconvexity and Uncertainty: Algorithms and Software

the goal of this work was to develop new algorithmic techniques for solving large-scale numerical optimization problems, focusing on problems classes that have proven to be among the most challenging for practitioners: those involving uncertainty and those involving nonconvexity. This research advanced the state-of-the-art in solving mixed integer linear programs containing symmetry, mixed integer nonlinear programs, and stochastic optimization problems. The focus of the work done in the continuation was on Mixed Integer Nonlinear Programs (MINLP)s and Mixed Integer Linear Programs (MILP)s, especially those containing a great deal of symmetry.

97 MATHEMATICS AND COMPUTING↗

Nonconvex Optimization and the High-Dimensional Geometry of Video [Slides]

Minimizing nonconvex penalty functions often produces better sparse solutions than ℓ 1 minimization. Using p-shrinkage and splitting allows this to be done very efficiently. Matrix decomposition methods can extract interesting features from high-dimensional data without explicit modeling. Important unsolved problems remain.

97 MATHEMATICS AND COMPUTING↗

Randomized Federated Learning Methods for Nonsmooth, Nonconvex, and Hierarchical Optimization (Final Technical Report)

This final technical report summarizes the outcomes of a DOE-funded project on federated scientific machine learning (FL) under nonsmooth, nonconvex, and hierarchical optimization settings. The project develops new mathematical models, algorithms, and theoretical guarantees for decentralized stochastic, bilevel, and minimax optimization problems arising in DOE mission-relevant applications. A unified framework of randomized and zeroth-order federated optimization methods is introduced, providing provable convergence, communication efficiency, and sample-complexity guarantees. The report documents algorithmic design, theoretical analysis, and empirical validation of the proposed federated learning methods. The project also contributes to workforce development through graduate training and dissemination of results via publications and seminars.

97 MATHEMATICS AND COMPUTING↗

The nonconvex multi-dimensional Riemann problem for Hamilton-Jacobi equations

Simple inequalities are presented for the viscosity solution of a Hamilton-Jacobi equation in N space dimensions when neither the initial data nor the Hamiltonian need be convex (or concave). The initial data are uniformly Lipschitz and can be written as the sum of a convex function in a group of variables and a concave function in the remaining variables, therefore including the nonconvex Riemann problem. The inequalities become equalities wherever a 'maxmin' equals a 'minmax', and thus a representation formula for this problem is obtained, generalizing the classical Hopi formulas.

Bardi, Martino↗

A Convexification-Based Outer-Approximation Method for Convex and Nonconvex MINLP

The advancement of domain reduction techniques has significantly enhanced the performance of solvers in mathematical programming. This paper delves into the impact of integrating convexification and domain reduction techniques within the Outer-Approximation method. We propose a refined convexification-based Outer-Approximation method alongside a Branch-and-Bound method for both convex and nonconvex Mixed-Integer Nonlinear Programming problems. These methods have been developed and incorporated into the open-source Mixed-Integer Nonlinear Decomposition Toolbox for Pyomo-MindtPy. Comprehensive benchmark tests were conducted, validating the effectiveness and reliability of our proposed algorithms. These tests highlight the improvements achieved by incorporating convexification and domain reduction techniques into the Outer-Approximation and Branch-and-Bound methods.

Optimization↗

Frequency extrapolation by nonconvex compressive sensing

Tomographic imaging modalities sample subjects with a discrete, finite set of measurements, while the underlying object function is continuous. Because of this, inversion of the imaging model, even under ideal conditions, necessarily entails approximation. The error incurred by this approximation can be important when there is rapid variation in the object function or when the objects of interest are small. In this work, we investigate this issue with the Fourier transform (FT), which can be taken as the imaging model for magnetic resonance imaging (MRl) or some forms of wave imaging. Compressive sensing has been successful for inverting this data model when only a sparse set of samples are available. We apply the compressive sensing principle to a somewhat related problem of frequency extrapolation, where the object function is represented by a super-resolution grid with many more pixels than FT measurements. The image on the super-resolution grid is obtained through nonconvex minimization. The method fully utilizes the available FT samples, while controlling aliasing and ringing. The algorithm is demonstrated with continuous FT samples of the Shepp-Logan phantom with additional small, high-contrast objects.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗