Probability maximization via Minkowski functionals: convex representations and tractable resolution
Not provided.
Computer Science↗
SEARCH · Engineering Papers
Results for “Convex functions”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Convergence analysis for a nonlocal gradient descent method via directional Gaussian smoothing
We analyze the convergence of a nonlocal gradient descent method for minimizing a class of high-dimensional non-convex functions, where a directional Gaussian smoothing (DGS) is proposed to define the nonlocal gradient (also referred to as the DGS gradient). The method was first proposed in [Zhang et al., Enabling long-range exploration in minimization of multimodal functions, UAI 2021], in which multiple numerical experiments showed that replacing the traditional local gradient with the DGS gradient can help the optimizers escape local minima more easily and significantly improve their performance. However, a rigorous theory for the efficiency of the method on nonconvex landscape is lacking. In this work, we investigate the scenario where the objective function is composed of a convex function, perturbed by deterministic oscillating noise. We provide a convergence theory under which the iterates exponentially converge to a tightened neighborhood of the solution, whose size is characterized by the noise wavelength. Here, we also establish a correlation between the optimal values of the Gaussian smoothing radius and the noise wavelength, thus justifying the advantage of using moderate or large smoothing radii with the method. Furthermore, if the noise level decays to zero when approaching the global minimum, we prove that DGS-based optimization converges to the exact global minimum with linear rates, similarly to standard gradient-based methods in optimizing convex functions. Several numerical experiments are provided to confirm our theory and illustrate the superiority of the approach over those based on the local gradient.
Tran, Hoang [Oak Ridge National Laboratory (ORNL),↗
A proximal trust-region method for nonsmooth optimization with inexact function and gradient evaluations
Many applications require minimizing the sum of smooth and nonsmooth functions. For example, basis pursuit denoising problems in data science require minimizing a measure of data misfit plus an $\ell^1$-regularizer. Similar problems arise in the optimal control of partial differential equations (PDEs) when sparsity of the control is desired. Here, we develop a novel trust-region method to minimize the sum of a smooth nonconvex function and a nonsmooth convex function. Our method is unique in that it permits and systematically controls the use of inexact objective function and derivative evaluations. When using a quadratic Taylor model for the trust-region subproblem, our algorithm is an inexact, matrix-free proximal Newton-type method that permits indefinite Hessians. We prove global convergence of our method in Hilbert space and demonstrate its efficacy on three examples from data science and PDE-constrained optimization.
97 MATHEMATICS AND COMPUTING↗
Memory-efficient nonsmooth dynamic optimization using adaptive randomized compression
Dynamic optimization problems arise in many applications including flow control, full waveform inversion, and medical imaging. These problems are plagued by significant computational challenges. One such challenge — and the focus of this work — is the memory limitation induced by the size of the underlying dynamical system. In particular, the entire dynamic trajectory is required for derivative computation and therefore must be stored or recomputed using, e.g., checkpointing. Although recent work demonstrated the use of adaptive randomized sketching to overcome the memory challenge, that work only applies to smooth unconstrained problems, prohibiting its use for nonsmooth regularized and constrained problems. The inclusion of nonsmooth regularizers and constraints is critical as they often arise in an attempt to preserve certain physical properties or to promote sparsity. To solve these problems, we introduce a trust-region algorithm for minimizing the sum of a smooth nonconvex function and a nonsmooth convex function that leverages randomized sketching to compress the dynamical system trajectories and adaptively adjust the sketch rank to satisfy a gradient inexactness condition. We prove convergence of this algorithm and demonstrate that it achieves substantial memory reduction on three discretized PDE-constrained optimization applications.
97 MATHEMATICS AND COMPUTING↗
Local convergence analysis of an inexact trust-region method for nonsmooth optimization
In Baraldi, we introduced an inexact trust-region algorithm for minimizing the sum of a smooth nonconvex function and a nonsmooth convex function in Hilbert space—a class of problems that is ubiquitous in data science, learning, optimal control, and inverse problems. Furthermore, this algorithm has demonstrated excellent performance and scalability with problem size. In this paper, we enrich the convergence analysis for this algorithm, proving strong convergence of the iterates with guaranteed rates. In particular, we demonstrate that the trust-region algorithm recovers superlinear, even quadratic, convergence rates when using a second-order Taylor approximation of the smooth objective function term.
97 MATHEMATICS AND COMPUTING↗
Closed-Form Approximation of the Total Variation Proximal Operator
Total variation (TV) is a widely used function for regularizing imaging inverse problems that is particularly appropriate for images whose underlying structure is piecewise constant. TV regularized optimization problems are typically solved using proximal methods, but the way in which they are applied is constrained by the absence of a closed-form expression for the proximal operator of the TV function. A closed-form approximation of the TV proximal operator has previously been proposed, but its accuracy was not theoretically explored in detail. Here, we address this gap by making several new theoretical contributions, proving that the approximation leads to a proximal operator of some convex function, it is equivalent to a gradient descent step on a smoothed version of TV, and that its error can be fully characterized and controlled with its scaling parameter. We experimentally validate our theoretical results on image denoising and sparse-view computed tomography (CT) image reconstruction.
97 MATHEMATICS AND COMPUTING↗
Exploring the strong-coupling region of SU( N ) Seiberg-Witten theory
We consider the Seiberg-Witten solution of pure N = 2 gauge theory in four dimensions, with gauge group SU(N). A simple exact series expansion for the dependence of the 2(N – 1) Seiberg-Witten periods a I (u), a DI (u) on the N – 1 Coulomb-branch moduli un is obtained around the Z 2N -symmetric point of the Coulomb branch, where all u n vanish. This generalizes earlier results for N = 2 in terms of hypergeometric functions, and for N = 3 in terms of Appell functions. Using these and other analytical results, combined with numerical computations, we explore the global structure of the Kähler potential K = 1/2Σ I Im(a¯ I a DI ), which is single valued on the Coulomb branch. Evidence is presented that K is a convex function, with a unique minimum at the Z 2N -symmetric point. Finally, we explore candidate walls of marginal stability in the vicinity of this point, and their relation to the surface of vanishing Kähler potential.
72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
Efficient proximal subproblem solvers for a nonsmooth trust-region method
In [R. J. Baraldi and D. P. Kouri, Mathematical Programming, (2022), pp. 1-40], we introduced an inexact trust-region algorithm for minimizing the sum of a smooth nonconvex and nonsmooth convex function. The principle expense of this method is in computing a trial iterate that satisfies the so-called fraction of Cauchy decrease condition—a bound that ensures the trial iterate produces sufficient decrease of the subproblem model. In this paper, we expound on various proximal trust-region subproblem solvers that generalize traditional trust-region methods for smooth unconstrained and convex-constrained problems. We introduce a simplified spectral proximal gradient solver, a truncated nonlinear conjugate gradient solver, and a dogleg method. Finally, we compare algorithm performance on examples from data science and PDE-constrained optimization.
97 MATHEMATICS AND COMPUTING↗
A method for convex black-box integer global optimization
Here we study the problem of minimizing a convex function on a nonempty, finite subset of the integer lattice when the function cannot be evaluated at noninteger points. We propose a new underestimator that does not require access to (sub)gradients of the objective; such information is unavailable when the objective is a blackbox function. Rather, our underestimator uses secant linear functions that interpolate the objective function at previously evaluated points. These linear mappings are shown to underestimate the objective in disconnected portions of the domain. Therefore, the union of these conditional cuts provides a nonconvex underestimator of the objective. We propose an algorithm that alternates between updating the underestimator and evaluating the objective function. We prove that the algorithm converges to a global minimum of the objective function on the feasible set. We present two approaches for representing the underestimator and compare their computational effectiveness. We also compare implementations of our algorithm with existing methods for minimizing functions on a subset of the integer lattice. We discuss the difficulty of this problem class and provide insights into why a computational proof of optimality is challenging even for moderate problem sizes.
97 MATHEMATICS AND COMPUTING↗
An Incremental Gradient Method for Optimization Problems With Variational Inequality Constraints
We consider minimizing a sum of agent-specific nondifferentiable merely convex functions over the solution set of a variational inequality (VI) problem in that each agent is associated with a local monotone mapping. This problem finds an application in computation of the best equilibrium in nonlinear complementarity problems arising in transportation networks. We develop an iteratively regularized incremental gradient method where at each iteration, agents communicate over a directed cycle graph to update their solution iterates using their local information about the objective and the mapping. The proposed method is single-timescale in the sense that it does not involve any excessive hard-to-project computation per iteration. We derive nonasymptotic agent-wise convergence rates for the suboptimality of the global objective function and infeasibility of the VI constraints measured by a suitably defined dual gap function. Finally, the proposed method appears to be the first fully iterative scheme equipped with iteration complexity that can address distributed optimization problems with VI constraints over cycle graphs.
convergence↗
Complementary variational principle and duality in mathematical programming.
The relationship between the complementary variational principle and duality in mathematical programming is demonstrated through a geometric approach in a Hilbert space setting. A necessary and sufficient condition for the existence of such a principle is given in the case of a convex functional constrained by linear dynamics. Its relationship to the Kuhn-Tucker saddle point theory is indicated. Applications to various programming and control problems are discussed.
Chan, W. L.↗
An analysis of scatter decomposition
A formal analysis of a powerful mapping technique known as scatter decomposition is presented. Scatter decomposition divides an irregular computational domain into a large number of equal sized pieces, and distributes them modularly among processors. A probabilistic model of workload in one dimension is used to formally explain why, and when scatter decomposition works. The first result is that if correlation in workload is a convex function of distance, then scattering a more finely decomposed domain yields a lower average processor workload variance. The second result shows that if the workload process is stationary Gaussian and the correlation function decreases linearly in distance until becoming zero and then remains zero, scattering a more finely decomposed domain yields a lower expected maximum processor workload. Finally it is shown that if the correlation function decreases linearly across the entire domain, then among all mappings that assign an equal number of domain pieces to each processor, scatter decomposition minimizes the average processor workload variance. The dependence of these results on the assumption of decreasing correlation is illustrated with situations where a coarser granularity actually achieves better load balance.
Nicol, David M.↗
An analysis of scatter decomposition
A formal analysis of a mapping method known as scatter decomposition (SD) is presented. SD divides an irregular domain into many equal-size pieces and distributes them modularly among processors. It is shown that, if a correlation in workload is a convex function of distance, then scattering a more finely decomposed domain yields a lower average processor workload variance; if the workload process is stationary Gaussian and the correlation function decreases linearly in distance to zero and then remains zero, scattering a more finely decomposed domain yields a lower expected maximum processor workload. Finally, if the correlation function decreases linearly across the entire domain, then (among all mappings that assign an equal number of domain pieces to each processor) SD minimizes the average processor workload variance. The dependence of these results on the assumption of decreasing correlation is illustrated with cases where a coarser granularity actually achieves better load balance.
Nicol, David M.↗
The nonconvex multi-dimensional Riemann problem for Hamilton-Jacobi equations
Simple inequalities are presented for the viscosity solution of a Hamilton-Jacobi equation in N space dimensions when neither the initial data nor the Hamiltonian need be convex (or concave). The initial data are uniformly Lipschitz and can be written as the sum of a convex function in a group of variables and a concave function in the remaining variables, therefore including the nonconvex Riemann problem. The inequalities become equalities wherever a 'maxmin' equals a 'minmax', and thus a representation formula for this problem is obtained, generalizing the classical Hopi formulas.
Bardi, Martino↗
Robust control of systems with real parameter uncertainty and unmodelled dynamics
During this research period we have made significant progress in the four proposed areas: (1) design of robust controllers via H infinity optimization; (2) design of robust controllers via mixed H2/H infinity optimization; (3) M-delta structure and robust stability analysis for structured uncertainties; and (4) a study on controllability and observability of perturbed plant. It is well known now that the two-Riccati-equation solution to the H infinity control problem can be used to characterize all possible stabilizing optimal or suboptimal H infinity controllers if the optimal H infinity norm or gamma, an upper bound of a suboptimal H infinity norm, is given. In this research, we discovered some useful properties of these H infinity Riccati solutions. Among them, the most prominent one is that the spectral radius of the product of these two Riccati solutions is a continuous, nonincreasing, convex function of gamma in the domain of interest. Based on these properties, quadratically convergent algorithms are developed to compute the optimal H infinity norm. We also set up a detailed procedure for applying the H infinity theory to robust control systems design. The desire to design controllers with H infinity robustness but H(exp 2) performance has recently resulted in mixed H(exp 2) and H infinity control problem formulation. The mixed H(exp 2)/H infinity problem have drawn the attention of many investigators. However, solution is only available for special cases of this problem. We formulated a relatively realistic control problem with H(exp 2) performance index and H infinity robustness constraint into a more general mixed H(exp 2)/H infinity problem. No optimal solution yet is available for this more general mixed H(exp 2)/H infinity problem. Although the optimal solution for this mixed H(exp 2)/H infinity control has not yet been found, we proposed a design approach which can be used through proper choice of the available design parameters to influence both robustness and performance. For a large class of linear time-invariant systems with real parametric perturbations, the coefficient vector of the characteristic polynomial is a multilinear function of the real parameter vector. Based on this multilinear mapping relationship together with the recent developments for polytopic polynomials and parameter domain partition technique, we proposed an iterative algorithm for coupling the real structured singular value.
Chang, Bor-Chin↗
On convexity of H-infinity Riccati solutions
The authors revealed several important eigen properties of the stabilizing solutions of the two H-infinity Riccati equations and their product. Among them, the most prominent one is that the spectral radius of the product of these two Riccati solutions is a continuous, nonincreasing, convex function of gamma in the domain of interest. Based on these properties, quadratically convergent algorithms are developed to compute the optimal H-infinity norm. Two examples are used to illustrate the algorithms.
Li, X. P.↗
On convexity of H-infinity Riccati solutions and its applications
The celebrated two-Riccati-equation solution to a standard H-infinity control problem can be used to characterize all possible stabilizing optimal or suboptimal H-infinity controllers if the optimal H-infinity norm or gamma, an upper bound of a suboptimal H-infinity norm, is given. In this note, some properties of these H-infinity Riccati solutions are revealed. Among them, the most prominent one is that the spectral radius of the product of these two Riccati solutions is a continuous, nonincreasing, convex function of gamma on the domain of interest. Based on these properties, a quadratically convergent algorithm is developed to compute the optimal H-infinity norm.
Li, X. P.↗
RAISHIN: A High-Resolution Three-Dimensional General Relativistic Magnetohydrodynamics Code
We have developed a new three-dimensional general relativistic magnetohydrodynamic (GRMHD) code, RAISHIN, using a conservative, high resolution shock-capturing scheme. The numerical fluxes are calculated using the Harten, Lax, & van Leer (HLL) approximate Riemann solver scheme. The flux-interpolated, constrained transport scheme is used to maintain a divergence-free magnetic field. In order to examine the numerical accuracy and the numerical efficiency, the code uses four different reconstruction methods: piecewise linear methods with Minmod and MC slope-limiter function, convex essentially non-oscillatory (CENO) method, and piecewise parabolic method (PPM) using multistep TVD Runge-Kutta time advance methods with second and third-order time accuracy. We describe code performance on an extensive set of test problems in both special and general relativity. Our new GRMHD code has proven to be accurate in second order and has successfully passed with all tests performed, including highly relativistic and magnetized cases in both special and general relativity.
Mizuno, Yosuke↗