Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “optimization problems”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6

Train Like a (Var)Pro: Efficient Training of Neural Networks with Variable Projection

Deep neural networks (DNNs) have achieved state-of-the-art performance across a variety of traditional machine learning tasks, e.g., speech recognition, image classification, and segmentation. The ability of DNNs to efficiently approximate high-dimensional functions has also motivated their use in scientific applications, e.g., to solve partial differential equations and to generate surrogate models. In this paper, we consider the supervised training of DNNs, which arises in many of the above applications. We focus on the central problem of optimizing the weights of the given DNN such that it accurately approximates the relation between observed input and target data. Devising effective solvers for this optimization problem is notoriously challenging due to the large number of weights, nonconvexity, data sparsity, and nontrivial choice of hyperparameters. To solve the optimization problem more efficiently, we propose the use of variable projection (VarPro), a method originally designed for separable nonlinear least-squares problems. Our main contribution is the Gauss--Newton VarPro method (GNvpro) that extends the reach of the VarPro idea to nonquadratic objective functions, most notably cross-entropy loss functions arising in classification. These extensions make GNvpro applicable to all training problems that involve a DNN whose last layer is an affine mapping, which is common in many state-of-the-art architectures. In our four numerical experiments from surrogate modeling, segmentation, and classification, GNvpro solves the optimization problem more efficiently than commonly used stochastic gradient descent (SGD) schemes. Finally, GNvpro finds solutions that generalize well, and in all but one example better than well-tuned SGD methods, to unseen data points.

97 MATHEMATICS AND COMPUTING↗

Robust A-Optimal Experimental Design for Sensor Placement in Bayesian Linear Inverse Problems

Optimal design of experiments for Bayesian inverse problems has recently gained wide popularity and attracted much attention, especially in the computational science and Bayesian inversion communities. An optimal design maximizes a predefined utility function that is formulated in terms of the elements of an inverse problem, an example being optimal sensor placement for parameter identification. The state-of-the-art algorithmic approaches following this simple formulation generally overlook misspecification of the elements of the inverse problem, such as the prior or the measurement uncertainties. This work presents an efficient algorithmic approach for designing optimal experimental design schemes for Bayesian linear inverse problems such that the optimal design is robust to misspecification of elements of the inverse problem. Specifically, we consider a worst-case scenario approach for the uncertain or misspecified parameters, formulate robust objectives, and propose an algorithmic approach for optimizing such objectives. Furthermore, both relaxation and stochastic solution approaches are discussed with detailed analysis and insight into the interpretation of the problem and the proposed algorithmic approach. Extensive numerical experiments to validate and analyze the proposed approach are carried out for sensor placement in a parameter identification problem.

Bayesian inverse problems↗

Quantum computational phase transition in combinatorial problems

Quantum Approximate Optimization algorithm (QAOA) aims to search for approximate solutions to discrete optimization problems with near-term quantum computers. As there are no algorithmic guarantee possible for QAOA to outperform classical computers, without a proof that bounded-error quantum polynomial time (BQP) ≠ nondeterministic polynomial time (NP), it is necessary to investigate the empirical advantages of QAOA. We identify a computational phase transition of QAOA when solving hard problems such as SAT—random instances are most difficult to train at a critical problem density. We connect the transition to the controllability and the complexity of QAOA circuits. Moreover, we find that the critical problem density in general deviates from the SAT-UNSAT phase transition, where the hardest instances for classical algorithms lies. Then, we show that the high problem density region, which limits QAOA’s performance in hard optimization problems (reachability deficits), is actually a good place to utilize QAOA: its approximation ratio has a much slower decay with the problem density, compared to classical approximate algorithms. Indeed, it is exactly in this region that quantum advantages of QAOA over classical approximate algorithms can be identified.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Extremum seeking for optimal control problems with unknown time-varying systems and unknown objective functions

We consider the problem of optimal feedback control of an unknown, noisy, time-varying, dynamic system that is initialized repeatedly. Examples include a robotic manipulator which must perform the same motion, such as assisting a human, repeatedly and accelerating cavities in particle accelerators which are turned on for a fraction of a second with given initial conditions and vary slowly due to temperature fluctuations. In this paper, we present an approach that applies to systems of practical interest. The method presented here is model independent; does not require knowledge of the objective function; is robust to measurement noise; is applicable for any set of initial conditions; is applicable to simultaneously controlling an arbitrary number of parameters; and may be implemented with a broad range of continuous or discontinuous functions such as sine or square waves. For systems with convex cost functions we prove that our algorithm will produce controllers that approach the minimal cost. For linear systems we reproduce the cost minimizing linear quadratic regulator optimal controller that could have been designed analytically had the system and cost function been known. We demonstrate the effectiveness of the algorithm with simulation studies of noisy and time-varying systems.

42 ENGINEERING↗

A frozen Gaussian approximation-based multi-level particle swarm optimization for seismic inversion

In this paper, we propose a frozen Gaussian approximation (FGA)-based multi-level particle swarm optimization (MLPSO) method for seismic inversion of high-frequency wave data. The method addresses two challenges in it: First, the optimization problem is highly non-convex, which makes hard for gradient-based methods to reach global minima. This is tackled by MLPSO which can escape from undesired local minima. Second, the character of high-frequency of seismic waves requires a large number of grid points in direct computational methods, and thus renders an extremely high computational demand on the simulation of each sample in MLPSO. We overcome this difficulty by three steps: First, we use FGA to compute high-frequency wave propagation based on asymptotic analysis on phase plane; Then we design a constrained full waveform inversion problem to prevent the optimization search getting into regions of velocity where FGA is not accurate; Last, we solve the constrained optimization problem by MLPSO that employs FGA solvers with different fidelity. The performance of the proposed method is demonstrated by a two-dimensional full-waveform inversion example of the smoothed Marmousi model.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Managing Model Data Introduced Uncertainties in Simulator Predictions for Generation IV Systems via Optimum Experimental Design

An optimization technique has been developed to select optimized experimental design specifications to produce data specifically designed to be assimilated to optimize a given reactor concept. Data from the optimized experiment is assimilated to generate posteriori uncertainties on the reactor concept’s core attributes from which the design responses are computed. The reactor concept is then optimized with the new data to realize cost savings by reducing margin. The optimization problem iterates until an optimal experiment is found to maximize the savings. A new generation of innovative nuclear reactor designs, in particular fast neutron spectrum recycle reactors, are being considered for the application of closing the nuclear fuel cycle in the future. Safe and economical design of these reactors will require uncertainty reduction in basic nuclear data which are input to the reactor design. These data uncertainty propagate to design responses which in turn require the reactor designer to incorporate additional safety margin into the design, which often increases the cost of the reactor. Therefore basic nuclear data needs to be improved and this is accomplished through experimentation. Considering the high cost of nuclear experiments, it is desired to have an optimized experiment which will provide the data needed for uncertainty reduction such that a reactor design concept can meet its target accuracies or to allow savings to be realized by reducing the margin required due to uncertainty propagated from basic nuclear data. However, this optimization is coupled to the reactor design itself because with improved data the reactor concept can be re-optimized itself. It is thus desired to find the experiment that gives the best optimized reactor design. Methods are first established to model both the reactor concept and the experiment and to efficiently propagate the basic nuclear data uncertainty through these models to outputs. The representativity of the experiment to the design concept is quantitatively determined. A technique is then established to assimilate this data and produce posteriori uncertainties on key attributes and responses of the design concept. Several experiment perturbations based on engineering judgment are used to demonstrate these methods and also serve as an initial generation of the optimization problem. Finally, an optimization technique is developed which will simultaneously arrive at an optimized experiment to produce an optimized reactor design. Solution of this problem is made possible by the use of the simulated annealing algorithm for solution of optimization problems. The optimization examined in this work is based on maximizing the reactor cost savings associated with the modified design made possible by using the design margin gained through reduced basic nuclear data uncertainties. Cost values for experiment design specifications and reactor design specifications are established and used to compute a total savings by comparing the posteriori reactor cost to the a priori cost plus the cost of the experiment. The optimized solution arrives at a maximized cost savings.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

Red-QAOA: Efficient Variational Optimization through Circuit Reduction

The Quantum Approximate Optimization Algorithm (QAOA) provides a quantum solution for combinatorial optimization problems. However, the optimal parameter searching process of QAOA is greatly affected by noise, leading to non-optimal solutions. This paper introduces a novel approach to optimize QAOA by exploiting the energy landscape concentration of similar instances via graph reduction, thus addressing the effect of noise. We formalize the notion of similar instances in QAOA and develop a Simulated Annealing-based graph reduction algorithm, called Red-QAOA, to identify the most similar subgraph for efficient parameter optimization. Red-QAOA outperforms state-of-the-art Graph Neural Network (GNN) based graph pooling techniques in performance and demonstrates effectiveness on a diverse set of real-world optimization problems encompassing 3200 graphs. Red-QAOA reduced the node counts and edge counts by 28% and 37%, respectively, while maintaining a low mean square error of 2%. These enable the identification of an optimal parameter set that is closer to the ideal true optimal solution in the presence of noise. By substantially streamlining the search for QAOA parameters, our approach sets the stage for the practical application of quantum algorithms in solving complex optimization problems.

Wang, Meng↗

Analysis of the Optimal Relaxed Control to an Optimal Control Problem

Relaxed controls are widely used to analyze the existence of optimal controls in the literature. Though there are many optimal control problems admitting no optimal control, rare examples were shown. This paper will solve a particular optimal control problem by analyzing the optimal relaxed controls, showing the ideas we used to study such kind of problems.

97 MATHEMATICS AND COMPUTING↗

A Nonlinear Optimal Control Problem Arising from a Sterilization Process for Packaged Foods

In this paper we study an optimization problem arising from a sterilization process for packaged foods by using a microwave heating method. The goal of the optimal control is to find the optimal frequency function such that the temperature profile at the final stage has a relative uniform distribution in the food product. The underlying state variables are electric, magnetic fields and temperature which satisfy the coupled nonlinear Maxwell’s system and a nonlinear heat equation. The control variable for the system is chosen to be the electric frequency function. We show that there exists an optimal frequency which minimizes the cost functional. Moreover, an optimality condition is also derived.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Outcomes and Insights From Simplified Analytic Trajectory Optimization for a Tethered Underwater Kite

This letter formulates and solves a periodic trajectory optimization problem for a tethered underwater kite. The goal is to maximize the average mechanical power harvested by the kite. The type of kite considered in this letter extracts electricity from ocean currents by moving cross-current as its reel away from its base station, and consumes electricity to reel back. The problem of optimizing this kite’s trajectory is challenging due to the high dimensionality and nonlinearity of its dynamics. To tackle this challenge, the literature often separates the problem into two subproblems focusing on optimizing the cross-current and the reel-in/reel-out components of the trajectory, respectively, which may be sub-optimal. In contrast, this letter solves for the combined cross-current and reel-in/reel-out trajectory by linearizing the dynamics of the kite around a zero-power reference equilibrium trajectory in spherical coordinates. This allows the trajectory optimization problem to be solved analytically for simple sinusoidal input perturbations from equilibrium. Here we use linear quadratic regulation to enable the nonlinear kite model to track the optimized trajectory. The result is a computationally efficient approach that achieves an attractive Loyd factor of 19.9%, while providing important insights into the nature of the optimal trajectory.

42 ENGINEERING↗

On a Highly Nonlinear Self-Obstacle Optimal Control Problem

We consider a non-quadratic optimal control problem associated to a nonlinear elliptic variational inequality, where the obstacle is the control itself. We show that, fixed a desired profile, there exists an optimal solution which is not far from it. Detailed characterizations of the optimal solution are given, also in terms of approximating problems.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

ParMOO: A Python library for parallel multiobjective simulation optimization

A multiobjective optimization problem (MOOP) is an optimization problem in which multiple objectives are optimized simultaneously. The goal of a MOOP is to find solutions that describe the tradeoff between these (potentially conflicting) objectives. Such a tradeoff surface is called the Pareto front. Real-world MOOPs may also involve constraints – additional hard rules that every solution must adhere to. In a multiobjective simulation optimization problem, the objectives are derived from the outputs of one or more computationally expensive simulations. Such problems are ubiquitous in science and engineering.

97 MATHEMATICS AND COMPUTING↗

Quantum annealing for combinatorial optimization: a benchmarking study

Quantum annealing (QA) has the potential to significantly improve solution quality and reduce time complexity in solving combinatorial optimization problems compared to classical optimization methods. However, due to the limited number of qubits and their connectivity, the QA hardware did not show such an advantage over classical methods in past benchmarking studies. Recent advancements in QA with more than 5000 qubits, enhanced qubit connectivity, and the hybrid architecture promise to realize the quantum advantage. Here, we use a quantum annealer with state-of-the-art techniques and benchmark its performance against classical solvers. To compare their performance, we solve over 50 optimization problem instances represented by large and dense Hamiltonian matrices using quantum and classical solvers. The results demonstrate that a state-of-the-art quantum solver has higher accuracy (~0.013%) and a significantly faster problem-solving time (~6561×) than the best classical solver. Our results highlight the advantages of leveraging QA over classical counterparts, particularly in hybrid configurations, for achieving high accuracy and substantially reduced problem solving time in large-scale real-world optimization problems.

97 MATHEMATICS AND COMPUTING↗

Feed Forward Neural Network and Optimal Control Problem with Control and State Constraints

A feed forward neural network based optimal control synthesis is presented for solving optimal control problems with control and state constraints. The paper extends adaptive critic neural network architecture proposed by [5] to the optimal control problems with control and state constraints. The optimal control problem is transcribed into a nonlinear programming problem which is implemented with adaptive critic neural network. The proposed simulation method is illustrated by the optimal control problem of nitrogen transformation cycle model. Results show that adaptive critic based systematic approach holds promise for obtaining the optimal control with control and state constraints.

99 GENERAL AND MISCELLANEOUS↗

Convergence Analysis of Fixed Point Chance Constrained Optimal Power Flow Problems

For optimal power flow problems with chance constraints, a particularly effective method is based on a fixed point iteration applied to a sequence of deterministic power flow problems. However, a priori, the convergence of such an approach is not necessarily guaranteed. Here this article analyses the convergence conditions for this fixed point approach, and reports numerical experiments including for large IEEE networks.

24 POWER TRANSMISSION AND DISTRIBUTION↗

An inexact semismooth Newton method with application to adaptive randomized sketching for dynamic optimization

In many applications, one can only access the inexact gradients and inexact hessian times vector products. Thus it is essential to consider algorithms that can handle such inexact quantities with a guaranteed convergence to solution. An inexact adaptive and provably convergent semismooth Newton method is considered to solve constrained optimization problems. In particular, dynamic optimization problems, which are known to be highly expensive, are the focus. A memory efficient semismooth Newton algorithm is introduced for these problems. The source of efficiency and inexactness is the randomized matrix sketching. Further, applications to optimization problems constrained by partial differential equations are also considered.

97 MATHEMATICS AND COMPUTING↗

Direct stellarator coil optimization for nested magnetic surfaces with precise quasi-symmetry

We present a robust optimization algorithm for the design of electromagnetic coils that generate vacuum magnetic fields with nested flux surfaces and precise quasi-symmetry. The method is based on a bilevel optimization problem, where the outer coil optimization is constrained by a set of inner least squares optimization problems whose solutions describe magnetic surfaces. The outer optimization objective targets coils that generate a field with nested magnetic surfaces and good quasi-symmetry. The inner optimization problems identify magnetic surfaces when they exist, and approximate surfaces in the presence of magnetic islands or chaos. We show that this formulation can be used to heal islands and chaos, thus producing coils that result in magnetic fields with precise quasi-symmetry. We show that the method can be initialized with coils from the traditional two-stage coil design process, as well as coils from a near-axis expansion optimization. We present a numerical example where island chains are healed and quasi-symmetry is optimized up to surfaces with aspect ratio 6. Another numerical example illustrates that the aspect ratio of nested flux surfaces with optimized quasi-symmetry can be decreased from 6 to approximately 4. In conclusion, the last example shows that our approach is robust and a cold-start using coils from a near-axis expansion optimization.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗