A Trust Region Approach to Optimal Estimation in Atmospheric Retrievals
Explore the source record for details and available documents.
optimization atmospheric retrievals↗
SEARCH · Engineering Papers
Results for “trust regions”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Numerical optimization in Hilbert space using inexact function and gradient evaluations
Trust region algorithms provide a robust iterative technique for solving non-convex unstrained optimization problems, but in many instances it is prohibitively expensive to compute high accuracy function and gradient values for the method. Of particular interest are inverse and parameter estimation problems, since function and gradient evaluations involve numerically solving large systems of differential equations. A global convergence theory is presented for trust region algorithms in which neither function nor gradient values are known exactly. The theory is formulated in a Hilbert space setting so that it can be applied to variational problems as well as the finite dimensional problems normally seen in trust region literature. The conditions concerning allowable error are remarkably relaxed: relative errors in the gradient error condition is automatically satisfied if the error is orthogonal to the gradient approximation. A technique for estimating gradient error and improving the approximation is also presented.
Carter, Richard G.↗
Recursive Branching Simulated Annealing Algorithm
This innovation is a variation of a simulated-annealing optimization algorithm that uses a recursive-branching structure to parallelize the search of a parameter space for the globally optimal solution to an objective. The algorithm has been demonstrated to be more effective at searching a parameter space than traditional simulated-annealing methods for a particular problem of interest, and it can readily be applied to a wide variety of optimization problems, including those with a parameter space having both discrete-value parameters (combinatorial) and continuous-variable parameters. It can take the place of a conventional simulated- annealing, Monte-Carlo, or random- walk algorithm. In a conventional simulated-annealing (SA) algorithm, a starting configuration is randomly selected within the parameter space. The algorithm randomly selects another configuration from the parameter space and evaluates the objective function for that configuration. If the objective function value is better than the previous value, the new configuration is adopted as the new point of interest in the parameter space. If the objective function value is worse than the previous value, the new configuration may be adopted, with a probability determined by a temperature parameter, used in analogy to annealing in metals. As the optimization continues, the region of the parameter space from which new configurations can be selected shrinks, and in conjunction with lowering the annealing temperature (and thus lowering the probability for adopting configurations in parameter space with worse objective functions), the algorithm can converge on the globally optimal configuration. The Recursive Branching Simulated Annealing (RBSA) algorithm shares some features with the SA algorithm, notably including the basic principles that a starting configuration is randomly selected from within the parameter space, the algorithm tests other configurations with the goal of finding the globally optimal solution, and the region from which new configurations can be selected shrinks as the search continues. The key difference between these algorithms is that in the SA algorithm, a single path, or trajectory, is taken in parameter space, from the starting point to the globally optimal solution, while in the RBSA algorithm, many trajectories are taken; by exploring multiple regions of the parameter space simultaneously, the algorithm has been shown to converge on the globally optimal solution about an order of magnitude faster than when using conventional algorithms. Novel features of the RBSA algorithm include: 1. More efficient searching of the parameter space due to the branching structure, in which multiple random configurations are generated and multiple promising regions of the parameter space are explored; 2. The implementation of a trust region for each parameter in the parameter space, which provides a natural way of enforcing upper- and lower-bound constraints on the parameters; and 3. The optional use of a constrained gradient- search optimization, performed on the continuous variables around each branch s configuration in parameter space to improve search efficiency by allowing for fast fine-tuning of the continuous variables within the trust region at that configuration point.
Bolcar, Matthew↗
A new algorithm for constrained nonlinear least-squares problems, part 1
A Gauss-Newton algorithm is presented for solving nonlinear least squares problems. The problem statement may include simple bounds or more general constraints on the unknowns. The algorithm uses a trust region that allows the objective function to increase with logic for retreating to best values. The computations for the linear problem are done using a least squares system solver that allows for simple bounds and linear constraints. The trust region limits are defined by a box around the current point. In its current form the algorithm is effective only for problems with small residuals, linear constraints and dense Jacobian matrices. Results on a set of test problems are encouraging.
Hanson, R. J.↗
Multilevel algorithms for nonlinear optimization
Multidisciplinary design optimization (MDO) gives rise to nonlinear optimization problems characterized by a large number of constraints that naturally occur in blocks. We propose a class of multilevel optimization methods motivated by the structure and number of constraints and by the expense of the derivative computations for MDO. The algorithms are an extension to the nonlinear programming problem of the successful class of local Brown-Brent algorithms for nonlinear equations. Our extensions allow the user to partition constraints into arbitrary blocks to fit the application, and they separately process each block and the objective function, restricted to certain subspaces. The methods use trust regions as a globalization strategy, and they have been shown to be globally convergent under reasonable assumptions. The multilevel algorithms can be applied to all classes of MDO formulations. Multilevel algorithms for solving nonlinear systems of equations are a special case of the multilevel optimization methods. In this case, they can be viewed as a trust-region globalization of the Brown-Brent class.
Alexandrov, Natalia↗
A new minimax algorithm
The representation min t s.t. F(I)(x). - t less than or equal to 0 for all i is examined. An active set strategy is designed of functions: active, semi-active, and non-active. This technique will help in preventing zigzagging which often occurs when an active set strategy is used. Some of the inequality constraints are handled with slack variables. Also a trust region strategy is used in which at each iteration there is a sphere around the current point in which the local approximation of the function is trusted. The algorithm is implemented into a successful computer program. Numerical results are provided.
Vardi, A.↗
An Improved Cryosat-2 Sea Ice Freeboard Retrieval Algorithm Through the Use of Waveform Fitting
We develop an empirical model capable of simulating the mean echo power cross product of CryoSat-2 SAR and SAR In mode waveforms over sea ice covered regions. The model simulations are used to show the importance of variations in the radar backscatter coefficient with incidence angle and surface roughness for the retrieval of surfaceelevation of both sea ice floes and leads. The numerical model is used to fit CryoSat-2 waveforms to enable retrieval of surface elevation through the use of look-up tables and a bounded trust region Newton least squares fitting approach. The use of a model to fit returns from sea ice regions offers advantages over currently used threshold retrackingmethods which are here shown to be sensitive to the combined effect of bandwidth limited range resolution and surface roughness variations. Laxon et al. (2013) have compared ice thickness results from CryoSat-2 and IceBridge, and found good agreement, however consistent assumptions about the snow depth and density of sea ice werenot used in the comparisons. To address this issue, we directly compare ice freeboard and thickness retrievals from the waveform fitting and threshold tracker methods of CryoSat-2 to Operation IceBridge data using a consistent set of parameterizations. For three IceBridge campaign periods from March 20112013, mean differences (CryoSat-2 IceBridge) of 0.144m and 1.351m are respectively found between the freeboard and thickness retrievals using a 50 sea ice floe threshold retracker, while mean differences of 0.019m and 0.182m are found when using the waveform fitting method. This suggests the waveform fitting technique is capable of better reconciling the seaice thickness data record from laser and radar altimetry data sets through the usage of consistent physical assumptions.
CryoSat-2↗
A conjugate gradients/trust regions algorithms for training multilayer perceptrons for nonlinear mapping
This paper addresses the issue of applying a globally convergent optimization algorithm to the training of multilayer perceptrons, a class of Artificial Neural Networks. The multilayer perceptrons are trained towards the solution of two highly nonlinear problems: (1) signal detection in a multi-user communication network, and (2) solving the inverse kinematics for a robotic manipulator. The research is motivated by the fact that a multilayer perceptron is theoretically capable of approximating any nonlinear function to within a specified accuracy. The algorithm that has been employed in this study combines the merits of two well known optimization algorithms, the Conjugate Gradients and the Trust Regions Algorithms. The performance is compared to a widely used algorithm, the Backpropagation Algorithm, that is basically a gradient-based algorithm, and hence, slow in converging. The performances of the two algorithms are compared with the convergence rate. Furthermore, in the case of the signal detection problem, performances are also benchmarked by the decision boundaries drawn as well as the probability of error obtained in either case.
Madyastha, Raghavendra K.↗
Numerical experience with a class of algorithms for nonlinear optimization using inexact function and gradient information
For optimization problems associated with engineering design, parameter estimation, image reconstruction, and other optimization/simulation applications, low accuracy function and gradient values are frequently much less expensive to obtain than high accuracy values. Here, researchers investigate the computational performance of trust region methods for nonlinear optimization when high accuracy evaluations are unavailable or prohibitively expensive, and confirm earlier theoretical predictions when the algorithm is convergent even with relative gradient errors of 0.5 or more. The proper choice of the amount of accuracy to use in function and gradient evaluations can result in orders-of-magnitude savings in computational cost.
Carter, Richard G.↗
Iterative procedures for space shuttle main engine performance models
Performance models of the Space Shuttle Main Engine (SSME) contain iterative strategies for determining approximate solutions to nonlinear equations reflecting fundamental mass, energy, and pressure balances within engine flow systems. Both univariate and multivariate Newton-Raphson algorithms are employed in the current version of the engine Test Information Program (TIP). Computational efficiency and reliability of these procedures is examined. A modified trust region form of the multivariate Newton-Raphson method is implemented and shown to be superior for off nominal engine performance predictions. A heuristic form of Broyden's Rank One method is also tested and favorable results based on this algorithm are presented.
Santi, L. Michael↗
Globally convergent techniques in nonlinear Newton-Krylov
Some convergence theory is presented for nonlinear Krylov subspace methods. The basic idea of these methods is to use variants of Newton's iteration in conjunction with a Krylov subspace method for solving the Jacobian linear systems. These methods are variants of inexact Newton methods where the approximate Newton direction is taken from a subspace of small dimensions. The main focus is to analyze these methods when they are combined with global strategies such as linesearch techniques and model trust region algorithms. Most of the convergence results are formulated for projection onto general subspaces rather than just Krylov subspaces.
Brown, Peter N.↗
A quadratic-tensor model algorithm for nonlinear least-squares problems with linear constraints
A new algorithm for solving nonlinear least-squares and nonlinear equation problems is proposed which is based on approximating the nonlinear functions using the quadratic-tensor model by Schnabel and Frank. The algorithm uses a trust region defined by a box containing the current values of the unknowns. The algorithm is found to be effective for problems with linear constraints and dense Jacobian matrices.
Hanson, R. J.↗
Algorithms for bilevel optimization
General multilevel nonlinear optimization problems arise in design of complex systems and can be used as a means of regularization for multi-criteria optimization problems. Here, for clarity in displaying our ideas, we restrict ourselves to general bi-level optimization problems, and we present two solution approaches. Both approaches use a trust-region globalization strategy, and they can be easily extended to handle the general multilevel problem. We make no convexity assumptions, but we do assume that the problem has a nondegenerate feasible set. We consider necessary optimality conditions for the bi-level problem formulations and discuss results that can be extended to obtain multilevel optimization formulations with constraints at each level.
Alexandrov, Natalia↗
A PDE Sensitivity Equation Method for Optimal Aerodynamic Design
The use of gradient based optimization algorithms in inverse design is well established as a practical approach to aerodynamic design. A typical procedure uses a simulation scheme to evaluate the objective function (from the approximate states) and its gradient, then passes this information to an optimization algorithm. Once the simulation scheme (CFD flow solver) has been selected and used to provide approximate function evaluations, there are several possible approaches to the problem of computing gradients. One popular method is to differentiate the simulation scheme and compute design sensitivities that are then used to obtain gradients. Although this black-box approach has many advantages in shape optimization problems, one must compute mesh sensitivities in order to compute the design sensitivity. In this paper, we present an alternative approach using the PDE sensitivity equation to develop algorithms for computing gradients. This approach has the advantage that mesh sensitivities need not be computed. Moreover, when it is possible to use the CFD scheme for both the forward problem and the sensitivity equation, then there are computational advantages. An apparent disadvantage of this approach is that it does not always produce consistent derivatives. However, for a proper combination of discretization schemes, one can show asymptotic consistency under mesh refinement, which is often sufficient to guarantee convergence of the optimal design algorithm. In particular, we show that when asymptotically consistent schemes are combined with a trust-region optimization algorithm, the resulting optimal design method converges. We denote this approach as the sensitivity equation method. The sensitivity equation method is presented, convergence results are given and the approach is illustrated on two optimal design problems involving shocks.
Borggaard, Jeff↗
Subauditory Speech Recognition based on EMG/EPG Signals
Sub-vocal electromyogram/electro palatogram (EMG/EPG) signal classification is demonstrated as a method for silent speech recognition. Recorded electrode signals from the larynx and sublingual areas below the jaw are noise filtered and transformed into features using complex dual quad tree wavelet transforms. Feature sets for six sub-vocally pronounced words are trained using a trust region scaled conjugate gradient neural network. Real time signals for previously unseen patterns are classified into categories suitable for primitive control of graphic objects. Feature construction, recognition accuracy and an approach for extension of the technique to a variety of real world application areas are presented.
Jorgensen, Charles↗
On Managing the Use of Surrogates in General Nonlinear Optimization and MDO
This paper is concerned with a trust region approximation management framework (AMF) for solving the nonlinear programming problem in general and multidisciplinary optimization problems in particular The intent of the AMF methodology is to facilitate the solution of optimization problems with high-fidelity models. While such models are designed to approximate the physical phenomena they describe to a high degree of accuracy, their use in a repetitive procedure, for example, iterations of an optimization or a search algorithm, make such use prohibitively expensive. An improvement in design with lower-fidelity, cheaper models, however, does not guarantee a corresponding improvement for the higher-fidelity problem. The AMF methodology proposed here is based on a class of multilevel methods for constrained optimization and is designed to manage the use of variable-fidelity approximations or models in a systematic way that assures convergence to critical points of the original high-fidelity problem.
Alexandrov, Natalia M.↗
Retrieval of Snow Freeboard of Antarctic Sea Ice Using Waveform Fitting of CryoSat-2 Returns
In this paper we develop a CryoSat-2 algorithm to retrieve the surface elevation of the air–snow interface over Antarctic sea ice. This algorithm utilizes a two-layer physical model that accounts for scattering from a snow layer atop sea ice as well as scattering from below the snow surface. The model produces waveforms that are fit to CryoSat-2 level 1B data through a bounded trust region least-squares fitting process. These fit waveforms are then used to track the air–snow interface and retrieve the surface elevation at each point along the CryoSat-2 ground track, from which the snow freeboard is computed. To validate this algorithm, we compare retrieved surface elevation measurements and snow surface radar return power levels with those from Operation IceBridge, which flew along a contemporaneous CryoSat-2 orbit in October 2011 and November 2012. Average elevation differences (standard deviations) along the flight lines (IceBridge Airborne Topographic Mapper, ATM – CryoSat-2) are found to be 0.016 cm (29.24 cm) in 2011 and 2.58 cm (26.65 cm) in 2012. The spatial distribution of monthly average pan-Antarctic snow freeboard found using this method is similar to what was observed from NASA's Ice, Cloud, and land Elevation Satellite (ICESat), where the difference (standard deviation) between October 2011–2017 CryoSat-2 mean snow freeboard and spring 2003–2007 mean freeboard from ICESat is 1.92 cm (9.23 cm). While our results suggest that this physical model and waveform fitting method can be used to retrieve snow freeboard from CryoSat-2, allowing for the potential to join laser and radar altimetry data records in the Antarctic, larger (∼30 cm) regional differences from ICESat and along-track differences from ATM do exist, suggesting the need for future improvements to the method. Snow–ice interface elevation retrieval is also explored as a potential to obtain snow depth measurements. However, it is found that this retrieval method often tracks a strong scattering layer within the snow layer instead of the actual snow–ice interface, leading to an overestimation of ice freeboard and an underestimation of snow depth in much of the Southern Ocean but with promising results in areas such as the East Antarctic sector.
Fons, Steven W.↗
Multifidelity Optimization with Transonic Flutter Constraints
This work considers static and dynamic aeroelastic optimization of a cantilevered platewing in transonic flow. Low- and high-fidelity aeroelastic predictions are fed into a standard trust region model management scheme in order to efficiently solve this expensive optimization problem with multifidelity methods. Differences in fidelity are entirely driven by different aerodynamic solvers: linear panel methods for the low-fidelity method, and inviscid CFD-based solvers for the high-fidelity response. The optimization problem utilizes shape, sizing, and trim variables to satisfy stress, trim, and flutter constraints, and is solved across a range of subsonic and transonic Mach numbers. The multifidelity method is able to provide a speed-up for all cases considered here, despite sizable inaccuracies in the low-fidelity response for transonic flows.
Bret K Stanford↗