Robust initializations of variational inference using global optimization and Laplace approximations.
Abstract not provided.
SEARCH · Engineering Papers
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Abstract not provided.
This award allowed Massachusetts Institute of Technology to demonstrate a number of key objectives. The focus of the project was on building a machine learning (ML) enhanced tools to accelerate the development of catalysts that promote the oxygen evolution reaction (OER) or the oxygen reduction reaction (ORR).
Explore the source record for details and available documents.
(Previously announced in STAR as N81-13673)
Explore the source record for details and available documents.
In this paper, an algorithm is introduced to find a minimum phase transfer function of specified order whose magnitude "tightly" overbounds a specified real-valued nonparametric function of frequency. This method has direct application to transforming nonparametric uncertainty bounds (available from system identification experiments and/or plant modeling) into parametric representations required for modern robust control design software (i.e., a minimum-phase transfer function multiplied by a norm-bounded perturbation).
While the response surface method is an effective method in engineering optimization, its accuracy is often affected by the use of limited amount of data points for model construction. In this chapter, the issues related to the accuracy of the RS approximations and possible ways of improving the RS model using appropriate treatments, including the iteratively re-weighted least square (IRLS) technique and the radial-basis neural networks, are investigated. A main interest is to identify ways to offer added capabilities for the RS method to be able to at least selectively improve the accuracy in regions of importance. An example is to target the high efficiency region of a fluid machinery design space so that the predictive power of the RS can be maximized when it matters most. Analytical models based on polynomials, with controlled level of noise, are used to assess the performance of these techniques.
Single trial evaluations Trial creation by Phase-wise GA-style or DE-inspired recombination Bin repository structure requires an initialization period Non-exclusionary Kill Distance Population collapse mechanic Main loop Creation Probabilistic switch between GA and DE creation types Locally optimize Submit to repository Repeat.
Low-thrust electric propulsion provides many advantages for mission to difficult targets-Comets and asteroids-Mercury-Outer planets (with sufficient power supply)Low-thrust electric propulsion is characterized by high power requirements but also very high specific impulse (Isp), leading to very good mass fractions. Low-thrust trajectory design is a very different process from chemical trajectory.
The coronal magnetic field directly or indirectly affects a majority of the phenomena studied in the heliosphere. It provides energy for coronal heating, controls the release of coronal mass ejections, and drives heliospheric and magnetospheric activity, yet the coronal magnetic field itself has proven difficult to measure. This difficulty has prompted a decades-long effort to develop accurate, timely, models of the field, an effort that continues today. We have developed a method for improving global coronal magnetic field models by incorporating the type of morphological constraints that could be derived from coronal images. Here we report promising initial tests of this approach on two theoretical problems, and discuss opportunities for application.
With the advancement in high performance computing and numerical optimization techniques,engineering design optimization problems are becoming more complex, larger scale,higher fidelity, and computationally more demanding, requiring longer run times than ever before. There exists methodologies and techniques that can address some of these challenges but very few can address all, and most are limited in the extent that these concerns can be addressed. With the goal of addressing such challenging engineering problems, we developed anew optimization framework, named AMIEGO, that combines concepts from surrogate-based optimization approaches, gradient-based numerical methods, Partial Least Squares, evolutionary algorithms, and Branch-and-Bound, providing newer capabilities that were not previouslyperceived. However, the original version of this framework, in the process of adaptive samplingto explore and exploit the design space, finds only a single sample point per iteration. The efforthere builds upon this previously developed optimization framework to include multiple infillsampling capability that combines the concept of generalized expected improvement function,unsupervised learning, and multi-objective evolutionary technique. To demonstrate, AMIEGOwith the multiple infill capability (called AMIEGO-MIMOS) solves a series of increasingly difficultengineering design optimization problems. The results reveal the performance of the newapproach is problem dependent. When applied to a ten-bar truss problem, the newly proposedmultiple infill strategy consistently leads to a better design solutions when compared to theexisting CPTV method (implemented with the context of the AMIEGO framework). On theother hand, when applied to a mixed-integer high fidelity wing topology optimization problem- MIMOS, despite showing a steeper convergence at the start, eventually leads to an inferiorsolution as compared to CPTV approach. These results also reveal that a small number ofstarting points, in general, are sufficient to lead to a good overall solution.
An algorithm is introduced to find a minimum phase trqansfer function of specified order whose magnitude
Generalized disjunctive programming (GDP) models with bilinear and concave constraints, often seen in water network design, are challenging optimization problems. This work proposes quadratic and piecewise linear approximations for nonlinear terms to reformulate GDP models into quadratic GDP (QGDP) models that suitable solvers may solve more efficiently. We illustrate the benefits of the quadratic reformulation with a water treatment network design problem in which nonconvexities arise from bilinear terms in the mixers’ mass balances and concave investment cost functions of treatment units. Given the similarities with water network design problems, we suggest quadratic approximation for the GDP model for the optimal design of a large-scale reverse electrodialysis (RED) process. This power technology can recover energy from salinity differences between by-product streams of the water sector, such as desalination brine mixed with regenerated wastewater effluents. The solver Gurobi excels in handling QGDP problems, but weighing the problem’s precision and tractability balance is crucial. The piecewise linear approximation yields more accurate, yet larger QGDP models that may require longer optimization times in large-scale process synthesis problems.
Global optimization of a multivariable function - constrained by bounds specified on each variable and also unconstrained - is an important problem with several real world applications. Deterministic methods such as the gradient algorithms as well as the randomized methods such as the genetic algorithms may be employed to solve these problems. In fact, there are optimization problems where a genetic algorithm/an evolutionary approach is preferable at least from the quality (accuracy) of the results point of view. From cost (complexity) point of view, both gradient and genetic approaches are usually polynomial-time; there are no serious differences in this regard, i.e., the computational complexity point of view. However, for certain types of problems, such as those with unacceptably erroneous numerical partial derivatives and those with physically amplified analytical partial derivatives whose numerical evaluation involves undesirable errors and/or is messy, a genetic (stochastic) approach should be a better choice. We have presented here the pros and cons of both the approaches so that the concerned reader/user can decide which approach is most suited for the problem at hand. Also for the function which is known in a tabular form, instead of an analytical form, as is often the case in an experimental environment, we attempt to provide an insight into the approaches focusing our attention toward accuracy. Such an insight will help one to decide which method, out of several available methods, should be employed to obtain the best (least error) output. *
Predictive modeling typically relies on Bayesian model calibration to provide uncertainty quantification. Variational inference utilizing fully independent (“mean-field”) Gaussian distributions are often used as approximate probability density functions. This simplification is attractive since the number of variational parameters grows only linearly with the number of unknown model parameters. However, the resulting diagonal covariance structure and unimodal behavior can be too restrictive to provide useful approximations of intractable Bayesian posteriors that exhibit highly non-Gaussian behavior, including multimodality. High-fidelity surrogate posteriors for these problems can be obtained by considering the family of Gaussian mixtures. Gaussian mixtures are capable of capturing multiple modes and approximating any distribution to an arbitrary degree of accuracy, while maintaining some analytical tractability. Unfortunately, variational inference using Gaussian mixtures with full-covariance structures suffers from a quadratic growth in variational parameters with the number of model parameters. The existence of multiple local minima due to strong nonconvex trends in the loss functions often associated with variational inference present additional complications, These challenges motivate the need for robust initialization procedures to improve the performance and computational scalability of variational inference with mixture models. In this work, we propose a method for constructing an initial Gaussian mixture model approximation that can be used to warm-start the iterative solvers for variational inference. The procedure begins with a global optimization stage in model parameter space. In this step, local gradient-based optimization, globalized through multistart, is used to determine a set of local maxima, which we take to approximate the mixture component centers. Around each mode, a local Gaussian approximation is constructed via the Laplace approximation. Finally, the mixture weights are determined through constrained least squares regression. The robustness and scalability of the proposed methodology is demonstrated through application to an ensemble of synthetic tests using high-dimensional, multimodal probability density functions. Here, the practical aspects of the approach are demonstrated with inversion problems in structural dynamics.
Recent experiences in utilizing the global optimization methodology, based on polynomial and neural network techniques for fluid machinery design are summarized. Global optimization methods can utilize the information collected from various sources and by different tools. These methods offer multi-criterion optimization, handle the existence of multiple design points and trade-offs via insight into the entire design space can easily perform tasks in parallel, and are often effective in filtering the noise intrinsic to numerical and experimental data. Another advantage is that these methods do not need to calculate the sensitivity of each design variable locally. However, a successful application of the global optimization method needs to address issues related to data requirements with an increase in the number of design variables and methods for predicting the model performance. Examples of applications selected from rocket propulsion components including a supersonic turbine and an injector element and a turbulent flow diffuser are used to illustrate the usefulness of the global optimization method.
The global optimization of protective structural designs for spacecraft subject to hypervelocity meteoroid and space debris impacts is presented. This nonlinear problem is first formulated for weight minimization of the space station core module configuration using the Nysmith impact predictor. Next, the equivalence and uniqueness of local and global optima is shown using properties of convexity. This analysis results in a new feasibility condition for this problem. The solution existence is then shown, followed by a comparison of optimization techniques. Finally, a sensitivity analysis is presented to determine the effects of variations in the systemic parameters on optimal design. The results show that global optimization of this problem is unique and may be achieved by a number of methods, provided the feasibility condition is satisfied. Furthermore, module structural design thicknesses and weight increase with increasing projectile velocity and diameter and decrease with increasing separation between bumper and wall for the Nysmith predictor.
Modern computational and experimental tools for aerodynamics and propulsion applications have matured to a stage where they can provide substantial insight into engineering processes involving fluid flows, and can be fruitfully utilized to help improve the design of practical devices. In particular, rapid and continuous development in aerospace engineering demands that new design concepts be regularly proposed to meet goals for increased performance, robustness and safety while concurrently decreasing cost. To date, the majority of the effort in design optimization of fluid dynamics has relied on gradient-based search algorithms. Global optimization methods can utilize the information collected from various sources and by different tools. These methods offer multi-criterion optimization, handle the existence of multiple design points and trade-offs via insight into the entire design space, can easily perform tasks in parallel, and are often effective in filtering the noise intrinsic to numerical and experimental data. However, a successful application of the global optimization method needs to address issues related to data requirements with an increase in the number of design variables, and methods for predicting the model performance. In this article, we review recent progress made in establishing suitable global optimization techniques employing neural network and polynomial-based response surface methodologies. Issues addressed include techniques for construction of the response surface, design of experiment techniques for supplying information in an economical manner, optimization procedures and multi-level techniques, and assessment of relative performance between polynomials and neural networks. Examples drawn from wing aerodynamics, turbulent diffuser flows, gas-gas injectors, and supersonic turbines are employed to help demonstrate the issues involved in an engineering design context. Both the usefulness of the existing knowledge to aid current design practices and the need for future research are identified.