Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Derivative-free optimization”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

A simplified, efficient approach to hybrid wind and solar plant site optimization

Abstract. Wind plant layout optimization is a difficult, complex problem with a large number of variables and many local minima. Layout optimization only becomes more difficult with the addition of solar generation. In this paper, we propose a parameterized approach to wind and solar hybrid power plant layout optimization that greatly reduces problem dimensionality while guaranteeing that the generated layouts have a desirable regular structure. Thus far, hybrid power plant optimization research has focused on system sizing. We go beyond sizing and present a practical approach to optimizing the physical layout of a wind–solar hybrid power plant. We argue that the evolution strategy class of derivative-free optimization methods is well-suited to the parameterized hybrid layout problem, and we demonstrate how hard layout constraints (e.g., placement restrictions) can be transformed into soft constraints that are amenable to optimization using evolution strategies. Next, we present experimental results on four test sites, demonstrating the viability, reliability, and effectiveness of the parameterized evolution strategy approach for generating optimized hybrid plant layouts. Completing the tool kit for parameterized layout generation, we include a brief tutorial describing how the parameterized evolutionary approach can be inspected, understood, and debugged when applied to hybrid plant layouts.

14 SOLAR ENERGY↗
DOE OSTI2022

Isotherm Modeling and Techno-Economic Analysis of Contactor Technologies for New Tetraamine-Appended MOF for NGCC Applications

Recently, a family of tetraamine-functionalized metal-organic frameworks (MOF) has been reported as promising sorbent materials for capturing CO2 from flue gas conditions relevant to natural gas combined cycle (NGCC) applications. The main advantages of these materials are their two-step cooperative CO2 adsorption, which gives rise to unusual two step-shaped CO2 adsorption profiles and their high thermal stability. This work presents the modelling of the two-transition isotherm of the tetraamine-appended MOF, N,N'-bis(3-aminopropyl)-1,4-diaminobutane (3-4-3)-appended Mg2(dobpdc), and the techno-economic analysis (TEA) of carbon capture processes utilizing this sorbent. Due to the unusual isotherm shapes of the experimental CO2 adsorption data for tetraamine-appended Mg2(dobpdc) and the strong nonlinearity of CO2 loading with respect to temperature and pressure, we tested two different models which use logistic functions for representing the different isotherm behaviors in the different pressure ranges. The first model uses the quadratic isotherm model in the low-pressure region, the Langmuir isotherm model in the middle pressure range, and the dual site Langmuir isotherm model in the high-pressure range. To model the transition between regions we used the arctangent functions independent of temperature and the thermal effect was accounted by using the Clausius-Clapeyron relation]. The second model is an extension of the weighted dual-site Langmuir isotherm model presented by Hughes et al.. In this extended model, the dual-site Langmuir isotherm is employed in the three transition regions, using temperature-dependent logistic functions to activate or deactivate the isotherm model in the low, middle, and high-pressure ranges. Both models fit the experimental data quite well with root mean squared errors (RMSE’s) of 0.41 and 0.17 for model 1 and model 2, respectively. Since model 2 resulted in a lower RMSE, it was leveraged for the development of the gas/solid contactor models used by the TEA. Specifically, two different contactor models, an axial-flow fixed bed and moving bed contactor, were developed as part of this work. These models are dynamic, pressure-driven, and consist of mass, energy, and momentum conservation equations. A kinetic model was also developed by performing parameter estimation using experimental fixed bed breakthrough data. These models are then used to simulate CO2 capture processes from the flue gas generated from a ~600 gross MW NGCC power plant. A cost model was developed which considers the capital cost of the reactors and the significant operating costs such as steam and electricity. Using NETL’s Framework for the Optimization and Quantification of Uncertainty of Uncertainty and Surrogates tool (FOQUS), which has the capability of linking models built using numerous modelling platforms with derivative-free optimization solvers, a techno-economic optimization of the carbon capture processes was performed which minimizes the cost of capture.

Caballero, Daison↗
DOE OSTI2023

Machine learning-based surrogate models and transfer learning for derivative free optimization of HT-PEM fuel cells

Widespread adoption of high-temperature polymer electrolyte membrane electrochemical systems, such as fuel cells (HT-PEMFCs), requires models and computational tools for accurate optimization and guiding new materials for enhancing performance and durability. In this contribution, knowledge-based modelling and data-driven modelling are combined using Few-Shot Learning and implementing an Automated Machine Learning framework for the generation of Machine Learning-based surrogate models. Applicability of the resulting model for derivative-free optimization is demonstrated. Additionally, a way of considering extrapolation in the optimization task is presented. Results show that although extrapolation is needed to achieve better solutions during optimization, it can be monitored and managed. As a result, tuning the electrode ionomer binder's properties, such as ionic conductivity, in the fuel cell represents a promising pathway for improving HT-PEMFC performance.

08 HYDROGEN↗
DOE OSTI2025

Evaluating pulse-shaping capabilities of next-generation pulsed power architectures

This project evaluated the pulse shaping capabilities of next-generation pulsed power (NGPP) architectures. NGPP architectures share several common attributes including multiple independent pulse-generation lines, a radial water-insulated impedance transformer, and a central vacuum insulated load region. A multi-module circuit model was developed, incorporating independent pulse-generation lines and a 2-D transmission line mesh of the radial impedance transformer to assess the effects of azimuthal asymmetry in pulse-shaped experiments. Circuit model simulations demonstrated that NGPP architectures are able to produce the the desired current pulse shapes for exemplar NGPP experiments. Additionally, the project explored automated methods for experiment design, including derivative -ree optimization and machine learning. Pulse-shaped experiments require designers to determine machine parameters that reliably produce the desired current pulse at the load, a process that typically relies on expert knowledge and iterative adjustments using the Z circuit model. Given the increased complexity of NGPP systems, this manual approach may be impractical. While the evaluated methods do not eliminate the need for manual iteration, they can reduce the time required for experiment design. Derivative-free optimization automates much of the trial-and-error process, providing a close starting point for manual adjustments or making small modifications to near-final designs. Meanwhile, deep neural network methods can generate a good qualitative match to the desired current pulse in under one second without requiring circuit model simulations.

42 ENGINEERING↗
DOE OSTI2026

Augmenting subspace optimization methods with linear bandits

In this work, we consider the framework of methods for unconstrained minimization that are, in each iteration, restricted to a model that is only a valid approximation to the objective function on some affine subspace containing an incumbent point. These methods are of practical interest in computational settings where derivative information is either expensive or impossible to obtain. Recent attention has been paid in the literature to employing randomized matrix sketching for generating the affine subspaces within this framework. We consider a relatively straightforward, deterministic augmentation of such a generic subspace optimization method. In particular, we consider a sequential optimization framework where actions consist of one-dimensional linear subspaces and rewards consist of (approximations to) the magnitudes of directional derivatives computed in the direction of the action subspace. Reward maximization in this context is consistent with maximizing lower bounds on descent guaranteed by first-order Taylor models. This sequential optimization problem can be analysed through the lens of dynamic regret. We modify an existing linear upper confidence bound (UCB) bandit method and prove sublinear dynamic regret in the subspace optimization setting. We demonstrate the efficacy of employing this linear UCB method in a setting where forward-mode algorithmic differentiation can provide directional derivatives in arbitrary directions and in a derivative-free setting. For the derivative-free setting, we propose SS-POUNDers, an extension of the derivative-free optimization method POUNDers that employs the linear UCB mechanism to identify promising subspaces. Our numerical experiments suggest a preference, in either computational setting, for employing a linear UCB mechanism within a subspace optimization method.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

A Class of Sparse Johnson–Lindenstrauss Transforms and Analysis of their Extreme Singular Values

The Johnson–Lindenstrauss (JL) lemma is a powerful tool for dimensionality reduction in modern algorithm design. The lemma states that any set of high-dimensional points in a Euclidean space can be projected into lower dimensions while approximately preserving pairwise Euclidean distances. Random matrices satisfying this lemma are called JL transforms (JLTs). Inspired by existing $s$-hashing JLTs with exactly $s$ nonzero elements on each column, the present work introduces an ensemble of sparse matrices encompassing so-called $s$-hashing-like matrices whose expected number of nonzero elements on each column is $s$. The independence of the sub-Gaussian entries of these matrices and the knowledge of their exact distribution play an important role in their analyses. Using properties of independent sub-Gaussian random variables, these matrices are demonstrated to be JLTs, and their smallest nontrivial singular values and largest singular values are estimated nonasymptotically using a technique from geometric functional analysis. As the dimensions of the matrix grow to infinity, these singular values are proved to converge almost surely to fixed quantities (by using the universal Bai–Yin law) and in distribution to the Gaussian orthogonal ensemble Tracy–Widom law after proper rescalings. Understanding the behaviors of extreme singular values is important in general because they are often used to define a measure of stability of matrix algorithms. For example, JLTs were recently used in derivative-free optimization algorithmic frameworks to select random subspaces in which are constructed random models or poll directions to achieve scalability, and hence estimating their smallest singular value in particular helps determine the dimension of these subspaces.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Structure-aware methods for expensive derivative-free nonsmooth composite optimization

We present new methods for solving a broad class of bound-constrained nonsmooth composite minimization problems. These methods are specially designed for objectives that are some known mapping of outputs from a computationally expensive function. We provide accompanying implementations of these methods: in particular, a novel manifold sampling algorithm (MS-P) with subproblems that are in a sense primal versions of the dual problems solved by previous manifold sampling methods and a method (GOOMBAH) that employs more difficult optimization subproblems. For these two methods, we provide rigorous convergence analysis and guarantees. We demonstrate extensive testing of these methods. Open-source implementations of the methods developed in this manuscript can be found at https://github.com/POptUS/ IBCDFO/.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Adaptive sampling quasi-Newton methods for zeroth-order stochastic optimization

Here, we consider unconstrained stochastic optimization problems with no available gradient information. Such problems arise in settings from derivative-free simulation optimization to reinforcement learning. We propose an adaptive sampling quasi-Newton method where we estimate the gradients using finite differences of stochastic function evaluations within a common random number framework. We develop modified versions of a norm test and an inner product quasi-Newton test to control the sample sizes used in the stochastic approximations and provide global convergence results to the neighborhood of a locally optimal solution. We present numerical experiments on simulation optimization problems to illustrate the performance of the proposed algorithm. When compared with classical zeroth-order stochastic gradient methods, we observe that our strategies of adapting the sample sizes significantly improve performance in terms of the number of stochastic function evaluations required.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Stochastic average model methods

We consider the solution of finite-sum minimization problems, such as those appearing in nonlinear least-squares or general empirical risk minimization problems. We are motivated by problems in which the summand functions are computationally expensive and evaluating all summands on every iteration of an optimization method may be undesirable. Here we present the idea of stochastic average model (SAM) methods, inspired by stochastic average gradient methods. SAM methods sample component functions on each iteration of a trust-region method according to a discrete probability distribution on component functions; the distribution is designed to minimize an upper bound on the variance of the resulting stochastic model. We present promising numerical results concerning an implemented variant extending the derivative-free model-based trust-region solver POUNDERS, which we name SAM-POUNDERS.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Long-term missing value imputation for time series data using deep neural networks

We present an approach that uses a deep learning model, in particular, a MultiLayer Perceptron, for estimating the missing values of a variable in multivariate time series data. We focus on filling a long continuous gap (e.g., multiple months of missing daily observations) rather than on individual randomly missing observations. Our proposed gap filling algorithm uses an automated method for determining the optimal MLP model architecture, thus allowing for optimal prediction performance for the given time series. We tested our approach by filling gaps of various lengths (three months to three years) in three environmental datasets with different time series characteristics, namely daily groundwater levels, daily soil moisture, and hourly Net Ecosystem Exchange. We compared the accuracy of the gap-filled values obtained with our approach to the widely used R-based time series gap filling methods ImputeTS and mtsdi. The results indicate that using an MLP for filling a large gap leads to better results, especially when the data behave nonlinearly. Thus, our approach enables the use of datasets that have a large gap in one variable, which is common in many long-term environmental monitoring observations.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Two-Stage Estimation and Variance Modeling for Latency-Constrained Variational Quantum Algorithms

The quantum approximate optimization algorithm (QAOA) has enjoyed increasing attention in noisy, intermediate-scale quantum computing with its application to combinatorial optimization problems. QAOA has the potential to demonstrate a quantum advantage for NP-hard combinatorial optimization problems. As a hybrid quantum-classical algorithm, the classical component of QAOA resembles a simulation optimization problem in which the simulation outcomes are attainable only through a quantum computer. The simulation that derives from QAOA exhibits two unique features that can have a substantial impact on the optimization process: (i) the variance of the stochastic objective values typically decreases in proportion to the optimality gap, and (ii) querying samples from a quantum computer introduces an additional latency overhead. In this paper, we introduce a novel stochastic trust-region method derived from a derivative-free, adaptive sampling trust-region optimization method intended to efficiently solve the classical optimization problem in QAOA by explicitly taking into account the two mentioned characteristics. The key idea behind the proposed algorithm involves constructing two separate local models in each iteration: a model of the objective function and a model of the variance of the objective function. Exploiting the variance model allows us to restrict the number of communications with the quantum computer and also helps navigate the nonconvex objective landscapes typical in QAOA optimization problems. In conclusion, we numerically demonstrate the superiority of our proposed algorithm using the SimOpt library and Qiskit when we consider a metric of computational burden that explicitly accounts for communication costs.

Derivative-free Optimization↗
DOE OSTI2023

Scalable computations for nonstationary Gaussian processes

Nonstationary Gaussian process models can capture complex spatially varying dependence structures in spatial datasets. However, the large number of observations in modern datasets makes fitting such models computationally intractable with conventional dense linear algebra. In addition, derivative-free or even first-order optimization methods can be very slow to converge when estimating many spatially varying parameters. In this paper, we present a computational framework which couples an algebraic block diagonal plus low-rank covariance matrix approximation with stochastic trace estimation to facilitate the efficient use of second-order solvers for maximum likelihood estimation of Gaussian process models with many parameters. We demonstrate the effectiveness of these methods by simultaneously fitting 192 parameters in the popular nonstationary model of Paciorek and Schervish using 107,600 sea surface temperature anomaly measurements.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Adaptive Sampling Trust Region Method for Bi-fidelity Simulation Optimization [SWR-25-166]

Adaptive Sampling Trust Region Method for Bi-fidelity Simulation Optimization aims to demonstrate the effect of adaptive sampling-based bi-fidelity stochastic trust region method (ASTRO-BFDF). ASTRO-BFDF, derived from a derivative-free adaptive sampling trust-region optimization (ASTRO-DF) (Shashaani et al. 2018, Ha and Shashaani 2023), intended to efficiently solve the bi-fidelity simulation optimization.

Mueller, Juliane [National Laboratory of the Rocki↗
DOE OSTI2021

Manifold Sampling for Optimizing Nonsmooth Nonconvex Compositions

Here we propose a manifold sampling algorithm for minimizing a nonsmooth composition $f= h\circ F$, where we assume $h$ is nonsmooth and may be inexpensively computed in closed form and $F$ is smooth but its Jacobian may not be available. We additionally assume that the composition $h\circ F$ defines a continuous selection. Manifold sampling algorithms can be classified as model-based derivative-free methods, in that models of $F$ are combined with particularly sampled information about $h$ to yield local models for use within a trust-region framework. We demonstrate that cluster points of the sequence of iterates generated by the manifold sampling algorithm are Clarke stationary. We consider the tractability of three particular subproblems generated by the manifold sampling algorithm and the extent to which inexact solutions to these subproblems may be tolerated. Numerical results demonstrate that manifold sampling as a derivative-free algorithm is competitive with state-of-the-art algorithms for nonsmooth optimization that utilize first-order information about $f$.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Enabling Scale-Up Through Multi-Fidelity Adaptive Computing

We present ideas from our ongoing work in adaptive computing - an optimization framework that allows us to strategically deploy various fidelity level experiments and simulations to guide decision making. The framework aims to enable uncertainty quantified scale-up of simulations and experiments, which causes increased complexity. A key feature of the framework is the integration of user-specified local model trustworthiness estimates. Adaptive sampling strategies allow us to optimally exploit the multiple fidelity level information and trustworthiness measures to arrive at the best decisions within a highly limited budget of objective function evaluations.

adaptive sampling↗
DOE OSTI2021

Physics-Informed Evolutionary Strategy Based Control for Mitigating Delayed Voltage Recovery

Here, in this work we propose a novel data-driven, real-time power system voltage stability control method based on the physics-informed guided meta evolutionary strategy (ES). The main objective is to quickly provide an adaptive control strategy to secure system voltage stability. The problem is challenging due to the high-dimensional feature of the power system model and the fast-changing and uncertain nature of power system operation scenarios. To this end, a model-free and derivative-free guided ES method is applied. The method is further combined with a meta-learning strategy to make the learnt control policy automatically adapted to unseen operation conditions and fault scenarios, which is highly desired for real-time emergency control. Last but not least, physical knowledge is embedded in the above method through a trainable action mask technique to rule out unnecessary load shedding actions for better learning and control performance. Case studies on the IEEE 300-bus system and comparisons with other state-of-the-art benchmark methods verify the superiority of the proposed physics-informed guided meta ES method in realizing fast and adaptive power system voltage stability control.

42 ENGINEERING↗