Exploring risk-averse design criteria for sequential optimal experimental design in a Bayesian setting.
Abstract not provided.
SEARCH · Engineering Papers
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Abstract not provided.
Abstract not provided.
We present a mathematical framework and computational methods for optimally designing a finite sequence of experiments. This sequential optimal experimental design (sOED) problem is formulated as a finite-horizon partially observable Markov decision process (POMDP) under a Bayesian setting and with information-theoretic utilities. The formulation is general and may accommodate continuous random variables, non-Gaussian posteriors, and nonlinear forward models. The sOED design policy incorporates elements of feedback and lookahead simultaneously, and we show it to generalize the commonly-used batch and greedy design strategies. We solve for the sOED policy using the policy gradient (PG) method from reinforcement learning, and provide a derivation for the PG expression in the sOED context. Adopting an actor-critic approach, the policy and value functions are parameterized using deep neural networks and improved via PG estimates produced from simulated episodes of designs and observations. The new PG-sOED algorithm is first validated on a linear-Gaussian benchmark, and then compared against other design baselines on a sensor movement problem for contaminant source inversion in a convection-diffusion field. As a result, we provide explanation for the policy behaviors using knowledge of the underlying physical process.
Experiments are indispensable for developing models of complex systems. Carefully designed experiments can provide substantial savings for these expensive data-acquisition opportunities. However, designs based on heuristics are often suboptimal for systems with multiphysics, nonlinear dynamics, and uncertain and noisy environments. Optimal experimental design, while leveraging predictive models, seeks to systematically quantify and maximize the value of experiments. In this project, we focused on the design of multiple experiments, where current approaches are largely suboptimal: batch-design does not adapt to new data acquired during the experiment campaign (no feedback), and greedy/myopic design ignores future dynamics and consequences (no lookahead). We developed the mathematical framework and computational methods for sequential optimal experimental design (sOED) for complex systems. We enabled tractable model-based sOED in a rigorous manner through novel algorithms based on reinforcement learning, and investigated the effects of human experimenters on the design process. Our methods are fully Bayesian, able to quantify and update uncertainty in a principled manner. The traits aimed by our approach—mathematical rigor and optimality, human effects and uncertainty quantification, computational practicality—are crucial for elevating the standards of artificial intelligence (AI) to support decision-making in scientific domains, and contribute toward trust and realistic adoption of AI in experimental design practice.
In computer simulation and optimal design, sequential batch sampling offers an appealing way to iteratively stipulate optimal sampling points based upon existing selections and efficiently construct surrogate modeling. Nonetheless, the issue of near duplicates poses tremendous quandary for sequential learning. It refers to the situation that selected critical points cluster together in each sampling batch, which are individually but not collectively informative towards the optimal design. Near duplicates severely diminish the computational efficiency as they barely contribute extra information towards update of the surrogate. To address this issue, we impose a dispersion criterion on concurrent selection of sampling points, which essentially forces a sparse distribution of critical points in each batch, and demonstrate the effectiveness of this approach in adaptive contour estimation. Specifically, we adopt Gaussian process surrogate to emulate the simulator, acquire variance reduction of the critical region from new sampling points as a dispersion criterion, and combine it with the modified expected improvement (EI) function for critical batch selection. The critical region here is the proximity of the contour of interest. This proposed approach is vindicated in numerical examples of a two-dimensional four-branch function, a four-dimensional function with a disjoint contour of interest and a time-delay dynamic system.
Abstract not provided.
The ensemble Kalman filter (EnKF) has been widely used in parameter estimation for hydrological models. The focus of most previous studies was to develop more efficient analysis (estimation) algorithms. On the other hand, it is intuitively understandable that a well-designed sampling (data-collection) strategy should provide more informative measurements and subsequently improve the parameter estimation. In this work, a Sequential Ensemble-based Optimal Design (SEOD) method, coupled with EnKF, information theory and sequential optimal design, is proposed to improve the performance of parameter estimation. Based on the first-order and second-order statistics, different information metrics including the Shannon entropy difference (SD), degrees of freedom for signal (DFS) and relative entropy (RE) are used to design the optimal sampling strategy, respectively. The effectiveness of the proposed method is illustrated by synthetic one-dimensional and two-dimensional unsaturated flow case studies. It is shown that the designed sampling strategies can provide more accurate parameter estimation and state prediction compared with conventional sampling strategies. Optimal sampling designs based on various information metrics perform similarly in our cases. Furthermore, the effect of ensemble size on the optimal design is also investigated. Overall, larger ensemble size improves the parameter estimation and convergence of optimal sampling strategy. Although the proposed method is applied to unsaturated flow problems in this study, it can be equally applied in any other hydrological problems.
While most performance metrics of high–explosive (HE) based devices like detonation velocity, detonation pressure, and energy output are expected to degrade over time, the evolution of initiation threshold appears less clear, with claims of both increasing and decreasing trends in threshold having been made in the literature. This work analyzes D–optimally designed sequential binary test data for a few thermally conditioned porous–powder and polymer–bonded HE initiator systems using a Bayesian likelihood method employing the probit regression model. Here we find that in most cases the initiation threshold decreases (i.e., sensitivity increases) upon accelerated thermal conditioning. However, such results are nuanced and influenced by factors like the contact area of initiating stimulus, HE characteristics like density and specific surface area, as well as possible thermally induced changes to other materials and interfaces involved.
Abstract not provided.
Abstract not provided.
Abstract not provided.
Abstract. In this study, wind farms were optimized to show the benefit of coupling complete turbine design and layout optimization as well as including two different turbine designs in a fixed 1-to-1 ratio in a single wind farm. For our purposes, the variables in each turbine optimization include hub height, rotor diameter, rated power, tower diameter, tower shell thickness, and implicit blade chord-and-twist distributions. A 32-turbine wind farm and a 60-turbine wind farm were both considered, as well as a variety of turbine spacings and wind shear exponents. Structural constraints as well as turbine costs were considered in the optimization. Results indicate that coupled turbine design and layout optimization is superior to sequentially optimizing turbine design, then turbine layout. Coupled optimization results in an additional 2 %–5 % reduction in the cost of energy compared to optimizing sequentially for wind farms with turbine spacings of 8.5–11 rotor diameters. Smaller wind farms benefit even more from coupled optimization. Furthermore, wind farms with closely spaced wind turbines can greatly benefit from nonuniform turbine design throughout the farm. Some of these wind farms with heterogeneous turbine design have an additional 10 % cost-of-energy reduction compared to wind farms with identical turbines throughout the farm.
Traditional approaches to design and optimization of a new system often use a system-centric objective and do not take into consideration how the operator will use this new system alongside other existing systems. When the new system design is incorporated into the broader group of systems, the performance of the operator-level objective can be sub-optimal due to the unmodeled interaction between the new system and the other systems. Among the few available references that describe attempts to address this disconnect, most follow an MDO (Multidisciplinary Design Optimization)-motivated sequential decomposition approach of first designing a very good system and then providing this system to the operator who, decides the best way to use this new system along with the existing systems. This paper addresses this issue by including aircraft design, airline operations, and revenue management "subspaces"; and presents an approach that could simultaneously solve these subspaces posed as a monolithic optimization problem rather than the traditional approach described above. The monolithic approach makes the problem an expensive Mixed Integer Non-Linear Programming problem, which are extremely difficult to solve. To address the problem, we use a recently developed optimization framework that simultaneously solves the subspaces to capture the "synergy" in the problem that the previous decomposition approaches did not exploit, addresses mixed-integer/discrete type design variables in an efficient manner, and accounts for computationally expensive analysis tools. This approach solves an 11-route airline network problem consisting of 94 decision variables including 33 integer and 61 continuous type variables. Simultaneously solving the subspaces leads to significant improvement in the fleet-level objective of the airline when compared to the previously developed sequential subspace decomposition approach.
Modern distributed hybrid and electric propulsion aircraft, including vertical, short, and conventional takeoff and landing configurations, exhibit significant aero-propulsive complexity and a large number of interacting test factors. This paper presents the development and evaluation of experiment designs for aero-propulsive characterization of distributed propulsion aircraft. Five different foundational response surface designs are evaluated to inform the development of two sequential design approaches tailored to complex aircraft aerodynamic characterization experiments. The first approach, which builds on sequential face-centered central composite designs, has been used previously to develop aero-propulsive models for complex aircraft using wind tunnel testing. The second approach is a new design strategy leveraging a regular I-optimal and nested I-optimal design that was developed for this study. The two sequential design strategies are compared for experiments with a large number of test factors using pre-experiment design evaluation metrics, as well as modeling results obtained from simulated wind tunnel data for the NASA LA-8 aircraft. The design evaluation metrics show that the sequential I-optimal base design has higher statistical power, lower correlation among candidate regressors, lower prediction variance, and more precise parameter estimates. The simulated wind tunnel experiments conducted using each design reveal that the sequential I-optimal base design has better predictive capability with fewer test points. The experiment design and evaluation procedures are described in detail to inform future aerodynamic characterization experiments for complex aircraft.
Presentation explaining the new advancements for optimal design of sequential experimental campaign. Scalable computation was achieved through decomposition strategies on models relevant to initiatives across the IDAES software and supporting research portfolio (e.g., PrOMMiS, CCSI2).