Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Gaussian process surrogate”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 127 records · Page 7
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Time-series forecasting using manifold learning, radial basis function interpolation, and geometric harmonics

We address a three-tier numerical framework based on nonlinear manifold learning for the forecasting of high-dimensional time series, relaxing the “curse of dimensionality” related to the training phase of surrogate/machine learning models. At the first step, we embed the high-dimensional time series into a reduced low-dimensional space using nonlinear manifold learning (local linear embedding and parsimonious diffusion maps). Then, we construct reduced-order surrogate models on the manifold (here, for our illustrations, we used multivariate autoregressive and Gaussian process regression models) to forecast the embedded dynamics. Finally, we solve the pre-image problem, thus lifting the embedded time series back to the original high-dimensional space using radial basis function interpolation and geometric harmonics. The proposed numerical data-driven scheme can also be applied as a reduced-order model procedure for the numerical solution/propagation of the (transient) dynamics of partial differential equations (PDEs). In conclusion, we assess the performance of the proposed scheme via three different families of problems: (a) the forecasting of synthetic time series generated by three simplistic linear and weakly nonlinear stochastic models resembling electroencephalography signals, (b) the prediction/propagation of the solution profiles of a linear parabolic PDE and the Brusselator model (a set of two nonlinear parabolic PDEs), and (c) the forecasting of a real-world data set containing daily time series of ten key foreign exchange rates spanning the time period 3 September 2001–29 October 2020.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2023

Leverage Score Sampling for Parametric PDEs (Final Technical Report)

This final technical report summarizes the accomplishments of work performed under DOE Office of Science Award DE-SC0022266, which is titled “Leverage Score Sampling for Parametric PDEs”. The goal of the project was to extend methods from Randomized Numerical Linear Algebra (RandNLA) to tackle central computational challenges in model order reduction and uncertainty quantification (UQ) for parametric partial differential equations (PDEs). In particular, we sought to use importance sampling methods originally developed for RandNLA to develop sample efficient active learning algorithms for approximating high-dimensional scalar functions, e.g. by polynomials, Gaussian process models, and simple neural networks. Such methods can be immediately applied to developing surrogate models or to approximating quantity of interest (QoI) surfaces. In the context of PDEs, each sample used for learning equates to the solution of the differential equation for a particular set of parameters, so sample efficiency translates to improved computational efficiency for a variety of downstream tasks.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Efficient Reliability Analysis using Generalized Multifidelity Modeling and Explainable Active Learning

To assess the reliability of critical technologies like nuclear plants and infrastructure systems and improve the robustness of design, engineers have to quantify the uncertainties surrounding the system behavior accurately. However, the complexity of the problem can make standard reliability analysis algorithms prohibitively expensive, primarily due to the high computational cost of estimating the system response at each iteration. This cost can be greatly reduced by using multi-fidelity modeling and machine learning to build a surrogate model to replace the expensive response function. We propose a general and robust method for building surrogates from multiple Low Fidelity (LF) models coupled with machine learning to retain accuracy. Our framework first constructs “Corrected Low Fidelity models” (CLFs) by coupling a High Fidelity (HF) model inferred Gaussian Process correction term with each of the LF models. It then uses the correction terms to assign model probabilities to each of these CLFs in an explainable way before using them to assemble the final surrogate. No assumptions are made about the type of the LF models or their correlation with the HF model. The proposed surrogate modeling framework is used within the subset simulation algorithm (a variance-reduced MCMC-based reliability analysis algorithm) for enhanced efficiency. Additionally, an active learning step is added to the algorithm to adaptively decide when the surrogate is not sufficiently accurate, at which point the HF model is called and used to refine the surrogate. Through a frame buckling example, our method is shown to be highly efficient at reducing the expensive HF model calls while accurately estimating the failure probability.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Weighted greedy-optimal design of computer experiments for kernel-based and Gaussian process model emulation and calibration

This article is concerned with the approximation of high-dimensional functions by kernel-based methods. Motivated by uncertainty quantification, which often necessitates the construction of approximations that are accurate with respect to a probability density function of random variables, we aim at minimizing the approximation error with respect to a weighted $L^p$-norm. We present a greedy procedure for designing computer experiments based upon a weighted modification of the pivoted Cholesky factorization. The method successively generates nested samples with the goal of minimizing error in regions of high probability. Numerical experiments validate that this new importance sampling strategy is superior to other sampling approaches, especially when used with non-product probability density functions. We also show how to use the proposed algorithm to efficiently generate surrogates for inferring unknown model parameters from data.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Scalable Risk Assessment of Rare Events in Power Systems With Uncertain Wind Generation and Loads

Risk assessment of rare events has become increasingly important in power system planning and operation with the increasing integration of renewable energy and the presence of system uncertainties. However, quantifying the risk posed by rare events via the traditional method, i.e., Monte Carlo sampling (MCS), incurs substantial computational expense stemming from the vast ensemble of power flow simulations. To accelerate the assessment, this paper proposes a Deep Neural Network (DNN)-kernelized vector-valued Gaussian Process (VVGP) approach with excellent computational efficiency while maintaining high accuracy. Consequently, serving as a surrogate model for the power flow solver, the DNN-kernelized VVGP enables significantly faster but accurate risk assessment compared to the power flow solver. The developed surrogate model evaluates low-order N - k events that contain more than 90% instances by adeptly capturing the topological features while the high-order N - k events are assessed via a power flow solver, thereby striking a balance between computational efficiency and uncertainty quantification accuracy. Moreover, the model incorporates a Support Vector Machine (SVM) classifier to resample concerning low-probability tail events to counteract the biases potentially introduced during the DNN-kernelized VVGP evaluations. Simulations conducted on the modified IEEE 24-bus, 118-bus, and European 1354-bus systems demonstrate that the proposed method maintains the accuracy benchmark set by MCS while significantly reducing computational demands in large-scale power systems as compared to other state-of-the-art methods.

17 WIND ENERGY↗
DOE OSTI2024

Hybrid Parameter Search and Dynamic Model Selection for Mixed-Variable Bayesian Optimization

Herein this article presents a new type of hybrid model for Bayesian optimization (BO) adept at managing mixed variables, encompassing both quantitative (continuous and integer) and qualitative (categorical) types. Our proposed new hybrid models (named hybridM) merge the Monte Carlo Tree Search structure (MCTS) for categorical variables with Gaussian Processes (GP) for continuous ones. hybridM leverages the upper confidence bound tree search (UCTS) for MCTS strategy, showcasing the tree architecture’s integration into Bayesian optimization. Our innovations, including dynamic online kernel selection in the surrogate modeling phase and a unique UCTS search strategy, position our hybrid models as an advancement in mixed-variable surrogate models. Numerical experiments underscore the superiority of hybrid models, highlighting their potential in Bayesian optimization.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

AI and extreme scale computing to learn and infer the physics of higher order gravitational wave modes of quasi-circular, spinning, non-precessing black hole mergers

We use artificial intelligence (AI) to learn and infer the physics of higher order gravitational wave modes of quasi-circular, spinning, non precessing binary black hole mergers. We trained AI models using 14 million waveforms, produced with the surrogate model NRHybSur3dq8, that include modes up to $\ell$ ≤ 4 and (5,5), except for (4,0) and (4,1), that describe binaries with mass-ratios $\textit{q}$ ≤ 8, individual spins $s^z_{\{1,2\}} \in$[–0.8,0.8], and inclination angle $θ \in$ [0,π]. Our probabilistic AI surrogates can accurately constrain the mass-ratio, individual spins, effective spin, and inclination angle of numerical relativity waveforms that describe such signal manifold. We compared the predictions of our AI models with Gaussian process regression, random forest, k-nearest neighbors, and linear regression, and with traditional Bayesian inference methods through the PyCBC Inference toolkit, finding that AI outperforms all these approaches in terms of accuracy, and are between three to four orders of magnitude faster than traditional Bayesian inference methods. Our AI surrogates were trained within 3.4 hours using distributed training on 1,536 NVIDIA V100 GPUs in the Summit supercomputer.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2024

Discovering the Unknowns: A First Step

This article aims at discovering the unknown variables in the system through data analysis. The main idea is to use the time of data collection as a surrogate variable and try to identify the unknown variables by modeling gradual and sudden changes in the data. We use Gaussian process modeling and a sparse representation of the sudden changes to efficiently estimate the large number of parameters in the proposed statistical model. The method is tested on a realistic dataset generated using a one-dimensional implementation of a Magnetized Liner Inertial Fusion (MagLIF) simulation model, and encouraging results are obtained.

42 ENGINEERING↗
DOE OSTI2025

Direct Deoxygenation of Phenol over Fe-Based Bimetallic Surfaces Using On-the-Fly Surrogate Models

We present an accelerated nudged elastic band (NEB) study of phenol direct deoxygenation (DDO) on Fe-based bimetallic surfaces using a recently developed Gaussian process regression (GPR) calculator. Our test calculations demonstrate that the GPR calculator achieves up to 3 times speedup compared to conventional density functional theory calculations while maintaining high accuracy, with energy barrier errors below 0.015 eV. Using GPR-NEB, we systematically examine the DDO mechanism on pure Fe(110) and surfaces modified with Co and Ni in both top and subsurface layers. Our results show that subsurface Co and Ni substitutions preserve favorable thermodynamics and kinetics for both C–O bond cleavage and C–H bond formation, comparable to those on the pure Fe(110) surface. In contrast, top-layer substitutions generally increase the C–O bond cleavage barrier, render the step endothermic, and result in significantly higher reverse reaction rates, making DDO unfavorable on these surfaces. This work demonstrates the effectiveness of GRR-accelerated transition state searches for complex surface reactions and provides insights into rational design of bimetallic catalysts for selective deoxygenation.

Aromatic compounds↗
DOE OSTI2022

Uncertainty Aware Deep Learning for Particle Accelerators

Standard deep learning models for classification and regression applications are ideal for capturing complex system dynamics. However, their predictions can be arbitrarily inaccurate when the input samples are not similar to the training data. Implementation of distance aware uncertainty estimation can be used to detect these scenarios and provide a level of confidence associated with their predictions. In this paper, we present results from using Deep Gaussian Process Approximation (DGPA) methods for errant beam prediction at Spallation Neutron Source (SNS) accelerator (classification) and we provide an uncertainty aware surrogate model for the Fermi National Accelerator Lab (FNAL) Booster Accelerator Complex (regression).

Rajput, Kishansingh↗
DOE OSTI2021

Testing Surrogate-Based Optimization with the Fortified Branin-Hoo Extended to Four Dimensions

Some popular functions used to test global optimization algorithms have multiple local optima, all with the same value, making them all global optima. It is easy to make them more challenging by fortifying them via adding a localized bump at the location of one of the optima. In previous work the authors illustrated this for the Branin-Hoo function and the popular differential evolution algorithm, showing that the fortified Branin-Hoo required an order of magnitude more function evaluations. This paper examines the effect of fortifying the Branin-Hoo function on surrogate- based optimization, which usually proceeds by adaptive sampling. Two algorithms are considered. The EGO algorithm, which is based on a Gaussian process (GP) and an algorithm based on radial basis functions (RBF). EGO is found to be more frugal in terms of the number of required function evaluations required to identify the correct basin, but it is expensive to run on a desktop, limiting the number of times the runs could be repeated to establish sound statistics on the number of required function evaluations. The RBF algorithm was cheaper to run, providing more sound statistics on performance. A four-dimensional version of the Branin-Hoo function was introduced in order to assess the effect of dimensionality. Furthermore, it was found that the difference between the ordinary function and the fortified one was much more pronounced for the four-dimensional function compared to the two dimensional one.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Short-Term Probabilistic Solar Forecasting via Reinforcement Learning over ECMWF

In this paper, we present an innovative reinforcement learning approach for short-term solar forecasting, leveraging data from the European Centre for Medium-Range Weather Forecasts (ECMWF). The methodology begins with the application of the System Advisor Model (SAM) to transform various ECMWF numerical weather prediction members into predictive photovoltaic power generation. To enhance the precision of deterministic forecasting, we introduce a dynamic model selection algorithm based on Q-learning. This algorithm dynamically identifies and utilizes the most accurate ensemble member for forecasting purposes. Furthermore, we employ a support vector regression surrogate model with a Gaussian distribution to generate probabilistic forecasts, providing a holistic view of solar energy generation uncertainty. To expedite the training process and make it more practical for real-world applications, we integrate a rolling update workflow. This innovative workflow reduces the training period from months to a mere 19 days, making our method highly efficient. Numerical results of the case study show that in comparison to benchmark models, the proposed method improves the deterministic and probabilistic solar forecasting accuracy by up to 40.84% and 48.42%, respectively.

ensemble forecasting↗
DOE OSTI2022

Multi-fidelity modeling to predict the rheological properties of a suspension of fibers using neural networks and Gaussian processes

Unveiling the rheological properties of fiber suspensions is of paramount interest to many industrial applications. There are multiple factors, such as fiber aspect ratio and volume fraction, that play a significant role in altering the rheological behavior of suspensions. Three-dimensional (3D) numerical simulations of coupled differential equations of the suspension of fibers are computationally expensive and time-consuming. Machine learning algorithms can be trained on the available data and make predictions for the cases where no numerical data are available. However, some widely used machine learning surrogates, such as neural networks, require a relatively large training dataset to produce accurate predictions. Multi-fidelity models, which combine high-fidelity data from numerical simulations and less expensive lower fidelity data from resources such as simplified constitutive equations, can pave the way for more accurate predictions. Here, we focus on neural networks and the Gaussian processes with two levels of fidelity, i.e., high and low fidelity networks, to predict the steady-state rheological properties, and compare them to the single-fidelity network. High-fidelity data are obtained from direct numerical simulations based on an immersed boundary method to couple the fluid and solid motion. The low-fidelity data are produced by using constitutive equations. Multiple neural networks and the Gaussian process structures are used for the hyperparameter tuning purpose. Results indicate that with the best choice of hyperparameters, both the multi-fidelity Gaussian processes and neural networks are capable of making predictions with a high level of accuracy with neural networks demonstrating marginally better performance.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2025

Active oversight and quality control in standard Bayesian optimization for autonomous experiments

The fusion of experimental automation and machine learning has catalyzed a new era in materials research, prominently featuring Gaussian Process (GP) Bayesian Optimization (BO) driven autonomous experiments. Here we introduce a Dual-GP approach that enhances traditional GPBO by adding a secondary surrogate model to dynamically constrain the experimental space based on real-time assessments of the raw experimental data. This Dual-GP approach enhances the optimization efficiency of traditional GPBO by isolating more promising space for BO sampling and more valuable experimental data for primary GP training. We also incorporate a flexible, human-in-the-loop intervention method in the Dual-GP workflow to adjust for unanticipated results. We demonstrate the effectiveness of the Dual-GP model with synthetic model data and implement this approach in autonomous pulsed laser deposition experimental data. This Dual-GP approach has broad applicability in diverse GPBO-driven experimental settings, providing a more adaptable and precise framework for refining autonomous experimentation for more efficient optimization.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Probabilistic projections of the Amery Ice Shelf catchment, Antarctica, under conditions of high ice-shelf basal melt

Abstract. Antarctica's Lambert Glacier drains about one-sixth of the ice from the East Antarctic Ice Sheet and is considered stable due to the strong buttressing provided by the Amery Ice Shelf. While previous projections of the sea-level contribution from this sector of the ice sheet have predicted significant mass loss only with near-complete removal of the ice shelf, the ocean warming necessary for this was deemed unlikely. Recent climate projections through 2300 indicate that sufficient ocean warming is a distinct possibility after 2100. This work explores the impact of parametric uncertainty on projections of the response of the Lambert–Amery system (hereafter “the Amery sector”) to abrupt ocean warming through Bayesian calibration of a perturbed-parameter ice-sheet model ensemble. We address the computational cost of uncertainty quantification for ice-sheet model projections via statistical emulation, which employs surrogate models for fast and inexpensive parameter space exploration while retaining critical features of the high-fidelity simulations. To this end, we build Gaussian process (GP) emulators from simulations of the Amery sector at a medium resolution (4–20 km mesh) using the Model for Prediction Across Scales (MPAS)-Albany Land Ice (MALI) model. We consider six input parameters that control basal friction, ice stiffness, calving, and ice-shelf basal melting. From these, we generate 200 perturbed input parameter initializations using space filling Sobol sampling. For our end-to-end probabilistic modeling workflow, we first train emulators on the simulation ensemble and then calibrate the input parameters using observations of the mass balance, grounding line movement, and calving front movement with priors assigned via expert knowledge. Next, we use MALI to project a subset of simulations to 2300 using ocean and atmosphere forcings from a climate model for both low- and high-greenhouse-gas-emission scenarios. From these simulation outputs, we build multivariate emulators by combining GP regression with principal component dimension reduction to emulate multivariate sea-level contribution time series data from the MALI simulations. We then use these emulators to propagate uncertainty from model input parameters to predictions of glacier mass loss through 2300, demonstrating that the calibrated posterior distributions have both greater mass loss and reduced variance compared to the uncalibrated prior distributions. Parametric uncertainty is large enough through about 2130 that the two projections under different emission scenarios are indistinguishable from one another. However, after rapid ocean warming in the first half of the 22nd century, the projections become statistically distinct within decades. Overall, this study demonstrates an efficient Bayesian calibration and uncertainty propagation workflow for ice-sheet model projections and identifies the potential for large sea-level rise contributions from the Amery sector of the Antarctic Ice Sheet after 2100 under high-greenhouse-gas-emission scenarios.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Development of advanced machine learning models for analysis of plutonium surrogate optical emission spectra

This work investigates and applies machine learning paradigms seldom seen in analytical spectroscopy for quantification of gallium in cerium matrices via processing of laser-plasma spectra. Ensemble regressions, support vector machine regressions, Gaussian kernel regressions, and artificial neural network techniques are trained and tested on cerium-gallium pellet spectra. A thorough hyperparameter optimization experiment is conducted initially to determine the best design features for each model. The optimized models are evaluated for sensitivity and precision using the limit of detection (LoD) and root mean-squared error of prediction (RMSEP) metrics, respectively. Gaussian kernel regression yields the superlative predictive model with an RMSEP of 0.33% and an LoD of 0.015% for quantification of Ga in a Ce matrix. This study concludes that these machine learning methods could yield robust prediction models for rapid quality control analysis of plutonium alloys.

Rao, Ashwin P. (ORCID:0000000319312568)↗
DOE OSTI2021

Cholesky-based experimental design for Gaussian process and kernel-based emulation and calibration.

Gaussian processes and other kernel-based methods are used extensively to construct approximations of multivariate data sets. The accuracy of these approximations is dependent on the data used. This paper presents a computationally efficient algorithm to greedily select training samples that minimize the weighted L p error of kernel-based approximations for a given number of data. The method successively generates nested samples, with the goal of minimizing the error in high probability regions of densities specified by users. The algorithm presented is extremely simple and can be implemented using existing pivoted Cholesky factorization methods. Training samples are generated in batches which allows training data to be evaluated (labeled) in parallel. For smooth kernels, the algorithm performs comparably with the greedy integrated variance design but has significantly lower complexity. Numerical experiments demonstrate the efficacy of the approach for bounded, unbounded, multi-modal and non-tensor product densities. We also show how to use the proposed algorithm to efficiently generate surrogates for inferring unknown model parameters from data using Bayesian inference.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Robust Parameter Design on Dual Stochastic Response Models With Constrained Bayesian Optimization

In engineering system design, minimizing the variations of the quality measurements while guaranteeing their overall quality up to certain levels, namely the robust parameter design (RPD), is crucial. Recent works have dealt with the design of a system whose response-control variables relationship is a deterministic function with a complex shape and function evaluation is expensive. In this work, we propose a Bayesian optimization method for the RPD of stochastic functions. Dual stochastic response models are carefully designed for stochastic functions. The heterogeneous variance of the sample mean is addressed by the predictive mean of the log variance surrogate model in a two-step approach. We establish an acquisition function that favors exploration across the feasible and optimality-improvable regions to effectively and efficiently solve the stochastic constrained optimization problem. Further, the performance of our proposed method is demonstrated by the extensive numerical and case studies. Note to Practitioners-Many manufacturing processes involve undesirable variations, which create variations in the final products. For example, many emerging manufacturing processes, such as nanomanufacturing, involve complex physical and chemical dynamics and transformation, creating variations in the manufacturing output. In such processes, it is crucial to design the manufacturing processes or products so that they have minimum variations in their quality. Meanwhile, it is also important to maintain the overall quality of the designed processes or products. Furthermore, acquiring data from many advanced manufacturing processes is often very costly, especially in the designing stage. In this work, we propose a data-driven method that automatically finds the best setting of manufacturing processes or products with the minimum variations of quality and a given constraint on the average quality satisfied. Our proposed method is used before conducting every experiment; It analyzes the historical data from previous experiments and provides a setting to be used in the next experiment. Our proposed method efficiently utilizes the historical data, and thus finds the best robust setting by conducting only a small number of experiments.

42 ENGINEERING↗