Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “gaussian process regression”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 127 records · Page 7

Adaptive sampling for accelerating neutron diffraction-based strain mapping *

Abstract Neutron diffraction is a useful technique for mapping residual strains in dense metal objects. The technique works by placing an object in the path of a neutron beam, measuring the diffracted signals and inferring the local lattice strain values from the measurement. In order to map the strains across the entire object, the object is stepped one position at a time in the path of the neutron beam, typically in raster order, and at each position a strain value is estimated. Typical dwell times at neutron diffraction instruments result in an overall measurement that can take several hours to map an object that is several tens of centimeters in each dimension at a resolution of a few millimeters, during which the end users do not have an estimate of the global strain features and are at risk of incomplete information in case of instruments outages. In this paper, we propose an object adaptive sampling strategy to measure the significant points first. We start with a small initial uniform set of measurement points across the object to be mapped, compute the strain in those positions and use a machine learning technique to predict the next position to measure in the object. Specifically, we use a Bayesian optimization based on a Gaussian process regression method to infer the underlying strain field from a sparse set of measurements and predict the next most informative positions to measure based on estimates of the mean and variance in the strain fields estimated from the previously measured points. We demonstrate our real-time measure-infer-predict workflow on additively manufactured steel parts—demonstrating that we can get an accurate strain estimate even with 30%–40% of the typical number of measurements—leading the path to faster strain mapping with useful real-time feedback. We emphasize that the proposed method is general and can be used for fast mapping of other material properties such as phase fractions from time-consuming point-wise neutron measurements.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Combining synchrotron X-ray diffraction, mechanistic modeling and machine learning for in situ subsurface temperature quantification during laser melting

Laser melting, such as that encountered during additive manufacturing, produces extreme gradients of temperature in both space and time, which in turn influence microstructural development in the material. Qualification and model validation of the process itself and the resulting material necessitate the ability to characterize these temperature fields. However, well established means to directly probe the material temperature below the surface of an alloy while it is being processed are limited. To address this gap in characterization capabilities, a novel means is presented to extract subsurface temperature-distribution metrics, with uncertainty, from in situ synchrotron X-ray diffraction measurements to provide quantitative temperature evolution data during laser melting. Temperature-distribution metrics are determined using Gaussian process regression supervised machine-learning surrogate models trained with a combination of mechanistic modeling (heat transfer and fluid flow) and X-ray diffraction simulation. The trained surrogate model uncertainties are found to range from 5 to 15% depending on the metric and current temperature. The surrogate models are then applied to experimental data to extract temperature metrics from an Inconel 625 nickel superalloy wall specimen during laser melting. The maximum temperatures of the solid phase in the diffraction volume through melting and cooling are found to reach the solidus temperature as expected, with the mean and minimum temperatures found to be several hundred degrees less. The extracted temperature metrics near melting are determined to be more accurate because of the lower relative levels of mechanical elastic strains. However, uncertainties for temperature metrics during cooling are increased due to the effects of thermomechanical stress.

36 MATERIALS SCIENCE↗

Industrial battery operation and utilization in the presence of electrical load uncertainty using Bayesian decision theory

Behind the meter battery storage is becoming increasing popular in all sectors, though enthusiasm has recently lagged in the industrial sector. Even though there may be many factors contributing to this including lack of innovation, prohibitive costs, and undesirable rate structures, a difficulty arises in accounting for uncertainty of electrical load in industrial facilities while still attempting to utilize battery storage as much as possible all while trying to achieve fiscal profitability. Here this study utilizes Gaussian process regression and Bayesian decision theory to organize load data and quantify electrical load uncertainty to properly and effectively discharge industrial battery storage. The study employs a simulation model to set battery load setpoints for the span of the utility billing period according to the degree of risk aversion. This combination of economic analysis according to utility billing period and utilization of degree of risk aversion to make decisions on the uncertainty of the data has not before been applied to battery storage. The method resulted in an annual average reduction of peak demand by 3.8 % at the lowest amount of savings and lowest risk aversion. The highest risk aversion resulted in an annual average reduction of peak demand of 7.5 %. The maximum reduction of peak load in any month was 13.8 % in the month of December with a relatively high risk aversion. With a the highest amount risk aversion tested, the model reduced demand ten of the twelve months of the year.

25 ENERGY STORAGE↗

Single Gaussian process method for arbitrary tokamak regimes with a statistical analysis

Abstract Gaussian process regression is a Bayesian method for inferring profiles based on input data. The technique is increasing in popularity in the fusion community due to its many advantages over traditional fitting techniques including intrinsic uncertainty quantification and robustness to over-fitting. This work investigates the use of a new method, the change-point method, for handling the varying length scales found in different tokamak regimes. The use of the Student’s t-distribution for the Bayesian likelihood probability is also investigated and shown to be advantageous in providing good fits in profiles with many outliers. To compare different methods, synthetic data generated from analytic profiles is used to create a database enabling a quantitative statistical comparison of which methods perform the best. Using a full Bayesian approach with the change-point method, Matérn kernel for the prior probability, and Student’s t-distribution for the likelihood is shown to give the best results.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Special Issue: Geostatistics and Machine Learning

Abstract Recent years have seen a steady growth in the number of papers that apply machine learning methods to problems in the earth sciences. Although they have different origins, machine learning and geostatistics share concepts and methods. For example, the kriging formalism can be cast in the machine learning framework of Gaussian process regression. Machine learning, with its focus on algorithms and ability to seek, identify, and exploit hidden structures in big data sets, is providing new tools for exploration and prediction in the earth sciences. Geostatistics, on the other hand, offers interpretable models of spatial (and spatiotemporal) dependence. This special issue on Geostatistics and Machine Learning aims to investigate applications of machine learning methods as well as hybrid approaches combining machine learning and geostatistics which advance our understanding and predictive ability of spatial processes.

58 GEOSCIENCES↗

Comparison of Machine Learning-Based Predictive Models of the Nutrient Loads Delivered from the Mississippi/Atchafalaya River Basin to the Gulf of Mexico

Predicting nutrient loads is essential to understanding and managing one of the environmental issues faced by the northern Gulf of Mexico hypoxic zone, which poses a severe threat to the Gulf’s healthy ecosystem and economy. The development of hypoxia in the Gulf of Mexico is strongly associated with the eutrophication process initiated by excessive nutrient loads. Due to the complexities in the excessive nutrient loads to the Gulf of Mexico, it is challenging to understand and predict the underlying temporal variation of nutrient loads. The study was aimed at identifying an optimal predictive machine learning model to capture and predict nonlinear behavior of the nutrient loads delivered from the Mississippi/Atchafalaya River Basin (MARB) to the Gulf of Mexico. For this purpose, monthly nutrient loads (N and P) in tons were collected from US Geological Survey (USGS) monitoring station 07373420 from 1980 to 2020. Machine learning models—including autoregressive integrated moving average (ARIMA), gaussian process regression (GPR), single-layer multilayer perceptron (MLP), and a long short-term memory (LSTM) with the single hidden layer—were developed to predict the monthly nutrient loads, and model performances were evaluated by standard assessment metrics—Root Mean Square Error (RMSE) and Correlation Coefficient (R). The residuals of predictive models were examined by the Durbin–Watson statistic. The results showed that MLP and LSTM persistently achieved better accuracy in predicting monthly TN and TP loads compared to GPR and ARIMA. In addition, GPR models achieved slightly better test RMSE score than ARIMA models while their correlation coefficients are much lower than ARIMA models. Moreover, MLP performed slightly better than LSTM in predicting monthly TP loads while LSTM slightly outperformed for TN loads. Furthermore, it was found that the optimizer and number of inputs didn’t show effects on the LSTM performance while they exhibited impacts on MLP outcomes. This study explores the capability of machine learning models to accurately predict nonlinearly fluctuating nutrient loads delivered to the Gulf of Mexico. Further efforts focus on improving the accuracy of forecasting using hybrid models which combine several machine learning models with superior predictive performance for nutrient fluxes throughout the MARB.

54 ENVIRONMENTAL SCIENCES↗

Absorption dissymmetry factor enhancement: A data-driven approach to unravel the synthesis knobs of chiral 2D perovskites

Chiral 2D metal halide perovskites (MHPs) are promising for spin-optoelectronic applications, yet their absorption dissymmetry factor (g abs ) exhibits significant variability due to complex, co-dependent structural and experimental factors. Here, we established a data-driven framework using Pearson’s correlation, ANOVA, and Gaussian process regression to identify and model key synthesis “knobs” governing these properties. The analysis revealed that solvent choice is the primary factor driving variability. For acetonitrile-based films, g abs was maximized by optimizing annealing temperature and film thickness. Conversely, films from higher boiling point solvents showed complex dependencies on annealing temperature, excitonic integral intensity, and film texture. These statistical correlations provide a roadmap for the rational design of high-performance chiral MHPs and establish a foundation for future machine learning-driven material exploration.

ANOVA↗

Uncertainty quantification in machine learning for engineering design and health prognostics: A tutorial

On top of machine learning (ML) models, uncertainty quantification (UQ) functions as an essential layer of safety assurance that could lead to more principled decision making by enabling sound risk assessment and management. The safety and reliability improvement of ML models empowered by UQ has the potential to significantly facilitate the broad adoption of ML solutions in high-stakes decision settings, such as healthcare, manufacturing, and aviation, to name a few. In this tutorial, we aim to provide a holistic lens on emerging UQ methods for ML models with a particular focus on neural networks and the applications of these UQ methods in tackling engineering design as well as prognostics and health management problems. Towards this goal, we start with a comprehensive classification of uncertainty types, sources, and causes pertaining to UQ of ML models. Next, we provide a tutorial-style description of several state-of-the-art UQ methods: Gaussian process regression, Bayesian neural network, neural network ensemble, and deterministic UQ methods focusing on spectral-normalized neural Gaussian process. Established upon the mathematical formulations, we subsequently examine the soundness of these UQ methods quantitatively and qualitatively (by a toy regression example) to examine their strengths and shortcomings from different dimensions. Then, we review quantitative metrics commonly used to assess the quality of predictive uncertainty in classification and regression problems. Afterward, we discuss the increasingly important role of UQ of ML models in solving challenging problems in engineering design and health prognostics. In conclusion, two case studies with source codes available on GitHub are used to demonstrate these UQ methods and compare their performance in the life prediction of lithium-ion batteries at the early stage (case study 1) and the remaining useful life prediction of turbofan engines (case study 2).

97 MATHEMATICS AND COMPUTING↗

Bayesian learning of orthogonal embeddings for multi-fidelity Gaussian Processes

Uncertainty propagation in complex engineering systems often poses significant computational challenges related to modeling and quantifying probability distributions of model outputs, as those emerge as the result of various sources of uncertainty that are inherent in the system under investigation. Gaussian Processes regression (GPs) is a robust meta-modeling technique that allows for fast model prediction and exploration of response surfaces. Multi-fidelity variations of GPs further leverage information from cheap and low fidelity model simulations in order to improve their predictive performance on the high fidelity model. In order to cope with the high volume of data required to train GPs in high dimensional design spaces, a common practice is to introduce latent design variables that are typically projections of the original input space to a lower dimensional subspace, and therefore substitute the problem of learning the initial high dimensional mapping, with that of training a GP on a low dimensional space. Here in this paper, we present a Bayesian approach to identify optimal transformations that map the input points to low dimensional latent variables. The \projection" mapping consists of an orthonormal matrix that is considered a priori unknown and needs to be inferred jointly with the GP parameters, conditioned on the available training data. The proposed Bayesian inference scheme relies on a two-step iterative algorithm that samples from the marginal posteriors of the GP parameters and the projection matrix respectively, both using Markov Chain Monte Carlo (MCMC) sampling. In order to take into account the orthogonality constraints imposed on the orthonormal projection matrix, a Geodesic Monte Carlo sampling algorithm is employed, that is suitable for exploiting probability measures on manifolds. We extend the proposed framework to multi-fidelity models using GPs including the scenarios of training multiple outputs together. We validate our framework on three synthetic problems with a known lower-dimensional subspace. The benefits of our proposed framework, are illustrated on the computationally challenging aerodynamic optimization of a last-stage blade for an industrial gas turbine, where we study the effect of an 85-dimensional shape parameterization of a three-dimensional airfoil on two output quantities of interest, specifically on the aerodynamic efficiency and the degree of reaction

42 ENGINEERING↗

Active- and transfer-learning applied to microscale-macroscale coupling to simulate viscoelastic flows

Active- and transfer-learning are applied to microscale dynamics of polymer flows for the multiscale discovery of effective constitutive approximations required in viscoelastic flow simulation. The result is macroscopic rheology directly connected to a microstructural model. Micro and macroscale simulations are adaptively coupled by means of Gaussian process regression (GPR) to run the expensive microscale computations only as necessary. This multiscale method is demonstrated with flows of a polymer solution as a model system. At the microscale level dissipative particle dynamics (DPD) is employed to model the fluid as a suspension of bead-spring micro-structures subjected to steady shear flow. The results yield the non-Newtonian viscosity and the first normal stress difference at strain rates as training data used in a GPR model. DPD parameters are calibrated with respect to experimental data for a real polymer solution. Compliance with these data requires adjustment of the DPD model's cutoff radius, which then becomes a function of the second invariant of the strain rate tensor. The FENE-P model is chosen for the macroscale description using the spectral element method (SEM) to simulate channel flow and flow past a circular cylinder. The DPD results at the lowest possible shear strain rate yield an estimate of the zero-shear rate viscosity, which allows the initiation of the macroscale flow by SEM as a Newtonian fluid. The resulting strain-rate field is surveyed to determine additional shear strain rate sampling points for the DPD system. This new information allows an initial fitting of parameters of the constitutive equation followed by new SEM simulations at the macroscale. Additionally, guided by active-learning GPR to select new sampling points, this process continues until convergence is achieved. The effectiveness of this new simulation paradigm for viscoelastic flows is tested with different macroscale operating conditions. The effective closure learned in the channel simulation is then transferred directly to the flow past a circular cylinder at low Reynolds number, where the results show that only two additional DPD simulations are required to achieve a satisfactory constitutive model. With an increase of the Reynolds number, the active-learning scheme automatically detects the inaccuracy of the learned constitutive model, and initiates additional DPD simulations for the extra data needed to once again close the microscale-macroscale coupled system. This new paradigm of active- and transfer-learning for multiscale modeling is readily applicable to other microscale-macroscale coupled simulations of complex fluids and other materials. Furthermore, the coupling between microscale and macroscale solvers can be seamlessly implemented with our open source multiscale universal interface (MUI) library.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Uncertainty-aware molecular dynamics from Bayesian active learning for phase transformations and thermal transport in SiC

Abstract Machine learning interatomic force fields are promising for combining high computational efficiency and accuracy in modeling quantum interactions and simulating atomistic dynamics. Active learning methods have been recently developed to train force fields efficiently and automatically. Among them, Bayesian active learning utilizes principled uncertainty quantification to make data acquisition decisions. In this work, we present a general Bayesian active learning workflow, where the force field is constructed from a sparse Gaussian process regression model based on atomic cluster expansion descriptors. To circumvent the high computational cost of the sparse Gaussian process uncertainty calculation, we formulate a high-performance approximate mapping of the uncertainty and demonstrate a speedup of several orders of magnitude. We demonstrate the autonomous active learning workflow by training a Bayesian force field model for silicon carbide (SiC) polymorphs in only a few days of computer time and show that pressure-induced phase transformations are accurately captured. The resulting model exhibits close agreement with both ab initio calculations and experimental measurements, and outperforms existing empirical models on vibrational and thermal properties. The active learning workflow readily generalizes to a wide range of material systems and accelerates their computational understanding.

36 MATERIALS SCIENCE↗

Predicting concrete compressive strength using hybrid ensembling of surrogate machine learning models

This study aims to implement a hybrid ensemble surrogate machine learning technique in predicting the compressive strength (CS) of concrete, an important parameter used for durability design and service life prediction of concrete structures in civil engineering projects. For this purpose, an experimental database consisting of 1030 records has been compiled from the machine learning repository of the University of California, Irvine. The database was used to train and validate four conventional machine learning (CML) models, namely Artificial Neural Network (ANN), Linear and Non-Linear Multivariate Adaptive Regression Splines (MARS-L and MARS-C), Gaussian Process Regression (GPR), and Minimax Probability Machine Regression (MPMR). Subsequently, the predicted outputs of CML models were combined and trained using ANN to construct the Hybrid Ensemble Model (HENSM). It is observed that the proposed HENSM produces higher predictive accuracy compared to the CML models used in the present study. The predictive performance of all models for CS prediction was compared using the testing dataset and it is found that the HENSM model attained the highest predictive accuracy in both phases. Based on the experimental results, the newly constructed HENSM model is very potential to be a new alternative in handling the overfitting issues of CML models and hence, can be used to predict the concrete CS, including the design of less polluting and more sustainable concrete constructions.

36 MATERIALS SCIENCE↗

Data-Driven Machine Learning for Wind Plant Flow Modeling

In this study, we introduce a data-driven machine learning framework for improving the accuracy of wind plant flow models by learning turbulence model corrections based on data from higher-fidelity simulations. First, a high-dimensional PDE-constrained optimization problem is solved using gradient-based optimization with adjoints to determine optimal eddy viscosity fields that improve the agreement of a medium-fidelity Reynolds-Averaged Navier Stokes (RANS) model with large eddy simulations (LES). A supervised learning problem is then constructed to find general, predictive representations of the optimal turbulence closure. A machine learning technique using Gaussian process regression is trained to predict the eddy viscosity field based on local RANS flow field information like velocities, pressures, and their gradients. The Gaussian process is trained on LES simulations of a single turbine and implemented in a wind plant simulation with 36 turbines. We show improvement over the baseline RANS model with the machine learning correction, and demonstrate the ability to provide accurate confidence levels for the corrections that enable future uncertainty quantification studies.

17 WIND ENERGY↗

Characterizing Spatiotemporal Uncertainty in Interpolated Meteorological Data

Interpolated meteorological data invariably contain errors. These errors have structure in time and space, particularly autocorrelation, which can cause the effects of errors to compound when model outputs are aggregated temporally or spatially. One way to account for this uncertainty is with a probabilistic model from which samples can be drawn that are coherent with respect to underlying spatial and temporal covariance structure. This work describes a probabilistic method for spatial interpolation of point-wise meteorological time series. Observational data from weather stations are generally sparse in space and dense in time (but sometimes missing). The method works by projecting time series onto orthogonal basis vectors and spatially interpolating each resulting component independently. Under suitable assumptions, and data transformations to better satisfy those assumptions, Gaussian process regression provides a complete description of the joint predictive distribution over a Gaussian random field. Spatiotemporally coherent realizations are generated as the sum of conditional (spatial) simulations of each orthogonal (temporal) component. Data-derived and generic orthogonal bases are considered. In addition to spatial interpolation, imputation of missing observational data is examined. The method is applied using near-surface air temperature over the Western United States and validated by comparing theoretical versus actual coverage of predictive distributions and analyzing the degree to which spatial and temporal covariance structure is reproduced. Computational considerations, relating to conditional simulation of random fields, are also addressed.

Conor T Doherty↗

A comparison of Gaussian processes and neural networks for computer model emulation and calibration

The Department of Energy relies on complex physics simulations for prediction in domains like cosmology, nuclear theory, and materials science. These simulations are often extremely computationally intensive, with some requiring days or weeks for a single simulation. In order to assure their accuracy, these models are calibrated against observational data in order to estimate inputs and systematic biases. Because of their great computational complexity, this process typically requires the construction of an emulator, a fast approximation to the simulation. In this paper, two emulator approaches are compared: Gaussian process regression and neural networks. Their emulation accuracy and calibration performance on three real problems of Department of Energy interest is considered. On these problems, the Gaussian process emulator tends to be more accurate with narrower, but still well-calibrated uncertainty estimates. The neural network emulator is accurate, but tends to have large uncertainty on its predictions. Finally, as a result, calibration with the Gaussian process emulator produces more constrained posteriors that still perform well in prediction.

97 MATHEMATICS AND COMPUTING↗

Nested polynomial trends for the improvement of Gaussian process-based predictors

The role of simulation keeps increasing for the sensitivity analysis and the uncertainty quantification of complex systems. Such numerical procedures are generally based on the processing of a huge amount of code evaluations. When the computational cost associated with one particular evaluation of the code is high, such direct approaches based on the computer code only, are not affordable. Surrogate models have therefore to be introduced to interpolate the information given by a fixed set of code evaluations to the whole input space. When confronted to deterministic mappings, the Gaussian process regression (GPR), or kriging, presents a good compromise between complexity, efficiency and error control. Such a method considers the quantity of interest of the system as a particular realization of a Gaussian stochastic process, whose mean and covariance functions have to be identified from the available code evaluations. In this context, this work proposes an innovative parametrization of this mean function, which is based on the composition of two polynomials. This approach is particularly relevant for the approximation of strongly non linear quantities of interest from very little information. After presenting the theoretical basis of this method, this work compares its efficiency to alternative approaches on a series of examples.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Hot planets around cool stars – two short-period mini-Neptunes transiting the late K-dwarf TOI-1260

We present the discovery and characterization of two sub-Neptunes in close orbits, as well as a tentative outer planet of a similar size, orbiting TOI-1260 – a low metallicity K6 V dwarf star. Photometry from Transiting Exoplanet Survey Satellite(TESS) yields radii of R(b) = 2.33 ± 0.10 and R(c) = 2.82 ± 0.15 Rꚛ, and periods of 3.13 and 7.49 d for TOI-1260 b and TOI-1260 c, respectively. We combined the TESS data with a series of ground-based follow-up observations to characterize the planetary system. From HARPS-N high-precision radial velocities we obtain M(b) = 8.6(+1.4,−1.5) and M(c) = 11.8(+3.4,−3.2) Mꚛ. The star is moderately active with a complex activity pattern, which necessitated the use of Gaussian process regression for both the light-curve detrending and the radial velocity modelling, in the latter case guided by suitable activity indicators. We successfully disentangle the stellar-induced signal from the planetary signals, underlining the importance and usefulness of the Gaussian process approach. We test the system’s stability against atmospheric photoevaporation and find that the TOI-1260 planets are classic examples of the structure and composition ambiguity typical for the 2–3 Rꚛ range.

I Y Georgieva↗

Molecular dipole moment learning via rotationally equivariant derivative kernels in molecular-orbital-based machine learning

This study extends the accurate and transferable molecular-orbital-based machine learning (MOB-ML) approach to modeling the contribution of electron correlation to dipole moments at the cost of Hartree–Fock computations. A MOB pairwise decomposition of the correlation part of the dipole moment is applied, and these pair dipole moments could be further regressed as a universal function of MOs. The dipole MOB features consist of the energy MOB features and their responses to electric fields. An interpretable and rotationally equivariant derivative kernel for Gaussian process regression (GPR) is introduced to learn the dipole moment more efficiently. The proposed problem setup, feature design, and ML algorithm are shown to provide highly accurate models for both dipole moments and energies on water and 14 small molecules. To demonstrate the ability of MOB-ML to function as generalized density-matrix functionals for molecular dipole moments and energies of organic molecules, we further apply the proposed MOB-ML approach to train and test the molecules from the QM9 dataset. The application of local scalable GPR with Gaussian mixture model unsupervised clustering GPR scales up MOB-ML to a large-data regime while retaining the prediction accuracy. In addition, compared with the literature results, MOB-ML provides the best test mean absolute errors of 4.21 mD and 0.045 kcal/mol for dipole moment and energy models, respectively, when training on 110 000 QM9 molecules. The excellent transferability of the resulting QM9 models is also illustrated by the accurate predictions for four different series of peptides.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗