Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “gaussian regression”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 127 records · Page 7
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

Uncertainty-aware molecular dynamics from Bayesian active learning for phase transformations and thermal transport in SiC

Abstract Machine learning interatomic force fields are promising for combining high computational efficiency and accuracy in modeling quantum interactions and simulating atomistic dynamics. Active learning methods have been recently developed to train force fields efficiently and automatically. Among them, Bayesian active learning utilizes principled uncertainty quantification to make data acquisition decisions. In this work, we present a general Bayesian active learning workflow, where the force field is constructed from a sparse Gaussian process regression model based on atomic cluster expansion descriptors. To circumvent the high computational cost of the sparse Gaussian process uncertainty calculation, we formulate a high-performance approximate mapping of the uncertainty and demonstrate a speedup of several orders of magnitude. We demonstrate the autonomous active learning workflow by training a Bayesian force field model for silicon carbide (SiC) polymorphs in only a few days of computer time and show that pressure-induced phase transformations are accurately captured. The resulting model exhibits close agreement with both ab initio calculations and experimental measurements, and outperforms existing empirical models on vibrational and thermal properties. The active learning workflow readily generalizes to a wide range of material systems and accelerates their computational understanding.

36 MATERIALS SCIENCE↗
DOE OSTI2021

A comparison of Gaussian processes and neural networks for computer model emulation and calibration

The Department of Energy relies on complex physics simulations for prediction in domains like cosmology, nuclear theory, and materials science. These simulations are often extremely computationally intensive, with some requiring days or weeks for a single simulation. In order to assure their accuracy, these models are calibrated against observational data in order to estimate inputs and systematic biases. Because of their great computational complexity, this process typically requires the construction of an emulator, a fast approximation to the simulation. In this paper, two emulator approaches are compared: Gaussian process regression and neural networks. Their emulation accuracy and calibration performance on three real problems of Department of Energy interest is considered. On these problems, the Gaussian process emulator tends to be more accurate with narrower, but still well-calibrated uncertainty estimates. The neural network emulator is accurate, but tends to have large uncertainty on its predictions. Finally, as a result, calibration with the Gaussian process emulator produces more constrained posteriors that still perform well in prediction.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Molecular dipole moment learning via rotationally equivariant derivative kernels in molecular-orbital-based machine learning

This study extends the accurate and transferable molecular-orbital-based machine learning (MOB-ML) approach to modeling the contribution of electron correlation to dipole moments at the cost of Hartree–Fock computations. A MOB pairwise decomposition of the correlation part of the dipole moment is applied, and these pair dipole moments could be further regressed as a universal function of MOs. The dipole MOB features consist of the energy MOB features and their responses to electric fields. An interpretable and rotationally equivariant derivative kernel for Gaussian process regression (GPR) is introduced to learn the dipole moment more efficiently. The proposed problem setup, feature design, and ML algorithm are shown to provide highly accurate models for both dipole moments and energies on water and 14 small molecules. To demonstrate the ability of MOB-ML to function as generalized density-matrix functionals for molecular dipole moments and energies of organic molecules, we further apply the proposed MOB-ML approach to train and test the molecules from the QM9 dataset. The application of local scalable GPR with Gaussian mixture model unsupervised clustering GPR scales up MOB-ML to a large-data regime while retaining the prediction accuracy. In addition, compared with the literature results, MOB-ML provides the best test mean absolute errors of 4.21 mD and 0.045 kcal/mol for dipole moment and energy models, respectively, when training on 110 000 QM9 molecules. The excellent transferability of the resulting QM9 models is also illustrated by the accurate predictions for four different series of peptides.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
NASA NTRS2021

Hot planets around cool stars – two short-period mini-Neptunes transiting the late K-dwarf TOI-1260

We present the discovery and characterization of two sub-Neptunes in close orbits, as well as a tentative outer planet of a similar size, orbiting TOI-1260 – a low metallicity K6 V dwarf star. Photometry from Transiting Exoplanet Survey Satellite(TESS) yields radii of R(b) = 2.33 ± 0.10 and R(c) = 2.82 ± 0.15 Rꚛ, and periods of 3.13 and 7.49 d for TOI-1260 b and TOI-1260 c, respectively. We combined the TESS data with a series of ground-based follow-up observations to characterize the planetary system. From HARPS-N high-precision radial velocities we obtain M(b) = 8.6(+1.4,−1.5) and M(c) = 11.8(+3.4,−3.2) Mꚛ. The star is moderately active with a complex activity pattern, which necessitated the use of Gaussian process regression for both the light-curve detrending and the radial velocity modelling, in the latter case guided by suitable activity indicators. We successfully disentangle the stellar-induced signal from the planetary signals, underlining the importance and usefulness of the Gaussian process approach. We test the system’s stability against atmospheric photoevaporation and find that the TOI-1260 planets are classic examples of the structure and composition ambiguity typical for the 2–3 Rꚛ range.

I Y Georgieva↗
DOE OSTI2023

Sparse regression for plasma physics

Many scientific problems can be formulated as sparse regression, i.e., regression onto a set of parameters when there is a desire or expectation that some of the parameters are exactly zero or do not substantially contribute. This includes many problems in signal and image processing, system identification, optimization, and parameter estimation methods such as Gaussian process regression. Sparsity facilitates exploring high-dimensional spaces while finding parsimonious and interpretable solutions. In the present work, we illustrate some of the important ways in which sparse regression appears in plasma physics and point out recent contributions and remaining challenges to solving these problems in this field. Further, a brief review is provided for the optimization problem and the state-of-the-art solvers, especially for constrained and high-dimensional sparse regression.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2024

Physics-Based Machine Learning Methods for U-235 Forensics Signatures

Signatures of low-intensity U-235 sources have been recently studied by utilizing a variety of machine learning (ML) classifiers using features derived from gamma spectral measurements collectedunder structured campaigns. Several ML classifiers, such as ensemble of tress and classification trees, revealed misleadingly-optimistic training error due to over-fitting, and furthermore,their performance is not directly relatable to the physical properties due to their data-driven, opaque designs. We present a regression-based ML method that first estimates the inverse distanceto the source and then utilizes a threshold to infer its presence, by representing the background as a source located at an infinite distance. For the inverse distance estimation, we study the ensembleof trees and Gaussian process regression methods, and a hyper parameter auto-tuning and selection method that employs five regression estimators. These methods avoid the over-fittingobserved in several ML classifiers, while providing the classification error nearly comparable to them based on independent test data. Their error is directly related to estimates of the inversephysical distance to source, and the precision of error determines the seperability property that determines the false alarm and missed detection rates. The property of monotonic decrease of thesource strength with increasing detector distance combined with Poisson distribution of measurements is utilized to analytically validate these methods by deriving the generalization equations ofunderlying regression methods.

Rao, Nageswara↗
DOE OSTI2024

Physics-Based Machine Learning Methods for U-235 Forensics Signatures

Signatures of low-intensity U-235 sources have been recently studied by utilizing a variety of machine learning (ML) classifiers using features derived from gamma spectral measurements collected under structured campaigns. Several ML classifiers, such as ensemble of tress and classification trees, revealed misleadingly-optimistic training error due to over-fitting, and furthermore, their performance is not directly relatable to the physical properties due to their data-driven, opaque designs. We present a regression-based ML method that first estimates the inverse distance to the source and then utilizes a threshold to infer its presence, by representing the background as a source located at an infinite distance. For the inverse distance estimation, we study the ensemble of trees and Gaussian process regression methods, and a hyper parameter auto-tuning and selection method that employs five regression estimators. These methods avoid the over-fitting observed in several ML classifiers, while providing the classification error nearly comparable to them based on independent test data. Their error is directly related to estimates of the inverse physical distance to source, and the precision of error determines the seperability property that determines the false alarm and missed detection rates. The property of monotonic decrease of the source strength with increasing detector distance combined with Poisson distribution of measurements is utilized to analytically validate these methods by deriving the generalization equations of underlying regression methods.

Rao, Nageswara↗
NASA NTRS2012

Can Selforganizing Maps Accurately Predict Photometric Redshifts?

We present an unsupervised machine-learning approach that can be employed for estimating photometric redshifts. The proposed method is based on a vector quantization called the self-organizing-map (SOM) approach. A variety of photometrically derived input values were utilized from the Sloan Digital Sky Survey's main galaxy sample, luminous red galaxy, and quasar samples, along with the PHAT0 data set from the Photo-z Accuracy Testing project. Regression results obtained with this new approach were evaluated in terms of root-mean-square error (RMSE) to estimate the accuracy of the photometric redshift estimates. The results demonstrate competitive RMSE and outlier percentages when compared with several other popular approaches, such as artificial neural networks and Gaussian process regression. SOM RMSE results (using delta(z) = z(sub phot) - z(sub spec)) are 0.023 for the main galaxy sample, 0.027 for the luminous red galaxy sample, 0.418 for quasars, and 0.022 for PHAT0 synthetic data. The results demonstrate that there are nonunique solutions for estimating SOM RMSEs. Further research is needed in order to find more robust estimation techniques using SOMs, but the results herein are a positive indication of their capabilities when compared with other well-known methods

Way, Michael J.↗
DOE OSTI2021

Gaussian processes for autonomous data acquisition at large-scale synchrotron and neutron facilities

The execution and analysis of complex experiments are challenged by the vast dimensionality of the underlying parameter spaces. Although an increase in data-acquisition rates should allow broader querying of the parameter space, the complexity of experiments and the subtle dependence of the model function on input parameters remains daunting owing to the sheer number of variables. New strategies for autonomous data acquisition are being developed, with one promising direction being the use of Gaussian process regression (GPR). GPR is a quick, non-parametric and robust approximation and uncertainty quantification method that can be applied directly to autonomous data acquisition. We review GPR-driven autonomous experimentation and illustrate its functionality using real-world examples from large experimental facilities in the USA and France. We introduce the basics of a GPR-driven autonomous loop with a focus on Gaussian processes, and then shift the focus to the infrastructure that needs to be built around GPR to create a closed loop. Finally, the case studies we discuss show that Gaussian-process-based autonomous data acquisition is a widely applicable method that can facilitate the optimal use of instruments and facilities by enabling the efficient acquisition of high-value datasets.

36 MATERIALS SCIENCE↗
DOE OSTI2021

A machine learning approach for efficient multi-dimensional integration

Many physics problems involve integration in multi-dimensional space whose analytic solution is not available. The integrals can be evaluated using numerical integration methods, but it requires a large computational cost in some cases, so an efficient algorithm plays an important role in solving the physics problems. We propose a novel numerical multi-dimensional integration algorithm using machine learning (ML). After training a ML regression model to mimic a target integrand, the regression model is used to evaluate an approximation of the integral. Then, the difference between the approximation and the true answer is calculated to correct the bias in the approximation of the integral induced by ML prediction errors. Because of the bias correction, the final estimate of the integral is unbiased and has a statistically correct error estimation. Three ML models of multi-layer perceptron, gradient boosting decision tree, and Gaussian process regression algorithms are investigated. The performance of the proposed algorithm is demonstrated on six different families of integrands that typically appear in physics problems at various dimensions and integrand difficulties. The results show that, for the same total number of integrand evaluations, the new algorithm provides integral estimates with more than an order of magnitude smaller uncertainties than those of the VEGAS algorithm in most of the test cases.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2023

Conditional Karhunen–Loève regression model with Basis Adaptation for high-dimensional problems: Uncertainty quantification and inverse modeling

Here, we propose a methodology for improving the accuracy of surrogate models of the observable response of physical systems as a function of the systems’ spatially heterogeneous parameter fields, with applications to uncertainty quantification and parameter estimation in high-dimensional problems. Practitioners often formulate finite-dimensional representations of spatially heterogeneous parameter fields using truncated unconditional Karhunen–Loève expansions (KLEs) for a certain choice of unconditional covariance kernel and construct surrogate models of the observable response with respect to the KLE coefficients. When direct measurements of the parameter fields are available, we propose improving the accuracy of these surrogate models by representing the parameter fields via conditional Karhunen-Loève expansions (CKLEs). CKLEs are constructed by conditioning the covariance kernel of the unconditional expansion on the direct measurements of the parameter field via Gaussian process regression, and then truncating the corresponding KLE. We apply the proposed methodology to constructing surrogate models via the Basis Adaptation (BA) method of the stationary hydraulic head response, measured at spatially discrete observation locations, of a groundwater flow model of the Hanford Site, as a function of the 1000-dimensional representation of the model’s log-transmissivity field. We find that BA surrogate models of the hydraulic head based on CKLEs are more accurate than BA surrogate models based on unconditional expansions for forward uncertainty quantification tasks. Furthermore, we find that inverse estimates of the hydraulic transmissivity field computed using CKLE-based BA surrogate models are more accurate than those computed using unconditional BA surrogate models.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Uncertainty quantification in multivariable regression for material property prediction with Bayesian neural networks

With the increased use of data-driven approaches and machine learning-based methods in material science, the importance of reliable uncertainty quantification (UQ) of the predicted variables for informed decision-making cannot be overstated. UQ in material property prediction poses unique challenges, including multi-scale and multi-physics nature of materials, intricate interactions between numerous factors, limited availability of large curated datasets, etc. In this work, we introduce a physics-informed Bayesian Neural Networks (BNNs) approach for UQ, which integrates knowledge from governing laws in materials to guide the models toward physically consistent predictions. To evaluate the approach, we present case studies for predicting the creep rupture life of steel alloys. Experimental validation with three datasets of creep tests demonstrates that this method produces point predictions and uncertainty estimations that are competitive or exceed the performance of conventional UQ methods such as Gaussian Process Regression. Additionally, we evaluate the suitability of employing UQ in an active learning scenario and report competitive performance. The most promising framework for creep life prediction is BNNs based on Markov Chain Monte Carlo approximation of the posterior distribution of network parameters, as it provided more reliable results in comparison to BNNs based on variational inference approximation or related NNs with probabilistic outputs.

36 MATERIALS SCIENCE↗
DOE OSTI2020

Simultaneous Optical Transmission Spectroscopy of a Terrestrial, Habitable-zone Exoplanet with Two Ground-based Multiobject Spectrographs

Investigating the atmospheres of rocky exoplanets is key to performing comparative planetology between these worlds and the terrestrial planets that reside in the inner solar system. Terrestrial exoplanet atmospheres exhibit weak signals, and attempting to detect them pushes at the boundaries of what is possible for current instrumentation. We focus on the habitable-zone terrestrial exoplanet LHS 1140b. Given its 25-day orbital period and 2 hr transit duration, capturing transits of LHS 1140b is challenging. We observed two transits of this object, approximately 1 yr apart, which yielded four data sets thanks to our simultaneous use of the IMACS and LDSS3C multiobject spectrographs mounted on the twin Magellan telescopes at Las Campanas Observatory. We present a jointly fit white light curve, as well as jointly fit 20 nm wavelength-binned light curves from which we construct a transmission spectrum. Binning the joint white light-curve residuals to 3-minute time bins gives an rms of 145 ppm; binning down to 10-minute time bins gives an rms of 77 ppm. Our median uncertainty in R{sub p}{sup 2}/R{sub s}{sup 2} in the 20 nm wavelength bins is 260 ppm, and we achieve an average precision of 1.3× the photon noise when fitting the wavelength-binned light curves with a Gaussian process regression. Our precision on R{sub p}{sup 2}/R{sub s}{sup 2} is a factor of four larger than the feature amplitudes of a clear, hydrogen-dominated atmosphere, meaning that we are not able to test realistic models of LHS 1140b’s atmosphere. The techniques and caveats presented here are applicable to the growing sample of terrestrial worlds in the Transiting Exoplanet Survey Satellite era, as well as to the upcoming generation of ground-based giant segmented mirror telescopes.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2020

Machine learning with bond information for local structure optimizations in surface science

Local optimization of adsorption systems inherently involves different scales: within the substrate, within the molecule, and between the molecule and the substrate. In this work, we show how the explicit modeling of different characteristics of the bonds in these systems improves the performance of machine learning methods for optimization. Furthermore, we introduce an anisotropic kernel in the Gaussian process regression framework that guides the search for the local minimum, and we show its overall good performance across different types of atomic systems. The method shows a speed-up of up to a factor of two compared with the fastest standard optimization methods on adsorption systems. Additionally, we show that a limited memory approach is not only beneficial in terms of overall computational resources but can also result in a further reduction of energy and force calculations.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
NASA NTRS2025

Gaussian Process for Flight Delay Prediction: Learning a Stochastic Process

This paper presents a machine-learning approach to predict flight delays. Whereas neural networks are extensively studied for predictive capabilities, they involve non-intuitive design and extensive analysis, particularly in training and optimization processes. Instead, the proposed framework employs Gaussian Processes as a supervised learning technique for flight delay prediction. This data-driven approach trains the model using prior information, specifically the mean and covariance tied to existing data. The proposed Gaussian Process Regression (GPR) model employs the day of flight as a pivotal feature for delay forecasting. We analyze flights from various routes and gauge the accuracy of the presented learning technique by comparing the predicted delays with the actual ones. Given the inherent challenges in precisely forecasting delays, we predict the delays with a 95 % confidence interval. Also, an error propagation analysis in the prediction horizon is carried out to determine the optimal time frame for prediction. The proposed method for flight delay prediction is important as airlines can strategize flight operations and issue timely advisories.

stochastic↗
DOE OSTI2024

Thinking Bayesian for plasma physicists

Bayesian statistics offers a powerful technique for plasma physicists to infer knowledge from the heterogeneous data types encountered. To explain this power, a simple example, Gaussian Process Regression, and the application of Bayesian statistics to inverse problems are explained. The likelihood is the key distribution because it contains the data model, or theoretic predictions, of the desired quantities. By using prior knowledge, the distribution of the inferred quantities of interest based on the data given can be inferred. Because it is a distribution of inferred quantities given the data and not a single prediction, uncertainty quantification is a natural consequence of Bayesian statistics. The benefits of machine learning in developing surrogate models for solving inverse problems are discussed, as well as progress in quantitatively understanding the errors that such a model introduces.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2022

Time-series forecasting using manifold learning, radial basis function interpolation, and geometric harmonics

We address a three-tier numerical framework based on nonlinear manifold learning for the forecasting of high-dimensional time series, relaxing the “curse of dimensionality” related to the training phase of surrogate/machine learning models. At the first step, we embed the high-dimensional time series into a reduced low-dimensional space using nonlinear manifold learning (local linear embedding and parsimonious diffusion maps). Then, we construct reduced-order surrogate models on the manifold (here, for our illustrations, we used multivariate autoregressive and Gaussian process regression models) to forecast the embedded dynamics. Finally, we solve the pre-image problem, thus lifting the embedded time series back to the original high-dimensional space using radial basis function interpolation and geometric harmonics. The proposed numerical data-driven scheme can also be applied as a reduced-order model procedure for the numerical solution/propagation of the (transient) dynamics of partial differential equations (PDEs). In conclusion, we assess the performance of the proposed scheme via three different families of problems: (a) the forecasting of synthetic time series generated by three simplistic linear and weakly nonlinear stochastic models resembling electroencephalography signals, (b) the prediction/propagation of the solution profiles of a linear parabolic PDE and the Brusselator model (a set of two nonlinear parabolic PDEs), and (c) the forecasting of a real-world data set containing daily time series of ten key foreign exchange rates spanning the time period 3 September 2001–29 October 2020.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2026

Taming nuclear mass models with Gaussian processes

We propose a new set of nuclear mass predictions based on multiple theoretical mass models. By employing Gaussian process regression with the Matérn kernel, we achieved root-mean-square (rms) deviations below 100 keV for the training dataset. The best-performing mass models achieved rms deviations below 150 keV for the new precise mass data from AME2020, whereas the ensemble average showed robust performance across the nuclear chart. Our approach uniquely combines: (1) systematic refinement of eight mass models through their residuals, (2) physics-informed features, including magic numbers, nucleon parity numbers, neutron excess, and nuclear collectivity, and (3) theory-to-theory validation demonstrating robust extrapolation capability. We find that the Matérn kernel provides superior uncertainty quantification compared to the RBF kernel, with a length-scale analysis revealing enhanced inter-nuclei correlations. We provide complete mass predictions for all unknown nuclides in AME2020, offering valuable constraints for nuclear structure studies and astrophysical modeling when used with proper uncertainty propagation.

Gaussian processes↗