SEARCH · Engineering Papers
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
The code enables to perform Bayesian inference in an efficient manner through the use of Hamiltonian Neural Networks (HNNs), Deep Neural Networks (DNNs), Neural ODEs, and Symplectic Neural Networks (SympNets) used with state-of-the-art sampling schemes like Hamiltonian Monte Carlo (HMC) and the No-U-Turn-Sampler (NUTS).
Knowledge of the amount of water in the soil is of great importance to many earth science disciplines. Soil moisture is a key variable in controlling the exchange of water and energy between the land surface and the atmosphere. Thus, soil moisture information is valuable in a wide range of applications including weather and climate, runoff potential and flood control, early warning of droughts, irrigation, crop yield forecasting, soil erosion, reservoir management, geotechnical engineering, and water quality. Despite the importance of soil moisture information, widespread and continuous measurements of soil moisture are not possible today. Although many earth surface conditions can be measured from satellites, we still cannot adequately measure soil moisture from space. Research in soil moisture remote sensing began in the mid 1970s shortly after the surge in satellite development. Recent advances in remote sensing have shown that soil moisture can be measured, at least qualitatively, by several methods. Quantitative measurements of moisture in the soil surface layer have been most successful using both passive and active microwave remote sensing, although complications arise from surface roughness and vegetation type and density. Early attempts to measure soil moisture from space-borne microwave instruments were hindered by what is now considered sub-optimal wavelengths (shorter than 5 cm) and the coarse spatial resolution of the measurements. L-band frequencies between 1 and 3 GHz (10-30 cm) have been deemed optimal for detection of soil moisture in the upper few centimeters of soil. The Electronically Steered Thinned Array Radiometer (ESTAR), an aircraft-based instrument operating a 1,4 GHz, has shown great promise for soil moisture determination. Initiatives are underway to develop a similar instrument for space. Existing space-borne synthetic aperture radars (SARS) operating at C- and L-band have also shown some potential to detect surface wetness. The advantage of radar is its much higher resolution than passive microwave systems, but it is currently hampered by surface roughness effects and the lack of a good algorithm based on a single frequency and single polarization. In addition, its repeat frequency is generally low (about 40 days). In the meantime, two new radiometers offer some hope for remote sensing of soil moisture from space. The Tropical Rainfall Measuring Mission (TRMM) Microwave Imager (TMI), launched in November 1997, possesses a 10.65 GHz channel and the Advanced Microwave Scanning Radiometer (AMSR) on both the ADEOS-11 and Earth Observing System AM-1 platforms to be launched in 1999 possesses a 6.9 GHz channel. Aside from issues about interference from vegetation, the coarse resolution of these data will provide considerable challenges pertaining to their application. The resolution of TMI is about 45 km and that of AMSR is about 70 km. These resolutions are grossly inconsistent with the scale of soil moisture processes and the spatial variability of factors that control soil moisture. Scale disparities such as these are forcing us to rethink how we assimilate data of various scales in hydrologic models. Of particular interest is how to assimilate soil moisture data by reconciling the scale disparity between what we can expect from present and future remote sensing measurements of soil moisture and modeling soil moisture processes. It is because of this disparity between the resolution of space-based sensors and the scale of data needed for capturing the spatial variability of soil moisture and related properties that remote sensing of soil moisture has not met with more widespread success. Within a single footprint of current sensors at the wavelengths optimal for this application, in most cases there is enormous heterogeneity in soil moisture created by differences in landcover, soils and topography, as well as variability in antecedent precipitation. It is difficult to interpret the meaning of 'mean' soil moisture under such conditions and even more difficult to apply such a value. Because of the non-linear relationships between near-surface soil moisture and other variables of interest, such as surface energy fluxes and runoff, mean soil moisture has little applicability at such large scales. It is for these reasons that the use of remote sensing in conjunction with a hydrologic model appears to be of benefit in capturing the complete spatial and temporal structure of soil moisture. This paper is Part I of a four-part series describing a method for intermittently assimilating remotely-sensed soil moisture information to improve performance of a distributed land surface hydrology model. The method, summarized in section II, involves the following components, each of which is detailed in the indicated section of the paper or subsequent papers in this series: Forward radiative transfer model methods (section II and Part IV); Use of a Kalman filter to assimilate remotely-sensed soil moisture estimates with the model profile (section II and Part IV); Application of a soil hydrology model to capture the continuous evolution of the soil moisture profile within and below the root zone (section III); Statistical aggregation techniques (section IV and Part II); Disaggregation techniques using a neural network approach (section IV and Part III); and Maximum likelihood and Bayesian algorithms for inversely solving for the soil moisture profile in the upper few cm (Part IV).
Here, the Deep Operator Network (DeepONet) is a neural network architecture used to approximate operators, including the solution operator of parametric PDEs. DeepONets have shown remarkable approximation ability. However, the performance of DeepONets deteriorates when the training data is polluted with noise, a scenario that occurs in practice. To handle noisy data, we propose a Bayesian DeepONet based on replica exchange Langevin diffusion (reLD). Replica exchange uses two particles. The first particle trains a DeepONet to exploit the loss landscape and make predictions. The other particle trains a different DeepONet to explore the loss landscape and escape local minima via swapping. Compared to DeepONets trained with state-of-the-art gradient-based algorithms (e.g., Adam), the proposed Bayesian DeepONet greatly improves the training convergence for noisy scenarios and accurately estimates the uncertainty. To further reduce the high computational cost of the reLD training of DeepONets, we propose (1) an accelerated training framework that exploits the DeepONet's architecture to reduce its computational cost up to 25% without compromising performance and (2) a transfer learning strategy that accelerates training DeepONets for PDEs with different parameter values. Finally, we illustrate the effectiveness of the proposed Bayesian DeepONet using four parametric PDE problems.
Accurate prediction of injection profiles is a critical aspect of linking injector operation with engine performance and emissions. However, highly resolved injector simulations can take one to two weeks of wall-clock time, which is incompatible with engine design cycles with desired turnaround times of less than a day. Hence, it is important to reduce the time-to-solution of the internal flow simulations by several orders of magnitude to make it compatible with engine simulations. This work demonstrates a data-driven approach for tackling the computational overhead of injector simulations, whereby the transient injection profiles are emulated for a side-oriented, single-hole diesel injector using a Bayesian machine-learning framework. First, an interpretable Bayesian learning strategy was employed to understand the effect of design parameters on the total void fraction field. Then, autoencoders are utilized for efficient dimensionality reduction of the flowfields. Gaussian process models are finally used to predict the spatiotemporal void fraction field at the injector exit for unknown operating conditions. The Gaussian process models produce principled uncertainty estimates associated with the emulated flowfields, which provide the engine designer with valuable information of where the data-driven predictions can be trusted in the design space. The Bayesian flowfield predictions are compared with the corresponding predictions from a deep neural network, which has been transfer-learned from static needle simulations from a previous work by the authors. The emulation framework can predict the void fraction field at the exit of the orifice within a few seconds, thus achieving a speed-up factor of up to 38 x 10(6) over the traditional simulation-based approach of generating transient injection maps.
With increasing simulation and measurement data, machine learning and artificial intelligence have been widely used in computational decision-making of complex engineering systems. The resulting tools, such as uncertainty quantification solvers, reinforcement learning, and physics-informed machine learning, have achieved great success in critical DOE tasks such as material discovery and design, energy system modeling and control, and numerical weather and climate prediction. A core topic in scientific machine learning and artificial intelligence is Bayesian inference: given an observed data set, people want to estimate the posterior distribution of a (possibly large) number of hidden parameters. Due to the flexibility and weak assumptions, Bayesian sampling has been the mainstream Bayesian inference solvers despite the rapid progress of approximate Bayesian inference. Classical Bayesian sampling methods such as Markov-chain Monte Carlo suffer from a low-acceptance rate due to the random walk nature, therefore state-of-the-art techniques use Hamiltonian Monte Carlo and its variants to efficiently draw posterior samples in a high dimension. The key idea of Hamiltonian Monte Carlo and its variants is to simulate the Hamiltonian dynamics of a classical particle with a fixed mass, and their performance significantly degrades when the posterior distribution is highly spiky or has multiple modes. Leveraging the idea of quantum physics, this project has investigated new theory, algorithms and applications of Bayesian inference (especially Bayesian sampling). The main results include: (1) novel quantum-inspired Bayesian sampling methods that can lead to better accuracy for challenging multi-modal or spiky distributions, (2) more scalable machine learning framework leveraging tensor-compressed Bayesian inference, and (3) Bayesian and sampling approaches for verifying the robustness of continuous and binary neural networks.
In numerous applications, the integration of prior knowledge and historical information is essential, particularly for tasks requiring the solution of ordinary or partial differential equations (ODEs/PDEs) in data-sparse or noisy environments. For instance, achieving accurate solutions to time-dependent PDEs with limited initial condition measurements necessitates an effective strategy for embedding prior knowledge. Hard-parameter sharing architectures in neural networks (NNs) have demonstrated success in both traditional and scientific machine learning domains, facilitating the learning of informative representations. Here, in this study, we introduce a novel, yet efficient, method to enhance physics-informed neural networks (PINNs) by incorporating a multi-head structure that enables the learning of functional priors from both empirical data and governing physical laws. This prior information can then be used to address data sparsity and high-level noise in solving ODE/PDE problems with uncertainty quantification (UQ). The approach, termed Multi-Head PINN (MH-PINN), consists of a shared body NN and multiple head NNs, each corresponding to an individual PINN instance. Our framework for functional prior learning is carried out in two stages: (1) training the MH-PINNs to develop a shared body NN alongside multiple head NNs, and (2) employing these trained head NNs to estimate a prior distribution through a normalizing flow-based density estimator. The learned functional prior can then be applied as a regularization mechanism in deterministic contexts or as an informative prior within a Bayesian inference framework, aiding in the resolution of subsequent ODE/PDE tasks. We evaluate the efficacy of MH-PINNs across five benchmark problems, including a high-dimensional parametric PDE, all characterized by data sparsity or substantial noise levels. Our findings reveal that MH-PINNs deliver accurate solutions and robust UQ, demonstrating adaptability across a range of complex and challenging scenarios.
We propose a reduced-order deep-learning surrogate model for dynamic systems described by time-dependent partial differential equations. This method employs space–time Karhunen–Loève expansions (KLEs) of the state variables and space-dependent KLEs of space-varying parameters to identify the reduced (latent) dimensions. Subsequently, a deep neural network (DNN) is used to map the parameter latent space to the state variable latent space. An approximate Bayesian method is developed for uncertainty quantification (UQ) in the proposed KL-DNN surrogate model. The KL-DNN method is tested for the linear advection–diffusion and nonlinear diffusion equations, and the Bayesian approach for UQ is compared with the deep ensembling (DE) approach, commonly used for quantifying uncertainty in DNN models. It was found that the approximate Bayesian method provides a more informative distribution of the PDE solutions in terms of the coverage of the reference PDE solutions (the percentage of nodes where the reference solution is within the confidence interval predicted by the UQ methods) and log predictive probability. The DE method is found to underestimate uncertainty and introduce bias. For the nonlinear diffusion equation, we compare the KL-DNN method with the Fourier Neural Operator (FNO) method and find that KL-DNN is 10% more accurate and needs less training time than the FNO method.
Abstract Connectivity and automated driving technologies have opened up new research directions in the energy management of vehicles which exploit look-ahead preview and enhance the situational awareness. Despite this advancement, the vehicle speed preview that can be obtained from vehicle-to-vehicle/infrastructure (V2V/I) communications is often limited to a relatively short time-horizon. The vehicular energy systems, specifically those of the electrified vehicles, consist of multiple interacting power and thermal subsystems that respond over different time-scales. Consequently, their optimal energy management can greatly benefit from long-term speed prediction beyond that available through V2V/I communications. Accurately extending the look-ahead preview, on the other hand, is fundamentally challenging due to the dynamic nature of the traffic environment. To address this challenge, we propose a data-driven multirange vehicle speed prediction strategy for arterial corridors with signalized intersections, providing the vehicle speed preview for three different ranges, i.e., short-, medium-, and long-range. The short-range preview is obtained by V2V/I communications. The medium-range preview is realized using a neural network (NN), while the long-range preview is predicted based on a Bayesian network (BN). The predictions are updated in real-time based on the current state of traffic and incorporated into a multihorizon model predictive control (MH-MPC) for integrated power and thermal management (iPTM) of connected vehicles. The results of design and evaluation of the performance of the proposed data-informed MH-MPC for iPTM of connected hybrid electric vehicles (HEVs) using traffic data for real-world city driving are reported.
This paper introduces an advanced Short-term Nodal Load Forecasting (STNLF) method that forecasts nodal load profiles for the next day in power systems, based on the combined use of three machine learning techniques. Least Absolute Shrinkage and Selection Operator (LASSO) is employed to reduce the number of features for a single nodal load forecasting. Principal Component Analysis (PCA) is used to capture the features of historical loads in low-dimensional space compared to the original high-dimensional load space where features are barely possible to depict. Additionally, Bayesian Ridge Regression (BRR) is utilized to decide the parameters of the prediction model from a statistics perspective. Tests based on modified PJM load data demonstrate the effectiveness of the proposed STNLF method compared to the state-of-the-art General Regression Neural Network (GRNN) method. Moreover, the reliability of the day-ahead Unit Commitment (UC) solution is shown to have been improved, based on the forecasted load data using the proposed STNLF method.
We propose a novel deep learning framework, named SYMHnet, which employs a graph neural network and a bidirectional long short-term memory network to cooperatively learn patterns from solar wind and interplanetary magnetic field parameters for short-term forecasts of the SYM-H index based on 1- and 5-min resolution data. SYMHnet takes, as input, the time series of the parameters' values provided by NASA's Space Science Data Coordinated Archive and predicts, as output, the SYM-H index value at time point t + w hours for a given time point t where w is 1 or 2. By incorporating Bayesian inference into the learning framework, SYMHnet can quantify both aleatoric (data) uncertainty and epistemic (model) uncertainty when predicting future SYM-H indices. Experimental results show that SYMHnet works well at quiet time and storm time, for both 1- and 5-min resolution data. The results also show that SYMHnet generally performs better than related machine learning methods. For example, SYMHnet achieves a forecast skill score (FSS) of 0.343 compared to the FSS of 0.074 of a recent gradient boosting machine (GBM) method when predicting SYM-H indices (1 hr in advance) in a large storm (SYM-H = -393 nT) using 5-min resolution data. When predicting the SYM-H indices (2 hr in advance) in the large storm, SYMHnet achieves an FSS of 0.553 compared to the FSS of 0.087 of the GBM method. In addition, SYMHnet can provide results for both data and model uncertainty quantification, whereas the related methods cannot.
The nuclear imaging system at the National Ignition Facility (NIF) is a crucial diagnostic for determining the geometry of inertial confinement fusion implosions. The geometry is reconstructed from a neutron aperture image via a set of reconstruction algorithms using an iterative Bayesian inference approach. An important step in these reconstruction algorithms is finding the fusion source location within the camera field-of-view. Currently, source localization is achieved via an iterative optimization algorithm. In this paper, we introduce a machine learning approach for source localization. Specifically, we train a convolutional neural network to predict source locations given a neutron aperture image. We show that this approach decreases computation time by several orders of magnitude compared to the current optimization-based source localization while achieving similar accuracy on both synthetic data and a collection of recent NIF deuterium–tritium shots.
The nuclear imaging system at the National Ignition Facility (NIF) is a crucial diagnostic for determining the geometry of inertial confinement fusion implosions. The geometry is reconstructed from a neutron aperture image via a set of reconstruction algorithms using an iterative Bayesian inference approach. An important step in these reconstruction algorithms is finding the fusion source location within the camera field-of-view. Currently, source localization is achieved via an iterative optimization algorithm. In this paper, we introduce a machine learning approach for source localization. Specifically, we train a convolutional neural network to predict source locations given a neutron aperture image. We show that this approach decreases computation time by several orders of magnitude compared to the current optimization-based source localization while achieving similar accuracy on both synthetic data and a collection of recent NIF deuterium–tritium shots.
Differential cross section measurements are the currency of scientific exchange in particle and nuclear physics. A key challenge for these analyses is the correction for detector distortions, known as deconvolution or unfolding. Binned unfolding of cross section measurements traditionally rely on the regularized inversion of the response matrix that represents the detector response, mapping pre-detector (`particle level') observables to post-detector (`detector level') observables. In this paper we introduce Neural Posterior Unfolding, a modern, Bayesian approach that leverages normalizing flows for unfolding. By using normalizing flows for neural posterior estimation, NPU offers several key advantages including implicit regularization through the neural network architecture, fast amortized inference that eliminates the need for repeated retraining, and direct access to the full uncertainty in the unfolded result. In addition to introducing NPU, we implement a classical Bayesian unfolding method called Fully Bayesian Unfolding (FBU) in modern Python so it can also be studied. These tools are validated on simple Gaussian examples and then tested on simulated jet substructure examples from the Large Hadron Collider (LHC). We find that the Bayesian methods are effective and worth additional development to be analysis ready for cross section measurements at the LHC and beyond.
An accurate estimation of spatially and temporally continuous global terrestrial evapotranspiration (ET) is essential in the assessment of surface energy, water and carbon cycles. The Global LAnd Surface Satellite (GLASS) ET product Version 4.0 (v4.0) based on the Bayesian model averaging (BMA) method was generated to estimate global terrestrial ET. However, certain uncertainty for the GLASS ET product v4.0 limits its application. In this study, we introduced the deep neural networks (DNN) merging framework to improve terrestrial ET estimation for GLASS ET product Version 5.0 (v5.0) generation by integrating five satellite-derived ET products [Moderate Resolution Imaging Spectroradiometer (MODIS) ET product (MOD16), Shuttleworth–Wallace dual-source ET product (SW), Priestley–Taylor-based ET product (PT-JPL), modified satellite-based Priestley–Taylor ET product (MS-PT) and simple hybrid ET product (SIM)]. We compared the performance of DNN method against other merging methods, including GLASS ET algorithm v4.0 (BMA), the gradient boosting regression tree (GBRT) method and the random forest (RF) method, based on 195 global eddy covariance (EC) flux towers covering observations from 2000 through 2015. Validations indicated that the DNN had the highest accuracy among four merging methods across different land cover types, yielding the highest average determination coefficients (R 2 , 0.62), root-mean-squared-error (RMSE, 24.1 W/m 2 ) and Kling–Gupta efficiency (KGE, 0.77) with a of 99% confidence interval. Compared with GLASS ET algorithm v4.0, the DNN improved on the R 2 by approximately 7% (p < 0.01) and the KGE by 10%. Based on the DNN, we then generated 8-day GLASS ET product v5.0 globally with a 1 km spatial resolution from 2001 to 2015 driven by GLASS vegetation and surface net radiation (R n ) datasets and Modern-Era Retrospective Analysis for Research and Applications, Version 2 (MERRA2) datasets. Finally, this global terrestrial ET product provides a valuable dataset for monitoring regional and global water resources and environmental changes.
Bayesian parameter inference is useful to improve Li-ion battery diagnostics and can help formulate battery aging models. However, it is computationally intensive and cannot be easily repeated for multiple cycles, multiple operating conditions, or multiple replicate cells. To reduce the computational cost of Bayesian calibration, numerical solvers for physics-based models can be replaced with faster surrogates. A physics-informed neural network (PINN) is developed as a surrogate for the pseudo-2D (P2D) battery model calibration. For the P2D surrogate, additional training regularization was needed as compared to the PINN single-particle model (SPM) developed in Part I. Both the PINN SPM and P2D surrogate models are exercised for parameter inference and compared to data obtained from a direct numerical solution of the governing equations. A parameter inference study highlights the ability to use these PINNs to calibrate scaling parameters for the cathode Li diffusion and the anode exchange current density. By realizing computational speed-ups of ~2250x for the P2D model, as compared to using standard integrating methods, the PINN surrogates enable rapid state-of-health diagnostics. Finally, in the low-data availability scenario, the testing error was estimated to ~2 mV for the SPM surrogate and ~10 mV for the P2D surrogate which could be mitigated with additional data.
Energy systems models typically take the form of complex partial differential equations which make multiple forward calculations prohibitively expensive. Fast and data-efficient construction of surrogate models is of utmost importance for applications that require parameter exploration such as design optimization and Bayesian calibration. In presence of a large number of parameters, surrogate models that capture correct dependencies may be difficult to construct with traditional techniques. The issue is addressed here with the formulation of the surrogate model constructed via Physics-Informed Neural Networks (PINN) which capture the dependence with respect to the parameters to estimate, while using a limited amount of data. Since forward evaluations of the surrogate model are cheap, parameter exploration is made inexpensive, even when considering a large number of parameters.
Fast analysis of collective Thomson scattering ion acoustic wave features using a deep convolutional neural network model is presented. The network was trained from spectra to predict the plasma parameters, including ion velocities, population fractions, and ion and electron temperatures. A fully kinetic particle-in-cell simulation was used to model a laboratory astrophysics experiment and simulate a diagnostic image of the ion acoustic wave feature. Network predictions were compared with Bayesian inference of the plasma model parameters for both the simulated and experimentally measured images. Both approaches were fairly accurate predicting the simulated image and the network predictions matched a good portion of the Bayesian results for the experimentally measured image. The Bayesian approach is more robust to noise and motivates future work to train deep learning models with realistic noise. The advantage of the deep learning model is making thousands of predictions in a few hundred milliseconds, compared to a few seconds to minutes per prediction for the optimization and Bayesian approaches presented here. The results demonstrate promising capabilities of deep learning models to analyze Thomson data orders of magnitude faster than conventional methods when using the neural network for standalone analysis. If more rigorous analysis is needed, neural network predictions can be used to quickly initialize other optimization methods and increase chances of success. This is especially useful when the dataset becomes very large or highly dimensional and manually refining initial conditions for the entire dataset are no longer tractable.
We present global predictions of the ground state mass of atomic nuclei based on a novel Machine Learning algorithm. We combine precision nuclear experimental measurements together with theoretical predictions of unmeasured nuclei. This hybrid data set is used to train a probabilistic neural network. In addition to training on this data, a physics-based loss function is employed to help refine the solutions. The resultant Bayesian averaged predictions have excellent performance compared to the testing set and come with well-quantified uncertainties which are critical for contemporary scientific applications. We assess extrapolations of the model’s predictions and estimate the growth of uncertainties in the region far from measurements.