SEARCH · Engineering Papers
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
This work demonstrates the applicability of partially Bayesian neural networks (PBNNs) to active and transfer learning within materials science and chemistry domains, combining reliable uncertainty quantification (UQ) with NN flexibility.
Machine learning (ML) has been leveraged to tackle a diverse range of tasks in almost all branches of nuclear engineering. Many of the successes in ML applications can be attributed to the recent performance breakthroughs in deep learning, the growing availability of computational power, data, and easy-to-use ML libraries. However, these empirical successes have often outpaced our formal understanding of the ML algorithms. An important but under-rated area is uncertainty quantification (UQ) of ML. ML-based models are subject to approximation uncertainty when they are used to make predictions, due to sources including but not limited to, data noise, data coverage, extrapolation, imperfect model architecture and the stochastic training process. The goal of this paper is to clearly explain and illustrate the importance of UQ of ML. We will elucidate the differences in the basic concepts of UQ of physics-based models and data-driven ML models. Various sources of uncertainties in physical modeling and data-driven modeling will be discussed, demonstrated, and compared. We will also present and demonstrate a few techniques to quantify the ML prediction uncertainties, including Monte Carlo dropout, deep ensemble, Bayesian neural networks, Gaussian Processes and conformal prediction. Lastly, we will discuss the need for building a verification, validation and UQ framework to establish ML credibility.
Abstract While methane is the second largest contributor to global warming after carbon dioxide, it has a larger warming effect over a much shorter lifetime. Despite accelerated technological efforts to radically reduce global carbon dioxide emissions, rapid reductions in methane emissions are needed to limit near-term warming. Being primarily emitted as a byproduct from agricultural activities and energy extraction, methane is currently monitored via bottom–up (i.e. activity level) or top–down (via airborne or satellite retrievals) approaches. However, significant methane leaks remain undetected and emission rates are challenging to characterize with current monitoring frameworks. In this paper, we study the design of a layered monitoring approach that combines bottom–up and top–down approaches as an integrated sensing network. By recognizing that varying meteorological conditions and emission rates impact the efficacy of bottom–up monitoring, we develop a probabilistic approach to optimal sensor placement in its bottom–up network. Subsequently, we derive an inverse Bayesian framework to quantify the improvement that a design-optimized integrated framework has on emission-rate quantifications and their uncertainties. We find that under realistic meteorological conditions, the overall error in estimating the true emission rates is approximately 1.3 times higher, with their uncertainties being approximately 2.4 times higher, when using a randomized network over an optimized network, highlighting the importance of optimizing the design of integrated methane sensing networks. Further, we find that optimized networks can improve scenario coverage fractions by more than a factor of 2 over experimentally-studied networks, and identify a budget threshold beyond which the rate of optimized-network coverage improvement exhibits diminishing returns, suggesting that strategic sensor placement is also crucial for maximizing network efficiency.
Not Available
The last decade has seen numerous record-shattering heatwaves in all corners of the globe. In the aftermath of these devastating events, there is interest in identifying worst-case thresholds or upper bounds that quantify just how hot temperatures can become. Generalized Extreme Value theory provides a data-driven estimate of extreme thresholds; however, upper bounds may be exceeded by future events, which undermines attribution and planning for heatwave impacts. Here, we show how the occurrence and relative probability of observed yet unprecedented events that exceed a priori upper bound estimates, so-called “impossible” temperatures, has changed over time. We find that many unprecedented events are actually within data-driven upper bounds, but only when using modern spatial statistical methods. Furthermore, there are clear connections between anthropogenic forcing and the “impossibility” of the most extreme temperatures. Robust understanding of heatwave thresholds provides critical information about future record-breaking events and how their extremity relates to historical measurements.
In this work, we propose a Stein variational gradient descent (SVGD) method to concurrently sparsify, train, and provide uncertainty quantification (UQ) of a complexly parameterized model, such as a neural network (NN). It employs a graph reconciliation and condensation process to reduce complexity and increase similarity in the Stein ensemble of parameterizations. Therefore, the proposed concurrent, condensed SVGD (ccSVGD) method can provide UQ on parameters, not just outputs. Furthermore, the parameter reduction speeds up the convergence of the Stein gradient descent as it reduces the combinatorial complexity by aligning and differentiating the sensitivity to parameters. These properties are demonstrated with an illustrative example and an application to a mechanical response representation problem in solid mechanics.
Hamiltonian Monte Carlo (HMC) is a powerful and accurate method to sample from the posterior distribution in Bayesian inference. However, HMC techniques are computationally demanding for Bayesian neural networks due to the high dimensionality of the network’s parameter space and the non-convexity of their posterior distributions. Therefore, various approximation techniques, such as variational inference (VI) or stochastic gradient MCMC, are often employed to infer the posterior distribution of the network parameters. Such approximations introduce inaccuracies in the inferred distributions, resulting in unreliable uncertainty estimates. In this work, we propose a hybrid approach that combines inexpensive VI and accurate HMC methods to efficiently and accurately quantify uncertainties in neural networks and neural operators. The proposed approach leverages an initial VI training on the full network. We examine the influence of individual parameters on the prediction uncertainty, which shows that a large proportion of the parameters do not contribute substantially to uncertainty in the network predictions. This information is then used to significantly reduce the dimension of the parameter space, and HMC is performed only for the subset of network parameters that strongly influence prediction uncertainties. This yields a framework for accelerating the full batch HMC for posterior inference in neural networks. We demonstrate the efficiency and accuracy of the proposed framework on deep neural networks and operator networks, showing that inference can be performed for large networks with tens to hundreds of thousands of parameters. Finally, we show that this method can effectively learn surrogates for complex physical systems by modeling the operator that maps from upstream conditions to wall-pressure data on a cone in hypersonic flow.
In this work, we present two neural network approaches that approximate the solutions of static and dynamic conditional optimal transport (COT) problems. Both approaches enable conditional sampling and conditional density estimation, which are core tasks in Bayesian inference—particularly in the simulation-based (“likelihood-free”) setting. Our methods represent the target conditional distribution as a transformation of a tractable reference distribution. Obtaining such a transformation, chosen here to be an approximation of the COT map, is computationally challenging even in moderate dimensions. To improve scalability, our numerical algorithms use neural networks to parameterize candidate maps and further exploit the structure of the COT problem. Our static approach approximates the map as the gradient of a partially input convex neural network. It uses a novel numerical implementation to increase computational efficiency compared to state-of-the-art alternatives. Our dynamic approach approximates the conditional optimal transport via the flow map of a regularized neural ODE; compared to the static approach, it is slower to train but offers more modeling choices and can lead to faster sampling. We demonstrate both algorithms numerically, comparing them with competing state-of-the-art approaches, using benchmark datasets and simulation-based Bayesian inverse problems.
The precipitating hydrometeor parameters used in cloud microphysics schemes carry inherent uncertainties. The quantification of these uncertainties, together with parameter optimization, can significantly improve precipitation forecasts. This study investigates the effects of 13 parameters in the Weather Research and Forecasting (WRF) Double-Moment 6-class (WDM6) microphysics scheme, which define the hydrometeor characteristics such as fall velocity–diameter and mass–diameter relationships, as well as the shape parameter of the drop size distribution for precipitating particles such as rain, snow, and graupel on simulated winter precipitation. A comparison between the model's pre-defined parameters and observations from the International Collaborative Experiments for the PyeongChang 2018 Olympic and Paralympic winter games (ICE-POP 2018) field campaign reveals that the fall velocity–diameter relationship for rain, the mass–diameter relationships for snow and graupel, and the shape parameters for all precipitating particles in the WDM6 scheme deviate from the median values observed by the two-dimensional video disdrometer (2DVD). To quantify parameter sensitivities, a perturbed parameter ensemble (PPE) of 256 simulations was conducted within parameter ranges constrained by 2DVD observations for three winter precipitation cases. Bayesian optimization was then applied to identify parameter sets that minimized the root mean square error (RMSE) for each case, achieving reductions of up to 30.2 %. These results demonstrate that ensemble-based uncertainty quantification and parameter optimization can help identify key parameters and provide a pathway to improving precipitation simulation performance. In addition, measurement sites can be strategically selected based on regions that show high sensitivity to variations in hydrometeor characteristic parameters.
In numerous applications, the integration of prior knowledge and historical information is essential, particularly for tasks requiring the solution of ordinary or partial differential equations (ODEs/PDEs) in data-sparse or noisy environments. For instance, achieving accurate solutions to time-dependent PDEs with limited initial condition measurements necessitates an effective strategy for embedding prior knowledge. Hard-parameter sharing architectures in neural networks (NNs) have demonstrated success in both traditional and scientific machine learning domains, facilitating the learning of informative representations. Here, in this study, we introduce a novel, yet efficient, method to enhance physics-informed neural networks (PINNs) by incorporating a multi-head structure that enables the learning of functional priors from both empirical data and governing physical laws. This prior information can then be used to address data sparsity and high-level noise in solving ODE/PDE problems with uncertainty quantification (UQ). The approach, termed Multi-Head PINN (MH-PINN), consists of a shared body NN and multiple head NNs, each corresponding to an individual PINN instance. Our framework for functional prior learning is carried out in two stages: (1) training the MH-PINNs to develop a shared body NN alongside multiple head NNs, and (2) employing these trained head NNs to estimate a prior distribution through a normalizing flow-based density estimator. The learned functional prior can then be applied as a regularization mechanism in deterministic contexts or as an informative prior within a Bayesian inference framework, aiding in the resolution of subsequent ODE/PDE tasks. We evaluate the efficacy of MH-PINNs across five benchmark problems, including a high-dimensional parametric PDE, all characterized by data sparsity or substantial noise levels. Our findings reveal that MH-PINNs deliver accurate solutions and robust UQ, demonstrating adaptability across a range of complex and challenging scenarios.
The computational cost of high-fidelity numerical models makes outer-loop analysis, which requires repeated interrogation of the model such as uncertainty quantification, computationally demanding. Multi-fidelity methods, which construct a surrogate model using data from an ensemble of models of varying cost and accuracy, can substantially reduce the cost of outer-loop analysis. However, these methods can be difficult to apply when the model ensemble does not admit a clear hierarchy a priori and the correlations between models are low. Consequently, in this paper, we present a multi-fidelity method that leverages dimension reduction to enhance the correlation between models, thereby reducing the amount of data needed to train a surrogate from an unordered ensemble of models. Our method utilizes basis adaptation to build low-dimensional polynomial chaos expansions of each model and employs Multi-fidelity Networks to encode the relationships among models. We show that the resulting method exhibit two notable advantages over its counterpart: (1) enhanced accuracy (both reduced bias and variance); and (2) reduced dependency on the graph structure encoding relationships among models. We demonstrate the approach on an analytical test problem and a challenging finite element model for a spent nuclear fuel. Our method produces a surrogate model that is significantly more accurate than either a single-fidelity surrogate or a multi-fidelity surrogate constructed without basis adaptation.
This manuscript proposes a novel information-theoretic approach to the quantification of experimental relevance, i.e., coverage, to achieve optimal data assimilation results for nuclear engineering applications. Specifically, this work posits the need for a new metric, called coverage (q C ) of an application’s quantity of interest, i.e., eigenvalue or power peaking for an advanced reactor concept, defined herein as the theoretically maximum achievable reduction in the quantity’s uncertainty given measurements from a pool of experiments in a manner that is independent of the data assimilation procedure employed. Currently, reduction in a quantity’s uncertainty is strongly biased by the underlying assumptions of the assimilation procedure to account for the under-determined nature of such problems and the similarity criterion employed to identify relevant experiments. To address this challenge, this work has developed a coverage metric, q C , based on mutual information, which establishes a new conceptual framework for assessing coverage, one that is independent of the model parameters and responses degree of variations in both the experimental and application domains, i.e., linear vs non-linear, and their prior uncertainty distributions, i.e., Gaussian vs. non-Gaussian. The q C is an entropic measure capable of addressing coverage for general nonlinear problems with non-Gaussian uncertainties and inclusive of the measurement uncertainties from multiple experiments. Numerical experiments from manufactured analytical problems as well as a set of benchmarks from the ICSBEP handbook are employed to demonstrate its theoretical and practical performance as compared to the c k -based experiment selection methodology, commonly employed in the neutronic community. The manuscript then employs other well-known adaptations to existing data assimilation methodologies for nonlinear and non-Gaussian problems capable of achieving the coverage posited by q C .
Here, this paper develops a novel method for reconstructing the full-field response of structural dynamic systems using sparse measurements. The singular value decomposition is applied to a frequency response matrix relating the structural response to physical loads, base motion, or modal loads. The left singular vectors form a non-physical reduced basis that can be used for response reconstruction with far fewer sensors than existing methods. The contributions of the singular vectors to measured response are termed singular-vector loads (SVLs) and are used in a regularized Bayesian framework to generate full-field response estimates and confidence intervals. The reconstruction framework is applicable to the estimation of single data records and power spectral densities from multiple records. Reconstruction is successfully performed in configurations where the number of SVLs to identify is less than, equal to, and greater than the number of sensors used for reconstruction. In a simulation featuring a seismically excited shear structure, SVL reconstruction significantly outperforms modal FRF-based reconstruction and successfully estimates full-field responses with as few as two uniaxial accelerometers. SVL reconstruction is further verified in a simulation featuring an acoustically excited cylinder. Finally, response reconstruction and uncertainty quantification are performed on an experimental structure with three shaker inputs and 27 triaxial accelerometer outputs.
Prediction of the creep rupture life of engineering metals is critical for qualification and design of new materials. The use of long-term creep tests and the need to quantify the performance variability in a priori similar systems hinder the rapid creep assessment of a given material. Therefore, it is essential to develop methods that can extrapolate the long-term performance of alloys and the associated variability from short-term experiments. To this end, this study introduces a new model which enables the estimation of the rupture life of a material for a given stress and temperature. This model relies on two components. First, a new relation for the minimum creep rate (MCR) of materials is introduced. It includes a stress dependent stress exponent allowing the model to capture the variation of MCR across a wide range of temperatures and stresses. Second, employing the Monkman-Grant (MG) law, we establish a relation between stress, temperature and creep rupture life. Together, these two elements yield a new closed-form mathematical expression for the Larson Miller parameter as a function of stress and temperature. This expression captures the creep rupture time for many metals (Gr91, Copper, Gr122 and 347H) and compares favorably with alternate empirical approaches. The model is then used to assess the minimum duration of creep rates necessary to qualify the material up to 100000h. Furthermore, it is found that depending on the material system, creep tests as few as five limited to 5000 h for steels (Gr91, Gr122, 347H) and 100 h for copper are sufficient to model creep lifetimes. Finally, using a Bayesian inference-based approach to calibrate the model, we demonstrate that variability in rupture life can be captured via the quantification of the uncertainty in the model parameters and extrapolated from a limited number of short to moderately short creep tests; thereby paving the way for accelerated creep testing.
Experiments are indispensable for developing models of complex systems. Carefully designed experiments can provide substantial savings for these expensive data-acquisition opportunities. However, designs based on heuristics are often suboptimal for systems with multiphysics, nonlinear dynamics, and uncertain and noisy environments. Optimal experimental design, while leveraging predictive models, seeks to systematically quantify and maximize the value of experiments. In this project, we focused on the design of multiple experiments, where current approaches are largely suboptimal: batch-design does not adapt to new data acquired during the experiment campaign (no feedback), and greedy/myopic design ignores future dynamics and consequences (no lookahead). We developed the mathematical framework and computational methods for sequential optimal experimental design (sOED) for complex systems. We enabled tractable model-based sOED in a rigorous manner through novel algorithms based on reinforcement learning, and investigated the effects of human experimenters on the design process. Our methods are fully Bayesian, able to quantify and update uncertainty in a principled manner. The traits aimed by our approach—mathematical rigor and optimality, human effects and uncertainty quantification, computational practicality—are crucial for elevating the standards of artificial intelligence (AI) to support decision-making in scientific domains, and contribute toward trust and realistic adoption of AI in experimental design practice.
MELCOR-TMAP is a combined thermal-hydraulics and tritium tracking code developed to simulate severe accident scenarios in fission and fusion power plants. Here, we demonstrate the results of MELCOR-TMAP analyses on historical ARIES program reference designs. By coupling MELCOR-TMAP with the open source RAVEN probabilistic risk analysis framework’s Bayesian UQ capabilities, we also demonstrate key uncertainties in material properties with the highest impact on tritium inventory and plant risk.
Dempster-Shafer theory (DST) is a mathematical framework that allows for uncertainty or ignorance to be quantified and included when making predictions from evidence. This is in contrast to Bayesian theory, which does not allow for any quantification of ignorance. The framework is described in great detail in [7]. DST is particularly useful for problems where the inclusion of additional evidence (for example, data from another sensor) could lead to a different conclusion. Thus, it is a useful data fusion method, especially in applications not suited to maximum likelihood or maximum a posteriori estimations due to limited samples or incomplete prior knowledge.
Sustained advances in the mathematics of modeling and simulation have resulted in the capability today for routine simulation of a number of large scale complex DOE-relevant systems. As remarkable as this capability for solving the so-called forward problem is, it is typically only the first step-an inner loop within an outer loop that explores the simulation model's parameter space and decision space to characterize uncertainty in the model's predictions, learn unknown model parameters from data, design the most informative experiments, determine optimal control strategies, and create optimal designs. Broadly, what unifies all of these outer loop problems is that they are, in one form or another, optimization problems over parameter/control/design space that are constrained by complex uncertain models. To fully realize the power of scientific simulation as a basis for scientific discovery, technological innovation, and rational decision-making, it is imperative to move beyond simulation to tackle the outer loop of optimization for learning from data, experimental design, and control with complex uncertain models. When the models under consideration are large-scale and complex, and when the optimization variable and uncertain parameter spaces are high (or infinite) dimensional, this constitutes a grand challenge of the highest order, and is intractable with conventional methods. To overcome these challenges, the AEOLUS Center was established to develop a unified mathematical, computational, and statistical framework for (1) Learning predictive models from complex data via Bayesian inference and optimization, and (2) Optimizing experiments, processes, and designs using the resulting uncertain models. These problems are intractable with conventional methods, for several reasons: (1) The simulation problems that govern the inner loops of the optimization problems are expensive to execute (due to severe nonlinearity, heterogeneity, multiphysics/multiscale coupling); (2) The optimization variable and uncertain parameter spaces are high dimensional, often stemming from discretizations of infinite dimensional fields such as initial conditions, sources, or material properties. We argue that the key to overcoming these challenges is to develop new mathematical, computational, and statistical methods that exploit the structure of the Bayesian inference and optimization problems mediated by their underlying complex uncertain models. This structure includes the regularity, sparsity, geometry, low intrinsic dimensionality, and multifidelity nature of the maps from uncertain parameter/optimization variable spaces to the specific objectives targeted: Bayesian inference, optimal experimental design, and optimal control design. Black box methods developed as generic tools are incapable of exploiting this structure. To be successful, we must create, integrate, and cross-fertilize ideas across multiple areas of applied math--including approximation theory, Bayesian inference, data science, experimental design, information theory, machine learning, model reduction, optimal control theory, parallel algorithms, PDE-constrained optimization, randomized algorithms, stochastic optimization, and uncertainty quantification--all while exploiting the structure of the problems at hand. With this goal in mind, we have marshaled a team of leading authorities in these areas. While the methods we develop will be broadly applicable across a wide spectrum of DOE problems in which experiments inform models and the systems those models describe must be optimized under uncertainty, we have chosen a specific area, advanced manufacturing and materials, to drive our work. AMM is characterized by complex models across multiple scales, and is a rich source of challenging problems in inference, experimental design, and optimal control, requiring multifaceted and integrated advances in applied mathematics. As such, AMM serves as an excellent vehicle to motivate and demonstrate the advances in applied mathematics developed by our center.