Framework for Optimization, Quantification of Uncertainty, and Surrogates (FOQUS) – Demonstration
Presented at the 2022 Carbon Management Project Review Meeting in Pittsburgh, PA
Deshpande, Anuja↗
SEARCH · Engineering Papers
Results for “uncertainty Quantification”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Framework for Optimization, Quantification of Uncertainty and Surrogates – Updated Capabilities
This poster presents updated capabilities within the FOQUS Toolset, covering core features as well as project-based applications. Specifically, the poster highlights advancements in FOQUS integration with cloud-based clients, plugin support for machine learning tools and external simulations, sequential design of experiments, and advanced support for dynamic process models. The applications demonstrate the breadth of FOQUS predictive capabilities for chemical, energy and economic system analysis.
Paul, Brandon↗
A Neural Optimizer With Decision-Focused Learning for Optimal Energy Storage Operation
Here, this article introduces a neural optimizer-based framework for optimizing battery energy storage system (BESS) control for grid services, including demand charge and energy cost reduction. By leveraging decision-focused learning (DFL), the proposed framework ensures seamless integration and adaptation, significantly enhancing control performance. A patch time-series transformer is employed for peak load forecasting, incorporating aleatoric uncertainty quantification to account for forecasting uncertainties within the decision-making process. The framework utilizes a solver-in-the-loop approach to generate optimal BESS actions, which are then used to train the neural optimizer-based agent. By co-optimizing both BESS operational modes and output power within the NN, the system achieves improved performance and robustness. After initial training, the forecasting and control models are jointly fine-tuned to account for forecasting errors, further improving decision precision and efficiency through DFL. Case studies are performed to validate the performance of the framework using multiple real-world datasets, demonstrating superior performance in monthly peak load forecasting compared to state-of-the-art models. In addition, the results are compared against existing decision-making approaches. The results demonstrate a reduction in monthly peak forecasting error by approximately 15% across various performance measures and achieve an optimization gap for BESS operation that is about three times smaller compared to existing methods.
Kim, Hyeonjin [Pacific Northwest National Laborato↗
Examples of Mission-driven Data Science from Jefferson Lab and ACES
This presentation details mission-driven data science initiatives at Jefferson Lab and the Joint Institute for Advanced Computing on Environmental Studies (ACES). JLab, a U.S. Department of Energy Office of Science national laboratory, operates the Continuous Electron Beam Accelerator Facility (CEBAF), and is the lead institute for the new High Performance Data Facility (HPDF) Hub. The Joint Institute for ACES brings together interdisciplinary teams in health informatics, climate modeling, computer science, and physics to address environmental challenges, including flood modeling. The Hampton Roads region, particularly Norfolk and Virginia Beach, faces increasing flood risks, motivating the need for rapid, reliable, and risk-aware decision support. ACES’s flooding work has a focus on uncertainty quantification (UQ) and machine learning (ML) for coastal flood management. The work is motivated by the increasing vulnerability of communities such as Norfolk and Virginia Beach, Virginia, to frequent coastal flooding events, and the need for rapid, reliable decision support. The research develops computationally efficient ML surrogate models to forecast water levels and flooding risk. A central theme is the quantification and calibration of predictive uncertainty, especially for out-of-distribution (OOD) scenarios, using techniques such as Monte Carlo Dropout, Deep Ensembles, Gaussian Processes, and Deep Quantile Regression (DQR). The study demonstrates that distance-aware UQ is critical for reliable scientific AI, particularly in high-dimensional, safety-critical, and real-time applications.
McSpadden, Diana [Thomas Jefferson National Accele↗
Adaptive learning-driven high-throughput synthesis of oxygen reduction reaction Fe–N–C electrocatalysts
Reducing human reliance on inefficient energy systems and fossil fuels has become more urgent due to the consequences of global climate change. However, traditional trial-and-error approaches have hampered our ability to accelerate the discovery and implementation of functional materials for efficient energy conversion devices, such as polymer electrolyte fuel cells (PEFCs). To address this, we develop an adaptive learning framework that integrates machine learning and state-of-the-art capabilities in high-throughput synthesis to achieve expedited optimization of iron-nitrogen-carbon PEFC oxygen reduction reaction (ORR) electrocatalysts. We use statistical inference, uncertainty quantification, and global optimization to build a computational design-of-experiment tool that identifies the optimum compositions to be investigated next to reduce the demands placed on experimental materials discovery. We benchmark the ability of the proposed strategy to discover optimum catalyst synthesis conditions in a six-dimensional search space when starting with a thirty-six-sample database. By following the adaptive learning strategy, we synthesize fourteen new catalysts from approximately ten billion unique compositions and discover four catalysts that outperform all original samples. The best machine learning-optimized catalyst is 33% more active than the highest-performing one in the initial database, showing an ORR activity seven times larger than those typically reported for the same class of materials.
36 MATERIALS SCIENCE↗
Quantifying the impact of temporal analysis of products reactor initial state uncertainties on kinetic parameters
The temporal analysis of products (TAP) reactor, a transient kinetic tool, provides users with information as the catalyst state evolves. However, the state of the art for TAP uncertainty quantification only considers the experimental noise present in the outlet flow signal. Additional sources of uncertainty, including, but not limited to, initial surface coverages, the catalyst zone location, the inert void fraction, and the gas pulse intensity and pulse delay, are not included. For this reason, a framework for quantifying all uncertainty sources present in TAP experiments is presented and applied to a carbon monoxide oxidation case study. Herein, two methods for quantifying these sources of uncertainty are introduced. The first utilizes initial state sensitivities to approximate the parameter variances, as well as to provide insights into the structural certainty of the model. The second generates parameter confidence distributions through an ensemble-based sampling algorithm. This initial state covariance matrix can ultimately be merged with the experimental noise covariance matrix, providing a unified description of the parameter uncertainties for a single TAP experiment.
36 MATERIALS SCIENCE↗
Surrogate-driven Variance-based Sensitivity Analysis of Thermal Storage Tanks in Integrated Energy Systems
Sensitivity analysis and uncertainty quantification are essential steps for enhancing the accuracy of computational models by identifying and mitigating uncertainties. This study focuses on these steps for the Thermal Energy Delivery System at Idaho National Laboratory, specifically targeting the thermocline tank. Using a Modelica/Dymola simulation model, the study perturbed various design parameters and boundary conditions, including shape factor, porosity, outlet temperature, inlet mass flow rate, and system pressure, to predict and quantify uncertainty in the tank’s ax- ial temperature. A dataset of over 1,000 simulations was generated, and surrogate models were developed using the pyMAISE (Michigan Artificial Intelligence Standard Environment) library, which is an Automatic Machine Learning library for nuclear engineering applications. The optimal model, a feedforward neural network with two hidden layers, achieved an R2 score above 0.99 and a mean absolute error below 1 Kelvin. Sensitivity analyses using Sobol indices and Fourier amplitude sensitivity testing methods on this surrogate model revealed that the inlet mass flow rate at initial timestamps and porosity significantly impacts predicted temperatures across all sensors and time steps.
22 - GENERAL STUDIES OF NUCLEAR REACTORS↗
Sensitivity Analysis of Genome-Scale Metabolic Flux Prediction
TRIMER, Transcription Regulation Integrated with MEtabolic Regulation, is a genome-scale modeling pipeline targeting at metabolic engineering applications. Using TRIMER, regulated metabolic reactions can be effectively predicted by integrative modeling of metabolic reactions with a Transcription Factor (TF)-gene regulatory network (TRN), which is modeled via a Bayesian network (BN). In this paper, we focus on sensitivity analysis of metabolic flux prediction for uncertainty quantification of BN structures for TRN modeling in TRIMER. We propose a computational strategy to construct the uncertainty class of TRN models based on the inferred regulatory order uncertainty given transcriptomic expression data. With that, we analyze the prediction sensitivity of the TRIMER pipeline for the metabolite yields of interest. The obtained sensitivity analyses can guide Optimal Experimental Design (OED) to help acquire new data that can enhance TRN modeling and achieve specific metabolic engineering objectives, including metabolite yield alterations. Here we have performed small- and large-scale simulated experiments, demonstrating the effectiveness of our developed sensitivity analysis strategy for BN structure learning to quantify the edge importance in terms of metabolic flux prediction uncertainty reduction and its potential to effectively guide OED.
59 BASIC BIOLOGICAL SCIENCES↗
Quantifying uncertainties in α -nucleus reaction dynamics informed from first principles
The ab initio symmetry-adapted no-core shell model is a microscopic many-body method which naturally describes challenging collective and clustering features of atomic nuclei. Wave functions and observables computed with realistic nucleon-nucleon forces in this framework are tied to first principles, and are hence well-suited for rigorous uncertainty quantification. We discuss α-deuteron and α- 12 C cluster potentials informed by symmetry-adapted calculations, and propagate uncertainties in the effective binary cluster method as well as those in the nuclear interaction to reaction observables, namely scattering phase shifts, cross sections, partial widths and resonance energies. Here, we find that the overall uncertainties are dominated by those originating in the underlying nuclear force, speaking to the need for tighter constraints on realistic nucleon-nucleon interactions.
Ab initio↗
Uncertainty-Aware Machine Learning for Small-Angle X-ray Scattering Analysis in Autonomous Experimentation
Small-angle X-ray scattering (SAXS) is a powerful high-throughput characterization tool for probing nanoscale structure in native sample environments, providing real-time morphological information such as nanoparticle size and shape during synthesis. However, automated SAXS data analysis for extracting meaningful structural parameters is non-trivial and remains a bottleneck in closed-loop experimentation towards autonomous materials discovery, which demands fast, reliable, and uncertainty-aware data analysis. Here, we develop a machine-learning approach for automated SAXS analysis tailored to closed-loop nanoparticle synthesis. A Random Forest (RF) regression model is trained on 100,000 synthetic SAXS curves generated from polydisperse spherical nanoparticles with realistic background contributions. Using normalized one-dimensional SAXS intensity profiles as input, the RF model directly predicts nanoparticle radius, size polydispersity, and background parameters, while the ensemble standard deviation across trees provides built-in uncertainty quantification (UQ). On synthetic data, we show that combining fit-quality metrics (R 2 , MAE) with thresholds on prediction uncertainty reliably identifies accurate parameter estimates without access to ground truth. We then apply the trained model to 365 experimental SAXS profiles of citrate-reduced gold nanoparticles synthesized using an automated droplet-flow microreactor with in situ SAXS at a synchrotron beamline, classifying the results into high- and low-confidence subsets based on UQ metrics. Finally, we integrate RF-based SAXS analysis into a simulated closed-loop optimization campaign using Gaussian process Bayesian optimization to minimize nanoparticle polydispersity, benchmarking against conventional automated Levenberg–Marquardt fitting. The RF-guided campaign exhibits substantially faster convergence and lower relative opportunity cost (∼0.07 vs ∼0.3), demonstrating that uncertainty-aware machine-learning SAXS analysis significantly enhances the efficiency and robustness of autonomous nanomaterials synthesis workflows.
Bayesian optimization↗
Multilevel Monte Carlo Estimators For Derivative-Free Optimization Under Uncertainty
Optimization is a key tool for scientific and engineering applications; however, in the presence of models affected by uncertainty, the optimization formulation needs to be extended to consider statistics of the quantity of interest. Optimization under uncertainty (OUU) deals with this endeavor and requires uncertainty quantification analyses at several design locations; i.e., its overall computational cost is proportional to the cost of performing a forward uncertainty analysis at each design location. An OUU workflow has two main components: an inner loop strategy for the computation of statistics of the quantity of interest, and an outer loop optimization strategy tasked with finding the optimal design, given a merit function based on the inner loop statistics. Here, in this work, we propose to alleviate the cost of the inner loop uncertainty analysis by leveraging the so-called multilevel Monte Carlo (MLMC) method, which is able to allocate resources over multiple models with varying accuracy and cost. The resource allocation problem in MLMC is formulated by minimizing the computational cost given a target variance for the estimator. We consider MLMC estimators for statistics usually employed in OUU workflows and solve the corresponding allocation problem. For the outer loop, we consider a derivative-free optimization strategy implemented in the SNOWPAC library; our novel strategy is implemented and released in the Dakota software toolkit. We discuss several numerical test cases to showcase the features and performance of our approach with respect to its Monte Carlo single fidelity counterpart.
97 MATHEMATICS AND COMPUTING↗
Estimation, UQ, and Sensitivity Analysis for a New Diagnostic Setup [Slides]
This is for an early career talk I am giving at the Association for Women in Math’s Research Symposium. It provides a general overview of three projects: the first focuses on velocity measurements from optical laser diagnostics, the second focuses on probe alignment and location (metrology), and the third is my Site-Directed Research and Development (SDRD) which focuses on neural networks for image datasets. All three projects have strong uncertainty quantification components.
97 MATHEMATICS AND COMPUTING↗
The Sensitivity of Variational Bayesian Neural Network Performance to Hyperparameters
In scientific applications, predictive modeling is often of limited use without accurate uncertainty quantification (UQ) to indicate when a model may be extrapolating or when more data needs to be collected. Bayesian Neural Networks (BNNs) produce predictive uncertainty by propagating uncertainty in neural network (NN) weights and offer the promise of obtaining not only an accurate predictive model but also accurate UQ. However, in practice, obtaining accurate UQ with BNNs is difficult due in part to the approximations used for model training (such as those made in variational inference) and in part to the need to choose a suitable set of hyperparameters; these hyperparameters outnumber those needed for traditional NNs and often have opaque effects on the results. We aim to shed light on the effects of hyperparameter choices for variational BNNs by performing a global sensitivity analysis of variational BNN performance under varying hyperparameter settings. Our results indicate that many of the hyperparameters interact with each other to affect both predictive accuracy and UQ. For improved usage of variational BNNs in real-world applications, we suggest that thorough hyperparameter tuning, including tuning of prior hyperparameters and loss function parameters, is essential for accurate UQ in variational BNNs.
97 MATHEMATICS AND COMPUTING↗
Sequential Design of Experiments for Pilot Testing of Novel Solvent System
The CCSI2 program is supporting a six-month test campaign at the National Carbon Capture Center (NCCC) for evaluation of a novel water-lean solvent. This presentation describes CCSI2’s efforts in process modeling of the solvent system for both coal and natural gas-based flue gas sources and initial uncertainty quantification (UQ) work to estimate parametric uncertainty in key sub-models of interest (e.g., thermodynamics, mass transfer, reaction kinetics). Moreover, perspective is provided on how UQ and sequential design of experiments (SDoE) tools are used to assess the impact of model uncertainty on projected process performance, use this information to optimize data collection during the campaign, and refine process models through data collection. This framework is expected to reduce the overall model uncertainty, and thus risk associated with scale-up as the process moves towards commercialization.
Morgan, Joshua↗
MatLib-1.2: Nuclear Material Properties Library
The U.S. Nuclear Regulatory Commission (NRC) uses the computer code Fuel Analysis under Steady-state and Transients (FAST) to model steady-state and transient fuel behavior to support regulatory decisions. FAST relies on a material properties library (MatLib) that contains the thermal and mechanical properties of the nuclear materials and coolants of interest to support the US commercial nuclear industry. MatLib contains properties for a variety of nuclear fuels, cladding and other structural materials, gases, and coolants. In this document, material property correlations for the materials contained within MatLib are presented and discussed. When available, comparisons are made between the material property correlations and available data. Additionally, uncertainties are quantified on the material proper- ties, which is then used by the NRC to support uncertainty quantification for best-estimate plus uncertainty safety evaluation reviews. This document describes MatLib-1.2, which was updated from MatLib-1.1 to include cleanup of the code for streamlined calls to MatLib from the main FAST source code, and to enable easier readability of the code by developers that maintain and enhance the FAST code. The results of MatLib did not change between MatLib-1.1 and MatLib-1.2; however, the source code changes warranted incrementing the minor numbering of the software version number. The changes may result in faster execution times of FAST-1.2 as compared to FAST-1.1. This document is one of a series of documents on FAST; the other documents detail the models used by FAST as well as its integral assessment to experiments and commercial data.
11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
MatLib-1.0: Nuclear Material Properties Library
The U.S. Nuclear Regulatory Commission (NRC) uses the computer code Fuel Analysis under Steady-state and Transients (FAST) to model steady-state and transient fuel behavior to support regulatory decisions. FAST relies on a material properties library (MatLib) that contains the thermal and mechanical properties of the nuclear materials and coolants of interest to support the US commercial nuclear industry. MatLib contains properties for a variety of nuclear fuels, cladding and other structural materials, gases, and coolants. In this document, material property correlations for the materials contained within MatLib are presented and discussed. When available, comparisons are made between the material property correlations and available data. Additionally, uncertainties are quantified on the material properties, which is then used by the NRC to support uncertainty quantification for best-estimate plus uncertainty safety evaluation reviews. This document describes MatLib-1.0, which is the first official version of the MatLib library. This document is one of a series of documents on FAST; the other documents detail the models used by FAST as well as its integral assessment to experiments and commercial data.
11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
MatLib-1.1: Nuclear Material Properties Library: Developed under NQA-1-2017
The U.S. Nuclear Regulatory Commission (NRC) uses the computer code Fuel Analysis under Steady-state and Transients (FAST) to model steady-state and transient fuel behavior to support regulatory decisions. FAST relies on a material properties library (MatLib) that contains the thermal and mechanical properties of the nuclear materials and coolants of interest to support the US commercial nuclear industry. MatLib contains properties for a variety of nuclear fuels, cladding and other structural materials, gases, and coolants. In this document, material property correlations for the materials contained within MatLib are presented and discussed. When available, comparisons are made between the material property correlations and available data. Additionally, uncertainties are quantified on the material properties, which is then used by the NRC to support uncertainty quantification for best-estimate plus uncertainty safety evaluation reviews. This document describes MatLib-1.1, which was updated from MatLib-1.0 to include the addition of Lua functionality and code changes for integration with FAST. The results of MatLib did not change between MatLib-1.0 and MatLib-1.1; however, the source code changes warranted incrementing the minor numbering of the software version number. This document is one of a series of documents on FAST; the other documents detail the models used by FAST as well as its integral assessment to experiments and commercial data.
11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
MatLib-1.2.2: Nuclear Material Properties Library
The U.S. Nuclear Regulatory Commission (NRC) uses the computer code Fuel Analysis under Steady-state and Transients (FAST) to model steady-state and transient fuel behavior to support regulatory decisions. FAST relies on a material properties library (MatLib) that contains the thermal and mechanical properties of the nuclear materials and coolants of interest to support the U.S. commercial nuclear industry. MatLib contains properties for a variety of nuclear fuels, cladding and other structural materials, gases, and coolants. In this document, material property correlations for the materials contained within MatLib are presented and discussed. When available, comparisons are made between the material property correlations and available data. Additionally, uncertainties are quantified on the material properties, which is then used by the NRC to support uncertainty quantification for best-estimate plus uncertainty safety evaluation reviews. This document describes MatLib-1.2.2, updated from MatLib-1.2.1 to include additional properties for metallic fuel. It is one of a series of documents on FAST; the other documents detail the models used by FAST as well as its integral assessment to experiments and commercial data.
11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗