Small increases in agent-based model complexity can result in large increases in required calibration data
Not Available
42 ENGINEERING↗
SEARCH · Engineering Papers
Results for “computer model calibration”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Towards Redefining the Reproducibility in Quantum Computing: A Data Analysis Approach on NISQ Devices
Although the building of quantum computers has kept making rapid progress in recent years, noise is still the main challenge for any application to leverage the power of quantum computing. Existing works addressing noise in quantum devices proposed noise reduction when deploying a quantum algorithm to a specified quantum computer. The reproducibility issue of quantum algorithms has been raised since the noise levels vary on different quantum computers. Importantly, existing works largely ignore the fact that the noise of quantum devices varies as time goes by. Therefore, reproducing the results on the same hardware will even become a problem. We analyze the reproducibility of quantum machine learning (QML) algorithms based on daily model training and execution data collection. Our analysis shows a correlation between our QML models’ test accuracy and quantum computer hardware’s calibration features. We also demonstrate that noisy simulators for quantum computers are not a reliable tool for quantum machine learning applications.
Senapati, Priyabrata↗
Study of Radiative Heat Transfer and Flow Physics from Medium-scale Methanol Pool Fire Simulations [Slides]
Analysis of methanol pool fire conducted as part of validation study for SIERRA/Fuego. Radiation model was effectively calibrated by modifying radiation model parameters for methanol. Computing integrated buoyancy flux, entrainment rate, and turbulent kinetic energy allowed for evaluation of less typical quantities in this validation study. Quantities were compared with experimental data or correlations and generally showed agreement. TKE needed fine mesh to be computed accurately. Predicted flame height less sensitive to variations in mixture fraction than temperature. Mixture fraction is a preferable threshold variable for this application.
37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
Policy-Driven Sustainable Saline Drainage Disposal and Forage Production in the Western San Joaquin Valley of California
Environmental policies to address water quality impairments in the San Joaquin River of California have focused on the reduction of salinity and selenium-contaminated subsurface agricultural drainage loads from westside sources. On 31 December 2019, all of the agricultural drainage from a 44,000 ha subarea on the western side of the San Joaquin River basin was curtailed. This policy requires the on-site disposal of all of the agricultural drainage water in perpetuity, except during flooding events, when emergency drainage to the River is sanctioned. The reuse of this saline agricultural drainage water to irrigate forage crops, such as ‘Jose’ tall wheatgrass and alfalfa, in a 2428 ha reuse facility provides an economic return on this pollutant disposal option. Irrigation with brackish water requires careful management to prevent salt accumulation in the crop root zone, which can impact forage yields. The objective of this study was to optimize the sustainability of this reuse facility by maximizing the evaporation potential while achieving cost recovery. This was achieved by assessing the spatial and temporal distribution of the root zone salinity in selected fields of ‘Jose’ tall wheatgrass and alfalfa in the drainage reuse facility, some of which have been irrigated with brackish subsurface drainage water for over fifteen years. Electromagnetic soil surveys using an EM-38 instrument were used to measure the spatial variability of the salinity in the soil profile. The tall wheatgrass fields were irrigated with higher salinity water (1.2–9.3 dS m -1 ) compared to the fields of alfalfa (0.5–6.5 dS m -1 ). Correspondingly, the soil salinity in the tall wheatgrass fields was higher (12.5 dS m -1 –19.3 dS m -1 ) compared to the alfalfa fields (8.97 dS mm -1 –14.4 dS mm -1 ) for the years 2016 and 2017. Better leaching of salts was observed in the fields with a subsurface drainage system installed (13–1 and 13–2). The depth-averaged root zone salinity data sets are being used for the calibration of the transient hydro-salinity computer model CSUID-ID (a one-dimensional version of the Colorado State University Irrigation Drainage Model). This user-friendly decision support tool currently provides a useful framework for the data collection needed to make credible, field-scale salinity budgets. In time, it will provide guidance for appropriate leaching requirements and potential blending decisions for sustainable forage production. This paper shows the tie between environmental drainage policy and the role of local governance in the development of sustainable irrigation practices, and how well-directed collaborative field research can guide future resource management.
54 ENVIRONMENTAL SCIENCES↗
XCal: model-based approach to X-ray CT spectral calibration
Transmission X-ray computed tomography (CT) is widely used to quantitatively reconstruct 3D objects composed of multiple materials. However, accurate CT reconstruction requires the system to be calibrated to account for the effective X-ray spectrum. Unfortunately, measurement of the effective spectrum is ill-posed, and existing calibration methods require that the system be recalibrated when the system parameters are changed. In this paper, we propose XCal, a multi-energy model-based spectral calibration approach for X-ray CT. The XCal approach models the effective spectrum using a separable physics-based model of the CT system. The model parameters are then estimated by fitting calibration data with known objects at multiple energies. An important advantage of XCal is that it allows the user to change scanner settings, such as the source voltage or X-ray filters, without the need for recalibration. Evaluations on simulated and measured datasets demonstrate that XCal significantly improves the accuracy of the estimated spectrum as compared to existing calibration methods.
Li, Wenrui [Purdue Univ., West Lafayette, IN (Unit↗
A New 1D Model for Thermal Mixing and Stratification in Advanced Reactor Transients
Thermal mixing and stratification in large pools and enclosures play a critical role in the safety and performance of pool-type nuclear reactors, particularly during transient scenarios involving significant temperature differences between incoming and bulk coolant. Accurate modeling of these phenomena is essential for predicting system behavior and supporting passive safety features such as natural circulation. Here, this paper presents a new 1D model for thermal mixing and stratification, developed and implemented in the SAM code. The model represents a large pool as 1D coolant jet channels and zero-dimensional bulk pool volumes, enabling the simulation of a wide range of flow configurations, including hot and cold jet interactions, stratified layers, and the influence of complex geometries such as ceilings, free surfaces, and internal obstacles. Heat exchange between jet and pool regions is governed by closure relations calibrated against 3D computational fluid dynamics (CFD) simulations. The model improves upon earlier approaches by incorporating time-dependent jet characteristics and capturing the associated delay effects more accurately. Code-to-code comparisons and validation against experimental data from the Thermal Stratification Test Facility demonstrate the model’s accuracy and flexibility. This work offers two key contributions: (1) an efficient and robust method for simulating thermal mixing and stratification at the system level, eliminating the need for external coupling between system analysis codes and CFD, and (2) a significant enhancement of SAM’s capabilities to analyze thermal stratification phenomena in advanced reactor systems.
SAM↗
Calibration and Uncertainty Estimation Using the Ensemble Kalman Filter with a Large Subsurface Flow and Transport Model - 20321
At routinely monitored groundwater contamination sites, periodically measured environmental conditions such as groundwater levels and contaminant concentrations are used to inform and confirm a conceptual site model (CSM) and guide the development and calibration of a numerical groundwater flow and transport model. The calibration of groundwater flow and transport models after each measurement (sampling) event can illuminate deficiencies in a CSM, identify areas where additional monitoring is warranted, and predict the behavior of the system to guide decision making. However, manual and automated (e.g. PEST) model calibration tools can be time-consuming and computationally expensive to implement after each sampling event. Perhaps as a result, such calibration tools generally utilize all available monitoring data simultaneously rather than sequentially assimilating monitoring data one sampling event at a time as the results from sampling become available. A more real-time data assimilation approach may reduce parameter uncertainty, quantify the value of additional monitoring data, and produce a usable model more quickly and with less effort. To mitigate the potential time-consuming aspects of manual and widely applied automated calibration techniques, a data assimilation algorithm called the ensemble Kalman filter (EnKF) was evaluated as a relatively efficient method of model calibration and uncertainty assessment via the sequential integration of monitoring data into a model. The EnKF was able to successfully and efficiently assimilate monitoring and modeling data to calibrate a complex flow and transport model at a real-world site with significant subsurface heterogeneity, uncertainty, and 12 years of monitoring data (over 4,000 individual measurements of groundwater levels and over 2,500 measurements of contaminant concentrations). Starting with an uncalibrated model data from annual sampling events were sequentially assimilated, and the resultant predication errors and estimated parameter uncertainties were tracked. After all monitoring data were assimilated, both flow and transport residuals at the end of the EnKF process were comparable to those produced via a concurrent PEST calibration effort but required fewer model simulations. Both uncertainty and prediction errors decreased over time. In a real-time application, the adequacy of the model could be assessed after each sampling event. The benefits of such a real-time approach to utilizing monitoring data include reduced costs (in the form of model updates or site characterization efforts), early flagging of possible errors in the CSM, and a reduced risk of overfitting and corresponding increased confidence in model predictions. This tool may be particularly useful compared to other calibration techniques (e.g. manual, PEST) when model runtimes are long, calibration parameters are many, or parameter uncertainty is large. (authors)
12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
Extension of SCALE/Sampler’s sensitivity index for the assessment of cross section, fission yield, and decay data uncertainties
Accurate prediction of nuclide compositions and neutron and gamma sources and spectra for fresh and spent nuclear fuel through computational modeling and simulations is the basis for safeguards instruments design, optimization, and calibration, and for validation of the measured responses. The accuracy of these simulations can be significantly impacted by uncertainties in input parameters to the applied computer code. Input parameters are, for example, dimensions, material compositions, and temperatures of the model. Additionally, important but sometimes neglected input parameters for simulations of nuclear systems are the nuclear data that include nuclear reaction cross sections, fission product yields, and decay data. It has been shown that uncertainties of calculated results are dominated by uncertainties of these nuclear data.
11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
Monte Carlo control loops for cosmic shear cosmology with DES Year 1 data
Weak lensing by large-scale structure is a powerful probe of cosmology and of the dark universe. This cosmic shear technique relies on the accurate measurement of the shapes and redshifts of background galaxies and requires precise control of systematic errors. Monte Carlo control loops (MCCL) is a forward modeling method designed to tackle this problem. It relies on the ultra fast image generator (UFig) to produce simulated images tuned to match the target data statistically, followed by calibrations and tolerance loops. Here, we present the first end-to-end application of this method, on the Dark Energy Survey (DES) Year 1 wide field imaging data. We simultaneously measure the shear power spectrum $C_ℓ$ and the redshift distribution $n(z)$ of the background galaxy sample. The method includes maps of the systematic sources, point spread function (PSF), an approximate Bayesian computation (ABC) inference of the simulation model parameters, a shear calibration scheme, and a fast method to estimate the covariance matrix. We find a close statistical agreement between the simulations and the DES Y1 data using an array of diagnostics. In a nontomographic setting, we derive a set of $C_ℓ$ and $n(z)$ curves that encode the cosmic shear measurement, as well as the systematic uncertainty. Following a blinding scheme, we measure the combination of $Ω_m$, $σ_8$, and intrinsic alignment amplitude $A_{IA}$, defined as $S_8D_{IA}=σ_8(Ω_m/0.3)^{0.5}D_{IA}$, where $D_{IA}=1-0.11(A_{IA}-1)$. We find $S_8D_{IA}=0.8954_{-0.039}^{+0.054}$, where systematics are at the level of roughly 60% of the statistical errors. We discuss these results in the context of earlier cosmic shear analyses of the DES Y1 data. Our findings indicate that this method and its fast runtime offer good prospects for cosmic shear measurements with future wide-field surveys.
79 ASTRONOMY AND ASTROPHYSICS↗
Formulation and solution approach for calibrating activity-based travel demand model-system via microsimulation
This study addresses the problem of calibrating utility-maximizing nested logit activity-based travel demand model-systems. After estimation, it is common practice to use aggregate measurements to calibrate the estimated model-system’s parameters prior to their application in transportation planning, policy making, and operations. However, calibration of activity-based model-systems has received much less attention. Existing calibration approaches are myopic heuristics in the sense that they do not consider the fundamental inter-dependencies among choice-models and do not have a systematic way to adjust model parameters. Also, other purely simulation-based approaches do not perform well in large-scale applications. In this study, we focus on utility-maximizing nested logit activity-based model-systems and calibrating aggregate statistics such as activity shares, mode shares, time-dependent & mode-specific OD flows, and time-dependent & mode-specific sensor counts. We formulate the calibration problem as a simulation-based optimization problem and propose a stochastic gradient-based solution procedure to solve it. The solution procedure relies on microsimulation to calculate expectations of the aggregate statistics of interest to the calibration problem. Additionally, we derive approximate analytical expressions for the gradient of the objective function —that are evaluated through microsimulation on mini-batches of the population. The proposed solution procedure is sensitive to the fundamental structure of the activity-based model-system and is non-myopic in considering the dependencies across its model components. The formulated optimization problem is non-convex, highly nonlinear, and potentially has multiple-minima. Lastly, we show —through a real-world application— that the proposed solution procedure outperforms other state-of-the-art purely simulation-based optimization approaches in terms of computational efficiency, stability, and convergence. We also compare various gradient-based solution algorithms to determine the best algorithm to update the parameters. This work has the potential to facilitate wider and easier application of activity-based model-systems.
97 MATHEMATICS AND COMPUTING↗
An initial exploration of Bayesian model calibration for estimating the composition of rocks and soils on Mars
The Mars Curiosity rover carries an instrument, ChemCam, designed to measure the composition of surface rocks and soil using laser-induced breakdown spectroscopy (LIBS). The measured spectra from this instrument must be analyzed to identify the component elements in the target sample, as well as their relative proportions. This process, which we call disaggregation, is complicated by so-called matrix effects, which describe nonlinear changes in the relative heights of emission lines as an unknown function of composition due to atomic interactions within the LIBS plasma. In this work, we explore the use of the plasma physics code ATOMIC, developed at Los Alamos National Laboratory, for the disaggregation task. ATOMIC has recently been used to model LIBS spectra and can robustly reproduce matrix effects from first principles. The ability of ATOMIC to predict LIBS spectra presents an exciting opportunity to perform disaggregation in a manner not yet tried in the LIBS community, namely via Bayesian model calibration. However, using it directly to solve our inverse problem is computationally intractable due to the large parameter space and the computation time required to produce a single output. Therefore, we also explore the use of emulators as a fast solution for this analysis. We discuss a proof of concept Gaussian process emulator for disaggregating two-element compounds of sodium and copper. The training and test datasets were simulated with ATOMIC using a Latin hypercube design. After testing the performance of the emulator, we successfully recover the composition of 25 test spectra with Bayesian model calibration.
97 MATHEMATICS AND COMPUTING↗
Neural Posterior Estimation for Scalable and Accurate Inverse Parameter Inference in Li-Ion Batteries
Diagnosing the internal state of Li-ion batteries is critical for battery research, operation of real-world systems, and prognostic evaluation of remaining lifetime. By using physics-based models to perform probabilistic parameter estimation via Bayesian calibration, diagnostics can account for the uncertainty due to model fitness, data noise, and the observability of any given parameter. However, Bayesian calibration in Li-ion batteries using electrochemical data is computationally intensive even when using a fast surrogate in place of physics-based models, requiring many thousands of model evaluations. A fully amortized alternative is neural posterior estimation (NPE). NPE shifts the computational burden from the parameter estimation step to data generation and model training, reducing the parameter estimation time from minutes to milliseconds, enabling real-time applications. The present work shows that NPE can infer parameters equally or more accurately than Bayesian calibration, even if it leads to higher voltage reconstruction errors. We also demonstrate that the higher computational costs for data generation are tractable even in high-dimensional cases (ranging from 6 to 27 estimated parameters). The NPE method also offers several interpretability advantages over Bayesian calibration, such as local parameter sensitivity to specific regions of the voltage curve. The NPE method is demonstrated using an experimental fast charge dataset, with parameter estimates validated against measurements of loss of lithium inventory and loss of active material. The implementation is made available in a companion repository (https://github.com/NatLabRockies/BatFIT).
25 ENERGY STORAGE↗
Emulator-Based Bayesian Calibration of the CISNET Colorectal Cancer Models
Purpose To calibrate Cancer Intervention and Surveillance Modeling Network (CISNET)'s SimCRC, MISCAN-Colon, and CRC-SPIN simulation models of the natural history colorectal cancer (CRC) with an emulator-based Bayesian algorithm and internally validate the model-predicted outcomes to calibration targets.Methods We used Latin hypercube sampling to sample up to 50,000 parameter sets for each CISNET-CRC model and generated the corresponding outputs. We trained multilayer perceptron artificial neural networks (ANNs) as emulators using the input and output samples for each CISNET-CRC model. We selected ANN structures with corresponding hyperparameters (i.e., number of hidden layers, nodes, activation functions, epochs, and optimizer) that minimize the predicted mean square error on the validation sample. We implemented the ANN emulators in a probabilistic programming language and calibrated the input parameters with Hamiltonian Monte Carlo-based algorithms to obtain the joint posterior distributions of the CISNET-CRC models' parameters. We internally validated each calibrated emulator by comparing the model-predicted posterior outputs against the calibration targets.Results The optimal ANN for SimCRC had 4 hidden layers and 360 hidden nodes, MISCAN-Colon had 4 hidden layers and 114 hidden nodes, and CRC-SPIN had 1 hidden layer and 140 hidden nodes. The total time for training and calibrating the emulators was 7.3, 4.0, and 0.66 h for SimCRC, MISCAN-Colon, and CRC-SPIN, respectively. The mean of the model-predicted outputs fell within the 95% confidence intervals of the calibration targets in 98 of 110 for SimCRC, 65 of 93 for MISCAN, and 31 of 41 targets for CRC-SPIN.Conclusions Using ANN emulators is a practical solution to reduce the computational burden and complexity for Bayesian calibration of individual-level simulation models used for policy analysis, such as the CISNET CRC models. In this work, we present a step-by-step guide to constructing emulators for calibrating 3 realistic CRC individual-level models using a Bayesian approach.
artificial neural networks↗
AWSD Reactive Burn Model for High Explosive LX‐14
ABSTRACT The results of an Arrhenius–Wescott–Stewart–Davis (AWSD) reactive flow calibration for the HMX‐based high explosive LX‐14 are presented. The parameters in the AWSD model are calibrated to experimental thermodynamic and gas gun data and to computational results from thermochemical calculations. There is no experimental rate stick data available for LX‐14; therefore, scaled experimental results from other PBX‐based high explosives are used in the calibration to fill this gap in data. Strong agreement is observed between the calibrated AWSD model and experimental data for LX‐14, including validation data that were not used in the calibration procedure. The developed model more accurately describes experimental shock‐to‐detonation results compared to several other reactive flow models for LX‐14 from the literature. The presented results illustrate that the AWSD model is capable of quantitatively describing the reactive burn of LX‐14.
45 MILITARY TECHNOLOGY, WEAPONRY, AND NATIONAL DEF↗
A Bayesian Calibration Framework with Embedded Model Error for Model Diagnostics
We study the utility and performance of a Bayesian model error embedding construction in the context of molecular dynamics modeling of metallic alloys, where we embed model error terms in existing interatomic potential model parameters. To alleviate the computational burden of this approach, we propose a framework combining likelihood approximation and Gaussian process surrogates. Here we leverage sparse Gaussian process techniques to construct a hierarchy of increasingly accurate but more expensive surrogate models. This hierarchy is then exploited by multilevel Markov chain Monte Carlo methods to efficiently sample from the target posterior distribution. We illustrate the utility of this approach by calibrating an interatomic potential model for a family of gold-copper alloys. In particular, this case study highlights effective means for dealing with computational challenges with Bayesian model error embedding in large-scale physical models, and the utility of embedded model error for model diagnostics.
Bayesian inference↗
Bayesian model updating with finite element vs surrogate models: Application to a miter gate structural system
Bayesian finite element (FE) model updating using direct model evaluations of large-scale high-fidelity FE models is extremely computationally expensive. Surrogate models can be used as fast emulators of FE models to accelerate the model calibration process. The physics/mechanics-based FE models are still the underpinning behind the surrogate models. Here, this paper evaluates the loss in accuracy and the gain in computational time while performing Bayesian model updating by using surrogate model evaluations compared to using direct FE model evaluations. This evaluation is crucial before entirely relying on surrogate models in model updating for structural health monitoring (SHM) and damage prognosis (DP) purposes. This paper also demonstrates Bayesian updating and surrogate model construction of large-scale high-fidelity FE models of infrastructure systems. In this regard, the miter gate structural system is considered as the testbed structure. Three predominant damage modes (loss of contact between gate and wall, loss of thickness due to corrosion, and loss of tension in the diagonal rods) are considered for model updating purposes. Bayesian model updating is performed using direct FE evaluations by leveraging parallel computing. Two types of surrogates, namely polynomial chaos expansion (PCE) and Gaussian process regression (GPR), are developed for the miter gate. Model updating is performed again using the trained surrogate models, and the updating results are compared with their counterparts obtained using the direct FE evaluation results. The posterior distribution of the FE model parameters obtained using the trained surrogates are sufficiently accurate with respect to the posterior obtained utilizing the direct FE evaluations. In addition, an approximate 4-fold decrease in the computational time was observed when using surrogate model evaluations instead of direct FE evaluations for model updating.
42 ENGINEERING↗
Enabling Predictive Scale-Bridging Simulations through Active Learning (Institutional Computing Annual Report (Project w21_alscalebridging)) [Slides]
The goal of this project was to develop, demonstrate, and provide a new capability to achieve greater physical fidelity in large-scale simulations, rather than the usual brute-force increases in the number of mesh elements or particles. Transport in nanoporous media, critical to hydrocarbon extraction from tight shale formations, is affected by molecular-level interactions. Several coarse-scale Lattice Boltzmann model (LBM) parameters cannot be directly computed, so instead we calibrate them to molecular dynamics (MD) simulations, by building emulators that mimic MD and LBM behavior and then training an upscaler. The resulting machine learning (ML)-based scale-bridging framework is up to 7 orders of magnitude faster than direct MD.
74 ATOMIC AND MOLECULAR PHYSICS↗
Predicting Geologic Behavior in Carbon Storage Projects Using Graph Neural Network
This study was invited to presented at NVIDIA's GTC conference to highlight the potential of Graph Neural Network as a novel and promising methodology for predicting pressure and saturation evolution in carbon storage projects. Carbon capture and storage (CCS) technology plays a pivotal role in mitigating greenhouse gas emissions, facilitating the transition to a low-carbon future. Effective management of subsurface reservoirs is essential to ensure the safe and efficient storage of captured carbon dioxide (CO₂). Accurate predictions of pressure and saturation over time are critical for evaluating the long-term performance and integrity of CCS projects. In recent years, Graph Neural Network (GNN) has emerged as a powerful framework for analyzing complex data in graph-structured domains. This abstract explores the application of GNN to forecast pressure and saturation evolution in carbon storage projects. Traditional numerical simulations of subsurface reservoirs have proven successful in providing pressure and saturation forecasts. However, these simulations involve massive amounts of computational effort and require extensive domain expertise for proper model calibration and validation. Graph Neural Operator offers an alternative approach that harnesses the inherent graph structure of reservoirs, where nodes represent reservoir grid cells and edges represent the geological connectivity between them.
Shih, Chung Yan↗