Calibration Propagation and Validation of Model Discrepancy Across Experimental Settings.
Abstract not provided.
Maupin, Kathryn Anne↗
SEARCH · Engineering Papers
Results for “model validation and calibration”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
1988 Delaware Valley Regional Planning Commission Household Travel Survey
The Delaware Valley Regional Planning Commission conducted household travel surveys in 1988 for the Delaware Valley region of Pennsylvania, which consists of Bucks, Chester, Delaware, Montgomery, and Philadelphia counties. The home interview travel survey was intended to measure the travel habits of area residents, and it generated needed information on trip generation rates, trip distribution, and model choice. This information was used for model calibration and validation on the trip generation, trip distribution, and model split models. The data includes demographic and socioeconomic details, as well as travel patterns, for 1,567 households. A total of 5,733 people reported a total of 18,899 trips. The survey was conducted on a weekday, Monday through Friday, and only ten survey locations contained weekend day travel.
1Hz data↗
Model Gap Quantification and Evaluation
In the real world, model gaps always exist because models cannot perfectly match the objective physical plants. Model gaps reflect the integrity of models, which is vital for model validation and calibration. This letter proposes a novel approach for model gap quantification and evaluation. First, a comprehensive metric is developed to quantify dynamic model gaps. Next, the model gaps are evaluated from a panoramic view of the probability distribution of the comprehensive metric for multiple scenarios. Finally, we demonstrate the proposed quantification and evaluation approach through synchronous machine models.
Wang, Shaobu↗
Modeling the Impacts of Hydrogen Extraction on Anaerobic Fermentation Systems Using a Modified ADM1 Model
Here, this paper presents a modified ADM1 (mADM1) model for use in assessing the influence of active dissolved H 2 extraction on biogas production from anaerobic fermentation reactors, specifically in the context of high-strength brewery wastewater as the feed. The modified model adds regulation factors to account for the varying product stoichiometry of glucose degradation reactions as a function of H 2 concentration in anaerobic digestion, and incorporates the use of both lactate and ethanol as intermediates. The changes to the model are expected to better predict the effects of H 2 removal on the production of H 2 in fermentation processes. Both ADM1 and mADM1 were calibrated against experimental data from a fluidized-bed reactor containing encapsulated acidogenic-acetogenic microbial communities treating real brewery wastewater. The calibrated models were validated against another fluidized-bed reactor identical to the calibration reactor, with the addition of a hollow-fiber membrane module that was used to reduce dissolved H 2 concentrations in the validation reactor. It was found that ADM1 over-estimated the H 2 production rates of the validation reactor (117 mL/day simulated vs. 16.48 mL/day experimental), whereas mADM1 was able to successfully predict fermentation, and specifically the H 2 production rate (17 mL/day). The presented model is a first step towards a model that can predict the effects of dissolved H 2 removal on fermentation processes.
Anaerobic Digestion↗
A Comprehensive Review and Qualitative Analysis of Micro-Combined Heat and Power Modeling Approaches
Concurrent production of electrical and thermal energy from a Combined Heat and Power (CHP) device is an attractive tool to address the growing energy needs of the planet. Micro CHP (µCHP) systems can reduce a building’s primary energy consumption, reduce carbon footprint, and enhance resiliency. Modeling of the µCHP helps understand the system from multiple perspectives and helps discover errors earlier, improves impact analysis and simulation of system solutions for ease of integration with the building. Consequently, there is a need for analysis of the impact of µCHP modeling approach on its reliability and flexibility. The primary objective of this paper is to review the state-of-the art models in the µCHP space with a focus towards internal combustion engine as the primary mover (PM) and limit the study to system modeling, calibration, and validation methodologies. Based on the analysis, recommendations for further model considerations and refinements are presented.
42 ENGINEERING↗
Geothermal Heat Pump System Showcase: Short-Term Validation of Borehole Heat Exchanger Performance from Field Data to Numerical Modeling: Preprint
Since 2011, a geothermal heat pump (GHP) system has been operating to provide space heating and cooling for the Solar Radiation and Research Laboratory building at the National Laboratory of the Rockies (NLR) in Golden, Colorado. The system consists of 23 vertical boreholes, each extending to a depth of 300 ft (91 m), connected to 11 water-to-air heat pump units and four circulation pumps. Between fiscal years 2023 and 2025, additional power meters and temperature sensors were retrofitted to support detailed system performance assessment and model development. This study presents preliminary monitoring results and the development of an initial numerical model of the borehole heat exchanger field. The model incorporated site-specific geometry, ground thermal properties derived from thermal response tests, and ambient temperatures, and simulated system behavior over a representative operating day in September. Model predictions of outlet temperatures were compared against corresponding field measurements. Results showed that modeling initialized with a simplified linear subsurface temperature gradient presents systematic discrepancies in outlet temperature, whereas incorporating depth-resolved borehole temperature measurements for initialization yields substantially improved agreement with observations. The findings highlight the sensitivity of short-term predictive modeling to the representation of initial subsurface thermal conditions and underscore the value of high-resolution field measurements for model calibration and validation. These preliminary results inform ongoing efforts to extend the modeling framework to longer time horizons and to refine monitoring and modeling strategies that support the design guidance and operational optimization of GHP systems in research and commercial buildings.
15 GEOTHERMAL ENERGY↗
Monitoring Noble Gases (Xe and Kr) and Aerosols (Cs and Rb) in a Molten Salt Reactor Surrogate Off-Gas Stream Using Laser-Induced Breakdown Spectroscopy (LIBS)
In this study with surrogate materials we show that laser-induced breakdown spectroscopy (LIBS) is a robust tool with promising capability toward monitoring gaseous (Xe and Kr) and aerosol (Cs and Rb) species in an off-gas stream from a molten salt reactor (MSR). MSRs will continually evolve fission products into the cover gas flowing across the reactor headspace. The cover gas entrains Xe and Kr gases, along with aerosol particles, before passing into an off-gas treatment system. Univariate models of Xe and Kr peaks showed a strong correlation to concentration indicated by their coefficients of determination of 0.983 and 0.997, respectively. Multivariate models were built for all four analytes using partial least squares regression coupled with preprocessing steps including normalization, trimming, and/or genetic algorithm derived filters. The models were evaluated by predicting the concentrations of the analytes in four validation samples, in which all calibration models were successfully validated at a confidence interval of 99.9%. Finally, pressure controllers were used to regulate the mass flow rate of Kr flowing into the measurement cell in sinusoidal and stepwise waveforms to test the real-time monitoring capabilities of the regression models. Both univariate and partial least squares Kr models were able to successfully quantify the gas concentration in the real-time evaluation. The root mean squared error of prediction (RMSEP) values for these real-time tests were calculated to be 0.051, 0.060, and 0.121 mol% demonstrating the measurement systems’ capability to perform online monitoring with acceptable accuracy.
22 GENERAL STUDIES OF NUCLEAR REACTORS↗
In-Situ Calibrated Modeling of Residual Stresses Induced in Machining under Various Cooling and Lubricating Environments
Although many functional characteristics, such as fatigue life and damage resistance depend on residual stresses, there are currently no industrially viable ‘Digital Process Twin’ models (DPTs) capable of efficiently and quickly predicting machining-induced stresses. By leveraging advances in ultra-high-speed in-situ experimental characterization of machining and finishing processes under plane strain (orthogonal/2D) conditions, we have developed a set of physics-based semi-analytical models to predict residual stress evolution in light of the extreme gradients of stress, strain and temperature, which are unique to these thermo-mechanical processes. Initial validation trials of this novel paradigm were carried out in Ti-6Al4V and AISI 4340 alloy steel. A variety dry, cryogenically cooled and oil lubricated conditions were evaluated to determine the model’s ability to capture the tribological changes induced due to lubrication and cooling. The preliminarily calibrated and validated model exhibited an average correlation of better than 20% between the predicted stresses and experimental data, with calculation times of less than a second. Based on such fast-acting DPTs, the authors envision future capabilities in pro-active surface engineering of advanced structural components (e.g., turbine blades).
36 MATERIALS SCIENCE↗
Operationalizing Synchrophasors for Enhanced Grid Reliability and Asset Utilization
The report discusses the development of production-grade synchrophasor software applications. The objective is to improve system reliability and asset utilization for offline as well as real-time considerations, as per the Department of Energy (DOE) guidelines specified by Funding Opportunity Announcement (FOA) DE-FOA-0001492. On the one hand, we have developed grid management applications on how to transform synchrophasor-based monitoring into operator guidance, and commercial grade tools addressing industry needs such as NERC Reliability Standards (MOD 26, MOD 27 and MOD 33). On the other hand, with the wide industrial adoption in mind, the developed applications have been fully discussed with utility partners and installed in the utility partners site including 6 field tests. The Model Validation and Calibration software module has been integrated into GE’s PhasorAnalyticsTM. The first commercial contract has been signed even before the close date of this project. The Angle-based Grid Management (AGM), Fast Voltage Security Assessment (FastVSA), Enhanced Islanding Monitoring(EIM) and Enhanced Disturbance Management (EDM) have been integrated into GE’s Grid Stability Assessment line of applications in e-terra Suite of EMS product.
24 POWER TRANSMISSION AND DISTRIBUTION↗
Grid Modeling and Hosting Capacity Analysis
An interface with the OpenDSS distribution grid simulator, facilitating users in calibrating and validating models. Additionally, three distinct tools have been developed: PV Hosting Capacity: This tool allows users to determine the additional amount of photovoltaic (PV) generation that can be integrated into the system without breaching operational constraints. EV Hosting Capacity: This tool focuses on identifying the capacity for incorporating extra electric vehicle (EV) load into the system without surpassing operational limitations. Project Impact Analysis Tool: This tool is designed to assess and report all potential grid violations associated with a specific project, providing valuable insights into its impact on the distribution grid.
Poudel, Shiva↗
VALIDATION, VERIFICATION, AND CALIBRATION THROUGH A CAUSAL LENS
This paper presents an alternative method based on causal inference to perform validation, verification, and calibration of simulation models. While classical validation and verification approaches focus on the identification of the associations between data elements using statistical and machine learning methods, the novel methods in this paper focus instead on the identification of causal relationships between data elements. Statistical and machine learning-based approaches are strictly data-driven, meaning that they provide quantitative comparison measures between datasets without explicitly considering the hypotheses behind them. This can lead to the erroneous conclusion that, if two data sets are close enough, then the models that generated them are similar. In addition, when experimental and simulated data differ to an extent that fails to meet the acceptance criteria, calibration techniques are used to tweak simulation model parameters to reduce the gap between the two types of data. This produces the false expectation that a simulation model will match reality. The methods presented in this paper move away from these strictly data-driven methods for validation and calibration toward more robust, model-driven methods based on causal inference. Causal inference aims to identify the possible mechanisms that might have generated data. Thus, this analysis targets the prediction of the effects when one (or more) of the identified mechanisms are altered. There are many approaches to identify, quantify and illustrate causal relationships. For the scope of this paper, directed graphs are employed as causal models. If the directed graph lacks cycles it is known as a directed acyclic graph (DAG). A node in such a graph represents an observed data element while a directed edge connecting two nodes represents a causal relationship between two variables. The developed causal methods are designed to extract causal models from simulation models and from experimental data. Causal models capture the causal relationships between data elements (e.g., simulated and experimental data). In this context, validation and verification are performed by comparing causal models. The proposed approach does not only inform system analysts on how a simulation model matches real-world data, but also identifies elements of the simulation model that should be revised when discrepancies between simulation and experimental data are observed. Through these causal methods, analysts have a means to identify the portion of the model equation(s) that are behind an edge connecting two variables. Hence, once the structural differences between causal models have been determined, model calibration can occur by changing only those model parameters that impact the identified causal relationships.
97 MATHEMATICS AND COMPUTING↗
A Process-based Model with Temperature, Water, and Lab-derived Data Improves Predictions of Daily Culex pipiens/restuans Mosquito Density
While the number of human cases of mosquito-borne diseases has increased in North America in the last decade, accurate modeling of mosquito population density has remained a challenge. Longitudinal mosquito trap data over the many years needed for model calibration, and validation is relatively rare. In particular, capturing the relative changes in mosquito abundance across seasons is necessary for predicting the risk of disease spread as it varies from year to year. We developed a discrete, semi-stochastic, mechanistic process-based mosquito population model that captures life-cycle egg, larva, pupa, adult stages, and diapause for Culex pipiens (Diptera, Culicidae) and Culex restuans (Diptera, Culicidae) mosquito populations. This model combines known models for development and survival into a fully connected age-structured model that can reproduce mosquito population dynamics. Mosquito development through these stages is a function of time, temperature, daylight hours, and aquatic habitat availability. The time-dependent parameters are informed by both laboratory studies and mosquito trap data from the Greater Toronto Area. The model incorporates city-wide water-body gauge and precipitation data as a proxy for aquatic habitat. This approach accounts for the nonlinear interaction of temperature and aquatic habitat variability on the mosquito life stages. We demonstrate that the full model predicts the yearly variations in mosquito populations better than a statistical model using the same data sources. This improvement in modeling mosquito abundance can help guide interventions for reducing mosquito abundance in mitigating mosquito-borne diseases like West Nile virus.
59 BASIC BIOLOGICAL SCIENCES↗
Bayesian optimal experimental design for constitutive model calibration
Computational simulation is increasingly relied upon for high/consequence engineering decisions, which necessitates a high confidence in the calibration of and predictions from complex material models. However, the calibration and validation of material models is often a discrete, multi-stage process that is decoupled from material characterization activities, which means the data collected does not always align with the data that is needed. To address this issue, an integrated workflow for delivering an enhanced characterization and calibration procedure—Interlaced Characterization and Calibration (ICC)—is introduced and demonstrated. Further, this framework leverages Bayesian optimal experimental design (BOED), which creates a line of communication between model calibration needs and data collection capabilities in order to optimize the information content gathered from the experiments for model calibration. Eventually, the ICC framework will be used in quasi real-time to actively control experiments of complex specimens for the calibration of a high-fidelity material model. This work presents the critical first piece of algorithm development and a demonstration in determining the optimal load path of a cruciform specimen with simulated data. Calibration results, obtained via Bayesian inference, from the integrated ICC approach are compared to calibrations performed by choosing the load path a priori based on human intuition, as is traditionally done. The calibration results are communicated through parameter uncertainties which are propagated to the model output space (i.e. stress–strain). In these exemplar problems, data generated within the ICC framework resulted in calibrated model parameters with reduced measures of uncertainty compared to the traditional approaches.
42 ENGINEERING↗
Cutting out the middleman: calibrating and validating a dynamic vegetation model (ED2-PROSPECT5) using remotely sensed surface reflectance
Canopy radiative transfer is the primary mechanism by which models relate vegetation composition and state to the surface energy balance, which is important to light- and temperature-sensitive plant processes as well as understanding land–atmosphere feedbacks. In addition, certain parameters (e.g., specific leaf area, SLA) that have an outsized influence on vegetation model behavior can be constrained by observations of shortwave reflectance, thus reducing model predictive uncertainty. Importantly, calibrating against radiative transfer outputs allows models to directly use remote sensing reflectance products without relying on highly derived products (such as MODIS leaf area index) whose assumptions may be incompatible with the target vegetation model and whose uncertainties are usually not well quantified. Here, we created the EDR model by coupling the two-stream representation of canopy radiative transfer in the Ecosystem Demography model version 2 (ED2) with a leaf radiative transfer model (PROSPECT-5) and a simple soil reflectance model to predict full-range, high-spectral-resolution surface reflectance that is dependent on the underlying ED2 model state. We then calibrated this model against estimates of hemispherical reflectance (corrected for directional effects) from the NASA Airborne Visible/Infrared Imaging Spectrometer (AVIRIS) and survey data from 54 temperate forest plots in the northeastern United States. The calibration significantly reduced uncertainty in model parameters related to leaf biochemistry and morphology and canopy structure for five plant functional types.
54 ENVIRONMENTAL SCIENCES↗
Solidification and crystallographic texture modeling of laser powder bed fusion Ti-6Al-4V using finite difference-monte carlo method
Laser powder bed fusion (LPBF) additive manufacturing makes near-net-shaped parts with reduced material cost and time, rising as a promising technology to fabricate Ti-6Al-4V, a widely used titanium alloy in aerospace and medical industries. However, LPBF Ti-6Al-4V parts produced with 67° rotation between layers, a scan strategy commonly used to reduce microstructure and property inhomogeneity, have varying grain morphologies and weak crystallographic textures that change depending on processing parameters. Here, this study predicts LPBF Ti-6Al-4V solidification at three energy levels using a finite difference-Monte Carlo method and validates the simulations with large-area electron backscatter diffraction (EBSD) scans. The developed model accurately shows that a <001> texture forms at low energy and a <111> texture occurs at higher energies parallel to the build direction but with a lower strength than the textures observed from EBSD. A validated and well-established method of combining spatial correlation and general spherical harmonics representation of texture is developed to calculate a difference score between simulations and experiments. The quantitative comparison enables effective fine-tuning of nucleation density (N 0 ) input, which shows a nonlinear relationship with increasing energy level. Future improvements in texture prediction code and a more comprehensive study of N 0 with different energy levels will further advance the optimization of LPBF Ti-6Al-4V components. These developments contribute a novel understanding of crystallographic texture formation in LPBF Ti-6Al-4V, the development of robust model validation and calibration pipeline methodologies, and provide a platform for mechanical property prediction and process parameter optimization.
70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
A scalable wave resource assessment methodology: Application to U.S. waters
Waves deliver large quantities of energy to populated coastlines around the world, and wave energy technology research and development has accelerated over the last two decades. Throughout this time national and regional resource assessments have utilized disparate methodologies, which can cause confusion and skepticism. Here, in this work, we describe a theoretical wave resource assessment methodology that addresses many of the major areas of inconsistency and debate. Applying this revised methodology to U.S. waters, we find the theoretical U.S. wave energy resource to be 3300 TWh/yr, with region totals of 2000 TWh/yr in Alaska, 510 TWh/yr along the U.S. west coast, 380 TWh/yr in Hawaii, 290 TWh/yr along the east coast, 69 TWh/yr in the Gulf of Mexico, and 17 TWh/yr in Puerto Rico and the U.S. Virgin Islands. We also find significant uncertainty in these estimates associated with the underlying model dataset, which emphasizes the importance of thorough model validation and calibration as well as quantifying uncertainty.
16 TIDAL AND WAVE POWER↗
AWSD reactive flow model for PBX 9404
An Arrhenius–Wescott–Stewart–Davis (AWSD) reactive flow model for high explosive PBX 9404 is developed. We specifically calibrate an AWSD model for PBX 9404 by fitting equations of state for reactants and detonation products to the results of thermochemical calculations and to experimental data from multiple sources. The calibrated equations of state are then coupled with an Arrhenius rate law based on shock temperature that describes the reaction progress during PBX 9404 detonation. The parameter values in the rate law are calibrated to experimental gas-gun data and diameter effect results. The results of the calibrated AWSD model are in strong agreement with available experimental data for PBX 9404. A similar level of agreement between predicted and experimental results is observed when the calibrated model is validated on data that were not used in the model parameterization procedure. Our results illustrate that the AWSD model is capable of accurately describing the many important properties and observables in the reactive burn of PBX 9404. Because of the historical significance of PBX 9404 in high explosives research and its current use in aging studies, this work provides an important model of a legacy material, which can be used to make comparisons to new high explosive formulations.
45 MILITARY TECHNOLOGY, WEAPONRY, AND NATIONAL DEF↗