Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “error modeling”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

A Bayesian Calibration Framework with Embedded Model Error for Model Diagnostics

We study the utility and performance of a Bayesian model error embedding construction in the context of molecular dynamics modeling of metallic alloys, where we embed model error terms in existing interatomic potential model parameters. To alleviate the computational burden of this approach, we propose a framework combining likelihood approximation and Gaussian process surrogates. Here we leverage sparse Gaussian process techniques to construct a hierarchy of increasingly accurate but more expensive surrogate models. This hierarchy is then exploited by multilevel Markov chain Monte Carlo methods to efficiently sample from the target posterior distribution. We illustrate the utility of this approach by calibrating an interatomic potential model for a family of gold-copper alloys. In particular, this case study highlights effective means for dealing with computational challenges with Bayesian model error embedding in large-scale physical models, and the utility of embedded model error for model diagnostics.

Bayesian inference↗
DOE OSTI2020

Time-series machine-learning error models for approximate solutions to parameterized dynamical systems

This work proposes a machine-learning framework for modeling the error incurred by approximate solutions to parameterized dynamical systems. In particular, we extend the machine-learning error models (MLEM) framework proposed in Ref. Freno and Carlberg (2019) to dynamical systems. The proposed Time-Series Machine-Learning Error Modeling (T-MLEM) method constructs a regression model that maps features – which comprise error indicators that are derived from standard a posteriori error-quantification techniques – to a random variable for the approximate-solution error at each time instance. The proposed framework considers a wide range of candidate features, regression methods, and additive noise models. We consider primarily recursive regression techniques developed for time-series modeling, including both classical time-series models (e.g., autoregressive models) and recurrent neural networks (RNNs), but also analyze standard non-recursive regression techniques (e.g., feed-forward neural networks) for comparative purposes. Finally, numerical experiments conducted on multiple benchmark problems illustrate that the long short-term memory (LSTM) neural network, which is a type of RNN, outperforms other methods and yields substantial improvements in error predictions over traditional approaches.

42 ENGINEERING↗
DOE OSTI2020

Characterizing NWP Model Errors Using Doppler-Lidar Measurements of Recurrent Regional Diurnal Flows: Marine-Air Intrusions into the Columbia-River Basin

Ground-based Doppler-lidar instrumentation provides atmospheric wind data at dramatically improved accuracies and spatial/temporal resolutions. These capabilities have provided new insights into atmospheric flow phenomena, but they also should have a strong role in NWP model improvement. Insight into the nature of model errors can be gained by studying recurrent atmospheric flows, here a regional summertime diurnal sea breeze and subsequent marine-air intrusion into the arid interior of Oregon-Washington, where these winds are an important windenergy resource. These marine intrusions were sampled by three scanning Doppler lidars in the Columbia-River Basin as part of the Second Wind Forecast Improvement Project (WFIP2), using data from summer 2016. Lidar time-height cross sections of wind speed identified eight days when the diurnal-flow cycle, (peak wind speeds at midnight, afternoon minima) was obvious and strong. Eight-day composite time-height cross sections of lidar wind speeds are used to validate those generated by the operational NCEP-HRRR model. HRRR simulated the diurnal wind cycle, but produced errors in the timing of onset and significant errors due to a premature nighttime demise of the intrusion flow, producing low-bias errors of 6 m s-1 . Day-to-day and in the composite, whenever a marine intrusion occurred, HRRR made these same errors. The errors occurred under a range of gradient wind conditions indicating that they resulted from the misrepresentation of physical processes within a limited region around the measurement locations. Because of their generation within a limited geographical area, field-measurement programs can be designed to find and address the sources of these NWP errors.

Banta, Robert M.↗
DOE OSTI2021

Combined selection of the dynamic model and modeling error in nonlinear aeroelastic systems using Bayesian Inference

Here, we report a Bayesian framework for concurrent selection of physics-based models and (modeling) error models. We investigate the use of colored noise to capture the mismatch between the predictions of calibrated models and observational data that cannot be explained by measurement error alone within the context of Bayesian estimation for stochastic ordinary differential equations. Proposed models are characterized by the average data-fit, a measure of how well a model fits the measurements, and the model complexity measured using the Kullback–Leibler divergence. The use of a more complex error models increases the average data-fit but also increases the complexity of the combined model, possibly over-fitting the data. Bayesian model selection is used to find the optimal physical model as well as the optimal error model. The optimal model is defined using the evidence, where the average data-fit is balanced by the complexity of the model. The effect of colored noise process is illustrated using a nonlinear aeroelastic oscillator representing a rigid NACA0012 airfoil undergoing limit cycle oscillations due to complex fluid–structure interactions. Several quasi-steady and unsteady aerodynamic models are proposed with colored noise or white noise for the model error. The use of colored noise improves the predictive capabilities of simpler models.

42 ENGINEERING↗
DOE OSTI2022

Using an Explainable Machine Learning Approach to Characterize Earth System Model Errors: Application of SHAP Analysis to Modeling Lightning Flash Occurrence

Abstract Computational models of the Earth System are critical tools for modern scientific inquiry. Efforts toward evaluating and improving errors in representations of physical and chemical processes in these large computational systems are commonly stymied by highly nonlinear and complex error behavior. Recent work has shown that these errors can be effectively predicted using modern Artificial Intelligence (A.I.) techniques. In this work, we go beyond these previous studies to apply an explainable A.I. technique to not only predict model errors but also move toward understanding the underlying reasons for successful error prediction. We use XGBoost classification trees and SHapley Additive exPlanations analysis to explore the errors in the prediction of lightning occurrence in the NASA Goddard Earth Observing System model, a widely used Earth System Model. This explainable error prediction system can effectively predict the model error and indicates that the errors are strongly related to convective processes and the characteristics of the land surface.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Stochastic modeling and statistical calibration with model error and scarce data

This paper introduces a procedure to assess the predictive accuracy of stochastic models subject to model error and sparse data. Model error is introduced as uncertainty on the coefficients of appropriate polynomial chaos expansions (PCE). The error associated with finite sample size allows us to conceive of these coefficients as statistics of the data that we describe as random variables whose influence on output quantities of interest is evaluated through the extended polynomial chaos expansion (EPCE). A Bayesian data assimilation scheme is introduced to update these expansions by considering the resulting nested chaos expansion as a hierarchical probabilistic model. Stochastic models of quantities of interest (QoI) are thus constructed and efficiently evaluated. Here, the Metropolis–Hastings Markov chain Monte Carlo procedure is used to sample the posterior. Two illustrative analytical and numerical problems are used to demonstrate the proposed approach.

Bayesian inference↗
DOE OSTI2023

Stochastic multiscale modeling for quantifying statistical and model errors with application to composite materials

This paper provides a coherent and efficient computational framework for stochastic multiscale analysis of material systems in the presence of parametric uncertainties and modeling errors. Uncertainty in those model parameters that are not deduced as upscaled quantities is attributed to an uncertainty “germ”. While such parameters can appear at any scale, they are predominant at the finest analysis scale. Additional uncertainties stemming from statistical estimation, attributed to lack of data and model error, are associated with each submodel contributing to the multiscale system. Here, a robust and efficient framework based on a generalized extended polynomial chaos expansion (gEPCE) is proposed to simultaneously propagate all these uncertainties in order to provide a probabilistic representation of specific quantities of interest (QoI). We characterize the full probability distribution of the QoI and the uncertainty in the failure probability pertaining to its tails. By combining gEPCE with kernel density estimation (KDE) and directional derivatives, we construct sensitivity measures that connect these statistical metrics of QoI to the various sources of uncertainty to assess their individual and combined impacts. An illustrative problem featuring three-point bending of a composite beam is investigated to demonstrate the presented approach.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Efficient flexible characterization of quantum processors with nested error models

We present a simple and powerful technique for finding a good error model for a quantum processor. The technique iteratively tests a nested sequence of models against data obtained from the processor, and keeps track of the best-fit model and its wildcard error (a metric of the amount of unmodeled error) at each step. Each best-fit model, along with a quantification of its unmodeled error, constitutes a characterization of the processor. We explain how quantum processor models can be compared with experimental data and to each other. We demonstrate the technique by using it to characterize a simulated noisy two-qubit processor.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2020

A robust dynamic state estimation approach against model errors caused by load changes

Dynamic state estimation (DSE) plays an important role in power system security monitoring and online control. In practice, there are two approaches to implementing DSE. The first approach is distributed DSE, which is based on the assumption that the terminal bus of each generator can be measured by PMUs (phasor measurement units). The assumption cannot be satisfied currently, however, because PMUs usually are installed at important high-voltage buses such as 500-kV buses installed in portions of the grid overseen by the Western Electricity Coordinating Council. Another issue of this approach is that performance of DSE is vulnerable to bad measurement data. The reason for this vulnerability is that DSE is performed separately through measurements at each terminal bus, and measurements at terminal buses are the only measurement upon which DSE can rely. Therefore, important redundant measurements are not included in this approach. The second approach is centralized DSE. This approach does not have the requirement for PMU location, and redundant measurements can be considered fully. However, load changes and grid topology changes impact centralized DSE. In this paper, we propose a new approach for handling the impact of load changes on DSE. We have developed a new algorithm that includes two sequential steps. In the first step, errors caused by load changes are detected by analyzing the difference between prediction results and measured results. In the second step, once model error is detected, a model optimization procedure is run to correct the error so the state estimation error can be mitigated. Simulation results from the IEEE 68 bus system show that the proposed approach can effectively handle model errors caused by load changes.

robust dynamic state estimation, load change, powe↗
DOE OSTI2023

Measurements and Model Improvement: Insight into NWP Model Error Using Doppler Lidar and Other WFIP2 Measurement Systems

Abstract Doppler-lidar wind-profile measurements at three sites were used to evaluate NWP model errors from two versions of NOAA’s 3-km-grid HRRR model, to see whether updates in the latest version 4 reduced errors when compared against the original version 1. Nested (750-m grid) versions of each were also tested to see how grid spacing affected forecast skill. The measurements were part of the field phase of the Second Wind Forecasting Improvement Project (WFIP2), an 18-month deployment into central Oregon–Washington, a major wind-energy-producing region. This study focuses on errors in simulating marine intrusions, a summertime, 600–800-m-deep, regional sea-breeze flow found to generate large errors. HRRR errors proved to be complex and site dependent. The most prominent error resulted from a premature drop in modeled marine-intrusion wind speeds after local midnight, when lidar-measured winds of greater than 8 m s −1 persisted through the next morning. These large negative errors were offset at low levels by positive errors due to excessive mixing, complicating the interpretation of model “improvement,” such that the updates to the full-scale versions produced mixed results, sometimes enhancing but sometimes degrading model skill. Nesting consistently improved model performance, with version 1’s nest producing the smallest errors overall. HRRR’s ability to represent the stages of sea-breeze forcing was evaluated using radiation budget, surface-energy balance, and near-surface temperature measurements available during WFIP2. The significant site-to-site differences in model error and the complex nature of these errors mean that field-measurement campaigns having dense arrays of profiling sensors are necessary to properly diagnose and characterize model errors, as part of a systematic approach to NWP model improvement. Significance Statement Dramatic increases in NWP model skill will be required over the coming decades. This paper describes the role of major deployments of accurate profiling sensors in achieving that goal and presents an example from the Second Wind Forecast Improvement Program (WFIP2). Wind-profile data from scanning Doppler lidars were used to evaluate two versions of HRRR, the original and an updated version, and nested versions of each. This study focuses on the ability of updated HRRR versions to improve upon predicting a regional sea-breeze flow, which was found to generate large errors by the original HRRR. Updates to the full-scale HRRR versions produced mixed results, but the finer-mesh versions consistently reduced model errors.

Meteorology & Atmospheric Sciences↗
DOE OSTI2023

High Impedance Fault Detection Through Quasi-Static State Estimation: A Parameter Error Modeling Approach

This paper presents a model for detecting high impedance faults using parameter error modeling and a two step per-phase weighted-least squares state estimation process. The proposed scheme leverages the use of Phasor Measurement Units and synthetic measurements to identify per-phase power flow and injection measurements which indicate a parameter error through ?2 Hypothesis Testing applied to the composed measurement error. Although current and voltage waveforms are commonly analyzed for high-impedance fault detection, wide area power flow and injection measurements, which are already inherent to the state estimation process, also show promise for real-world high-impedance fault detection applications. The error distributions after detection share the measurement function error spread observed in proven parameter error diagnostics and can be applied to high-impedance fault identification. Further, this error spread across measurement functions related to the fault will be clearly discerned from measurement error. Case studies are performed on the IEEE 33-Bus Distribution System along with the proposed model in Simulink.

Cooper, Austin↗
DOE OSTI2023

Uncertainty Error Modeling for Non-Linear State Estimation With Unsynchronized SCADA and µPMU Measurements

Distribution systems of the future smart grid require enhancements to the reliability of distribution system state estimation (DSSE) in the face of low measurement redundancy, unsynchronized measurements, and dynamic load profiles. Micro phasor measurement units (µPMUs) facilitate co-synchronized measurements with high granularity, albeit at an often prohibitively expensive installation cost. Supervisory control and data acquisition (SCADA) measurements can supplement µPMU data, although they are received at a slower sampling rate. Further complicating matters is the uncertainty associated with load dynamics and unsynchronized measurements–not only are the SCADA and µPMU measurements not synchronized with each other, but the SCADA measurements themselves are received at different time intervals with respect to one another. This paper proposes a non-linear state estimation framework which models dynamic load uncertainty error by updating the variances of the unsynchronized measurements, leading to a time-varying system of weights in the weighted least squares state estimator. Case studies are performed on the 33-Bus Distribution System in MATPOWER, using Ornstein–Uhlenbeck stochastic processes to simulate dynamic load conditions.

Cooper, Austin↗
DOE OSTI2022

Goal-oriented a-posteriori estimation of model error as an aid to parameter estimation

In this work, a Bayesian model calibration framework is presented that utilizes goal-oriented a-posterior error estimates in quantities of interest (QoIs) for classes of high-fidelity models characterized by PDEs. It is shown that for a large class of computational models, it is possible to develop a computationally inexpensive procedure for calibrating parameters of high-fidelity models of physical events when the parameters of low-fidelity (surrogate) models are known with acceptable accuracy. The main ingredients in the proposed model calibration scheme are goal-oriented a-posteriori estimates of error in QoIs computed using a so-called lower fidelity model compared to those of an uncalibrated higher fidelity model. The estimates of error in QoIs are used to define likelihood functions in Bayesian inversion analysis. A standard Bayesian approach is employed to compute the posterior distribution of model parameters of high-fidelity models. As applications, parameters in a quasi-linear second-order elliptic boundary-value problem (BVP) are calibrated using a second-order linear elliptic BVP. In a second application, parameters of a tumor growth model involving nonlinear time-dependent PDEs are calibrated using a lower fidelity linear tumor growth model with known parameter values.

A-posterior estimates↗
DOE OSTI2021

Analytical Bit-Error Model of NAND Flash Memories for Dosimetry Application

In this work, we provide an analytical model for the total ionizing dose (TID) effects on the bit error statistics of commercial flash memory chips. We have validated the model with experimental data collected by irradiating several commercial NAND flash memory chips from different technology nodes. We find that our analytical model can project bit errors at higher TID values (~ 20 krad(Si)) from measured data at lower TID values (<1 krad(Si)). Based on our model and the measured data, we have formulated basic design rules for using a commercial flash memory chip as a dosimeter. We discuss the impact of NAND chip-to-chip variability, noise margin and the intrinsic errors on the dosimeter design using detailed experimentation.

3-D NAND↗
DOE OSTI2023

Effects of head modeling errors on the spatial frequency representation of MEG

Abstract Objectives. We aim to investigate the effects of head model inaccuracies on signal and source reconstruction accuracies for various sensor array distances to the head. This allows for the assessment of the importance of head modeling for next-generation magnetoencephalography (MEG) sensors, optically-pumped magnetometers (OPM). Approach. A 1-shell boundary element method (BEM) spherical head model with 642 vertices of radius 9 cm and conductivity of 0.33 S m −1 was defined. The vertices were then randomly perturbed radially up to 2%, 4%, 6%, 8% and 10% of the radius. For each head perturbation case, the forward signal was calculated for dipolar sources located at 2 cm, 4 cm, 6 cm and 8 cm from the origin (center of the sphere), and for a 324 sensor array located at 10 cm to 15 cm from the origin. Equivalent current dipole (ECD) source localization was performed for each of these forward signals. The signal for each perturbed spherical head case was then analyzed in the spatial frequency domain, and the signal and ECD errors were quantified relative to the unperturbed case. Main results. In the noiseless and high signal-to-noise ratio (SNR) case of approximately ≥6 dB, inaccuracies in our spherical BEM head conductor models lead to increased signal and ECD inaccuracies when sensor arrays are placed closer to the head. This is true especially in the case of deep and superficial sources. In the noisy case however, the higher SNR for closer sensor arrays allows for an improved ECD fit and outweighs the effects of head geometry inaccuracies. Significance. OPMs may be placed directly on the head, as opposed to the more commonly used superconducting quantum interference device sensors which must be placed a few centimeters away from the head. OPMs thus allow for signals of higher spatial resolution to be captured, resulting in potentially more accurate source localizations. Our results suggest that an increased emphasis on accurate head modeling for OPMs may be necessary to fully realize its improved source localization potential.

Yeo, Wan-Jin (ORCID:0000000241417123)↗
DOE OSTI2025

Reducing model error using optimized galaxy selection: weak lensing cluster mass estimation

Galaxy clusters are one of the most powerful probes to study extensions of General Relativity and the Standard Cosmological Model. Upcoming surveys like the Vera Rubin Observatory’s Legacy Survey of Space and Time are expected to revolutionise the field, by enabling the analysis of cluster samples of unprecedented size and quality. To reach this era of high-precision cluster cosmology, the mitigation of sources of systematic error is crucial. A particularly important challenge is bias in cluster mass measurements induced by the mismodelling of photometric redshift estimates of source galaxies. This work proposes a method to optimise the source sample selection in cluster weak lensing analyses drawn from wide-field survey lensing catalogs to reduce the bias on reconstructed cluster masses. We use a combinatorial optimisation scheme and methods from variational inference to select galaxies in latent space to produce a probabilistic galaxy source sample catalog for highly accurate cluster mass estimation. We show that our method reduces the critical surface mass density Σ crit relative modelling bias on the 60-70% level, while maintaining up to 90% of galaxies. We highlight that our methodology has applications beyond cluster mass estimation as an approach to jointly combine galaxy selection and model inference under sources of systematics.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2023

Diagnosing Near-Surface Model Errors with Candidate Physics Parameterization Schemes for the Multiphysics Rapid Refresh Forecast System (RRFS) Ensemble during Winter over the Northeastern United States and Southern Great Plains

Abstract During the winter of 2020/21 an ensemble of FV3-LAM forecasts was produced over the contiguous United States for the Winter Weather Experiment using five physics suites. These forecasts are evaluated with the goal of optimizing physics parameterizations within the future operational Rapid Refresh Forecast System (RRFS) in the Unified Forecast System (UFS) realm and for selecting suitable physics suites for a multiphysics RRFS ensemble. The five physics suites have different combinations of land surface models (LSMs), planetary boundary layer (PBL) parameterizations, and surface layer schemes, chosen from those used in current and possible future operational systems and likely to be supported in the operational UFS. Full-season evaluation reveals a persistent near-surface cold bias in the U.S. Northeast from one suite and a nighttime warm bias in the southern Great Plains in another suite, while other suites have smaller biases. A representative case is chosen to diagnose the cause for each of these biases using sensitivity simulations with different physics combinations or modified parameters and verified with additional mesonet observations. The cold bias in the Northeast is attributed to aspects of the Noah-MP LSM over snow cover, where Noah-MP simulates lower soil water content, and thus lower thermal conductivity than other LSMs, leading to less upward ground heat flux during nighttime and consequently lower surface temperature. The nighttime warm bias found in the southern Great Plains is attributed to overestimation of vertical mixing in the K -profile-based eddy-diffusivity mass-flux (K-EDMF) PBL scheme and insufficient land–atmospheric coupling from the GFS surface layer scheme over short vegetation. A few key parameters driving these systematic biases are identified.

Meteorology & Atmospheric Sciences↗