Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Error Modeling”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 163 records · Page 9

Successive Procedure for Solution Verification Based on User Needs

This paper discusses a revised solution verification procedure for computational fluid dynamics simulations to estimate the uncertainties in the quantities of interest based on discretization error models. This proposed procedure builds upon current procedures described in ASME V&V 20 but provides more guidance in determining the necessary number of mesh levels to build reliable discretization error models. Such guidance is particularly useful for practicing engineers without prior experience in solution verification. The key features of this proposed solution verification procedure are the ability to determine the need for additional mesh levels iteratively and the seamless treatment for underdetermined, exact, and overdetermined solutions of the power series approximation to the discretization error models. This study applies the proposed procedure to a set of synthetic examples to demonstrate the revised procedure’s clarity in determining the number of mesh solutions required for a reliable estimate of the discretization error in computational fluid dynamics settings. Additionally, this proposed procedure prevents a potential pathway in the current procedure in ASME V&V 20 that may lead to unreasonably small discretization errors.

Weinmeister, Justin↗

A Comparison of Forecast Error Generators for Modeling Wind and Load Uncertainty

This paper presents four algorithms to generate random forecast error time series, including a truncated-normal distribution model, a state-space based Markov model, a seasonal autoregressive moving average (ARMA) model, and a stochastic-optimization based model. The error time series are used to create real-time (RT), hour-ahead (HA), and day-ahead (DA) wind and load forecast time series that statistically match historically observed forecasting data sets, used for variable generation integration studies. A comparison is made using historical DA load forecast and actual load values to generate new sets of DA forecasts with similar stoical forecast error characteristics. This paper discusses and compares the capabilities of each algorithm to preserve the characteristics of the historical forecast data sets.

Lu, Ning↗

Moving beyond the total sea ice extent in gauging model biases

Here, reproducing characteristics of observed sea ice extent remains an important climate modeling challenge. This study describes several approaches to improve how model biases in total sea ice distribution are quantified, and applies them to historically forced simulations contributed to phase 5 of the Coupled Model Intercomparison Project (CMIP5). The quantity of hemispheric total sea ice area, or some measure of its equatorward extent, is often used to evaluate model performance. A new approach is introduced that investigates additional details about the structure of model errors, with an aim to reduce the potential impact of compensating errors when gauging differences between simulated and observed sea ice. Using multiple observational datasets, several new methods are applied to evaluate the climatological spatial distribution and the annual cycle of sea ice cover in 41 CMIP5 models. It is shown that in some models, error compensation can be substantial, for example resulting from too much sea ice in one region and too little in another. Error compensation tends to be larger in models that agree more closely with the observed total sea ice area, which may result from model tuning. The results herein suggest that consideration of only the total hemispheric sea ice area or extent can be misleading when quantitatively comparing how well models agree with observations. Further work is needed to fully develop robust methods to holistically evaluate the ability of models to capture the finescale structure of sea ice characteristics; however, the “sector scale” metric used here aids in reducing the impact of compensating errors in hemispheric integrals.

54 ENVIRONMENTAL SCIENCES↗

Uncertainty quantification for nuclear density functional theory and information content of new measurements

Statistical tools of uncertainty quantification can be used to assess the information content of measured observables with respect to present-day theoretical models, to estimate model errors and thereby improve predictive capability, to extrapolate beyond the regions reached by experiment, and to provide meaningful input to applications and planned measurements. To showcase new opportunities offered by such tools, we make a rigorous analysis of theoretical statistical uncertainties in nuclear density functional theory using Bayesian inference methods. By considering the recent mass measurements from the Canadian Penning Trap at Argonne National Laboratory, we demonstrate how the Bayesian analysis and a direct least-squares optimization, combined with high-performance computing, can be used to assess the information content of the new data with respect to a model based on the Skyrme energy density functional approach. Employing the posterior probability distribution computed with a Gaussian process emulator, we apply the Bayesian framework to propagate theoretical statistical uncertainties in predictions of nuclear masses, two-neutron dripline, and fission barriers. Overall, we find that the new mass measurements do not impose a constraint that is strong enough to lead to significant changes in the model parameters. In addition, the example discussed in this study sets the stage for quantifying and maximizing the impact of new measurements with respect to current modeling and guiding future experimental efforts, thus enhancing the experiment-theory cycle in the scientific method.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Quantification of modeling uncertainty in the Rayleigh damping model

Understanding and accurately characterizing energy dissipation mechanisms in civil structures during earthquakes is an important element of seismic assessment and design. The most commonly used model is attributed to Rayleigh. This paper proposes a systematic approach to quantify the uncertainty associated with Rayleigh's damping model. Bayesian calibration with embedded model error is employed to treat the coefficients of the Rayleigh model as random variables using modal damping ratios. Through a numerical example, we illustrate how this approach works and how the calibrated model can address modeling uncertainty associated with the Rayleigh damping model.

42 ENGINEERING↗

A Proposed Global Optimum Algebraic Iterative Solver for Modeling of Lattice Element Errors

Model-based electron accelerator control is the maintenance of optimal parameters of an electron beam such as its orbit, size, and shape, as well as machine parameters such as tunes. It works well when the model reflects reality. SLAC pioneered this technique in SPEAR about thirty decades ago. Similar techniques are now employed in particle accelerator and synchrotron laboratories around the world. There is still an inherently complex problem related to the employment of such techniques to manage the operation and analysis of accelerators and storage rings. The problem arises from the use by those techniques of complex numerical algorithms commonly known as nonlinear solvers that are difficult to control and operate. Lessons learned at SLAC have led to the development of a new, simple-to-use, and iterative nonlinear solver that holds much promise not only in advancing the derivation of errors in accelerators and storage rings at SLAC, but also in its ability to tackle a range of complex engineering problems. We intend to further develop and validate this nonlinear solver for robust SLAC accelerator and other applications.

ACCSYS↗

Bayesian Calibration of the Community Land Model using Surrogates

We present results from the Bayesian calibration of hydrological parameters of the Community Land Model (CLM), which is often used in climate simulations and Earth system models. A statistical inverse problem is formulated for three hydrological parameters, conditioned on observations of latent heat surface fluxes over 48 months. Our calibration method uses polynomial and Gaussian process surrogates of the CLM, and solves the parameter estimation problem using a Markov chain Monte Carlo sampler. Posterior probability densities for the parameters are developed for two sites with different soil and vegetation covers. Our method also allows us to examine the structural error in CLM under two error models. We find that accurate surrogate models can be created for CLM in most cases. The posterior distributions lead to better prediction than the default parameter values in CLM. Climatologically averaging the observations does not modify the parameters’ distributions significantly. The structural error model reveals a correlation time-scale which can potentially be used to identify physical processes that could be contributing to it. While the calibrated CLM has a higher predictive skill, the calibration is under-dispersive.

Bayesian calibration↗

AMiBA: cluster Sunyaev-Zel'dovich effect observations with the expanded 13-element array

The Yuan-Tseh Lee Array for Microwave Background Anisotropy (AMiBA) is a co-planar interferometer array operating at a wavelength of 3 mm to measure the Sunyaev–Zel’dovich effect (SZE) of galaxy clusters at arcminute scales. The first phase of operation—with a compact 7-element array with 0.6 m antennas (AMiBA-7)—observed six clusters at angular scales from 5{sup ′} to 23{sup ′}. Here, we describe the expansion of AMiBA to a 13-element array with 1.2 m antennas (AMiBA-13), its subsequent commissioning, and cluster SZE observing program. The most noticeable changes compared to AMiBA-7 are (1) array re-configuration with baselines ranging from 1.4 m to 4.8 m, allowing us to sample structures between 2{sup ′} and 10{sup ′}, (2) 13 new lightweight carbon-fiber-reinforced plastic (CFRP) 1.2 m reflectors, and (3) additional correlators and six new receivers. Since the reflectors are co-mounted on and distributed over the entire six-meter CFRP platform, a refined hexapod pointing error model and phase error correction scheme have been developed for AMiBA-13. These effects—entirely negligible for the earlier central close-packed AMiBA-7 configuration—can lead to additional geometrical delays during observations. Our correction scheme recovers at least 80 ± 5% of the point-source fluxes. We, therefore, apply an upward correcting factor of 1.25 to our visibilities to correct for phase decoherence, and a ±5% systematic uncertainty is added in quadrature with our statistical errors. We demonstrate the absence of further systematics with a noise level consistent with zero in stacked uv-visibilities. From the AMiBA-13 SZE observing program, we present here maps of a subset of 12 clusters with signal-to-noise ratios above five. We demonstrate combining AMiBA-7 with AMiBA-13 observations on Abell 1689, by jointly fitting their data to a generalized Navarro–Frenk–White model. Our cylindrically integrated Compton-y values for five radii are consistent with results from the Berkeley-Illinois-Maryland Array, the Owens Valley Radio Observatory, the Sunyaev–Zel’dovich Array, and the Planck Observatory. We also report the first targeted SZE detection toward the optically selected cluster RCS J1447+0828, and we demonstrate the ability of AMiBA SZE data to serve as a proxy for the total cluster mass. Finally, we show that our AMiBA-SZE derived cluster masses are consistent with recent lensing mass measurements in the literature.

79 ASTRONOMY AND ASTROPHYSICS↗

A novel network-based approach to determining measurement representation error for model evaluation of aerosol microphysical properties

Atmospheric aerosol size and abundance influence radiative effects and climate change. To date, efforts to constrain global climate models’ radiative forcing with in situ aerosol observations have been hamstrung by uncertainty. One source of error, the regional “representation error,” arises when accurate but sparse single-point measurements of atmospheric aerosol distributions are compared with a model value, assuming that the single-point measurement is representative of the model domain. The Portable Optical Particle Spectrometer network in the Southern Great Plains (POPSnet-SGP) campaign has demonstrated that a network of nearly autonomous aerosol instruments operating at ambient temperature and relative humidity (with low measurement error) may be used to quantify measurement representation error and investigate the factors introducing heterogeneity in aerosol distributions across a rural, continental background region. Measurements were made using Portable Optical Particle Spectrometer (POPS) instruments at several sites for five months across the Department of Energy’s Aerosol Radiation Measurement Southern Great Plains (ARM-SGP) User Facility in the central USA. Measurement representation error decreased with longer averaging periods (20-40 % between 1 sec and 1 day), varied between sites by 10 – 20 % for aerosol concentration 140 – 2500 nm in diameter (N_140), and was higher for aerosols > 400 nm in diameter (N_400). Our measurements also show the influence of local meteorology on aerosol surface area (A_140) and size distributions: A_140 is positively correlated with wind speed and relative humidity, negatively correlated with precipitation, and lower given westerly winds. Based on this study, we conclude that the POPSnet approach provides considerably more insight into the spatial variability in the aerosol population that can be used to constrain climate models than would be available from similar networks of PM 2.5 monitors.

54 ENVIRONMENTAL SCIENCES↗

A novel implicit hybrid machine learning model and its application for reinforcement learning

A novel methodology to develop implicit hybrid models is presented. PyTorch is used to integrate physics-based equations with machine learning models. Automatic differentiation of the hybrid model is leveraged to solve the implicit equations. Iterative solving enables gradient based updates to the machine learning model. The novel methodology is compared to an explicit hybrid approach on a continuously stirred tank reactor (CSTR). The novel method results in a lower modelling error. Both hybrid models effectively train with noisy data. To test the implicit hybrid model, it is employed as a reinforcement learning (RL) training model. The RL algorithm trained on the hybrid model outperforms real time optimization of the CSTR and performs nearly as well as RL trained directly on the CSTR and a traditional gradient based approach. Training RL directly on the CSTR requires over 60,000 system interactions compared to 6000 historical data points for hybrid model development.

42 ENGINEERING↗

Regional-scale, sector-specific evaluation of global CO2 inversion models using aircraft data from the ACT-America project

We use vertical profiles of airborne measurements of CO2 from frontal cases during the summer 2016 Atmospheric Carbon and Transport – America (ACT-America) campaign to evaluate the skill of a set of ten global CO2 inversion models participating in the Orbiting Carbon Observatory – 2 (OCO-2) Model Intercomparison Project (MIP). Model errors and biases (model minus observation) were categorized by region (Mid-Atlantic, Midwest, and South), frontal sector (warm or cold), and transport model (predominantly Tracer Model 5 (TM5) and Goddard Earth Observing System – Chemistry (GEOS-Chem)). Overall, the inversions reproduce the general structures of the observed vertical profiles and the enhanced / depleted low-level CO2 in warm / cold sectors, but tend to underestimate the magnitude of the sector difference in each region. In the Midwest and South warm sectors, inversion biases were about 1 ppm above 1500 m AGL, though model spread (quantified by interquartile range) was often even smaller; below 1500 m AGL, model biases were negative and about -2 ppm near the surface. For the Midwest and South cold sectors, models had +2-3 ppm biases below 1500 m AGL, but with comparable model spread. Uniquely, in the Mid-Atlantic there was a consistent difference between TM5 and GEOS-Chem mole fractions for both sectors up to 3000 m AGL (TM5 lower by 2 ppm), on the order of the observation-relative biases. In the MidAtlantic TM5 inversions had negative biases in warm sectors, while GEOS-Chem inversions had positive biases in cold sectors. Possible reasons for the regional variability are discussed.

Gaudet, Brian J.↗

A Hybrid Biophysical‐Machine Learning Framework for Diurnal Surface Energy Flux Estimation Using Proximal Sensing

Thermal infrared-based remote sensing of surface energy fluxes has traditionally relied on high spatial resolution satellite data with revisit frequencies on the order of weeks. In this study, we evaluate a biophysics-based analytical surface energy balance model for predicting latent energy ( LE ) and sensible heat ( H ) fluxes using proximal sensing observations. The Surface Temperature Initiated Closure (STIC1.2) model has been extensively validated across a wide range of spatial and temporal scales using various satellite-derived thermal infrared data sets. Here we extend this validation by applying STIC at sub-hourly temporal resolution over multiple growing seasons for four distinct agricultural systems. We further develop and evaluate novel STIC variants that incorporate machine learning (ML) techniques to eliminate the need for surface energy balance observations, specifically net radiation and soil heat flux, thereby enhancing model applicability in data-sparse settings. The integration of a ML component to estimate surface available energy is shown to have strong predictive performance for both LE (R 2 = 0.81–0.94) and H (R 2 = 0.46–0.72) across all agricultural systems examined here, demonstrating the potential of hybrid biophysical-machine learning approaches for surface energy balance modeling with minimal data requirements. This study concludes with a novel application of explainable machine learning (exML) to diagnose sources of model error. This exML framework attributes residual prediction errors to both model input variables and environmental drivers not explicitly included in the simulation experiments. This approach provides a new pathway for improving model design and integrating previously overlooked yet influential variables into future model iterations.

evapotranspiration↗

Uncertainty quantification in application of the enrichment meter principle for nondestructive assay of special nuclear material

The various methods of nondestructive assay (NDA) of special nuclear material (SNM) have applications in nuclear nonproliferation, including detection and identification of illicit SNM at border crossings, and quantifying SNM at nuclear facilities for safeguards. No assay method is complete without “error bars,” which provide one way of expressing confidence in the assay result. Consequently, NDA specialists typically quantify total uncertainty in terms of “random” and “systematic” components, and then specify error bars for the total mass estimate in multiple items. Uncertainty quantification (UQ) for NDA has always been important, but it is recognized that greater rigor is needed and achievable using modern statistical methods. To this end, we describe the extent to which the guideline for expressing uncertainty in measurements (GUM) can be used for NDA. Also, we propose improvements over GUM for NDA by illustrating UQ challenges that it does not address, including calibration with errors in predictors, model error, and item-specific biases. A case study is presented using low-resolution NaI spectra and applying the enrichment meter principle to estimate the U-235 mass in an item. The case study illustrates how to update the current American Society for Testing and Materials guide for application of the enrichment meter principle using gamma spectra from a NaI detector.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Data for: A hybrid biophysical-machine learning framework for diurnal surface energy flux estimation using proximal sensing

Thermal-based remote sensing of surface energy fluxes has traditionally relied on high spatial resolution satellite data with revisit frequencies on the order of weeks. In this study, we evaluate a biophysics-based analytical surface energy balance model for predicting latent energy (LE) and sensible heat (H) fluxes using proximal sensing observations. The Surface Temperature Initiated Closure (STIC1.2) model has been extensively validated across a wide range of spatial and temporal scales using various satellite-derived thermal datasets. Here we extend this validation by applying STIC at sub-hourly temporal resolution over multiple growing seasons for four distinct agricultural systems. We further develop and evaluate novel STIC variants that incorporate machine learning (ML) techniques to eliminate the need for specific surface energy balance observations, specifically net radiation and soil heat flux, thereby enhancing model applicability in data-sparse settings. The integration of an ML component to estimate surface available energy is shown to have strong predictive performance for both LE (R2 = 0.81-0.94) and H (R2 = 0.46-0.72) across all agricultural systems examined here, demonstrating the potential of hybrid biophysical – machine learning approaches for surface energy balance modeling with minimal data requirements. This study concludes with a novel application of explainable machine learning (exML) to diagnose sources of model error. This exML framework attributes residual prediction errors to both model input variables and environmental drivers not explicitly included in the simulation experiments. This approach provides a new pathway for improving model design and integrating previously overlooked yet influential variables into future model iterations.

Agricultural Sciences↗

Toward Joint Hypothesis-Tests Seismic Event Screening Analysis: Ms|mb and Event Depth

Well established theory can be used to combine single-phenomenology hypothesis tests into a multi-phenomenology event screening hypothesis test (Fisher's and Tippett's tests). Commonly used standard error in Ms:mb event screening hypothesis test is not fully consistent with physical basis. Improved standard error - Better agreement with physical basis, and correctly partitions error to include Model Error as a component of variance, correctly reduces station noise variance through network averaging. For 2009 DPRK test - Commonly used standard error 'rejects' H0 even with better scaling slope ({beta} = 1, Selby et al.), improved standard error 'fails to rejects' H0.

58 GEOSCIENCES↗

Evaluating the WFIP2 updates to the HRRR model using scanning Doppler lidar measurements in the complex terrain of the Columbia River Basin

The wind-energy (WE) industry relies on numerical weather prediction (NWP) forecast models as foundational or base models for many purposes, including wind-resource assessment and wind-power forecasting. During the Second Wind Forecast Improvement Project (WFIP2) in the Columbia River Basin of Oregon and Washington, a significant effort was made to improve NWP forecasts through focused model development, to include experimental refinements to the High Resolution Rapid Refresh (HRRR) model physics and horizontal grid spacing. In this study, the performance of an experimental version of HRRR that includes these refinements is tested against a control version, which corresponds to that of the operational HRRR run by National Oceanic and Atmospheric Administration/National Centers for Environmental Protection at the outset of WFIP2. Furthermore, the effects of horizontal grid resolution were also tested by comparing wind forecasts from the HRRR (with 3-km grid spacing) with those from a finer-resolution HRRR nest with 750-m grid spacing. Model forecasts are validated against accurate wind-profile measurements by three scanning, pulsed Doppler lidars at sites separated by a total distance of 71 km. Model skill and improvements in model skill, attributable to physics refinements and improved horizontal grid resolution, varied by season, by site, and during periods of atmospheric phenomena relevant to WE. In general, model errors were the largest below 150 m above ground level (AGL). Experimental HRRR refinements tended to reduce the mean absolute error (MAE) and other error metrics for many conditions, but degradation in skill (increased MAE) was noted below 150 m AGL at the two lowest-elevation sites at night. Finer resolution was found to produce the most significant reductions in the error metrics.

17 WIND ENERGY↗

Free kick instead of cross-validation in maximum-likelihood refinement of macromolecular crystal structures

The maximum-likelihood free-kick target, which calculates model error estimates from the work set and a randomly displaced model, proved superior in the accuracy and consistency of refinement of crystal structures compared with the maximum-likelihood cross-validation target, which calculates error estimates from the test set and the unperturbed model. The refinement of a molecular model is a computational procedure by which the atomic model is fitted to the diffraction data. The commonly used target in the refinement of macromolecular structures is the maximum-likelihood (ML) function, which relies on the assessment of model errors. The current ML functions rely on cross-validation. They utilize phase-error estimates that are calculated from a small fraction of diffraction data, called the test set, that are not used to fit the model. An approach has been developed that uses the work set to calculate the phase-error estimates in the ML refinement from simulating the model errors via the random displacement of atomic coordinates. It is called ML free-kick refinement as it uses the ML formulation of the target function and is based on the idea of freeing the model from the model bias imposed by the chemical energy restraints used in refinement. This approach for the calculation of error estimates is superior to the cross-validation approach: it reduces the phase error and increases the accuracy of molecular models, is more robust, provides clearer maps and may use a smaller portion of data for the test set for the calculation of R{sub free} or may leave it out completely.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗