Results for “Model error” — Page 75

DOE OSTI2024

Generalization error guaranteed auto-encoder-based nonlinear model reduction for operator learning

Many physical processes in science and engineering are naturally represented by operators between infinite-dimensional function spaces. The problem of operator learning, in this context, seeks to extract these physical processes from empirical data, which is challenging due to the infinite or high dimensionality of data. An integral component in addressing this challenge is model reduction, which reduces both the data dimensionality and problem size. In this paper, we utilize low-dimensional nonlinear structures in model reduction by investigating Auto-Encoder-based Neural Network (AENet). AENet first learns the latent variables of the input data and then learns the transformation from these latent variables to corresponding output data. Our numerical experiments validate the ability of AENet to accurately learn the solution operator of nonlinear partial differential equations. Furthermore, we establish a mathematical and statistical estimation theory that analyzes the generalization error of AENet. Finally, our theoretical framework shows that the sample complexity of training AENet is intricately tied to the intrinsic dimension of the modeled process, while also demonstrating the robustness of AENet to noise.

Auto-encoder↗

DOE OSTI2023

Closing in on Hydrologic Predictive Accuracy: Combining the Strengths of High-Fidelity and Physics-Agnostic Models

Applications of process-based models (PBM) for predictions are confounded by multiple uncertainties and computational burdens, resulting in appreciable errors. A novel modeling framework combining a high-fidelity PBM with surrogate and machine learning (ML) models is developed to tackle these challenges and applied for streamflow prediction. A surrogate model permits high computational efficiency of a PBM solution at a minimum loss of its accuracy. A novel probabilistic ML model partitions the PBM-surrogate prediction errors into reducible and irreducible types, quantifying their distributions that arise due to both explicitly perceived uncertainties (such as parametric) or those that are entirely hidden to the modeler (not included or unexpected). Using this approach, we demonstrate a substantial improvement of streamflow predictive accuracy for a case study urbanized watershed. Such a framework provides an efficient solution combining the strengths of high-fidelity and physics-agnostic models for a wide range of prediction problems in geosciences.

58 GEOSCIENCES↗

DOE OSTI2023

Benchmarking quantum logic operations relative to thresholds for fault tolerance

Contemporary methods for benchmarking noisy quantum processors typically measure average error rates or process infidelities. However, thresholds for fault-tolerant quantum error correction are given in terms of worst-case error rates—defined via the diamond norm—which can differ from average error rates by orders of magnitude. One method for resolving this discrepancy is to randomize the physical implementation of quantum gates, using techniques like randomized compiling (RC). In this work, we use gate set tomography to perform precision characterization of a set of two-qubit logic gates to study RC on a superconducting quantum processor. We find that, under RC, gate errors are accurately described by a stochastic Pauli noise model without coherent errors, and that spatially correlated coherent errors and non-Markovian errors are strongly suppressed. We further show that the average and worst-case error rates are equal for randomly compiled gates, and measure a maximum worst-case error of 0.0197(3) for our gate set. Our results show that randomized benchmarks are a viable route to both verifying that a quantum processor’s error rates are below a fault-tolerance threshold, and to bounding the failure rates of near-term algorithms, if—and only if—gates are implemented via randomization methods which tailor noise.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2022

Fusion RF Modeling Machine Learning (FusionML_RF) v1.0

FusionML_RF consists of multiple codes and trained machine learning (ML) models that perform low-cost output modeling from the Genray-CQL3D. Three machine learning techniques (multilayer perceptron, random forest, and Gaussian process) provide fast surrogate models for lower hybrid current drive (LHCD) simulations. For example, completing a single GENRAY/CQL3D simulation without radial diffusion of fast electrons requires several minutes of wall-clock time. On the other hand, these ML models achieve ~ms of inference time with high accuracy across the input parameter space. This software collection consists of multiple components. (1) codes that use ML methods and precomputed Genray-CQL3D simulation output to build regression models that enable approximate computations of Genray-CLQ3D outputs from arbitrary but physically meaningful input parameters (surrogate modeling); (2) three trained models created by the team, using a database of 16,000+ GENRAY/CQL3D simulations, to study the performance of ML models for surrogate modeling; (3) codes that load the trained models and simulation data, and then compute mean squared error between the models' predictions and the ground truth of simulation output data. This collection is being made available in conjunction with a scientific publication about the work to promote reusability and provide an artifact of the scientific work.

Bai, Zhe↗

DOE OSTI2022

Adaptive Determination of the Optimal Exchange Location in Wall-Modeled Large-Eddy Simulation

Wall-modeled large-eddy simulation introduces a modeling interface (or exchange location) separating the wall-modeled layer from the rest of the domain. The current state-of-the-art is to rely on user expertise when choosing where to place this modeling interface, whether this choice is tied to the grid or not. This paper presents a postprocessing algorithm that determines the exchange location systematically. The algorithm is based on a model for the error in the predicted wall shear stress and a model for the computational cost, and then finds the exchange location that minimizes a combination of the two. Here, the algorithm is tested both a priori and a posteriori using an equilibrium wall model for the flow over a wall-mounted hump, a boundary layer in an adverse pressure gradient, and a shock/boundary-layer interaction. The algorithm produces exchange locations that mostly agree with what an experienced user would suggest, with thinner wall-modeled layers in nonequilibrium flow regions and thicker wall-modeled layers where the boundary layer is closer to equilibrium. This suggests that the algorithm should be useful in simulations of realistic and highly complex geometries.

42 ENGINEERING↗

NASA NTRS1980

Nonlinear grid error effects on numerical solution of partial differential equations

Finite difference solutions of nonlinear partial differential equations require discretizations and consequently grid errors are generated. These errors strongly affect stability and convergence properties of difference models. Previously such errors were analyzed by linearizing the difference equations for solutions. Properties of mappings of decadence were used to analyze nonlinear instabilities. Such an analysis is directly affected by initial/boundary conditions. An algorithm was developed, applied to nonlinear Burgers equations, and verified computationally. A preliminary test shows that Navier-Stokes equations may be treated similarly.

Dey, S. K.↗

NASA NTRS1981

Burst statistics of Viterbi decoding

A mathematical model of Viterbi decoder burst error performance is presented. This model allows for computer generation of Viterbi-like error sequences quickly and inexpensively for applications where large amounts of data are required. The model is corroborated through comparisons with actual software decoder simulations.

Deutsch, L. J.↗

NASA NTRS2009

Simulation of the Impact of New Aircraft- and Satellite-based Ocean Surface Wind Measurements on Estimates of Hurricane Intensity

The Hurricane Imaging Radiometer (HIRAD) is a new airborne microwave remote sensor currently under development to enhance real-time hurricane ocean surface wind observations. HIRAD builds on the capabilities of the Stepped Frequency Microwave Radiometer (SFMR), which now operates on NOAA P-3, G-4, and AFRC C-130 aircraft. Unlike the SFMR, which measures wind speed and rain rate along the ground track directly beneath the aircraft, HIRAD will provide images of the surface wind and rain field over a wide swath (approximately 3 times the aircraft altitude). To demonstrate potential improvement in the measurement of peak hurricane winds, we present a set of Observing System Simulation Experiments (OSSEs) in which measurements from the new instrument as well as those from existing platforms (air, surface, and space-based) are simulated from the output of a high-resolution (approximately 1.7 km) numerical model. Simulated retrieval errors due to both instrument noise as well as model function accuracy are considered over the expected range of incidence angles, wind speeds and rain rates. Based on numerous simulated flight patterns and data source combinations, statistics are developed to describe relationships between the observed and true (from the model s perspective) peak wind speed. These results have implications for improving the estimation of hurricane intensity (as defined by the peak sustained wind anywhere in the storm), which may often go un-observed due to sampling limitations.

Uhlhorn, Eric↗

NASA NTRSDate not supplied

Recent Developments in the Assimilation of Microwave and Radar Observations Into NWP Models

Microwave observations play a very important role in improving the weather forecasts. Although these observations are routinely assimilated into NWP models in clear-sky conditions, assimilation of all-sky microwave observations is very limited. Two main factors contributing to this limitation are inaccuracy in the input cloud and hydrometeor profiles used as input to the radiative transfer model and also error in scattering calculations performed by the radiative transfer model itself. The Mie theory is used by many fast RT models to estimate the optical properties of single particles. The Mie theory assumes spherical shapes for ice or snow particles with mixture of air and ice. However, hydrometeors scattering radiation at microwave frequencies have different shapes, sizes, and orientations. Therefore, using Mie theory to determine their optical properties leads to large uncertainties in all-sky radiative transfer calculations. The discrete dipole approximation (DDA) which approximates the optical properties of large objects in terms of discrete dipoles has shown promise in calculating the scattering properties of particles with different shapes in the microwave frequencies. This presentation focuses on recent advancements in the CRTM scattering calculations for frozen hydrometeors in the microwave frequencies using the DDA technique. In addition to using stand-alone CRTM calculations using collocated ATMS and reanalysis profiles, the data assimilation experiments conducted using the NOAA FV3GFS forecast system are used to evaluate the scattering improvements. Additionally, the backscattering information from the DDA database was used to implement a radar simulator into CRTM. The radar operator takes advantage of CRTM different modules to calculate clouds absorption and scattering properties. In addition to the forward model both adjoint and tangent linear of the radar simulator are implemented and evaluated as well. The radar simulator is currently being tested within the JEDI/GEOS data assimilation framework to facilitate the assimilation of radar measurements such as CloudSat CPR and GPM DPR into the NASA GEOS model.

Isaac Moradi↗

NASA NTRSDate not supplied

Enhancing CRTM All-Sky Simulations and Implementation of A New Active Sensor Module

Microwave observations play a very important role in improving the weather forecasts. Although these observations are routinely assimilated into NWP models in clear-sky conditions, assimilation of all-sky microwave observations is very limited. Two main factors contributing to this limitation are inaccuracy in the input cloud and hydrometeor profiles used as input to the radiative transfer model and also error in scattering calculations performed by the radiative transfer model itself. The Mie theory is used by many fast RT models to estimate the optical properties of single particles. The Mie theory assumes spherical shapes for ice or snow particles with mixture of air and ice. However, hydrometeors scattering radiation at microwave frequencies have different shapes, sizes, and orientations. Therefore, using Mie theory to determine their optical properties leads to large uncertainties in all-sky radiative transfer calculations. The discrete dipole approximation (DDA) which approximates the optical properties of large objects in terms of discrete dipoles has shown promise in calculating the scattering properties of particles with different shapes in the microwave frequencies. This presentation focuses on recent advancements in the CRTM scattering calculations for frozen hydrometeors in the microwave frequencies using the DDA technique. In addition to using stand-alone CRTM calculations using collocated ATMS and reanalysis profiles, the data assimilation experiments conducted using the NOAA FV3GFS forecast system are used to evaluate the scattering improvements. Additionally, the backscattering information from the DDA database was used to implement a radar simulator into CRTM. The radar operator takes advantage of CRTM different modules to calculate clouds absorption and scattering properties. In addition to the forward model both adjoint and tangent linear of the radar simulator are implemented and evaluated as well. The radar simulator is currently being tested within the JEDI/GEOS data assimilation framework to facilitate the assimilation of radar measurements such as CloudSat CPR and GPM DPR into the NASA GEOS model.

Isaac Moradi↗

NASA NTRSDate not supplied

CRTM Microwave Cloud Scattering Lookup Tables and Radar Simulator

Microwave observations play a very important role in improving the weather forecasts. Although these observations are routinely assimilated into NWP models in clear-sky conditions, assimilation of all-sky microwave observations is very limited. Two main factors contributing to this limitation are inaccuracy in the input cloud and hydrometeor profiles used as input to the radiative transfer model and also error in scattering calculations performed by the radiative transfer model itself. The Mie theory is used by many fast RT models to estimate the optical properties of single particles. The Mie theory assumes spherical shapes for ice or snow particles with mixture of air and ice. However, hydrometeors scattering radiation at microwave frequencies have different shapes, sizes, and orientations. Therefore, using Mie theory to determine their optical properties leads to large uncertainties in all-sky radiative transfer calculations. The discrete dipole approximation (DDA) which approximates the optical properties of large objects in terms of discrete dipoles has shown promise in calculating the scattering properties of particles with different shapes in the microwave frequencies. This presentation focuses on recent advancements in the CRTM scattering calculations for frozen hydrometeors in the microwave frequencies using the DDA technique. In addition to using stand-alone CRTM calculations using collocated ATMS and reanalysis profiles, the data assimilation experiments conducted using the NOAA FV3GFS forecast system are used to evaluate the scattering improvements. Additionally, the backscattering information from the DDA database was used to implement a radar simulator into CRTM. The radar operator takes advantage of CRTM different modules to calculate clouds absorption and scattering properties. In addition to the forward model both adjoint and tangent linear of the radar simulator are implemented and evaluated as well. The radar simulator is currently being tested within the JEDI/GEOS data assimilation framework to facilitate the assimilation of radar measurements such as CloudSat CPR and GPM DPR into the NASA GEOS model.

Isaac Moradi↗

DOE OSTI2021

Toward Predictive RANS and SRS Computations of Turbulent External Flows of Practical Interest

In this work, we investigate the main challenges to prediction of turbulent external flows of practical interest with Reynolds-Averaged Navier–Stokes equations (RANS) and Scale-Resolving Simulation (SRS) models. This represents a crucial step toward further developing and establishing these formulations so they can be confidently utilized in engineering problems without reference data. The study initiates by identifying the major challenges to prediction. A literature review is performed to illustrate their effects in RANS and SRS computations. Afterward, we evaluate the impact of the challenges to prediction by analyzing representative statistically steady and unsteady flows with prominent RANS and SRS methods. These include multiple turbulent viscosity and second-moment RANS closures, and hybrid and bridging SRS models. The results demonstrate the potential of the selected SRS models to predict engineering flows. Yet, they also show the importance of considering the challenges to prediction during the setup and conduction of numerical experiments. These can suppress the advantages of using SRS formulations. The data also indicate that only SRS models can confidently predict statistically unsteady flows. In contrast, the results demonstrate that mean-flow quantities of statistically steady flows can be efficiently calculated with RANS closures, especially second-moment closures. Among the selected SRS methods, bridging models reveal better suited for prediction due to their ability to prevent commutation errors and enable the robust evaluation of numerical and modeling errors. This last property allows the use of a new validation technique that does not require reference data.

42 ENGINEERING↗

DOE OSTI2021

Model-based approach to investigate equipment-induced error in pressure-waveform derived hemodynamic measurements

Advanced hemodynamic monitoring systems have provided less invasive methods for estimating pressure-derived measurements such as pressure-derived cardiac output (CO) measurements. These devices apply algorithms to arterial pressure waveforms recorded via pressure recording components that transmit the pressure signal to a pressure monitor. While standards have been developed for pressure monitoring equipment, it is unclear how the equipment-induced error can affect secondary measurements from pressure waveforms. We propose an approach for modelling different components of a pressure monitoring system and use this model-based approach to investigate the effect of different pressure recording configurations on pressure-derived hemodynamic measurements. The proposed model-based approach is a three-step process. (1) Modelling the response of pressure recording components using bench tests; (2) verifying the identified models through nonparametric equivalence tests; and (3) assessing the effects of pressure recording components on pressure-derived measurements. To delineate the application of this approach, we performed a series of model-based analyses to quantify the combined effect of a wide range of tubing configurations with various damping ratios and natural frequencies and monitors with different bandwidths on pressure waveforms and CO measurements by six pulse contour algorithms. Model-based results show the error in pressure-derived CO measurements because of tubing configurations with different natural frequencies and damping ratios. Tubing configurations with low natural frequencies (<23 Hz) altered characteristics of pressure waveforms in a way that affected the CO measurement, some by as much as 20%. Our method can serve as a tool to quantify the performance of pressure recording systems with different dynamic properties. In conclusion, this approach can be applied to investigate the effects of physiologic signal recording configurations on various pressure-derived hemodynamic measurements.

42 ENGINEERING↗

NASA NTRS1988

Performability modeling based on real data: A case study

Described is a measurement-based performability model based on error and resource usage data collected on a multiprocessor system. A method for identifying the model structure is introduced and the resulting model is validated against real data. Model development from the collection of raw data to the estimation of the expected reward is described. Both normal and error behavior of the system are characterized. The measured data show that the holding times in key operational and error states are not simple exponentials and that a semi-Markov process is necessary to model system behavior. A reward function, based on the service rate and the error rate in each state, is then defined in order to estimate the performability of the system and to depict the cost of apparent types of errors.

Hsueh, M. C.↗

NASA NTRS1991

Evaporation over global oceans derived from satellite data and AGCM

Evaporation over ocean at 100 km has been estimated using operational satellite data. Recent satellite results are compared to data obtained through atmospheric general circulation models (AGCMs). A large systematic error is found in AGCM data which is considered to be due to overestimation of atmospheric moisture. It is suggested that the discrepancy may be caused by errors in operational infrared sounder data assimilated into the model. The error may also be caused by interpolation of very sparse in situ data in areas of sharp gradients.

Liu, W. T.↗

NASA NTRS1987

Performability modeling based on real data: A casestudy

Described is a measurement-based performability model based on error and resource usage data collected on a multiprocessor system. A method for identifying the model structure is introduced and the resulting model is validated against real data. Model development from the collection of raw data to the estimation of the expected reward is described. Both normal and error behavior of the system are characterized. The measured data show that the holding times in key operational and error states are not simple exponentials and that a semi-Markov process is necessary to model the system behavior. A reward function, based on the service rate and the error rate in each state, is then defined in order to estimate the performability of the system and to depict the cost of different types of errors.

Hsueh, M. C.↗

DOE OSTI2025

Parameter uncertainties for imperfect surrogate models in the low-noise regime

Abstract Bayesian regression determines model parameters by minimizing the expected loss, an upper bound to the true generalization error. However, this loss ignores model form error, or misspecification, meaning parameter uncertainties are significantly underestimated and vanish in the large data limit. As misspecification is the main source of uncertainty for surrogate models of low-noise calculations, such as those arising in atomistic simulation, predictive uncertainties are systematically underestimated. We analyze the true generalization error of misspecified, near-deterministic surrogate models, a regime of broad relevance in science and engineering. We show that posterior parameter distributions must cover every training point to avoid a divergence in the generalization error and design a compatible ansatz which incurs minimal overhead for linear models. The approach is demonstrated on model problems before application to thousand-dimensional datasets in atomistic machine learning. Our efficient misspecification-aware scheme gives accurate prediction and bounding of test errors in terms of parameter uncertainties, allowing this important source of uncertainty to be incorporated in multi-scale computational workflows.

Swinburne, Thomas D. (ORCID:0000000232554257)↗

DOE OSTI2023

An Investigation of Time Distributions for Task Primitives to Support the HUNTER Dynamic Human Reliability Analysis

As an effort to support data collection for dynamic human reliability analysis (HRA), this study investigates time distributions for task primitives defined in the Goals, Operators, Methods, and Selection rules (GOMS)–Human Reliability Analysis (HRA) method. GOMS-HRA was developed to provide cognition-based time and human error probability information for dynamic HRA calculation in the Human Unimodel for Nuclear Technology to Enhance Reliability (HUNTER) framework. HUNTER is a framework to support the dynamic modelling of human error in conjunction with other modelling tools. In this paper, we investigate time distributions using experimental data collected from the Simplified Human Error Experimental Program (SHEEP) study, which suggests an HRA data collection framework to complement full-scope simulator research as well as collect input data for dynamic HRA using simplified simulators such as the Rancor Microworld Simulator. In this study, time required for GOMS-HRA task primitives to satisfy thirteen statistical distributions is investigated. Then, the time distributions from student operators and professional operators are compared and discussed. As a result, this study identified several time distributions on five GOMS-HRA task primitives at a statistically significant level. According to analyses to date, a greater number of significant time distributions was found in abnormal or emergency operating procedures rather than standard operating procedures. In the future, it is expected that the result of this study can provide objective reference on elapsed time data for task primitives as well as help to realistically simulate scenarios within dynamic HRA.

99 GENERAL AND MISCELLANEOUS↗