NOT ALL ERRORS ARE CREATED EQUAL: EXAMINING HUMAN-ALGORITHM SYSTEM PERFORMANCE FOR INTERNATIONAL SAFEGUARDS-INFORMED VISUAL SEARCH TASKS .
Abstract not provided.
Gastelum, Zoe↗
SEARCH · Engineering Papers
Results for “error performance”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Robust verification of stochastic simulation codes
We introduce a robust verification tool for computational codes, which we call Stochastic Robust Extrapolation based Error Quantification (StREEQ). Unlike the prevalent Grid Convergence Index (GCI) [1] method, our approach is suitable for both stochastic and deterministic computational codes and is generalizable to any number of discretization variables. Building on ideas introduced in the Robust Verification [2] approach, we estimate the converged solution and orders of convergence with uncertainty using multiple fits of a discretization error model. In contrast to Robust Verification, we perform these fits to many bootstrap samples yielding a larger set of predictions with smoother statistics. Here, bootstrap resampling is performed on the lack-of-fit errors for deterministic code responses, and directly on the noisy data set for stochastic responses. This approach lends a degree of robustness to the overall results, capable of yielding precise verification results for sufficiently resolved data sets, and appropriately expanding the uncertainty when the data set does not support a precise result. For stochastic responses, a credibility assessment is also performed to give the analyst an indication of the trustworthiness of the results. Furthermore, this approach is suitable for both code and solution verification, and is particularly useful for solution verification of high-consequence simulations..
97 MATHEMATICS AND COMPUTING↗
SPIDARman: System-Level Physics-Informed Detection of Anomalies in Reactor Collected Data Considering Human Errors
In nuclear power plants (NPPs), anomalies arising from sensors or human errors (HEs) can undermine the performance and reliability of plant operations. Anomaly detection models can be employed to detect sensor errors and HEs. Additionally, physics-informed machine learning models can utilize the known physics of the system, as described by mathematical equations, to ensure that sensor values are consistent with physical laws. Hence, we propose SPIDARman: System-level Physics-Informed Detection of Anomalies in Reactor Collected Data Considering Human Errors, a holistic physics-informed anomaly detection approach based on generative adversarial networks (GANs) to detect anomalies in both automatically collected sensor data and manually collected surveillance data. Here we test our approach on data collected from a flow loop testbed, showcasing its potential to detect anomalies. Results demonstrate that the proposed model performs better than the baseline GAN-based models in detecting sensor and surveillance anomalies, suggesting the potential of physics-informed anomaly detection GAN models in NPPs.
46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
Determining circuit model parameters from operation data for PV system degradation analysis: $\mathrm{PVPRO}$
Physics-based circuit parameters like series and shunt resistance are essential to provide insights into the degradation status of photovoltaic (PV) arrays. However, calculating these parameters typically requires a full current-voltage characteristic (I-V curve), the acquisition of which involves specific measurement devices and costly methods. Thus, I-V curves of the PV system level are often not available. Here this paper proposes a methodology (PVPRO) to estimate these I-V curve parameters using only operation (string-level DC voltage and current) and weather data (irradiance and temperature). PVPRO first performs multi-stage data pre-processing to remove noisy data. Next, the time-series DC data are used to fit an equivalent circuit single-diode model (SDM) to estimate the circuit parameters by minimizing the differences between the measured and estimated values. In this way, the time evolutions of the SDM parameters are obtained. We evaluate PVPRO on synthetic datasets and find an excellent estimation of both SDM and the key I-V parameters (e.g., open-circuit voltage, short-circuit current, maximum power, etc.) with an average relative error of 0.55%. The performance, especially the extracted degradation rate of parameters, is robust to various measurement noises and the presence of faults. In addition, PVPRO is applied to a 271 kW PV field system. The relative error between the real and estimated operation voltage and current is less than 1%, suggesting that degradation trends are well captured. PVPRO represents a promising open-source tool to extract the time-series degradation trends of key PV parameters from routine operation data.
14 SOLAR ENERGY↗
Universal Optimization and Tighter Fidelity Bounds for Approximate Quantum Error Correction
Approximate quantum error correction (AQEC) not only dictates the performance of discrete- and continuous-variable quantum error correction codes but also serves as a unifying framework across various physical disciplines. Identifying the optimal recovery channel to maximize the entanglement fidelity via standard semidefinite programming is computationally bottlenecked by the exponentially growing number of Kraus operators with system size, rendering large-scale optimization prohibitive. While analytical near-optimal maps exist, they typically work only when the Knill-Laflamme conditions are nearly satisfied. In this Letter, we establish an efficient framework by leveraging the duality between recovery and environment decoupling. This framework yields a tighter analytical lower bound on entanglement fidelity than the conventional limit set by the transpose channel. Furthermore, by exploiting the decayed weights of noise Kraus operators, we introduce a framework based on principal component analysis to reduce the dimension. In thermal loss channels where the weights decay exponentially, our approach achieves a 33-fold computational speedup while maintaining rigorous accuracy. Our approach enables high-precision optimization for AQEC codes that were previously intractable due to the curse of dimensionality.
Wu, Jing [Fermilab] (ORCID:0000000249460732)↗
Assessing Cognitive Impacts of Errors from Machine Learning and Deep Learning Models: Final Report
Due to their recent increases in performance, machine learning and deep learning models are being increasingly adopted across many domains for visual processing tasks. One such domain is international nuclear safeguards, which seeks to verify the peaceful use of commercial nuclear energy across the globe. Despite recent impressive performance results from machine learning and deep learning algorithms, there is always at least some small level of error. Given the significant consequences of international nuclear safeguards conclusions, we sought to characterize how incorrect responses from a machine or deep learning-assisted visual search task would cognitively impact users. We found that not only do some types of model errors have larger negative impacts on human performance than other errors, the scale of those impacts change depending on the accuracy of the model with which they are presented and they persist in scenarios of evenly distributed errors and single-error presentations. Further, we found that experiments conducted using a common visual search dataset from the psychology community has similar implications to a safeguards- relevant dataset of images containing hyperboloid cooling towers when the cooling tower images are presented to expert participants. While novice performance was considerably different (and worse) on the cooling tower task, we saw increased novice reliance on the most challenging cooling tower images compared to experts. These findings are relevant not just to the cognitive science community, but also for developers of machine and deep learning that will be implemented in multiple domains. For safeguards, this research provides key insights into how machine and deep learning projects should be implemented considering their special requirements that information not be missed.
97 MATHEMATICS AND COMPUTING↗
Experimental Analysis of the Effects of Simulator Complexity on Human Performance
Human Reliability Analysis predicts accidents caused by human errors and is an important factor in Probabilistic Safety Assessment that comprehensively evaluates the safety of nuclear power plants. This study compares and analyzes the human performance of nuclear power plant operators according to the simulator complexity as part of the HRA data collection support method development project conducted by Idaho National Laboratory. This experiment was conducted by setting two types of simulators and scenarios as independent variables. The data collected by conducting the experiments in two different simulators were analyzed using an analysis of variance test and a correlation analysis, and four human performance charts were derived.
99 GENERAL AND MISCELLANEOUS↗
Analysis of Integrated Vapor Transport Biases
Abstract High-fidelity analyses and forecasts of integrated vapor transport (VT) are central to the study of Earth’s hydrological cycle as well as high-impact phenomena such as monsoons and atmospheric rivers. The impact of the in-line analysis correction-based additive inflation (ACAI) on IVT biases and forecast errors is examined within the Navy Earth System Prediction Capability (Navy ESPC) global coupled system. The ACAI technique uses atmospheric analysis corrections from the data assimilation system to approximate model bias and as a representation of stochastic model error to simultaneously reduce systematic and random errors and improve ensemble performance. ACAI reduces the global average magnitude of the 7- and 14-day IVT bias by 16%–17% during Northern Hemisphere summer, reaching 70% reductions in some tropical regions. The global average IVT bias reduction is similar to the bias reduction for low-level wind speed bias and considerably smaller than the bias reduction in total precipitable water. The localized regions where ACAI increases IVT bias occur where the control IVT biases change sign and structure with increasing forecast lead time, such as the South Asian monsoon region. Substituting analyzed wind or moisture fields for the forecast fields when calculating the forecast IVT confirms that, on average, wind errors dominate the IVT error calculation in the tropics, although wind and moisture error contributions are comparable in the extratropics. The existence of regions where using either analyzed winds or analyzed moisture increases IVT bias or mean absolute error reveals areas with compensating errors.
Reynolds, Carolyn A.↗
JWST Noise Floor. I. Random Error Sources in JWST NIRCam Time Series
James Webb Space Telescope (JWST) transmission and emission spectra will provide invaluable glimpses of transiting exoplanet atmospheres, including possible biosignatures. This promising science from JWST, however, will require exquisite precision and understanding of systematic errors that can impact the time series of planets crossing in front of and behind their host stars. Here, we provide estimates of the random noise sources affecting JWST Near-Infrared Camera (NIRCam) time-series data on the integration-to-integration level. We find that 1/f noise can limit the precision of grism time series for two groups (230–1000 ppm depending on the extraction method and extraction parameters), but will average down like the square root of N frames/reads. The current NIRCam grism time-series mode is especially affected by 1/f noise because its GRISMR dispersion direction is parallel to the detector fast-read direction, but could be alleviated in the GRISMC direction. Care should be taken to include as many frames as possible per visit to reduce this 1/f noise source: thus, we recommend the smallest detector subarray sizes one can tolerate, four output channels, and readout modes that minimize the number of skipped frames (RAPID or BRIGHT2). We also describe a covariance-weighting scheme that can significantly lower the contributions from 1/f noise as compared to sum extraction. We evaluate the noise introduced by pre-amplifier offsets, random telegraph noise, and high dark current resistor capacitor (RC) pixels and find that these are correctable below 10 ppm once background subtraction and pixel masking are performed. We explore systematic error sources in a companion paper.
79 ASTRONOMY AND ASTROPHYSICS↗
Computational screening of fly ash zeolite sorbents for boric acid removal
In the United States, many impoundments at coal-fired power plants contain elevated contaminants like arsenic, boron, barium, and selenium. Zeolites synthesized from fly ash show promise as sorbents for these contaminants. However, optimizing sorption capacity is challenging due to numerous possible topologies, silicon to aluminum (Si/Al) ratios, and cation types. In this study, molecular simulations are used to design cationic zeolites for boric acid adsorption. Force field models based on quantum mechanical calculations (PBE + D2) for Na-, Ca-, Mn-, and Fe-exchanged chabazite and LTA are presented. The new D2FF force fields reproduce DFT energies with about half the error of UFF. Zeolite performance depends on Si/Al ratio and cation type, with low Si/Al ratio chabazite (CHA) and phillipsite (PHI) zeolite frameworks exchanged with Ca 2+ or Na + /Ca 2+ mixtures showing the highest adsorption. In conclusion, these findings suggest tailored fly ash-derived zeolites could provide effective boron removal from leachate ponds.
CCR impoundment↗
Uncertainty quantification of a physics-informed model based on sparse identification of a Thermal Energy Distribution System
Integrated energy systems (IES)s are crucial for enhancing the economy and efficiency of power generation sources (e.g., nuclear energy) necessary to unleash American energy dominance. These systems can be integrated with thermal energy storage (TES) and intermittent renewable energies to optimize overall energy use, peak-load regulation, and demand-side responses. However, the stabilization of energy generation, transport, and utilization introduces operational complexities that exceed the challenges of managing each sub-component individually. Currently, though IESs rely on human operators for efficiency and stability, reducing human error risk and enhancing performance through automation is highly desirable. Recent advances at Idaho National Laboratory have demonstrated successful control of the Thermal Energy Distributed System (TEDS). However, the automatic control system depends on a deterministic Sparse Identification of Nonlinear Dynamics with Control (SINDyC) model, which are trained based on simulation data from physics-based simulations. Because of uncertainties in physics-based simulation, SINDyC model results in large discrepancies against experimental data and cannot be reliably used in automatic control. In this paper, we present an innovative approach to address these discrepancies by quantifying uncertainties and developing a more robust model. We first generated trajectories by using first-principles physics codes to encapsulate the experiment. Next, we trained thousands of models by randomly sampling these trajectories. We then collapsed all those models into one probabilistic SINDyC by fitting a multivariate Gaussian distribution onto the resulting coefficient’s distribution. Despite its simplicity, our approach successfully produced 95% confidence intervals that captured the experimental trajectories. It even did so with a higher probability and better U-pooling score across six of the seven relevant quantities of interest (QoIs), as compared to other classical approaches. In conclusion, ongoing research is focusing on generating new experimental trajectories to validate this approach, and on employing Bayesian calibration to refine parametric uncertainties and guide future model development efforts.
22 - GENERAL STUDIES OF NUCLEAR REACTORS↗
High order interpolation of magnetic fields with vector potential reconstruction for particle simulations
We propose a method for interpolating divergence-free continuous magnetic fields via vector potential reconstruction using Hermite interpolation, which ensures high-order continuity for applications requiring adaptive, high-order ordinary differential equation (ODE) integrators, such as the Dormand-Prince method. The method provides C(m) continuity and achieves high-order accuracy, making it particularly suited for particle trajectory integration and Poincaré section analysis under optimal integration order and timestep adjustments. Through numerical experiments, we demonstrate that the Hermite interpolation method preserves volume and continuity, which are critical for conserving toroidal canonical momentum and magnetic moment in guiding center simulations, especially over long-term trajectory integration. Furthermore, we analyze the impact of insufficient derivative continuity on Runge-Kutta schemes and show how it degrades accuracy at low error tolerances, introducing discontinuity-induced truncation errors. Lastly, we demonstrate performant Poincaré section analysis in two relevant settings of field data collocated from finite element meshes.
70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
Super-resolution surface slope metrology of x-ray mirrors
We present experimental, analytical, and numerical methods developed for reconstruction (deconvolution) of one-dimensional (1D) surface slope profiles over the spatial frequency range where the raw data are significantly perturbed due to the limited resolution of the measurement instrument. We characterize the spatial resolution properties of a profiler with the instrument's transfer function (ITF). To precisely measure the ITF, we apply a recently developed method utilizing test surfaces with 1D linear chirped height profiles of constant slope amplitude. Based on the results of the ITF calibration, we determine parameters of an analytical model for the ITF that is used in the original reconstruction software. Here, we treat surface slope metrology data obtained with the Optical Surface Measuring System (OSMS), using as a sensor an electronic autocollimator (AC) ELCOMAT-3000. The spatial resolution of the OSMS is limited by the size of the AC light-beam-collimating aperture. For the purposes of this investigation, the OSMS is equipped with a circular aperture with a diameter of 2.5 mm. This is a typical arrangement of most AC-based slope profilers developed for surface slope metrology of state-of-the-art x-ray mirrors. Using the example of surface slope metrology of two state-of-the-art elliptically shaped x-ray focusing mirrors, we demonstrate that the developed data reconstruction procedure allows us to significantly improve the accuracy of surface slope metrology with the OSMS over the spatial wavelength range from ~1.6 mm to 7 mm. Thus, the amplitude of the quasi-periodic error characteristic of the deterministic polishing process used appears to be higher by a factor of~2 than is apparent from the rough metrology data. Underestimation of the surface slope errors in this spatial wavelength range can lead to serious errors in the expected performance of x-ray mirrors in synchrotron beamlines, especially at modern light sources utilizing coherent x rays, where the perturbations can lead to increased speckle-like intensity variation.
47 OTHER INSTRUMENTATION↗
An evaluation for geometries, formation enthalpies, and dissociation energies of diatomic and triatomic (C, H, N, O), NO 3 , and HNO 3 molecules from the PAW DFT method with PBE and optB88-vdW functionals
The structural geometries, formation enthalpies, and dissociation energies of all diatomic and triatomic molecules consisting of the four basic elements C, H, N, and/or O are calculated using the projector augmented wave density functional theory (DFT) method with the Perdew–Burke–Ernzerhof and optB88-vdW exchange-correlation functionals. The calculations are also extended to two larger molecules NO 3 and HNO 3 , which consist of four and five atoms, respectively. In total, 82 molecules or isomers are considered in the calculations. The geometric parameters including 42 bond lengths and 15 bond angles of these molecules calculated using the planewave DFT method are highly satisfactory, relative to the available experimental data. The error analysis is also performed for 49 formation enthalpies and 138 dissociation energies (including 51 atomization energies as well as the corresponding bond dissociation energies). The results are also compared with the previous data from various atomic-orbital-based methods for molecules and from similar or different planewave DFT methods for various solids and other molecules. This provides an informative and instructive evaluation especially for calculating the large-size material systems containing these small molecules as well as for developing the DFT methods further.
74 ATOMIC AND MOLECULAR PHYSICS↗
Robust two-colour pyrometry uncertainty analysis to acquire spatially-resolved measurements
Two-colour pyrometry (2CP) has been used over several decades to study engine-relevant combustion processes, but results are generally regarded as qualitative or semi-quantitative. In many current 2CP systems, large measurement errors are introduced by parallax because the two measured wavelengths are not from the same line of sight. Here this work presents a spatially-resolved 2CP system with the objective of quantifying and reducing measurement uncertainty. An optical setup that eliminates parallax in 2CP is used together with pixel-by-pixel calibration of the camera sensor to increase measurement accuracy. Primary uncertainty terms are identified, and an error propagation analysis is performed to compute uncertainties in the final results of soot temperature, soot concentration parameter, KL, and soot mass. These methodologies are applied to investigate an auto-igniting fuel spray in a constant pressure flow rig at diesel-like conditions of high ambient pressure and temperature. Results show bias uncertainty of around 200 K (≈10%) for temperature and about 40%–60% for KL. High uncertainty was found to occur on the diffusion flame front where both optical thickness and soot concentrations are small. However, these uncertain measurement zones with relatively low soot concentrations contribute minimally to the total soot mass present in the reacting jet during the temporal evolution of the flame.
42 ENGINEERING↗
Selecting durable building envelope systems with machine learning assisted hygrothermal simulations database
Hygrothermal simulations provide insight into the energy performance and moisture durability of building envelope components under dynamic conditions. The inputs required for hygrothermal simulations are extensive, and carrying out simulations and analyses requires expert knowledge. An expert system, the Building Science Advisor (BSA), has been developed to predict the performance and select the energy-efficient and durable building envelope systems for different climates. The BSA consists of decision rules based on expert opinions and thousands of parametric simulation results for selected wall systems. The number of potential wall systems results in millions, too many to simulate all of them. We present how machine learning can help predict durability data, such as mold growth, while minimizing the number of simulations needed to run. The simulation results are used for training and validation of machine learning tools for predicting wall durability. We tested Artificial Neural Network (ANN) and Gradient Boosted Decision Trees (GBDT) for their applicability and model accuracy. Models developed with both methods showed adequate prediction performance (root mean square error of 0.195 and 0.209, respectively). Finally, we introduce how the information supports guidance for envelope design via an easy-to-use web-based tool that does not require the end-user to run hygrothermal simulations.
Salonvaara, Mikael↗
Dark Energy Survey Year 3 results: Curved-sky weak lensing mass map reconstruction
ABSTRACT We present reconstructed convergence maps, mass maps, from the Dark Energy Survey (DES) third year (Y3) weak gravitational lensing data set. The mass maps are weighted projections of the density field (primarily dark matter) in the foreground of the observed galaxies. We use four reconstruction methods, each is a maximum a posteriori estimate with a different model for the prior probability of the map: Kaiser–Squires, null B-mode prior, Gaussian prior, and a sparsity prior. All methods are implemented on the celestial sphere to accommodate the large sky coverage of the DES Y3 data. We compare the methods using realistic ΛCDM simulations with mock data that are closely matched to the DES Y3 data. We quantify the performance of the methods at the map level and then apply the reconstruction methods to the DES Y3 data, performing tests for systematic error effects. The maps are compared with optical foreground cosmic-web structures and are used to evaluate the lensing signal from cosmic-void profiles. The recovered dark matter map covers the largest sky fraction of any galaxy weak lensing map to date.
79 ASTRONOMY AND ASTROPHYSICS↗
Automated label flows for excited states of correlation functions in lattice gauge theory
Extracting excited states from lattice gauge theory correlation functions can be achieved through $χ^2$ minimization fits or algebraic approaches such as the variational method and Prony's method. Performing any kind of error analysis often leads to overlapping confidence regions of model parameters, even when the spectrum is not particularly dense. To correctly estimate errors, one must beware of mislabeling the states. In this work, we provide an algorithm that we call automated label flows which consistently and systematically identifies a deterministic labeling of states. This is a black-box approach in the sense that it gives a sensible set of labels without user guidance. Further, as an example, we pair one black box method with another, analyzing a lattice correlation function from real data using automated label flows in the context of Prony's method.
72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗