Results for “error bounds” — Page 45

DOE OSTI2024

Effects of machine learning errors on human decision-making: manipulations of model accuracy, error types, and error importance

Abstract This study addressed the cognitive impacts of providing correct and incorrect machine learning (ML) outputs in support of an object detection task. The study consisted of five experiments that manipulated the accuracy and importance of mock ML outputs. In each of the experiments, participants were given the T and L task with T-shaped targets and L-shaped distractors. They were tasked with categorizing each image as target present or target absent. In Experiment 1, they performed this task without the aid of ML outputs. In Experiments 2–5, they were shown images with bounding boxes, representing the output of an ML model. The outputs could be correct (hits and correct rejections), or they could be erroneous (false alarms and misses). Experiment 2 manipulated the overall accuracy of these mock ML outputs. Experiment 3 manipulated the proportion of different types of errors. Experiments 4 and 5 manipulated the importance of specific types of stimuli or model errors, as well as the framing of the task in terms of human or model performance. These experiments showed that model misses were consistently harder for participants to detect than model false alarms. In general, as the model’s performance increased, human performance increased as well, but in many cases the participants were more likely to overlook model errors when the model had high accuracy overall. Warning participants to be on the lookout for specific types of model errors had very little impact on their performance. Overall, our results emphasize the importance of considering human cognition when determining what level of model performance and types of model errors are acceptable for a given task.

97 MATHEMATICS AND COMPUTING↗

NASA NTRS1966

Parameter optimization in systems subject to worst /bounded/ disturbance.

Parameter optimization in systems subject to worst bounded disturbance that maximizes chosen performance index

STEEPEST DESCENT METHOD↗

DOE OSTI2026

Determining Optimal Magnetometer Configuration on MAGIS-100

Long-baseline atom interferometers such as the Matter-wave Atomic Gradiometer Interferometric Sensor (MAGIS-100) require stringent control and continuous characterization of background magnetic fields and spatial gradients to prevent systemic phase shifts that mimic ultralight dark matter or gravitational wave signatures. Because direct sensor placement within the ultra-high vacuum beam pipe is infeasible, in-situ magnetic field monitoring relies on external sensor arrays situated in the surrounding annular region. This work demonstrates a field reconstruction framework for a 5.3-meter MAGIS-100 modular section using finite-element Opera simulations. Transverse magnetic fields are expanded using a cylindrical multipole framework as informed by Fermilab’s Muon g-2 experiment, with magnetometer array configurations optimized via Fisher information matrix D-optimality. Inverting external sensor readings through a Gauss-Newton scheme recovers interior tube fields across distinct axial positions. In the discontinuity-averse uniform region (slice pair P4), the model achieves sub-noise-floor performance with a cross-validated root-mean-square error (RMSE) of $6.7227 \times 10^{-4}\text{ A/m}$ ($0.845\times$ sensor noise floor) and an interior field coefficient of variation of $1.71\%$. An elbow criterion in the Fisher bounds establishes $n_{\text{max}} = 2$ as the optimal multipole truncation order to prevent noise amplification from over-parameterization, with $n_{\text{max}} = 3$ (sextupole) order chosen for analysis to demonstrate further complexity and cross-pair comparison. Furthermore, analytical differentiation of the fitted multipole coefficients yields dense spatial maps of the transverse Jacobian gradient matrix $\nabla \mathbf{H}$ along with propagated $1\sigma$ uncertainty bounds across the beam region ($r \le 2.75\text{ in}$). This operational framework confirms that external magnetometer arrays can reliably monitor magnetic field uniformity and spatial gradients along the 100-meter flight path given appropriate sampling for any complexity order.

Appleby, Darwin [William Rainey Harper Coll.] (ORC↗

DOE OSTI2026

Determining Optimal Magnetometer Configuration on MAGIS-100

Long-baseline atom interferometers such as the Matter-wave Atomic Gradiometer Interferometric Sensor (MAGIS-100) require stringent control and continuous characterization of background magnetic fields and spatial gradients to prevent systemic phase shifts that mimic ultralight dark matter or gravitational wave signatures. Because direct sensor placement within the ultra-high vacuum beam pipe is infeasible, in-situ magnetic field monitoring relies on external sensor arrays situated in the surrounding annular region. This work demonstrates a field reconstruction framework for a 5.3-meter MAGIS-100 modular section using finite-element Opera simulations. Transverse magnetic fields are expanded using a cylindrical multipole framework as informed by Fermilab’s Muon g-2 experiment, with magnetometer array configurations optimized via Fisher information matrix D-optimality. Inverting external sensor readings through a Gauss-Newton scheme recovers interior tube fields across distinct axial positions. In the discontinuity-averse uniform region (slice pair P4), the model achieves sub-noise-floor performance with a cross-validated root-mean-square error (RMSE) of $6.7227 \times 10^{-4}\text{ A/m}$ ($0.845\times$ sensor noise floor) and an interior field coefficient of variation of $1.71\%$. An elbow criterion in the Fisher bounds establishes $n_{\text{max}} = 2$ as the optimal multipole truncation order to prevent noise amplification from over-parameterization, with $n_{\text{max}} = 3$ (sextupole) order chosen for analysis to demonstrate further complexity and cross-pair comparison. Furthermore, analytical differentiation of the fitted multipole coefficients yields dense spatial maps of the transverse Jacobian gradient matrix $\nabla \mathbf{H}$ along with propagated $1\sigma$ uncertainty bounds across the beam region ($r \le 2.75\text{ in}$). This operational framework confirms that external magnetometer arrays can reliably monitor magnetic field uniformity and spatial gradients along the 100-meter flight path given appropriate sampling for any complexity order.

Appleby, Darwin [William Rainey Harper Coll.] (ORC↗

NASA NTRS1977

Analytic studies of local-severe-storm observables by satellites

Attention is concentrated on the exceptionally violet whirlwind, often characterized by a fairly vertical axis of rotation. For a cylindrical polar coordinate system with axis coincident with the axis of rotation, the secondary flow involves the radial and axial velocity components. The thesis advanced is, first, that a violent whirlwind is characterized by swirl speeds relative to the axis of rotation on the order of 90 m/s, with 100 m/s being close to an upper bound. This estimate is based on interpretation of funnel-cloud shape (which also suggests properties of the radial profile of swirl, as well as the maximum magnitude); an error assessment of the funnel-cloud interpretation procedure is developed. Second, computation of ground-level pressure deficits achievable from typical tornado-spawning ambients by idealized thermohydrostatic processes suggests that a two-cell structure is required to sustain such large speeds.

Dergarabedian, P.↗

NASA NTRS1977

The local interstellar helium density

During the Apollo-Soyuz Test Project, an extreme-ultraviolet telescope observed the region of the sky from which the interstellar medium approaches the sun. The instrument had a tin filter whose bandpass included the 584-A line of neutral helium. The observations set an upper bound for the number density of neutral helium in the local interstellar medium of 0.004 + or - 0.0022 per cu cm, which is significantly lower than previously reported. The stated error is dominated by the present uncertainty in the solar 584-A flux, which is taken to be approximately 200 million photons/sq cm per sec at earth's orbit. The result is not consistent with previous resonant-scattering Ly-alpha observations of interstellar hydrogen passing through the solar system, which generally yield a hydrogen number density of about 0.1 per cu cm, unless interstellar helium is locally depleted below its cosmic abundance. The result is consistent with cosmic abundances if the much lower average hydrogen densities inferred from recent column-density measurements to nearby hot stars are in fact representative of the immediate solar neighborhood.

Freeman, J.↗

NASA NTRS1988

Multicolor pyrometer for materials processing in space

The program goals are to design, construct, and program a prototype imaging pyrometer capable of measuring the temperature distribution across the surface of a moving object suspended in space. The approach is to utilize an optical system which operates at short wavelengths compared to the peak of the blackbody spectrum for the temperature range of interest, thus minimizing errors associated with a lack of knowledge about heated sample emissivity. An analysis of the system's temperature measurement capability based on the camera's responsivity was performed and bound to be satisfactory. Details for the evolving optical design and the progress towards construction of a working model are discussed. Details of the algorithm developed for selecting the optimum colors to be used by the pyrometer are reported. Though final selection of the colors will have to await a final design of the optical system, results using a preliminary optical design are presented.

Frish, Michael↗

NASA NTRS2005

Performance Analysis on Fault Tolerant Control System

In a fault tolerant control (FTC) system, a parameter varying FTC law is reconfigured based on fault parameters estimated by fault detection and isolation (FDI) modules. FDI modules require some time to detect fault occurrences in aero-vehicle dynamics. In this paper, an FTC analysis framework is provided to calculate the upper bound of an induced-L(sub 2) norm of an FTC system with existence of false identification and detection time delay. The upper bound is written as a function of a fault detection time and exponential decay rates and has been used to determine which FTC law produces less performance degradation (tracking error) due to false identification. The analysis framework is applied for an FTC system of a HiMAT (Highly Maneuverable Aircraft Technology) vehicle. Index Terms fault tolerant control system, linear parameter varying system, HiMAT vehicle.

Shin, Jong-Yeob↗

NASA NTRS2014

Extreme Thunderstorms as Seen by Satellite

Extreme events by their nature fall outside the bounds of routine experience. With imperfect or ambiguous measuring systems, it is appropriate to question whether an unusual measurement represents an extreme event or is the result of instrument errors or other sources of noise. About three weeks after the Tropical Rainfall Measuring Mission (TRMM) satellite began collecting data in Dec 1997, a thunderstorm was observed over northern Argentina with 85 GHz brightness temperatures below 50 K and 37 GHz brightness temperatures below 70 K (Zipser et al. 2006). These values are well below what had previously been observed from satellite sensors with lower resolution. The 37 GHz brightness temperatures are also well below those measured by TRMM for any other storm in the subsequent 16 years. Without corroborating evidence, it would be natural to suspect a problem with the instrument, or perhaps an irregularity with the platform during the first weeks of the satellite mission. But the TRMM satellite also carries a radar and a lightning sensor, both confirming the presence of an intense thunderstorm. The radar recorded 40+ dBZ (decibels relative to Z) reflectivity up to about 19 km altitude. More than 200 lightning flashes per minute were recorded. That same storm's 19 GHz brightness temperatures below 150 K would normally be interpreted as the result of a low-emissivity water surface (e.g., a lake, or flood waters) if not for the simultaneous measurements of such intense convection. This paper will examine records from TRMM and related satellite sensors including SSMI and AMSR-E to find the strongest signatures resulting from thunderstorms, and distinguishing those from sources of noise. The lowest brightness temperatures resulting from thunderstorms as seen by TRMM have been in Argentina in November and December. For SSMI sensors carried on five DMSP satellites examined so far, the lowest thunderstorm-related brightness temperatures have been from Argentina in November - December and from Minnesota in June-July. The Minnesota cases were associated with spotter reports of large hail, significant severe wind, and tornadoes. Those locations have the record holders for each satellite. This paper will show examples of cases with the lowest brightness temperatures, and map the locations of these and other storms with brightness temperatures nearly as low. Higher resolution data from the field program MC3E and possibly from IPHEX will be considered for context.

Cecil, Daniel J.↗

DOE OSTI2026

In-Situ Magnetic Field Reconstruction in the MAGIS-100 Experiment

Long-baseline atom interferometers such as the Matter-wave Atomic Gradiometer Interferometric Sensor (MAGIS-100) require stringent control and continuous characterization of background magnetic fields and spatial gradients to prevent systemic phase shifts that mimic ultralight dark matter or gravitational wave signatures. Because direct sensor placement within the ultra-high vacuum beam pipe is infeasible, in-situ magnetic field monitoring relies on external sensor arrays situated in the surrounding annular region. This work demonstrates a field reconstruction framework for a 5.3-meter MAGIS-100 modular section using finite-element Opera simulations. Transverse magnetic fields are expanded using a cylindrical multipole framework as informed by Fermilab’s Muon g-2 experiment, with magnetometer array configurations optimized via Fisher information matrix D-optimality. Inverting external sensor readings through a Gauss-Newton scheme recovers interior tube fields across distinct axial positions. In the discontinuity-averse uniform region (slice pair P4), the model achieves sub-noise-floor performance with a cross-validated root-mean-square error (RMSE) of $6.7227 \times 10^{-4}\text{ A/m}$ ($0.845\times$ sensor noise floor) and an interior field coefficient of variation of $1.71\%$. An elbow criterion in the Fisher bounds establishes $n_{\text{max}} = 2$ as the optimal multipole truncation order to prevent noise amplification from over-parameterization, with $n_{\text{max}} = 3$ (sextupole) order chosen for analysis to demonstrate further complexity and cross-pair comparison. Furthermore, analytical differentiation of the fitted multipole coefficients yields dense spatial maps of the transverse Jacobian gradient matrix $\nabla \mathbf{H}$ along with propagated $1\sigma$ uncertainty bounds across the beam region ($r \le 2.75\text{ in}$). This operational framework confirms that external magnetometer arrays can reliably monitor magnetic field uniformity and spatial gradients along the 100-meter flight path given appropriate sampling for any complexity order.

Appleby, Darwin [William Rainey Harper Coll.; Ferm↗

DOE OSTI2025

Covariant Quantum Error-Correcting Codes with Metrological Entanglement Advantage

Here, we show that a subset of the basis for the irreducible representations of a tensor-product SU(2) rotation forms a covariant approximate quantum error-correcting code with transversal U(1) logical gates. Generalizing previous work on “thermodynamic codes” to general local spin and different irreducible representations using only properties of the angular momentum algebra, we obtain bounds on the code inaccuracy under generic noise on any known 𝑑 sites, under independent and identically distributed noise, and under heralded 𝑑-local erasures. We demonstrate that this family of codes protects a probe state with quantum Fisher information surpassing the standard quantum limit when the sensing parameter couples to the generator of the U(1) logical gate.

quantum error correction↗

NASA NTRS1986

Error control techniques for satellite and space communications

The performance of NASA Telecommand System was analyzed. A random coding approach was taken to determine the optimum code rate to use in forward error correcting (FEC) system with a fixed signal energy to noise power density ration, but no bandwidth constraint. Capacity and cutoff rates of concatened coding systems were determined. A lower bound on the minium distance growth rate between unmerged codewords was obtained for time invarient convolutional codes.

Costello, D. J., Jr.↗

NASA NTRS1993

Transient Ejector Analysis (TEA) code user's guide

A FORTRAN computer program for the semi analytic prediction of unsteady thrust augmenting ejector performance has been developed, based on a theoretical analysis for ejectors. That analysis blends classic self-similar turbulent jet descriptions with control-volume mixing region elements. Division of the ejector into an inlet, diffuser, and mixing region allowed flexibility in the modeling of the physics for each region. In particular, the inlet and diffuser analyses are simplified by a quasi-steady-analysis, justified by the assumption that pressure is the forcing function in those regions. Only the mixing region is assumed to be dominated by viscous effects. The present work provides an overview of the code structure, a description of the required input and output data file formats, and the results for a test case. Since there are limitations to the code for applications outside the bounds of the test case, the user should consider TEA as a research code (not as a production code), designed specifically as an implementation of the proposed ejector theory. Program error flags are discussed, and some diagnostic routines are presented.

Drummond, Colin K.↗

DOE OSTI2024

Complexity-calibrated benchmarks for machine learning reveal when prediction algorithms succeed and mislead

Abstract Recurrent neural networks are used to forecast time series in finance, climate, language, and from many other domains. Reservoir computers are a particularly easily trainable form of recurrent neural network. Recently, a “next-generation” reservoir computer was introduced in which the memory trace involves only a finite number of previous symbols. We explore the inherent limitations of finite-past memory traces in this intriguing proposal. A lower bound from Fano’s inequality shows that, on highly non-Markovian processes generated by large probabilistic state machines, next-generation reservoir computers with reasonably long memory traces have an error probability that is at least $$\sim 60\%$$ ∼ 60 % higher than the minimal attainable error probability in predicting the next observation. More generally, it appears that popular recurrent neural networks fall far short of optimally predicting such complex processes. These results highlight the need for a new generation of optimized recurrent neural network architectures. Alongside this finding, we present concentration-of-measure results for randomly-generated but complex processes. One conclusion is that large probabilistic state machines—specifically, large $$\epsilon$$ ϵ -machines—are key to generating challenging and structurally-unbiased stimuli for ground-truthing recurrent neural network architectures.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2026

Spatially Accelerated Winding Numbers for Curved Geometry

The generalized winding number (GWN) is a scalar field that supports robust containment queries on curved geometry, including non-watertight, overlapping, and nested boundary representations. While queries can be easily parallelized over samples, direct evaluation on parametric curves and surfaces remains costly for large and complex models. Fast, state-of-the-art GWN approaches leverage a spatial index to approximate the GWN, typically coupled with a Taylor expansion which approximates the GWN contribution for far clusters of geometric primitives. However, such methods operate only on discrete inputs such as triangle meshes and point clouds, and would introduce containment errors near boundaries if applied to curved input. We extend support for fast GWN evaluation over arbitrary collections of NURBS curves in 2D and trimmed NURBS patches in 3D via a Bounding Volume Hierarchy that stores efficiently precomputed moment data in the hierarchy nodes. When querying the hierarchy, approximations for far clusters are used alongside direct evaluation for nearby NURBS primitives, achieving sub-linear complexity while preserving the geometric features in the vicinity of the query point. Central to our performance improvements is an adaptive subdivision strategy for NURBS primitives during a preprocessing phase, creating better spatial partitions while retaining the same accuracy for containment decisions as a direct evaluation. We demonstrate the performance and accuracy of our approach across a large collection of 2D and 3D datasets.

Computer science↗

NASA NTRS1975

Worst error performance of continuous Kalman filters

The worst error performance of estimation filters is investigated for continuous systems in this paper. The pathological performance study, without assuming any dynamical model such as Markov processes for perturbations, except for its bounded amplitude, will give practical and dependable criteria in establishing the navigation and maneuver strategy in deep space missions.

Nishimura, T.↗

NASA NTRS1985

Cockpit automation - In need of a philosophy

Concern has been expressed over the rapid development and deployment of automatic devices in transport aircraft, due mainly to the human interface and particularly the role of automation in inducing human error. The paper discusses the need for coherent philosophies of automation, and proposes several approaches: (1) flight management by exception, which states that as long as a crew stays within the bounds of regulations, air traffic control and flight safety, it may fly as it sees fit; (2) exceptions by forecasting, where the use of forecasting models would predict boundary penetration, rather than waiting for it to happen; (3) goal-sharing, where a computer is informed of overall goals, and subsequently has the capability of checking inputs and aircraft position for consistency with the overall goal or intentions; and (4) artificial intelligence and expert systems, where intelligent machines could mimic human reason.

Wiener, E. L.↗

NASA NTRS1990

Computational efficiency of parallel combinatorial OR-tree searches

The performance of parallel combinatorial OR-tree searches is analytically evaluated. This performance depends on the complexity of the problem to be solved, the error allowance function, the dominance relation, and the search strategies. The exact performance may be difficult to predict due to the nondeterminism and anomalies of parallelism. The authors derive the performance bounds of parallel OR-tree searches with respect to the best-first, depth-first, and breadth-first strategies, and verify these bounds by simulation. They show that a near-linear speedup can be achieved with respect to a large number of processors for parallel OR-tree searches. Using the bounds developed, the authors derive sufficient conditions for assuring that parallelism will not degrade performance and necessary conditions for allowing parallelism to have a speedup greater than the ratio of the numbers of processors. These bounds and conditions provide the theoretical foundation for determining the number of processors required to assure a near-linear speedup.

Li, Guo-Jie↗