Estimating Errors in GLM Detection Efficiency and False Alarm Rate
Explore the source record for details and available documents.
Katrina S Virts↗
SEARCH · Engineering Papers
Results for “error rate”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Automated contouring error detection based on supervised geometric attribute distribution models for radiation therapy: A general strategy
Purpose: One of the most critical steps in radiation therapy treatment is accurate tumor and critical organ-at-risk (OAR) contouring. Both manual and automated contouring processes are prone to errors and to a large degree of inter- and intraobserver variability. These are often due to the limitations of imaging techniques in visualizing human anatomy as well as to inherent anatomical variability among individuals. Physicians/physicists have to reverify all the radiation therapy contours of every patient before using them for treatment planning, which is tedious, laborious, and still not an error-free process. In this study, the authors developed a general strategy based on novel geometric attribute distribution (GAD) models to automatically detect radiation therapy OAR contouring errors and facilitate the current clinical workflow. Methods: Considering the radiation therapy structures’ geometric attributes (centroid, volume, and shape), the spatial relationship of neighboring structures, as well as anatomical similarity of individual contours among patients, the authors established GAD models to characterize the interstructural centroid and volume variations, and the intrastructural shape variations of each individual structure. The GAD models are scalable and deformable, and constrained by their respective principal attribute variations calculated from training sets with verified OAR contours. A new iterative weighted GAD model-fitting algorithm was developed for contouring error detection. Receiver operating characteristic (ROC) analysis was employed in a unique way to optimize the model parameters to satisfy clinical requirements. A total of forty-four head-and-neck patient cases, each of which includes nine critical OAR contours, were utilized to demonstrate the proposed strategy. Twenty-nine out of these forty-four patient cases were utilized to train the inter- and intrastructural GAD models. These training data and the remaining fifteen testing data sets were separately employed to test the effectiveness of the proposed contouring error detection strategy. Results: An evaluation tool was implemented to illustrate how the proposed strategy automatically detects the radiation therapy contouring errors for a given patient and provides 3D graphical visualization of error detection results as well. The contouring error detection results were achieved with an average sensitivity of 0.954/0.906 and an average specificity of 0.901/0.909 on the centroid/volume related contouring errors of all the tested samples. As for the detection results on structural shape related contouring errors, an average sensitivity of 0.816 and an average specificity of 0.94 on all the tested samples were obtained. The promising results indicated the feasibility of the proposed strategy for the detection of contouring errors with low false detection rate. Conclusions: The proposed strategy can reliably identify contouring errors based upon inter- and intrastructural constraints derived from clinically approved contours. It holds great potential for improving the radiation therapy workflow. ROC and box plot analyses allow for analytically tuning of the system parameters to satisfy clinical requirements. Future work will focus on the improvement of strategy reliability by utilizing more training sets and additional geometric attribute constraints.
60 APPLIED LIFE SCIENCES↗
Comanche Helmet-Mounted Display Symbology Simulation
The Aeroflightdynamics Directorate (AMRDEC) conducted a simulation to examine the performance of the Comanche Contact Analog world-referenced symbology displayed on the Comanche HIDSS when compared with a compressed symbology design similar to that specified by the former MIL-STD 1295. Six experimental test pilots flew one modified ADS-33 maneuver (hover turn, bob-up), an unusual attitude recovery, and two terrain flight tactical tasks in the NASA Vertical Motion Simulator (VMS). Analysis of the pilot objective performance data and subjective data showed the following results. Objective test results showed that 1295 symbology yielded more rapid maneuvering in the hover turn bob-up than Contact Analog symbology. The average margin of difference in the time to complete the maneuver was approximately two seconds, which was statistically significant. There were no significant differences measured between symbology sets with respect to altitude or position performance measures for all other maneuvers. The NOE target ID task data showed improved accuracy in determining heading to target when using Contact Analog over MIL-STD-1295. Subjective test results, including handling qualities ratings (HQRs) and NASA-TLX workload ratings, showed small but consistent advantages of 1295 symbology over Contact Analog for most parameters. For the bob-up maneuver, 1295 symbology handling qualities were rated Desired for lateral position error and time to complete whereas Contact Analog was rated adequate. The average HQRs for all other maneuvers were rated the same for both symbology sets. Pilot comments and the results of an online questionnaire more strongly favored 1295 over Contact Analog. Repeated comments from all six pilots led to a focus on design issues with six Contact Analog symbols. Those symbols were the heading tape, horizon line, radar altitude six-second predictor, the position of the torque symbol, the absence of a hover position cue, and the widespread positioning of the symbology to the outer edges of the display field-of-view. Pilots rated the present design of three of the six symbols as having safety-of-flight implications. Those symbols were the heading tape, horizon line, and six-second predictor. To summarize, test results showed no objective data that would warrant restricting experimental test pilots from flying constrained tasks. Small but consistent advantages were recorded for the MILSTD-1295 symbology design when executing ADS-33 constrained tasks. No general performance differences were recorded for the operational maneuvers (NOE, contour flight modes) except for the azimuth-to-target task, which favored the Contact Analog heading tape design. There were consistent and strong pilot comments supporting the MIL-STD-1295 design over Contact Analog in both this current simulation and in Comanche Sim I. Three Contact Analog symbols warrant modification and further evaluation to mitigate safety-of-flight implications noted by participating test pilots. A plan for symbology redesign and testing was developed. However, the Comanche program was cancelled by Department of Army in February 2004.
Turpin, Terry S.↗
Numerical modeling based machine learning approach for the optimization of falling - film evaporator in thermal desalination application
Scale formation that drastically increases thermal resistance and reduces freshwater production remains a critical challenge in thermal desalination. Novel designs of falling film evaporator and optimal operating condition hold great promise to mitigate scale formation, and increase heat transfer performance and fresh water production. In this work, CFD simulation based machine learning and multi-objective optimization are performed to identify optimal conditions and tube arrangement for evaporator. Non-dominated sorting genetic algorithm is adopted to determine and analyze the optimal pareto front for multiple objectives in desalination criteria. The errors of training, validation, and testing set are computed to identify an optimal hyperparameter set. For performance ratio, fouling resistance, and water production rate, the average relative error is 2.26%, 3.67%, and 3.24%. At pareto front, both performance ratio and water production rate increase at high temperature with fouling resistance (thermal resistance of the fouling layer) increasing as well. Tradeoffs between mitigating scale formation and enhancing desalination performance are evaluated in optimizations for different objectives. Finally, potential optima are identified and can be applied as guidelines to determine evaporator design and system operating conditions.
42 ENGINEERING↗
The Importance of Semi-Major Axis Knowledge in the Determination of Near-Circular Orbits
Modem orbit determination has mostly been accomplished using Cartesian coordinates. This usage has carried over in recent years to the use of GPS for satellite orbit determination. The unprecedented positioning accuracy of GPS has tended to focus attention more on the system's capability to locate the spacecraft's location at a particular epoch than on its accuracy in determination of the orbit, per se. As is well-known, the latter depends on a coordinated knowledge of position, velocity, and the correlation between their errors. Failure to determine a properly coordinated position/velocity state vector at a given epoch can lead to an epoch state that does not propagate well, and/or may not be usable for the execution of orbit adjustment maneuvers. For the quite common case of near-circular orbits, the degree to which position and velocity estimates are properly coordinated is largely captured by the error in semi-major axis (SMA) they jointly produce. Figure 1 depicts the relationships among radius error, speed error, and their correlation which exist for a typical low altitude Earth orbit. Two familiar consequences are the relationship Figure 1 shows are the following: (1) downrange position error grows at the per orbit rate of 3(pi) times the SMA error; (2) a velocity change imparted to the orbit will have an error of (pi) divided by the orbit period times the SMA error. A less familiar consequence occurs in the problem of initializing the covariance matrix for a sequential orbit determination filter. An initial covariance consistent with orbital dynamics should be used if the covariance is to propagate well. Properly accounting for the SMA error of the initial state in the construction of the initial covariance accomplishes half of this objective, by specifying the partition of the covariance corresponding to down-track position and radial velocity errors. The remainder of the in-plane covariance partition may be specified in terms of the flight path angle error of the initial state. Figure 2 illustrates the effect of properly and not properly initializing a covariance. This figure was produced by propagating the covariance shown on the plot, without process noise, in a circular low Earth orbit whose period is 5828.5 seconds. The upper subplot, in which the proper relationships among position, velocity, and their correlation has been used, shows overall error growth, in terms of the standard deviations of the inertial position coordinates, of about half of the lower subplot, whose initial covariance was based on other considerations.
Carpenter, J. Russell↗
Simulation results for the Viterbi decoding algorithm
Concepts involved in determining the performance of coded digital communications systems are introduced. The basic concepts of convolutional encoding and decoding are summarized, and hardware implementations of sequential and maximum likelihood decoders are described briefly. Results of parametric studies of the Viterbi decoding algorithm are summarized. Bit error probability is chosen as the measure of performance and is calculated, by using digital computer simulations, for various encoder and decoder parameters. Results are presented for code rates of one-half and one-third, for constraint lengths of 4 to 8, for both hard-decision and soft-decision bit detectors, and for several important systematic and nonsystematic codes. The effect of decoder block length on bit error rate also is considered, so that a more complete estimate of the relationship between performance and decoder complexity can be made.
Batson, B. H.↗
Simultaneous 3D–2D image registration and C-arm calibration: Application to endovascular image-guided interventions
Purpose: Three-dimensional to two-dimensional (3D–2D) image registration is a key to fusion and simultaneous visualization of valuable information contained in 3D pre-interventional and 2D intra-interventional images with the final goal of image guidance of a procedure. In this paper, the authors focus on 3D–2D image registration within the context of intracranial endovascular image-guided interventions (EIGIs), where the 3D and 2D images are generally acquired with the same C-arm system. The accuracy and robustness of any 3D–2D registration method, to be used in a clinical setting, is influenced by (1) the method itself, (2) uncertainty of initial pose of the 3D image from which registration starts, (3) uncertainty of C-arm’s geometry and pose, and (4) the number of 2D intra-interventional images used for registration, which is generally one and at most two. The study of these influences requires rigorous and objective validation of any 3D–2D registration method against a highly accurate reference or “gold standard” registration, performed on clinical image datasets acquired in the context of the intervention. Methods: The registration process is split into two sequential, i.e., initial and final, registration stages. The initial stage is either machine-based or template matching. The latter aims to reduce possibly large in-plane translation errors by matching a projection of the 3D vessel model and 2D image. In the final registration stage, four state-of-the-art intrinsic image-based 3D–2D registration methods, which involve simultaneous refinement of rigid-body and C-arm parameters, are evaluated. For objective validation, the authors acquired an image database of 15 patients undergoing cerebral EIGI, for which accurate gold standard registrations were established by fiducial marker coregistration. Results: Based on target registration error, the obtained success rates of 3D to a single 2D image registration after initial machine-based and template matching and final registration involving C-arm calibration were 36%, 73%, and 93%, respectively, while registration accuracy of 0.59 mm was the best after final registration. By compensating in-plane translation errors by initial template matching, the success rates achieved after the final stage improved consistently for all methods, especially if C-arm calibration was performed simultaneously with the 3D–2D image registration. Conclusions: Because the tested methods perform simultaneous C-arm calibration and 3D–2D registration based solely on anatomical information, they have a high potential for automation and thus for an immediate integration into current interventional workflow. One of the authors’ main contributions is also comprehensive and representative validation performed under realistic conditions as encountered during cerebral EIGI.
60 APPLIED LIFE SCIENCES↗
SU-C-BRD-03: Closing the Loop On Virtual IMRT QA
Purpose: To develop an algorithm that predicts a priori IMRT QA passing rates. Methods: 416 IMRT plans from all treatment sites were planned in Eclipse version 11 and delivered using a dynamic sliding window technique on Clinac iX or TrueBeam linacs (Varian Medical Systems, Palo Alto, CA). The 3%/3mm and 2%/2mm local distance to agreement (DTA) were recorded during clinical operations using a commercial 2D diode array (MapCHECK 2, Sun Nuclear, Melbourne, FL). Each plan was characterized by 37 metrics that describe different failure modes between the calculated and measured dose. Machine-learning algorithms (MLAs) were trained to learn the relation between the plan characteristics and each passing rate. Minimization of the cross validated error, together with maximum a posteriori estimation (MAP), were used to choose the model parameters. Results: 3%/3mm local DTA can be predicted with an error smaller than 3% for 98% of the plans. For the remaining 2% of plans, the residual error was within 5%. For 2%/2mm local DTA passing rates, 96% percent of the plans were successfully predicted with an error smaller than 5%. All high-risk plans that failed the 2%/2mm local criteria were correctly identified by the algorithm. The most important metric to describe the passing rates was determined to be the MU per Gray (modulation factor). Conclusions: Logs files and independent dose calculations have been suggested as possible substitutes for measurement based IMRT QA. However, none of these methods answer the fundamental question of whether a plan can be delivered with a clinically acceptable error given the limitations of the linacs and the treatment planning system. Predicting the IMRT QA passing rates a priori closes that loop. For additional robustness, virtual IMRT QA can be combined with Linac QA and log file analysis to confirm appropriate delivery.
60 APPLIED LIFE SCIENCES↗
Bit error probability calculations for convolutional codes with short constraint lengths on very noisy channels
A technique for estimating convolutional code performance on very noisy channels is considered. Specifically, the performance of short constraint length codes operating near the channel cutoff rate is estimated. Decoding convolutional codes with a sliding window decoder (SWD) are considered. This decoder is an optimal (maximum likelihood) symbol decoder as the window size grows toward infinity, while the Viterbi decoder is the maximum-likelihood sequence estimator. The difference in the decoded BERs (bit error rates) between the two decoders is very small and approaches zero asymptotically as the channel BER decreases. Therefore, an estimate on the decoded BER for the SWD can also be used as an estimate of the decoded BER for Viterbi decoding.
Herro, Mark A.↗
Rate constants for fine-structure excitations in O-H collisions with error bars obtained by machine learning
We present an approach using a combination of coupled channel scattering calculations with a machine-learning technique based on Gaussian Process regression to determine the sensitivity of the rate constants for non-adiabatic transitions in inelastic atomic collisions to variations of the underlying adiabatic interaction potentials. Using this approach, we improve the previous computations of the rate constants for the fine-structure transitions in collisions of O({sup 3}P{sub j}) with atomic H. We compute the error bars of the rate constants corresponding to 20% variations of the ab initio potentials and show that this method can be used to determine which of the individual adiabatic potentials are more or less important for the outcome of different fine-structure changing collisions.
37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
A simple parameterization of ozone infrared absorption for atmospheric heating rate calculations
A simple parameterization of ozone absorption in the 9.6-micron region which is suitable for two- and three-dimensional stratospheric and tropospheric models is presented. The band is divided into two parts, a brand center region and a band wing region, grouping together regions for which the temperature dependence of absorption is similar. Each of the two regions is modeled with a function having the form of the Goody random model, with pressure and temperature dependent band parameters chosen by empirically fitting line-by-line equivalent widths for pressures between 0.25 and 1000 mbar and ozone absorber amounts between 1.0 x 10 to the -7th and 1.0 cm atm. The model has been applied to calculations of atmospheric heating rates using an absorber amount weighted mean pressure and temperature along the inhomogeneous paths necessary for flux computations. In the stratosphere, maximum errors in the heating rates relative to line-by-line calculations are 0.1 K/d, or 5 percent of the peak cooling at the stratopause. In the troposphere the errors are at most 0.005 K/d.
Rosenfield, Joan E.↗
Inorganic Chlorine Partitioning in the Summer Lower Stratosphere: Modeled and Measured [ClONO2/HCl] During POLARIS
We examine inorganic chlorine (Cly) partitioning in the summer lower stratosphere using in situ ER-2 aircraft observations made during the Photochemistry of Ozone Loss in the Arctic Region in Summer (POLARIS) campaign. New steady state and numerical models estimate [ClONO2]/[HCl] using currently accepted photochemistry. These models are tightly constrained by observations with OH (parameterized as a function of solar zenith angle) substituting for modeled HO2 chemistry. We find that inorganic chlorine photochemistry alone overestimates observed [ClONO2]/[HCl] by approximately 55-60% at mid and high latitudes. On the basis of POLARIS studies of the inorganic chlorine budget, [ClO]/[ClONO2], and an intercomparison with balloon observations, the most direct explanation for the model-measurement discrepancy in Cly partitioning is an error in the reactions, rate constants, and measured species concentrations linking HCl and ClO (simulated [ClO]/[HCl] too high) in combination with a possible systematic error in the ER-2 ClONO2 measurement (too low). The high precision of our simulation (+/-15% 1-sigma for [ClONO2]/[HCl], which is compared with observations) increases confidence in the observations, photolysis calculations, and laboratory rate constants. These results, along with other findings, should lead to improvements in both the accuracy and precision of stratospheric photochemical models.
Voss, P. B.↗
Nonlinear analysis of phased-locked loops with rapidly varying phase
The performance of command and telemetry systems, useful in deep-space communications, is frequently affected by the radio-frequency phase error which is introduced at the point of reception by means of the carrier tracking loop. In low data rate communications, this phase error may vary rapidly over the duration of the signaling interval. In this paper such phase variation is characterized by a sinusoidal input phase, K sin (omega sub o t+, pi/6), which models a typical phase variation in communication over turbulent media. Conditions for synchronization stability and the acquisition behavior are examined by detailed computer study of the phase-plane trajectories for the second and third-order loops with perfect integrator. It is demonstrated that for the phase variation considered the third-order loop has no real advantage over the second-order loop. Finally, it is shown that nonzero initial conditions may result in large steady-state phase error.
Chen, C.↗
Design Calculations for Thermoelectric Generators
Nine simplified analytic models based on average properties accurately predict heat rates for silicon/germanium thermoelectric generators. Solutions from simplified models were compared with those obtained using sophisticated numerical analysis. Maximum errors in calculated heat rate range from about 4 percent to about 0.2 percent. Models also used to calculate power delivered to load and thermodynamic efficiency.
Zeldin, B.↗
PREDICTION OF SOLAR FLARE SIZE AND TIME-TO-FLARE USING SUPPORT VECTOR MACHINE REGRESSION
We study the prediction of solar flare size and time-to-flare using 38 features describing magnetic complexity of the photospheric magnetic field. This work uses support vector regression to formulate a mapping from the 38-dimensional feature space to a continuous-valued label vector representing flare size or time-to-flare. When we consider flaring regions only, we find an average error in estimating flare size of approximately half a geostationary operational environmental satellite (GOES) class. When we additionally consider non-flaring regions, we find an increased average error of approximately three-fourths a GOES class. We also consider thresholding the regressed flare size for the experiment containing both flaring and non-flaring regions and find a true positive rate of 0.69 and a true negative rate of 0.86 for flare prediction. The results for both of these size regression experiments are consistent across a wide range of predictive time windows, indicating that the magnetic complexity features may be persistent in appearance long before flare activity. This is supported by our larger error rates of some 40 hr in the time-to-flare regression problem. The 38 magnetic complexity features considered here appear to have discriminative potential for flare size, but their persistence in time makes them less discriminative for the time-to-flare problem.
79 ASTRONOMY AND ASTROPHYSICS↗
Convergence Analysis of Turbulent Flow Solutions
Data from the "Turbulence Modeling Resource" website for turbulent flow over an NACA-0012 airfoil is analyzed to determine the convergence behavior of three second-order CFD (Computational Fluid Dynamics) codes: CFL3D (Computational Fluids Lab 3 Dimensional flow solver), FUN3D (Fully Unstructured Navier-stokes flow solver), and TAU (German Aerospace Center (DLR) 2 dimensional code for unstructured hybrid grids solving the Reynolds-Averaged Navier-Stokes equations or the Euler equations). The convergence of both integrated properties and pointwise data are examined. Several different methods for estimating errors and computing convergence rates are compared. A high-order extension to the Richardson extrapolation is developed that improves the accuracy of the mesh limit values and provides a quantitative estimate of the threshold of the asymptotic regime. The coefficient of total drag exhibits second-order convergence for all three codes, and convergence is monotone over a sequence of 7 grids. Other force coefficients are not so well behaved. The convergence rates of the viscous component of drag on the three nest grids ranges from 3:0 for CFL3D to 1:0 for FUN3D. The three codes are converging to similar but not identical solutions. The largest differences between the codes are in the coefficient of lift for which the difference between CFL3D and FUN3D is greater than 10 (sup minus 4). The best agreement occurs in the viscous component of drag, which is the only force component for which all three codes are converging towards each other at a rate of second-order. The agreement between the two unstructured grid codes is good with all properties except lift converging towards common values at a rate of second-order. No one code was universally better than the other. The TAU code has the lowest error in total drag, FUN3D has the lowest error in lift, and CFL3D has the lowest error in the viscous component of drag.
Atkins, Harold L.↗
Convergence properties of finite-difference hydrodynamics schemes in the presence of shocks
We investigate the asymptotic convergence of finite-difference schemes for the Euler equations when the limiting solution contains shocks. The Lax-Wendroff theorem guarantees that certain conservative schemes converge to correct, physically valid solutions. We focus on two one-dimensional operator-split schemes with explicit artificial-viscosity terms. One, an internal-energy scheme, does not satisfy the assumptions of Lax-Wendroff; the other, a conservative total-energy scheme, does. With viscous lengths chosen proportional to the grid size, we find that both schemes converge to their zero-grid-size limits at the theoretically expected rate, but only the conversative scheme converges toward correct solutions of the inviscid fluid equations. We show that the difference in their behaviors results directly from the presence of shocks in the limiting solution. Empirically, we find that when the viscous lenghts tend toward zero more slowly than the grid size, however the nonconservative scheme also converges toward correct solutions. We characterize the asymptotic behavior of the total-energy scheme in a particular problem in which a shock forms. As the grid is refined, a Cauchy error approaches the expected rate of change slowly. We show that the changes in the artificial viscosity alter the diffusion of small-amplitude waves. The differences associated with such waves make the dominant contribution to the Cauchy error. We formulate an analytic model to relate the rate of approach to the effect of varying diffusion in waves and find quantitative agreement with our numerical results.
Kimoto, Paul A.↗
A study of the viability of exploiting memory content similarity to improve resilience to memory errors
Building the next-generation of extreme-scale distributed systems will require overcoming several challenges related to system resilience. As the number of processors in these systems grow, the failure rate increases proportionally. One of the most common sources of failure in large-scale systems is memory. In this paper, we propose a novel runtime for transparently exploiting memory content similarity to improve system resilience by reducing the rate at which memory errors lead to node failure. We evaluate the viability of this approach by examining memory snapshots collected from eight high-performance computing (HPC) applications and two important HPC operating systems. Based on the characteristics of the similarity uncovered, we conclude that our proposed approach shows promise for addressing system resilience in large-scale systems.
97 MATHEMATICS AND COMPUTING↗