Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “error metric”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 343 records · Page 19
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

Safeguards Modeling for Advanced Nuclear Facility Design.

Future nuclear fuel cycle facilities will see a significant benefit from considering materials accountancy requirements early in the design process. The Material Protection, Accounting, and Control Technologies (MPACT) working group is demonstrating Safeguards and Security by Design (SSBD) for a notional electrochemical reprocessing facility as part of a 2020 Milestone. The idea behind SSBD is to consider regulatory requirements early in the design process to provide more optimized systems and avoid costly retrofits later in the design process. Safeguards modeling, using single analyst tools, allows the designer to efficiently consider materials accountancy approaches that meet regulatory requirements. However, safeguards modeling also allows the facility designer to go beyond current regulations and work toward accountancy designs with rapid response and lower thresholds for detection of anomalies. This type of modeling enables new safeguards approaches and may inform future regulatory changes. The Separation and Safeguards Performance Model (SSPM) has been used for materials accountancy system design and analysis. This paper steps through the process of designing a Material Control and Accountancy (MC&A) system, presents the baseline system design for an electrochemical reprocessing facility, and provides performance metrics from the modeling analysis. The most critical measurements in the electrochemical facility are the spent fuel input, electrorefiner salt, and U/TRU product output measurements. Finally, material loss scenario analysis found that measurement uncertainties (relative standard deviations) for Pu would need to be at 1% (random and systematic error components) or better in order to meet domestic detection goals or as high as 3% in order to meet international detection goals, based on a 100 metric ton per year plant size.

98 NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL P↗
NASA NTRS1982

A study of image quality for radar image processing

Methods developed for image quality metrics are reviewed with focus on basic interpretation or recognition elements including: tone or color; shape; pattern; size; shadow; texture; site; association or context; and resolution. Seven metrics are believed to show promise as a way of characterizing the quality of an image: (1) the dynamic range of intensities in the displayed image; (2) the system signal-to-noise ratio; (3) the system spatial bandwidth or bandpass; (4) the system resolution or acutance; (5) the normalized-mean-square-error as a measure of geometric fidelity; (6) the perceptual mean square error; and (7) the radar threshold quality factor. Selective levels of degradation are being applied to simulated synthetic radar images to test the validity of these metrics.

King, R. W.↗
NASA NTRS2020

Observing System Simulation Experiments Investigating Atmospheric Motion Vectors and Radiances from a Constellation of 4-5 μm Infrared Sounders

A set of Observing System Simulation Experiments (OSSEs) was performed to investigate the utility of a constellation of passive infrared spectrometers, strategically designed with the aim of deriving the three-dimensional retrievals of the horizontal wind via atmospheric motion vectors (AMVs) from instruments with the spectral resolution of an infrared sounder. The instrument and constellation designs were performed in the context of the Midwave Infrared Sounding of Temperature and humidity in a Constellation for Winds, or MISTiC Winds. The Global Modeling and Assimilation Office OSSE system, which includes a full suite of operational meteorological observations, served as the control. To illustrate the potential impact of this observing strategy, two experiments were performed by adding the new simulated observations to the control. First, perfect (error-free) simulated AMVs and radiances were assimilated. Second, the data were made imperfect by adding realistic modeled errors to the AMVs and radiances that were assimilated. The experimentation showed beneficial impacts on both the mass and wind fields, as based on analysis verification, forecast verification, and the assessment of the observations using the Forecast Sensitivity to Observation Impact (FSOI) metric. In all variables and metrics, the impacts of the imperfect observations were smaller than those of the perfect observations, though much of the positive benefit was retained. The FSOI metric illustrated two key points. First, the largest impacts were seen in the middle troposphere AMVs, which is a targeted capability of the constellation strategy. Second, the addition of modeled errors showed that the assimilation system was unable to fully exploit the 4.3 μm carbon dioxide absorption radiances.

Will McCarty↗
NASA NTRS2021

Observing System Simulation Experiments Investigating Atmospheric Motion Vectors and Radiances from a Constellation of 4–5-μm Infrared Sounders

A set of observing system simulation experiments (OSSEs) was performed to investigate the utility of a constellation of passive infrared spectrometers, strategically designed with the aim of deriving the three-dimensional retrievals of the horizontal wind via atmospheric motion vectors (AMVs) from instruments with the spectral resolution of an infrared sounder. The instrument and constellation designs were performed in the context of the Midwave Infrared Sounding of Temperature and humidity in a Constellation for Winds (MISTiC Winds). The Global Modeling and Assimilation Office OSSE system, which includes a full suite of operational meteorological observations, served as the control. To illustrate the potential impact of this observing strategy, two experiments were performed by adding the new simulated observations to the control. First, perfect (error free) simulated AMVs and radiances were assimilated. Second, the data were made imperfect by adding realistic modeled errors to the AMVs and radiances that were assimilated. The experimentation showed beneficial impacts on both the mass and wind fields, as based on analysis verification, forecast verification, and the assessment of the observations using the forecast sensitivity to observation impact (FSOI) metric. In all variables and metrics, the impacts of the imperfect observations were smaller than those of the perfect observations, although much of the positive benefit was retained. The FSOI metric illustrated two key points. First, the largest impacts were seen in the middle troposphere AMVs, which is a targeted capability of the constellation strategy. Second, the addition of modeled errors showed that the assimilation system was unable to fully exploit the 4.3-μm carbon dioxide absorption radiances.

Will McCarty↗
DOE OSTI2021

Evaluation of pressure reconstruction techniques for Model Order Reduction in incompressible convective heat transfer

This paper compares pressure reconstruction strategies in Model Order Reduction for incompressible flows with convective heat transfer. The Navier-Stokes equation are reduced along with the passive scalar transport equation for the temperature using the POD-Galerkin technique. Six different pressure reconstruction methods are evaluated, two of which are novel to the best of the authors’ knowledge. Accurate pressure reconstruction is key to avoid error buildup in when solving for the conservation of linear momentum at the reduced level. The six approaches are compared using Direct Numerical Simulations of convective heat exchange processes in a 3D Backward Facing Step with a heated cylinder. Additionally, when comparing time-averaged metrics, we observe that the reconstruction methods that approximate the reduced pressure field using techniques borrowed from full order models (mechanical analogy, pressure Poisson, and velocity supremizers) yield higher errors than the methods that seek to stabilize the reduced systems (reduced residual stabilization, artificial divergence, and Uzawa operator).

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
NASA NTRS1974

Upper bounds on sequential decoding performance parameters

This paper presents the best obtainable random coding and expurgated upper bounds on the probabilities of undetectable error, of t-order failure (advance to depth t into an incorrect subset), and of likelihood rise in the incorrect subset, applicable to sequential decoding when the metric bias G is arbitrary. Upper bounds on the Pareto exponent are also presented. The G-values optimizing each of the parameters of interest are determined, and are shown to lie in intervals that in general have nonzero widths. The G-optimal expurgated bound on undetectable error is shown to agree with that for maximum likelihood decoding of convolutional codes, and that on failure agrees with the block code expurgated bound. Included are curves evaluating the bounds for interesting choices of G and SNR for a binary-input quantized-output Gaussian additive noise channel.

Jelinek, F.↗
NASA NTRS1995

Orbit determination of highly elliptical Earth orbiters using improved Doppler data-processing modes

A navigation error covariance analysis of four highly elliptical Earth orbits is described, with apogee heights ranging from 20,000 to 76,800 km and perigee heights ranging from 1,000 to 5,000 km. This analysis differs from earlier studies in that improved navigation data-processing modes were used to reduce the radio metric data. For this study, X-band (8.4-GHz) Doppler data were assumed to be acquired from two Deep Space Network radio antennas and reconstructed orbit errors propagated over a single day. Doppler measurements were formulated as total-count phase measurements and compared to the traditional formulation of differenced-count frequency measurements. In addition, an enhanced data-filtering strategy was used, which treated the principal ground system calibration errors affecting the data as filter parameters. Results suggest that a 40- to 60-percent accuracy improvement may be achievable over traditional data-processing modes in reconstructed orbit errors, with a substantial reduction in reconstructed velocity errors at perigee. Historically, this has been a regime in which stringent navigation requirements have been difficult to meet by conventional methods.

Estefan, J. A.↗
DOE OSTI2025

AAPM Truth‐based CT (TrueCT) reconstruction grand challenge

Background: This Special Report summarizes the 2022, AAPM grand challenge on Truth-based CT image reconstruction. Purpose: To provide an objective framework for evaluating CT reconstruction methods using virtual imaging resources consisting of a library of simulated CT projection images of a population of human models with various diseases. Methods: Two hundred unique anthropomorphic, computational models were created with varied diseases consisting of 67 emphysema, 67 lung lesions, and 66 liver lesions. The organs were modeled based on clinical CT images of real patients. The emphysematous regions were modeled using segmentations from patient CT cases in the COPDGene Phase I dataset. For the lung and liver lesion cases, 1–6 malignant lesions were created and inserted into the human models, with lesion diameters ranging from 5.6 to 21.9 mm for lung lesions and 3.9 to 14.9 mm for liver lesions. The contrast defined between the liver lesions and liver parenchyma was 82 ± 12 HU, ranging from 50 to 110 HU. Similarly, the contrast between the lung lesions and the lung parenchyma was defined as 781 ± 11 HU, ranging from 725 to 805 HU. For the emphysematous regions, the defined HU values were −950 ± 17 HU ranging from −918 to −979 HU. The developed human models were imaged with a validated CT simulator. The resulting CT sinograms were shared with the participants. The participants reconstructed CT images from the sinograms and sent back their reconstructed images. Further, the reconstructed images were then scored by comparing the results against the corresponding ground truth values. The scores included both task-generic (root mean square error [RMSE] and structural similarity matrix [SSIM]), and task-specific (detectability index [d’] and lesion volume accuracy) metrics. For the cases with multiple lesions, the measured metric was averaged across all the lesions. To combine the metrics with each other, each metric was normalized to a range of 0 to 1 per disease type, with “0” and “1” being the worst and best measured values across all cases of the disease type for all received reconstructions. Results: The True-CT challenge attracted 52 participants, out of which 5 successfully completed the challenge and submitted the requested 200 reconstructions. Across all participants and disease types, SSIM absolute values ranged from 0.22 to 0.90, RMSE from 77.6 to 490.5 HU, d’ from 0.1 to 64.6, and volume accuracy ranged from 1.2 to 753.1 mm3. The overall scores demonstrated that participant “A” had the best performance in all categories, except for the metrics of d’ for lung lesions and RMSE for liver lesions. Participant “A” had an average normalized score of 0.41 ± 0.22, 0.48 ± 0.32, and 0.42 ± 0.33 for the emphysema, lung lesion, and liver lesion cases, respectively. Conclusions: The True-CT challenge successfully enabled objective assessment of CT reconstructions with the unique advantage of access to a diverse population of diseased human models with known ground truth. This study highlights the significant potential of virtual imaging trials in objective assessment of medical imaging technologies.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2020

A control-oriented combustion model framework for compression ignition engines operating on low-reactivity fuel

This work focuses on zero-dimensional modeling of the heat release rate in a compression ignition engine operating on gasoline-like fuels. Due to the properties of gasoline, such as high volatility and longer ignition delay than diesel, the injection strategies can vary significantly from the operation with conventional diesel fuel. Different injection strategies are commonly used to achieve varying degrees of in-cylinder stratification in order to shape the combustion event and maximize efficiency. The proposed zero-dimensional combustion model was developed to account for the different stages in combustion caused by the fuel stratification. As the ignition delay model is an integral part of the entire combustion process and significantly affects the prediction accuracy, special attention has been paid to local phenomena influencing ignition delay. A one-dimensional spray model by Musculus and Kattke was employed in conjunction with a Lagrangian tracking approach in order to estimate the local air–fuel ratio within the spray tip, as a proxy for reactivity. The local air–fuel ratio, in-cylinder temperature and pressure were used in an integral fashion to estimate the ignition delay. Heat release rates were modeled using first-order non-linear differential equations. The proposed combustion model was validated against experimental data of a heavy-duty compression ignition engine with up to three injection events at mostly 1038 r/min and 14 bar brake mean effective pressure. Further validation of the model was carried out at other engine loads and speeds. Model prediction errors in CA50 of less than 1 °CA across all conditions were found. Modeling results of other combustion metrics such as combustion duration and indicated mean effective pressure are also highly satisfactory. In addition, the model has been shown to be capable of estimating the ringing intensity for most conditions.

Pamminger, Michael↗
NASA NTRS1985

Strategies for FH/MFSK signaling with diversity in worst-case partial-band noise

Optimum diversity and worst-case partial-band noise jamming conditions have been derived for noncoherent energy detection of frequency-hopped (FH) M-ary frequency-shift keyed (MFSK) signals using a soft-chip decision suboptimum linear combining metric with perfect jamming-state side information. However, the assumption implicit in previous publications is that the error is first maximized over the jammer's partial-band duty factor for arbitrary diversity, and the result is then minimized over the amount of diversity. This paper shows that if the order of optimization is reversed, different conditions and performance are produced; that is, the previous solution is not a saddlepoint. This introduces some game-theoretic considerations for the communicator and the jammer, the risks and advantages of which are explored.

Levitt, B. K.↗
NASA NTRS1994

Enhanced orbit determination filter: Inclusion of ground system errors as filter parameters

The theoretical aspects of an orbit determination filter that incorporates ground-system error sources as model parameters for use in interplanetary navigation are presented in this article. This filter, which is derived from sequential filtering theory, allows a systematic treatment of errors in calibrations of transmission media, station locations, and earth orientation models associated with ground-based radio metric data, in addition to the modeling of the spacecraft dynamics. The discussion includes a mathematical description of the filter and an analytical comparison of its characteristics with more traditional filtering techniques used in this application. The analysis in this article shows that this filter has the potential to generate navigation products of substantially greater accuracy than more traditional filtering procedures.

Masters, W. C.↗
NASA NTRS2014

Strain Gage Load Calibration of the Wing Interface Fittings for the Adaptive Compliant Trailing Edge Flap Flight Test

The safety-of-flight parameters for the Adaptive Compliant Trailing Edge (ACTE) flap experiment require that flap-to-wing interface loads be sensed and monitored in real time to ensure that the structural load limits of the wing are not exceeded. This paper discusses the strain gage load calibration testing and load equation derivation methodology for the ACTE interface fittings. Both the left and right wing flap interfaces were monitored; each contained four uniquely designed and instrumented flap interface fittings. The interface hardware design and instrumentation layout are discussed. Twenty-one applied test load cases were developed using the predicted in-flight loads. Pre-test predictions of strain gage responses were produced using finite element method models of the interface fittings. Predicted and measured test strains are presented. A load testing rig and three hydraulic jacks were used to apply combinations of shear, bending, and axial loads to the interface fittings. Hardware deflections under load were measured using photogrammetry and transducers. Due to deflections in the interface fitting hardware and test rig, finite element model techniques were used to calculate the reaction loads throughout the applied load range, taking into account the elastically-deformed geometry. The primary load equations were selected based on multiple calibration metrics. An independent set of validation cases was used to validate each derived equation. The 2-sigma residual errors for the shear loads were less than eight percent of the full-scale calibration load; the 2-sigma residual errors for the bending moment loads were less than three percent of the full-scale calibration load. The derived load equations for shear, bending, and axial loads are presented, with the calculated errors for both the calibration cases and the independent validation load cases.

Miller, Eric J.↗
NASA NTRSDate not supplied

Error Localization Examples: Looking for a Needle in a Hay-stack

Finite element models (FEM) are routinely developed and used during fabrication of high dollar-value hardware. NASA as part of the pre-flight certification of launch vehicles routinely conducts vibration and static tests to calibrate models used for flight-risk assessments. As part of the calibration process, certain areas in the model are modified, using engineering judgment and sensitivity analysis, to match the test results. Unfortunately, tools to identify problem areas in the FEM using test data directly are scarce. Over the years, Error Localization Algorithms (ELA) have been proposed with very limited success. Recently, the Analytical Dynamics Model Improvement (ADMI) algorithm, which computes closed-form mass and stiffness corrections to match the test data exactly, have been shown to be effective for error localization. The paper will present several FEM example problems where ELA is used with simulated test data to determine FEM problem areas. For each example, the correct answer is shown along with ELA results. It is shown that the ELA process is able to identify general problem areas in the FEM, which are consistent with known model perturbations. However, in most cases the ELA identified area of improvement is larger than the true answer. Nonetheless, with proper optimization tools, calibration results using the ELA identified areas provide excellent results.

error localization↗
NASA NTRS2022

Robustness of Vegetation Optical Depth Retrievals Based on L-Band Global Radiometry

Microwave vegetation optical depth (VOD) and soil moisture (SM) can be simultaneously retrieved based on L-band radiometry with polarization information. VOD is indicative of the vegetation water content (VWC) because it captures the extinction of land surface emission. If the connectivity of VOD to VWC is robust, the pair of VWC-SM observations can be viable bases for understanding soil–plant–atmosphere water relations, providing new perspectives on ecosystem science. Simultaneous SM–VOD retrievals are feasible by inverting the τ−ω model with two independent datasets in dual-channel algorithms. However, given correlated satellite vertical and horizontal brightness temperatures (TBs; TB v and TB h ), an ill-posed inverse problem arises where TB errors result in high uncertainties of retrievals. In this study, we apply the degrees-of-information (DoI) metric and propose a signal-to-noise ratio (SNR) metric to assess the “retrievability” of VOD given the Soil Moisture Active Passive (SMAP) TB v –TB h linear dependence. The application of these metrics allows determining where the VOD retrievals are robust and reliable. This is a necessary step in supporting the applications of VOD in ecology and hydrology. Results show that regions with mainly nonwoody vegetation have the best potential for VOD retrievals, though regularization is necessary. We then assess VOD time variations from two regularization products that reduce the impact of underdetermined inversions: the L3 dual-channel algorithm (L3-DCA) and the multitemporal dual-channel algorithm (MTDCA), which constrain VOD time dynamics with and without using a priori VOD climatology, respectively. Though they both reduce noise, especially in the VOD retrievals, they result in differences in VOD seasonal amplitude and coupling to SM at high frequencies as we outline here.

Microwave↗
DOE OSTI2021

Solution Irregularity Remediation for Spatial Discretization Error Estimation for S N Transport Solutions

The discrete ordinates linear Boltzmann transport equation is typically solved in its spatially discretized form, incurring spatial discretization error. Quantification of this error for purposes such as adaptive mesh refinement or error analysis requires an a posteriori estimator, which utilizes the numerical solution to the spatially discretized equation to compute an estimate. Because the quality of the numerical solution informs the error estimate, irregularities, present in the true solution for any realistic problem configuration, tend to cause the largest deviation in the error estimate vis-a-vis the true error. In this paper, an analytical partial singular characteristic tracking (pSCT) procedure for reducing the estimator’s error is implemented within our novel residual source estimator for a zeroth-order discontinuous Galerkin scheme, at the additional cost of a single inner iteration. Here, a metric-based evaluation of the pSCT scheme versus the standard residual source estimator is performed over the parameter range of a Method of Manufactured Solutions test suite. The pSCT scheme generates near-ideal accuracy in the estimate in problems where the dominant source of the estimator’s error is the solution irregularity, namely, problems where the true solution is discontinuous and problems where the true solution’s first derivative is discontinuous and the scattering ratio is low. In problems where the scattering ratio is high and the true solution is discontinuous in the first derivative, the error in the scattering source, which is not converged by the pSCT scheme, is greater than the error incurred due to the irregularity. Ultimately, a pSCT scheme is judged to be useful for error estimation in problems where the computational cost of the scheme is justified. In the presence of many irregularities, such a scheme may be intractable for general use, but in benchmarks, as an analytical tool, or in problems that have nondissipative discontinuities, the scheme may prove invaluable.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2021

Extreme metrics from large ensembles: investigating the effects of ensemble size on their estimates

Abstract. We consider the problem of estimating the ensemble sizes required to characterize the forced component and the internal variability of a number of extreme metrics. While we exploit existing large ensembles, our perspective is that of a modeling center wanting to estimate a priori such sizes on the basis of an existing small ensemble (we assume the availability of only five members here). We therefore ask if such a small-size ensemble is sufficient to estimate accurately the population variance (i.e., the ensemble internal variability) and then apply a well-established formula that quantifies the expected error in the estimation of the population mean (i.e., the forced component) as a function of the sample size n, here taken to mean the ensemble size. We find that indeed we can anticipate errors in the estimation of the forced component for temperature and precipitation extremes as a function of n by plugging into the formula an estimate of the population variance derived on the basis of five members. For a range of spatial and temporal scales, forcing levels (we use simulations under Representative Concentration Pathway 8.5) and two models considered here as our proof of concept, it appears that an ensemble size of 20 or 25 members can provide estimates of the forced component for the extreme metrics considered that remain within small absolute and percentage errors. Additional members beyond 20 or 25 add only marginal precision to the estimate, and this remains true when statistical inference through extreme value analysis is used. We then ask about the ensemble size required to estimate the ensemble variance (a measure of internal variability) along the length of the simulation and – importantly – about the ensemble size required to detect significant changes in such variance along the simulation with increased external forcings. Using the F test, we find that estimates on the basis of only 5 or 10 ensemble members accurately represent the full ensemble variance even when the analysis is conducted at the grid-point scale. The detection of changes in the variance when comparing different times along the simulation, especially for the precipitation-based metrics, requires larger sizes but not larger than 15 or 20 members. While we recognize that there will always exist applications and metric definitions requiring larger statistical power and therefore ensemble sizes, our results suggest that for a wide range of analysis targets and scales an effective estimate of both forced component and internal variability can be achieved with sizes below 30 members. This invites consideration of the possibility of exploring additional sources of uncertainty, such as physics parameter settings, when designing ensemble simulations.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS1971

Metric analysis of minitrack optical and interferometer data

The network analysis program (NAP-2), which has the capability of simultaneously solving for orbits and tracking station error model terms, was, after several modifications, used in the calibration of minitrack stations using minitrack satellite measurements (self-calibration). Several support programs were written to aid in this task. A simultaneous four-arc solution was obtained. A comparison with optically determined arcs for the same time spans showed rms position differences of 67m, 86m, 124m and 168m for the 4 arcs considered. An apparatus incorporating a diffraction grating was designed and successfully used to measure the drift rate of the Fort Myers MOTS camera drive.

Brown, D. C.↗
NASA NTRS2010

Normalized Point Source Sensitivity for Off-Axis Optical Performance Evaluation of the Thirty Meter Telescope

The Normalized Point Source Sensitivity (PSSN) has previously been defined and analyzed as an On-Axis seeing-limited telescope performance metric. In this paper, we expand the scope of the PSSN definition to include Off-Axis field of view (FoV) points and apply this generalized metric for performance evaluation of the Thirty Meter Telescope (TMT). We first propose various possible choices for the PSSN definition and select one as our baseline. We show that our baseline metric has useful properties including the multiplicative feature even when considering Off-Axis FoV points, which has proven to be useful for optimizing the telescope error budget. Various TMT optical errors are considered for the performance evaluation including segment alignment and phasing, segment surface figures, temperature, and gravity, whose On-Axis PSSN values have previously been published by our group.

optical performance metric↗