SEARCH · Engineering Papers

Results for “error metric”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 469 records · Page 26

All sources NASA NTRS DOE OSTI

DOE OSTI2023

Evaluating Retention Index Score Assumptions to Refine GC–MS Metabolite Identification

As metabolomics grows into a high-throughput and high demand research field, current metrics for the identification of small molecules in gas chromatography–mass spectrometry (GC–MS) still require manual verification. Though steps have been taken to improve scoring metrics by combining spectral similarity (SS) and retention index (RI), the problem persists. A large body of literature has analyzed and refined SS scores, but few studies have explicitly studied improvements to RI scores. Here, in this study, we examined whether uninvestigated assumptions of the RI score are valid and propose ways to improve them. Query RIs were matched to library RI with a generous window of ±35 to avoid unintentional removal of valid compound identifications. Each match was manually verified as a true positive (TP), true negative, or unknown. Metabolites with at least 30 TP identifications were included in downstream analyses, resulting in a total of 87 metabolites from samples of varying complexity and type (e.g., amino acid mixtures, human urine, fungal species, and so on.). Our results showed that the RI score assumptions of normality, consistent variance across metabolites, and a mean error centered at 0 are often violated. We demonstrated through a cross-validation analysis that modifying these underlying assumptions according to empirical metabolite-specific distributions improved the TP and negative rankings. Further, we statistically determined the minimum number of samples required to estimate distributional parameters for scoring metrics. Overall, this work proposes a robust statistical pipeline to reduce the time bottleneck of metabolite identification by improving RI scores and thus minimize the effort to complete manual verification.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

NASA NTRSDate not supplied

Near-Body Mesh Adaption for Transitional Flows Using OVERFLOW

Accurate modeling of boundary-layer transition is an important aspect of developing greener air transport technologies. In that regard, transition models based on auxiliary transport equations offer a robust approach that is easily integrated into the Reynolds-averaged Navier-Stokes (RANS) solvers. Recent workshops under NATO and AIAA have identified the verification of transport-equations-based transition modeling as a critical aspect of reducing the scatter between the predictions of different CFD codes. Follow-on work has highlighted the need for highly dense grids to achieve an asymptotic convergence of transition related flow metrics. The present work examines the role of automatic near-body mesh adaptation capability in the NASA OVERFLOW CFD solver to enable verification studies in an efficient manner, and for establishing best practices for designing grids for the RANS-based transition models. A sensor function relevant to the Langtry-Menter \gamma-Re_{\theta t}\ transition model has been identified and used for error-based mesh adaptation for canonical configurations comprising a flat plate, and the S809 and NLR-7301 airfoils. The efficacy of the mesh adaptation approach is assessed for flow conditions involving multiple transition scenarios such as natural transition, separation-induced transition, and shock-induced transition. The results from this exploratory study indicate that the meshes adapted using the proposed sensor provide solutions that approach the references solutions obtained with uniformly refined hand-crafted meshes, in terms of the chosen metrics, and yield modest yet significant savings in grid count. We also highlight areas for improvement in the grid adaptation methodology within OVERFLOW.

CFD↗

NASA NTRSDate not supplied

Near-Body Mesh Adaption for Transitional Flows Using OVERFLOW

CFD↗

NASA NTRS2017

Comparison of Commonly-Used Microwave Radiative Transfer Models for Snow Remote Sensing

This paper reviews four commonly-used microwave radiative transfer models that take different electromagnetic approaches to simulate snow brightness temperature (T(sub B)): the Dense Media Radiative Transfer - Multi-Layer model (DMRT-ML), the Dense Media Radiative Transfer - Quasi-Crystalline Approximation Mie scattering of Sticky spheres (DMRT-QMS), the Helsinki University of Technology n-Layers model (HUT-nlayers) and the Microwave Emission Model of Layered Snowpacks (MEMLS). Using the same extensively measured physical snowpack properties, we compared the simulated T(sub B) at 11, 19 and 37 GHz from these four models. The analysis focuses on the impact of using different types of measured snow microstructure metrics in the simulations. In addition to density, snow microstructure is defined for each snow layer by grain optical diameter (Do) and stickiness for DMRT-ML and DMRT-QMS, mean grain geometrical maximum extent (D(sub max)) for HUT n-layers and the exponential correlation length for MEMLS. These metrics were derived from either in-situ measurements of snow specific surface area (SSA) or macrophotos of grain sizes (D(sub max)), assuming non-sticky spheres for the DMRT models. Simulated T(sub B) sensitivity analysis using the same inputs shows relatively consistent T(sub B) behavior as a function of Do and density variations for the vertical polarization (maximum deviation of 18 K and 27 K, respectively), while some divergences appear in simulated variations for the polarization ratio (PR). Comparisons with ground based radiometric measurements show that the simulations based on snow SSA measurements have to be scaled with a model-specific factor of Do in order to minimize the root mean square error (RMSE) between measured and simulated T(sub B). Results using in-situ grain size measurements (SSA or D(sub max), depending on the model) give a mean T(sub B) RMSE (19 and 37 GHz) of the order of 16-26 K, which is similar for all models when the snow microstructure metrics are scaled. However, the MEMLS model converges to better results when driven by the correlation length estimated from in-situ SSA measurements rather than D(sub max) measurements. On a practical level, this paper shows that the SSA parameter, a snow property that is easy to retrieve in-situ, appears to be the most relevant parameter for characterizing snow microstructure, despite the need for a scaling factor.

Snow↗

DOE OSTI2023

Cluster characterization in atom probe tomography: Machine learning using multiple summary functions

In this work, we develop a machine learning-based method to characterize intracluster concentration (ρ c ), background concentration (ρ b ), clustering radius (r̄), and radius dispersity (δ r ) in simulated atom probe tomography data using multiple spatial statistics summary functions to train a Bayesian regularized neural network. Here, we build upon previous work that utilized Ripley’s K-function by incorporating additional features from nearest-neighbor spatial statistics summary functions to better characterize concentration-based metrics. The addition of nearest-neighbor based features allows for highly accurate estimates of ρ c and ρ b , both with 90% of the predictions within 4.0% of the real value; the root-mean-square errors are reduced by 81.5% and 92.8% from predictions using only K-function based features, respectively. Additionally, including these nearest-neighbor based features improves the ability to differentiate between r̄ and δ r .

36 MATERIALS SCIENCE↗

NASA NTRS1993

Methods of evaluating the effects of coding on SAR data

It is recognized that mean square error (MSE) is not a sufficient criterion for determining the acceptability of an image reconstructed from data that has been compressed and decompressed using an encoding algorithm. In the case of Synthetic Aperture Radar (SAR) data, it is also deemed to be insufficient to display the reconstructed image (and perhaps error image) alongside the original and make a (subjective) judgment as to the quality of the reconstructed data. In this paper we suggest a number of additional evaluation criteria which we feel should be included as evaluation metrics in SAR data encoding experiments. These criteria have been specifically chosen to provide a means of ensuring that the important information in the SAR data is preserved. The paper also presents the results of an investigation into the effects of coding on SAR data fidelity when the coding is applied in (1) the signal data domain, and (2) the image domain. An analysis of the results highlights the shortcomings of the MSE criterion, and shows which of the suggested additional criterion have been found to be most important.

Dutkiewicz, Melanie↗

NASA NTRS2023

A Blind Convolutional Deep Autoencoder for Spectral Unmixing of Hyperspectral Images Over Waterbodies

Harmful algal blooms have dangerous repercussions for biodiversity, the ecosystem, and public health. Automatic identification based on remote sensing hyperspectral image analysis provides a valuable mechanism for extracting the spectral signatures of harmful algal blooms and their respective percentage in a region of interest. This paper proposes a new model called a non-symmetrical autoencoder for spectral unmixing to perform endmember extraction and fractional abundance estimation. The model is assessed in benchmark datasets, such as Jasper Ridge and Samson. Additionally, a case study of the HSI2 image acquired by NASA over Lake Erie in 2017 is conducted for extracting optical water types. The results using the proposed model for the benchmark datasets improve unmixing performance, as indicated by the spectral angle distance compared to five baseline algorithms. Improved results were obtained for various metrics. In the Samson dataset, the proposed model outperformed other methods for water (0.060) and soil (0.025) endmember extraction. Moreover, the proposed method exhibited superior performance in terms of mean spectral angle distance compared to the other five baseline algorithms. The non-symmetrical autoencoder for the spectral unmixing approach achieved better results for abundance map estimation, with a root mean square error of 0.091 for water and 0.187 for soil, compared to the ground truth. For the Jasper Ridge dataset, the non-symmetrical autoencoder for the spectral unmixing model excelled in the tree (0.039) and road (0.068) endmember extraction and also demonstrated improved results for water abundance maps (0.1121). The proposed model can identify the presence of chlorophyll-a in waterbodies. Chlorophyll-a is an essential indicator of the presence of the different concentrations of macrophytes and cyanobacteria. The non-symmetrical autoencoder for spectral unmixing achieves a value of 0.307 for the spectral angle distance metric compared to a reference ground truth spectral signature of chlorophyll-a. The source code for the proposed model, as implemented in this manuscript, can be found at https://github.com/EstefaniaAlfaro/autoencoder_owt_spectral.git.

hyperspectral imaging↗

DOE OSTI2023

Cost-constrained adaptive simulations of transient spray combustion in a gas turbine combustor

Predictive high-fidelity simulations of turbulent spray combustion must capture the combined effects of complex chemistry, multiphase evaporating flow and spray-flame interactions to achieve physical accuracy. Finite-rate chemistry (FRC) combined with a realistic chemical mechanism is a combustion model well-suited for this purpose, but has a high computational cost due to the large number and stiffness of transported chemical species. In contrast, flamelet-based models achieve lower cost by transporting a small number of quantities of reduced stiffness, but assumptions regarding local flame topology, boundary conditions and inter-phase coupling limit their physical accuracy. Recently, the Pareto-efficient combustion (PEC) framework was developed to dynamically assign combustion models based on local cost and accuracy metrics in gas-phase reacting flows. In this work, we extend this PEC framework to spray combustion through the rigorous analysis of the multiphase coupling terms in the governing equations. The derivation shows that spray evaporation causes errors in the prediction of species mass fractions for flamelet-based models due to the sensitivity of the local thermo-chemical state to changes in composition caused by fuel vaporization across combustion regimes present in practical spray combustion devices. Sub-model assignment is formulated as a multiple-choice knapsack problem, where computational cost is directly controlled through the fraction of the domain assigned to the FRC sub-model. The extended PEC formulation is applied to the simulation of a realistic rich-quench-lean gas turbine combustor at steady-state conditions, as well as transient operation resulting in lean blow-out (LBO). Analysis of transient simulations during LBO demonstrates the extended PEC formulation’s capacity to dynamically adapt to changing conditions within the combustor. Transient combustor dynamics are shown to approach convergence with limited increases in computational cost, while retaining substantial computational cost reduction compared to monolithic FRC simulations. Through PEC simulations with increasing fractions of the domain assigned to FRC, monolithic flamelet simulations are shown to over-predict flame stability during LBO. Finally, the extended PEC formulation is thus shown to overcome deficiencies of monolithic models by controlling modeling error for multiphase combustion modeling.

33 ADVANCED PROPULSION SYSTEMS↗

DOE OSTI2026

An Integrated Framework for Memory-Centric Analysis: From Trace Collection to Co-Design

The memory wall phenomenon—where advances in processor performance significantly outpace those in memory subsystems—poses a fundamental challenge for contemporary computing systems. In memory-bound applications, memory subsystem behavior dominates performance, yet existing analysis approaches present significant limitations: detailed microarchitectural simulators require days to weeks to simulate modest workloads; hardware performance counters provide only aggregate statistics that obscure temporal and spatial access patterns; and scaled simulation approaches face challenges in capturing certain behaviors that emerge at larger scales. These limitations reflect a processor-centric design philosophy increasingly misaligned with memory-bound workloads where detailed understanding of memory access patterns, cache hierarchy interactions, and contention is critical for effective optimization. This paper presents an integrated framework for memory-centric analysis that enables effective hardware-software co-design. We describe practical trace collection techniques, including hardware-assisted processor tracing with minimal overhead and portable software-based instrumentation with statistical sampling. We present multi-perspective analysis methods that examine memory behavior from temporal, sequential, spatial, and relational viewpoints, revealing distinct optimization opportunities invisible in aggregate metrics. We detail an architectural modeling framework that uses sampled traces with temporal interpolation and confidence-based filtering to evaluate cache and memory configurations. Evaluation on representative benchmarks demonstrates that this framework achieves practical accuracy (L2 cache errors of 2.64\%, confidence-filtered L3 errors of 9.92\%, bandwidth errors of 7.33\%) while providing substantial speedup (26.8×) over cycle-accurate simulation, enabling rapid design space exploration. We demonstrate how this integrated framework enables systematic identification of both hardware optimizations (memory controller tuning, bank partitioning, NUMA configuration) and software optimizations (data layout restructuring, prefetching strategies, memory-aware scheduling). Through this comprehensive treatment of the memory-centric analysis pipeline—from trace collection through architectural modeling to co-design application—we provide researchers and practitioners with practical techniques for addressing memory bottlenecks in contemporary computing systems.

Gajaria, Dhruv Mayur↗

NASA NTRS2015

Ocean Heat and Carbon Uptake in Transient Climate Change: Identifying Model Uncertainty

Global warming on decadal and centennial timescales is mediated and ameliorated by the oceansequestering heat and carbon into its interior. Transient climate change is a function of the efficiency by whichanthropogenic heat and carbon are transported away from the surface into the ocean interior (Hansen et al. 1985).Gregory and Mitchell (1997) and Raper et al. (2002) were the first to identify the importance of the ocean heat uptakeefficiency in transient climate change. Observational estimates (Schwartz 2012) and inferences from coupledatmosphere-ocean general circulation models (AOGCMs; Gregory and Forster 2008; Marotzke et al. 2015), suggest thatocean heat uptake efficiency on decadal timescales lies in the range 0.5-1.5 W/sq m/K and is thus comparable to theclimate feedback parameter (Murphy et al. 2009). Moreover, the ocean not only plays a key role in setting the timing ofwarming but also its regional patterns (Marshall et al. 2014), which is crucial to our understanding of regional climate,carbon and heat uptake, and sea-level change. This short communication is based on a presentation given by A.Romanou at a recent workshop, Oceans Carbon and Heat Uptake: Uncertainties and Metrics, co-hosted by US CLIVARand OCB. As briefly reviewed below, we have incomplete but growing knowledge of how ocean models used in climatechange projections sequester heat and carbon into the interior. To understand and thence reduce errors and biases inthe ocean component of coupled models, as well as elucidate the key mechanisms at work, in the final section we outlinea proposed model intercomparison project named FAFMIP. In FAFMIP, coupled integrations would be carried out withprescribed overrides of wind stress and freshwater and heat fluxes acting at the sea surface.

periodic variations↗

NASA NTRS2022

Validation of IMERG Oceanic Precipitation over Kwajalein

The integrated Multi-satellitE Retrievals for GPM (IMERG) Version V05B and V06B precipitation products from the Global Precipitation Measurement (GPM) mission are validated against ground-based observations from the Kwajalein Polarimetric S-band Weather Radar (KPOL) deployed at Kwajalein Atoll in the central Pacific Ocean. Such a validation is particularly important as comprehensive surface measurements over the oceans are practically infeasible, which hampers the identification of possible errors, and improvement of future versions of IMERG and other satellitebased retrieval algorithms. The V05B and V06B IMERG products are validated at their native 0.1°, 30 min resolution from 2014 to 2018 based on both volumetric and categorical metrics. This validation study indicates that precipitation rates from both IMERG V05B and V06B are underestimated with respect to radar surface estimates, but the underestimation is much reduced from V05B IMERG V06B outperforms V05B with reduced systematic bias and improved precipitation detectability. The IMERG performance is further traced back to its individual sensors and morphing-based algorithms. The overall underestimation in V05B is mainly driven by the negative relative biases from morphing-based algorithms which are largely corrected in V06B. Imagers perform generally better than sounders because of the usage of low-frequency channels in imagers which can better detect emission signals by the hydrometeors. Among imagers, the GPM Microwave Imager (GMI) and Advanced Microwave Scanning Radiometer Version 2 (AMSR2) are the best, followed by Special Sensor Microwave Imager/Sounder (SSMIS). Among sounders, the Microwave Humidity Sounder (MHS) is the best, followed by Advanced Technology Microwave Sounder (ATMS) and the Sounder for Atmospheric Profiling of Humidity in the Intertropics by Radiometry (SAPHIR) for V06B. Among all categories, morph-only and IR+morph only perform better than SAPHIR. SAPHIR shows the worst performance among all categories, likely due to its limited channel selection. It is envisaged that these results will improve our understanding of IMERG performance over oceans and aid in the improvement of future versions of IMERG.

KPOL↗

NASA NTRS2011

Reliability Analysis and Standardization of Spacecraft Command Generation Processes

center dot In order to reduce commanding errors that are caused by humans, we create an approach and corresponding artifacts for standardizing the command generation process and conducting risk management during the design and assurance of such processes. center dot The literature review conducted during the standardization process revealed that very few atomic level human activities are associated with even a broad set of missions. center dot Applicable human reliability metrics for performing these atomic level tasks are available. center dot The process for building a "Periodic Table" of Command and Control Functions as well as Probabilistic Risk Assessment (PRA) models is demonstrated. center dot The PRA models are executed using data from human reliability data banks. center dot The Periodic Table is related to the PRA models via Fault Links.

human errors.↗

DOE OSTI2023

An automated and portable method for selecting an optimal GPU frequency

Power consumption poses a significant challenge in current and emerging graphics processing unit (GPU) enabled high-performance computing systems. In modern GPUs, dynamic voltage frequency scaling (DVFS) appears to be a reliable control to regulate power consumption and performance. However, the DVFS design space is large - hence, brute-force approaches are infeasible to select the optimal frequency. Furthermore, no single frequency can be universally optimal for applications with varying computational intensities. Thus, the application's complexity and the availability of a wide range of frequency settings are a challenge in selecting the optimal frequency configuration for a given GPU workload. To that end, this paper proposes a systematic approach that consists of three steps. The feature characterization study identifies the fine-grain GPU utilization metrics that influence the power consumption and execution time of a given workload. To understand the performance, power, and energy consumption behaviors of a workload across GPU's DVFS design space, we derived analytical power and performance models using the identified fine-grain features. Here, it is shown that the same set of GPU utilization metrics can estimate both the power consumption and execution time while being agnostic of changes to frequency and input sizes. Applying a power control with the single objective of reducing power may cause performance degradation, leading to more energy consumption. A multi-objective approach is proposed to select the optimal GPU DVFS configuration for a workload that reduces power consumption with negligible degradation in performance. The evaluation was conducted using SPEC ACCEL benchmarks and three real applications - NAMD LAMMPS, and LSTM on NVIDIA GV100, GA100, and AMD MI210 GPUs. On average, real applications showed 29.6% energy savings with a performance loss of 5.2% on GA100 and 22.6% energy savings with a performance loss of 4.7% on GV100. Moreover, the proposed models are portable to real applications, GPU architectures, and vendors, and require metric collection at only the default frequency rather than all supported DVFS configurations. Additionally, we conducted a comparison between our models and the GPU assembly instructions (PTX)-based static models. The results revealed a significant reduction in the average error rates, with a decrease from 19.7% to 3.1% for power models and from 29.4% to 5.2% for performance models.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2021

Validation of Jezebel Reactivity Coefficients and Sensitivity Analysis

Nuclear data validation is often performed today using criticality measurements. The gold standard for criticality measurements is the International Criticality Safety Benchmark Experiment Project (ICSBEP). The validation specifically focuses on the effective multiplication factor ($k_{eff}$). $K_{eff}$ is a relatively easy parameter to infer and has reduced uncertainty due to being at or above critical. However, while $k_{eff}$ is the most documented parameter and its uncertainties and sensitivities have been evaluated in great detail, it cannot be used as a standalone metric to determine inaccuracies in nuclear data (e.g., cross section data, PFNS, nu), which is based on theory, physics, and differential measurements. The Experiments Underpinned by Computational Learning for Improvements in nuclear Data (EUCLID) project aims to identify compensating errors in specific isotope nuclear data by optimally designing experiments that are, or are not sensitive to a suite of measurement parameters beyond $k_{eff}$. By identifying parameters that are sensitive to each other, oppositely sensitive, or have substantial magnitude differences in sensitivity, experiments can be designed to constrain questionable nuclear data. One sensitivity that is of particular interest to this project includes the sensitivity of reactivity coefficients. Reactivity coefficients compare reactivity, which is related to $k_{eff}$ at two different states therefore being sensitive to small changes in the system. The most common type of reactivity coefficient measurements is comparison to void for a small sample within the assembly. It is key that the sample sizes are small enough to not affect the flux of the full system. Reactivity coefficients were evaluated for many early experiments to better understand transport corrected cross sections. In fact, ICSBEP includes reactivity coefficient results as “Supplemental Measurements” in appendices for a handful of older benchmarks. One of those benchmarks is Jezebel, the bare Pu critical assembly. This paper compares new simulations of reactivity coefficients for Jezebel, and explores the sensitivity of reactivity coefficients to small changes in nuclear data.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

DOE OSTI2025

SOC Microstructural Property Estimator

This pre-trained ML model is a tool that uses basic compositional parameters for porous solid oxide cell (SOC) electrodes - the phase fractions and mean particle/pore diameters – as inputs and uses them to estimate additional electrochemical performance parameters: active (i.e., connected) TPB density, all tortuosity factors, and phase pair specific interfacial areas. The electrode is assumed to be composed of two solid phases and a pore phase. The property calculations are performed using neural network regression models trained on a large bank of synthetic electrode microstructural data that NETL has generated using the program DREAM3D (that bank is also hosted on EDX: https://edx.netl.doe.gov/dataset/soc-synthetic-microstructure-bank). This means the generated parameters are based on training from actual measured properties from 3D microstructures, not estimated from geometric simplifications. This tool was developed and is intended to replace percolation theory calculations in models that use hypothetical electrode properties. An example use case would be running SOC performance simulations across a parametric sweep of electrode designs (e.g., varying phase fractions and particle sizes) and assessing how it impacts the electrochemical performance of the SOC. Within the parameter space of the training data (statistics of that parameter space is provided in the readme file), this model achieves sub-5% mean absolute percent errors, an order of magnitude less error than percolation theory across the same parameter space. However, be aware that this tool was developed with parametric simulations in mind, and users are encouraged to assess accuracy for their own specific use case rather than taking accuracy metrics at face value. More info, including a usage guide, is in the included readme file. This tool should be cited with the DOI number provided.

Electrode Microstructure↗

NASA NTRS1980

Measuring the development process: A tool for software design evaluation

The design metrics evaluator (DME), a component of an automated software design analysis system, is described. The DME quantitatively evaluates software design attributes. Its use directs attention to areas of a procedure, module, or complete program having a high potential for error.

Moy, S. S.↗

NASA NTRS1998

Testing, Requirements, and Metrics

The criticality of correct, complete, testable requirements is a fundamental tenet of software engineering. Also critical is complete requirements based testing of the final product. Modern tools for managing requirements allow new metrics to be used in support of both of these critical processes. Using these tools, potential problems with the quality of the requirements and the test plan can be identified early in the life cycle. Some of these quality factors include: ambiguous or incomplete requirements, poorly designed requirements databases, excessive or insufficient test cases, and incomplete linkage of tests to requirements. This paper discusses how metrics can be used to evaluate the quality of the requirements and test to avoid problems later. Requirements management and requirements based testing have always been critical in the implementation of high quality software systems. Recently, automated tools have become available to support requirements management. At NASA's Goddard Space Flight Center (GSFC), automated requirements management tools are being used on several large projects. The use of these tools opens the door to innovative uses of metrics in characterizing test plan quality and assessing overall testing risks. In support of these projects, the Software Assurance Technology Center (SATC) is working to develop and apply a metrics program that utilizes the information now available through the application of requirements management tools. Metrics based on this information provides real-time insight into the testing of requirements and these metrics assist the Project Quality Office in its testing oversight role. This paper discusses three facets of the SATC's efforts to evaluate the quality of the requirements and test plan early in the life cycle, thus preventing costly errors and time delays later.

Rosenberg, Linda↗

NASA NTRS2000

Wind Prediction Accuracy for Air Traffic Management Decision Support Tools

The performance of Air Traffic Management and flight deck decision support tools depends in large part on the accuracy of the supporting 4D trajectory predictions. This is particularly relevant to conflict prediction and active advisories for the resolution of conflicts and the conformance with of traffic-flow management flow-rate constraints (e.g., arrival metering / required time of arrival). Flight test results have indicated that wind prediction errors may represent the largest source of trajectory prediction error. The tests also discovered relatively large errors (e.g., greater than 20 knots), existing in pockets of space and time critical to ATM DST performance (one or more sectors, greater than 20 minutes), are inadequately represented by the classic RMS aggregate prediction-accuracy studies of the past. To facilitate the identification and reduction of DST-critical wind-prediction errors, NASA has lead a collaborative research and development activity with MIT Lincoln Laboratories and the Forecast Systems Lab of the National Oceanographic and Atmospheric Administration (NOAA). This activity, begun in 1996, has focussed on the development of key metrics for ATM DST performance, assessment of wind-prediction skill for state of the art systems, and development/validation of system enhancements to improve skill. A 13 month study was conducted for the Denver Center airspace in 1997. Two complementary wind-prediction systems were analyzed and compared to the forecast performance of the then standard 60 km Rapid Update Cycle - version 1 (RUC-1). One system, developed by NOAA, was the prototype 40-km RUC-2 that became operational at NCEP in 1999. RUC-2 introduced a faster cycle (1 hr vs. 3 hr) and improved mesoscale physics. The second system, Augmented Winds (AW), is a prototype en route wind application developed by MITLL based on the Integrated Terminal Wind System (ITWS). AW is run at a local facility (Center) level, and updates RUC predictions based on an optimal interpolation of the latest ACARS reports since the RUC run. This paper presents an overview of the study's results including the identification and use of new large mor wind-prediction accuracy metrics that are key to ATM DST performance.

Cole, Rod↗