Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Performance benchmark”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 433 records · Page 24
All sourcesNASA NTRSDOE OSTI
NASA NTRS2023

Atmospheric Correction Inter-comparison eXercise, ACIX-II Land: An Assessment of Amospheric Correction Processors for Landsat 8 and Sentinel-2 Over Land

The correction of the atmospheric effects on optical satellite images is essential for quantitative and multi-temporal remote sensing applications. In order to study the performance of the state-of-the-art methods in an integrated way, a voluntary and open-access benchmark Atmospheric Correction Inter-comparison eXercise (ACIX) was initiated in 2016 in the frame of Committee on Earth Observation Satellites (CEOS) Working Group on Calibration & Validation (WGCV). The first exercise was extended in a second edition wherein twelve atmospheric correction (AC) processors, a substantially larger testing dataset and additional validation metrics were involved. The sites for the inter-comparison analysis were defined by investigating the full catalogue of the Aerosol Robotic Network (AERONET) sites for coincident measurements with satellites' overpass. Although there were more than one hundred sites for Copernicus Sentinel-2 and Landsat 8 acquisitions, the analysis presented in this paper concerns only the common matchups amongst all processors, reducing the number to 79 and 62 sites respectively. Aerosol Optical Depth (AOD) and Water Vapour (WV) retrievals were consequently validated based on the available AERONET observations. The processors mostly succeeded in retrieving AOD for relatively light to medium aerosol loading (AOD < 0.2) with uncertainties <0.08, while the overall uncertainty values were typically 0.23 ± 0.15. Better performances were observed for WV retrievals with >90% of the results falling within the suggested empirical specifications and with the Root Mean Square Error (RMSE) being mostly <0.25 g/cm2. Regarding Surface Reflectance (SR) validation two main approaches were followed. For the first one, a simulated SR reference dataset was computed over all of the test sites by using the 6SV (Second Simulation of the Satellite Signal in the Solar Spectrum vector code) full radiative transfer modelling (RTM) and AERONET measurements for the required aerosol variables and water vapour content. The performance assessment demonstrated that the retrievals were not biased for most of the bands. The uncertainties ranged from approximately 0.003 to 0.01 (excluding B01) for the best performing processors in both sensors' analyses. For the second one, measurements from the radiometric calibration network RadCalNet over La Crau (France) and Gobabeb (Namibia) were involved in the validation. The performance of the processors was in general consistent across all bands for both sensors and with low standard deviations (<0.04) between on-site and estimated surface reflectance. Overall, our study provides a good insight of AC algorithms' performance to developers and users, pointing out similarities and differences for AOD, WV and SR retrievals. Such validation though still lacks of ground-based measurements of known uncertainty to better assess and characterize the uncertainties in SR retrievals.

Atmospheric correction↗
NASA NTRS2000

Automatic Generation of Directive-Based Parallel Programs for Shared Memory Parallel Systems

The shared-memory programming model is a very effective way to achieve parallelism on shared memory parallel computers. As great progress was made in hardware and software technologies, performance of parallel programs with compiler directives has demonstrated large improvement. The introduction of OpenMP directives, the industrial standard for shared-memory programming, has minimized the issue of portability. Due to its ease of programming and its good performance, the technique has become very popular. In this study, we have extended CAPTools, a computer-aided parallelization toolkit, to automatically generate directive-based, OpenMP, parallel programs. We outline techniques used in the implementation of the tool and present test results on the NAS parallel benchmarks and ARC3D, a CFD application. This work demonstrates the great potential of using computer-aided tools to quickly port parallel programs and also achieve good performance.

Jin, Hao-Qiang↗
NASA NTRS2014

The Sensitivity of WRF Daily Summertime Simulations over West Africa to Alternative Parameterizations. Part 1: African Wave Circulation

The performance of the NCAR Weather Research and Forecasting Model (WRF) as a West African regional-atmospheric model is evaluated. The study tests the sensitivity of WRF-simulated vorticity maxima associated with African easterly waves to 64 combinations of alternative parameterizations in a series of simulations in September. In all, 104 simulations of 12-day duration during 11 consecutive years are examined. The 64 combinations combine WRF parameterizations of cumulus convection, radiation transfer, surface hydrology, and PBL physics. Simulated daily and mean circulation results are validated against NASA's Modern-Era Retrospective Analysis for Research and Applications (MERRA) and NCEP/Department of Energy Global Reanalysis 2. Precipitation is considered in a second part of this two-part paper. A wide range of 700-hPa vorticity validation scores demonstrates the influence of alternative parameterizations. The best WRF performers achieve correlations against reanalysis of 0.40-0.60 and realistic amplitudes of spatiotemporal variability for the 2006 focus year while a parallel-benchmark simulation by the NASA Regional Model-3 (RM3) achieves higher correlations, but less realistic spatiotemporal variability. The largest favorable impact on WRF-vorticity validation is achieved by selecting the Grell-Devenyi cumulus convection scheme, resulting in higher correlations against reanalysis than simulations using the Kain-Fritch convection. Other parameterizations have less-obvious impact, although WRF configurations incorporating one surface model and PBL scheme consistently performed poorly. A comparison of reanalysis circulation against two NASA radiosonde stations confirms that both reanalyses represent observations well enough to validate the WRF results. Validation statistics for optimized WRF configurations simulating the parallel period during 10 additional years are less favorable than for 2006.

variability↗
NASA NTRS2020

Towards a Soil Moisture Drought Monitoring System for South Korea

The Korea Land Data Assimilation System (KLDAS) has been established for agricultural drought (i.e. soil moisture deficit) monitoring in South Korea, running the Noah-MP land surface model within the NASA Land Information System (LIS) framework with the added value of local precipitation forcing dataset and soil texture maps. KLDAS soil moisture is benchmarked against three global products: the Global Land Data Assimilation System (GLDAS), the Famine Early Warning Systems Network (FEWS NET) Land Data Assimilation System (FLDAS), and the European Space Agency Climate Change Initiative (ESA CCI) satellite product. The evaluation is performed using in situ measurements for 2013–2015 and one month standardized precipitation index (SPI-1) for 1982–2016, focusing on four major river basins in South Korea. The KLDAS outperforms all benchmark products in capturing soil moisture states and variability at a basin scale. Compared to GLDAS and FLDAS products, the EAS CCI product is not feasible for long term agricultural monitoring due to lower data quality for early periods (1979–1991) of soil moisture estimates. KLDAS shows that the most recent 2015 drought event leads to highest drought areas in the Han and Geum River basins in the past 35 years. This work supports KLDAS as an effective agricultural drought monitoring system to provide continuous regional high-resolution soil moisture estimates in South Korea.

Agricultural drought↗
NASA NTRS1988

Constitutive modeling for isotropic materials

The third and fourth years of a 4-year research program, part of the NASA HOST Program, are described. The program goals were: (1) to develop and validate unified constitutive models for isotropic materials, and (2) to demonstrate their usefulness for structural analysis of hot section components of gas turbine engines. The unified models selected for development and evaluation were those of Bodner-Partom and of Walker. The unified approach for elastic-viscoplastic constitutive equations is a viable method for representing and predicting material response characteristics in the range where strain rate and temperature dependent inelastic deformations are experienced. This conclusion is reached by extensive comparison of model calculations against the experimental results of a test program of two high temperature Ni-base alloys, B1900+Hf and Mar-M247, over a wide temperature range for a variety of deformation and thermal histories including uniaxial, multiaxial, and thermomechanical loading paths. The applicability of the Bodner-Partom and the Walker models for structural applications has been demonstrated by implementing these models into the MARC finite element code and by performing a number of analyses including thermomechanical histories on components of hot sections of gas turbine engines and benchmark notch tensile specimens. The results of the 4-year program have been published in four annual reports. The results of the base program are summarized in this report. The tasks covered include: (1) development of material test procedures, (2) thermal history effects, and (3) verification of the constitutive model for an alternative material.

Chan, K. S.↗
NASA NTRS2000

BOREAS TE-2 NSA Soil Lab Data

This data set contains the major soil properties of soil samples collected in 1994 at the tower flux sites in the Northern Study Area (NSA). The soil samples were collected by Hugo Veldhuis and his staff from the University of Manitoba. The mineral soil samples were largely analyzed by Barry Goetz, under the supervision of Dr. Harold Rostad at the University of Saskatchewan. The organic soil samples were largely analyzed by Peter Haluschak, under the supervision of Hugo Veldhuis at the Centre for Land and Biological Resources Research in Winnipeg, Manitoba. During the course of field investigation and mapping, selected surface and subsurface soil samples were collected for laboratory analysis. These samples were used as benchmark references for specific soil attributes in general soil characterization. Detailed soil sampling, description, and laboratory analysis were performed on selected modal soils to provide examples of common soil physical and chemical characteristics in the study area. The soil properties that were determined include soil horizon; dry soil color; pH; bulk density; total, organic, and inorganic carbon; electric conductivity; cation exchange capacity; exchangeable sodium, potassium, calcium, magnesium, and hydrogen; water content at 0.01, 0.033, and 1.5 MPascals; nitrogen; phosphorus: particle size distribution; texture; pH of the mineral soil and of the organic soil; extractable acid; and sulfur. These data are stored in ASCII text files. The data files are available on a CD-ROM (see document number 20010000884), or from the Oak Ridge National Laboratory (ORNL) Distributed Active Archive Center (DAAC).

Veldhuis, Hugo↗
NASA NTRS2001

2-D Modeling of Nanoscale MOSFETs: Non-Equilibrium Green's Function Approach

We have developed physical approximations and computer code capable of realistically simulating 2-D nanoscale transistors, using the non-equilibrium Green's function (NEGF) method. This is the most accurate full quantum model yet applied to 2-D device simulation. Open boundary conditions and oxide tunneling are treated on an equal footing. Electrons in the ellipsoids of the conduction band are treated within the anisotropic effective mass approximation. Electron-electron interaction is treated within Hartree approximation by solving NEGF and Poisson equations self-consistently. For the calculations presented here, parallelization is performed by distributing the solution of NEGF equations to various processors, energy wise. We present simulation of the "benchmark" MIT 25nm and 90nm MOSFETs and compare our results to those from the drift-diffusion simulator and the quantum-corrected results available. In the 25nm MOSFET, the channel length is less than ten times the electron wavelength, and the electron scattering time is comparable to its transit time. Our main results are: (1) Simulated drain subthreshold current characteristics are shown, where the potential profiles are calculated self-consistently by the corresponding simulation methods. The current predicted by our quantum simulation has smaller subthreshold slope of the Vg dependence which results in higher threshold voltage. (2) When gate oxide thickness is less than 2 nm, gate oxide leakage is a primary factor which determines off-current of a MOSFET (3) Using our 2-D NEGF simulator, we found several ways to drastically decrease oxide leakage current without compromising drive current. (4) Quantum mechanically calculated electron density is much smaller than the background doping density in the poly silicon gate region near oxide interface. This creates an additional effective gate voltage. Different ways to. include this effect approximately will be discussed.

Svizhenko, Alexei↗
NASA NTRSDate not supplied

Model Predictive Control Strategies for Turbine Electrified Energy Management

The increasing electrification of aircraft propulsion systems is leading to new control architectures being developed to address integration between electric machines and gas-based turbine engines. For hybrid-electric propulsion systems, current conceptual architectures often couple electric machines with the shafts of gas turbine engines and introduce energy storage. Leveraging the electrical power system of hybridized engines, Turbine Electrified Energy Management (TEEM) is a recent control approach that improves transient operability in an effort to enable more efficient and lighter weight turbomachinery. This study seeks to expand TEEM’s application beyond traditional proportional-integral (PI) control by presenting linear model predictive control (MPC) schemes to execute the TEEM concept. Through constraint selection and cost function design, transient operability goals for TEEM are considered with no external logic or saturation. Unique to the designs are the use of a washout filter, which simplifies transient detection and motor activation logic. The proposed architectures are implemented with both centralized MPC and distributed MPC approaches, and comparisons are drawn to a benchmark PI controller simulated on a nonlinear turbofan engine model at one ground condition and one cruise condition. Performance is evaluated using compressor maps, stall margin performance, and two novel metrics: transient stack usage and transient excursion integral. Results reveal the linear MPC scheme performs comparably to the baseline controller and can be implemented in at least two distinct configurations with potential for further modifications, thus establishing the groundwork for future investigations.

Model Predictive Control↗
NASA NTRSDate not supplied

Model Predictive Control Strategies for Turbine Electrified Energy Management

The increasing electrification of aircraft propulsion systems is leading to new control architectures being developed to address integration between electric machines and gas-based turbine engines. For hybrid-electric propulsion systems, current conceptual architectures often couple electric machines with the shafts of gas turbine engines and introduce energy storage. Leveraging the electrical power system of hybridized engines, Turbine Electrified Energy Management (TEEM) is a recent control approach that improves transient operability in an effort to enable more efficient and lighter weight turbomachinery. This study seeks to expand TEEM’s application beyond traditional proportional integral (PI) control by presenting linear model predictive control (MPC) schemes to execute the TEEM concept. Through constraint selection and cost function design, transient operability goals for TEEM are considered with no external logic or saturation. Unique to the designs are the use of a washout filter, which simplifies transient detection and motor activation logic. The proposed architectures are implemented with both centralized MPC and distributed MPC approaches, and comparisons are drawn to a benchmark PI controller simulated on a nonlinear turbofan engine model at one ground condition and one cruise condition. Performance is evaluated using compressor maps, stall margin performance, and two novel metrics: transient stack usage and transient excursion integral. Results reveal the linear MPC scheme performs comparably to the baseline controller and can be implemented in at least two distinct configurations with potential for further modifications, thus establishing the groundwork for future investigations.

Model Predictive Control↗
NASA NTRS2023

Model Predictive Control Strategies for Turbine Electrified Energy Management

The increasing electrification of aircraft propulsion systems is leading to new control architectures being developed to address integration between electric machines and gas-based turbine engines. For hybrid-electric propulsion systems, current conceptual architectures often couple electric machines with the shafts of gas turbine engines and introduce energy storage. Leveraging the electrical power system of hybridized engines, Turbine Electrified Energy Management (TEEM) is a recent control approach that improves transient operability in an effort to enable more efficient and lighter weight turbomachinery. This study seeks to expand TEEM’s application beyond traditional proportional-integral (PI) control by presenting linear model predictive control (MPC) schemes to execute the TEEM concept. Through constraint selection and cost function design, transient operability goals for TEEM are considered with no external logic or saturation. Unique to the designs are the use of a washout filter, which simplifies transient detection and motor activation logic. The proposed architectures are implemented with both centralized MPC and distributed MPC approaches, and comparisons are drawn to a benchmark PI controller simulated on a nonlinear turbofan engine model at one ground condition and one cruise condition. Performance is evaluated using compressor maps, stall margin performance, and two novel metrics: transient stack usage and transient excursion integral. Results reveal the linear MPC scheme performs comparably to the baseline controller and can be implemented in at least two distinct configurations with potential for further modifications, thus establishing the groundwork for future investigations.

Model Predictive Control↗
NASA NTRS2019

In Situ Observation of Growth Dynamics in DECLIC Directional Solidification Insert Onboard ISS: DSI-R Flight Campaign

The study of solidification microstructure formation is of utmost importance for materials design and processing, as solid-liquid interface patterns largely govern mechanical and physical properties. Pattern selection occurs under dynamic conditions of growth in which the initial morphological instability evolves nonlinearly and undergoes a reorganization process. The dynamic and nonlinear nature of this instability renders in situ observation of the interface an invaluable tool to gain knowledge on the time-evolution of the interface pattern. Transparent organic analogs, which solidify like metallic alloys, allow direct visualization of interface dynamics. Extensive ground-based studies of both metallic and organic bulk samples have established the presence of significant convection during solidification processes that alters the formation of interfacial microstructures. A reduced-gravity environment is therefore mandatory for fluid flow elimination in bulk samples. In the framework of the CNES project MISOL3D (MIcrostrutures de SOLidification 3D) and the NASA projects DSIP (Dynamical Selection of 3D Interface Patterns), SPADES (SPAtiotemporal Evolution of three-dimensional DEndritic array Structures) and CAMUS (ComputAtional Studies of MicrostrUcture Formation During Alloy Solidification in Microgravity), we participated in the development of the Directional Solidification Insert (DSI) of the DEvice for the study of Critical Liquids and Crystallization (DECLIC). The DECLIC-DSI is dedicated to in situ and real-time characterization of solid-liquid interface patterns during directional solidification of transparent alloys in diffusive transport regime. Between April 2010 and March 2011, the first ISS campaign (DSI) explored the entire range of microstructures resulting in unprecedented observations. A second campaign (DSI-R), performed between October 2017 and December 2018, in which the insert contained an alloy of higher solute concentration, allowed to complete the benchmark database. The increase of solute concentration resulted in well-developed dendritic patterns at lower velocities (lower interface curvature and larger tip radius). The microstructure resulting from dendritic growth is dominant in metallurgy so that it is fundamental to understand the mechanisms of its formation. The main aims of this experimental campaign are to understand: the mechanisms of the cell to dendrite transition, the fundamental mechanisms of sidebranching formation, the dependence of dendrite tip shapes on growth conditions, the interaction of primary array and secondary sidebranches, and the influence of subgrain boundaries on the spatiotemporal organization of the array structure. Preparation, analysis and interpretation of the experiments performed onboard ISS are considerably enhanced by experiments performed on ground using thin-samples (Pr. Trivedi’s group) and phase-field simulations of microstructure formation in a diffuse growth regime (Pr. Karma’s group). In this summary, we will present an initial assessment of the results obtained during the DSI-R campaign. Nomenclature Vp: pulling velocity G: thermal gradient L: solidified length rtip: tip curvature radius Acronyms/Abbreviations CADMOS: Centre d'Aide au Développement des activités en Micro-pesanteur et des Opérations Spatiales CNES: Centre National d’Études Spatiales DECLIC: Device for the study of Critical LIquids and Crystallization DSI: Directional Solidification Insert DSI-R: Directional Solidification Insert - Refurbish ISS: International Space Station NASA: National Aeronautics and Space Administration PF: phase-field SCN: Succinonitrile 3D: three-dimensional

Mota, F.L.↗
NASA NTRS2016

An Integrated Extravehicular Activity Research Plan

Multiple organizations within NASA and outside of NASA fund and participate in research related to extravehicular activity (EVA). In October 2015, representatives of the EVA Office, the Crew and Thermal Systems Division (CTSD), and the Human Research Program (HRP) at NASA Johnson Space Center agreed on a formal framework to improve multi-year coordination and collaboration in EVA research. At the core of the framework is an Integrated EVA Research Plan and a process by which it will be annually reviewed and updated. The over-arching objective of the collaborative framework is to conduct multi-disciplinary cost-effective research that will enable humans to perform EVAs safely, effectively, comfortably, and efficiently, as needed to enable and enhance human space exploration missions. Research activities must be defined, prioritized, planned and executed to comprehensively address the right questions, avoid duplication, leverage other complementary activities where possible, and ultimately provide actionable evidence-based results in time to inform subsequent tests, developments and/or research activities. Representation of all appropriate stakeholders in the definition, prioritization, planning and execution of research activities is essential to accomplishing the over-arching objective. A formal review of the Integrated EVA Research Plan will be conducted annually. External peer review of all HRP EVA research activities including compilation and review of published literature in the EVA Evidence Book is already performed annually. Coordination with stakeholders outside of the EVA Office, CTSD, and HRP is already in effect on a study-by-study basis; closer coordination on multi-year planning with other EVA stakeholders including academia is being actively pursued. Details of the current Integrated EVA Research Plan are presented including description of ongoing and planned research activities in the areas of: Benchmarking; Anthropometry and Suit Fit; Sensors; Human-Suit Modeling; Suit Trauma Monitoring and Countermeasures; EVA Workload and Duration Effects; Decompression Sickness Risk Mitigation; Deconditioned EVA Performance; and Exploration EVA Concept of Operations.

Abercromby, Andrew F. J.↗
NASA NTRS2016

Performance Evaluation of an Intel Haswell- and Ivy Bridge-Based Supercomputer Using Scientific and Engineering Applications

We present a performance evaluation conducted on a production supercomputer of the Intel Xeon Processor E5- 2680v3, a twelve-core implementation of the fourth-generation Haswell architecture, and compare it with Intel Xeon Processor E5-2680v2, an Ivy Bridge implementation of the third-generation Sandy Bridge architecture. Several new architectural features have been incorporated in Haswell including improvements in all levels of the memory hierarchy as well as improvements to vector instructions and power management. We critically evaluate these new features of Haswell and compare with Ivy Bridge using several low-level benchmarks including subset of HPCC, HPCG and four full-scale scientific and engineering applications. We also present a model to predict the performance of HPCG and Cart3D within 5%, and Overflow within 10% accuracy.

Ivy Bridge↗
NASA NTRS1994

Automated Instrumentation and Monitoring of Data Movement for Parallel Programs

Writing efficient parallel programs is complicated by the need to select the right data structure alignments and distributions, which determine the nature and volume of inter-processor communications. A large number of performance tools for parallel programs have been developed recently to expose these inter-processor communications. However, none of them support performance views or provide statistics in terms of inter-processor data structure interactions. A performance tool that tracks the interaction between individual data structures and the context of these interactions is essential for understanding the performance of both explicit message passing programs and data-parallel languages such as HPF. In this paper we discuss the use of compiler front end tools for automatically tracking data structure movements in message passing programs, and low-overhead monitoring and postprocessing of such codes. We demonstrate that robust instrumentation and low overhead monitoring of inter-processor data structure movements is possible, with the use of a number of NAS benchmark codes, run on the i860 hypercube. We also show that the data so collected can be used effectively by post processing tools that expose performance bottlenecks using graphical displays and performance statistics.

Sarukkai, Sekhar↗
NASA NTRS2020

Durability Characterization of Low Ice Adhesion and Impact Resistance Coatings

Glaze, rime and mixed icing conditions can cause significant variation in aerodynamic performance, fuel efficiency, and flight safety. For commercial aircraft, active icing mitigation strategies are utilized to enable safe flight within icing conditions according to the FAR Part 25/29 Appendix C icing envelope. For general aviation and unmanned aerial vehicles that cannot support active anti-icing technologies, avoidance is the only recourse. Passive approaches have been investigated to reduce weight and energy consumption for active systems and to expand the operational envelope for smaller aircraft. Coatings are one passive method to reduce or mitigate ice accretion on frontal surfaces of commercial aircraft with the greatest region of interest being wing leading edges. Aircraft wing leading edges represent an extreme environment. Durability must be considered for any material to be applied as a coating in this area. Currently, there are no specifications regarding coating durability of a low ice adhesion material applied on the wing leading edge. Therefore, a reasonable starting point is to use the durability specifications put forth for an aircraft external coating. Besides the tests called out in these specifications, others were included such as Taber abrasion to simulate wear and erosion. Beyond durability, performance metrics with regards to adhesion strength of accreted in-flight (i.e., impact) icing are central to determine coating usefulness. Determination of this property though is particularly challenging and only a few facilities have the capability to quantify this. An instrument at NASA Langley Research Center [Adverse Environment Rotary Test Stand Jr., (AERTS Jr.)], based on the AERTS system at The Pennsylvania State University, allows for screening numerous coatings subjected to the icing environment. To establish a benchmark for comparison with research coatings, a state-of-the-art (SOA) commercial aircraft coating was subjected to an array of salient durability and performance experiments. The same tests were conducted using a baseline epoxy (BE) resin formulation that demonstrated some initial promising results for comparison. Results from this initial screening will be discussed herein.

Miranda L Beaudry↗
NASA NTRS1983

Tilt rotor technology thrusts

This paper provides an overview of the results of on-going research investigations associated with the XV-15 program. The XV-15 data and an understanding of the technological elements that go into tilt rotor design can provide a series of benchmark points from which to proceed with new design. The work to refine the estimates of vertical-lift performance, the influence of airframe aerodynamics, and the work planned to establish rotor-limiting flight conditions are discussed. An initial look at short takeoff and landing (STOL) performance is provided and the lessons learned and on-going work in aeroelastic testing and data handling is described. Structural loads and stability and control systems work which have applicability to new design efforts are also shown. The paper concludes with some ideas for longer-range generic investigations to advance the state of the art.

Magee, J. P.↗
NASA NTRS1992

Exploratory study on performance measures as indicators of IS effectiveness

The Information Systems Directorate at JSC/NASA has undertaken the reevaluation of its performance measures process and measures. Under the direction of a quality approach it is essential to identify an external perspective of how well an organization is performing. This study was conducted with two major objectives: (1) survey and summarize the academic literature on performance measures as indicators of information systems (IS) effectiveness; and (2) survey organizations for their experience in measuring for IS effectiveness. Four approaches to measuring the effectiveness of IS performance were identified: (1) listen to the customer for the things they need; (2) align with corporate goals; (3) benchmark against well-respected organizations; and (4) ask yourself what critical factors lead to success. The list of known methods for soliciting customer feedback are as follows: (1) executive visit; (2) survey, interview, and focus group; (3) complaints and compliments; and (4) service level agreements. A common set of characteristics that satisfy customers was identified from the literature. The list includes elements such as the following: accuracy, timeliness, relevance, understandability, reliability, and completeness. Future research in this topic area should prove beneficial to determine the metrics for external validity.

Bishop, Peter C.↗
NASA NTRS2004

The Processing of Airspace Concept Evaluations Using FASTE-CNS as a Pre- or Post-Simulation CNS Analysis Tool

As NASA speculates on and explores the future of aviation, the technological and physical aspects of our environment increasing become hurdles that must be overcome for success. Research into methods for overcoming some of these selected hurdles have been purposed by several NASA research partners as concepts. The task of establishing a common evaluation environment was placed on NASA's Virtual Airspace Simulation Technologies (VAST) project (sub-project of VAMS), and they responded with the development of the Airspace Concept Evaluation System (ACES). As one examines the ACES environment from a communication, navigation or surveillance (CNS) perspective, the simulation parameters are built with assumed perfection in the transactions associated with CNS. To truly evaluate these concepts in a realistic sense, the contributions/effects of CNS must be part of the ACES. NASA Glenn Research Center (GRC) has supported the Virtual Airspace Modeling and Simulation (VAMS) project through the continued development of CNS models and analysis capabilities which supports the ACES environment. NASA GRC initiated the development a communications traffic loading analysis tool, called the Future Aeronautical Sub-network Traffic Emulator for Communications, Navigation and Surveillance (FASTE-CNS), as part of this support. This tool allows for forecasting of communications load with the understanding that, there is no single, common source for loading models used to evaluate the existing and planned communications channels; and that, consensus and accuracy in the traffic load models is a very important input to the decisions being made on the acceptability of communication techniques used to fulfill the aeronautical requirements. Leveraging off the existing capabilities of the FASTE-CNS tool, GRC has called for FASTE-CNS to have the functionality to pre- and post-process the simulation runs of ACES to report on instances when traffic density, frequency congestion or aircraft spacing/distance violations have occurred. The integration of these functions require that the CNS models used to characterize these avionic system be of higher fidelity and better consistency then is present in FASTE-CNS system. This presentation will explore the capabilities of FASTE-CNS with renewed emphasis on the enhancements being added to perform these processing functions; the fidelity and reliability of CNS models necessary to make the enhancements work; and the benchmarking of FASTE-CNS results to improve confidence for the results of the new processing capabilities.

Mainger, Steve↗