SEARCH · Engineering Papers

Results for “performance metrics”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2

All sources NASA NTRS DOE OSTI

NASA NTRSDate not supplied

A Framework for Evaluating Climate Model Performance Metrics

The CMIP5 archive contains future climate projections from over 50 models provided by dozens of modeling centers from around the world. Individual model projections, however, are subject to biases created by structural model uncertainties. As a result, ensemble averaging of multiple models is often used to add value to model projections: consensus projections have been shown to consistently outperform individual models. Previous reports for the IPCC establish climate change projections based on an equal-weighted average of all model projections. However, certain models reproduce climate processes better than other models. Should models be weighted based on performance? Unequal ensemble averages have previously been constructed using a variety of mean state metrics. What metrics are most relevant for constraining future climate projections? This project develops a framework for systematically testing metrics in models to identify optimal metrics for unequal weighting multi-model ensembles. A unique aspect of this project is the construction and testing of climate process-based model evaluation metrics. A climate process-based metric is defined as a metric based on the relationship between two physically related climate variables?e.g., outgoing longwave radiation and surface temperature. Metrics are constructed using high-quality Earth radiation budget data from NASA's Clouds and Earth's Radiant Energy System (CERES) instrument and surface temperature data sets. It is found that regional values of tested quantities can vary significantly when comparing weighted and unweighted model ensembles. For example, one tested metric weights the ensemble by how well models reproduce the time-series probability distribution of the cloud forcing component of reflected shortwave radiation. The weighted ensemble for this metric indicates lower simulated precipitation (up to .7 mm/day) in tropical regions than the unweighted ensemble: since CMIP5 models have been shown to overproduce precipitation, this result could indicate that the metric is effective in identifying models which simulate more realistic precipitation. Ultimately, the goal of the framework is to identify performance metrics for advising better methods for ensemble averaging models and create better climate predictions.

Noel C Baker↗

NASA NTRS2018

Performance Metrics for the Assessment of Satellite Data Products: An Ocean Color Case Study

Performance assessment of ocean color satellite data has generally relied on statistical metrics chosen for their common usage and the rationale for selecting certain metrics is infrequently explained. Commonly reported statistics based on mean squared errors, such as the coefficient of determination (r2), root mean square error, and regression slopes, are most appropriate for Gaussian distributions without outliers and, therefore, are often not ideal for ocean color algorithm performance assessment, which is often limited by sample availability. In contrast, metrics based on simple deviations, such as bias and mean absolute error, as well as pair-wise comparisons, often provide more robust and straightforward quantities for evaluating ocean color algorithms with non-Gaussian distributions and outliers. This study uses a SeaWiFS chlorophyll-a validation data set to demonstrate a framework for satellite data product assessment and recommends a multimetric and user-dependent approach that can be applied within science, modeling, and resource management communities.

remote sensing↗

NASA NTRS1994

Performance Metrics for Monitoring Parallel Program Executions

Existing tools for debugging performance of parallel programs either provide graphical representations of program execution or profiles of program executions. However, for performance debugging tools to be useful, such information has to be augmented with information that highlights the cause of poor program performance. Identifying the cause of poor performance necessitates the need for not only determining the significance of various performance problems on the execution time of the program, but also needs to consider the effect of interprocessor communications of individual source level data structures. In this paper, we present a suite of normalized indices which provide a convenient mechanism for focusing on a region of code with poor performance and highlights the cause of the problem in terms of processors, procedures and data structure interactions. All the indices are generated from trace files augmented with data structure information.. Further, we show with the help of examples from the NAS benchmark suite that the indices help in detecting potential cause of poor performance, based on augmented execution traces obtained by monitoring the program.

Sarukkai, Sekkar R.↗

NASA NTRS2009

Stability and Performance Metrics for Adaptive Flight Control

This paper addresses the problem of verifying adaptive control techniques for enabling safe flight in the presence of adverse conditions. Since the adaptive systems are non-linear by design, the existing control verification metrics are not applicable to adaptive controllers. Moreover, these systems are in general highly uncertain. Hence, the system's characteristics cannot be evaluated by relying on the available dynamical models. This necessitates the development of control verification metrics based on the system's input-output information. For this point of view, a set of metrics is introduced that compares the uncertain aircraft's input-output behavior under the action of an adaptive controller to that of a closed-loop linear reference model to be followed by the aircraft. This reference model is constructed for each specific maneuver using the exact aerodynamic and mass properties of the aircraft to meet the stability and performance requirements commonly accepted in flight control. The proposed metrics are unified in the sense that they are model independent and not restricted to any specific adaptive control methods. As an example, we present simulation results for a wing damaged generic transport aircraft with several existing adaptive controllers.

Stepanyan, Vahram↗

NASA NTRS1988

C3 generic workstation: Performance metrics and applications

The large number of integrated dependent measures available on a command, control, and communications (C3) generic workstation under development are described. In this system, embedded communications tasks will manipulate workload to assess the effects of performance-enhancing drugs (sleep aids and decongestants), work/rest cycles, biocybernetics, and decision support systems on performance. Task performance accuracy and latency will be event coded for correlation with other measures of voice stress and physiological functioning. Sessions will be videotaped to score non-verbal communications. Physiological recordings include spectral analysis of EEG, ECG, vagal tone, and EOG. Subjective measurements include SWAT, fatigue, POMS and specialized self-report scales. The system will be used primarily to evaluate the effects on performance of drugs, work/rest cycles, and biocybernetic concepts. Performance assessment algorithms will also be developed, including those used with small teams. This system provides a tool for integrating and synchronizing behavioral and psychophysiological measures in a complex decision-making environment.

Eddy, Douglas R.↗

NASA NTRS2009

Evaluating Algorithm Performance Metrics Tailored for Prognostics

Prognostics has taken a center stage in Condition Based Maintenance (CBM) where it is desired to estimate Remaining Useful Life (RUL) of the system so that remedial measures may be taken in advance to avoid catastrophic events or unwanted downtimes. Validation of such predictions is an important but difficult proposition and a lack of appropriate evaluation methods renders prognostics meaningless. Evaluation methods currently used in the research community are not standardized and in many cases do not sufficiently assess key performance aspects expected out of a prognostics algorithm. In this paper we introduce several new evaluation metrics tailored for prognostics and show that they can effectively evaluate various algorithms as compared to other conventional metrics. Specifically four algorithms namely; Relevance Vector Machine (RVM), Gaussian Process Regression (GPR), Artificial Neural Network (ANN), and Polynomial Regression (PR) are compared. These algorithms vary in complexity and their ability to manage uncertainty around predicted estimates. Results show that the new metrics rank these algorithms in different manner and depending on the requirements and constraints suitable metrics may be chosen. Beyond these results, these metrics offer ideas about how metrics suitable to prognostics may be designed so that the evaluation procedure can be standardized. 1

Saxena, Abhinav↗

NASA NTRS2020

Controller Design for Propeller Phase Synchronization with Aeroacoustic Performance Metrics

Active noise reduction using phase control takes advantage of the propellers used in a distributed electric air vehicle by treating each propeller as an independent acoustic source. These acoustic sources can destructively interfere if the propellers are synchronized. With this method, the radiated sound power around the vehicle can be reduced. The purpose of this paper is to design a controller that regulates propeller positions to reduce the radiated sound power through destructive interference and to demonstrate the performance of the controller through acoustic testing. There are two control requirements for reducing the sound pressure level considered in this paper. The first is accurately regulating the difference between the propeller azimuthal blade positions (phase) relative to their neighbors. By changing the phase at the source, we can control the phase of the sound wave at an arrival position to create destructive interference. The second consideration is maintaining high coherence between the two propeller sources. To achieve significant attenuation, the controller needs to regulate phase error and suppress sources of incoherence. Performance is demonstrated through sound recordings performed in the NASA Langley Structural Acoustic Loads and Transmission (SALT) anechoic chamber. We show that the controller is capable of reducing sound pressure level at a given observer location by 17 dB at the blade passage frequency and that this method can reduce the radiated sound power by 6 dB at the blade passage frequency.

Andrew Patterson↗

NASA NTRS2004

Defining Exercise Performance Metrics for Flight Hardware Development

The space industry has prevailed over numerous design challenges in the spirit of exploration. Manned space flight entails creating products for use by humans and the Johnson Space Center has pioneered this effort as NASA's center for manned space flight. NASA Astronauts use a suite of flight exercise hardware to maintain strength for extravehicular activities and to minimize losses in muscle mass and bone mineral density. With a cycle ergometer, treadmill, and the Resistive Exercise Device available on the International Space Station (ISS), the Space Medicine community aspires to reproduce physical loading schemes that match exercise performance in Earth s gravity. The resistive exercise device presents the greatest challenge with the duty of accommodating 20 different exercises and many variations on the core set of exercises. This paper presents a methodology for capturing engineering parameters that can quantify proper resistive exercise performance techniques. For each specified exercise, the method provides engineering parameters on hand spacing, foot spacing, and positions of the point of load application at the starting point, midpoint, and end point of the exercise. As humans vary in height and fitness levels, the methodology presents values as ranges. In addition, this method shows engineers the proper load application regions on the human body. The methodology applies to resistive exercise in general and is in use for the current development of a Resistive Exercise Device. Exercise hardware systems must remain available for use and conducive to proper exercise performance as a contributor to mission success. The astronauts depend on exercise hardware to support extended stays aboard the ISS. Future plans towards exploration of Mars and beyond acknowledge the necessity of exercise. Continuous improvement in technology and our understanding of human health maintenance in space will allow us to support the exploration of Mars and the future of space exploration.

Beyene, Nahon M.↗

NASA NTRS1994

Contrasting Target, Stray-Light, and Other Performance Metrics for MISR

The Multi-angle Imaging Spectrometer (MISR) is an Earth-observing sensor to be flown as part of the Earth Observing System (EOS) in 1998. The radiometric and spectral calibration of the nine cameras which compose this instrument will be done using targets which are uniform in space and in angle, unpolarized, and lacking in absorption lines. A calibration uncertainty will also be determined for this configuration. This allows one to estimate the accuracy of measured radiances, assuming the scene is likewise featureless with respect to these parameters. In addition to these calibrations, the MISR engineering team will be responsible for verification of certain performance specifications which assure data products can be produced for a range of target types.

Multi-angle Imaging Spectrometer MISR Earth Observ↗

NASA NTRSDate not supplied

Comparisons of Performance Metrics and Machine Learning Methods on an Entry Descent and Landing Database

This work focuses on evaluating machine learning methods and their applicability to the generation of an aerodynamic database, particularly for trajectory analysis of a capsule during entry, descent, and landing with a focus on uncertainty quantification. The source data to be used is the wind tunnel and computational data for the Integrated Design Assessment Team (IDAT) configuration of the Orion project, which has been publicly released. The methods used to generate the proposed databases are designed to naturally include a prediction interval, which will be evaluated both for their mean response as well as how well the prediction interval performs. These machine learning methods are compared to a traditionally generated database used by the Orion team as a baseline. It is found that while these machine learning methods perform well, the Orion database tends to still outperform them showing that engineering experience is still needed to make the best database possible. However, these methods still provide comparable results with significantly less effort.

Orion↗

NASA NTRSDate not supplied

Comparisons of Performance Metrics and Machine Learning Methods on an Entry, Descent, and Landing Database

Orion↗

DOE OSTI2024

A Global Sensitivity Analysis of Traffic Microsimulation Input Parameters on Performance Metrics

Not Available

Schrader, Maxwell↗

DOE OSTI2024