Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “METRICS”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6
All sourcesNASA NTRSDOE OSTI
NASA NTRS2003

NASA Aviation Safety Program Systems Analysis/Program Assessment Metrics Review

The goal of this project is to evaluate the metrics and processes used by NASA's Aviation Safety Program in assessing technologies that contribute to NASA's aviation safety goals. There were three objectives for reaching this goal. First, NASA's main objectives for aviation safety were documented and their consistency was checked against the main objectives of the Aviation Safety Program. Next, the metrics used for technology investment by the Program Assessment function of AvSP were evaluated. Finally, other metrics that could be used by the Program Assessment Team (PAT) were identified and evaluated. This investigation revealed that the objectives are in fact consistent across organizational levels at NASA and with the FAA. Some of the major issues discussed in this study which should be further investigated, are the removal of the Cost and Return-on-Investment metrics, the lack of the metrics to measure the balance of investment and technology, the interdependencies between some of the metric risk driver categories, and the conflict between 'fatal accident rate' and 'accident rate' in the language of the Aviation Safety goal as stated in different sources.

Louis, Garrick E.↗
NASA NTRS1998

Towards a Visual Quality Metric for Digital Video

The advent of widespread distribution of digital video creates a need for automated methods for evaluating visual quality of digital video. This is particularly so since most digital video is compressed using lossy methods, which involve the controlled introduction of potentially visible artifacts. Compounding the problem is the bursty nature of digital video, which requires adaptive bit allocation based on visual quality metrics. In previous work, we have developed visual quality metrics for evaluating, controlling, and optimizing the quality of compressed still images. These metrics incorporate simplified models of human visual sensitivity to spatial and chromatic visual signals. The challenge of video quality metrics is to extend these simplified models to temporal signals as well. In this presentation I will discuss a number of the issues that must be resolved in the design of effective video quality metrics. Among these are spatial, temporal, and chromatic sensitivity and their interactions, visual masking, and implementation complexity. I will also touch on the question of how to evaluate the performance of these metrics.

Watson, Andrew B.↗
NASA NTRS2006

Metrics for Linear Kinematic Features in Sea Ice

The treatment of leads as cracks or discontinuities (see Coon et al. presentation) requires some shift in the procedure of evaluation and comparison of lead-resolving models and their validation against observations. Common metrics used to evaluate ice model skills are by and large an adaptation of a least square "metric" adopted from operational numerical weather prediction data assimilation systems and are most appropriate for continuous fields and Eilerian systems where the observations and predictions are commensurate. However, this class of metrics suffers from some flaws in areas of sharp gradients and discontinuities (e.g., leads) and when Lagrangian treatments are more natural. After a brief review of these metrics and their performance in areas of sharp gradients, we present two new metrics specifically designed to measure model accuracy in representing linear features (e.g., leads). The indices developed circumvent the requirement that both the observations and model variables be commensurate (i.e., measured with the same units) by considering the frequencies of the features of interest/importance. We illustrate the metrics by scoring several hypothetical "simulated" discontinuity fields against the lead interpreted from RGPS observations.

Levy, G.↗
NASA NTRS2009

Evaluating Algorithm Performance Metrics Tailored for Prognostics

Prognostics has taken a center stage in Condition Based Maintenance (CBM) where it is desired to estimate Remaining Useful Life (RUL) of the system so that remedial measures may be taken in advance to avoid catastrophic events or unwanted downtimes. Validation of such predictions is an important but difficult proposition and a lack of appropriate evaluation methods renders prognostics meaningless. Evaluation methods currently used in the research community are not standardized and in many cases do not sufficiently assess key performance aspects expected out of a prognostics algorithm. In this paper we introduce several new evaluation metrics tailored for prognostics and show that they can effectively evaluate various algorithms as compared to other conventional metrics. Specifically four algorithms namely; Relevance Vector Machine (RVM), Gaussian Process Regression (GPR), Artificial Neural Network (ANN), and Polynomial Regression (PR) are compared. These algorithms vary in complexity and their ability to manage uncertainty around predicted estimates. Results show that the new metrics rank these algorithms in different manner and depending on the requirements and constraints suitable metrics may be chosen. Beyond these results, these metrics offer ideas about how metrics suitable to prognostics may be designed so that the evaluation procedure can be standardized. 1

Saxena, Abhinav↗
NASA NTRS2009

On Applying the Prognostic Performance Metrics

Prognostics performance evaluation has gained significant attention in the past few years. As prognostics technology matures and more sophisticated methods for prognostic uncertainty management are developed, a standardized methodology for performance evaluation becomes extremely important to guide improvement efforts in a constructive manner. This paper is in continuation of previous efforts where several new evaluation metrics tailored for prognostics were introduced and were shown to effectively evaluate various algorithms as compared to other conventional metrics. Specifically, this paper presents a detailed discussion on how these metrics should be interpreted and used. Several shortcomings identified, while applying these metrics to a variety of real applications, are also summarized along with discussions that attempt to alleviate these problems. Further, these metrics have been enhanced to include the capability of incorporating probability distribution information from prognostic algorithms as opposed to evaluation based on point estimates only. Several methods have been suggested and guidelines have been provided to help choose one method over another based on probability distribution characteristics. These approaches also offer a convenient and intuitive visualization of algorithm performance with respect to some of these new metrics like prognostic horizon and alpha-lambda performance, and also quantify the corresponding performance while incorporating the uncertainty information.

Saxena, Abhinav↗
NASA NTRS2010

Metrics for Offline Evaluation of Prognostic Performance

Prognostic performance evaluation has gained significant attention in the past few years. Currently, prognostics concepts lack standard definitions and suffer from ambiguous and inconsistent interpretations. This lack of standards is in part due to the varied end-user requirements for different applications, time scales, available information, domain dynamics, etc. to name a few. The research community has used a variety of metrics largely based on convenience and their respective requirements. Very little attention has been focused on establishing a standardized approach to compare different efforts. This paper presents several new evaluation metrics tailored for prognostics that were recently introduced and were shown to effectively evaluate various algorithms as compared to other conventional metrics. Specifically, this paper presents a detailed discussion on how these metrics should be interpreted and used. These metrics have the capability of incorporating probabilistic uncertainty estimates from prognostic algorithms. In addition to quantitative assessment they also offer a comprehensive visual perspective that can be used in designing the prognostic system. Several methods are suggested to customize these metrics for different applications. Guidelines are provided to help choose one method over another based on distribution characteristics. Various issues faced by prognostics and its performance evaluation are discussed followed by a formal notational framework to help standardize subsequent developments.

Saxena, Abhinav↗
NASA NTRS2005

Semantic Metrics for Analysis of Software

A recently conceived suite of object-oriented software metrics focus is on semantic aspects of software, in contradistinction to traditional software metrics, which focus on syntactic aspects of software. Semantic metrics represent a more human-oriented view of software than do syntactic metrics. The semantic metrics of a given computer program are calculated by use of the output of a knowledge-based analysis of the program, and are substantially more representative of software quality and more readily comprehensible from a human perspective than are the syntactic metrics.

Etzkorn, Letha H.↗
NASA NTRS2016

Up Periscope! Designing a New Perceptual Metric for Imaging System Performance

Modern electronic imaging systems include optics, sensors, sampling, noise, processing, compression, transmission and display elements, and are viewed by the human eye. Many of these elements cannot be assessed by traditional imaging system metrics such as the MTF. More complex metrics such as NVTherm do address these elements, but do so largely through parametric adjustment of an MTF-like metric. The parameters are adjusted through subjective testing of human observers identifying specific targets in a set of standard images. We have designed a new metric that is based on a model of human visual pattern classification. In contrast to previous metrics, ours simulates the human observer identifying the standard targets. One application of this metric is to quantify performance of modern electronic periscope systems on submarines.

imaging↗
NASA NTRS2018

Overview of Metrics Pertinent to Human Response to UAM Noise

This presentation reviews metrics pertinent to noise from eVTOL aircraft, including Noise dose metrics; DNL, CTL; Statistical metrics; Spectrum-based metrics; metrics based on event frequency; and metrics based on partial loudness and detection.

aircraft noise disturbance↗
NASA NTRS2018

Psychoacoustic Test to Determine Sound Quality Metric Indicators of Rotorcraft Noise Annoyance

Noise certification metrics such as Effective Perceived Noise Level and Sound Exposure Level are used to ensure that helicopters meet regulations, but these metrics may not be good indicators of annoyance since noise complaints against helicopters persist. Sound quality (SQ) metrics, specifically fluctuation strength, tonality, impulsiveness, roughness, and sharpness, are explored to determine their relationship with annoyance. A psychoacoustic test was conducted at the NASA Langley Research Center Exterior Effects Room to assess annoyance to helicopter-like sounds over a range of SQ metric values. The amplitude, phase, and frequency of the AS350 helicopter main and tail rotor blade passage signal harmonics were manipulated to produce 105 unique helicopter-like sounds with prescribed values of SQ metrics. All sounds were set to roughly the same loudness level. These sounds were played to 40 subjects who rated each sound for annoyance. Analyses given in this paper point to which SQ metrics are important to the helicopter noise annoyance response.

Krishnamurthy, Siddhartha↗
NASA NTRSDate not supplied

Mineralogical, Elemental and Tomographic Reconnaissance Investigation for CLPS (“METRIC”)

METRIC comprises a suite of two instruments in a Lunar lander that perform X-ray diffraction (XRD) for mineral structure, X-ray fluorescence (XRF) for elemental composition, and X-ray Micro Computed Tomography (XCT) for 3D internal micromorphology. The instruments are accompanied by optical-near IR cameras to provide local geologic context. The Honeybee Robotics PlanetVac pneumatic sampling and transfer system [1],positioned on a lander footpad, will deliver sieved regolith to the X-ray instruments for analysis. The instrument suite is intended for delivery to the lunar surface on a Commercial Lunar Payload Services (CLPS) lander. The METRICXRD/F instrument draws on heritage from the Mars Science Laboratory CheMin instrument [2] and improves upon the design in multiple ways [3]. Like CheMin, Rietveld refinement and full-pattern fitting of METRIC XRD data can identify minerals at a detection limit of ~1 wt.%, quantify their abundances when present at >3 wt.%, and determine mineral composition (e.g., Fo#in olivine) from lattice parameters for minerals present at >5 wt.%[4,5]. An optimized XRF geometry provides for improved detection and quantification of major, minor and trace elements. The METRIC XCT instrument is a miniaturized X-ray Computed Tomography scanner[6]. Lunar regolith is delivered to a 3 mm diameter, 10 mm long graphite tube inside the instrument. The tube is rotated through 360°in 0.9deg.incrementsand a divergent point source X-ray beam is directed through the material. A Charge Coupled Device (CCD)records attenuation images whose brightness and contrast area function of average atomic number and density. Quantitative data, including particle and void sizes, 3D particle shape parameters, modal volumes and pore geometry can be derived from the resulting 3D reconstructions(voxel resolution: 30 μm). Crystal morphologies derived from METRIC XCT data complement the bulk mineralogy determined by the METRIC XRD/F and provide a measure of grain size distribution for the different phases. Taken together, the METRIC instrument suite determines crystal structure, elemental composition and morphology, three principal characteristics of geological materials that are highly useful in determining the origin and subsequent processing of lunar regolith.

Moon↗
DOE OSTI2026

Resilience Metrics for Solar Photovoltaics

This workshop presentation proposes the development of solar photovoltaic (PV) system resilience metrics and a methodology and framework for evaluation of PV resilience metrics. PV resilience metrics are needed to establish a consistent basis for reporting, evaluation, and data collection by industry, evaluate performance of PV systems that have been subject to natural hazards, correlating resilience to system attributes, and predicting resilience for any PV system. PV resilience metrics can guide improved system design, standards, and insurance coverage. Establishing consistent metrics can foster data collection on impacts of natural hazards on PV systems.

14 SOLAR ENERGY↗
DOE OSTI2022

Improving energy performance metrics to maximize the benefits of disruptive technologies

Advances in appliance and equipment systems' technologies also cause changes in their technical characteristics that can affect energy consumption and associated greenhouse gas emissions. Energy performance evaluation methods that use testing and metrics can evolve to account for energy consumption contributed from new technological characteristics. However, there is no systematic effort to enable the advancement of test procedures and performance metrics to maximize the energy, climate, and financial benefits of disruptive technologies. Therefore, it is important for countries to regularly update their energy-efficiency policy programs, such as standards and labels, by improving test procedures and metrics to reflect the innovation in emerging technologies as well as to mitigate the risk of deploying obsolete technology. This paper shows how energy performance metrics have been improving in selected appliances and equipment and provides insights to design a systematic effort to improve metrics and test procedures at the speed of ongoing changes in technologies and markets.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2022

Characterization factors and other air quality impact metrics: Case study for PM 2.5 -emitting area sources from biofuel feedstock supply

In this paper, we develop a framework and metrics for estimating the impact of emission sources on regulatory compliance and human health for applications in air quality planning and life cycle impact assessment (LCIA). Our framework is based on a pollutant's characterization factor (CF) and three new metrics: Available Regulatory Capacity for Incremental Emissions (ARCIE), Source CF Ratio, and Activity Health Impact (AHI) Ratio. ARCIE can be used to assess whether a receptor location has capacity to accommodate additional source emissions while complying with regulatory limits. We present CF as a midpoint indicator of health impacts per unit mass of emitted pollutant. Source CF Ratio enables comparison of potential new-source locations based on human health impacts. The AHI Ratio estimates the health impacts of a pollutant in relation to the utilization of the source for each unit of product or service. These metrics can be applied to any pollutant, energy source sector (e.g., agriculture, electricity), source type (point, line, area), and spatial modeling domain (nation, state, city, region). We demonstrate these metrics through a case study of fine particulate (PM 2.5 ) emissions from U.S. corn stover harvesting and local processing at various scales, representing steps in the biofuel production process. We model PM 2.5 formation in the atmosphere using a novel reduced-complexity chemical transport model called the Intervention Model for Air Pollution (InMAP). Through this case study, we present the first area-source PM 2.5 CFs that address the recommendations of several LCIA studies to establish spatially explicit CFs specific to an energy source sector or type. Overall, the framework developed in this work provides multiple new ways to consider the potential impacts of air emissions through spatially differentiated metrics.

09 BIOMASS FUELS↗
DOE OSTI2021

An evaluation of electricity system technology competitiveness metrics: The case for profitability

In this work we develop standardized functional forms for electricity technology competitiveness metrics and show how the mathematical relationship between value and cost can affect the robustness of the metric. Here, we make the case to replace established metrics with economic profitability metrics – including a new profitability-based adjustment to LCOE. These profitability metrics are shown to be more robust for evaluating technology competitiveness by keeping comparisons on an equivalent monetary basis.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2023

How to select distracted driving countermeasures evaluation metrics: A systematic review

While there are numerous performance metrics that have been developed for the evaluation of distracted driving prevention programs, there is little information on how to select them depending on the requirements and/or objectives of the study. Here, this paper describes a systematic literature review that was conducted on the metrics for evaluating distracted driving countermeasures in order to bridge this research gap. A summary of the evaluation metrics used for the existing distracted driving countermeasures was provided. Guidance for choosing an evaluation measure was provided by analyzing the metrics from the perspectives of functionality, spatial-temporal dimension, and equity. Three examples of distracted driving countermeasure evaluation processes were thoroughly reviewed in order to offer insight into metric selection and measurement. This paper contributes to the body of knowledge by discussing the implications for policy on how to enhance the thoroughness and accuracy of the evaluation of distracted driving countermeasures. Analysis from multiple angles, the development of data collection tools and direct behavior indicators, taking into account temporal dimensions, and equity considerations, are proposed.

99 GENERAL AND MISCELLANEOUS↗
DOE OSTI2021

Advanced Performance Metrics and their Application to the Sensitivity Analysis for Model Validation and Calibration

High-quality generator dynamic models are critical to reliable and accurate power systems studies and planning. With the availability of PMUs, measurement-based approach for model validation has gained significant prominence. In this approach, the quality of a model is analyzed by visually comparing measured generator response with the model-based simulated response for large system disturbances. This paper proposes a new set of performance metrics to assess the model validation results to facilitate automation of the model validation process. In the proposed methodology, first, the slow governor response and comparatively faster oscillatory response are separated, and then a separate set of performance metrics is calculated for each of these two components. These proposed metrics quantify the mismatch between the actual and model-based response in a comprehensive manner without missing any information enabling automation of the process. Furthermore, in this paper, we are also proposing that the sensitivity analysis for model calibration be performed with respect to the proposed metrics for the systematic identification of key parameters. In this work, results obtained using both simulated and real-world case-studies validate the effectiveness of the proposed performance metrics for model validation and their application to the sensitivity analysis for model calibration.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2023

Metrics for Decision-Making in Energy Justice

Energy equity and justice have become priority considerations for policymakers, practitioners, and scholars alike. To ensure that energy equity is incorporated into actual decisions and analysis, it is necessary to design, use, and continually improve energy equity metrics. In this article, we review the literature and practices surrounding such metrics. We present a working definition for energy justice and equity, and connect them to both criteria for and frameworks of metrics. We then present a large sampling of energy equity metrics, including those focused on vulnerability, wealth creation, energy poverty, life cycle, and comparative country-level dynamics. We conclude with a discussion of the limitations, gaps, and trade-offs associated with these various metrics and their interactions thereof.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗