Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Metric”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6
All sourcesNASA NTRSDOE OSTI
NASA NTRS1998

Testing, Requirements, and Metrics

The criticality of correct, complete, testable requirements is a fundamental tenet of software engineering. Also critical is complete requirements based testing of the final product. Modern tools for managing requirements allow new metrics to be used in support of both of these critical processes. Using these tools, potential problems with the quality of the requirements and the test plan can be identified early in the life cycle. Some of these quality factors include: ambiguous or incomplete requirements, poorly designed requirements databases, excessive or insufficient test cases, and incomplete linkage of tests to requirements. This paper discusses how metrics can be used to evaluate the quality of the requirements and test to avoid problems later. Requirements management and requirements based testing have always been critical in the implementation of high quality software systems. Recently, automated tools have become available to support requirements management. At NASA's Goddard Space Flight Center (GSFC), automated requirements management tools are being used on several large projects. The use of these tools opens the door to innovative uses of metrics in characterizing test plan quality and assessing overall testing risks. In support of these projects, the Software Assurance Technology Center (SATC) is working to develop and apply a metrics program that utilizes the information now available through the application of requirements management tools. Metrics based on this information provides real-time insight into the testing of requirements and these metrics assist the Project Quality Office in its testing oversight role. This paper discusses three facets of the SATC's efforts to evaluate the quality of the requirements and test plan early in the life cycle, thus preventing costly errors and time delays later.

Rosenberg, Linda↗
NASA NTRS2001

A Sensor-Independent Gust Hazard Metric

A procedure for calculating an intuitive hazard metric for gust effects on airplanes is described. The hazard metric is for use by pilots and is intended to replace subjective pilot reports (PIREPs) of the turbulence level. The hazard metric is composed of three numbers: the first describes the average airplane response to the turbulence, the second describes the positive peak airplane response to the gusts, and the third describes the negative peak airplane response to the gusts. The hazard metric is derived from any time history of vertical gust measurements and is thus independent of the sensor making the gust measurements. The metric is demonstrated for one simulated airplane encountering different types of gusts including those derived from flight data recorder measurements of actual accidents. The simulated airplane responses to the gusts compare favorably with the hazard metric.

Stewart, Eric C.↗
NASA NTRS2002

Evaluation of Standard Gear Metrics in Helicopter Flight Operation

Each false alarm made by a machine monitoring system carries a high price tag. The machine must be taken out of service, thoroughly inspected with possible disassembly, and then made ready for service. Loss of use of the machine and the efforts to inspect it are costly. In addition, if a monitoring system is prone to false alarms, the system will soon be turned off or ignored. For aircraft applications, one growing concern is that the dynamic flight environment differs from the laboratory environment where fault detection methods are developed and tested. Vibration measurements made in flight are less stationary than those made in a laboratory, or test facility, and thus a given fault detection method may produce more false alarms in flight than might be anticipated. In 1977. Stewart introduced several metrics, including FM0 and FM4, for evaluating the health of a gear. These metrics are single valued functions of the vibration signal that indicate if the signal deviates from an ideal model of the signal. FM0 is a measure of the ratio of the peak-to-peak level to the harmonic energy in the signal. FM4 is the kurtosis of the signal with the gear mesh harmonics and first order side bands removed. The underlying theory is that a vibration signal from a gear in good condition is expected to be dominated by a periodic signal at the gear mesh frequency. If one or a small number of gear teeth contain damage or faults, the signal will change, possibly showing increased amplitude, local phase changes or both near the damaged region of the gear. FM0 increases if a signal contains a local increase in amplitude. FM4 increases if a signal contains a local increase in amplitude or local phase change in a periodic signal. Over the years, other single value metrics were also introduced to detect the onset and growth of damage in gears. These various metrics have detected faults in several gear tests in experimental test rigs. Conditions in these tests have been steady state in the sense that the rpm, torque and forces on the gear have been held steady. For gears used in a dynamic environment such as that occurring in aircraft, the rpm, torque and forces on the gear are constantly changing. The authors have measured significant variation in rpm and torque in the transmissions of helicopters in controlled steady flight conditions flown by highly proficient test pilots. Statistical analyses of the data taken in flight show significant nonstationarity in the vibration measurements. These deviations from stationarity may increase false alarms in gear monitoring during aircraft flight. In the proposed paper, the authors will study vibration measurements made in flight on an AH- 1 Cobra and an OH-58C Kiowa helicopters. The primary focus will be the development of a methodology to assess the impact of nonstationarity on false alarms. Issues to be addressed include how time synchronous averages are constructed from raw data as well as how lack of stationarity effects the behavior of single value metrics. Emphasis will be placed on the occurrence of false alarms with the use of standard metrics. In order to maintain an acceptable level of false alarms in the flight environment, this study will also address the determination of appropriate threshold levels, which may need to be higher than for test rigs.

Mosher, M.↗
NASA NTRS2003

NASA Aviation Safety Program Systems Analysis/Program Assessment Metrics Review

The goal of this project is to evaluate the metrics and processes used by NASA's Aviation Safety Program in assessing technologies that contribute to NASA's aviation safety goals. There were three objectives for reaching this goal. First, NASA's main objectives for aviation safety were documented and their consistency was checked against the main objectives of the Aviation Safety Program. Next, the metrics used for technology investment by the Program Assessment function of AvSP were evaluated. Finally, other metrics that could be used by the Program Assessment Team (PAT) were identified and evaluated. This investigation revealed that the objectives are in fact consistent across organizational levels at NASA and with the FAA. Some of the major issues discussed in this study which should be further investigated, are the removal of the Cost and Return-on-Investment metrics, the lack of the metrics to measure the balance of investment and technology, the interdependencies between some of the metric risk driver categories, and the conflict between 'fatal accident rate' and 'accident rate' in the language of the Aviation Safety goal as stated in different sources.

Louis, Garrick E.↗
NASA NTRS1998

Towards a Visual Quality Metric for Digital Video

The advent of widespread distribution of digital video creates a need for automated methods for evaluating visual quality of digital video. This is particularly so since most digital video is compressed using lossy methods, which involve the controlled introduction of potentially visible artifacts. Compounding the problem is the bursty nature of digital video, which requires adaptive bit allocation based on visual quality metrics. In previous work, we have developed visual quality metrics for evaluating, controlling, and optimizing the quality of compressed still images. These metrics incorporate simplified models of human visual sensitivity to spatial and chromatic visual signals. The challenge of video quality metrics is to extend these simplified models to temporal signals as well. In this presentation I will discuss a number of the issues that must be resolved in the design of effective video quality metrics. Among these are spatial, temporal, and chromatic sensitivity and their interactions, visual masking, and implementation complexity. I will also touch on the question of how to evaluate the performance of these metrics.

Watson, Andrew B.↗
NASA NTRS2006

Metrics for Linear Kinematic Features in Sea Ice

The treatment of leads as cracks or discontinuities (see Coon et al. presentation) requires some shift in the procedure of evaluation and comparison of lead-resolving models and their validation against observations. Common metrics used to evaluate ice model skills are by and large an adaptation of a least square "metric" adopted from operational numerical weather prediction data assimilation systems and are most appropriate for continuous fields and Eilerian systems where the observations and predictions are commensurate. However, this class of metrics suffers from some flaws in areas of sharp gradients and discontinuities (e.g., leads) and when Lagrangian treatments are more natural. After a brief review of these metrics and their performance in areas of sharp gradients, we present two new metrics specifically designed to measure model accuracy in representing linear features (e.g., leads). The indices developed circumvent the requirement that both the observations and model variables be commensurate (i.e., measured with the same units) by considering the frequencies of the features of interest/importance. We illustrate the metrics by scoring several hypothetical "simulated" discontinuity fields against the lead interpreted from RGPS observations.

Levy, G.↗
NASA NTRS2009

Evaluating Algorithm Performance Metrics Tailored for Prognostics

Prognostics has taken a center stage in Condition Based Maintenance (CBM) where it is desired to estimate Remaining Useful Life (RUL) of the system so that remedial measures may be taken in advance to avoid catastrophic events or unwanted downtimes. Validation of such predictions is an important but difficult proposition and a lack of appropriate evaluation methods renders prognostics meaningless. Evaluation methods currently used in the research community are not standardized and in many cases do not sufficiently assess key performance aspects expected out of a prognostics algorithm. In this paper we introduce several new evaluation metrics tailored for prognostics and show that they can effectively evaluate various algorithms as compared to other conventional metrics. Specifically four algorithms namely; Relevance Vector Machine (RVM), Gaussian Process Regression (GPR), Artificial Neural Network (ANN), and Polynomial Regression (PR) are compared. These algorithms vary in complexity and their ability to manage uncertainty around predicted estimates. Results show that the new metrics rank these algorithms in different manner and depending on the requirements and constraints suitable metrics may be chosen. Beyond these results, these metrics offer ideas about how metrics suitable to prognostics may be designed so that the evaluation procedure can be standardized. 1

Saxena, Abhinav↗
NASA NTRS2009

On Applying the Prognostic Performance Metrics

Prognostics performance evaluation has gained significant attention in the past few years. As prognostics technology matures and more sophisticated methods for prognostic uncertainty management are developed, a standardized methodology for performance evaluation becomes extremely important to guide improvement efforts in a constructive manner. This paper is in continuation of previous efforts where several new evaluation metrics tailored for prognostics were introduced and were shown to effectively evaluate various algorithms as compared to other conventional metrics. Specifically, this paper presents a detailed discussion on how these metrics should be interpreted and used. Several shortcomings identified, while applying these metrics to a variety of real applications, are also summarized along with discussions that attempt to alleviate these problems. Further, these metrics have been enhanced to include the capability of incorporating probability distribution information from prognostic algorithms as opposed to evaluation based on point estimates only. Several methods have been suggested and guidelines have been provided to help choose one method over another based on probability distribution characteristics. These approaches also offer a convenient and intuitive visualization of algorithm performance with respect to some of these new metrics like prognostic horizon and alpha-lambda performance, and also quantify the corresponding performance while incorporating the uncertainty information.

Saxena, Abhinav↗
NASA NTRS2010

Metrics for Offline Evaluation of Prognostic Performance

Prognostic performance evaluation has gained significant attention in the past few years. Currently, prognostics concepts lack standard definitions and suffer from ambiguous and inconsistent interpretations. This lack of standards is in part due to the varied end-user requirements for different applications, time scales, available information, domain dynamics, etc. to name a few. The research community has used a variety of metrics largely based on convenience and their respective requirements. Very little attention has been focused on establishing a standardized approach to compare different efforts. This paper presents several new evaluation metrics tailored for prognostics that were recently introduced and were shown to effectively evaluate various algorithms as compared to other conventional metrics. Specifically, this paper presents a detailed discussion on how these metrics should be interpreted and used. These metrics have the capability of incorporating probabilistic uncertainty estimates from prognostic algorithms. In addition to quantitative assessment they also offer a comprehensive visual perspective that can be used in designing the prognostic system. Several methods are suggested to customize these metrics for different applications. Guidelines are provided to help choose one method over another based on distribution characteristics. Various issues faced by prognostics and its performance evaluation are discussed followed by a formal notational framework to help standardize subsequent developments.

Saxena, Abhinav↗
NASA NTRS2005

Semantic Metrics for Analysis of Software

A recently conceived suite of object-oriented software metrics focus is on semantic aspects of software, in contradistinction to traditional software metrics, which focus on syntactic aspects of software. Semantic metrics represent a more human-oriented view of software than do syntactic metrics. The semantic metrics of a given computer program are calculated by use of the output of a knowledge-based analysis of the program, and are substantially more representative of software quality and more readily comprehensible from a human perspective than are the syntactic metrics.

Etzkorn, Letha H.↗
NASA NTRS2016

Up Periscope! Designing a New Perceptual Metric for Imaging System Performance

Modern electronic imaging systems include optics, sensors, sampling, noise, processing, compression, transmission and display elements, and are viewed by the human eye. Many of these elements cannot be assessed by traditional imaging system metrics such as the MTF. More complex metrics such as NVTherm do address these elements, but do so largely through parametric adjustment of an MTF-like metric. The parameters are adjusted through subjective testing of human observers identifying specific targets in a set of standard images. We have designed a new metric that is based on a model of human visual pattern classification. In contrast to previous metrics, ours simulates the human observer identifying the standard targets. One application of this metric is to quantify performance of modern electronic periscope systems on submarines.

imaging↗
NASA NTRS2018

Overview of Metrics Pertinent to Human Response to UAM Noise

This presentation reviews metrics pertinent to noise from eVTOL aircraft, including Noise dose metrics; DNL, CTL; Statistical metrics; Spectrum-based metrics; metrics based on event frequency; and metrics based on partial loudness and detection.

aircraft noise disturbance↗
NASA NTRS2018

Psychoacoustic Test to Determine Sound Quality Metric Indicators of Rotorcraft Noise Annoyance

Noise certification metrics such as Effective Perceived Noise Level and Sound Exposure Level are used to ensure that helicopters meet regulations, but these metrics may not be good indicators of annoyance since noise complaints against helicopters persist. Sound quality (SQ) metrics, specifically fluctuation strength, tonality, impulsiveness, roughness, and sharpness, are explored to determine their relationship with annoyance. A psychoacoustic test was conducted at the NASA Langley Research Center Exterior Effects Room to assess annoyance to helicopter-like sounds over a range of SQ metric values. The amplitude, phase, and frequency of the AS350 helicopter main and tail rotor blade passage signal harmonics were manipulated to produce 105 unique helicopter-like sounds with prescribed values of SQ metrics. All sounds were set to roughly the same loudness level. These sounds were played to 40 subjects who rated each sound for annoyance. Analyses given in this paper point to which SQ metrics are important to the helicopter noise annoyance response.

Krishnamurthy, Siddhartha↗
NASA NTRSDate not supplied

Mineralogical, Elemental and Tomographic Reconnaissance Investigation for CLPS (“METRIC”)

METRIC comprises a suite of two instruments in a Lunar lander that perform X-ray diffraction (XRD) for mineral structure, X-ray fluorescence (XRF) for elemental composition, and X-ray Micro Computed Tomography (XCT) for 3D internal micromorphology. The instruments are accompanied by optical-near IR cameras to provide local geologic context. The Honeybee Robotics PlanetVac pneumatic sampling and transfer system [1],positioned on a lander footpad, will deliver sieved regolith to the X-ray instruments for analysis. The instrument suite is intended for delivery to the lunar surface on a Commercial Lunar Payload Services (CLPS) lander. The METRICXRD/F instrument draws on heritage from the Mars Science Laboratory CheMin instrument [2] and improves upon the design in multiple ways [3]. Like CheMin, Rietveld refinement and full-pattern fitting of METRIC XRD data can identify minerals at a detection limit of ~1 wt.%, quantify their abundances when present at >3 wt.%, and determine mineral composition (e.g., Fo#in olivine) from lattice parameters for minerals present at >5 wt.%[4,5]. An optimized XRF geometry provides for improved detection and quantification of major, minor and trace elements. The METRIC XCT instrument is a miniaturized X-ray Computed Tomography scanner[6]. Lunar regolith is delivered to a 3 mm diameter, 10 mm long graphite tube inside the instrument. The tube is rotated through 360°in 0.9deg.incrementsand a divergent point source X-ray beam is directed through the material. A Charge Coupled Device (CCD)records attenuation images whose brightness and contrast area function of average atomic number and density. Quantitative data, including particle and void sizes, 3D particle shape parameters, modal volumes and pore geometry can be derived from the resulting 3D reconstructions(voxel resolution: 30 μm). Crystal morphologies derived from METRIC XCT data complement the bulk mineralogy determined by the METRIC XRD/F and provide a measure of grain size distribution for the different phases. Taken together, the METRIC instrument suite determines crystal structure, elemental composition and morphology, three principal characteristics of geological materials that are highly useful in determining the origin and subsequent processing of lunar regolith.

Moon↗
DOE OSTI2026

Resilience Metrics for Solar Photovoltaics

This workshop presentation proposes the development of solar photovoltaic (PV) system resilience metrics and a methodology and framework for evaluation of PV resilience metrics. PV resilience metrics are needed to establish a consistent basis for reporting, evaluation, and data collection by industry, evaluate performance of PV systems that have been subject to natural hazards, correlating resilience to system attributes, and predicting resilience for any PV system. PV resilience metrics can guide improved system design, standards, and insurance coverage. Establishing consistent metrics can foster data collection on impacts of natural hazards on PV systems.

14 SOLAR ENERGY↗
DOE OSTI2022

Improving energy performance metrics to maximize the benefits of disruptive technologies

Advances in appliance and equipment systems' technologies also cause changes in their technical characteristics that can affect energy consumption and associated greenhouse gas emissions. Energy performance evaluation methods that use testing and metrics can evolve to account for energy consumption contributed from new technological characteristics. However, there is no systematic effort to enable the advancement of test procedures and performance metrics to maximize the energy, climate, and financial benefits of disruptive technologies. Therefore, it is important for countries to regularly update their energy-efficiency policy programs, such as standards and labels, by improving test procedures and metrics to reflect the innovation in emerging technologies as well as to mitigate the risk of deploying obsolete technology. This paper shows how energy performance metrics have been improving in selected appliances and equipment and provides insights to design a systematic effort to improve metrics and test procedures at the speed of ongoing changes in technologies and markets.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2022

Characterization factors and other air quality impact metrics: Case study for PM 2.5 -emitting area sources from biofuel feedstock supply

In this paper, we develop a framework and metrics for estimating the impact of emission sources on regulatory compliance and human health for applications in air quality planning and life cycle impact assessment (LCIA). Our framework is based on a pollutant's characterization factor (CF) and three new metrics: Available Regulatory Capacity for Incremental Emissions (ARCIE), Source CF Ratio, and Activity Health Impact (AHI) Ratio. ARCIE can be used to assess whether a receptor location has capacity to accommodate additional source emissions while complying with regulatory limits. We present CF as a midpoint indicator of health impacts per unit mass of emitted pollutant. Source CF Ratio enables comparison of potential new-source locations based on human health impacts. The AHI Ratio estimates the health impacts of a pollutant in relation to the utilization of the source for each unit of product or service. These metrics can be applied to any pollutant, energy source sector (e.g., agriculture, electricity), source type (point, line, area), and spatial modeling domain (nation, state, city, region). We demonstrate these metrics through a case study of fine particulate (PM 2.5 ) emissions from U.S. corn stover harvesting and local processing at various scales, representing steps in the biofuel production process. We model PM 2.5 formation in the atmosphere using a novel reduced-complexity chemical transport model called the Intervention Model for Air Pollution (InMAP). Through this case study, we present the first area-source PM 2.5 CFs that address the recommendations of several LCIA studies to establish spatially explicit CFs specific to an energy source sector or type. Overall, the framework developed in this work provides multiple new ways to consider the potential impacts of air emissions through spatially differentiated metrics.

09 BIOMASS FUELS↗
DOE OSTI2021

An evaluation of electricity system technology competitiveness metrics: The case for profitability

In this work we develop standardized functional forms for electricity technology competitiveness metrics and show how the mathematical relationship between value and cost can affect the robustness of the metric. Here, we make the case to replace established metrics with economic profitability metrics – including a new profitability-based adjustment to LCOE. These profitability metrics are shown to be more robust for evaluating technology competitiveness by keeping comparisons on an equivalent monetary basis.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗