Engineering PapersSearch

SEARCH · Engineering Papers

Results for “Metrics”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4

A Sensor-Independent Gust Hazard Metric

A procedure for calculating an intuitive hazard metric for gust effects on airplanes is described. The hazard metric is for use by pilots and is intended to replace subjective pilot reports (PIREPs) of the turbulence level. The hazard metric is composed of three numbers: the first describes the average airplane response to the turbulence, the second describes the positive peak airplane response to the gusts, and the third describes the negative peak airplane response to the gusts. The hazard metric is derived from any time history of vertical gust measurements and is thus independent of the sensor making the gust measurements. The metric is demonstrated for one simulated airplane encountering different types of gusts including those derived from flight data recorder measurements of actual accidents. The simulated airplane responses to the gusts compare favorably with the hazard metric.

Stewart, Eric C.

Evaluation of Standard Gear Metrics in Helicopter Flight Operation

Each false alarm made by a machine monitoring system carries a high price tag. The machine must be taken out of service, thoroughly inspected with possible disassembly, and then made ready for service. Loss of use of the machine and the efforts to inspect it are costly. In addition, if a monitoring system is prone to false alarms, the system will soon be turned off or ignored. For aircraft applications, one growing concern is that the dynamic flight environment differs from the laboratory environment where fault detection methods are developed and tested. Vibration measurements made in flight are less stationary than those made in a laboratory, or test facility, and thus a given fault detection method may produce more false alarms in flight than might be anticipated. In 1977. Stewart introduced several metrics, including FM0 and FM4, for evaluating the health of a gear. These metrics are single valued functions of the vibration signal that indicate if the signal deviates from an ideal model of the signal. FM0 is a measure of the ratio of the peak-to-peak level to the harmonic energy in the signal. FM4 is the kurtosis of the signal with the gear mesh harmonics and first order side bands removed. The underlying theory is that a vibration signal from a gear in good condition is expected to be dominated by a periodic signal at the gear mesh frequency. If one or a small number of gear teeth contain damage or faults, the signal will change, possibly showing increased amplitude, local phase changes or both near the damaged region of the gear. FM0 increases if a signal contains a local increase in amplitude. FM4 increases if a signal contains a local increase in amplitude or local phase change in a periodic signal. Over the years, other single value metrics were also introduced to detect the onset and growth of damage in gears. These various metrics have detected faults in several gear tests in experimental test rigs. Conditions in these tests have been steady state in the sense that the rpm, torque and forces on the gear have been held steady. For gears used in a dynamic environment such as that occurring in aircraft, the rpm, torque and forces on the gear are constantly changing. The authors have measured significant variation in rpm and torque in the transmissions of helicopters in controlled steady flight conditions flown by highly proficient test pilots. Statistical analyses of the data taken in flight show significant nonstationarity in the vibration measurements. These deviations from stationarity may increase false alarms in gear monitoring during aircraft flight. In the proposed paper, the authors will study vibration measurements made in flight on an AH- 1 Cobra and an OH-58C Kiowa helicopters. The primary focus will be the development of a methodology to assess the impact of nonstationarity on false alarms. Issues to be addressed include how time synchronous averages are constructed from raw data as well as how lack of stationarity effects the behavior of single value metrics. Emphasis will be placed on the occurrence of false alarms with the use of standard metrics. In order to maintain an acceptable level of false alarms in the flight environment, this study will also address the determination of appropriate threshold levels, which may need to be higher than for test rigs.

Mosher, M.

NASA Aviation Safety Program Systems Analysis/Program Assessment Metrics Review

The goal of this project is to evaluate the metrics and processes used by NASA's Aviation Safety Program in assessing technologies that contribute to NASA's aviation safety goals. There were three objectives for reaching this goal. First, NASA's main objectives for aviation safety were documented and their consistency was checked against the main objectives of the Aviation Safety Program. Next, the metrics used for technology investment by the Program Assessment function of AvSP were evaluated. Finally, other metrics that could be used by the Program Assessment Team (PAT) were identified and evaluated. This investigation revealed that the objectives are in fact consistent across organizational levels at NASA and with the FAA. Some of the major issues discussed in this study which should be further investigated, are the removal of the Cost and Return-on-Investment metrics, the lack of the metrics to measure the balance of investment and technology, the interdependencies between some of the metric risk driver categories, and the conflict between 'fatal accident rate' and 'accident rate' in the language of the Aviation Safety goal as stated in different sources.

Louis, Garrick E.

Towards a Visual Quality Metric for Digital Video

The advent of widespread distribution of digital video creates a need for automated methods for evaluating visual quality of digital video. This is particularly so since most digital video is compressed using lossy methods, which involve the controlled introduction of potentially visible artifacts. Compounding the problem is the bursty nature of digital video, which requires adaptive bit allocation based on visual quality metrics. In previous work, we have developed visual quality metrics for evaluating, controlling, and optimizing the quality of compressed still images. These metrics incorporate simplified models of human visual sensitivity to spatial and chromatic visual signals. The challenge of video quality metrics is to extend these simplified models to temporal signals as well. In this presentation I will discuss a number of the issues that must be resolved in the design of effective video quality metrics. Among these are spatial, temporal, and chromatic sensitivity and their interactions, visual masking, and implementation complexity. I will also touch on the question of how to evaluate the performance of these metrics.

Watson, Andrew B.

Metrics for Linear Kinematic Features in Sea Ice

The treatment of leads as cracks or discontinuities (see Coon et al. presentation) requires some shift in the procedure of evaluation and comparison of lead-resolving models and their validation against observations. Common metrics used to evaluate ice model skills are by and large an adaptation of a least square "metric" adopted from operational numerical weather prediction data assimilation systems and are most appropriate for continuous fields and Eilerian systems where the observations and predictions are commensurate. However, this class of metrics suffers from some flaws in areas of sharp gradients and discontinuities (e.g., leads) and when Lagrangian treatments are more natural. After a brief review of these metrics and their performance in areas of sharp gradients, we present two new metrics specifically designed to measure model accuracy in representing linear features (e.g., leads). The indices developed circumvent the requirement that both the observations and model variables be commensurate (i.e., measured with the same units) by considering the frequencies of the features of interest/importance. We illustrate the metrics by scoring several hypothetical "simulated" discontinuity fields against the lead interpreted from RGPS observations.

Levy, G.

Evaluating Algorithm Performance Metrics Tailored for Prognostics

Prognostics has taken a center stage in Condition Based Maintenance (CBM) where it is desired to estimate Remaining Useful Life (RUL) of the system so that remedial measures may be taken in advance to avoid catastrophic events or unwanted downtimes. Validation of such predictions is an important but difficult proposition and a lack of appropriate evaluation methods renders prognostics meaningless. Evaluation methods currently used in the research community are not standardized and in many cases do not sufficiently assess key performance aspects expected out of a prognostics algorithm. In this paper we introduce several new evaluation metrics tailored for prognostics and show that they can effectively evaluate various algorithms as compared to other conventional metrics. Specifically four algorithms namely; Relevance Vector Machine (RVM), Gaussian Process Regression (GPR), Artificial Neural Network (ANN), and Polynomial Regression (PR) are compared. These algorithms vary in complexity and their ability to manage uncertainty around predicted estimates. Results show that the new metrics rank these algorithms in different manner and depending on the requirements and constraints suitable metrics may be chosen. Beyond these results, these metrics offer ideas about how metrics suitable to prognostics may be designed so that the evaluation procedure can be standardized. 1

Saxena, Abhinav

On Applying the Prognostic Performance Metrics

Prognostics performance evaluation has gained significant attention in the past few years. As prognostics technology matures and more sophisticated methods for prognostic uncertainty management are developed, a standardized methodology for performance evaluation becomes extremely important to guide improvement efforts in a constructive manner. This paper is in continuation of previous efforts where several new evaluation metrics tailored for prognostics were introduced and were shown to effectively evaluate various algorithms as compared to other conventional metrics. Specifically, this paper presents a detailed discussion on how these metrics should be interpreted and used. Several shortcomings identified, while applying these metrics to a variety of real applications, are also summarized along with discussions that attempt to alleviate these problems. Further, these metrics have been enhanced to include the capability of incorporating probability distribution information from prognostic algorithms as opposed to evaluation based on point estimates only. Several methods have been suggested and guidelines have been provided to help choose one method over another based on probability distribution characteristics. These approaches also offer a convenient and intuitive visualization of algorithm performance with respect to some of these new metrics like prognostic horizon and alpha-lambda performance, and also quantify the corresponding performance while incorporating the uncertainty information.

Saxena, Abhinav

Metrics for Offline Evaluation of Prognostic Performance

Prognostic performance evaluation has gained significant attention in the past few years. Currently, prognostics concepts lack standard definitions and suffer from ambiguous and inconsistent interpretations. This lack of standards is in part due to the varied end-user requirements for different applications, time scales, available information, domain dynamics, etc. to name a few. The research community has used a variety of metrics largely based on convenience and their respective requirements. Very little attention has been focused on establishing a standardized approach to compare different efforts. This paper presents several new evaluation metrics tailored for prognostics that were recently introduced and were shown to effectively evaluate various algorithms as compared to other conventional metrics. Specifically, this paper presents a detailed discussion on how these metrics should be interpreted and used. These metrics have the capability of incorporating probabilistic uncertainty estimates from prognostic algorithms. In addition to quantitative assessment they also offer a comprehensive visual perspective that can be used in designing the prognostic system. Several methods are suggested to customize these metrics for different applications. Guidelines are provided to help choose one method over another based on distribution characteristics. Various issues faced by prognostics and its performance evaluation are discussed followed by a formal notational framework to help standardize subsequent developments.

Saxena, Abhinav

Semantic Metrics for Analysis of Software

A recently conceived suite of object-oriented software metrics focus is on semantic aspects of software, in contradistinction to traditional software metrics, which focus on syntactic aspects of software. Semantic metrics represent a more human-oriented view of software than do syntactic metrics. The semantic metrics of a given computer program are calculated by use of the output of a knowledge-based analysis of the program, and are substantially more representative of software quality and more readily comprehensible from a human perspective than are the syntactic metrics.

Etzkorn, Letha H.

Up Periscope! Designing a New Perceptual Metric for Imaging System Performance

Modern electronic imaging systems include optics, sensors, sampling, noise, processing, compression, transmission and display elements, and are viewed by the human eye. Many of these elements cannot be assessed by traditional imaging system metrics such as the MTF. More complex metrics such as NVTherm do address these elements, but do so largely through parametric adjustment of an MTF-like metric. The parameters are adjusted through subjective testing of human observers identifying specific targets in a set of standard images. We have designed a new metric that is based on a model of human visual pattern classification. In contrast to previous metrics, ours simulates the human observer identifying the standard targets. One application of this metric is to quantify performance of modern electronic periscope systems on submarines.

imaging

Overview of Metrics Pertinent to Human Response to UAM Noise

This presentation reviews metrics pertinent to noise from eVTOL aircraft, including Noise dose metrics; DNL, CTL; Statistical metrics; Spectrum-based metrics; metrics based on event frequency; and metrics based on partial loudness and detection.

aircraft noise disturbance

Psychoacoustic Test to Determine Sound Quality Metric Indicators of Rotorcraft Noise Annoyance

Noise certification metrics such as Effective Perceived Noise Level and Sound Exposure Level are used to ensure that helicopters meet regulations, but these metrics may not be good indicators of annoyance since noise complaints against helicopters persist. Sound quality (SQ) metrics, specifically fluctuation strength, tonality, impulsiveness, roughness, and sharpness, are explored to determine their relationship with annoyance. A psychoacoustic test was conducted at the NASA Langley Research Center Exterior Effects Room to assess annoyance to helicopter-like sounds over a range of SQ metric values. The amplitude, phase, and frequency of the AS350 helicopter main and tail rotor blade passage signal harmonics were manipulated to produce 105 unique helicopter-like sounds with prescribed values of SQ metrics. All sounds were set to roughly the same loudness level. These sounds were played to 40 subjects who rated each sound for annoyance. Analyses given in this paper point to which SQ metrics are important to the helicopter noise annoyance response.

Krishnamurthy, Siddhartha

Mineralogical, Elemental and Tomographic Reconnaissance Investigation for CLPS (“METRIC”)

METRIC comprises a suite of two instruments in a Lunar lander that perform X-ray diffraction (XRD) for mineral structure, X-ray fluorescence (XRF) for elemental composition, and X-ray Micro Computed Tomography (XCT) for 3D internal micromorphology. The instruments are accompanied by optical-near IR cameras to provide local geologic context. The Honeybee Robotics PlanetVac pneumatic sampling and transfer system [1],positioned on a lander footpad, will deliver sieved regolith to the X-ray instruments for analysis. The instrument suite is intended for delivery to the lunar surface on a Commercial Lunar Payload Services (CLPS) lander. The METRICXRD/F instrument draws on heritage from the Mars Science Laboratory CheMin instrument [2] and improves upon the design in multiple ways [3]. Like CheMin, Rietveld refinement and full-pattern fitting of METRIC XRD data can identify minerals at a detection limit of ~1 wt.%, quantify their abundances when present at >3 wt.%, and determine mineral composition (e.g., Fo#in olivine) from lattice parameters for minerals present at >5 wt.%[4,5]. An optimized XRF geometry provides for improved detection and quantification of major, minor and trace elements. The METRIC XCT instrument is a miniaturized X-ray Computed Tomography scanner[6]. Lunar regolith is delivered to a 3 mm diameter, 10 mm long graphite tube inside the instrument. The tube is rotated through 360°in 0.9deg.incrementsand a divergent point source X-ray beam is directed through the material. A Charge Coupled Device (CCD)records attenuation images whose brightness and contrast area function of average atomic number and density. Quantitative data, including particle and void sizes, 3D particle shape parameters, modal volumes and pore geometry can be derived from the resulting 3D reconstructions(voxel resolution: 30 μm). Crystal morphologies derived from METRIC XCT data complement the bulk mineralogy determined by the METRIC XRD/F and provide a measure of grain size distribution for the different phases. Taken together, the METRIC instrument suite determines crystal structure, elemental composition and morphology, three principal characteristics of geological materials that are highly useful in determining the origin and subsequent processing of lunar regolith.

Moon

Resilience Metrics for Solar Photovoltaics

This workshop presentation proposes the development of solar photovoltaic (PV) system resilience metrics and a methodology and framework for evaluation of PV resilience metrics. PV resilience metrics are needed to establish a consistent basis for reporting, evaluation, and data collection by industry, evaluate performance of PV systems that have been subject to natural hazards, correlating resilience to system attributes, and predicting resilience for any PV system. PV resilience metrics can guide improved system design, standards, and insurance coverage. Establishing consistent metrics can foster data collection on impacts of natural hazards on PV systems.

14 SOLAR ENERGY

Performance Metrics, Error Modeling, and Uncertainty Quantification

A common set of statistical metrics has been used to summarize the performance of models or measurements-­ the most widely used ones being bias, mean square error, and linear correlation coefficient. They assume linear, additive, Gaussian errors, and they are interdependent, incomplete, and incapable of directly quantifying un­certainty. The authors demonstrate that these metrics can be directly derived from the parameters of the simple linear error model. Since a correct error model captures the full error information, it is argued that the specification of a parametric error model should be an alternative to the metrics-based approach. The error-modeling meth­odology is applicable to both linear and nonlinear errors, while the metrics are only meaningful for linear errors. In addition, the error model expresses the error structure more naturally, and directly quantifies uncertainty. This argument is further explained by highlighting the intrinsic connections between the performance metrics, the error model, and the joint distribution between the data and the reference.

Quantification

Metric Learning to Accelerate Convergence of Operator Splitting Methods

Recent developments in machine learning have led to promising advances in accelerating the solution of constrained optimization problems. Increasing demand for real-time decision-making capabilities in applications such as artificial intelligence and optimal control has led to a variety of proposed strategies for learning to produce fast solutions to optimization problems. For example, recent works have shown that it is possible to accelerate the convergence of optimization algorithms by learning to select their parameters, such as gradient descent stepsizes. This work proposes a new approach, in which the underlying metric spaces of proximal operator splitting algorithms are learned to maximize convergence rate. While prior works in optimization theory have derived optimal metrics in simple cases, no such result exists for many practical problem forms including general Quadratic Programming (QP). This paper shows how differentiable optimization can enable the end-to-end learning of proximal metrics, enhancing the convergence of proximal algorithms for QP problems beyond what is possible based on known theory. Additionally, the results illustrate a strong connection between the learned proximal metrics and active constraints at the optima, leading to an interpretation in which the predicted proximal metrics can be viewed as a form of active set prediction.

King, Ethan [BATTELLE (PACIFIC NW LAB)]

Integration of equitable resilience metrics into climate-informed electric utility planning processes: phase one

Working together, Sandia National Laboratories, Southern California Edison (SCE) - an Investor-Owned Utility (IOU) - and the California Public Utilities Commission (CPUC) are studying how electric utilities can use equity and resilience metrics to help inform the prioritization and sequencing of resilience-driven infrastructure investments. To this end, this project evaluated “Social Burden,” an equitable resilience metric which measures the potential impact of disruptions in access to non-electric critical services on people and estimates community resilience to these disruptions. The Social Burden was expanded to incorporate SCE’s existing equity metric and applied to evaluate the potential impacts from a range of climate-informed hypothetical outage scenarios developed under SCE’s 2022 Climate Adaptation Vulnerability Assessment. One baseline (“blue-sky”) state and eight different outage scenarios were evaluated to measure the potential impacts of the outages on non-electric infrastructure, critical services, and people. Key findings include: 1) the Social Burden framework is flexible enough to adapt to and build upon existing utility equity and/or resilience metrics, 2) Social Burden results highlight the high degree of non-electric service redundancy within the SCE service area with most (6/8) hypothetical outage scenarios predicted to increase people’s Social Burden by less than 10%; however, 3) access to critical services and people’s ability to obtain them is unequal and spatially clustered, meaning that there are some hypothetical outage scenarios (2/8) that will exert a higher toll on communities directly experiencing the outage as well as some nearby communities with pre-existing vulnerabilities. The report concludes with recommendations for potential use cases of the expanded Social Burden metric and identifies priority follow-on work. Potential use cases may include incorporating equity into IOU’s prioritization of climate resilience investments. Additionally, Social Burden analysis may provide additional data and insights to augment grid planning, potentially by identifying additional needs and/or prioritizing previously identified needs.

24 POWER TRANSMISSION AND DISTRIBUTION

Information on the metric system and related fields

This document contains about 7,600 references on the metric system and conversion to the metric system. These references include all known documents on the metric system as of December 1975, the month of enactment of the Metric Conversion Act of 1975. This bibliography includes books, reports, articles, presentations, periodicals, legislation, motion pictures, TV series, film strips, slides, posters, wall charts, education and training courses, addresses for information, and sources for metric materials and services. A comprehensive index is provided.

Lange, E.