Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “test metrics”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2

Testing of the Apollo 15 Metric Camera System.

Description of tests conducted (1) to assess the quality of Apollo 15 Metric Camera System data and (2) to develop production procedures for total block reduction. Three strips of metric photography over the Hadley Rille area were selected for the tests. These photographs were utilized in a series of evaluation tests culminating in an orbitally constrained block triangulation solution. Results show that film deformations up to 25 and 5 microns are present in the mapping and stellar materials, respectively. Stellar reductions can provide mapping camera orientations with an accuracy that is consistent with the accuracies of other parameters in the triangulation solutions. Pointing accuracies of 4 to 10 microns can be expected for the mapping camera materials, depending on variations in resolution caused by changing sun angle conditions.

Helmering, R. J.↗

An investigation of fighter aircraft agility

This report attempts to unify in a single document the results of a series of studies on fighter aircraft agility funded by the NASA Ames Research Center, Dryden Flight Research Facility and conducted at the University of Kansas Flight Research Laboratory during the period January 1989 through December 1993. New metrics proposed by pilots and the research community to assess fighter aircraft agility are collected and analyzed. The report develops a framework for understanding the context into which the various proposed fighter agility metrics fit in terms of application and testing. Since new metrics continue to be proposed, this report does not claim to contain every proposed fighter agility metric. Flight test procedures, test constraints, and related criteria are developed. Instrumentation required to quantify agility via flight test is considered, as is the sensitivity of the candidate metrics to deviations from nominal pilot command inputs, which is studied in detail. Instead of supplying specific, detailed conclusions about the relevance or utility of one candidate metric versus another, the authors have attempted to provide sufficient data and analyses for readers to formulate their own conclusions. Readers are therefore ultimately responsible for judging exactly which metrics are 'best' for their particular needs. Additionally, it is not the intent of the authors to suggest combat tactics or other actual operational uses of the results and data in this report. This has been left up to the user community. Twenty of the candidate agility metrics were selected for evaluation with high fidelity, nonlinear, non real-time flight simulation computer programs of the F-5A Freedom Fighter, F-16A Fighting Falcon, F-18A Hornet, and X-29A. The information and data presented on the 20 candidate metrics which were evaluated will assist interested readers in conducting their own extensive investigations. The report provides a definition and analysis of each metric; details of how to test and measure the metric, including any special data reduction requirements; typical values for the metric obtained using one or more aircraft types; and a sensitivity analysis if applicable. The report is organized as follows. The first chapter in the report presents a historical review of air combat trends which demonstrate the need for agility metrics in assessing the combat performance of fighter aircraft in a modern, all-aspect missile environment. The second chapter presents a framework for classifying each candidate metric according to time scale (transient, functional, instantaneous), further subdivided by axis (pitch, lateral, axial). The report is then broadly divided into two parts, with the transient agility metrics (pitch lateral, axial) covered in chapters three, four, and five, and the functional agility metrics covered in chapter six. Conclusions, recommendations, and an extensive reference list and biography are also included. Five appendices contain a comprehensive list of the definitions of all the candidate metrics; a description of the aircraft models and flight simulation programs used for testing the metrics; several relations and concepts which are fundamental to the study of lateral agility; an in-depth analysis of the axial agility metrics; and a derivation of the relations for the instantaneous agility and their approximations.

Valasek, John↗

Evaluation of Two Crew Module Boilerplate Tests Using Newly Developed Calibration Metrics

The paper discusses a application of multi-dimensional calibration metrics to evaluate pressure data from water drop tests of the Max Launch Abort System (MLAS) crew module boilerplate. Specifically, three metrics are discussed: 1) a metric to assess the probability of enveloping the measured data with the model, 2) a multi-dimensional orthogonality metric to assess model adequacy between test and analysis, and 3) a prediction error metric to conduct sensor placement to minimize pressure prediction errors. Data from similar (nearly repeated) capsule drop tests shows significant variability in the measured pressure responses. When compared to expected variability using model predictions, it is demonstrated that the measured variability cannot be explained by the model under the current uncertainty assumptions.

Horta, Lucas G.↗

Coverage Metrics for Requirements-Based Testing: Evaluation of Effectiveness

In black-box testing, the tester creates a set of tests to exercise a system under test without regard to the internal structure of the system. Generally, no objective metric is used to measure the adequacy of black-box tests. In recent work, we have proposed three requirements coverage metrics, allowing testers to objectively measure the adequacy of a black-box test suite with respect to a set of requirements formalized as Linear Temporal Logic (LTL) properties. In this report, we evaluate the effectiveness of these coverage metrics with respect to fault finding. Specifically, we conduct an empirical study to investigate two questions: (1) do test suites satisfying a requirements coverage metric provide better fault finding than randomly generated test suites of approximately the same size?, and (2) do test suites satisfying a more rigorous requirements coverage metric provide better fault finding than test suites satisfying a less rigorous requirements coverage metric? Our results indicate (1) only one coverage metric proposed -- Unique First Cause (UFC) coverage -- is sufficiently rigorous to ensure test suites satisfying the metric outperform randomly generated test suites of similar size and (2) that test suites satisfying more rigorous coverage metrics provide better fault finding than test suites satisfying less rigorous coverage metrics.

Staats, Matt↗

Crew Exploration Vehicle (CEV) (Orion) Occupant Protection

The purpose of this study was to determine the similarity between the response of the THUMS model and the Hybrid III Anthropometric Test Device (ATD) given existing Wright-Patterson (WP) sled tests. There were four tests selected for this comparison with frontal, spinal, rear, and lateral loading. The THUMS was placed in a sled configuration that replicated the WP configuration and the recorded seat acceleration for each test was applied to model seat. Once the modeling simulations were complete, they were compared to the WP results using two methods. The first was a visual inspection of the sled test videos compared to the THUMS d3plot files. This comparison resulted in an assessment of the overall kinematics of the two results. The other comparison was a comparison of the plotted data recorded for both tests. The metrics selected for comparison were seat acceleration, belt forces, head acceleration and chest acceleration. These metrics were recorded in all WP tests and were outputs of the THUMS model. Once the comparison of the THUMS to the WP tests was complete, the THUMS model output was also examined for possible injuries in these scenarios. These outputs included metrics for injury risk to the head, neck, thorax, lumbar spine and lower extremities. The metrics to evaluate head response were peak head acceleration, HIC15, and HIC36. For the neck, N (sub ij) was calculated. The thorax response was evaluated with peak chest acceleration, the Combined Thoracic Index (CTI), sternal deflection, chest deflection, and chest acceleration- 3 ms clip. The lumbar spine response was evaluated with lumbar spine force. Finally the lower extremity response was evaluated by femur and tibia force. The results of the simulation comparisons indicate the THUMS model had a similar response to the Hybrid III dummy given the same input. The primary difference seen between the two was a more flexible response of the THUMS compared to the Hybrid III. This flexibility was most pronounced in the neck flexion, shoulder deflection and chest deflection. Due to the flexibility of the THUMS, the resulting head and chest accelerations tended to lag the Hybrid III acceleration trace and have a lower peak value. The results of the injury metric comparison identified possible injury trends between simulations. Risk of head injury was highest for the lateral simulations. The risk of chest injury was highest for the rear impact. However, neck injury risk was approximately the same for all simulations. The injury metric value for lumbar spine force was highest for the spinal impact. The leg forces were highest for the rear and lateral impacts. The results of this comparison indicate the THUMS model performs in a similar manner as the Hybrid III ATD. The differences in the responses of model and the ATD are primarily due to the flexibility of the THUMS. This flexibility of the THUMS would be a more human like response. Based on the similarity between the two models, the THUMS should be used in further testing to assess risk of injury to the occupant.

Currie-Gregg, Nancy J.↗

Using Trajectory Smoothness Metrics to Identify Drones in Radar Track Data

The identification of unmanned aircraft systems (UAS) using trajectory data is considered. Specifically, a number of smoothness metrics are proposed, which can be used to distinguish UAS from other aerial objects even when they are engaged in accelerative maneuvers (non-constant-velocity flight). The metrics are evaluated on a data set from a UAS sense-and-avoid field test, which contains track data of aerial objects recorded by a vehicle-board radar system during a flight test. The metrics are found to effectively differentiate UAS from other objects such as birds for this data set. In addition, an initial statistical performance analysis of one of the smoothness metrics is undertaken, using 15 data sets deriving from multiple flight tests. The smoothness metric is shown to identify the target UAS with 95% accuracy (95% true positive rate), while achieving a false positive rate of less than 9%.

Sandip Roy↗

Dynamics and control of multipayload platforms - The Middeck Active Control Experiment (MACE)

A flight experiment entitled the Middeck Active Control Experiment (MACE) proposed by the Space Engineering Research Center (SERC) at the Massachusetts Institute of Technology is described. The objective of this program is to investigate and validate the modeling of the dynamics of an actively controlled flexible, articulating, multibody platform free floating in zero gravity. A rationale and experimental approach for the program are presented. The rationale shows that on-orbit testing, coupled with ground testing and a strong analytical program, is necessary in order to fully understand both how flexibility of the platform affects the pointing problem, as well as how gravity perturbs this structural flexibility causing deviations between 1-and 0-gravity behavior. The experimental approach captures the essential physics of multibody platforms, by identifying the appropriate attributes, tests, and performance metrics of the test article, and defines the tests required to successfully validate the analytical framework.

Miller, David W.↗

The MODE family of on-orbit experiments: The Middeck Active Control Experiment (MACE)

A flight experiment entitled the Middeck Active Control Experiment (MACE), proposed by the Space Engineering Research Center (SERC) at the Massachusetts Institute of Technology, is described. This is the second in a family of flight experiments being developed at MIT. The first is the Middeck 0-Gravity Dynamics Experiment (MODE) which investigates the nonlinear behavior of contained fluids and truss structures in zero gravity. The objective of the MACE program is to investigate and validate the modeling of the dynamics of an actively controlled flexible, articulating, multibody platform free floating in zero gravity. A rationale and experimental approach for the program are presented. The rationale shows that on-orbit testing, coupled with ground testing and a strong analytical program, is necessary in order to fully understand both how flexibility of the platform affects the pointing problem, as well as how gravity perturbs this structural flexibility causing deviations between 1- and 0-gravity behavior. The experimental approach captures the essential physics of multibody platforms, by identifying the appropriate attributes, tests, and performance metrics of the test article and defines the tests required to successfully validate the analytical framework.

Crawley, Edward F.↗

JPL/NASA/IEEE Test Effectiveness Workshop

(none given)From OBJECTIVES: Specific objectives of the working group are to support the innovation, development, evaluation and implementation of test methods, metrics and tools based on failure engineering/physics and/or root cause evaluations. Data sources systems and tools shall be developed and implemented that: 1) provide improved preventions, controls, analyses and tests (PACT) & field failure data collection, 2) facilities data analysis, archiving, retrieval, failure physics and/or root cause evaluations and 3) enable new and existing technology suitability evaluations to be performed.

effectiveness concurrent engineering metrics test ↗

Improving Climate Projections Using "Intelligent" Ensembles

Recent changes in the climate system have led to growing concern, especially in communities which are highly vulnerable to resource shortages and weather extremes. There is an urgent need for better climate information to develop solutions and strategies for adapting to a changing climate. Climate models provide excellent tools for studying the current state of climate and making future projections. However, these models are subject to biases created by structural uncertainties. Performance metrics-or the systematic determination of model biases-succinctly quantify aspects of climate model behavior. Efforts to standardize climate model experiments and collect simulation data-such as the Coupled Model Intercomparison Project (CMIP)-provide the means to directly compare and assess model performance. Performance metrics have been used to show that some models reproduce present-day climate better than others. Simulation data from multiple models are often used to add value to projections by creating a consensus projection from the model ensemble, in which each model is given an equal weight. It has been shown that the ensemble mean generally outperforms any single model. It is possible to use unequal weights to produce ensemble means, in which models are weighted based on performance (called "intelligent" ensembles). Can performance metrics be used to improve climate projections? Previous work introduced a framework for comparing the utility of model performance metrics, showing that the best metrics are related to the variance of top-of-atmosphere outgoing longwave radiation. These metrics improve present-day climate simulations of Earth's energy budget using the "intelligent" ensemble method. The current project identifies several approaches for testing whether performance metrics can be applied to future simulations to create "intelligent" ensemble-mean climate projections. It is shown that certain performance metrics test key climate processes in the models, and that these metrics can be used to evaluate model quality in both current and future climate states. This information will be used to produce new consensus projections and provide communities with improved climate projections for urgent decision-making.

Baker, Noel C.↗

Improving Climate Projections Using "Intelligent" Ensembles

Recent changes in the climate system have led to growing concern, especially in communities which are highly vulnerable to resource shortages and weather extremes. There is an urgent need for better climate information to develop solutions and strategies for adapting to a changing climate. Climate models provide excellent tools for studying the current state of climate and making future projections. However, these models are subject to biases created by structural uncertainties. Performance metrics-or the systematic determination of model biases-succinctly quantify aspects of climate model behavior. Efforts to standardize climate model experiments and collect simulation data-such as the Coupled Model Intercomparison Project (CMIP)-provide the means to directly compare and assess model performance. Performance metrics have been used to show that some models reproduce present-day climate better than others. Simulation data from multiple models are often used to add value to projections by creating a consensus projection from the model ensemble, in which each model is given an equal weight. It has been shown that the ensemble mean generally outperforms any single model. It is possible to use unequal weights to produce ensemble means, in which models are weighted based on performance (called "intelligent" ensembles). Can performance metrics be used to improve climate projections? Previous work introduced a framework for comparing the utility of model performance metrics, showing that the best metrics are related to the variance of top-of-atmosphere outgoing longwave radiation. These metrics improve present-day climate simulations of Earth's energy budget using the "intelligent" ensemble method. The current project identifies several approaches for testing whether performance metrics can be applied to future simulations to create "intelligent" ensemble-mean climate projections. It is shown that certain performance metrics test key climate processes in the models, and that these metrics can be used to evaluate model quality in both current and future climate states. This information will be used to produce new consensus projections and provide communities with improved climate projections for urgent decision-making.

Baker, Noel C.↗

A report on the gravitational redshift test for non-metric theories of gravitation

The frequencies of two atomic hydrogen masers and of three superconducting cavity stabilized oscillators were compared as the ensemble of oscillators was moved in the Sun's gravitational field by the rotation and orbital motion of the Earth. Metric gravitation theories predict that the gravitational redshifts of the two types of oscillators are identical, and that there should be no relative frequency shift between the oscillators; nonmetric theories, in contrast, predict a frequency shift between masers and SCSOs that is proportional to the change in solar gravitational potential experienced by the oscillators. The results are consistent with metric theories of gravitation at a level of 2%.

Source record↗

Microcomputer-based tests for repeated-measures: Metric properties and predictive validities

A menu of psychomotor and mental acuity tests were refined. Field applications of such a battery are, for example, a study of the effects of toxic agents or exotic environments on performance readiness, or the determination of fitness for duty. The key requirement of these tasks is that they be suitable for repeated-measures applications, and so questions of stability and reliability are a continuing, central focus of this work. After the initial (practice) session, seven replications of 14 microcomputer-based performance tests (32 measures) were completed by 37 subjects. Each test in the battery had previously been shown to stabilize in less than five 90-second administrations and to possess retest reliabilities greater than r = 0.707 for three minutes of testing. However, all the tests had never been administered together as a battery and they had never been self-administered. In order to provide predictive validity for intelligence measurement, the Wechsler Adult Intelligence Scale-Revised and the Wonderlic Personnel Test were obtained on the same subjects.

Kennedy, Robert S.↗

Aerocapture, Entry, Descent and Landing (AEDL) Human Planetary Landing Systems. Section 10: AEDL Analysis, Test and Validation Infrastructure

Contents include the following: 3 Listing of critical capabilities (knowledge, procedures, training, facilities) and metrics for validating that they are mission ready. Examples of critical capabilities and validation metrics: ground test and simulations. Flight testing to prove capabilities are mission ready. Issues and recommendations.

Arnold, J.↗

Psychoacoustic Test to Determine Sound Quality Metric Indicators of Rotorcraft Noise Annoyance

Noise certification metrics such as Effective Perceived Noise Level and Sound Exposure Level are used to ensure that helicopters meet regulations, but these metrics may not be good indicators of annoyance since noise complaints against helicopters persist. Sound quality (SQ) metrics, specifically fluctuation strength, tonality, impulsiveness, roughness, and sharpness, are explored to determine their relationship with annoyance. A psychoacoustic test was conducted at the NASA Langley Research Center Exterior Effects Room to assess annoyance to helicopter-like sounds over a range of SQ metric values. The amplitude, phase, and frequency of the AS350 helicopter main and tail rotor blade passage signal harmonics were manipulated to produce 105 unique helicopter-like sounds with prescribed values of SQ metrics. All sounds were set to roughly the same loudness level. These sounds were played to 40 subjects who rated each sound for annoyance. Analyses given in this paper point to which SQ metrics are important to the helicopter noise annoyance response.

Krishnamurthy, Siddhartha↗

A Superposed Metric for Spinning Black Hole Binaries Approaching Merger

We construct an approximate metric that represents the spacetime of spinning binary black holes (BBH) approaching merger. We build the metric as an analytical superposition of two Kerr metrics in harmonic coordinates, where we transform each black hole term with time-dependent boosts describing an inspiral trajectory. The velocities and trajectories of the boost are obtained by solving the post-Newtonian (PN) equations of motion at 3.5 PN order. We analyze the spacetime scalars of the new metric and we show that it is an accurate approximation of Einstein’s field equations in vacuum for a BBH system in the inspiral regime. Furthermore, to prove the effectiveness of our approach, we test the metric in the context of a 3D general relativistic magnetohydrodynamical (GRMHD) simulation of accreting minidisks around the black holes. We compare our results with a previous well-tested spacetime construction based on the asymptotic matching method. We conclude that our new spacetime is well-suited for long-term GRMHD simulations of spinning binary black holes on their way to the merger.

Luciano Combi↗