Engineering PapersSearch

SEARCH · Engineering Papers

Results for “metrics assessment”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2

NASA metrication activities

NASA's organization and policy for metrification, history from 1964, NASA participation in Federal agency activities, interaction with nongovernmental metrication organizations, and the proposed metrication assessment study are reviewed.

Vlannes, P. N.

Sonic boom acceptability studies

The determination of the magnitude of sonic boom exposure which would be acceptable to the general population requires, as a starting point, a method to assess and compare individual sonic booms. There is no consensus within the scientific and regulatory communities regarding an appropriate sonic boom assessment metric. Loudness, being a fundamental and well-understood attribute of human hearing was chosen as a means of comparing sonic booms of differing shapes and amplitudes. The figure illustrates the basic steps which yield a calculated value of loudness. Based upon the aircraft configuration and its operating conditions, the sonic boom pressure signature which reaches the ground is calculated. This pressure-time history is transformed to the frequency domain and converted into a one-third octave band spectrum. The essence of the loudness method is to account for the frequency response and integration characteristics of the auditory system. The result of the calculation procedure is a numerical description (perceived level, dB) which represents the loudness of the sonic boom waveform.

Shepherd, Kevin P.

SPICE Module for the Satellite Orbit Analysis Program (SOAP)

A SPICE module for the Satellite Orbit Analysis Program (SOAP) precisely represents complex motion and maneuvers in an interactive, 3D animated environment with support for user-defined quantitative outputs. (SPICE stands for Spacecraft, Planet, Instrument, Camera-matrix, and Events). This module enables the SOAP software to exploit NASA mission ephemeris represented in the JPL Ancillary Information Facility (NAIF) SPICE formats. Ephemeris types supported include position, velocity, and orientation for spacecraft and planetary bodies including the Sun, planets, natural satellites, comets, and asteroids. Entire missions can now be imported into SOAP for 3D visualization, playback, and analysis. The SOAP analysis and display features can now leverage detailed mission files to offer the analyst both a numerically correct and aesthetically pleasing combination of results that can be varied to study many hypothetical scenarios. The software provides a modeling and simulation environment that can encompass a broad variety of problems using orbital prediction. For example, ground coverage analysis, communications analysis, power and thermal analysis, and 3D visualization that provide the user with insight into complex geometric relations are included. The SOAP SPICE module allows distributed science and engineering teams to share common mission models of known pedigree, which greatly reduces duplication of effort and the potential for error. The use of the software spans all phases of the space system lifecycle, from the study of future concepts to operations and anomaly analysis. It allows SOAP software to correctly position and orient all of the principal bodies of the Solar System within a single simulation session along with multiple spacecraft trajectories and the orientation of mission payloads. In addition to the 3D visualization, the user can define numeric variables and x-y plots to quantitatively assess metrics of interest.

Coggi, John

Neuropsychological Testing of Astronauts

The Spaceflight Cognitive Assessment Tool for Windows (WinSCAT) is a computer program that administers a battery of five timed neuro-cognitive tests. WinSCAT was developed to give astronauts an objective and automated means of assessing their cognitive functioning during space flight, as compared with their own baseline performances measured during similar prior testing on the ground. WinSCAT is also intended for use by flight surgeons to assess cognitive impairment after exposure of astronauts to such cognitive assaults as head trauma, decompression sickness, and exposure to toxic gas. The tests were selected from among a group of tests, denoted the Automated Neuropsychological Assessment Metrics, that were created by the United States Navy and Army for use in evaluating the cognitive impairment of military personnel who have been subjected to medication or are suspected to have sustained brain injuries. These tests have been validated in a variety of clinical settings and are now in the public domain. The tests are presented in a Microsoft Windows shell that facilitates administration and enables immediate reporting of test scores in numerical and graphical forms.

Flynn, Christopher

Thermal Imaging for Assessment of Electron-Beam Free Form Fabrication (EBF(sup 3)) Additive Manufacturing Welds

Additive manufacturing is a rapidly growing field where 3-dimensional parts can be produced layer by layer. NASA s electron beam free-form fabrication (EBF(sup 3)) technology is being evaluated to manufacture metallic parts in a space environment. The benefits of EBF(sup 3) technology are weight savings to support space missions, rapid prototyping in a zero gravity environment, and improved vehicle readiness. The EBF(sup 3) system is composed of 3 main components: electron beam gun, multi-axis position system, and metallic wire feeder. The electron beam is used to melt the wire and the multi-axis positioning system is used to build the part layer by layer. To insure a quality weld, a near infrared (NIR) camera is used to image the melt pool and solidification areas. This paper describes the calibration and application of a NIR camera for temperature measurement. In addition, image processing techniques are presented for weld assessment metrics.

Zalameda, Joseph N.

Identification of Flights for Cost-Efficient Climate Impact Reduction

The aircraft-induced climate impact has drawn attention in recent years. Aviation operations affect the environment mainly through the release of carbon-dioxide, nitrogen-oxides, and by the formation of contrails. Recent research has shown that altering trajectories can reduce aviation environmental cost by reducing Absolute Global Temperature Change Potential, a climate assessment metric that adapts a linear system for modeling the global temperature response to aviation emissions and contrails. However, these methods will increase fuel consumption that leads to higher fuel costs imposed on airlines. The goal of this work is to identify ights for which the environmental cost of climate impact reduction outweighs the increase in operational cost on an individual aircraft basis. Environmental cost is quanti ed using the monetary social cost of carbon. The increase in operational cost is considering cost of additional fuel usage only. For this paper, an algorithm has been developed that modi es the trajectories of ights to evaluate the e ect of environ- mental cost and operational cost of ights in the United States National Airspace System. The algorithm identi es ights for which the environmental cost of climate impact can be reduced and modi es their trajectories to achieve maximum environmental net bene t, which is the di erence between reduction in environmental cost and additional operational cost. The result shows on a selected day, 16% of the ights among eight major airlines, or 2,043 ights, can achieve environmental net bene t using weather forecast data, resulting in net bene t of around $500,000. The result also suggests that the long-haul ights would be better candidates for cost-ecient climate impact reduction than the short haul ights. The algorithm will help to identify the characteristics of ights that are capable of applying cost-ecient climate impact reduction strategy.

climate impact reduction

Evaluating Tradeoffs Between Environmental Impact and Operational Costs for Enroute Air Traffic

The rapid growth of air traffic has drawn attention to aircraft-induced environmental impact. Aviation operations affect the environment mainly through the release of emissions and by the formation of contrails. Recent research has shown that altering aircraft cruise altitudes can reduce aviation environmental impact by reducing Absolute Global Temperature Change Potential, a climate assessment metric that adapts a linear system for modeling the global temperature response to aviation emissions and contrails. However, these methods will increase fuel consumption that leads to higher operational costs imposed on airlines resulting in reluctance to adopt a new routing strategy. This paper evaluates the tradeoff between environmental impact reduction and the corresponding added operational costs for enroute air traffic. The concept of social cost of carbon and the carbon auction price from California's recent cap-and-trade system were used to provide estimates and a methodology to evaluate environmental costs for carbon dioxide emissions and contrail formations. Depending on the specific environmental policy, the strategy is considered favorable when the reduction in environmental costs exceeds the increase in operational costs. The results show how the net environmental bene t varies with different decision-making time horizons, different carbon and fuel costs, and different days. The study provides guidance towards the development of the environmental reduction strategies.

environmental impact

Limitations to Biomass Estimation for FAV Using NDVI Analysis of Satellite Acquisitions

The distribution and standing biomass (mass per unit area) of Floating Aquatic Vegetation (FAV)provide assessment metrics for productivity of vegetative communities. Remote sensing acquisition data has been used to estimate standing biomass in various ecosystems using vegetation indices, particularly the normalized difference vegetation index (NDVI), comparing the relative strength of radiation absorbance and reflectance of vegetation in specific wavebands. Obstacles to estimating vegetation biomass using NDVI reported are related to diversity of the vegetation community and high-density saturation. We evaluate the potential for using NDVI derived estimates for FAV biomass in the California Delta by focusing on patches of predominantly single species and compare biomass estimates for specific patches with measurement of total biomass and partitioning within the vegetation canopy. Using Landsat satellite acquisitions, biomass estimates are made for individual pixels and vegetation samples were collected from those same pixels. Focus on patches of predominantly single FAV species resulted in successful estimation of standing biomass beyond that previously reported in other ecosystems. A saturation response for the estimations was observed, however, with a much higher threshold high than previously reported for diverse population aquatic ecosystems. We discuss the likely cause for the saturation phenomenon and potential improvement utilizing altered spectral indices possible with higher spectral resolution from satellites carrying higher waveband multispectral and hyperspectral instruments.

Biomass

Exploration Medical Capability IMPACT Medical Database – Medical Item Database (MedID) Content Development Methods

The Informing Mission Planning via Analysis of Complex Tradespaces (IMPACT) project encompasses a suite of computational tools that are used to inform systematic trade study evaluations and research prioritizations regarding the optimization of spaceflight medical systems, including the provision of risk assessment metrics, for a given design reference mission (DRM). The IMPACT Medical Database (IMPACT-MD) is the component that virtually houses the clinical and engineering data for medical conditions, capabilities, and resources used to support these analyses. In addition, IMPACT-MD also provides data to the associated SysML model to support the Level 2, Level 3, and Level 4 requirements development needed for IMPACT. IMPACT-MD contains an internal Evidence Library (EL) sub-component that hosts the clinical evidence, treatment and outcome metric data, and incidence data for each medical condition identified for exploration-class missions. Furthermore, a separate Medical Item Database (MedID) sub-component hosts the engineering data and physical attributes (e.g., mass, volume, power, etc.) associated with each medical resource item identified to address the requisite medical conditions. The IMPACT project is conducted under the Exploration Medical Capability (ExMC) element of the Human Research Program (HRP) within NASA's Space Operations Mission Directorate (SOMD). The reader is referred to the documents in Appendix A for a more complete overview of the IMPACT tool suite and functionality as well as further detail on any associated aspects of the project. The primary purpose of this document is to describe the methods utilized in the collection and verification of engineering data content during the project development phase for the MedID sub-component of the IMPACT-MD project. Engineering content data (both existing and new/future) will be stored in IMPACT-MD for use by the IMPACT project. This document will detail the content development and data credibility for the engineering data housed within the MedID subcomponent of IMPACT-MD, which is used to inform the IMPACT 1.0 project deliverables.

Exploration Medical Capability

NASA EOSDIS Data Usage Metrics- Insight and Assessment

NASA's Earth Science Data and Information System (ESDIS) Project collects Earth science data usage metrics on a daily basis through the ESDIS Metrics System (EMS). This includes metrics on distribution of data products, users, data volumes, and number of files, which are key parameters in evaluating system-level performance of any of the Distributed Active Archive Centers (DAACs) encompassed by NASA's Earth Observing System (EOS) Data and Information System (EOSDIS). EOSDIS data usage metrics illustrate the benefits of making NASA data openly available to the public and show a rapid growth in data distribution to a worldwide user community. In fact, each year since 2014 the EOSDIS has distributed over one billion data files of products from EOS satellite, airborne, and in situ observations. An assessment of the long-term trends of data usage metrics and user characterization provides insights into data usability.This study will focus on describing the EMS as a metrics collection tool and will provide a comprehensive analysis of EOSDIS data usage metrics over the last 10 years. This study will also characterize the product distribution metrics by various tools and services, such as Giovanni, the Open-source Project for a Network Data Access Protocol (OPeNDAP), and subsets, to address how these tools/services have extended the usage of data in the EOSDIS collection. Data usage patterns based on discipline and study area will further assist in understanding how EOSDIS data user needs have evolved over time. Results from this study will provide useful information for the DAACs that can help them improve the functionality of their tools and services as well as more efficiently allocate the resources necessary for enhanced access and availability of their data products. Knowledge of these metrics may also benefit user discovery of data in the EOSDIS collection, promote research collaboration, and stimulate new ideas from work and research conducted using specific datasets and data collections.

Kafle, Durga N.

A Software Quality Model and Metrics for Risk Assessment

A software quality model and its associated attributes are defined and used as the model for the basis for a discussion on risk. Specific quality goals and attributes are selected based on their importance to a software development project and their ability to be quantified. Risks that can be determined by the model's metrics are identified. A core set of metrics relating to the software development process and its products is defined. Measurements for each metric and their usability and applicability are discussed.

L Hyatt

Setting the Bar for the Replacement of the Probability of Collision Metric

To date, satellite conjunction assessment (CA) risk analysis has largely embraced the probability of collision (Pc) as the omnibus metric to evaluate collision likelihood, and its use in such assessments has mostly been straightforward: at the point at which a conjunction mitigation decision is required, the calculated Pc is compared to a threshold; and if the calculated Pc exceeds that threshold, then a mitigation action is warranted. With only minor variation, this approach is employed by major CA risk assessment centers (e.g., NASA, EUSST, CNES, JAXA) and is advanced as the preferred method in the published CA best practices handbooks. Despite this near unanimity of operational practice, there is a major strain of secondary literature critical of the Pc and willing to propose alternatives. Alfano (2005) pointed out the ability of the Pc to underrepresent the risk in certain situations and counselled a maximum Pc construct. Carpenter (2017, 2019) reiterated this criticism and proposed using instead a confidence interval on the miss distance. Balch et al. (2019) identified what they argued was a defect in the entire Bayesian Pc construct and believed that the use of a more conservative methodology based on covariance ellipsoid overlap was necessary. Delande (2022) introduced the framework of collision “plausibility” to the risk assessment process and sketched out how this might be used operationally. Elkantassi (2022) published a full development of the miss distance confidence interval approach and applied it to several worked examples. While these different approaches to collision risk assessment do differ in their details, they all converge on two central points: first, the Pc’s failure to give an adequate expression of the risk in dilution region situations is a fatal flaw; and second, a conjunction should be presumed risky and in need of mitigation until the evidence of the situation can establish otherwise. These criticisms, if correct, would counsel a number of modifications to current CA operational practice; as such, they force a re-examination of fundamental aspects of the CA problem, including the following: 1. Is the CA risk assessment a probability problem, a statistics problem, or something else? 2. If it is a statistics problem, does it lend itself naturally to a hypothesis test construction? 3. If it can be construed as a hypothesis test, what form should the null hypothesis take, to wit: what constraints exist on the choice of the null hypothesis, what selections are in best alignment with all of the attendant parameters of the problem, and what is implied philosophically by different choices? 4. What are the implications of using the different proposed risk assessment parameters for CA? This question should be answered both in determining how frequently the dilution region situation cited by the critics of the Pc actually appears in an operationally significant manner and the missed detection and false alarm rates of all of the proposed risk assessment metrics, compared both to the Pc and to each other. This paper explores and offers preliminary answers to the above questions, presenting a researched treatment of the philosophical nature of the CA problem and the null hypothesis choice that achieves the greatest consistency with all of the different aspects of operational CA conduct. It then profiles all of the different proposed risk assessment metrics enumerated in the earlier paragraph against an extremely large database of conjunction events at both the 550km and 700km altitudes. The combination of the philosophical exploration of the CA problem and the results of the profiling activity articulates what a risk assessment metric will need to demonstrate, in terms of both innate construction and performance, in order to be a true competitor to the Pc.

conjunction assessment

Total Probability of Collision as a Metric for Finite Conjunction Assessment and Collision Risk Management

On-orbit collision risk is becoming an increasing mission risk to all operational satellites in Earth orbit. Managing this risk can be disruptive to mission and operations, present challenges for decision-makers, and is time-consuming for all parties involved. With the planned capability improvements to detecting and tracking smaller orbital debris and capacity improvements to routinely predict on-orbit conjunctions, this mission risk will continue to grow in terms of likelihood and effort. It is very real possibility that the future space environment will not allow collision risk management and mission operations to be conducted in the same manner as it is today. This paper presents the concept of a finite conjunction assessment-one where each discrete conjunction is not treated separately but, rather, as a continuous event that must be managed concurrently. The paper also introduces the Total Probability of Collision as an analogous metric for finite conjunction assessment operations and provides several options for its usage in a Concept of Operations.

Conjunction Assessment

Real-Time Assessment of Robot Performance During Remote Exploration Operations

To ensure that robots are used effectively for exploration missions, it is important to assess their performance during operations. We are investigating the definition and computation of performance metrics for assessing remote robotic operations in real-time. Our approach is to monitor data streams from robots, compute performance metrics, and provide Web-based displays of these metrics for assessing robot performance during operations. We evaluated our approach for measuring robot performance with the K10 rovers from NASA Ames Research Center during a field test at Moses Lake Sand Dunes (WA) in June 2008. In this paper we present the results of evaluating our software for robot performance and discuss our conclusions from this evaluation for future robot operations.

SBIR TOPIC X7.02 PHASE 1

Evaluation of Two Crew Module Boilerplate Tests Using Newly Developed Calibration Metrics

The paper discusses a application of multi-dimensional calibration metrics to evaluate pressure data from water drop tests of the Max Launch Abort System (MLAS) crew module boilerplate. Specifically, three metrics are discussed: 1) a metric to assess the probability of enveloping the measured data with the model, 2) a multi-dimensional orthogonality metric to assess model adequacy between test and analysis, and 3) a prediction error metric to conduct sensor placement to minimize pressure prediction errors. Data from similar (nearly repeated) capsule drop tests shows significant variability in the measured pressure responses. When compared to expected variability using model predictions, it is demonstrated that the measured variability cannot be explained by the model under the current uncertainty assumptions.

Horta, Lucas G.

Resilience Metrics Framework for Solar Photovoltaics

This presentation was given at the Photovoltaic Specialist Conference (PVSC) 54 in New Orleans, Louisiana. Photovoltaic (PV) systems are routinely exposed to extreme weather, including wind and hail storms. Historically, most systems have proven to be resilient to such events, but some storms have damaged PV systems, leading to physical and financial loss. Storm hardening measures and specific system attributes can reduce this risk. This work introduces a set of resilience metrics and a framework for quantifying, comparing, and predicting PV system resilience. The framework is divided into two parts: 1) predictive, attribute metrics based on site and component characteristics, and 2) impact metrics that assess post-storm performance. Metrics are weighted and aggregated, producing hazard-specific resilience scores. We derive damage functions from storm-impacted PV systems, establishing a baseline against which post-storm performance can be compared. This damage was widely variable across hail and wind intensities, and field hail damage was less than predicted by laboratory tests, suggesting that system features - in addition to storm conditions - influence damage likelihood. Finally, the metrics framework is demonstrated using three case studies of storm damaged PV systems. Although additional data are needed to create attribute specific damage functions and establish metric weights, this study presents a methodology for evaluating PV resilience and contributes new damage functions to the literature.

14 SOLAR ENERGY

Justice Underpinning Science and Technology Research (JUST-R) Offline Tool: A Tool for Guiding Researchers in Addressing Energy Justice Considerations in Early-Stage Research

The Justice Underpinning Science and Technology Research (JUST-R) Metrics Framework is a set of metrics for assessing the energy justice implications of technologies that are currently under research and development (R&D). The JUST-R Metrics Framework guides researchers through an analysis of the many facets of their research processes, from material inputs to knowledge sources, that may contribute to energy injustice both during the research period and when the technology is scaled. This JUST-R Offline Tool consists of an Excel file and PDF guide to aid researchers, engineers, and project managers in their application of the JUST-R Metrics Framework. This tool enables researchers to 1) evaluate the baseline energy justice implications of their research; 2) develop justice-oriented changes to the research process; and 3) track the implementation of proposed changes. The metrics included in the framework are sorted into five aspects of research: Team Dynamics, Sources & Inputs, Processes & Protocols, Waste & Hazards, and Results & Dissemination. The JUST-R Tool can be found here: https://www.nrel.gov/analysis/just-r.html.

29 ENERGY PLANNING, POLICY, AND ECONOMY

A Validation of Object-Oriented Design Metrics as Quality Indicators

This paper presents the results of a study in which we empirically investigated the suits of object-oriented (00) design metrics introduced in another work. More specifically, our goal is to assess these metrics as predictors of fault-prone classes and, therefore, determine whether they can be used as early quality indicators. This study is complementary to the work described where the same suite of metrics had been used to assess frequencies of maintenance changes to classes. To perform our validation accurately, we collected data on the development of eight medium-sized information management systems based on identical requirements. All eight projects were developed using a sequential life cycle model, a well-known 00 analysis/design method and the C++ programming language. Based on empirical and quantitative analysis, the advantages and drawbacks of these 00 metrics are discussed. Several of Chidamber and Kamerer's 00 metrics appear to be useful to predict class fault-proneness during the early phases of the life-cycle. Also, on our data set, they are better predictors than 'traditional' code metrics, which can only be collected at a later phase of the software development processes.

Basili, Victor R.