SEARCH · Engineering Papers

Results for “Metrics”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 289 records · Page 16

All sources NASA NTRS DOE OSTI

DOE OSTI2016

SU-F-T-600: Influence of Acuros XB and AAA Dose Calculation Algorithms On Plan Quality Metrics and Normal Lung Doses in Lung SBRT

Purpose: To study the influence of superposition-beam model (AAA) and determinant-photon transport-solver (Acuros XB) dose calculation algorithms on the treatment plan quality metrics and on normal lung dose in Lung SBRT. Methods: Treatment plans of 10 Lung SBRT patients were randomly selected. Patients were prescribed to a total dose of 50-54Gy in 3–5 fractions (10?5 or 18?3). Doses were optimized accomplished with 6-MV using 2-arcs (VMAT). Doses were calculated using AAA algorithm with heterogeneity correction. For each plan, plan quality metrics in the categories- coverage, homogeneity, conformity and gradient were quantified. Repeat dosimetry for these AAA treatment plans was performed using AXB algorithm with heterogeneity correction for same beam and MU parameters. Plan quality metrics were again evaluated and compared with AAA plan metrics. For normal lung dose, V{sub 20} and V{sub 5} to (Total lung- GTV) were evaluated. Results: The results are summarized in Supplemental Table 1. PTV volume was mean 11.4 (±3.3) cm{sup 3}. Comparing RTOG 0813 protocol criteria for conformality, AXB plans yielded on average, similar PITV ratio (individual PITV ratio differences varied from −9 to +15%), reduced target coverage (−1.6%) and increased R50% (+2.6%). Comparing normal lung doses, the lung V{sub 20} (+3.1%) and V{sub 5} (+1.5%) were slightly higher for AXB plans compared to AAA plans. High-dose spillage ((V105%PD - PTV)/ PTV) was slightly lower for AXB plans but the % low dose spillage (D2cm) was similar between the two calculation algorithms. Conclusion: AAA algorithm overestimates lung target dose. Routinely adapting to AXB for dose calculations in Lung SBRT planning may improve dose calculation accuracy, as AXB based calculations have been shown to be closer to Monte Carlo based dose predictions in accuracy and with relatively faster computational time. For clinical practice, revisiting dose-fractionation in Lung SBRT to correct for dose overestimates attributable to algorithm may very well be warranted.

60 APPLIED LIFE SCIENCES↗

DOE OSTI2016

TU-D-209-04: How Useful Are Indirect Dose Metrics for Estimating Peak Skin Dose in Interventional Cardiology?

Purpose: The purpose of this study was to determine if a relationship between indirect dose metrics and PSD could be established for fluoroscopically-guided interventional cardiology procedures. Methods: PSD were measured directly using XR-RV3 radiochromic film for 94 consecutive fluoroscopically guided interventional cardiology procedures performed at two sites. Procedures were both diagnostic and therapeutic in nature. Radiation dose structured reports (RDSR) were collected for each procedure and used to calculate indirect estimates of PSD which were compared to the measured PSD. Reference air kerma (Ka,r) was also compared to the measured PSD. Pearson’s correlation coefficient was calculated for each metric and metrics were compared to measured PSD using a two-tailed t-test. Data were log transformed prior to statistical analysis. Results: Both Ka,r and the calculated PSD were closely correlated with measured PSD at each sites (Ka,r: 0.92 and 0.86, indirect PSD: 0.91 and 0.88). At one site, neither Ka,r nor indirect PSD was significantly different from the measured PSD (p = 0.22 and p=0.054, respectively), while at the second site both Ka,r and indirect PSD were significantly higher than measured PSD (p<0.0001 and p<0.0001, respectively). In almost all cases, both Ka,r and indirect PSD overestimated the true PSD. Conclusions: The use of a range of gantry angles and table positions, along with variation in procedural imaging requirements, limits the utility of indirect dose metrics for predicting PSD for interventional cardiology procedures. A. Kyle Jones and Alexander S. Pasciak are owners of Fluoroscopic Safety, LLC.

60 APPLIED LIFE SCIENCES↗

DOE OSTI2015

Automatic Classification of Protein Structure Using the Maximum Contact Map Overlap Metric

In this paper, we propose a new distance measure for comparing two protein structures based on their contact map representations. We show that our novel measure, which we refer to as the maximum contact map overlap (max-CMO) metric, satisfies all properties of a metric on the space of protein representations. Having a metric in that space allows one to avoid pairwise comparisons on the entire database and, thus, to significantly accelerate exploring the protein space compared to no-metric spaces. We show on a gold standard superfamily classification benchmark set of 6759 proteins that our exact k-nearest neighbor (k-NN) scheme classifies up to 224 out of 236 queries correctly and on a larger, extended version of the benchmark with 60; 850 additional structures, up to 1361 out of 1369 queries. Finally, our k-NN classification thus provides a promising approach for the automatic classification of protein structures based on flexible contact map overlap alignments.

59 BASIC BIOLOGICAL SCIENCES↗

DOE OSTI2017

A Metric-Based Validation Process to Assess the Realism of Synthetic Power Grids

Public power system test cases that are of high quality benefit the power systems research community with expanded resources for testing, demonstrating, and cross-validating new innovations. Building synthetic grid models for this purpose is a relatively new problem, for which a challenge is to show that created cases are sufficiently realistic. This paper puts forth a validation process based on a set of metrics observed from actual power system cases. These metrics follow the structure, proportions, and parameters of key power system elements, which can be used in assessing and validating the quality of synthetic power grids. Though wide diversity exists in the characteristics of power systems, the paper focuses on an initial set of common quantitative metrics to capture the distribution of typical values from real power systems. The process is applied to two new public test cases, which are shown to meet the criteria specified in the metrics of this paper.

24 POWER TRANSMISSION AND DISTRIBUTION↗

DOE OSTI2021

Extreme metrics from large ensembles: investigating the effects of ensemble size on their estimates

Abstract. We consider the problem of estimating the ensemble sizes required to characterize the forced component and the internal variability of a number of extreme metrics. While we exploit existing large ensembles, our perspective is that of a modeling center wanting to estimate a priori such sizes on the basis of an existing small ensemble (we assume the availability of only five members here). We therefore ask if such a small-size ensemble is sufficient to estimate accurately the population variance (i.e., the ensemble internal variability) and then apply a well-established formula that quantifies the expected error in the estimation of the population mean (i.e., the forced component) as a function of the sample size n, here taken to mean the ensemble size. We find that indeed we can anticipate errors in the estimation of the forced component for temperature and precipitation extremes as a function of n by plugging into the formula an estimate of the population variance derived on the basis of five members. For a range of spatial and temporal scales, forcing levels (we use simulations under Representative Concentration Pathway 8.5) and two models considered here as our proof of concept, it appears that an ensemble size of 20 or 25 members can provide estimates of the forced component for the extreme metrics considered that remain within small absolute and percentage errors. Additional members beyond 20 or 25 add only marginal precision to the estimate, and this remains true when statistical inference through extreme value analysis is used. We then ask about the ensemble size required to estimate the ensemble variance (a measure of internal variability) along the length of the simulation and – importantly – about the ensemble size required to detect significant changes in such variance along the simulation with increased external forcings. Using the F test, we find that estimates on the basis of only 5 or 10 ensemble members accurately represent the full ensemble variance even when the analysis is conducted at the grid-point scale. The detection of changes in the variance when comparing different times along the simulation, especially for the precipitation-based metrics, requires larger sizes but not larger than 15 or 20 members. While we recognize that there will always exist applications and metric definitions requiring larger statistical power and therefore ensemble sizes, our results suggest that for a wide range of analysis targets and scales an effective estimate of both forced component and internal variability can be achieved with sizes below 30 members. This invites consideration of the possibility of exploring additional sources of uncertainty, such as physics parameter settings, when designing ensemble simulations.

54 ENVIRONMENTAL SCIENCES↗

NASA NTRS2014

Don't Trust a Management Metric, Especially in Life Support

Goodhart's law states that metrics do not work. Metrics become distorted when used and they deflect effort away from more important goals. These well-known and unavoidable problems occurred when the closure and system mass metrics were used to manage life support research. The intent of life support research should be to develop flyable, operable, reliable systems, not merely to increase life support system closure or to reduce its total mass. It would be better to design life support systems to meet the anticipated mission requirements and user needs. Substituting the metrics of closure and total mass for these goals seems to have led life support research to solve the wrong problems.

life support↗

NASA NTRS2017

An Alternative Time Metric to Modified Tau for Unmanned Aircraft System Detect And Avoid

A new horizontal time metric, Time to Protected Zone, is proposed for use in the Detect and Avoid (DAA) Systems equipped by unmanned aircraft systems (UAS). This time metric has three advantages over the currently adopted time metric, modified tau: it corresponds to a physical event, it is linear with time, and it can be directly used to prioritize intruding aircraft. The protected zone defines an area around the UAS that can be a function of each intruding aircraft's surveillance measurement errors. Even with its advantages, the Time to Protected Zone depends explicitly on encounter geometry and may be more sensitive to surveillance sensor errors than modified tau. To quantify its sensitivity, simulation of 972 encounters using realistic sensor models and a proprietary fusion tracker is performed. Two sensitivity metrics, the probability of time reversal and the average absolute time error, are computed for both the Time to Protected Zone and modified tau. Results show that the sensitivity of the Time to Protected Zone is comparable to that of modified tau if the dimensions of the protected zone are adequately defined.

alert↗

NASA NTRS2017

An Alternative Time Metric to Modified Tau for Unmanned Aircraft System Detect And Avoid

UAS↗

NASA NTRS2017

Real Time Metrics and Analysis of Integrated Arrival, Departure, and Surface Operations

A real time dashboard was developed in order to inform and present users notifications and integrated information regarding airport surface operations. The dashboard is a supplement to capabilities and tools that incorporate arrival, departure, and surface air-traffic operations concepts in a NextGen environment. As trajectory-based departure scheduling and collaborative decision making tools are introduced in order to reduce delays and uncertainties in taxi and climb operations across the National Airspace System, users across a number of roles benefit from a real time system that enables common situational awareness. In addition to shared situational awareness the dashboard offers the ability to compute real time metrics and analysis to inform users about capacity, predictability, and efficiency of the system as a whole. This paper describes the architecture of the real time dashboard as well as an initial set of metrics computed on operational data. The potential impact of the real time dashboard is studied at the site identified for initial deployment and demonstration in 2017; Charlotte-Douglas International Airport. Analysis and metrics computed in real time illustrate the opportunity to provide common situational awareness and inform users of metrics across delay, throughput, taxi time, and airport capacity. In addition, common awareness of delays and the impact of takeoff and departure restrictions stemming from traffic flow management initiatives are explored. The potential of the real time tool to inform the predictability and efficiency of using a trajectory-based departure scheduling system is also discussed.

integrated arrival departure surface operations↗

NASA NTRS2016

A New Paradigm for Quality Metrics

SMA metrics have been traditionally represented as either leading and lagging indicators. Traditional ways of viewing these metrics and trending the data is via graphs which reflect one attribute in each graph. The KSC Independent Assessment Team developed a non-traditional method for expressing variation as a ratio (the deviation between the Expected and Actual). This method allows different measures to be plotted on the same graph for trending and comparison to provide a holistic view across a particular theme or discipline. Since all measurements are a ratio of variation and dimensionless, this variation-based metrics approach can be used a new analytical tool which is complimentary to existing value-added metrics tools.

Dimensionless↗

NASA NTRS2024

Literature Review of Cumulative Noise Metrics Relevant to X-59 Supersonic Overflights

Upcoming studies with the X-59 aircraft will collect dose-response data to understand community response to en-route supersonic noise. This data collection effort will inform regulators in their efforts to potentially update the ban on commercial supersonic flight with acceptable noise standards. Part of the dataset will consist of end-of-day responses to the noise from multiple supersonic overflights. This document provides rationale from relevant literature surrounding which cumulative noise metric to pair with end-of-day survey responses for upcoming community studies with the X-59 aircraft. Despite its deficiencies, day-night average sound level (DNL) summation of candidate single-event metrics remains the most likely choice for a cumulative noise metric for multiple supersonic overflights because 1) it is widely used for other transportation noise sources and 2) its relevance for infrequent events is somewhat plausible, unlike some recent alternative metrics.

sonic boom↗

DOE OSTI2012

Towards Efficient Supercomputing: Searching for the Right Efficiency Metric

The efficiency of supercomputing has traditionally been in the execution time. In early 2000 s, the concept of total cost of ownership was re-introduced, with the introduction of efficiency measure to include aspects such as energy and space. Yet the supercomputing community has never agreed upon a metric that can cover these aspects altogether and also provide a fair basis for comparison. This paper exam- ines the metrics that have been proposed in the past decade, and proposes a vector-valued metric for efficient supercom- puting. Using this metric, the paper presents a study of where the supercomputing industry has been and how it stands today with respect to efficient supercomputing.

99 GENERAL AND MISCELLANEOUS↗

DOE OSTI2014

Resilient Control Systems Practical Metrics Basis for Defining Mission Impact

"Resilience” describes how systems operate at an acceptable level of normalcy despite disturbances or threats. In this paper we first consider the cognitive, cyber-physical interdependencies inherent in critical infrastructure systems and how resilience differs from reliability to mitigate these risks. Terminology and metrics basis are provided to integrate the cognitive, cyber-physical aspects that should be considered when defining solutions for resilience. A practical approach is taken to roll this metrics basis up to system integrity and business case metrics that establish “proper operation” and “impact.” A notional chemical processing plant is the use case for demonstrating how the system integrity metrics can be applied to establish performance, and

42 ENGINEERING↗

DOE OSTI2011

Making Energy Savings Easier: Usability Metrics for Thermostats

U.S. residential thermostats control approximately 9% of the nation’s energy use. Many building codes now require programmable thermostats (PTs) because of their assumed energy savings. However, several recent field studies have shown no significant savings or even higher energy use in households using PTs compared to those using non-PTs. These studies point to usability problems that lead to incorrect use and wasted energy. However, the lack of clear, consistent metrics has hampered the acceptance of usability concerns by thermostat manufacturers. Thus there is a need for metrics specific to PTs that manufacturers can use to evaluate their products. Here, we report on the results of a usability study conducted on five commercially available PTs and the development of four new metrics suitable for use in evaluating thermostat usability. Our study confirmed usability deficits in the current generation of PTs and showed the metrics are correlated with each other as well as agreeing with the qualitative results of the study.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

DOE OSTI2022

Resilience Metrics for Building-Level Electrical Distribution Systems with Energy Storage: Preprint

The energy system infrastructure that delivers power to a building's loads needs to be resilient, such that it can withstand and recover from extreme outages (e.g., grid faults that leave millions of people without power during severe weather events). Building-level electrical distribution systems (BEDSs) distribute power from a building's energy sources - including the grid, solar photovoltaic (PV) panels, and batteries - to its loads, including lighting, HVAC, and plug loads. BEDS with storage can provide resilience by distributing local electricity supply to critical loads during an outage. Quantitative metrics are needed to assess the resilience improvements associated with new BEDS and storage system technologies. In this paper, we apply an existing metric, the probability of outage survival curve (POSC), to BEDS with storage and propose a set of novel metrics that improve upon POSC. Through a simulation-based case study, we demonstrate how these metrics are impacted by the BEDS design and how they can be used to design a resilient system.

buildings↗

DOE OSTI2022

A Unified Metric for Fast Frequency Response in Low-Inertia Power Systems: Preprint

Future power system with more inverter-based resources (IBRs) is vulnerable to the frequency-decline contingency. Fast frequency response (FFR) provided by IBRs is a good candidate to arrest the frequency excursion. Diverse types of FFRs have been integrated into the power system. Without a unified quantification of FFRs, it is hard for the grid operators to fully leverage the FFR capabilities of IBRs. This work introduces potential unified metrics of prevailing FFRs. We utilized the metric-to-frequency (M2F) mapping to validate the accuracy of the metrics. The results show the proposed metric to be simple yet accurate.

effective inertia↗

DOE OSTI2025

Understanding ERE and iVOC Metrics for Graded CdSeTe Absorbers

PL-based external radiative efficiency (ERE) and implied open-circuit voltage (iVOC) metrics were introduced for thin-film solar absorbers to better understand the voltage deficit and diagnose losses in solar cells. Traditionally, elevated ERE and iVOC measurements are associated with diminished recombination within the solar device, a rationale heavily reliant on the assumption of a uniform bandgap and high carrier mobilities in the absorber. Recently, very low mobilities in CdSeTe absorbers (< 1 cm2/(V.s)) were measured using the light-induced transient grading technique. In this study, we use a detailed numerical model of iVOC to investigate the possible reasons of elevated iVOC in realistic CdSeTe absorbers with a graded Se profile. In particular, we examine how the bandgap nonuniformity and the reduced hole mobility in graded CdSeTe absorbers affect iVOC measurements. We show that high iVOC may result from inflated quasi-Fermi level splitting in the high-Se region in the front part of a CdSeTe absorber with slow hole transport. We reproduce the experimentally reported 360 mV increase in iVOC-VOC gap with reduced doping using a model with sub-1 cm2/(V.s) hole mobility in the high-Se region. Based on our results, we conclude that the iVOC metric (or ERE metric) should not be used as a sole metric of CdSeTe absorber quality. We discuss possible ways to extract useful information from the iVOC-VOC gap by supplementing the front-side illumination measurements with back-side illumination measurements.

14 SOLAR ENERGY↗

DOE OSTI2025

Explainable artificial intelligence relates perovskite luminescence images to current-voltage metrics

As the demand for low-cost, high-efficiency solar energy technologies grows, metal halide perovskite (MHP) solar cells have emerged as a promising candidate for next-generation photovoltaics due to their high power conversion efficiencies. However, their poor durability and issues with manufacturing consistency remain significant barriers to commercialization. In this work, we develop deep learning models to support materials characterization and provide insight into features and processes influencing performance. The models are trained using transfer learning of a pretrained model to predict relevant current-voltage (IV) metrics based on different combinations of input electroluminescence (EL) and photoluminescence (PL) images of MHP devices. We examine which image types are most informative in accurately predicting different IV metrics. Additionally, we use explainable artificial intelligence (XAI) techniques to provide insights into specific spatial features in the devices that drive differences in performance. We find that stabilized luminescence images (e.g. those collected after biasing the devices for at least 1 min) are better for predicting metrics of open-circuit voltage (by PL) and short-circuit current (by PL with EL), but that predicting fill factor and overall power output may use the time-evolution of EL images. Based on attribution masks generated by integrated gradients for each device performance metric, we further suggest different loss mechanisms associated with categories of large and small spatial defects. Overall, this case study highlights the potential applicability of XAI methodology for streamlining MHP device analysis and accelerating detailed understanding of the relationships between spatial defects and impacts on performance.

14 SOLAR ENERGY↗