SEARCH · Engineering Papers

Results for “METRICS”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 181 records · Page 10

All sources NASA NTRS DOE OSTI

DOE OSTI2022

Resilience Metrics for Building-Level Electrical Distribution Systems with Energy Storage

The energy system infrastructure that delivers power to a building's loads needs to be resilient to withstand and recover from extreme outages (e.g., grid faults that leave millions of people without power during severe weather events). Building-level electrical distribution systems (BEDSs) distribute power from a building's energy sources - including the grid, solar photovoltaic (PV) panels, and batteries - to its loads, including lighting, HVAC, and plug loads. BEDS with storage can provide resilience by distributing local electricity supply to critical loads during an outage. Quantitative metrics are needed to assess the resilience improvements associated with new BEDS and storage system technologies. In this paper, we apply an existing metric, the probability of outage survival curve (POSC), to BEDS with storage and propose novel metrics that improve upon POSC. Through a simulation-based case study, we demonstrate how these metrics are impacted by the BEDS design and how they can be used to design a resilient system.

buildings↗

DOE OSTI2021

Extreme metrics from large ensembles: investigating the effects of ensemble size on their estimates

Abstract. We consider the problem of estimating the ensemble sizes required to characterize the forced component and the internal variability of a number of extreme metrics. While we exploit existing large ensembles, our perspective is that of a modeling center wanting to estimate a priori such sizes on the basis of an existing small ensemble (we assume the availability of only five members here). We therefore ask if such a small-size ensemble is sufficient to estimate accurately the population variance (i.e., the ensemble internal variability) and then apply a well-established formula that quantifies the expected error in the estimation of the population mean (i.e., the forced component) as a function of the sample size n, here taken to mean the ensemble size. We find that indeed we can anticipate errors in the estimation of the forced component for temperature and precipitation extremes as a function of n by plugging into the formula an estimate of the population variance derived on the basis of five members. For a range of spatial and temporal scales, forcing levels (we use simulations under Representative Concentration Pathway 8.5) and two models considered here as our proof of concept, it appears that an ensemble size of 20 or 25 members can provide estimates of the forced component for the extreme metrics considered that remain within small absolute and percentage errors. Additional members beyond 20 or 25 add only marginal precision to the estimate, and this remains true when statistical inference through extreme value analysis is used. We then ask about the ensemble size required to estimate the ensemble variance (a measure of internal variability) along the length of the simulation and – importantly – about the ensemble size required to detect significant changes in such variance along the simulation with increased external forcings. Using the F test, we find that estimates on the basis of only 5 or 10 ensemble members accurately represent the full ensemble variance even when the analysis is conducted at the grid-point scale. The detection of changes in the variance when comparing different times along the simulation, especially for the precipitation-based metrics, requires larger sizes but not larger than 15 or 20 members. While we recognize that there will always exist applications and metric definitions requiring larger statistical power and therefore ensemble sizes, our results suggest that for a wide range of analysis targets and scales an effective estimate of both forced component and internal variability can be achieved with sizes below 30 members. This invites consideration of the possibility of exploring additional sources of uncertainty, such as physics parameter settings, when designing ensemble simulations.

54 ENVIRONMENTAL SCIENCES↗

NASA NTRS2014

Don't Trust a Management Metric, Especially in Life Support

Goodhart's law states that metrics do not work. Metrics become distorted when used and they deflect effort away from more important goals. These well-known and unavoidable problems occurred when the closure and system mass metrics were used to manage life support research. The intent of life support research should be to develop flyable, operable, reliable systems, not merely to increase life support system closure or to reduce its total mass. It would be better to design life support systems to meet the anticipated mission requirements and user needs. Substituting the metrics of closure and total mass for these goals seems to have led life support research to solve the wrong problems.

life support↗

NASA NTRS2017

An Alternative Time Metric to Modified Tau for Unmanned Aircraft System Detect And Avoid

A new horizontal time metric, Time to Protected Zone, is proposed for use in the Detect and Avoid (DAA) Systems equipped by unmanned aircraft systems (UAS). This time metric has three advantages over the currently adopted time metric, modified tau: it corresponds to a physical event, it is linear with time, and it can be directly used to prioritize intruding aircraft. The protected zone defines an area around the UAS that can be a function of each intruding aircraft's surveillance measurement errors. Even with its advantages, the Time to Protected Zone depends explicitly on encounter geometry and may be more sensitive to surveillance sensor errors than modified tau. To quantify its sensitivity, simulation of 972 encounters using realistic sensor models and a proprietary fusion tracker is performed. Two sensitivity metrics, the probability of time reversal and the average absolute time error, are computed for both the Time to Protected Zone and modified tau. Results show that the sensitivity of the Time to Protected Zone is comparable to that of modified tau if the dimensions of the protected zone are adequately defined.

alert↗

NASA NTRS2017

An Alternative Time Metric to Modified Tau for Unmanned Aircraft System Detect And Avoid

UAS↗

NASA NTRS2017

Real Time Metrics and Analysis of Integrated Arrival, Departure, and Surface Operations

A real time dashboard was developed in order to inform and present users notifications and integrated information regarding airport surface operations. The dashboard is a supplement to capabilities and tools that incorporate arrival, departure, and surface air-traffic operations concepts in a NextGen environment. As trajectory-based departure scheduling and collaborative decision making tools are introduced in order to reduce delays and uncertainties in taxi and climb operations across the National Airspace System, users across a number of roles benefit from a real time system that enables common situational awareness. In addition to shared situational awareness the dashboard offers the ability to compute real time metrics and analysis to inform users about capacity, predictability, and efficiency of the system as a whole. This paper describes the architecture of the real time dashboard as well as an initial set of metrics computed on operational data. The potential impact of the real time dashboard is studied at the site identified for initial deployment and demonstration in 2017; Charlotte-Douglas International Airport. Analysis and metrics computed in real time illustrate the opportunity to provide common situational awareness and inform users of metrics across delay, throughput, taxi time, and airport capacity. In addition, common awareness of delays and the impact of takeoff and departure restrictions stemming from traffic flow management initiatives are explored. The potential of the real time tool to inform the predictability and efficiency of using a trajectory-based departure scheduling system is also discussed.

integrated arrival departure surface operations↗

NASA NTRS2016

A New Paradigm for Quality Metrics

SMA metrics have been traditionally represented as either leading and lagging indicators. Traditional ways of viewing these metrics and trending the data is via graphs which reflect one attribute in each graph. The KSC Independent Assessment Team developed a non-traditional method for expressing variation as a ratio (the deviation between the Expected and Actual). This method allows different measures to be plotted on the same graph for trending and comparison to provide a holistic view across a particular theme or discipline. Since all measurements are a ratio of variation and dimensionless, this variation-based metrics approach can be used a new analytical tool which is complimentary to existing value-added metrics tools.

Dimensionless↗

NASA NTRS2024

Literature Review of Cumulative Noise Metrics Relevant to X-59 Supersonic Overflights

Upcoming studies with the X-59 aircraft will collect dose-response data to understand community response to en-route supersonic noise. This data collection effort will inform regulators in their efforts to potentially update the ban on commercial supersonic flight with acceptable noise standards. Part of the dataset will consist of end-of-day responses to the noise from multiple supersonic overflights. This document provides rationale from relevant literature surrounding which cumulative noise metric to pair with end-of-day survey responses for upcoming community studies with the X-59 aircraft. Despite its deficiencies, day-night average sound level (DNL) summation of candidate single-event metrics remains the most likely choice for a cumulative noise metric for multiple supersonic overflights because 1) it is widely used for other transportation noise sources and 2) its relevance for infrequent events is somewhat plausible, unlike some recent alternative metrics.

sonic boom↗

DOE OSTI2022

Resilience Metrics for Building-Level Electrical Distribution Systems with Energy Storage: Preprint

The energy system infrastructure that delivers power to a building's loads needs to be resilient, such that it can withstand and recover from extreme outages (e.g., grid faults that leave millions of people without power during severe weather events). Building-level electrical distribution systems (BEDSs) distribute power from a building's energy sources - including the grid, solar photovoltaic (PV) panels, and batteries - to its loads, including lighting, HVAC, and plug loads. BEDS with storage can provide resilience by distributing local electricity supply to critical loads during an outage. Quantitative metrics are needed to assess the resilience improvements associated with new BEDS and storage system technologies. In this paper, we apply an existing metric, the probability of outage survival curve (POSC), to BEDS with storage and propose a set of novel metrics that improve upon POSC. Through a simulation-based case study, we demonstrate how these metrics are impacted by the BEDS design and how they can be used to design a resilient system.

buildings↗

DOE OSTI2022

A Unified Metric for Fast Frequency Response in Low-Inertia Power Systems: Preprint

Future power system with more inverter-based resources (IBRs) is vulnerable to the frequency-decline contingency. Fast frequency response (FFR) provided by IBRs is a good candidate to arrest the frequency excursion. Diverse types of FFRs have been integrated into the power system. Without a unified quantification of FFRs, it is hard for the grid operators to fully leverage the FFR capabilities of IBRs. This work introduces potential unified metrics of prevailing FFRs. We utilized the metric-to-frequency (M2F) mapping to validate the accuracy of the metrics. The results show the proposed metric to be simple yet accurate.

effective inertia↗

DOE OSTI2025

Understanding ERE and iVOC Metrics for Graded CdSeTe Absorbers

PL-based external radiative efficiency (ERE) and implied open-circuit voltage (iVOC) metrics were introduced for thin-film solar absorbers to better understand the voltage deficit and diagnose losses in solar cells. Traditionally, elevated ERE and iVOC measurements are associated with diminished recombination within the solar device, a rationale heavily reliant on the assumption of a uniform bandgap and high carrier mobilities in the absorber. Recently, very low mobilities in CdSeTe absorbers (< 1 cm2/(V.s)) were measured using the light-induced transient grading technique. In this study, we use a detailed numerical model of iVOC to investigate the possible reasons of elevated iVOC in realistic CdSeTe absorbers with a graded Se profile. In particular, we examine how the bandgap nonuniformity and the reduced hole mobility in graded CdSeTe absorbers affect iVOC measurements. We show that high iVOC may result from inflated quasi-Fermi level splitting in the high-Se region in the front part of a CdSeTe absorber with slow hole transport. We reproduce the experimentally reported 360 mV increase in iVOC-VOC gap with reduced doping using a model with sub-1 cm2/(V.s) hole mobility in the high-Se region. Based on our results, we conclude that the iVOC metric (or ERE metric) should not be used as a sole metric of CdSeTe absorber quality. We discuss possible ways to extract useful information from the iVOC-VOC gap by supplementing the front-side illumination measurements with back-side illumination measurements.

14 SOLAR ENERGY↗

DOE OSTI2025

Explainable artificial intelligence relates perovskite luminescence images to current-voltage metrics

As the demand for low-cost, high-efficiency solar energy technologies grows, metal halide perovskite (MHP) solar cells have emerged as a promising candidate for next-generation photovoltaics due to their high power conversion efficiencies. However, their poor durability and issues with manufacturing consistency remain significant barriers to commercialization. In this work, we develop deep learning models to support materials characterization and provide insight into features and processes influencing performance. The models are trained using transfer learning of a pretrained model to predict relevant current-voltage (IV) metrics based on different combinations of input electroluminescence (EL) and photoluminescence (PL) images of MHP devices. We examine which image types are most informative in accurately predicting different IV metrics. Additionally, we use explainable artificial intelligence (XAI) techniques to provide insights into specific spatial features in the devices that drive differences in performance. We find that stabilized luminescence images (e.g. those collected after biasing the devices for at least 1 min) are better for predicting metrics of open-circuit voltage (by PL) and short-circuit current (by PL with EL), but that predicting fill factor and overall power output may use the time-evolution of EL images. Based on attribution masks generated by integrated gradients for each device performance metric, we further suggest different loss mechanisms associated with categories of large and small spatial defects. Overall, this case study highlights the potential applicability of XAI methodology for streamlining MHP device analysis and accelerating detailed understanding of the relationships between spatial defects and impacts on performance.

14 SOLAR ENERGY↗

DOE OSTI2021

Deep-Learning-Derived Evaluation Metrics Enable Effective Benchmarking of Computational Tools for Phosphopeptide Identification

Tandem mass spectrometry (MS/MS)-based phosphoproteomics is a powerful technology for global phosphorylation analysis. However, applying four computational pipelines to a typical mass spectrometry (MS)-based phosphoproteomic dataset from a human cancer study, we observed a large discrepancy among the reported phosphopeptide identification and phosphosite localization results, underscoring a critical need for benchmarking. While efforts have been made to compare performance of computational pipelines using data from synthetic phosphopeptides, evaluations involving real application data have been largely limited to comparing the numbers of phosphopeptide identifications due to the lack of appropriate evaluation metrics. We investigated three deep learning-derived features as potential evaluation metrics: phosphosite probability, Delta RT and spectral similarity. Predicted phosphosite probability is computed by MusiteDeep, which provides high accuracy as previously reported; Delta RT is defined as the absolute retention time (RT) difference between RTs observed and predicted by AutoRT; and spectral similarity is defined as the Pearson’s correlation coefficient between spectra observed and predicted by pDeep2. Using a synthetic peptide dataset, we found that both Delta RT and spectral similarity provided excellent discrimination between correct and incorrect peptide-spectrum matches (PSMs) both when incorrect PSMs involved wrong peptide sequences and even when incorrect PSMs were caused by only incorrect phosphosite localization. Based on these results, we used all the three deep learning-derived features as evaluation metrics to compare different computational pipelines on diverse set of phosphoproteomic datasets and showed their utility in benchmarking performance of the pipelines. The benchmark metrics demonstrated in this study will enable users to select computational pipelines and parameters for routine analysis of phosphoproteomics data and will offer guidance for developers to improve computational methods.

59 BASIC BIOLOGICAL SCIENCES↗

DOE OSTI2023

Earth Mover’s Distance as a Metric to Evaluate the Extent of Charge Transfer in Excitations Using Discretized Real-Space Densities

This paper presents a novel theoretical measure, μ EMD , based on the earth mover's distance (EMD), for quantifying the density shift caused by electronic excitations in molecules. As input, the EMD metric uses only the discretized ground- and excited-state electron densities in real space, rendering it compatible with almost all electronic structure methods used to calculate excited states. The EMD metric is compared against other popular theoretical metrics for describing the extent of electron-hole separation in a wide range of excited states (valence, Rydberg, charge transfer, etc.). Further, the results showcase the EMD metric's effectiveness across all excitation types and suggest that it is useful as an additional tool to characterize electronic excitations. The study also reveals that μ EMD can function as a promising diagnostic tool for predicting the failure of pure exchange-correlation functionals. Specifically, we show statistical relationships among the functional-driven errors, the exact exchange content within the functional, and the magnitude of μ EMD values.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

DOE OSTI2024

Assessing Radiative Feedbacks and Their Contribution to the Arctic Amplification Measured by Various Metrics

Arctic amplification (AA), characterized by a more rapid surface air temperature (SAT) warming in the Arctic than the global average, is a major feature of global climate warming. Various metrics have been used to quantify AA based on SAT anomalies, trends, or variability, and they can yield quite different conclusions regarding the magnitude and temporal patterns of AA. This study examines and compares various AA metrics for their temporal consistency in the region north of 70°N from the early twentieth to the early 21st century using observational data and reanalysis products. We also quantify contributions of different radiative feedback mechanisms to AA based on short-term climate variability in reanalysis and model data using the Kernel-Gregory approach. Albedo and lapse rate feedbacks are positive and comparable, with albedo feedback being the leading contributor for all AA metrics. The net cloud feedback, which has large uncertainties, depends strongly on the data sets and AA metrics used. By quantifying the influence of internal variability on AA and related feedbacks based on global climate model ensemble simulations, we find that water vapor and cloud feedbacks are most heavily affected by internal variability.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2021

The efficacy of Lewis affinity scale metrics to represent solvent interactions with reagent salts in all-inorganic metal halide perovskite solutions

Solvents employed in the solution processing of metal halide perovskites are known to play a key role in defining the morphology and properties of the resulting thin film, and thus the performance of perovskite solar cell devices. Accurate metrics are needed that are capable of differentiating among candidates, finding solvents that adequately solubilize the various precursor species in solution and facilitate the nucleation and growth of these materials. Existing metrics such as the unsaturated Mayer bond order (UMBO) and the Gutmann donor number (DN) have been tested for lead iodide perovskite systems; but there has yet to be a comprehensive study on their transferability to lead-free perovskite solutions. Here, we use ab initio methods (density functional theory) and regression analysis tools to study the usefulness of DN and BF 3 affinity scales in this regard. We compared the relative effectiveness of these scales to describe interactions between solvents and BXn perovskite salts of lead (Pb 2+ ), tin (Sn 2+ and Sn 4+ ), germanium (Ge 2+ ), bismuth (Bi 3+ ), and antimony (Sb 3+ and Sb 5+ ). The DN proved to be a better representation than the BF 3 of such interactions, reflecting the closer similarity of these species to the “parent” SbCl 5 Lewis acid than to BF 3 . In addition, we have uncovered the usefulness of the lithium cation affinity metric (LCA) to describe the strength of interactions between solvents and A-site cations (e.g. Na + , K + , Rb + and Cs + ) in all-inorganic metal halide perovskite solutions. We find that the coordination strengths of solvents towards species in all-inorganic metal halide perovskite solutions are best described by two different metrics with distinct modes of action: DN differentiates among BX n salt complexes, and LCA among A-site cation species. This revelation can help guide the choice of solvent to optimize processing conditions. It also emphasizes the importance of selecting solvents whose DN and LCA optimize coordination to key Lewis acid species in all-inorganic perovskite solutions.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

DOE OSTI2023

Operational resilience metrics for power systems with penetration of renewable resources

Abstract Modern power grid is evolving towards carbon neutrality by deploying increasing amount of renewable energy resources. However, the impact of renewable generation on power system planning and operation is not sufficiently investigated, especially the capability of renewable penetrated power systems to resist and recover from major disturbances, which is a critical concern for system operators. Novel metrics and evaluation methodologies are needed to depict systems’ ability in response to events caused by natural disasters, and quantitatively evaluate system performance in various time scales. In this paper, operational resilience metrics are proposed for power systems with penetration of renewable energy resources based on transient stability principles. A systematic methodology is proposed to quantitatively assess the evolution of system performance during various stages of the disaster process. Based on the proposed metrics, a resilience‐oriented disaster management strategy is designed and validated using the modified IEEE 39‐bus test system. The simulation results demonstrate the validity of the proposed metrics and strategy, and show that the system resilience is enhanced during the mitigation of fault conditions.

Gui, Jianzhong↗

DOE OSTI2025

Metrics and extrapolation of resonant magnetic perturbation thresholds for ELM suppression

This large database study of resonant magnetic perturbation (RMP) edge localized mode (ELM) suppression thresholds in the AUG, DIII-D, EAST, and KSTAR tokamaks details the key strengths and weaknesses of RMP metrics. The RMP ELM suppression database used for this work contains plasma information at the time of transition from ELMing to ELM suppressed states where a clear experimental threshold is identified. The experimental threshold distributions are compared for five metrics: (1) the island overlap width, (2) pedestal top Chirikov overlap, (3) peeling edge displacement, (4) pedestal top resonant drive, and (5) edge dominant mode overlap. The distributions, the regularity of the dependence on RMP coil currents, and the sensitivities of a given metric to equilibrium reconstruction details are compared. The overlap metric proves to be a good compromise between including the appropriate plasma response physics and maintaining a numerical robustness. This quantity does not exhibit clear power-law scalings for projection, but machine learning can assist in predicting thresholds within the existing parameter ranges and providing uncertainty quantification of those predictions. Two new first-principles models, one utilizing a threshold from the non-linear Modified Rutherford equation evaluated at the pedestal top and one utilizing the SLAYER code to calculate the linear tearing threshold from torque balance, offer possible paths to extrapolation beyond the existing database parameter space.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗