SEARCH · Engineering Papers

Results for “Metrics”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 325 records · Page 18

All sources NASA NTRS DOE OSTI

DOE OSTI2019

Validation Metrics for Deterministic and Probabilistic Data

Computational modeling and simulation are paramount to modern science. Computational models often replace physical experiments that are prohibitively expensive, dangerous, or occur at extreme scales. Thus, it is critical that these models accurately represent and can be used as replacements for reality. This paper provides an analysis of metrics that may be used to determine the validity of a computational model. While some metrics have a direct physical meaning and a long history of use, others, especially those that compare probabilistic data, are more difficult to interpret. Furthermore, the process of model validation is often application-specific, making the procedure itself challenging and the results difficult to defend. We therefore provide guidance and recommendations as to which validation metric to use, as well as how to use and decipher the results. Furthermore an example is included that compares interpretations of various metrics and demonstrates the impact of model and experimental uncertainty on validation processes.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2015

WE-AB-BRA-01: 3D-2D Image Registration for Target Localization in Spine Surgery: Comparison of Similarity Metrics Against Robustness to Content Mismatch

Purpose: In image-guided spine surgery, mapping 3D preoperative images to 2D intraoperative images via 3D-2D registration can provide valuable assistance in target localization. However, the presence of surgical instrumentation, hardware implants, and soft-tissue resection/displacement causes mismatches in image content, confounding existing registration methods. Manual/semi-automatic methods to mask such extraneous content is time consuming, user-dependent, error prone, and disruptive to clinical workflow. We developed and evaluated 2 novel similarity metrics within a robust registration framework to overcome such challenges in target localization. Methods: An IRB-approved retrospective study in 19 spine surgery patients included 19 preoperative 3D CT images and 50 intraoperative mobile radiographs in cervical, thoracic, and lumbar spine regions. A neuroradiologist provided truth definition of vertebral positions in CT and radiography. 3D-2D registration was performed using the CMA-ES optimizer with 4 gradient-based image similarity metrics: (1) gradient information (GI); (2) gradient correlation (GC); (3) a novel variant referred to as gradient orientation (GO); and (4) a second variant referred to as truncated gradient correlation (TGC). Registration accuracy was evaluated in terms of the projection distance error (PDE) of the vertebral levels. Results: Conventional similarity metrics were susceptible to gross registration error and failure modes associated with the presence of surgical instrumentation: for GI, the median PDE and interquartile range was 33.0±43.6 mm; similarly for GC, PDE = 23.0±92.6 mm respectively. The robust metrics GO and TGC, on the other hand, demonstrated major improvement in PDE (7.6 ±9.4 mm and 8.1± 18.1 mm, respectively) and elimination of gross failure modes. Conclusion: The proposed GO and TGC similarity measures improve registration accuracy and robustness to gross failure in the presence of strong image content mismatch. Such registration capability could offer valuable assistance in target localization without disruption of clinical workflow. G. Kleinszig and S. Vogt are employees of Siemens Healthcare.

60 APPLIED LIFE SCIENCES↗

DOE OSTI2021

Regional Relative Risk, a Physics-Based Metric for Characterizing Airborne Infectious Disease Transmission

Airborne infectious disease transmission events occur over a wide range of spatial scales and can be an important means of disease transmission. Physics- and biology-based models can assist in predicting airborne transmission events, overall disease incidence, and disease control strategy efficacy. We describe a new theory that extends current approaches for the case in which an individual is infected by a single airborne particle, including the scenario in which numerous infectious particles are present in the air but only one causes infection. A single infectious particle can contain more than one pathogenic microorganism and be physically larger than the pathogen itself. This approach allows robust relative risk estimates even when there is wide variation in (i) individual exposures and (ii) the individual response to that exposure (the pathogen dose-response function can take any mathematical form and vary by individual). Based on this theory, we propose the regional relative risk—a new metric, distinct from the traditional relative risk metric, that compares the risk between two regions. In theory, these regions can range from individual rooms to large geographic areas. In this paper, we apply the regional relative risk metric to outdoor disease transmission events over spatial scales ranging from 50 m to 20 km, demonstrating that in many common cases minimal input information is required to use the metric. Also, we demonstrate that the model predictions are consistent with data from prior outbreaks. Future efforts could apply and validate this theory for other spatial scales, such as transmission within indoor environments. This work provides context for (i) the initial stages of an airborne disease outbreak and (ii) larger-scale disease spread, including unexpected low-probability disease “sparks” that potentially affect remote populations, a key practical issue in controlling airborne disease outbreaks.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2024

HIPSTER (Harmonized Impacts across Products, Scenarios, and Technologies for Environmental and Resource metrics) [SWR-24-17]

Development of a prototype, code-based life cycle assessment framework, the Harmonized Impacts across Products, Scenarios, and Technologies for Environmental and Resource metrics (HIPSTER), to coherently assess outputs of prospective NREL models across environmental and resource use metrics. Eventually, HIPSTER is aimed to also cover socioeconomic and justice metrics. At present, NREL model scenario impact assessments are performed incoherently, as they as based on varying assumptions and inputs, specifically life cycle inventories, and cover different system boundaries. HIPSTER was created to align the assumptions and boundaries with those of cradle-to-grave life cycle assessment. In FY22, HIPSTER was successfully applied using scenarios from the ReEDS™ model, NREL's flagship capacity expansion model resulting in cradle-to-grace time-series life cycle impacts across LCA midpoints and resource use metrics.

Ghosh, Tapajyoti↗

DOE OSTI2024

Metric DBSCAN

SAND2025-11725O Metric DBSCAN is an implementation of the popular DBSCAN clustering algorithm that works in general metric spaces. DBSCAN is a clustering algorithm, a fundamental building block in machine learning. It takes a set of objects and, given some notion of distance, identifies coherent groups of objects. With Metric DBSCAN, users can provide an arbitrary function to compute distance. Nearly all existing implementations of DBSCAN restrict distance to one of a few formulations. Metric DBScan accomplishes this cleanly and efficiently. The Python source code is on Github. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525.

Dalbey, Keith↗

DOE OSTI2016

Metrics for the Diurnal Cycle of Precipitation: Toward Routine Benchmarks for Climate Models

In this paper, metrics are proposed—that is, a few summary statistics that condense large amounts of data from observations or model simulations—encapsulating the diurnal cycle of precipitation. Vector area averaging of Fourier amplitude and phase produces useful information in a reasonably small number of harmonic dial plots, a procedure familiar from atmospheric tide research. The metrics cover most of the globe but down-weight high-latitude wintertime ocean areas where baroclinic waves are most prominent. This enables intercomparison of a large number of climate models with observations and with each other. The diurnal cycle of precipitation has features not encountered in typical climate model intercomparisons, notably the absence of meaningful “average model” results that can be displayed in a single two-dimensional map. Displaying one map per model guides development of the metrics proposed here by making it clear that land and ocean areas must be averaged separately, but interpreting maps from all models becomes problematic as the size of a multimodel ensemble increases. Global diurnal metrics provide quick comparisons with observations and among models, using the most recent version of the Coupled Model Intercomparison Project (CMIP). This includes, for the first time in CMIP, spatial resolutions comparable to global satellite observations. Finally, consistent with earlier studies of resolution versus parameterization of the diurnal cycle, the longstanding tendency of models to produce rainfall too early in the day persists in the high-resolution simulations, as expected if the error is due to subgrid-scale physics.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2020

Flux Seasonality Metrics Database: A companion dataset for FLUXNET annual product

This dataset include standardized phenological transition dates, and derived season lengths of active, dormant and transition seasons of ecosystem carbon and energy fluxes, as reported in FLUXNET. Such metrics are unique in that they characterize the seasonal dynamics of these fluxes in their own terms rather than in reference to traditional phenology indicators that mostly relate to vegetation structural characteristics. The Flux Seasonality Metrics Database (FSMD) generated 17 standardized seasonality metrics for 4 different fluxes including gross primary productivity (GPP), ecosystem respiration (RE), latent heat (LE) and sensible heat (H) reported in FLUXNET2015 Dataset for about 200 sites and 1500 site-years of data globally. Each metric is reported as an estimate and a confidence interval, which is a function of data coverage and flux variability. We anticipate that the FSMD will facilitate new model validation exercises, and stimulate new research in global change and Earth science disciplines where land-atmosphere exchange dynamics play a central role.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2014

Light Water Reactor Sustainability Program Operator Performance Metrics for Control Room Modernization: A Practical Guide for Early Design Evaluation

As control rooms are modernized with new digital systems at nuclear power plants, it is necessary to evaluate the operator performance using these systems as part of a verification and validation process. There are no standard, predefined metrics available for assessing what is satisfactory operator interaction with new systems, especially during the early design stages of a new system. This report identifies the process and metrics for evaluating human system interfaces as part of control room modernization. The report includes background information on design and evaluation, a thorough discussion of human performance measures, and a practical example of how the process and metrics have been used as part of a turbine control system upgrade during the formative stages of design. The process and metrics are geared toward generalizability to other applications and serve as a template for utilities undertaking their own control room modernization activities.

99 GENERAL AND MISCELLANEOUS↗

DOE OSTI2014

Deep Energy Retrofit Performance Metric Comparison: Eight California Case Studies

In this paper we will present the results of monitored annual energy use data from eight residential Deep Energy Retrofit (DER) case studies using a variety of performance metrics. For each home, the details of the retrofits were analyzed, diagnostic tests to characterize the home were performed and the homes were monitored for total and individual end-use energy consumption for approximately one year. Annual performance in site and source energy, as well as carbon dioxide equivalent (CO 2 e) emissions were determined on a per house, per person and per square foot basis to examine the sensitivity to these different metrics. All eight DERs showed consistent success in achieving substantial site energy and CO 2 e reductions, but some projects achieved very little, if any source energy reduction. This problem emerged in those homes that switched from natural gas to electricity for heating and hot water, resulting in energy consumption dominated by electricity use. This demonstrates the crucial importance of selecting an appropriate metric to be used in guiding retrofit decisions. Also, due to the dynamic nature of DERs, with changes in occupancy, size, layout, and comfort, several performance metrics might be necessary to understand a project’s success.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

DOE OSTI2016

Key metrics for HFIR HEU and LEU models

This report compares key metrics for two fuel design models of the High Flux Isotope Reactor (HFIR). The first model represents the highly enriched uranium (HEU) fuel currently in use at HFIR, and the second model considers a low-enriched uranium (LEU) interim design fuel. Except for the fuel region, the two models are consistent, and both include an experiment loading that is representative of HFIR's current operation. The considered key metrics are the neutron flux at the cold source moderator vessel, the mass of 252 Cf produced in the flux trap target region as function of cycle time, the fast neutron flux at locations of interest for material irradiation experiments, and the reactor cycle length. These key metrics are a small subset of the overall HFIR performance and safety metrics. They were defined as a means of capturing data essential for HFIR's primary missions, for use in optimization studies assessing the impact of HFIR's conversion from HEU fuel to different types of LEU fuel designs.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

DOE OSTI2018

Methodology to Calculate the ACE and HPQ Metrics Used in the Wave Energy Prize

The U.S. Department of Energy's Wave Energy Prize Competition encouraged the development of innovative deep-water wave energy conversion technologies that at least doubled device performance above the 2014 state of the art. Because levelized cost of energy (LCOE) metrics are challenging to apply equitably to new technologies where significant uncertainty exists in design and operation, the prize technical team developed a reduced metric as proxy for LCOE, which provides an equitable comparison of low technology readiness level wave energy converter (WEC) concepts. The metric is called 'ACE' which is short for the ratio of the average climate capture width to the characteristic capital expenditure. The methodology and application of the ACE metric used to evaluate the performance of the technologies that competed in the Wave Energy Prize are explained in this report.

17 WIND ENERGY↗

DOE OSTI2018

Implementing Snowball Metrics in the Battelle-Affiliated DOE National Laboratories

The purpose of the Battelle Snowball Metric Working Group is to use the Snowball Metrics framework to build and employ a method to calculate metrics that will enable Battelle-affiliated National Laboratories to better understand their strengths and weaknesses in a few representative areas. This effort supports the overall aim of these Laboratories to use data-driven and evidence-based metrics to build and monitor effective strategies for evaluating science and technology (S&T) as individual laboratories, and as a Battelle institution. Battelle-affiliated National Laboratories include Brookhaven National Laboratory, Idaho National Laboratory, National Renewable Energy Laboratory, Oak Ridge National Laboratory, and Pacific Northwest National Laboratory.

99 GENERAL AND MISCELLANEOUS↗

DOE OSTI2018

Metrics and Benchmarks for Quantum Processors: State of Play

A compelling narrative has taken hold as quantum computing explodes into the commercial sector: Quantum computing in 2018 is like classical computing in 1965. In 1965 Gordon Moore wrote his famous paper about integrated circuits, saying: "At present, [minimum cost] is reached when 50 components are used per circuit. But... the complexity for minimum component costs has increased at a rate of roughly a factor of two per year... by 1975, the number of components per integrated circuit for minimum cost will be 65,000." This narrative is both appealing (we want to believe that quantum computing will follow the incredibly successful path of classical computing!) and plausible (2018 saw IBM, Intel, and Google announce 50-qubit integrated chips). But it is also deeply misleading. Here is an alternative: Quantum computing in 2018 is like classical computing in 1938. In 1938, John Atanasoff and Clifford Berry built the very first electronic digital computer. It had no program, and was not Turing-complete. Vacuum tubes — the standard "bit" for 20 years — were still 5 years in the future. ENIAC and the achievement of "computational supremacy" (over hand calculation) wouldn't arrive for 8 years, despite the accelerative effect of WWII. Integrated circuits and the information age were more than 20 years away. Neither of these analogies is perfect. Quantum computing technology is more like 1938, while the level of funding and excitement suggest 1965 (or later!). But the point of the cautionary analogy to 1938 is simple: Quantum computing in 2018 is a research field. It is far too early to establish metrics or benchmarks for performance. The best role for neutral organizations like IEEE is to encourage and shape research into metrics and benchmarks, so as to be ready when they become necessary. This white paper presents the evidence and reasoning for this claim. We explain what it means to say that quantum computing is a "research field", and why metrics and benchmarks for quantum processors also constitute a research field. We discuss the potential for harmful consequences of prematurely establishing standards or frameworks. We conclude by suggesting specific actions that IEEE or similar organizations can take to accelerate the development of good metrics and benchmarks for quantum computing.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2019

Evaluation of Machine Learning Approaches to Estimate Aerosol Mixing State Metrics in Atmospheric Models

Aerosol mixing state describes how aerosol compositions are distributed among atmospheric aerosol particles in a population. Oversimplified assumptions of aerosol mixing state in atmospheric modeling can introduce errors in estimations of weather and climate-relevant aerosol microphysical properties. A more comprehensive representation of the aerosol mixing state can be achieved in principle with a Particle-resolved Monte Carlo (PartMC) model but at added computational cost that may be prohibitive for direct invocation in operational numerical weather prediction or multi-year climate simulations. The aim of our research is to explore the machine learning (ML) methodologies for estimating aerosol mixing state metrics, which we define here in three different ways: with respect to (a) hygroscopicity; (b) optical properties; and (c) chemical species abundance. We adopted a data-driven approach, leveraging deep learning and statistical learning techniques, to take advantage of massive PartMC model simulations. First, we performed particle-resolved simulations by PartMC to create a series of scenarios considering a range of global environmental conditions. Each scenario consists of aerosol populations and corresponding mixing state metrics. The gas concentration, aerosol mass concentration, environmental variables and mixing state metrics of each population constitute the datasets for machine learning implementations. We have adopted and evaluated various configurations of machine learning methodologies in this investigation, embracing deep learning, Extreme Gradient Boosting (XGBoost) algorithm, and the ensemble approaches. After a rigorous model selection process, we identified an appropriate model to derive estimates of aerosol mixing state metrics. We used the computational and data resources of the Oak Ridge Leadership Computing Facility (OLCF) and the ORNL Compute and Data Environment for Science (CADES). The NVIDIA DGX-1 hardware was used for the prototyping of the ML models. Our approach has allowed us to gain a new understanding of how machine learning methodologies can be applied to improve the representation of aerosol mixing state in atmospheric models and benefit the atmospheric research community. Next, we plan to extend our research and methodology to quantify some of the aerosol-related uncertainties in the E3SM Atmospheric Model (EAM) Version 1.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2016

National Hurricane Program Metrics Framework

The need for metrics for planning and response measures was identified as key gap to be addressed in the National Hurricane Program's (NHP) Technology Modernization effort. This document proposes a framework for defining a set of metrics for planning and response that will be implemented in the NHP products of hurricane evacuation studies (HES) and post-storm assessments (PSA). To determine the feasibility of this framework, a survey of current HES and PSAs was carried out followed by and then used to determine if the proposed metrics are currently captured. While there is a wide variety in data availability and detail, the implementation of these metrics is not only feasible but presents an opportunity to improve on current practices. The final implementation of this framework shall require the ongoing feedback from local, state, tribal, and federal stakeholders.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2017

Applying Image Clutter Metrics to Domain-Specific Expert Visual Search

Visual clutter metrics play an important role in both the design of information visualizations and in the continued theoretical development of visual search models. In visualization design, clutter metrics provide a mathematical prediction of the complexity of the display and the difficulty associated with locating and identifying key pieces of information. In visual search models, they offer a proxy to set size, which represents the number of objects in the search scene, but is difficult to estimate in real-world imagery. In this article, we first briefly review the literature on clutter metrics and then contribute our own results drawn from studies in two security-oriented visual search domains: airport X-ray imagery and radar imagery. We analyze our results with an eye toward bridging the gap between the scene features evaluated by current clutter metrics and the features that are relevant to our security tasks. The article concludes with a brief discussion of possible research steps to close this gap.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2019

Grid Modernization: Metrics Analysis (GMLC1.1) - Affordability (Reference Document, Volume 6)

The GMLC Metric Team: 1) Identified important, but often overlooked, affordability metrics that measure the impacts of electricity costs on residential customers (customer cost burden, affordability gap, and affordability headcount gap); 2) Created a public-facing website for the residential sector that compiles publicly available data and displays affordability metrics at both the state and county level; and 3) demonstrated the use of the metrics and website for three remote village utilities in Alaska and a major electricity utility in California, Southern California Edison (SCE).

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

DOE OSTI2021

Review of Energy Equity Metrics

A literature review of energy equity and energy justice metrics was performed to support efforts to develop an energy equity metrics framework. Pacific Northwest National Laboratory (PNNL) reviewed the available literature, surveyed work in progress on the topic, and solicited expert feedback to lay the groundwork for metrics development and provide reference material for equity research and development applications. The literature review identified three distinct equity metric types: target population identification, investment decision making, and program impact assessment.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗