Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Metrics”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 235 records · Page 13
All sourcesNASA NTRSDOE OSTI
DOE OSTI2015

SU-E-T-776: Use of Quality Metrics for a New Hypo-Fractionated Pre-Surgical Mesothelioma Protocol

Purpose: The “SMART” (Surgery for Mesothelioma After Radiation Therapy) approach involves hypo-fractionated radiotherapy of the lung pleura to 25Gy over 5 days followed by surgical resection within 7. Early clinical results suggest that this approach is very promising, but also logistically challenging due to the multidisciplinary involvement. Due to the compressed schedule, high dose, and shortened planning time, the delivery of the planned doses were monitored for safety with quality metric software. Methods: Hypo-fractionated IMRT treatment plans were developed for all patients and exported to Quality Reports™ software. Plan quality metrics or PQMs™ were created to calculate an objective scoring function for each plan. This allows for an objective assessment of the quality of the plan and a benchmark for plan improvement for subsequent patients. The priorities of various components were incorporated based on similar hypo-fractionated protocols such as lung SBRT treatments. Results: Five patients have been treated at our institution using this approach. The plans were developed, QA performed, and ready within 5 days of simulation. Plan Quality metrics utilized in scoring included doses to OAR and target coverage. All patients tolerated treatment well and proceeded to surgery as scheduled. Reported toxicity included grade 1 nausea (n=1), grade 1 esophagitis (n=1), grade 2 fatigue (n=3). One patient had recurrent fluid accumulation following surgery. No patients experienced any pulmonary toxicity prior to surgery. Conclusion: An accelerated course of pre-operative high dose radiation for mesothelioma is an innovative and promising new protocol. Without historical data, one must proceed cautiously and monitor the data carefully. The development of quality metrics and scoring functions for these treatments allows us to benchmark our plans and monitor improvement. If subsequent toxicities occur, these will be easy to investigate and incorporate into the metrics. This will improve the safe delivery of large doses for these patients.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2016

SU-C-BRB-05: Determining the Adequacy of Auto-Contouring Via Probabilistic Assessment of Ensuing Treatment Plan Metrics in Comparison with Manual Contours

Purpose: To determine if auto-contour and manual-contour—based plans differ when evaluated with respect to probabilistic coverage metrics and biological model endpoints for prostate IMRT. Methods: Manual and auto-contours were created for 149 CT image sets acquired from 16 unique prostate patients. A single physician manually contoured all images. Auto-contouring was completed utilizing Pinnacle’s Smart Probabilistic Image Contouring Engine (SPICE). For each CT, three different 78 Gy/39 fraction 7-beam IMRT plans are created; PD with drawn ROIs, PAS with auto-contoured ROIs, and PM with auto-contoured OARs with the manually drawn target. For each plan, 1000 virtual treatment simulations with different sampled systematic errors for each simulation and a different sampled random error for each fraction were performed using our in-house GPU-accelerated robustness analyzer tool which reports the statistical probability of achieving dose-volume metrics, NTCP, TCP, and the probability of achieving the optimization criteria for both auto-contoured (AS) and manually drawn (D) ROIs. Metrics are reported for all possible cross-evaluation pairs of ROI types (AS,D) and planning scenarios (PD,PAS,PM). Bhattacharyya coefficient (BC) is calculated to measure the PDF similarities for the dose-volume metric, NTCP, TCP, and objectives with respect to the manually drawn contour evaluated on base plan (D-PD). Results: We observe high BC values (BC≥0.94) for all OAR objectives. BC values of max dose objective on CTV also signify high resemblance (BC≥0.93) between the distributions. On the other hand, BC values for CTV’s D95 and Dmin objectives are small for AS-PM, AS-PD. NTCP distributions are similar across all evaluation pairs, while TCP distributions of AS-PM, AS-PD sustain variations up to %6 compared to other evaluated pairs. Conclusion: No significant probabilistic differences are observed in the metrics when auto-contoured OARs are used. The prostate auto-contour needs improvement to achieve clinically equivalent plans.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2023

Quantum metric nonlinear Hall effect in a topological antiferromagnetic heterostructure

Quantum geometry in condensed matter physics has two components: the real part quantum metric and the imaginary part Berry curvature. Whereas the effects of Berry curvature have been observed through phenomena such as the quantum Hall effect in 2D electron gases and the anomalous Hall effect (AHE) in ferromagnets, quantum metric has rarely been explored. Here, in this paper, we report a nonlinear Hall effect induced by quantum metric dipole by interfacing even-layered MnBi 2 Te 4 with black phosphorus. The quantum metric nonlinear Hall effect switches direction upon reversing the AFM spins and exhibits distinct scaling that is independent of the scattering time. Our results open the door to discovering quantum metric responses predicted theoretically and pave the way for applications that bridge nonlinear electronics with AFM spintronics.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
DOE OSTI2012

Implementing the Data Center Energy Productivity Metric

As data centers proliferate in both size and number, their energy efficiency is becoming increasingly important. We discuss the properties of a number of the proposed metrics of energy efficiency and productivity. In particular, we focus on the Data Center Energy Productivity (DCeP) metric, which is the ratio of useful work produced by the data center to the energy consumed performing that work. We describe our approach for using DCeP as the principal outcome of a designed experiment using a highly instrumented, high performance computing data center. We found that DCeP was successful in clearly distinguishing between different operational states in the data center, thereby validating its utility as a metric for identifying configurations of hardware and software that would improve (or even maximize) energy productivity. We also discuss some of the challenges and benefits associated with implementing the DCeP metric, and we examine the efficacy of the metric in making comparisons within a data center and among data centers.

Sego, Landon H↗
DOE OSTI2021

Evaluating Climate Models with the CLIVAR 2020 ENSO Metrics Package

El Niño–Southern Oscillation (ENSO) is the dominant mode of interannual climate variability on the planet, with far-reaching global impacts. It is therefore key to evaluate ENSO simulations in state-of-the-art numerical models used to study past, present, and future climate. Recently, the Pacific Region Panel of the International Climate and Ocean: Variability, Predictability and Change (CLIVAR) Project, as a part of the World Climate Research Programme (WCRP), led a community-wide effort to evaluate the simulation of ENSO variability, teleconnections, and processes in climate models. The new CLIVAR 2020 ENSO metrics package enables model diagnosis, comparison, and evaluation to 1) highlight aspects that need improvement; 2) monitor progress across model generations; 3) help in selecting models that are well suited for particular analyses; 4) reveal links between various model biases, illuminating the impacts of those biases on ENSO and its sensitivity to climate change; and to 5) advance ENSO literacy. By interfacing with existing model evaluation tools, the ENSO metrics package enables rapid analysis of multipetabyte databases of simulations, such as those generated by the Coupled Model Intercomparison Project phases 5 (CMIP5) and 6 (CMIP6). The CMIP6 models are found to significantly outperform those from CMIP5 for 8 out of 24 ENSO-relevant metrics, with most CMIP6 models showing improved tropical Pacific seasonality and ENSO teleconnections. Only one ENSO metric is significantly degraded in CMIP6, namely, the coupling between the ocean surface and subsurface temperature anomalies, while the majority of metrics remain unchanged.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

Evaluation of a land-atmosphere coupling metric computed from a ground-based infrared interferometer

Land-atmosphere feedbacks are a critical component of the hydrologic cycle. Vertical profiles of boundary layer temperature and moisture, together with information about the land surface, are used to compute land-atmosphere coupling metrics. Ground based remote sensing platforms, such as the Atmospheric Emitted Radiance Interferometer (AERI), can provide high temporal resolution vertical profiles of temperature and moisture. When co-located with soil moisture, surface flux, and surface meteorological observations, such as at the Atmospheric Radiation Measurement (ARM) Southern Great Plains (SGP) site, it is possible to observe both the terrestrial and atmospheric legs of land-atmosphere feedbacks. In this study, we compare a commonly used coupling metric computed from radiosonde-based data to that obtained from the AERI to characterize the accuracy and uncertainty in the metric derived from the two distinct platforms. This approach demonstrates the AERI’s utility where radiosonde observations are absent in time and/or space. Radiosonde and AERI based observations of the Convective Triggering Potential and Low-Level Humidity Index (CTP-HI low ) were computed during the 1200 UTC observation time and displayed good agreement during both 2017 and 2019 warm seasons. Furthermore, radiosonde and AERI derived metrics diagnosed the same atmospheric preconditioning based upon the CTP-HI low framework a majority of the time. When retrieval uncertainty was considered, even greater agreement was found between radiosonde and AERI derived classification. The AERI’s ability to represent this coupling metric well enabled novel exploration of temporal variability within the overnight period in CTP and HI low . Observations of CTP-HI low computed within a few hours of 1200 UTC were essentially equivalent, however with greater differences in time arose greater differences in CTP and HI low .

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Uncertain of uncertainties? A comparison of uncertainty quantification metrics for chemical data sets

Abstract With the increasingly more important role of machine learning (ML) models in chemical research, the need for putting a level of confidence to the model predictions naturally arises. Several methods for obtaining uncertainty estimates have been proposed in recent years but consensus on the evaluation of these have yet to be established and different studies on uncertainties generally uses different metrics to evaluate them. We compare three of the most popular validation metrics (Spearman’s rank correlation coefficient, the negative log likelihood (NLL) and the miscalibration area) to the error-based calibration introduced by Levi et al. ( Sensors 2022 , 22 , 5540). Importantly, metrics such as the negative log likelihood (NLL) and Spearman’s rank correlation coefficient bear little information in themselves. We therefore introduce reference values obtained through errors simulated directly from the uncertainty distribution. The different metrics target different properties and we show how to interpret them, but we generally find the best overall validation to be done based on the error-based calibration plot introduced by Levi et al. Finally, we illustrate the sensitivity of ranking-based methods (e.g. Spearman’s rank correlation coefficient) towards test set design by using the same toy model ferent test sets and obtaining vastly different metrics (0.05 vs. 0.65).

Rasmussen, Maria H.↗
DOE OSTI2021

Sensitivity-based Similarity Metrics for New Experiment Design Optimization

The nuclear data used in advanced reactor simulations requires validation. Data from nuclear criticality experiments can provide this validation. New nuclear criticality experiment design requires extensive knowledge and expert judgement such that the experimental design parameters are selected in such a way to keep the experiment subcritical. To aide in this experimental design process, professionals can utilize sensitivity and uncertainty analysis. Sensitivity and uncertainty analysis relies on matching new application experiments with currently existing benchmark experiments. Currently, there is functionality in the Whisper 1.1 software package to calculate a similarity metric based on neutron multiplication factor sensitivity coefficients between a new application designed by the user and existing International Criticality Safety Benchmark Experiment Project (ICSBEP) benchmarks. The Whisper 1.1 software package is included in Monte Carlo N-Particle ® Code Version 6.21 (MCNP ® 6.2). This work is geared toward expanding this capability to new similarity metrics based on beta-effective sensitivity coefficients and reactivity coefficient sensitivity coefficients. While the investigation of these sensitivity coefficients is presented in detail in separate works at this same conference, this work will be primarily focused on studying the similarity metrics in more detail. These similarity metrics will then be incorporated into the optimization algorithms used for experiment design in EUCLID (Experiments Underpinned by Computational Learning for Improvements in nuclear Data), which is a Los Alamos National Laboratory (LANL) project designed to constrain nuclear data of interest, such that adjustments can be made to possible inaccuracies. A more detailed optimization can be subsequently performed by breaking down these similarity metrics by isotope, reaction, and energy.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2020

Gaussian Process Optimization of Sensitivity-Based Similarity Metrics between New Nuclear Applications and New/Existing Benchmarks [Slides]

This presentation discusses Nuclear Criticality Safety (NCS) and how designing safe, new nuclear criticality experiments requires expert judgement, which could take years of experience. Sensitivity/uncertainty (S/U) analysis can be utilized by less experienced individuals to conservatively estimate uncertainties in important parameters, such as k eff , in newly proposed nuclear experiments. The presentation poses the question of how this analysis can be performed and states that the answer lies in matching new nuclear experiments with existing benchmark experiments using similarity metrics. By increasing the criticality safety of the application in this work, higher mass limits could be used in PF-4 operations. Additionally, the presentation discusses MCNP6.2®, Whisper-1.1, the software that can be used in this analysis. Also discussed is the fact that International Criticality Safety Benchmark Evaluation Project (ICSBEP) benchmarks rarely match new nuclear applications and that there are significant differences in given set of materials and/or geometry. If there are no benchmarks that match the application, the presentation discusses the possibility of creating new benchmarks. In conclusion, this work presents a Gaussian process (GP) optimization scheme that was used to generate new benchmarks with the highest sensitivity-based similarity metrics to user-defined nuclear applications. The Gaussian process optimization successfully designed 3 new experimental benchmarks that were highly correlated to the application of interest and had k eff values near critical. Optimization over c k,i-r has shown that investigating specific isotope reactions for different applications is crucial to designing benchmark experiments. Partial contribution from Pu dominates c k similarity metric. Future work includes testing new stand-alone similarity metrics or new combinations of similarity metrics as the design criterion of this optimization – design criterion is application dependent.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2019

Existing Ocean Energy Performance Metrics

This document summarizes existing Marine and Hydrokinetic (MHK) performance metrics known to the United States Department of Energy and national laboratories. This document was updated based on feedback from the MHK Energy community, however, this summary still may not be exhaustive. There are a wide variety of needs and uses for metrics. All stakeholders, such as developers, funding agencies, investors, and researchers, have a need for metrics and their many uses. It is evident that the sector will benefit from clear techno economic performance metrics to guide development towards success. There are international efforts underway to bring the community together to (1) understand what metrics/approaches are being used currently and (2) reach a global framework on the approach to the measurement of success. This document serves to list existing metrics known to the U.S. at the present, and is not meant to represent international efforts or consensus.

16 TIDAL AND WAVE POWER↗
DOE OSTI2021

Power System Resilience Metrics Augmentation for Critical Load Prioritization

One of the major goals of new grid operation regimes, such as transactive energy systems (TESs), is to make the power grid more resilient to withstand natural or man-made disasters and potential reliability events, and to continue to serve the maximum number of its customers. But it is a well-known fact to system operators that not all customers are the same. This implies that any discussion of TESs’ impacts on the resilience of the power system should consider the needs of its critical customers (such as the power system operation centers, fire and police stations, and hospitals) over those of other customers. When evaluating the resilience of the system, bonus points must be awarded to any system that could maintain its power supply to critical customers during a disturbance that may cause an outage. This report discusses critical infrastructure (CI) as found in the literature and then categorizes it based on the field to which the operations belong (such as human life/safety-related, operations management, necessary city operation, industrial customers, etc.). Each of these CI categories is further divided into types of critical customers (e.g., the human life/safety-related category has different types of customers like hospitals, fire and police stations, etc.). The entire demand of each of the critical customer types is not categorized as critical load (CL); instead, only a portion of the total load of these critical customers is characterized as critical load. This is done based on the categories of equipment, the function of which is crucial in the operation of the overall facility. CL categorization is performed to provide the ratio of the critical load portion to the overall load , so that it can serve as a parameter in the resilience evaluation of the grid through a metrics-based approach. Such categorization is important as it helps to augment the existing quantifiable resilience metrics with CL categorization. The metrics for a power system need to not only consider how well a system performed during a disturbance event, but also how it reduced strain and supplied power to its CLs. The first step in this process is characterize CLs in the system. After CL characterization, the next step is the inclusion of these loads in the resilience metrics. To that end, in this report weight-based augmentation of resilience metrics is proposed, where certain customers (the ones that are categorized as critical) are assigned higher weights than others. Though an overview of assigning weights to customers is discussed, there is no one-size-fits-all approach for every power system. The decisions made about assigning such weights to customers vary greatly from one operator to another, based on their unique systems and the current and predicted states of critical customers. This decision-making can include the type of disturbance event, which might only affect certain parts of the system. In general, analyzing critical customers before an event helps understand system vulnerabilities. It also helps in planning and conducting operations during the event, evaluating system performance after the event, and supporting better planning for future events. An alternative to the current practices of managing the grid for outages is an innovative TES, which has the potential to provide a platform for including distributed energy resources for managing CLs. This report also describes how TES qualities can help (1) to maintain power supply to critical customers for uninterrupted operations and (2) to restore lost power supply to the critical customers rapidly.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2023

Power System Wildfire Risks and Potential Solutions: A Literature Review & Proposed Metric

Several fire risk evaluation, fire tracking, and fire response resources are available. The risk metrics and fire response programs are sometimes modified to include a power system context. The risk metrics often evaluate the risk of fires causing power system faults or outages, especially on transmission systems. The response programs are modified to ensure the safety of power system equipment and first responders as well as to coordinate power system outages to both ensure safety during an active fire and prevent fire ignition during high risk periods. Although some aspects of wildfire responses have been adapted to include power system concerns, adaptations to power system operations and maintenance to include wildfire risks and responses are still nascent. In particular, a risk metric that evaluates the potential for power system components to ignite wildfires is needed to help guide power system upgrade efforts and power system fire safety measures. This document serves as a brief literature review of wildfire risk metrics and response programs and how they relate to power systems. It also includes a proposed risk metric and structure for describing the risk of a power system component igniting a fire.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2022

Performance Monitoring Program: Developing Comparative Metrics for Fitness-for-Duty Programs

To comply with U.S. Nuclear Regulatory Commission (NRC) regulations, licensees and entities authorized under Title 10 of the Code of Federal Regulations (CFR) Part 26 Section 26.3 (§ 26.3) are required to have fitness-for-duty (FFD) programs. Under § 26.3, the expectation of these FFD programs is to provide reasonable assurance that individuals who are granted unescorted access to nuclear power reactor protected areas and Category I fuel cycle facility material control areas are trustworthy, will perform their tasks in a reliable manner, are not under the influence of any substance, legal or illegal, that may impair their ability to perform their duties, and are not mentally or physically impaired from any cause that can adversely affect their ability to safely and competently perform their duties. Pacific Northwest National Laboratory (PNNL) was tasked with developing a performance monitoring program to risk inform NRC inspection and policy regarding quantitative FFD performance data. To meet this need, PNNL developed methodologies that could be implemented within a performance monitoring program. Throughout this report, these methodologies are referred to as comparative metrics. The NRC provided PNNL with 2016–2019 data from annual reporting forms and single positive test forms provided by licensees and other entities. For most of the comparative metrics, the analyses required customized processing, such as creating filtering fields, adding data fields and summaries, and joining datasets. The comparative metrics developed include: random testing rate, random policy violation rate, pre-access policy violation rate, subversion attempt rate, and number of policy violations by labor category. The comparative metrics can be used for parsing and visualizing FFD program data and monitoring FFD performance at the labor category, facility, licensee, and industry levels to risk inform NRC inspection and policy with regard to the FFD data currently collected from licensees and other entities that implement Part 26 requirements. Furthermore, these developed comparative metrics allow a more in-depth look at the FFD programs for the industry to discern trends and patterns that may warrant changes at an industry level and to inform policy decisions. These analyses should be refreshed as new FFD data become available.

99 GENERAL AND MISCELLANEOUS↗
DOE OSTI2023

Confidence Calibration Metrics

This technical report serves to summarize a literature search conducted that covered confidence calibration. This report is meant to serve as a solid starting reference for individuals interested in learning more about the confidence calibration domain as well as for individuals more familiar with this work – as a summarizing document for calibration metrics is notably lacking in the literature. This report is not meant to serve as a comprehensive review of everything that has been done in this field – in fact, the reader is encouraged to look further into this domain. We describe confidence and calibration and discuss properties of good calibration metrics. We detail various calibration and calibration-tangential metrics, presenting equations, algorithms, parameters, and an analysis of strengths and weaknesses. We apply a subset of these metrics to eight proxy confidence assessment datasets. We examine the various metrics in the context of model confidence. Finally, we discuss promising future directions and outstanding questions.

99 GENERAL AND MISCELLANEOUS↗
DOE OSTI2024

Integration of equitable resilience metrics into climate-informed electric utility planning processes: phase one

Working together, Sandia National Laboratories, Southern California Edison (SCE) - an Investor-Owned Utility (IOU) - and the California Public Utilities Commission (CPUC) are studying how electric utilities can use equity and resilience metrics to help inform the prioritization and sequencing of resilience-driven infrastructure investments. To this end, this project evaluated “Social Burden,” an equitable resilience metric which measures the potential impact of disruptions in access to non-electric critical services on people and estimates community resilience to these disruptions. The Social Burden was expanded to incorporate SCE’s existing equity metric and applied to evaluate the potential impacts from a range of climate-informed hypothetical outage scenarios developed under SCE’s 2022 Climate Adaptation Vulnerability Assessment. One baseline (“blue-sky”) state and eight different outage scenarios were evaluated to measure the potential impacts of the outages on non-electric infrastructure, critical services, and people. Key findings include: 1) the Social Burden framework is flexible enough to adapt to and build upon existing utility equity and/or resilience metrics, 2) Social Burden results highlight the high degree of non-electric service redundancy within the SCE service area with most (6/8) hypothetical outage scenarios predicted to increase people’s Social Burden by less than 10%; however, 3) access to critical services and people’s ability to obtain them is unequal and spatially clustered, meaning that there are some hypothetical outage scenarios (2/8) that will exert a higher toll on communities directly experiencing the outage as well as some nearby communities with pre-existing vulnerabilities. The report concludes with recommendations for potential use cases of the expanded Social Burden metric and identifies priority follow-on work. Potential use cases may include incorporating equity into IOU’s prioritization of climate resilience investments. Additionally, Social Burden analysis may provide additional data and insights to augment grid planning, potentially by identifying additional needs and/or prioritizing previously identified needs.

24 POWER TRANSMISSION AND DISTRIBUTION↗
NASA NTRS1976

Information on the metric system and related fields

This document contains about 7,600 references on the metric system and conversion to the metric system. These references include all known documents on the metric system as of December 1975, the month of enactment of the Metric Conversion Act of 1975. This bibliography includes books, reports, articles, presentations, periodicals, legislation, motion pictures, TV series, film strips, slides, posters, wall charts, education and training courses, addresses for information, and sources for metric materials and services. A comprehensive index is provided.

Lange, E.↗
NASA NTRS1982

Psychoacoustic considerations for helicopter noise quantification: Metrics, human response and criteria

The purpose of this paper is to examine several aspects of helicopter noise quantification from the standpoint of psychoacoustics. Noise metrics in common use to describe far-field aircraft noise and the noise characteristics which they consider are discussed. Some findings of recent psychoacoustic research related specifically to helicopter noise quantification are presented. Criteria for the accuracy of noise metrics to quantify helicopter noise are discussed. Finally, the prospects for improved metrics and research needed to develop and validate improved metrics or existing metrics are discussed.

Powell, C. A.↗
NASA NTRS1991

An evaluation of software testing metrics for NASA's mission control center

Software metrics are used to evaluate the software development process and the quality of the resulting product. Five metrics were used during the testing phase of the Shuttle Mission Control Center Upgrade at the NASA Johnson Space Center. All but one metric provided useful information. Based on the experience, it is recommended that metrics be used during the test phase of software development and additional candidate metrics are proposed for further study.

Stark, George E.↗