Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “reliability metrics”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4

Estimation of pipe failure frequencies in the absence of operational experience data: A pilot study

Probabilistic failure metrics such as leak frequency and rupture frequency are commonly used to characterize piping reliability. The methodologies for calculating the failure metrics rely on a complex set of input parameters. Operating experience data and experimental data play an important role in informing the different input parameters. The paper describes results and conclusions of a coordinated research project to benchmark three different reliability models using a four-step procedure: reference case definition of relevance to advanced reactor designs, input parameter calibration, validation of results, and application of different methodologies upon completion of the calibration and validation steps. The reference case is a weld consisting of nickel-base alloy 152/52 and located within a primary pressure boundary of an advanced reactor. This alloy is a class of structural materials known to be highly resistant to stress corrosion cracking. Synergies between the different methods are noted and the importance of a multi-disciplinary approach to input parameter development is underscored. A key conclusion is that the three methods are equally suitable for estimating failure frequencies. In any specific application, a selection of the most practical or effective computational tool can be considered. The comparison of alternative models confirms and helps to gain confidence in the computed failure frequency estimates. The study was part of a coordinated research project organized by the International Atomic Energy Agency.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

Assessing the Resource Adequacy and Community Impacts of an Evolving Grid: Considerations for State Regulators

As U.S. power systems continue to decarbonize, state regulators must be able to broadly assess the cost and reliability implications of proposed fossil retirements and the corresponding resources that are coming online to replace them. Specifically, regulators must evaluate whether proposed infrastructure investments are sufficient to ensure the long-term reliability of the system, while also protecting rate payers by ensuring that such investments are necessary and not excessive. Broadly speaking, this objective is typically framed in terms of maintaining long-term resource adequacy at the lowest cost, while also ensuring that other social and/or environmental objectives are satisfied. System planners currently have a host of sophisticated analytical tools at their disposal to evaluate the cost-benefit trade-offs of different approaches to ensuring resource adequacy. However, most of these tools and associated metrics were designed with traditional power systems in mind. Over the past decade, it has become increasingly clear that the metrics and models that have reliably assessed resource adequacy in systems dominated by large thermal and hydropower resources will be insufficient for systems with significant contributions from emerging technologies such as wind, solar and storage. Therefore, in recent years many new metrics and modeling approaches have been developed, and are increasingly being implemented, to meet the needs of the clean energy transition.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Automated model generation and parameter estimation of building energy models using an ontology-based framework

This study presents a methodology for automated model generation and parameter estimation of building energy models using semantic modeling and Bayesian estimation. Semantic modeling techniques are used to represent the system components and their interactions, facilitating the automatic generation of a simulation model from dynamic component models. The proposed approach is applied to a case study of a ventilation system where a simulation model is generated, calibrated, and assessed through different performance metrics. These metrics demonstrate the accuracy and reliability of both model point estimates and probabilistic prediction intervals across all model outputs. Overall, the proposed methodology offers a systematic and automated approach to model development and calibration in building energy systems, with potential applications in building performance analysis, monitoring, and optimization.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

A VOI Web Application for Distinct Geothermal Domains: Statistical Evaluation of Different Data Types within the Great Basin

The Great Basin region contains different domains that have different structural and hydrothermal flow patterns. Depending on the characteristics of these patterns, certain data types may be more successful at detecting hidden geothermal resources. In this paper, we quantitatively evaluate if certain data types are more successful in certain domains. Given different aquifer, strain and structural conditions, we explore which data types statistically reveal positively labeled geothermal sites. We utilize value of information (VOI) metrics to help quantify the reliability of data types to discriminate against "positive" and "negative" labeled geothermal sites. We also evaluate how kernel density estimation can help generalize the statistics that inform VOI, which is necessary given the limited data in geothermal exploration. Except for the Carbonate Aquifer, the highest ranking of the Vimperfect is the Local Structural Setting. Next, the slip and dilation tendency is first for Carbonate Aquifer and second for Central Nevada Seismic Belt and Western Great Basin. For the Carbonate Aquifer, heat flow is has the lowest Vimperfect value compared to the other three domains, which is consistent with the understanding of how heat flow measurements are masked by regional groundwater flow.

Bayesian analysis↗

State Spotlight on Resilience: The Michigan Public Service Commission and Data Informed Accountability

Resilience activities are an emerging area within regulatory utility policy. As resilience frameworks develop, many Public Utility Commissions (PUCs) are looking to initiate resilience activities and have identified enhancing grid reliability as a practical starting point. Reliability describes the grid’s ability to deliver electricity steadily and without interruptions under normal or “blue sky” conditions. Reliability is measured using industry standard metrics such as outage frequency (SAIFI) and outage duration (SAIDI). Resilience is broader, reflecting the grid’s ability to both withstand and recover from disruptions, particularly those caused by extreme events such as severe storms. A reliable grid minimizes routine disruptions; a resilient grid ensures the system can recover quickly when disruptions occur, preventing them from escalating into widespread or prolonged crises.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Robust Explanations using Diverse Adversarially Trained Ensembles, Multi-Modal Contrastive Learning, and Attribution-based Confidence Metrics

The primary objective of this project is to strengthen the trustworthiness of AI systems by designing algorithms that make their internal decision-making processes more understandable to human users. This involves creating clear, interpretable explanations for AI decisions and developing metrics to assess these explanations' validity and reliability. Significant progress has been achieved through (i) developing symbolic explanations, (ii) generating meaningful interpretive insights, (iii) establishing accuracy and confidence metrics, and (iv) devising methods to evaluate the knowledge boundaries of AI models. To date, the research findings have been shared in peer-reviewed publications, with accompanying scientific and technical information (STI) detailed below.

97 MATHEMATICS AND COMPUTING↗

Model-driven prediction for accelerator magnet diagnostics to improve operation reliability

Reliability is one of the most critical metrics for accelerator operation, especially in user facilities. To reduce costly facility downtime and provide an operational environment where system performance can be reliably predicted in support of scientific studies, we are developing a model-driven approach for prediction and anomaly detection. Here, in this study, we present the application of a model-driven method that employs a linear regression model to predict the future temperature, in real time, of accelerator magnets at the NSLS-II light source. This approach enables proactive identification of magnet-heating issues, facilitating magnet flushing prior to the occurrence of permanent damage without interrupting machine operation. The implementation of this method in the NSLS-II control room is described and the analysis of the online results is presented. The results demonstrate the model’s effectiveness in providing early alerts to engineers and improving the reliability of accelerator operations.

36 MATERIALS SCIENCE↗

A ductility metric for refractory-based multi-principal-element alloys

We propose a quantum-mechanical dimensionless metric, the local-lattice distortion (LLD), as a reliable predictor of ductility in refractory multi-principal-element alloys (RMPEAs). The LLD metric is based on electronegativity differences in localized chemical environments and combines atomic-scale displacements due to local lattice distortions with a weighted average of valence-electron count. To evaluate the effectiveness of this metric, we examined body-centered cubic (bcc) refractory alloys that exhibit ductile-to-brittle behavior. Our findings demonstrate that local-charge behavior can be tuned via composition to enhance ductility in RMPEAs. With finite-sized cell effects eliminated, the LLD metric accurately predicted the ductility of arbitrary alloys, which compares well with existing tensile-elongation experiments. To validate further, we qualitatively evaluated the ductility of two refractory RMPEAs, i.e., NbTaMoW and Mo 72 W 13 Ta 10 Ti 2.5 Zr 2.5 , through the observation of crack formation under indentation, again showing excellent agreement with LLD predictions. Additionally, a comparative study of three refractory alloys provides further insights into the electronic-structure origin of ductility in refractory RMPEAs. This proposed metric enables rapid and accurate assessment of ductility behavior in the vast RMPEA composition space.

36 MATERIALS SCIENCE↗

Zero-Emission Transit Bus Needs Assessment

The transition to zero-emissions vehicles (ZEVs) in public transit has gained traction due to significant federal investments from the Bipartisan Infrastructure Law (BIL) and the Inflation Reduction Act (IRA). This needs assessment, commissioned by the Joint Office of Energy and Transportation and conducted by researchers at the Idaho National Laboratory, explores the current state of electrification in transit agencies, identifying barriers to implementation, potential funding sources, and operational considerations necessary for a successful transition. The assessment involved qualitative interviews with representatives from 19 transit service providers across diverse geographic regions. Key findings highlight the challenges related to bus facilities and operations, which require careful planning for charging infrastructure and maintenance capabilities to accommodate battery electric buses (BEBs) and hydrogen fuel cell buses (HFCBs). Agencies reported operational hurdles due to the shorter range of BEBs compared to diesel buses, necessitating revised scheduling and routing strategies. Despite these challenges, many agencies expressed optimism about their capacity to adapt. Funding availability emerged as a critical factor influencing the transition to ZEVs. While agencies welcomed increased financial support, particularly from the Low or No Emission Grant Program (Lo-No), concerns about the sustainability of this funding and the ongoing operational costs were prevalent. The need for a comprehensive funding inventory was underscored to ensure transit agencies are aware of all available resources. Technological constraints were significant barriers to ZEV adoption. The limited range of BEBs was frequently cited as a concern, leading to operational challenges and reliability issues. Agencies reported difficulties in sourcing replacement parts, which exacerbated downtime and maintenance challenges. Workforce development and training were identified as pivotal for a successful transition. Many agencies rely heavily on manufacturers for technician training, highlighting the need for scalable training programs that equip staff with the necessary skills to maintain electric powertrains effectively. This assessment offers actionable recommendations for the Joint Office, including enhancing outreach to transit agencies, developing resources for effective utility partnerships, and facilitating comprehensive training programs. Establishing a zero-emission bus evaluation program to track performance metrics such as cost, range, and reliability could provide valuable insights for transit agencies. The needs assessment provides a detailed examination of the challenges and opportunities facing transit agencies in their transition to zero-emissions bus fleets. By addressing these issues through targeted support, stakeholders can collaboratively work towards a cleaner, more sustainable public transportation system that benefits all communities.

33 - ADVANCED PROPULSION SYSTEMS↗

Velocity- and pressure-based metrics for estimating strike injuries during fish passage through hydro turbines

Hydropower is a crucial source of clean and reliable energy worldwide, and its importance will continue to grow. To enhance the environmental sustainability of hydropower development and operations, it is essential to predict the strike injury rates of fish passing through turbines accurately and cost-effectively. However, conventional experiments involving a large number of live fish are still commonly conducted in practice, and previous attempts mainly focused on the dose-response relationships at an individual level or relative comparisons of biological characterization between different conditions. Thus, this study proposes two novel strike metrics based on velocity and pressure (M V and M P ) measured by the cutting-edge Sensor Fish (SF) technology, designed to quantify the biological effects of strikes and collisions between fish and rigid hydraulic structures. SF measurements and 48-hour survival rates of live fish from a study conducted at the Ice Harbor Dam on the Snake River were used to establish thresholds for the two proposed metrics. These metrics and thresholds were then applied to three additional SF studies conducted at hydropower facilities featuring Kaplan turbines within the Columbia River basin. Results from the application of the strike metrics indicate that the estimated survival rates derived from both metrics agree well with previously reported live fish survival rates. Furthermore, the study identified specific passage regions with elevated risks of fish mortality due to strikes and collisions. Overall, the proposed strike metrics present a reliable and cost-effective approach to reducing reliance on live fish and lowering costs in fish passage studies.

Biological characterization↗

Scaling open-weight large language models for hydropower regulatory information extraction: A systematic analysis

Information extraction from regulatory and technical documents using large language models (LLMs) involves practical trade-offs between extraction quality and computational cost. We evaluate eight open-weight LLMs spanning 0.6B–70B parameters on hydropower licensing documents and report deployment-oriented evidence under a unified extraction schema and evaluation protocol. Across the model set, we observe clear scale-dependent trends in both baseline extraction quality and the effectiveness of reflective reasoning (self-checking) under our fixed-prompt, no-augmentation setting. Mid-scale models often provide a favorable balance of accuracy and efficiency, whereas the smallest models show limited or inconsistent gains from the reasoning variants tested. Larger models achieve the highest overall F1 scores but incur substantially greater compute and infrastructure requirements. We further find that reliability failure modes can distort conventional metrics in this domain: in particular, high recall can coincide with systematic extraction errors when models fabricate values for fields that are absent from the source text, underscoring the importance of conservative null handling and evidence-grounded evaluation. Overall, our study provides a reproducible resource–performance comparison for open-weight LLM-based extraction in hydropower regulatory documentation and offers practical guidance for model selection under different deployment constraints.

Evaluation protocol↗

Comprehensive Analysis of Streaming and Shutdown Dose Rate Experiments at JET with ORNL Fusion Neutronics Workflows

Current experimental fusion systems and conceptual designs of fusion pilot plants (FPPs) are growing in complexity and size. Several radiation metrics are crucial to the safe operation of fusion machines, including neutron flux streaming through openings and the shutdown dose rate (SDDR). Most current designs of advanced experimental fusion systems—and the most probable candidates for FPPs—are based on the tokamak concept, which is prone to neutron streaming through the myriad openings needed for diagnostic and support systems. SDDR is caused by decay gamma rays from radionuclides that become activated by neutrons during the operation of a fusion system that use deuterium-deuterium (DD), tritium-tritium, or deuterium-tritium plasma. Because computational tools have become essential for determining these radiation metrics, they must be validated against reliable and applicable experimental data. Experiments at the Joint European Torus (JET) provide a unique source of experimental data for validating computational tools and nuclear data used to determine SDDR and neutron fluxes in streaming-dominated geometries. Here, this paper presents the comprehensive analysis of the high-performance DD JET SDDR, and streaming experiments performed using Oak Ridge National Laboratory (ORNL) fusion workflows. The computational results were compared with experimental results that consist of online SDDR measurements with ionization chambers and neutron fluence streaming measurements using thermoluminescent detectors. The ratio of calculated-to-experimental SDDR values ranges from 0.6 to 2.5, and the streaming results range from 0.5 to 8.0. Future work will include analyzing the JET 2021 DTE2 campaign alongside the integration of the Shift Monte Carlo transport code into all ORNL fusion neutronics workflows.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Operations, maintenance, and cost considerations for PV+Storage in the United States

Battery storage systems are increasingly being installed at photovoltaic (PV) sites to address supply-demand balancing needs. Although there is some understanding of costs associated with PV operations and maintenance (O&M), costs associated with emerging technologies such as PV plus storage lack details about the specific systems and/or activities that contribute to the cost values. This study aims to address this gap by exploring the specific factors and drivers contributing to utility-scale PV plus storage systems (UPVS) O&M activities costs, including how technology selection, data collection, and related and ongoing challenges. Specifically, we used semi-structured interviews and questionnaires to collect information and insights from utility-scale owners and operators. Data was collected from 14 semi-structured interviews and questionnaires representing 51.1 MW with 64.1 MWh of installed battery storage capacity within the United States (U.S.). Differences in degradation rate, expected life cycle, and capital costs are observed across different storage technologies. Most O&M activities at UPVS related to correcting under-performance. Fires and venting issues are leading safety concerns, and owner operators have installed additional systems to mitigate these issues. There are ongoing O&M challenges due the lack of storage-specific performance metrics as well as poor vendor reliability and parts availability. Insights from this work will improve our understanding of O&M consideration at PV plus storage sites.

14 SOLAR ENERGY↗

The Current State of Light-Duty Electric Vehicle Supply Equipment Costs: An Assessment of Contemporary Understanding

This study uses a hybrid meta-analysis and literature-review approach to understand the current state of knowledge regarding the costs of electric vehicle supply equipment (EVSE). We present a novel way to consider, categorize, and label measures of cost and show cost measure estimates from a sample of 13 recent studies. We find that in general, there is too much variation and too few commonly represented EVSE cost measures to reasonably provide aggregate figures for these measures. We propose a convention for presenting EVSE cost measures that includes the application (commercial or residential), the power level (Level 1, Level 2, DCFC [with further distinction based on rated power capacity]), and the type of cost measure (hardware, installation, operation, and total cost). We contend that providing researchers with standard cost measures will help to advance our knowledge of EVSE costs by ensuring that future work will use common metrics. Establishing common metrics will enable conventional meta-analyses that will make assessments of EVSE costs even more accessible. Additionally, common metrics will make tracking costs more reliable as the technology continues to evolve and become more ubiquitous.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

Functional protein mining with conformal guarantees

Molecular structure prediction and homology detection offer promising paths to discovering protein function and evolutionary relationships. However, current approaches lack statistical reliability assurances, limiting their practical utility for selecting proteins for further experimental and in-silico characterization. To address this challenge, we introduce a statistically principled approach to protein search leveraging principles from conformal prediction, offering a framework that ensures statistical guarantees with user-specified risk and provides calibrated probabilities (rather than raw ML scores) for any protein search model. Our method (1) lets users select many biologically-relevant loss metrics (i.e. false discovery rate) and assigns reliable functional probabilities for annotating genes of unknown function; (2) achieves state-of-the-art performance in enzyme classification without training new models; and (3) robustly and rapidly pre-filters proteins for computationally intensive structural alignment algorithms. Our framework enhances the reliability of protein homology detection and enables the discovery of uncharacterized proteins with likely desirable functional properties.

59 BASIC BIOLOGICAL SCIENCES↗

PV Evolution in the Light of Circular Economy

As photovoltaic (PV) deployments increase, the increasing material volume presents challenges throughout the supply chain and eventual end-of-life waste management. Quantifying the value of lifetime, reliability, repair, reuse, recycling, and other options to increase circularity can help identify and prioritize research and technical solutions required to integrate PV into the circular economy. In this work, we capture the evolution of PV module technology, such as improved reliability and increasing market share of glass/glass modules. We discuss material circularity, lifetime and reliability, and energy return on investment as quantitative metrics to define and evaluate circularity and sustainability.

bifacial↗

Establishing performance metrics for quantitative non-targeted analysis: a demonstration using per- and polyfluoroalkyl substances

Abstract Non-targeted analysis (NTA) is an increasingly popular technique for characterizing undefined chemical analytes. Generating quantitative NTA (qNTA) concentration estimates requires the use of training data from calibration “surrogates,” which can yield diminished predictive performance relative to targeted analysis. To evaluate performance differences between targeted and qNTA approaches, we defined new metrics that convey predictive accuracy, uncertainty (using 95% inverse confidence intervals), and reliability (the extent to which confidence intervals contain true values). We calculated and examined these newly defined metrics across five quantitative approaches applied to a mixture of 29 per- and polyfluoroalkyl substances (PFAS). The quantitative approaches spanned a traditional targeted design using chemical-specific calibration curves to a generalizable qNTA design using bootstrap-sampled calibration values from “global” chemical surrogates. As expected, the targeted approaches performed best, with major benefits realized from matched calibration curves and internal standard correction. In comparison to the benchmark targeted approach, the most generalizable qNTA approach (using “global” surrogates) showed a decrease in accuracy by a factor of ~4, an increase in uncertainty by a factor of ~1000, and a decrease in reliability by ~5%, on average. Using “expert-selected” surrogates ( n = 3) instead of “global” surrogates ( n = 25) for qNTA yielded improvements in predictive accuracy (by ~1.5×) and uncertainty (by ~70×) but at the cost of further-reduced reliability (by ~5%). Overall, our results illustrate the utility of qNTA approaches for a subclass of emerging contaminants and present a framework on which to develop new approaches for more complex use cases. Graphical Abstract

Pu, Shirley (ORCID:0000000201223797)↗

Temporal complementarity of marine renewables with wind and solar generation: Implications for GB system benefits

Wave and tidal energy have the potential to provide benefits to power systems with high proportions of stochastic renewable generation. This is particularly applicable with wind and solar photovoltaics, as the offsetting of these renewable resources results in more reliable renewable generation. This study utilises ten metrics to quantify the temporal complementarity and supply-demand balancing requirements of the energy mix in Great Britain, to investigate the potential magnitude of these system benefits. Wave and tidal generation profiles are created using historical resource data and hydrodynamic models. The results show that the inclusion of wave and tidal generation creates a renewable energy mix which is more available under multiple conditions: throughout a year of operation; at times of peak demand; for multiple consecutive hourly time periods; and at times when wind and solar generation are not available. Three regional case studies also show that the inclusion of marine energy allows for improved regional supply-demand matching, reducing instances of energy shortage and excess and potentially relieving transmission congestion at particularly constrained locations within GB. Finally, the implications of these findings are discussed in terms of GB wholesale market operation, system balancing and system security.

16 TIDAL AND WAVE POWER↗