Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “reliability metrics”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 127 records · Page 7
All sourcesNASA NTRSDOE OSTI
NASA NTRS2024

Expanded Signal to Noise Ratio Estimates for Validating Next-Generation Satellite Sensors in Oceanic, Coastal, and Inland Waters

The launch of the NASA Plankton, Aerosol, Cloud, ocean Ecosystem (PACE) and the Surface Biology and Geology (SBG) satellite sensors will provide increased spectral resolution compared to existing platforms. These new sensors will require robust calibration and validation datasets, but existing field-based instrumentation is limited in its availability and potential for geographic coverage, particularly for coastal and inland waters, where optical complexity is substantially greater than in the open ocean. The minimum signal-to-noise ratio (SNR) is an important metric for assessing the reliability of derived biogeochemical products and their subsequent use as proxies, such as for biomass, in aquatic systems. The SNR can provide insight into whether legacy sensors can be used for algorithm development as well as calibration and validation activities for next-generation platforms. We extend our previous evaluation of SNR and associated uncertainties for representative coastal and inland targets to include the imaging sensors PRISM and AVIRIS-NG, the airborne-deployed C-AIR radiometers, and the shipboard HydroRad and HyperSAS radiometers, which were not included in the original analysis. Nearly all the assessed hyperspectral sensors fail to meet proposed criteria for SNR or uncertainty in remote sensing reflectance (R rs ) for some part of the spectrum, with the most common failures (>20% uncertainty) below 400 nm, but all the sensors were below the proposed 17.5% uncertainty for derived chlorophyll-a. Instrument suites for both in-water and airborne platforms that are capable of exceeding all the proposed thresholds for SNR and R rs uncertainty are commercially available. Thus, there is a straightforward path to obtaining calibration and validation data for current and next-generation sensors, but the availability of suitable high spectral resolution sensors is limited.

signal-to-noise ratio↗
DOE OSTI2024

Hydrogen and its Vital Role in a Clean Energy Future

Large-scale, low -cost hydrogen production can enable an economically competitive, secure, and environmentally beneficial future energy system across multiple sectors. Furthermore, clean hydrogen can address specific sectors that are hard to decarbonize (e.g., heavy-duty trucking, load-following electricity, iron, steel, and cement) and can help the U.S. meet the net zero carbon goal by 2050. To achieve this goal, tens of millions of metric tons of clean, reliable, and affordable hydrogen will be needed annually1. In 2021, the Hydrogen Energy Earthshot was launched, and its goal is to reduce the cost of clean hydrogen to $1 per $1 kilogram in 1 decade (1 1 1) 2. One very promising pathway for large-scale hydrogen production is water splitting. Water splitting technologies range from commercial technologies such as electrolyzers to approaches that are at a much earlier stage of development, such as photoelectrochemical (PEC) and thermochemical (TCH) processes. All these water splitting pathways offer diverse benefits in energy storage, grid services, and cross-sector emissions reductions while taking advantage of the diverse domestic resources. However, critical materials-, component- and system-level challenges must be addressed to improve efficiency and durability and reduce cost. To address these barriers and move these promising and high impact technologies forward, the HydroGEN Advanced Water Splitting Materials (AWSM) and the H2 from the Next-generation of Electrolyzers of Water (H2NEW) consortia were formed and supported by the Department of Energy (DOE) EERE Hydrogen and Fuel Cell Technologies Office (HFTO). HydroGEN (https://www.energy.gov/eere/h2awsm/) consortium, established in 2016, is an Energy Materials Network (EMN) that aims to accelerate the materials R&D of low technology readiness level (TRL) advanced water splitting (AWS) technologies. The consortium comprises five core national laboratories and focuses on four early-stage AWS pathways: alkaline exchange membrane (AEM) electrolysis, proton conducting solid oxide electrolysis (p-SOEC), photoelectrochemical, and thermochemical water splitting. Liquid alkaline and PEM electrolyzers are already commercial and significant advancements in oxygen conducting solid oxide electrolysis cells (o-SOECs) have been realized. Yet, these systems are still too expensive and not sufficiently durable for wide-scale commercialization. To enable high-volume manufacturing of affordable, durable, efficient electrolyzers, H2NEW (https://h2new.energy.gov/), another multi-lab consortium, was established in 2020. This comprehensive, concerted effort is focused on overcoming barriers related to components and materials integration and scale-up to achieve performance, durability, with an initial focus to achieve $2/kg H2 by 2026.

AEM↗
DOE OSTI2020

Method for accurately predicting solvation structure

Accurately predicting the molecular structure of solutions is a fundamental scientific challenge. Using quantum mechanical density functional theory (DFT) to make these predictions is hindered by significant variation depending on which DFT functional is used. Here, we present a simple metric that can determine the reliability of a DFT functional for predicting solvation structure. We then show that including a simple interaction term to correct this metric leads to quantitative agreement with experimental measurements of liquid structure. We demonstrate the utility of this method by using it to accurately describe the hydration structure around the Na+ a

basis sets, molecular structure, cluster chemistry↗
NASA NTRS1990

Multiversion software reliability through fault-avoidance and fault-tolerance

In this project we have proposed to investigate a number of experimental and theoretical issues associated with the practical use of multi-version software in providing dependable software through fault-avoidance and fault-elimination, as well as run-time tolerance of software faults. In the period reported here we have working on the following: We have continued collection of data on the relationships between software faults and reliability, and the coverage provided by the testing process as measured by different metrics (including data flow metrics). We continued work on software reliability estimation methods based on non-random sampling, and the relationship between software reliability and code coverage provided through testing. We have continued studying back-to-back testing as an efficient mechanism for removal of uncorrelated faults, and common-cause faults of variable span. We have also been studying back-to-back testing as a tool for improvement of the software change process, including regression testing. We continued investigating existing, and worked on formulation of new fault-tolerance models. In particular, we have partly finished evaluation of Consensus Voting in the presence of correlated failures, and are in the process of finishing evaluation of Consensus Recovery Block (CRB) under failure correlation. We find both approaches far superior to commonly employed fixed agreement number voting (usually majority voting). We have also finished a cost analysis of the CRB approach.

Vouk, Mladen A.↗
DOE OSTI2025

Q -score as a reliability measure for protein, nucleic acid and small-molecule atomic coordinate models derived from 3DEM maps

Atomic coordinate models are important for the interpretation of 3D maps produced with cryoEM and cryoET (3D electron microscopy; 3DEM). In addition to visual inspection of such maps and models, quantitative metrics can inform about the reliability of the atomic coordinates, in particular how well the model is supported by the experimentally determined 3DEM map. A recently introduced metric, Q-score, was shown to correlate well with the reported resolution of the map for well fitted models. Here, we present new statistical analyses of Q-score based on its application to ∼10 000 maps and models archived in the EMDB (Electron Microscopy Data Bank) and PDB (Protein Data Bank). Further, we introduce two new metrics based on Q-score to represent each map and model relative to all entries in the EMDB and those with similar resolution. We explore through illustrative examples of proteins, nucleic acids and small molecules how Q-scores can indicate whether the atomic coordinates are well fitted to 3DEM maps and also whether some parts of a map may be poorly resolved due to factors such as molecular flexibility, radiation damage and/or conformational heterogeneity. These examples and statistical analyses provide a basis for how Q-scores can be interpreted effectively in order to evaluate 3DEM maps and atomic coordinate models prior to publication and archiving.

B factors↗
DOE OSTI2020

Cell degradation quantification—a performance metric-based approach

A safe and reliable battery operation needs effective diagnostic tools. A quantitative failure analysis (FA) to enable cell qualification and quantify its effectiveness for reliable and safe operation of rechargeable Li batteries (RLB) is shown here. The method can identify and quantify potential failure based on the state of charge (SOC) under any operating conditions. A precise and accurate electrochemical analytic diagnosis (eCAD) of 14 rechargeable Li || NMC622 cells of the same build are used as an example. The FA by eCAD can quantitatively decipher good, bad and ugly cells in cycle aging. The cell qualification is based on thermodynamic SOC, not experimental conditions. The method provides a quantitative failure mode and effect analysis (FMEA) to reveal diverse “dead Li” formation that affects the reversibility of the Li anode and charge retention in the cell. This cell qualification method highlights the potential to improve cell quality for safe operation, with strong implications for early fault detection, FA, risk mitigation, state estimation and life prediction for reliable and safe RLB operations

25 ENERGY STORAGE↗
DOE OSTI2022

Development and testing of a performance evaluation methodology to assess the reliability of occupancy sensor systems in residential buildings

With the emergence of advanced occupancy sensor technologies to better detect occupancy in buildings, a universal methodology and metrics are required to evaluate and report sensor systems’ reliability and compare the performance across multiple sensor systems. Herein this research presents a methodology to assess the reliability of occupancy sensor systems in residential buildings in a controlled laboratory environment, including both “typical” and “failure” testing scenarios. The developed methodology was then implemented to evaluate a novel occupancy detection sensor system’s reliability. “Typical” testing evaluates the overall accuracy of the sensor system, which suggest how reliable the occupancy sensor system is over time. Results show that on average, the precision and recall are 0.75 and 0.70, indicating similar numbers of false positives and false negatives across the dataset. The overall accuracy of the tested sensor system was 62.4% to 76.4%. Failure testing results indicate whether there are influential variables impacting the sensor performance. For the tested sensor system, the number of occupants, presence of large objects, presence of interior light sources, and number of doors are not influential, while lighting level, location of occupants, additional door in the entry/exit area, and having the TV on are variables determined to impact the sensor system performance.

47 OTHER INSTRUMENTATION↗
DOE OSTI2022

Estimation of pipe failure frequencies in the absence of operational experience data: A pilot study

Probabilistic failure metrics such as leak frequency and rupture frequency are commonly used to characterize piping reliability. The methodologies for calculating the failure metrics rely on a complex set of input parameters. Operating experience data and experimental data play an important role in informing the different input parameters. The paper describes results and conclusions of a coordinated research project to benchmark three different reliability models using a four-step procedure: reference case definition of relevance to advanced reactor designs, input parameter calibration, validation of results, and application of different methodologies upon completion of the calibration and validation steps. The reference case is a weld consisting of nickel-base alloy 152/52 and located within a primary pressure boundary of an advanced reactor. This alloy is a class of structural materials known to be highly resistant to stress corrosion cracking. Synergies between the different methods are noted and the importance of a multi-disciplinary approach to input parameter development is underscored. A key conclusion is that the three methods are equally suitable for estimating failure frequencies. In any specific application, a selection of the most practical or effective computational tool can be considered. The comparison of alternative models confirms and helps to gain confidence in the computed failure frequency estimates. The study was part of a coordinated research project organized by the International Atomic Energy Agency.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
NASA NTRS2017

The Challenge Posed by Geomagnetic Activity to Electric Power Reliability: Evidence from England and Wales

This paper addresses whether geomagnetic activity challenged the reliability of the electric power system during part of the declining phase of solar cycle 23. Operations by National Grid in England and Wales are examined over the period of 11 March 2003 through 31 March 2005. This paper examines the relationship between measures of geomagnetic activity and a metric of challenged electric power reliability known as the net imbalance volume (NIV). Measured in megawatt hours, NIV represents the sum of all energy deployments initiated by the system operator to balance the electric power system. The relationship between geomagnetic activity and NIV is assessed using a multivariate econometric model. The model was estimated using half-hour settlement data over the period of 11 March 2003 through 31 December 2004. The results indicate that geomagnetic activity had a demonstrable effect on NIV over the sample period. Based on the parameter estimates, out-of-sample predictions of NIV were generated for each half hour over the period of 1 January to 31 March 2005. Consistent with the existence of a causal relationship between geomagnetic activity and the electricity market imbalance, the root-mean-square error of the out-of-sample predictions of NIV is smaller; that is, the predictions are more accurate, when the statistically significant estimated effects of geomagnetic activity are included as drivers in the predictions.

Forbes, Kevin F.↗
DOE OSTI2023

Assessing the Resource Adequacy and Community Impacts of an Evolving Grid: Considerations for State Regulators

As U.S. power systems continue to decarbonize, state regulators must be able to broadly assess the cost and reliability implications of proposed fossil retirements and the corresponding resources that are coming online to replace them. Specifically, regulators must evaluate whether proposed infrastructure investments are sufficient to ensure the long-term reliability of the system, while also protecting rate payers by ensuring that such investments are necessary and not excessive. Broadly speaking, this objective is typically framed in terms of maintaining long-term resource adequacy at the lowest cost, while also ensuring that other social and/or environmental objectives are satisfied. System planners currently have a host of sophisticated analytical tools at their disposal to evaluate the cost-benefit trade-offs of different approaches to ensuring resource adequacy. However, most of these tools and associated metrics were designed with traditional power systems in mind. Over the past decade, it has become increasingly clear that the metrics and models that have reliably assessed resource adequacy in systems dominated by large thermal and hydropower resources will be insufficient for systems with significant contributions from emerging technologies such as wind, solar and storage. Therefore, in recent years many new metrics and modeling approaches have been developed, and are increasingly being implemented, to meet the needs of the clean energy transition.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2025

Automated model generation and parameter estimation of building energy models using an ontology-based framework

This study presents a methodology for automated model generation and parameter estimation of building energy models using semantic modeling and Bayesian estimation. Semantic modeling techniques are used to represent the system components and their interactions, facilitating the automatic generation of a simulation model from dynamic component models. The proposed approach is applied to a case study of a ventilation system where a simulation model is generated, calibrated, and assessed through different performance metrics. These metrics demonstrate the accuracy and reliability of both model point estimates and probabilistic prediction intervals across all model outputs. Overall, the proposed methodology offers a systematic and automated approach to model development and calibration in building energy systems, with potential applications in building performance analysis, monitoring, and optimization.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
NASA NTRS2020

Mid Lift-to-Drag Ratio Rigid Vehicle 6-DoF Performance for Human Mars Entry, Descent, and Landing: A Fractional Polynomial Powered Descent Guidance Approach

Defining a feasible vehicle design and mission architecture capable of reliably delivering apayload of 20 metric tons (mt) or more is a great challenge for landing humans on Mars. TheMid Lift-to-Drag Rigid Vehicle (MRV), a rigid decelerator studied in NASA’s Entry, Descent,and Landing Architecture Study (EDLAS), has shown to be a viable vehicle candidate forfuture human Mars missions. As the vehicle concept matures, models of increasing fidelity areadded to the six-degree-of-freedom (6DoF) EDL simulation. This paper presents 6DoFsimulation results using model updates for vehicle mass properties, fineness ratio, andaerodynamic-propulsive interactions. Additionally, an assessment of the Fractional-Polynomial Powered Descent Guidance (FP2DG) performance is presented, and the vehicleperformance is compared with the Tunable Apollo Powered Descent Guidance (TAPDG).Finally, Monte Carlo results of the vehicle design trades are presented.

Johnson, Breanna J.↗
DOE OSTI2023

A VOI Web Application for Distinct Geothermal Domains: Statistical Evaluation of Different Data Types within the Great Basin

The Great Basin region contains different domains that have different structural and hydrothermal flow patterns. Depending on the characteristics of these patterns, certain data types may be more successful at detecting hidden geothermal resources. In this paper, we quantitatively evaluate if certain data types are more successful in certain domains. Given different aquifer, strain and structural conditions, we explore which data types statistically reveal positively labeled geothermal sites. We utilize value of information (VOI) metrics to help quantify the reliability of data types to discriminate against "positive" and "negative" labeled geothermal sites. We also evaluate how kernel density estimation can help generalize the statistics that inform VOI, which is necessary given the limited data in geothermal exploration. Except for the Carbonate Aquifer, the highest ranking of the Vimperfect is the Local Structural Setting. Next, the slip and dilation tendency is first for Carbonate Aquifer and second for Central Nevada Seismic Belt and Western Great Basin. For the Carbonate Aquifer, heat flow is has the lowest Vimperfect value compared to the other three domains, which is consistent with the understanding of how heat flow measurements are masked by regional groundwater flow.

Bayesian analysis↗
DOE OSTI2025

State Spotlight on Resilience: The Michigan Public Service Commission and Data Informed Accountability

Resilience activities are an emerging area within regulatory utility policy. As resilience frameworks develop, many Public Utility Commissions (PUCs) are looking to initiate resilience activities and have identified enhancing grid reliability as a practical starting point. Reliability describes the grid’s ability to deliver electricity steadily and without interruptions under normal or “blue sky” conditions. Reliability is measured using industry standard metrics such as outage frequency (SAIFI) and outage duration (SAIDI). Resilience is broader, reflecting the grid’s ability to both withstand and recover from disruptions, particularly those caused by extreme events such as severe storms. A reliable grid minimizes routine disruptions; a resilient grid ensures the system can recover quickly when disruptions occur, preventing them from escalating into widespread or prolonged crises.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2026

Robust Explanations using Diverse Adversarially Trained Ensembles, Multi-Modal Contrastive Learning, and Attribution-based Confidence Metrics

The primary objective of this project is to strengthen the trustworthiness of AI systems by designing algorithms that make their internal decision-making processes more understandable to human users. This involves creating clear, interpretable explanations for AI decisions and developing metrics to assess these explanations' validity and reliability. Significant progress has been achieved through (i) developing symbolic explanations, (ii) generating meaningful interpretive insights, (iii) establishing accuracy and confidence metrics, and (iv) devising methods to evaluate the knowledge boundaries of AI models. To date, the research findings have been shared in peer-reviewed publications, with accompanying scientific and technical information (STI) detailed below.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Model-driven prediction for accelerator magnet diagnostics to improve operation reliability

Reliability is one of the most critical metrics for accelerator operation, especially in user facilities. To reduce costly facility downtime and provide an operational environment where system performance can be reliably predicted in support of scientific studies, we are developing a model-driven approach for prediction and anomaly detection. Here, in this study, we present the application of a model-driven method that employs a linear regression model to predict the future temperature, in real time, of accelerator magnets at the NSLS-II light source. This approach enables proactive identification of magnet-heating issues, facilitating magnet flushing prior to the occurrence of permanent damage without interrupting machine operation. The implementation of this method in the NSLS-II control room is described and the analysis of the online results is presented. The results demonstrate the model’s effectiveness in providing early alerts to engineers and improving the reliability of accelerator operations.

36 MATERIALS SCIENCE↗
NASA NTRS1978

Numerical aerodynamic simulation facility. Preliminary study extension. Executive summary

An optimized functional design of key elements of the Numerical Aerodynamic Simulation Facility was investigated. The following tasks were performed and are discussed: (1) develop, optimize, and describe the functional description of the custom hardware; (2) delineate trade-off areas between performance, reliability, availability, serviceability, and programmability; (3) develop metrics and models for validation of the candidate system's performance; (4) conduct a functional simulation of the system design; (5) perform a reliability analysis of the system design; and (6) develop the software specifications to include a user level high level programming language, a correspondence between the programming language and instruction set, and outline the operating system requirements.

Source record↗