Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “reliability metrics”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5
All sourcesNASA NTRSDOE OSTI
DOE OSTI2026

From observation to replication: machine-learning-driven quantification and replication of fine-scale fish kinematics and behavior

Long-term quantification of fish behavior is essential for aquatic ecology, wildlife telemetry, and biomechanical device development. However, the observation duration required to obtain reliable behavioral and kinematic metrics remains unclear, and few tools exist to physically reproduce natural swimming motion for controlled experimentation. We address these challenges by developing a generalizable framework that models behavioral reliability (Spearman–Brown reliability index) as a function of observation duration and derives metric-specific monitoring thresholds. Using juvenile white sturgeon (Acipenser transmontanus) as a case study, we demonstrate that the minimum duration needed for reliable estimates varies substantially across kinematic features: to exceed a reliability of 0.8, total distance traveled requires 12 days, average curvature (mm?¹) 15 days, tail-beat frequency (Hz) 8 days, and average speed (body length/s) 17 days. We further bridge digital analysis and physical testing by developing a hardware-in-the-loop simulator that reconstructs machine-learning-derived swimming kinematics with high fidelity (correlation coefficient 0.98–0.99, RMSE 1.22–1.27 mm over a 5-minute segment). This platform enables realistic, repeatable motion stimuli for evaluating aquatic sensing technologies and bio-integrated devices under controlled conditions. Together, these contributions provide a scalable approach for designing long-term behavioral studies and a data-driven connection between ecological observation and robotic experimentation.

Hwang, SungJoo↗
NASA NTRS2000

Comprehensive Design Reliability Activities for Aerospace Propulsion Systems

This technical publication describes the methodology, model, software tool, input data, and analysis result that support aerospace design reliability studies. The focus of these activities is on propulsion systems mechanical design reliability. The goal of these activities is to support design from a reliability perspective. Paralleling performance analyses in schedule and method, this requires the proper use of metrics in a validated reliability model useful for design, sensitivity, and trade studies. Design reliability analysis in this view is one of several critical design functions. A design reliability method is detailed and two example analyses are provided-one qualitative and the other quantitative. The use of aerospace and commercial data sources for quantification is discussed and sources listed. A tool that was developed to support both types of analyses is presented. Finally, special topics discussed include the development of design criteria, issues of reliability quantification, quality control, and reliability verification.

Christenson, R. L.↗
DOE OSTI2022

Optimizing Rate of Penetration and Tripping Decision-Making using Real-Time Bit Wear Monitoring While Drilling Geothermal Wells

Understanding bit wear while drilling is critical to minimizing non-productive time (NPT) and optimizing rate of penetration (ROP). Lengthening drilling runs with damaged bits does not only lower the ROP, but also elevates the risk of inducing severe bit damage, which could potentially lead to time-consuming fishing operations. When drillers believe the bit has worn off substantially, the bit is tripped out to be replaced. On geothermal wells, tripping can take up to 20% of the overall well construction time, and this is generally acknowledged as an opportunity for improvement. Ideally, a bit run should be terminated before the bit is damaged beyond repair. At the same time, premature bit pulls are to be avoided as well. This study aims to leverage bit and tooth wear metrics that can be obtained in real time to characterize bit condition in order to optimize ROP and determine the optimal time to pull the bit. Two metrics were explored in this study: a bit wear metric that incorporated depth-of-cut, and a tooth wear metric developed by Bourgoyne & Young characterizing the state of bit teeth dull. Both metrics were computed using recorded data from 12¼ inches roller cone insert bit runs in five geothermal wells targeting a granodiorite formation in the western United States. Together with the actual dull grades, determined after the bits were pulled to surface, the metric trends were interpreted to characterize the downhole bit condition and identify the point at which the bit should have optimally been tripped out. The insights from studying the actual dull grades and how they relate to the two metrics were used to establish a reliable bit pull criterion. The bit wear metric trend correctly showed a noticeable departure from baseline for bits experiencing major dulling behavior. Additionally, the tooth wear model predicted the cutter dull within two dull grades for most runs, with better performance in predicting the inner teeth dull. Moreover, the combination of the bit wear and tooth wear metrics was effective in revealing the cause of the bit performance impairment. Proactive tracking of these two metrics in real-time can facilitate geothermal drilling ROP optimization and better-informed tripping decision-making, thereby avoiding wasted time and cost.

Ashari, Rahmat↗
DOE OSTI2026

New framework for benchmarking decadal predictions leveraging the PCMDI Metric Package with interactive visualization

Reliable climate predictions across multiple timescales are increasingly critical as climate-related risks continue to rise. With the growing number and diversity of climate prediction systems, systematic intercomparison has become essential. Here, we present a comprehensive evaluation framework based on the PCMDI Metric Package to assess the performance of multiple decadal climate prediction systems. Unlike uninitialized simulations, initialized predictions exhibit bias and predictive skill that evolve with forecast lead time. To address this, we introduce (1) model-by-lead-time portrait plots, which efficiently summarize metrics of global temperature, precipitation, and Arctic/Antarctic sea-ice extent, and (2) an HTML-based interactive visualization platform that provides detailed regional and seasonal diagnostics of model bias, skill scores, and ensemble spread for each model and lead time. Comparisons with uninitialized simulations further quantify the relative impacts of initialization and external forcing on prediction skill. The proposed framework provides a scalable and transparent approach for multi-model climate prediction assessments and can be readily extended to a wide range of operational and research forecasting systems.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS2026

Reliability Models and Demonstration of a Fault-Tolerant Motor Concept for Vertical Takeoff and Landing Vehicles

This report documents the completion of the Revolutionary Vertical Lift Technology Project Annual Performance Indicator 24-3.2.4.1: “Apply and document reliability prediction for high reliability motor concept.” Two modeling tools were completed for calculation of reliability of fault-tolerant (FT) motors, and key FT operations of a modular FT motor were demonstrated experimentally. The two models are complementary tools for the stakeholder and user community. Both models employ Markov chain theory. The first model is a time-homogeneous Markov chain model, and the second is a time-inhomogeneous Markov-Weibull model. This report’s main sections are as follows: 1.0 Introduction, 2.0 Theory, 3.0 Motor Reliability Models, 4.0 Validation of FT Operation by Hardware Demonstration, and 5.0 Concluding Remarks. Novel contributions to the field include development of a modular FT motor concept for electrified vertical takeoff and landing (eVTOL) application, solution methods to solve the reliability calculations, development of figures of merit, and the introduction of “linked chains” to formulate a building-block approach for time-inhomogeneous Markov-Weibull modeling of motor reliability. Example case studies have been completed, and results are provided and discussed herein. A four-module FT motor concept was developed to a preliminary-design level of detail. This eVTOL FT motor concept was designed for galvanic, magnetic, and thermal isolation of stator winding faults. The reliability of the concept motor was calculated using a time-inhomogeneous Markov chain model. Employing average failure rate as a metric, 570 times greater reliability was achieved as compared to a baseline motor without fault tolerance. A demonstrator motor was built and tested. The testing demonstrated the key features of FT operation and validated the essential premises of the FT motor concepts presented herein. The experiments included successful demonstration of the feasibility of the following four key FT features: (1) terminal open-circuit operation, (2) thermal isolation after fault, (3) terminal short-circuit operation, and (4) internal short-circuit operation. These works indicate that FT modular motor drives offer promise for addressing the daunting reliability gap that electric aircraft propulsor drives are facing relative to the best conventional motor drive technology that is available today.

Electric Motor↗
NASA NTRS2020

A Demonstration that Correcting for Completeness and Reliability Is Critical for Robust Occurrence Rates

A measurement of planetary occurrence rates based on a planet catalog should be robust against details of how initial detections were classified as planets or false positives. This is accomplished by supplying the catalog’s rate of missed planets (completeness) and rate of non-planets incorrectly called planets (reliability). The final Kepler data release (DR25) includes products that can be used with the DR25 planet candidate catalog to correct for completeness and reliability in occurrence rate estimates. This is made possible by the Kepler Robovetter, which algorithmically and uniformly selects planets based on a variety of metrics and thresholds. Completeness, reliability, and occurrence rates potentially depend on these Robovetter thresholds. We study the impact of varying these vetting thresholds using the techniques of Bryson et al. 2019 (arXiv:1906.03575). We explore sets of thresholds that result in more or fewer planets (trading off completeness for reliability), as well as thresholds tuned to pass DR25 false positives identified as possible planets by the Kepler False Positive Working Group. We find that when correcting only for completeness, and not reliability, the resulting occurrence rates have a strong dependence on these threshold sets. For example, the value of SAG13 eta-Earth varies by over a factor of 4 when not corrected for reliability. However, when correcting for both completeness and reliability, occurrence rates using our threshold sets are statistically indistinguishable, with differences being well inside 1-sigma error bars. We present occurrence rates integrated over several period-radius ranges. For example, SAG13 eta-Earth is consistent with 0.127 (+0.094)(-0.054) (from Bryson et al. 2019) for all the Robovetter threshold sets. This result emphasizes the importance of correcting occurrence rates for both completeness and reliability. This suggests that inconsistent completeness and reliability correction may be a significant contributor to the large variation of occurrence rates in recent literature. We plan to make the Robovetter results for our threshold sets available, and encourage the community to use them to examine whether other occurrence rate methods yield similarly robust results.

Bryson, S.↗
NASA NTRS1989

High rate concatenated coding systems using bandwidth efficient trellis inner codes

High-rate concatenated coding systems with bandwidth-efficient trellis inner codes and Reed-Solomon (RS) outer codes are investigated for application in high-speed satellite communication systems. Two concatenated coding schemes are proposed. In one the inner code is decoded with soft-decision Viterbi decoding, and the outer RS code performs error-correction-only decoding (decoding without side information). In the other, the inner code is decoded with a modified Viterbi algorithm, which produces reliability information along with the decoded output. In this algorithm, path metrics are used to estimate the entire information sequence, whereas branch metrics are used to provide reliability information on the decoded sequence. This information is used to erase unreliable bits in the decoded output. An errors-and-erasures RS decoder is then used for the outer code. The two schemes have been proposed for high-speed data communication on NASA satellite channels. The rates considered are at least double those used in current NASA systems, and the results indicate that high system reliability can still be achieved.

Deng, Robert H.↗
DOE OSTI2022

PyPVRPM: Photovoltaic Reliability and Performance Model in Python

The ability to perform accurate techno-economic analysis of solar photovoltaic (PV) systems is essential for bankability and investment purposes. Most energy yield models assume an almost flawless operation (i.e., no failures); however, realistically, components fail and get repaired stochastically. This package, PyPVRPM, is a Python translation and improvement of the Language Kit (LK) based PhotoVoltaic Reliability Performance Model (PVRPM), which was first developed at Sandia National Laboratories in Goldsim software (Granata et al., 2011) (Miller et al., 2012). PyPVRPM allows the user to define a PV system at a specific location and incorporate failure, repair, and detection rates and distributions to calculate energy yield and other financial metrics such as the levelized cost of energy and net present value (Klise, Lavrova, et al., 2017). Our package is a simulation tool that uses NREL’s Python interface for System Advisor Model (SAM) (National Renewable Energy Laboratory, 2020b) (National Renewable Energy Laboratory, 2020a) to evaluate the performance of a PV plant throughout its lifetime by considering component reliability metrics. Besides the numerous benefits from migrating to Python (e.g., speed, libraries, batch analyses), it also expands on the failure and repair processes from the LK version by including the ability to vary monitoring strategies. These failures, repairs, and monitoring processes are based on user-defined distributions and values, enabling a more accurate and realistic representation of cost and availability throughout a PV system’s lifetime.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Detecting defects that reduce breakdown voltage using machine learning and optical profilometry

Abstract Semiconductor wafer manufacturing relies on the precise control of various performance metrics to ensure the quality and reliability of integrated circuits. In particular, GaN has properties that are advantageous for high voltage and high frequency power devices; however, defects in the substrate growth and manufacturing are preventing vertical devices from performing optimally. This paper explores the application of machine learning techniques utilizing data obtained from optical profilometry as input variables to predict the probability of a wafer meeting performance metrics, specifically the breakdown voltage (V bk ). By incorporating machine learning techniques, it is possible to reliably predict performance metrics that cause devices to fail at low voltage. For diodes that fail at a higher (but still below theoretical) breakdown voltage, alternative inspection methods or a combination of several experimental techniques may be necessary.

42 ENGINEERING↗
NASA NTRS2006

An Integrated Approach to Life Cycle Analysis

Life Cycle Analysis (LCA) is the evaluation of the impacts that design decisions have on a system and provides a framework for identifying and evaluating design benefits and burdens associated with the life cycles of space transportation systems from a "cradle-to-grave" approach. Sometimes called life cycle assessment, life cycle approach, or "cradle to grave analysis", it represents a rapidly emerging family of tools and techniques designed to be a decision support methodology and aid in the development of sustainable systems. The implementation of a Life Cycle Analysis can vary and may take many forms; from global system-level uncertainty-centered analysis to the assessment of individualized discriminatory metrics. This paper will focus on a proven LCA methodology developed by the Systems Analysis and Concepts Directorate (SACD) at NASA Langley Research Center to quantify and assess key LCA discriminatory metrics, in particular affordability, reliability, maintainability, and operability. This paper will address issues inherent in Life Cycle Analysis including direct impacts, such as system development cost and crew safety, as well as indirect impacts, which often take the form of coupled metrics (i.e., the cost of system unreliability). Since LCA deals with the analysis of space vehicle system conceptual designs, it is imperative to stress that the goal of LCA is not to arrive at the answer but, rather, to provide important inputs to a broader strategic planning process, allowing the managers to make risk-informed decisions, and increase the likelihood of meeting mission success criteria.

Chytka, T. M.↗
NASA NTRS2010

Eye Tracking Metrics for Workload Estimation in Flight Deck Operation

Flight decks of the future are being enhanced through improved avionics that adapt to both aircraft and operator state. Eye tracking allows for non-invasive analysis of pilot eye movements, from which a set of metrics can be derived to effectively and reliably characterize workload. This research identifies eye tracking metrics that correlate to aircraft automation conditions, and identifies the correlation of pilot workload to the same automation conditions. Saccade length was used as an indirect index of pilot workload: Pilots in the fully automated condition were observed to have on average, larger saccadic movements in contrast to the guidance and manual flight conditions. The data set itself also provides a general model of human eye movement behavior and so ostensibly visual attention distribution in the cockpit for approach to land tasks with various levels of automation, by means of the same metrics used for workload algorithm development.

Ellis, Kyle↗
NASA NTRS2009

Shuttle Wing Leading Edge Root Cause NDE Team Findings and Implementation of Quantitative Flash Infrared Thermography

Comparison metrics can be established to reliably and repeatedly establish the health of the joggle region of the Orbiter Wing Leading Edge reinforced carbon carbon (RCC) panels. Using these metrics can greatly reduced the man hours needed to perform, wing leading edge scanning for service induced damage. These time savings have allowed for more thorough inspections to be preformed in the necessary areas with out affecting orbiter flow schedule. Using specialized local inspections allows for a larger margin of safety by allowing for more complete characterizations of panel defects. The presence of the t-seal during thermographic inspection can have adverse masking affects on ability properly characterize defects that exist in the joggle region of the RCC panels. This masking affect dictates the final specialized inspection should be preformed with the t-seal removed. Removal of the t-seal and use of the higher magnification optics has lead to the most effective and repeatable inspection method for characterizing and tracking defects in the wing leading edge. Through this study some inadequacies in the main health monitoring system for the orbiter wing leading edge have been identified and corrected. The use of metrics and local specialized inspection have lead to a greatly increased reliability and repeatable inspection of the shuttle wing leading edge.

Burke, Eric R.↗
NASA NTRS2003

Developing Fault Models for Space Mission Software

A viewgraph presentation on the development of fault models for space mission software is shown. The topics include: 1) Goal: Improve Understanding of Technology Fault Generation Process; 2) Required Measurement; 3) Measuring Structural Evolution; 4) Module Attributes; 5) Principal Components of Raw Metrics; 6) The Measurement Process; 7) View of Structural Evolution at the System and Module Level; 8) Identifying and Counting Faults; 9) Fault Enumeration; 10) Modeling Fault Content; 11) Modeling Results; 12) Current and Future Work; and 13) Discussion and Conclusions.

software measurements↗
DOE OSTI2025

Logarithmic Resilience Risk Metrics That Address the Huge Variations in Blackout Cost

Resilience risk metrics must address the customer cost of the largest blackouts of greatest impact. However, there are huge variations in blackout cost in observed distribution utility data that make it impractical to properly estimate the mean large blackout cost and the corresponding risk. These problems are caused by the heavy tail observed in the distribution of customer costs. To solve these problems, we propose resilience metrics that describe large blackout risk using the mean of the logarithm of the cost of large-cost blackouts, the slope index of the heavy tail, and the frequency of large-cost blackouts.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2023

Resilience Analytics for Interdependent Power and Water Distribution Systems

The occurrence of large-scale outages in power distribution systems (PDSs) caused, for instance, by extreme weather events has raised concerns on the cascading impact of the power outages on the resilience of other critical infrastructures, including water distribution systems (WDSs), whose healthy operation highly relies on the availability of power from PDSs. Here, this paper proposes an analytics model for quantifying the interdependence between the resilience of power and water distribution systems. The proposed model first performs spatio-temporal outage analysis on the PDS and WDS to determine the PDS power serving capability and the resulting cascading impact on the WDS water serving capability during the occurrence of disruptions. The resilience interdependence is then determined by a proposed set of six metrics calculated using the power and water serving capability profiles, which trace and quantify the cascading impacts of power outages in the operation of WDSs during the degradation and recovery stages. The proposed analytics model is implemented on the IEEE 33-bus power distribution system supplying power for pumping stations of a 15-node test WDS, considering multiple cases of water storage capacity, single and multiple outage scenarios, and availability of distributed generation. The numerical results demonstrate that the proposed metrics track the temporal and spatial interdependence between the resilience of power and water distribution systems, providing a mechanism to identify weak spots, and assess options to enhance the resilience of the critical water infrastructure.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2026

Nuclear Data Impact on Key Metrics for a Representative Molten Chloride Fast Reactor Model

Nuclear data are an essential component of the foundation on which all modeling and simulation methods and tools are relying upon, from the front end to the back end of the nuclear fuel cycle. In this study, the impact of uncertainties in nuclear data is investigated for a representative molten chloride fast reactor, for several important metrics, including eigenvalue, reactivity differences, and nuclide inventories in fuel at 5-yr irradiation. Uncertainty of keff for a full core model was found to be similar between the fresh fuel and the irradiated fuel states (1.7-1.8%), with its primary driver being the uncertainty in the 235U (n,γ) cross section. The results obtained for the reactivity differences show large uncertainties, of over 100%, in elastic scattering sensitivities of several nuclides, which led to large uncertainties of temperature reactivity differences for cladding and reflector. These results provide evidence that the currently applied methods may not be sufficiently adequate for ensuring the reliable determination of such metrics.

Procop, Germina [ORNL] (ORCID:0000000342226393)↗
NASA NTRS1990

Development of microcomputer-based mental acuity tests for repeated-measures studies

The purpose of this report is to detail the development of the Automated Performance Test System (APTS), a computer battery of mental acuity tests that can be used to assess human performance in the presence of toxic elements and environmental stressors. There were four objectives in the development of APTS. First, the technical requirements for developing APTS followed the tenets of the classical theory of mental tests which requires that tests meet set criteria like stability and reliability (the lack of which constitutes insensitivity). To be employed in the study of the exotic conditions of protracted space flight, a battery with multiple parallel forms is required. The second criteria was for the battery to have factorial multidimensionality and the third was for the battery to be sensitive to factors known to compromise performance. A fourth objective was for the tests to converge on the abilities entailed in mission specialist tasks. A series of studies is reported in which candidate APTS tests were subjected to an examination of their psychometric properties for repeated-measures testing. From this work, tests were selected that possessed the requisite metric properties of stability, reliability, and factor richness. In addition, studies are reported which demonstrate the predictive validity of the tests to holistic measures of intelligence.

Kennedy, R. S.↗