SEARCH · Engineering Papers

Results for “reliability metrics”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 217 records · Page 12

All sources NASA NTRS DOE OSTI

NASA NTRS2001

Space Transportation Operations: Assessment of Methodologies and Models

The systems design process for future space transportation involves understanding multiple variables and their effect on lifecycle metrics. Variables such as technology readiness or potential environmental impact are qualitative, while variables such as reliability, operations costs or flight rates are quantitative. In deciding what new design concepts to fund, NASA needs a methodology that would assess the sum total of all relevant qualitative and quantitative lifecycle metrics resulting from each proposed concept. The objective of this research was to review the state of operations assessment methodologies and models used to evaluate proposed space transportation systems and to develop recommendations for improving them. It was found that, compared to the models available from other sources, the operations assessment methodology recently developed at Kennedy Space Center has the potential to produce a decision support tool that will serve as the industry standard. Towards that goal, a number of areas of improvement in the Kennedy Space Center's methodology are identified.

Joglekar, Prafulla↗

NASA NTRS2002

Space Transportation Operations: Assessment of Methodologies and Models

Joglekar, Prafulla↗

NASA NTRS2008

Stability Metrics for Simulation and Flight-Software Assessment and Monitoring of Adaptive Control Assist Compensators

Due to a need for improved reliability and performance in aerospace systems, there is increased interest in the use of adaptive control or other nonlinear, time-varying control designs in aerospace vehicles. While such techniques are built on Lyapunov stability theory, they lack an accompanying set of metrics for the assessment of stability margins such as the classical gain and phase margins used in linear time-invariant systems. Such metrics must both be physically meaningful and permit the user to draw conclusions in a straightforward fashion. We present in this paper a roadmap to the development of metrics appropriate to nonlinear, time-varying systems. We also present two case studies in which frozen-time gain and phase margins incorrectly predict stability or instability. We then present a multi-resolution analysis approach that permits on-line real-time stability assessment of nonlinear systems.

Hodel, A. S.↗

DOE OSTI2022

Application of Margin-Based Methods to Assess System Health

Health management of complex systems such as nuclear power plants is an essential task to guarantee system reliability. This task can be greatly enhanced by constantly monitoring asset status and performances and process such data (through anomaly detection, diagnostic, and prognostic computational algorithms) to identify asset degradation trends and faulty states. While such information and data are typically available for many of the assets, they are not propagated from the asset to the system level in order to identify the most critical assets and prioritize maintenance and surveillance activities. The main reason is driven by the fact that current reliability modeling techniques are inadequate to process such information/data. This is due to the nature of these techniques which are based on the concept of failure rate/probability that do not serve an operational context where quantitative asset health information is available. Simply stated, current reliability techniques serve a run-to-failure operational setting and not a predictive maintenance one where the goal is to perform maintenance and surveillance activities only when they are needed based on asset health. The risk informed asset management (RIAM) project is focusing on the development of a different kind of reliability modeling techniques designed to adequately serve a predictive operational setting. Such reliability techniques move aways from a failure rate/probability to a margin-based mindset where margin is here used as a metric to quantify asset health based only on current and past operational experience of the asset under consideration. In addition, margin-based reliability techniques are able to propagate asset health information from the component to system level and provide importance measure to each asset. This report summarizes a recent activity performed in collaboration with plant modernization pathway designed to integrate monitoring data into margin-based reliability models. Such activity focuses on a specific system of an existing nuclear power plant where large amount of historic monitoring data is used to monitor asset and system health.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2025

Real-Time Lifetime Prediction of Semiconductor Devices Using Hardware-in-the-Loop

This paper presents a unique approach to enable real-time lifespan prediction of semiconductor power modules using a Hardware-in-the-Loop (HIL) system. By integrating the module's overall loss characteristics-specifically switching and conduction losses-with a thermoelectric model of the thermal management system, this research demonstrates that the model can dynamically estimates the junction temperature profile of the semiconductor devices in response to a changing torque demand profile for the motor drive system. This capability enables continuous monitoring of the module's operational time and cumulative stress induced on the devices to compute accumulated remaining lifetime or time-to-failure (TTF). This study provides an architectural framework for the HIL system with high-fidelity component models of multiple physical domains, allowing simulation of dynamic behaviors of a closely-coupled motor drive system. The advanced real-time computation and measurement functionalities of the HIL system allow for both dynamic lifetime calculations based on simulated data and aggregate lifetime predictions utilizing historical data. Moreover, this paper details an algorithm that not only computes cumulative damage but also synthesizes these data into a comprehensive aggregated lifetime metric. This methodology can enhance the maintenance scheduling strategies and operational reliability of semiconductor devices in critical applications, ultimately extending their service life while optimizing performance.

hardware-in-the-loop (HIL)↗

DOE OSTI2022

Valuing Resilience for Microgrids: Challenges, Innovative Approaches, and State Needs

The United States depends on the delivery of reliable, affordable, clean, and safe electricity. Electric utilities invest billions of dollars each year in generation, transmission, and distribution assets to meet this need. However, experiences with recent natural disasters of increasing frequency and duration demonstrate the shortcomings of this approach in the face of modern threats. Further, as customers rely on electricity for a broader range of important needs, such as transportation, as well as critical life-saving services and mission critical facilities such as water treatment, medical care, shelters, telecommunications, and more, the need to minimize the likelihood and impacts of outages grows. Against this backdrop, resilience has emerged as a key consideration to guide electricity spending, whether from utilities, customers, or taxpayers. Although reliability has been defined and measured for decades with broadly accepted metrics that measure how many customers lose power and at what frequency and duration, resilience considers the electricity system’s response to a disruption and its subsequent impacts on customers. Developing tools and methods to accurately assess the costs and benefits of resilience investments is a critical step toward the goal of mitigating the impacts of outages on customers and society. Today, electric system resilience is largely treated as an externality due to challenges estimating the costs of long-duration outages, impacts of outages on society, and increasing reliance on electricity for a growing set of interdependent services. These interdependencies include the water, wastewater, telecommunications, natural gas, and health sectors. Without knowing how much a given resilience investment will benefit customers or society more broadly, investors, policymakers, and regulators are less likely to make or approve such investments, and less able to prioritize those investments. State Energy Offices and public utility commissions (PUCs) lead the development of state-level energy policy and utility regulation, respectively, and each has interests in encouraging appropriate public and private investments in resilience. To this end, the National Association of State Energy Officials (NASEO) and National Association of Regulatory Utility Commissioners (NARUC), with the support of the U.S. Department of Energy (DOE) Office of Electricity (OE), formed a joint Microgrids State Working Group to explore the costs and benefits of microgrids, barriers to broader deployment of microgrids to meet resilience and other objectives, and policy and regulatory strategies to optimize investments in resilience, including but not limited to microgrids. Although no universally accepted valuation tool for resilience exists, National Laboratories, utilities, researchers, and state and federal agencies have collaborated to develop, apply, and improve a number of approaches to quantify resilience, several of which are still in progress at the time this report is published. This report seeks to share these important advances by discussing current definitions of resilience (Section 1), how microgrids are defined and used to meet resilience objectives (Section 2), new approaches to valuing resilience (Section 3), steps State Energy Offices and PUCs have taken to further resilience valuation efforts (Section 4), and finally, considerations and suggested next steps for State Energy Offices and PUCs (Section 5). Relevant examples of specific microgrid projects and resilience valuation efforts are included throughout the report. While this report is written specifically for NASEO and NARUC members, it may be useful for utilities, local governments, and individual customers interested in improving the way public and private dollars are spent to achieve resilience outcomes.

24 POWER TRANSMISSION AND DISTRIBUTION↗

DOE OSTI2024

Comparison of ML-Based Proxy Modeling Strategies: Lessons Learned from the SMART Initiative

Teams of researchers on Task 5 of the SMART project have developed a variety of modeling architectures to predict subsurface behavior during carbon injection and post-injection periods. One important part of this task was to compare the candidate approaches in terms of accuracy, reliability, speed, and memory use, all using a common set of metrics and visualizations for an “apples to apples” comparison. Dr. Jared Schuetter will share the details of this task, the results that were obtained, and the lessons learned.

Schuetter, Jared↗

NASA NTRS2024

Evaluation of Machine Learning and Deep Learning Algorithms for Fire Prediction in Southeast Asia

Vegetation fires are prevalent in South/Southeast Asian countries, making fire prediction crucial due to their potential environmental, economic, and social impacts. Accurate predictions of fires facilitate timely interventions, helping to mitigate uncontrolled fires that can lead to biodiversity loss and air quality issues. In this study, we utilize VIIRS satellite-derived fire data alongside six machine learning and deep learning models—Simple Persistence, Multi-Layer Perceptron (MLP), Convolutional Neural Network (CNN), Long Short-Term Memory (LSTM), CNN-LSTM, and ConvLSTM—to determine the most effective fire prediction model, using Root Mean Square Error (RMSE) as the metric. Our results indicate that the CNN model is the most reliable in regions with spatial dependencies, such as Brunei, Indonesia, Malaysia, the Philippines, Timor-Leste, and Thailand. Conversely, the ConvLSTM model excels in countries with complex spatiotemporal dynamics like Laos, Myanmar, and Vietnam. The CNN-LSTM hybrid model also performed well in Cambodia, suggesting a need for a balanced approach in areas requiring both spatial and temporal feature extraction. Furthermore, simpler models like Persistence and MLP showed limitations in capturing dynamic patterns and temporal dependencies. Our findings highlight the importance of evaluating models before implementing any decision support systems (DSS) in fire management. By tailoring models to specific regional fire data, we can enhance prediction accuracy and responsiveness, ultimately improving fire risk management in Southeast Asia and beyond.

Deep learning↗

DOE OSTI2026

SimLBR: Learning to Detect Fake Images by Learning to Detect Real Images

The rapid advancement of generative models has made the detection of AI-generated images a critical challenge for both research and society. Recent works have shown that most state-of-the-art fake image detection methods overfit to their training data and catastrophically fail when evaluated on curated hard test sets with strong distribution shifts. In this work, we argue that it is more principled to learn a tight decision boundary around the real image distribution and treat the fake category as a sink class. To this end, we propose SimLBR, a simple and efficient framework for fake image detection with Latent Blending Regularization (LBR). Our method significantly improves cross-generator generalization, achieving up to +24.85% accuracy and +69.62% recall on the challenging Chameleon benchmark. SimLBR is also highly efficient, training orders of magnitude faster than existing approaches. Furthermore, we emphasize the need for reliability-oriented evaluation in fake image detection, introducing risk-adjusted metrics and worst-case estimates to better assess model robustness. All the code and models are availabe at: https://github.com/mvrl/SimLBR

Dhakal, Aayush [Washington University, St. Louis]↗

DOE OSTI2020

Assessing the sensitivity and repeatability of permanganate oxidizable carbon as a soil health metric: An interlab comparison across soils

Soil organic matter is central to the soil health framework. Therefore, reliable indicators of Soil organic matter is central to the soil health framework. Therefore, reliable indicators of changes in soil organic matter are essential to inform land management decisions. Permanganate oxidizable carbon (PDXC), an emerging soil health indicator, has shown promise for being sensitive to soil management. However, strict standardization is required for widespread implementation in research and commercial contexts. Here, we used 36 soils-three from each of the 12 USDA soil orders-to determine the effects of sieve size and soil mass of analysis on PDXC results. Using replicated measurements across 12 labs in the US and the EU (n = 7951 samples), we quantified the relative importance of 1) variation between labs, 2) variation within labs, 3) effect soil mass, and 4) effect of soil sieve size on the repeatability of PDXC. We found a wide range of overall variability in PDXC values across labs (0.03 to 171.8%; mean = 13.4%), and much of this variability was attributable to within-lab variation (median = 6.5%) independently of soil mass or sieve size. Greater soil mass (2.5 g) decreased absolute PDXC values by a mean of 177 mg kg -1 soil and decreased analytical variability by 6.5%. For soils with organic carbon (SOC) >10%, greater soil mass (2.5 g) resulted in more frequent PDXC values above the limit of detection whereas the lower soil mass (0.75 g) resulted in PDXC values below the limit of detection for SOC contents <5%. A finer sieve size increased absolute values of PDXC by 124 mg kg -1 while decreasing the analytical variability by 1.8%. In general, soils with greater SOC contents had lower analytical variability. These results point to potential standardizations of the PDXC protocol that can decrease the variability of the metric. We recommend that the PDXC protocol be standardized to use 2.5 g for soils <10% SOC. Sieve size was a relatively small contributor to analytical variability and therefore we recommend that this decision be tailored to the study purpose. Tradeoffs associated with these standardizations can be mitigated, ultimately providing guidance on how to standardize PDXC for routine analysis.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2026

A call to standardize metrics for monitoring baleen whales near marine construction activities

Effective monitoring is necessary to protect marine mammal species during the construction of offshore infrastructure. The tools for detecting or monitoring marine mammals span traditional (e.g., visual observers, optical cameras), to newer (e.g., passive acoustic monitoring, infrared cameras, tags), and emerging (e.g., satellite imagery, environmental DNA, dimethyl sulfide concentration) technologies. Some are better suited for use during offshore development; however, peer-reviewed literature does not typically evaluate and report on the performance of these various technologies. We define a minimum set of metrics related to efficacy (i.e., confusion matrix, precision and recall, probability of missed mitigation), detection range (i.e., maximum and reliable detection range, spatial resolution), and data delivery (i.e., detection latency, system reliability, temporal resolution) that we recommend are needed to assess the utility of monitoring technologies for this purpose. Following a literature review of relevant studies, we highlight which publications reported these metrics and used multiple technologies to compare relative performance. We also emphasize the benefits of multi-modal approaches and recommend performance assessments through modeling or large-scale collaborative field testing. These metrics will standardize data collection, reporting, and analysis; promote consistent and comparable results; and foster collaboration among developers, regulatory agencies, and scientists. This may lead to the co-development of technology that achieves multiple goals, has greater application, and can answer research questions while collecting data to fulfill permitting requirements. These metrics may also inform decisions on what systems regulatory agencies might consider using and reduce monitoring costs, which is critical to support the marine sector's rapid growth alongside marine mammal conservation.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

DOE OSTI2022

Trends in surface equivalent potential temperature: A more comprehensive metric for global warming and weather extremes

Significance The Earth has warmed by 1.2 ± 0.1 °C since the preindustrial era. The most common metric to measure the ongoing global warming is surface air temperature since it has long and reliable observational records. However, surface air temperature alone does not fully describe the nature of global warming and its impact on climate and weather extremes. Here we show that surface equivalent potential temperature, which combines the surface air temperature and humidity, is a more comprehensive metric not only for the global warming but also for its impact on climate and weather extremes including tropical deep convection and extreme heat waves. We recommend that it should be used more widely in future climate change studies.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2026

Avoidance of disruptions on KSTAR due to vertical displacement events via novel real-time stability assessment

Disruption avoidance via the DECAF approach has been achieved on KSTAR using a novel real-time vertical stability assessment and a multiactuator feedback control strategy. The development of disruption avoidance strategies with reactor-relevant reliability is an urgent activity, enabling future fusion power plants. The stability metric employed is based on a new formulation of a vertical force gradient balance metric evaluated across the poloidal cross section of the plasma, with parameters tuned using historical data. Evaluation of this metric on a validation set of 400 recent KSTAR shots indicates >82% of Vertical displacement events can be avoided via feedback control. Essential to its calculation is the two-dimensional toroidal current density distribution in the plasma. Measurement of this profile faster than fully-converged equilibrium reconstructions can deliver is found to improve forecaster performance and is achieved with a surrogate model that takes as input magnetic diagnostic measurements and outputs the current profile on a basis comprising the top principal components of historical current profiles (from past equilibrium reconstructions). This method solves the non-uniqueness problem typically faced when reconstructing current profiles directly from diagnostics, while improving computational time and accuracy. On average, profiles produced by this model reach coefficients of determination of >0.99 with respect to those from equilibrium reconstructions. The avoidance actuators employed include poloidal field coils and an electron cyclotron current drive system. The multiactuator approach, as shown in this first demonstration, allows disruption avoidance while minimizing impact to operational performance. This ability, along with its flexibility and speed, makes this new approach an attractive option for avoiding these types of disruptions in reactors.

Tobin, Matthew [Columbia Univ., New York, NY (Unit↗

DOE OSTI2025

Integrated Metrics for County-Level Resilience Ranking Using Entropy and TOPSIS

In the face of atypical weather events, power infrastructure failures, and limited resources for resilience investment, energy decision-makers need data-driven metrics to allocate resilience investments and maximize the reduction of power outage impacts. For state-level planning, for instance, ranking the resilience of each county is key to ensuring effective distribution of resources. In such cases, resilience for each spatial unit is multifaceted and is captured by a set of indicators (i.e., metrics) that can be combined into an overall score that reduces the complexity of power outage dynamics to a single decision metric. However, weighting of these indicators is often addressed by simplifying assumptions (i.e., equal weights) or semi-subjective methods that rely on user-defined weights that can introduce biases (e.g., weighted average score). Within the disaster risk reduction and resilience engineering community, a recurring challenge in multicriteria decision-making is the objective weighting of indicators for composite indices. To address this issue, we have leveraged a Technique for Order of Preference by Similarity to Ideal Solution (TOPSIS) combined with an entropy-based weighting approach to calculated the integrated scores. This method objectively determines the importance of each metric, better discerns between spatial units (i.e., counties), and offers a more reliable ranking of counties according to their relative resilience attributes. By improving methods for integrating resilience indicators, our approach helps planners and decision-makers prioritize resources more effectively for more efficient resilience investments.

Bhusal, Narayan [Oak Ridge National Laboratory (OR↗

DOE OSTI2025

Developments to the Distributed Holdup Monitoring System in Fiscal Year 2025

Permanently installed holdup monitoring would provide a myriad of benefits to nuclear facilities, ranging among decreased facility burden, increased safety bases, and increased accuracy of relevant material controls and accountancy metrics. However, the cost per detection system must be affordable to provide reliable coverage. For this project, the target unit price per system has been $\$$1,000. Recent work has focused on pushing costs lower through developing a plastic scintillator and silicon photomultiplier (SiPM)–based front end, which are components that are not commercially available in conjunction with each other. Other work has included further system development and use of a low-cost SiPM. Finally, a limited deployment at a processing facility was achieved. Results from this deployment and the development process are discussed in detail.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

NASA NTRS1997

Computational Modeling as a Design Tool in Microelectronics Manufacturing

Plans to introduce pilot lines or fabs for 300 mm processing are in progress. The IC technology is simultaneously moving towards 0.25/0.18 micron. The convergence of these two trends places unprecedented stringent demands on processes and equipments. More than ever, computational modeling is called upon to play a complementary role in equipment and process design. The pace in hardware/process development needs a matching pace in software development: an aggressive move towards developing "virtual reactors" is desirable and essential to reduce design cycle and costs. This goal has three elements: reactor scale model, feature level model, and database of physical/chemical properties. With these elements coupled, the complete model should function as a design aid in a CAD environment. This talk would aim at the description of various elements. At the reactor level, continuum, DSMC(or particle) and hybrid models will be discussed and compared using examples of plasma and thermal process simulations. In microtopography evolution, approaches such as level set methods compete with conventional geometric models. Regardless of the approach, the reliance on empricism is to be eliminated through coupling to reactor model and computational surface science. This coupling poses challenging issues of orders of magnitude variation in length and time scales. Finally, database development has fallen behind; current situation is rapidly aggravated by the ever newer chemistries emerging to meet process metrics. The virtual reactor would be a useless concept without an accompanying reliable database that consists of: thermal reaction pathways and rate constants, electron-molecule cross sections, thermochemical properties, transport properties, and finally, surface data on the interaction of radicals, atoms and ions with various surfaces. Large scale computational chemistry efforts are critical as experiments alone cannot meet database needs due to the difficulties associated with such controlled experiments and costs.

Meyyappan, Meyya↗

NASA NTRS2007

Robust Design Optimization via Failure Domain Bounding

This paper extends and applies the strategies recently developed by the authors for handling constraints under uncertainty to robust design optimization. For the scope of this paper, robust optimization is a methodology aimed at problems for which some parameters are uncertain and are only known to belong to some uncertainty set. This set can be described by either a deterministic or a probabilistic model. In the methodology developed herein, optimization-based strategies are used to bound the constraint violation region using hyper-spheres and hyper-rectangles. By comparing the resulting bounding sets with any given uncertainty model, it can be determined whether the constraints are satisfied for all members of the uncertainty model (i.e., constraints are feasible) or not (i.e., constraints are infeasible). If constraints are infeasible and a probabilistic uncertainty model is available, upper bounds to the probability of constraint violation can be efficiently calculated. The tools developed enable approximating not only the set of designs that make the constraints feasible but also, when required, the set of designs for which the probability of constraint violation is below a prescribed admissible value. When constraint feasibility is possible, several design criteria can be used to shape the uncertainty model of performance metrics of interest. Worst-case, least-second-moment, and reliability-based design criteria are considered herein. Since the problem formulation is generic and the tools derived only require standard optimization algorithms for their implementation, these strategies are easily applicable to a broad range of engineering problems.

Crespo, Luis G.↗

DOE OSTI2024

A Large-Scale Analysis to Optimize the Control and V2V Communication Protocols for CDA Agreement-Seeking Cooperation

Cooperative driving automation (CDA) Class C, agreement-seeking cooperation, is an innovative and practical solution that can promote cooperation among general passenger vehicles on the road. However, more comprehensive studies are needed before establishing the standard protocols of agreement-seeking cooperation, such as communication frequency and the duration of cooperation. Here, this article presents an initiative study on the impacts of communication capabilities on agreement-seeking cooperation. Through a large-scale analysis by regulating vehicle-to-vehicle (V2V) communication metrics, this work suggests desirable system parameters that can maximize the benefits of cooperation and ensure reliable operability while avoiding exhaustive communication loads. As the first step, an example agreement-seeking cooperation system is created for a car-following scenario, including decision-making and control algorithms for autonomous vehicles. Then, software-in-the-loop tests explore the performance of the developed system as it encounters various communication risks, such as latency and message packet drops. The system performance metrics are evaluated from various angles, including the time consumed for the agreement-seeking process, cooperation ratio, and the ratio of faulty cooperation. Energy saving from the cooperation is assessed by using simulation software that can run multiple high-fidelity vehicle models simultaneously. Based on the analyses, this article suggests the V2V communication requirements for the reliable operation of CDA agreement-seeking, which can be referred to when developing the standard protocols of agreement-seeking cooperation.

42 ENGINEERING↗