Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “reliability metrics”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 451 records · Page 25

Comfort and Safety Metrics to Evaluate Customer Centric Resilience Enhancement Strategies

As the coincident events of extreme weather and major power outages continue to rise there is an increased need to implement resilience solutions. Many of these solutions are customer-centric, requiring the flexibility of demand, and many of the standard economic evaluations of these solutions do not include the benefits of resilience that are not easily quantified and monetized. Indoor air temperature has a large impact on the comfort and safety of building occupants during an extreme weather event. This work proposes metrics that can be used to compare the performance of resilience solutions in the context of comfort and safety.

Cost benefit analysis, Demand Side Management, Pow↗

Error metrics for partially coherent wave fields

Lensless imaging methods that account for partial coherence have become very common in the past decade. However, there are no metrics in use for comparing partially coherent light fields, despite the widespread use of such metrics to compare fully coherent objects and wave fields. Here, we show how reformulating the mean squared error and Fourier ring correlation in terms of quantum state fidelity naturally generalizes them to partially coherent wave fields. We report these results fill an important gap in the lensless imaging literature and will enable quantitative assessments of the reliability and resolution of reconstructed partially coherent wave fields.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Sensitivity of Optical Satellites to Estimate Windthrow Tree-Mortality in a Central Amazon Forest

Windthrow (i.e., trees broken and uprooted by wind) is a major natural disturbance in Amazon forests. Images from medium-resolution optical satellites combined with extensive field data have allowed researchers to assess patterns of windthrow tree-mortality and to monitor forest recovery over decades of succession in different regions. Although satellites with high spatial-resolution have become available in the last decade, they have not yet been employed for the quantification of windthrow tree-mortality. Here, we address how increasing the spatial resolution of satellites affects plot-to-landscape estimates of windthrow tree-mortality. We combined forest inventory data with Landsat 8 (30 m pixel), Sentinel 2 (10 m), and WorldView 2 (2 m) imagery over an old-growth forest in the Central Amazon that was disturbed by a single windthrow event in November 2015. Remote sensing estimates of windthrow tree-mortality were produced from Spectral Mixture Analysis and evaluated with forest inventory data (i.e., ground true) by using Generalized Linear Models. Field measured windthrow tree-mortality (3 transects and 30 subplots) crossing the entire disturbance gradient was 26.9 ± 11.1% (mean ± 95% CI). Although the three satellites produced reliable and statistically similar estimates (from 26.5% to 30.3%, p < 0.001), Landsat 8 had the most accurate results and efficiently captured field-observed variations in windthrow tree-mortality across the entire gradient of disturbance (Sentinel 2 and WorldView 2 produced the second and third best results, respectively). As expected, mean-associated uncertainties decreased systematically with increasing spatial resolution (i.e., from Landsat 8 to Sentinel 2 and WorldView 2). However, the overall quality of model fits showed the opposite pattern. We suggest that this reflects the influence of a relatively minor disturbance, such as defoliation and crown damage, and the fast growth of natural regeneration, which were not measured in the field nor can be captured by coarser resolution imagery. Our results validate the reliability of Landsat imagery for assessing plot-to-landscape patterns of windthrow tree-mortality in dense and heterogeneous tropical forests. Satellites with high spatial resolution can improve estimates of windthrow severity by allowing the quantification of crown damage and mortality of lower canopy and understory trees. However, this requires the validation of remote sensing metrics using field data at compatible scales.

54 ENVIRONMENTAL SCIENCES↗

Controls concepts for next generation reuseable rocket engines

Three primary issues will drive the design and control used in next generation reuseable rocket engines. In addition to steady-state and dynamic performance, the requirements for increased durability, reliability and operability (with faults) will dictate which new controls and design technologies and features will be brought to bear. An array of concepts which have been brought forward will be tested against the measures of cost and benefit as reflected in the above 'ilities'. This paper examines some of the new concepts and looks for metrics to judge their value.

Lorenzo, Carl F.↗

Benchmarking Diagnostic Algorithms on an Electrical Power System Testbed

Diagnostic algorithms (DAs) are key to enabling automated health management. These algorithms are designed to detect and isolate anomalies of either a component or the whole system based on observations received from sensors. In recent years a wide range of algorithms, both model-based and data-driven, have been developed to increase autonomy and improve system reliability and affordability. However, the lack of support to perform systematic benchmarking of these algorithms continues to create barriers for effective development and deployment of diagnostic technologies. In this paper, we present our efforts to benchmark a set of DAs on a common platform using a framework that was developed to evaluate and compare various performance metrics for diagnostic technologies. The diagnosed system is an electrical power system, namely the Advanced Diagnostics and Prognostics Testbed (ADAPT) developed and located at the NASA Ames Research Center. The paper presents the fundamentals of the benchmarking framework, the ADAPT system, description of faults and data sets, the metrics used for evaluation, and an in-depth analysis of benchmarking results obtained from testing ten diagnostic algorithms on the ADAPT electrical power system testbed.

Kurtoglu, Tolga↗

Comparison of Taxi Time Prediction Performance Using Different Taxi Speed Decision Trees

In the STBO modeler and tactical surface scheduler for ATD-2 project, taxi speed decision trees are used to calculate the unimpeded taxi times of flights taxiing on the airport surface. The initial taxi speed values in these decision trees did not show good prediction accuracy of taxi times. Using the more recent, reliable surveillance data, new taxi speed values in ramp area and movement area were computed. Before integrating these values into the STBO system, we performed test runs using live data from Charlotte airport, with different taxi speed settings: 1) initial taxi speed values and 2) new ones. Taxi time prediction performance was evaluated by comparing various metrics. The results show that the new taxi speed decision trees can calculate the unimpeded taxi-out times more accurately.

taxi time prediction↗

An Overview of Printed Circuit Board Risks and Supplier Capability

This NESC Webinar provides the audience a snapshot of quality variations in printed circuit boards, as assessed, using experiences in processing and risk analysis of PCB structural integrity coupons. The presentation will focus on PCB quality metrics, the relative type and number of non-conformances observed and trend analysis using statistical methods. Trend analysis shows the top five non-conformances observed across suppliers, the associated root cause(s) behind these non-conformance, suggestions of mitigation plans and most importantly – the relation of these non-conformances to end item reliability. The statistical trends will then be matched with the current state of the domestic PCB supplier base, its key challenges and opportunities. The presentation further discusses the methods applied at GSFC for evaluating candidate PCB technologies that promote the adoption of higher throughput and faster processing technology for GSFC missions.

printed cricuit boards, supplier capability, risk ↗

An Overview of Printed Circuit Board Risks and Supplier Capability (Reuse Document ID 20205007963)

This HDPUG Webinar provides the audience a snapshot of quality variations in printed circuit boards, as assessed, using experiences in processing and risk analysis of PCB structural integrity coupons. The presentation will focus on PCB quality metrics, the relative type and number of non-conformances observed and trend analysis using statistical methods. Trend analysis shows the top five non-conformances observed across suppliers, the associated root cause(s) behind these non-conformance, suggestions of mitigation plans and most importantly – the relation of these non-conformances to end item reliability. The statistical trends will then be matched with the current state of the domestic PCB supplier base, its key challenges and opportunities. The presentation further discusses the methods applied at GSFC for evaluating candidate PCB technologies that promote the adoption of higher throughput and faster processing technology for GSFC missions.

printed cricuit boards, supplier capability, risk ↗

Predicting Initial Trans-Membrane Pressure for Optimized Operations in UF Unit Using Random Forest

With the growing scarcity of freshwater, innovative process design mechanisms like Reverse Osmosis (RO) are increasingly gaining attention among water treatment utilities to address the rising demand. Ensuring reliable water production necessitates efficient resource utilization, minimizing downtime in (ultra-filtration) UF systems. Recent advancements in machine learning (ML) have enabled the development of accurate data-driven models for Model Predictive Control (MPC), often requiring minimal prior knowledge of underlying physical processes. In this study, we present predictive regression models based on Random Forest (RF) and Auto-Regressive (AR) approaches to forecast the initial Trans-Membrane Pressure (TMP) for each filtration cycle in data generated by Direct Potable Reuse (DPR) systems. The proposed RF-based model demonstrates superior performance compared to baseline methods, including historical mean, Last Observation Carried Forward (LOCF), and naïve AR models, across various forecasting horizons in terms of root mean square error (RMSE) metric. To evaluate how different classes of process variables contribute to TMP dynamics over time, we examine the feature importance of independent covariates across multiple forecast horizons. This analysis provides insight into the temporal relevance of operational and sensor-derived features, guiding control and monitoring strategies. Additionally, the impact of hyperparameter tuning on TMP prediction performance is studied for both direct and recursive RF modelling approaches across increasing forecast horizons. Accurate prediction of initial TMP is critical for optimizing RO operations, as it enables the development of robust modelling frameworks by accurately estimating membrane fouling trends, thereby enhancing process efficiency and long-term reliability. The demonstrated efficacy of the RF-based approach highlights its potential as a tool for real-time decision-making in water treatment systems, paving the way for advanced process optimization and sustainable water resource management.

Mukherjee, Subrata [ORNL] (ORCID:0000000309930338)↗

Independent Fuel Property Effects of Fuel Volatility on Low Temperature Heat Release and Fuel Autoignition (Final Report)

This Cooperative Research and Development Agreement (CRADA) project between Argonne National Laboratory (ANL), Oak Ridge National Laboratory (ORNL), and Shell Global Solutions (Shell) was initiated as part of a Directed Funding Opportunity (DFO) call for proposals from the Co-Optimization of Fuels and Engines (Co-Optima) initiative. Shell had observed that volatile fuels suppress low temperature heat release (LTHR) more than expected based on conventional gasoline autoignition metrics: research octane number (RON) and motor octane number (MON). The role of LTHR contributes to autoignition phenomena for both boosted spark ignition (BSI) and advanced compression ignition (ACI) combustion modes. ACI combustion modes are applicable to large engines in the hard-to-electrify applications such as off-road, rail, and marine. Thus, having a reliable understanding of autoignition phenomena, including being able to accurately account for the effects of fuel volatility, is particularly important as new synthetic and bio-fuel compositions are considered.This CRADA project aimed to test the hypothesis that the decreased LTHR is due to preferential evaporation of multicomponent fuels when using direct injection (DI) fueling technology, creating composition and reactivity stratification. A custom set of fuels was designed and blended to test this hypothesis by Shell, with experimental engine studies at ORNL and engine combustion modeling by ANL. However, the initial experimental findings did not show the expected effect of fuel volatility suppressing LTHR. Instead, the LTHR propensity observed was independent of the fuel volatility. Due to the unexpected experimental result, the remainder of the experimental effort was redirected to study the effect of fuel volatility on emissions under spark-ignited cold-start conditions. However, as with the LTHR experiments, the cold start effort did not show a meaningful effect of fuel volatility on cold start emissions. Meanwhile, improved engine CFD models have been developed for both LTHR and cold start operations for the Shell fuels with different volatilities. While the simulation efforts were not pursued further due to the insignificant effects of fuel volatility as shown in experiments, the models developed can be easily retooled for off-road, rail, and marine applications.

09 BIOMASS FUELS↗

Analyzing Terawatt Scale Sustainability with PV ICE Tool

Renewable energy reduces environmental impacts and decarbonizes the production of other goods. But, manufacturing renewable energy sources, such as photovoltaic (PV) modules, require energy inputs that are currently carbon intensive. So, how do we decarbonize and circularize these critical technologies to achieve a sustainable energy transition? As we manufacture and deploy solar PV at a fast pace to support the Energy Transition, this talk will cover some of the most relevant metrics for evaluating the success of our circular economy and sustainability choices.

circular economy↗

Energy Systems Integration Facility Stewardship Summary: Fiscal Year 2025

A summary of NLR's stewardship of the nationally unique Energy Systems Integration Facility (ESIF) highlighting performance metrics, capability upgrades, and examples of R&D impact. 2025 brought a national focus on energy and ESIF is meeting the moment. All eyes are on data centers and domestic manufacturing and bringing the benefits of artificial intelligence to power system planning and operations. In step with national priorities, ESIF is building out capabilities that advance secure, reliable, and affordable power. ESIF hosted 190 multidisciplinary research projects, 855 high-performance computer users, and collaborated with 81 partners from industry, academia, research, and federal agencies. These research projects resulted in an AI method for detecting high-impedance faults with 90% accuracy, a grid controls demonstration in Connecticut, power quality validation of CorePower's flagship inductor, and a cybersecurity assessment of potential rogue capabilities in digitally connected energy devices. Facility infrastructure improvements enhanced the thermal research network, the SCADA system, the cyber range, power hardware-in-the-loop testing, and more. With support from the U.S Department of Energy (DOE), the ESIF laboratories continue to deliver leading solutions for secure, reliable, and affordable power.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Small Punch Testing of Molybdenum-99 Targets

Northstar Medical Radioisotopes is developing an accelerator-based method to produce 99 Mo, which is a parent isotope of the commonly used 99m Tc medical isotope. The Mo targets being designed for the accelerator will be produced from enriched 100 Mo, also known as aMo . Pressed and sintered powder feedstock is used to fabricate aMo targets, producing 29 mm disk-shaped targets. The targets are subjected to 1–6 days in line of an electron beam with subsequent dissolution of the disk to retain the 99 Mo, which decays to 99m Tc at radio-pharmacies. The press and sinter method is advantageous because the inherent porosity produced by this method enables increased surface area and therefore increased flow of dissolution media, decreasing the dissolution time and reducing the need for a highly acidic media. Although porosity aids in dissolution, it reduces the mechanical strength and ductility. Targets require good mechanical integrity when subjected to the conditions in the accelerator. Therefore, Northstar is seeking methods to rapidly test disk samples after fabrication to assure mechanical performance metrics are achieved. This report details the design and testing of a small punch test (SPT) that accommodates the 29 mm disk. Initial data were used to relate the SPT data to tensile properties, such as the yield strength (YS), ultimate tensile strength (UTS), and total elongation to failure. Although the SPT has been established as a somewhat reliable method for testing metallic materials, few studies have applied the SPT to refractory materials such as Mo. Based on tensile testing performed at Oak Ridge National Laboratory on different Mo samples, correlation between the Mo tensile and SPT properties could be performed, establishing a standard calibration that could be applied to other Mo samples. To test the efficacy of the SPT with Mo, multiple different disk batches were fabricated under different conditions (e.g., pressure, lubricant) with commercially available pure Mo powder. Generally, only a UTS could be well defined because the press and sinter disks failed under brittle fracture, making it difficult to determine the YS and elongation. Compared with disks fabricated with aMo powder, the aMo samples underperformed their pure nat Mo counterparts. This report summarizes the current status of the SPT, but further evaluation is needed before it can be applied as a reliable quality assurance tool.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

Post-Newtonian gravitational bremsstrahlung

Formulas and numerical results for the gravitational radiation emitted during a low-deflection encounter between two massive bodies are presented. The results are valid through post-Newtonian order within general relativity. The gravitational waveform (transverse-traceless part of the metric perturbation tensor), the total luminosity and total emitted energy, the angular distribution of emitted energy (antenna pattern), and the frequency spectrum are discussed. A method of boosting the accuracy of these quantities to post-3/2-Newtonian order is presented. A numerical comparison of the results with previously reported results shows that the post-Newtonian method is reliable to better than 0.1% at v = 0.1c, to a few percent at v = 0.35c, and to 10-20% at v = 0.5c.

Turner, M.↗

Quality Assessment of Landsat Surface Reflectance Products Using MODIS Data

Surface reflectance adjusted for atmospheric effects is a primary input for land cover change detection and for developing many higher level surface geophysical parameters. With the development of automated atmospheric correction algorithms, it is now feasible to produce large quantities of surface reflectance products using Landsat images. Validation of these products requires in situ measurements, which either do not exist or are difficult to obtain for most Landsat images. The surface reflectance products derived using data acquired by the Moderate Resolution Imaging Spectroradiometer (MODIS), however, have been validated more comprehensively. Because the MODIS on the Terra platform and the Landsat 7 are only half an hour apart following the same orbit, and each of the 6 Landsat spectral bands overlaps with a MODIS band, good agreements between MODIS and Landsat surface reflectance values can be considered indicators of the reliability of the Landsat products, while disagreements may suggest potential quality problems that need to be further investigated. Here we develop a system called Landsat-MODIS Consistency Checking System (LMCCS). This system automatically matches Landsat data with MODIS observations acquired on the same date over the same locations and uses them to calculate a set of agreement metrics. To maximize its portability, Java and open-source libraries were used in developing this system, and object-oriented programming (OOP) principles were followed to make it more flexible for future expansion. As a highly automated system designed to run as a stand-alone package or as a component of other Landsat data processing systems, this system can be used to assess the quality of essentially every Landsat surface reflectance image where spatially and temporally matching MODIS data are available. The effectiveness of this system was demonstrated using it to assess preliminary surface reflectance products derived using the Global Land Survey (GLS) Landsat images for the 2000 epoch. As surface reflectance likely will be a standard product for future Landsat missions, the approach developed in this study can be adapted as an operational quality assessment system for those missions.

Landsat↗

Review of Technical Photovoltaic Key Performance Indicators and the Importance of Data Quality Routines

Technical key performance indicators (KPIs) are important metrics used to assess and quantitatively summarize various aspects of photovoltaic (PV) systems, including long-term performance, economic viability, and carbon footprint. Herein, a group of experts of the International Energy Agency's Photovoltaic Power Systems Programme Task 13 collect and describ the most important technical KPIs used in the industry. Thereby, a set of best practices for reliably handling PV system data is presented and the impact of data quality and climatic variability on KPI calculation is investigated. Further, the effective use of technical KPIs allows triggering data-driven and informed decisions to optimize PV systems and providing a comprehensive overview of how PV systems operate across different conditions and climates. With the worldwide growth of the PV industry, more companies operate/own PV systems in different regions, where the climatic and seasonal profiles differ. This requires context-aware evaluation of KPIs, or the judicious application of multiple KPIs, to ensure that each asset is evaluated correctly. Beyond that, there is untapped potential in the utilization of KPIs through geospatial mapping and extrapolation of fleet KPIs. This study demonstrates that the uncertainty in KPI estimation is not well understood and depends on data quality, climatic variability, and system configuration.

14 SOLAR ENERGY↗

Training calibration-based counterfactual explainers for deep learning models in medical image analysis

The rapid adoption of artificial intelligence methods in healthcare is coupled with the critical need for techniques to rigorously introspect models and thereby ensure that they behave reliably. This has led to the design of explainable AI techniques that uncover the relationships between discernible data signatures and model predictions. In this context, counterfactual explanations that synthesize small, interpretable changes to a given query while producing desired changes in model predictions have become popular. This under-constrained, inverse problem is vulnerable to introducing irrelevant feature manipulations, particularly when the model’s predictions are not well-calibrated. Hence, in this paper, we propose the TraCE (training calibration-based explainers) technique, which utilizes a novel uncertainty-based interval calibration strategy for reliably synthesizing counterfactuals. Given the wide-spread adoption of machine-learned solutions in radiology, our study focuses on deep models used for identifying anomalies in chest X-ray images. Using rigorous empirical studies, we demonstrate the superiority of TraCE explanations over several state-of-the-art baseline approaches, in terms of several widely adopted evaluation metrics. Our findings show that TraCE can be used to obtain a holistic understanding of deep models by enabling progressive exploration of decision boundaries, to detect shortcuts, and to infer relationships between patient attributes and disease severity.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

Explainable AI for Multivariate Time Series Pattern Exploration: Latent Space Visual Analytics With Temporal Fusion Transformer and Variational Autoencoders in Power Grid Event Diagnosis

Detecting and analyzing complex patterns in multivariate time-series data is crucial for decision-making in urban and environmental system operations. However, challenges arise from the high dimensionality, intricate complexity, and interconnected nature of complex patterns, which hinder the understanding of their underlying physical processes. Existing AI methods often face limitations in interpretability, computational efficiency, and scalability, reducing their applicability in real-world scenarios. This paper proposes a novel visual analytics framework that integrates two generative AI models, Temporal Fusion Transformer (TFT) and Variational Autoencoders (VAEs), to reduce complex patterns into lower-dimensional latent spaces and visualize them in 2D using dimensionality reduction techniques such as PCA, t-SNE, and UMAP with DBSCAN. These visualizations, presented through coordinated and interactive views and tailored glyphs, enable intuitive exploration of complex multivariate temporal patterns, identifying patterns’ similarities and uncover their potential correlations for a better interpretability of the AI outputs. The framework is demonstrated through a case study on power grid signal data, where it identifies multi-label grid event signatures, including faults and anomalies with diverse root causes. Additionally, novel metrics and visualizations are introduced to validate the models and assess the performance, efficiency, and consistency of latent maps generated by VAE, which have been utilized in prior studies for latent space cartography and used as a benchmark in this study, and the emerging TFT architecture under various configurations. These analyses provide actionable insights for model parameter tuning and reliability improvements. Comparative results highlight that TFT achieves shorter run times and superior scalability to diverse time-series data shapes compared to VAE. This work advances fault diagnosis in multivariate time series, fostering explainable AI to support critical system operations.

Explainable AI↗