SEARCH · Engineering Papers
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
- Objective - Background - Capability Resource Tables (CRT) - Root Cause Analysis(RCA) - Error types - RCA - Fishbone; 5 whys - Lessons Learned
The primary salt pump installed in the high-temperature FLUoride Salt Test Facility (FLUSTFA) was successfully operated for some time, but later ceased operation. To understand what occurred, a Root Cause Analysis (RCA) was performed. Steps taken to try to get the pump operational include adjusting the shaft position, increasing the heating power of the tape heaters on the pump volute, and manually rotating the pump shaft. While removing the insulation, corrosion was noted on the outside of the pump volute, and decolorization of the insulation and tape heaters was observed. Significant corrosion products were also observed in the pump itself and the piping connected to the pump. The nitrogen cover gas was maintained from before salt was introduced into the loop until the pump was dismounted and continues to be maintained even after the pump was removed. After considering probable scenarios, causes were assigned and corrective actions were developed to prevent those causes. Then, the RCA was presented to an advisory committee for review, the “Review Committee,” consisting of experts in large molten salt systems: Brandon Haugh, David Holcomb, Kevin Robb, and Vicente Rojas. As a result, the advisory committee provided comprehensive feedback, which have been incorporated into a revised RCA. Findings have then been summarized and reported in this publication.
Explore the source record for details and available documents.
The medical system for spaceflight thus far has been designed to support missions in low earth orbit (LEO). Crew capabilities are limited and heavily dependent on the team of medical support staff at Mission Control Center (MCC) to guide diagnosis and management. However, missions to the Moon and Mars will suffer from several constraints that will make this ground support focused approach to care ineffective. In order to update and modify medical system design, NASA has relied on Probabilistic Risk Assessment (PRA) modeling to mitigate medical risk through trade space analysis. Specifically, capability resource tables (CRT’s) were developed to create a dataset of resources required to manage a list of accepted medical conditions significant in exploration spaceflight. With 120 conditions, this dataset contained hundreds of capabilities and thousands of resources with tens of thousands of cells of data. Initially these tables were built in excel for high throughput during development, but ultimately had to be transferred, managed, and modified into the Evidence Library database for modeling purposes. The process of collating and reviewing the Evidence Library revealed numerous errors in the dataset that had to be corrected through iterative changes. Several error types emerged during this process and can be broken into specific classifications defined as “input”, “transcription”, “structural”, “branching”, and “information”. In reviewing these error types through the root cause analysis (RCA) approach, we were able to identify the contributors to these errors which included single data review points, changing product end goals, limited software selection, time constraints and several others. By reviewing and evaluating the underlying causes we can provide possible system improvements that can be implemented for current and future data management in PRA model inputs.
Polyvinyl toluene (PVT) plastic is often used to make gamma ray sensitive scintillators and is used in many important applications. For example, PVT is used to scan recycled steel going into a plant to be processed. It is also used in personnel portals to scan employees, and is used at the borders of many countries to scan cargo and cars passing through. In recent years, it was discovered that PVT can become degraded when it absorbs water and then is subjected to daily temperature swings, as is evidence by a reduction in scintillation light. Water absorption by PVT was notionally associated with temporary fogging (i.e., a rapid decrease in opacity which was reversible over time) and permanent fogging (i.e., permanent crack-like defects) and was for some time suspected to be the leading cause of the fogging due to circumstantial evidence. However, definitive proof was not established and in particular the specific mechanisms by which water entered the plastic and ultimately created temporary and permanent point-like defects was not fully understood. This paper is an overview of an effort initiated to reveal the fundamental root cause and also the dynamics of the fogging and degradation processes. This understanding is important not only to solve degradation problems for PVT in fielded systems, but also (1) to provide direction for future procurements for new equipment, and (2) to provide direction for future R&D efforts into advanced plastics with enhanced spectroscopic capabilities or dual particle gamma/neutron capabilities. Unexpected outcomes of this investigation included (1) the capability to predict the onset of fogging based on models using weather data alone, and (2) new modified formulations of PVT and PS which are fog-resistant. This paper provides an overview of the water uptake and associated mechanisms for various compositions, characterization of resulting temporary and permanent defects, predictive modeling of fogging, and possible solutions including encapsulation, heaters and new formulations. While this paper is an overview of the root cause analysis and solution, five accompanying papers (Lance et al., 2018, Myllenbeck and Feng, 2018, Payne et al., 2018, Zaitseva et al. 2018, Kouzes et al., 2018) from the same conference (SORMA 2018) delve further into the details and also expand upon the scope of what will be reported herein.
Abstract Protonic ceramic electrochemical cells (PCECs) offer promising paths for energy storage and conversion. Despite considerable achievements made, PCECs still face challenges such as physiochemical compatibility between componenets and suboptimal solid–solid contact at the interfaces between the electrolytes and electrodes. In this study, a novel approach is proposed that combines in situ electrochemical characterization of interfacial electrical sensor embedded PCECs and machine learning to quantify the contributions of different cell components to total degradation, as well as to predict the remaining useful life. The experimental results suggest that the overpotential induced by the oxygen electrode is 48% less than that of oxygen electrode/electrolyte interfacial contact for up to 1171 h. The data‐driven machine learning simulation predicts the RUL of up to 2132 h. The root cause of degradation is overpotential increase induced by oxygen electrode, which accounts for 82.9% of total cell degradation. The success of the failure diagnostic model is demonstrated by its consistency with degradation modes that do not manifest in electrolysis fade during early real operations. This synergistic approach provides valuable insights into practical failure diagnosis of PCECs and has the potential to revolutionize their development by enabling improved performance prediction and material selection for enhanced durability and efficiency.
On July 7 2020, approximately 1,000 MW reduction in solar photovoltaic (PV) output was experienced by the bulk power system in Southern California after a three-phase fault. Disturbance analysis indicates that instantaneous AC current caused inverter tripping. The projection of the root cause of AC overcurrent from the NERC report is that inverter current control is not tight. This letter provides a quantitative analysis on ac overcurrent and addresses the following questions: (i) what type of current controls may lead to ac overcurrent; and (ii) how controller parameters and grid strength influence ac overcurrent. Here, the letter provides frequency-domain analysis and time-domain simulation results of a grid-integrated inverter to illustrate the effect of control parameters and grid strength on ac overcurrent.
The NASA's Evolutionary Xenon Thruster (NEXT) project is developing an advanced ion propulsion system for future NASA missions for solar system exploration. A critical element of the propulsion system is the Power Processing Unit (PPU) which supplies regulated power to the key components of the thruster. The PPU contains six different power supplies including the beam, discharge, discharge heater, neutralizer, neutralizer heater, and accelerator supplies. The beam supply is the largest and processes up to 93+% of the power. The NEXT PPU had been operated for approximately 200+ hours and has experienced a series of three capacitor failures in the beam supply. The capacitors are in the same, nominally non-critical location the input filter capacitor to a full wave switching inverter. The three failures occurred after about 20, 30, and 135 hours of operation. This paper provides background on the NEXT PPU and the capacitor failures. It discusses the failure investigation approach, the beam supply power switching topology and its operating modes, capacitor characteristics and circuit testing. Finally, it identifies root cause of the failures to be the unusual confluence of circuit switching frequency, the physical layout of the power circuits, and the characteristics of the capacitor.
The NASA s Evolutionary Xenon Thruster (NEXT) project is developing an advanced ion propulsion system for future NASA missions for solar system exploration. A critical element of the propulsion system is the Power Processing Unit (PPU) which supplies regulated power to the key components of the thruster. The PPU contains six different power supplies including the beam, discharge, discharge heater, neutralizer, neutralizer heater, and accelerator supplies. The beam supply is the largest and processes up to 93+% of the power. The NEXT PPU had been operated for approximately 200+ hr and has experienced a series of three capacitor failures in the beam supply. The capacitors are in the same, nominally non-critical location-the input filter capacitor to a full wave switching inverter. The three failures occurred after about 20, 30, and 135 hr of operation. This paper provides background on the NEXT PPU and the capacitor failures. It discusses the failure investigation approach, the beam supply power switching topology and its operating modes, capacitor characteristics and circuit testing. Finally, it identifies root cause of the failures to be the unusual confluence of circuit switching frequency, the physical layout of the power circuits, and the characteristics of the capacitor.
Not Available
Dynamic phenomena linked to inverter-based resources (IBRs) have gained global attention. Several IBR-induced dynamics have caused bulk power system-connected wind or solar power plants to trip, and some have even led to widespread outages. In addition, many oscillations have been observed involving IBR power plants. In 2023, the IEEE Power & Energy Society (PES) IBR Subsynchronous Oscillations (SSO) task force published a journal article, “Real-World Subsynchronous Oscillation Events in Power Grids With High Penetrations of Inverter-Based Resources,” in which 19 IBR oscillation events were examined for their causation. Earlier in 2020, another PES task force article, “Definition and Classification of Power System Stability-Revisited & Extended,” authored by prominent academics, introduced converter-driven stability as a new category of stability. The international power grid industry community also took action by publishing the CIGRE Green Book, Power System Dynamic Modelling and Analysis in Evolving Networks (led by Babak Badrzadeh and Zia Emin) in 2024. In August 2024, the Energy Systems Integration Group (ESIG) released a practical guide led by Nick Miller, “Diagnosis and Mitigation of Observed Oscillations in IBR-Dominant Power System: A Practical Guide.” The goal of the guide is to assist practicing engineers in making initial judgments and conducting detailed analyses about oscillations. Finally, when addressing the classification of stability and oscillations, the guide emphasizes a causality-based taxonomy for grouping, such as voltage control-induced oscillations, synchronization-induced oscillations, and frequency or active power control-induced oscillations.
A gradual, but persistent, decrease in the optical throughput was detected during the early commissioning phase for the Suomi National Polar-Orbiting Partnership (SNPP) Visible Infrared Imager Radiometer Suite (VIIRS) Near Infrared (NIR) bands. Its initial rate and unknown cause were coincidently coupled with a decrease in sensitivity in the same spectral wavelength of the Solar Diffuser Stability Monitor (SDSM) raising concerns about contamination or the possibility of a system-level satellite problem. An anomaly team was formed to investigate and provide recommendations before commissioning could resume. With few hard facts in hand, there was much speculation about possible causes and consequences of the degradation. Two different causes were determined as will be explained in this paper. This paper will describe the build and test history of VIIRS, why there were no indicators, even with hindsight, of an on-orbit problem, the appearance of the on-orbit anomaly, the initial work attempting to understand and determine the cause, the discovery of the root cause and what Test-As-You-Fly (TAYF) activities, can be done in the future to greatly reduce the likelihood of similar optical anomalies. These TAYF activities are captured in the lessons learned section of this paper.
The complex interdependencies of cyber systems (sensors and communications), physical grids and associated electricity market operations make protecting electric power grids a significant challenge. The energy sector is constantly under new, targeted, advanced and dangerous cyber-attacks that have the potential to result in the loss of human life. These threats are further exacerbated by our need to modernize the grid. One focus of cyber security research in smart grids is the securing of the SCADA system through advanced intrusion detection systems (IDS) and bad data detection algorithms in state estimation. These methods either require full knowledge of the system topology and parameters or fail to understand the physical behaviors under attack. WISP (Watching grid Infrastructure Stealthily through Proxies) is designed to provide additional protection to the power grid using only publicly available data. In particular, WISP exploits the spatio-temporal nature of the real time locational marginal prices (LMPs), in conjunction with other information such as bids, weather, outages and load data to analyze anomalous power pricing behaviors and then correlate those observations to localize regions of interest and identify potential cyber events. WISP is non-intrusive as the tool is deployed as a service in the Cloud or on premise and provides reliable information to system operators for enhanced situational awareness, without impeding energy delivery functions. The WISP technology comprises three modules: the data-driven anomaly detection core, the vulnerability and risk analysis and the root cause analysis. The data-driven anomaly detection core performs the tasks of feature selection, anomaly detection and attack region localization. The vulnerability and risk analysis module provides system level information of the vulnerable variables and times, assisting the operators in selecting monitoring and protection nodes. The root cause analysis module takes the detection results and identifies potential operational conditions that contribute to the detected anomalies. In Phase I, we have demonstrated the feasibility and effectiveness of WISP. We developed a realistic electricity market simulator capable of generating normal and attack market data under various operational conditions. We developed a series of cyber-attack detection and analysis algorithms and evaluated them under multiple data sources. Finally, we integrated all modules into an end-to-end software, providing functions for data management, data analytics and visualization. Specifically, we have achieved: (i) real-time data acceptance from external utility interfaces with >99% acceptance rate; (ii) high performance anomaly detection algorithms with >98% detection accuracy and <0.1% false alarm rate; and (iii) ultra-low computing delay <50 milliseconds. Additionally, our team developed algorithms to identify the vulnerable variables in electricity market operations and root cause analysis functions to identify major contributors to the price spikes. These ancillary modules are necessary when deploying WISP in real world industry environment. In Phase II, we have demonstrated the effectiveness of WISP software on realistic largescale power systems. We performed red team testing for the Phase I WISP software and identified software vulnerabilities and implemented corresponding mitigation solutions. We adapted the electricity market simulator for the Texas synthetic 2000-bus system and generated datasets for the false data injection attacks. We created database and visualization interfaces for the Texas system and the ISO New England system. We performed software optimization in terms of operation efficiency, computing speed and detection accuracy. Finally, we tested the software on the Texas system and the ISO New England system and evaluated the detection performance. Overall, we achieved above 89% detection rate, below 3% false alarm rate and below 37 seconds of end-to-end detection delay.
Electronic logbooks contain valuable information about activities and events concerning their associated particle accelerator facilities. However, the highly technical nature of logbook entries can hinder their usability and automation. As natural language processing (NLP) continues advancing, it offers opportunities to address various challenges that logbooks present. This work explores jointly testing a tailored Retrieval Augmented Generation (RAG) model for enhancing the usability of particle accelerator logbooks at institutes like DESY, BESSY, Fermilab, BNL, SLAC, LBNL and CERN. The RAG model uses a corpus built on logbook contributions and aims to unlock insights from these logbooks by leveraging retrieval over facility datasets, including discussion about potential multimodal sources. Our goals are to increase the FAIR-ness (findability, accessibility, interoperability, and reusability) of logbooks by exploiting their information content to streamline everyday use, to enable macro-analysis for root cause analysis, and to facilitate problem-solving automation.
Electronic logbooks contain valuable information about activities and events concerning their associated particle accelerator facilities. However, the highly technical nature of logbook entries can hinder their usability and automation. As natural language processing (NLP) continues advancing, it offers opportunities to address various challenges that logbooks present. This work explores jointly testing a tailored Retrieval Augmented Generation (RAG) model for enhancing the usability of particle accelerator logbooks at institutes like DESY, BESSY, Fermilab, BNL, SLAC, LBNL, and CERN. The RAG model uses a corpus built on logbook contributions and aims to unlock insights from these logbooks by leveraging retrieval over facility datasets, including discussion about potential multimodal sources. Our goals are to increase the FAIR-ness (findability, accessibility, interoperability, and reusability) of logbooks by exploiting their information content to streamline everyday use, enable macro-analysis for root cause analysis, and facilitate problem-solving automation.
Systems theoretic process analysis (STPA) is becoming an increasingly popular technique to assess how complex digital software systems can fail. Rather than defining failures by their observable failure events, which may be sparse especially for safety rated nuclear digital instrumentation and control systems (DI&C), failures are defined as postulated unsafe actions under specific contextual conditions. This permits a top-down analysis of system hazards and identifies whether imposed constraints and requirements can sufficiently address undesirable hazards. However, STPA is a qualitative approach at identifying inadequacies in the development process and cannot currently be used to quantify unsafe action likelihoods for probabilistic risk assessment. Therefore, in this work, we examine the root causes of software failure and explore whether a consistent correlation can be linked to specific unsafe action classes. We implement Lbl2Vec, an unsupervised document classification and retrieval algorithm, on a database of 4,096 software defect reports acquired from various open-source software systems. By analyzing sentence structure, embedded labels, and word vectors, we show that certain defect types positively correlate to specific unsafe action classes over others. The correlations developed can be used to estimate the failure probability of safety intended DI&C systems which provides a licensing basis for nuclear plant modernization efforts.
Systems theoretic process analysis (STPA) is becoming an increasingly popular technique to assess how complex digital software systems can fail. Rather than defining failures by their observable failure events, which may be sparse especially for safety rated nuclear digital instrumentation and control systems (DI&C), failures are defined as postulated unsafe actions under specific contextual conditions. This permits a top-down analysis of system hazards and identifies whether imposed constraints and requirements can sufficiently address undesirable hazards. However, STPA is a qualitative approach at identifying inadequacies in the development process and cannot currently be used to quantify unsafe action likelihoods for probabilistic risk assessment. Therefore, in this work, we examine the root causes of software failure and explore whether a consistent correlation can be linked to specific unsafe action classes. We implement Lbl2Vec, an unsupervised document classification and retrieval algorithm, on a database of 4,096 software defect reports acquired from various open-source software systems. By analyzing sentence structure, embedded labels, and word vectors, we show that certain defect types positively correlate to specific unsafe action classes over others. The correlations developed can be used to estimate the failure probability of safety intended DI&C systems which provides a licensing basis for nuclear plant modernization efforts.