Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “failure recovery”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6

Electro-Thermo-Mechanical (ETM) Study on Submarine Dynamic Power Cables (SDPC) for Offshore Wind Transmission

Understanding the failure mechanisms of submarine dynamic power cables (SDPC) is critical for innovative design to meet the 2035 cost reduction target of U.S. DOE Floating Offshore Wind Shot. This is important because the current design suffers a significant failure rate in the field. This project conducted a systematic electro-thermo-mechanical (ETM) experimental study on the power cores extracted from a 15kV power cable with three cores of copper conductor, ethylene propylene rubber (EPR) insulation, and continuously corrugated welded aluminum armor (CCWA). An ETM testing system was developed by integrating a high voltage (HV) amplifier, two ceramic heaters, and a rod-plate transverse compression setup into a mechanical testing machine. Increasing temperature from room temperature (RT, 22 degree C) to 90 degree C resulted in the 67% decrease in the failure mechanical load as defined by the dielectric breakdown. Under creep mechanical loading, the dielectric breakdown time was decreased by 70% for a given mechanical load when the specimen temperature increased from RT to 90oC. The failure strain in both monotonic and creep mechanical loading modes was related to the maximum mechanical load applied. Although the dielectric breakdown occurred in the ETM test, the post-test measurement revealed an impressive recovery of electric resistance. The failure analysis on cross section of tested specimens indicated a sizable gap across insulation layer near the area between copper conductor and loading rod, which apparently resulted from online dielectric failure and offline elastic recovery of components including conductor and insulation layer.

17 WIND ENERGY↗

Enhanced Component Performance Study: Emergency Diesel Generators 1998-2020

This report presents an enhanced performance evaluation of emergency power system (EPS) and high-pressure core spray (HPCS) emergency diesel generators (EDGs) at U.S. commercial nuclear power plants. This report evaluates component performance over time using (1) Institute of Nuclear Power Operations (INPO) Industry Reporting and Information System (IRIS) data from 1998 through 2020 and (2) maintenance unavailability performance data from Mitigating Systems Performance Index (MSPI) Basis Document data from 2002 through 2020. The objective is to show estimates of current failure probabilities and rates related to EDGs, trend these data on an annual basis, determine if the current data are consistent with the probability distributions currently recommended for use in NRC probabilistic risk assessments, show how the reliability data differ for different EDG manufacturers and for EDGs with different ratings; and summarize the subcomponents, causes, detection methods, and recovery associated with each EDG failure mode. The EDG failure modes considered are fail to start (FTS), fail to load and run (FTLR), and fail to run after one hour of operation (FTR>1H). Engineering analyses were performed with respect to time period and failure mode without regard to the actual number of EDGs at each plant. The factors analyzed are: subcomponent, failure cause, detection method, recovery, manufacturer, and EDG rating. The following increasing trends were identified for EDGs for the most recent 10-year period: • HPCS EDG FTS failure probability • EPS and HPCS EDG frequency of start demands (demands per reactor year) • EPS and HPCS EDG frequency of FTLR demands. The following decreasing trends were identified for EDGs for the most recent 10-year period: • EPS EDG FTS failure probability • EPS EDG FTLR failure probability • EPS EDG unavailability • EPS and HPCS EDG frequency of FTLR events (failures per reactor year).

99 GENERAL AND MISCELLANEOUS↗

Enhanced Component Performance Study: Emergency Diesel Generators 1998–2022

This report presents an enhanced performance evaluation of the emergency power system (EPS) and high-pressure core spray (HPCS) emergency diesel generators (EDGs) at U.S. commercial nuclear power plants. This report evaluates component performance over time using (1) Institute of Nuclear Power Operations (INPO) Industry Reporting and Information System (IRIS) data from 1998 through 2022 and (2) maintenance unavailability performance data from Mitigating Systems Performance Index (MSPI) Basis Document data from 2002 through 2022. The objective is to show estimates of current failure probabilities and rates related to EDGs, trend these data on an annual basis, determine if the current data are consistent with the probability distributions currently recommended for use in Nuclear Regulatory Commission (NRC) probabilistic risk assessments, show how the reliability data differ for different EDG manufacturers and for EDGs with different ratings; and summarize the subcomponents, causes, detection methods, and recovery associated with each EDG failure mode. The EDG failure modes considered are fail to start (FTS), fail to load and run (FTLR), and fail to run after one hour of operation (FTR>1H). Engineering analyses were performed with respect to time-period and failure mode without regard to the actual number of EDGs at each plant. The factors analyzed include subcomponent, failure cause, detection method, recovery, manufacturer, and EDG rating. The following increasing trends were identified for EDGs for the most recent 10-year period: (1) HPCS EDG FTS failure probability, (2) EPS and HPCS EDG frequency of start demands (demands per reactor year), and (3) EPS and HPCS EDG frequency of FTLR demands. The following decreasing trends were identified for EDGs for the most recent 10-year period: (1) EPS EDG FTS failure probability, (2) EPS EDG FTLR failure probability, (3) EPS EDG unavailability, and (4) EPS and HPCS EDG frequency of FTLR events (failures per reactor year).

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Swift BAT Thermal Recovery After Loop Heat Pipe #0 Secondary Heater Controller Failure in October 2015

The Swift BAT LHP #0 primary heater controller failed on March 31, 2010. It has been disabled. On October 31, 2015, the secondary heater controller of this LHP failed. On November 1, 2015, the LHP #0 CC temperature increased to as 18.6 C, despite that the secondary heater controller set point was 8.8 C. It caused the average DM XA1 temperature to increase to 25.9 C, which was 5 C warmer than nominal. As a result, the detectors became noisy. To solve this problem, the LHP #1 secondary heater controller set point was decreased in 0.5 C decrements to 2.2 C. The set-point decrease restored the average DM XA1 temperature to a nominal value of 19.7 C on November 21.

thermal recovery in orbit↗

Enhanced Component Performance Study: Emergency Diesel Generators 1998-2024

This report presents an enhanced performance evaluation of the emergency power system (EPS) and high-pressure core spray (HPCS) emergency diesel generators (EDGs) at U.S. commercial nuclear power plants. This report evaluates component performance over time using (1) Institute of Nuclear Power Operations (INPO) Industry Reporting and Information System (IRIS) data from 1998 through 2024 and (2) maintenance unavailability performance data from Mitigating Systems Performance Index (MSPI) Basis Document data from 2002 through 2024. The objective is to show estimates of current failure probabilities and rates related to EDGs, trend these data on an annual basis, determine if the current data are consistent with the probability distributions currently recommended for use in Nuclear Regulatory Commission (NRC) probabilistic risk assessments, show how the reliability data differ for different EDG manufacturers and for EDGs with different ratings; and summarize the subcomponents, causes, detection methods, and recovery associated with each EDG failure mode. The EDG failure modes considered are fail to start (FTS), fail to load and run (FTLR), and fail to run after one hour of operation (FTR>1H). Engineering analyses were performed with respect to time-period and failure mode without regard to the actual number of EDGs at each plant. The factors analyzed include subcomponent, failure cause, detection method, recovery, manufacturer, and EDG rating. The following increasing trends were identified for EDGs for the most recent 10-year period: • EPS and HPCS EDG frequency of start demands (demands per reactor year) • EPS and HPCS EDG frequency of FTLR demands • EPS and HPCS EDG frequency of run>1H hours. The following decreasing trends were identified for EDGs for the most recent 10-year period: • EPS EDG FTR>1H failure rate • EPS EDG unreliability • EPS and HPCS EDG frequency of FTR>1H events (failures per reactor year).

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Enhanced Component Performance Study: Emergency Diesel Generators 1998–2018

This report presents an enhanced performance evaluation of emergency diesel generators (EDGs) at U.S. commercial nuclear power plants. This report evaluates component performance over time using (1) Institute of Nuclear Power Operations (INPO) Consolidated Events Database (ICES) data from 1998 through 2018 and (2) maintenance unavailability (UA) performance data from Mitigating Systems Performance Index (MSPI) Basis Document data from 2002 through 2018. The objective is to show estimates of current failure probabilities and rates related to EDGs, trend these data on an annual basis, determine if the current data are consistent with the probability distributions currently recommended for use in NRC probabilistic risk assessments, show how the reliability data differ for different EDG manufacturers and for EDGs with different ratings; and summarize the subcomponents, causes, detection methods, and recovery associated with each EDG failure mode. Engineering analyses were performed with respect to time period and failure mode without regard to the actual number of EDGs at each plant. The factors analyzed are: sub-component, failure cause, detection method, recovery, manufacturer, and EDG rating. A statistically significant increasing trend was identified in the frequency of FTLR demands for emergency power system (EPS) and high pressure core spray (HPCS) EDGs and a statistically significant decreasing trend was identified in the frequency of run > 1H hours for EPS and HPCS EDGs.

99 GENERAL AND MISCELLANEOUS↗

Space Reclamation for Uncoordinated Checkpointing in Message-Passing Systems

Checkpointing and rollback recovery are techniques that can provide efficient recovery from transient process failures. In a message-passing system, the rollback of a message sender may cause the rollback of the corresponding receiver, and the system needs to roll back to a consistent set of checkpoints called recovery line. If the processes are allowed to take uncoordinated checkpoints, the above rollback propagation may result in the domino effect which prevents recovery line progression. Traditionally, only obsolete checkpoints before the global recovery line can be discarded, and the necessary and sufficient condition for identifying all garbage checkpoints has remained an open problem. A necessary and sufficient condition for achieving optimal garbage collection is derived and it is proved that the number of useful checkpoints is bounded by N(N+1)/2, where N is the number of processes. The approach is based on the maximum-sized antichain model of consistent global checkpoints and the technique of recovery line transformation and decomposition. It is also shown that, for systems requiring message logging to record in-transit messages, the same approach can be used to achieve optimal message log reclamation. As a final topic, a unifying framework is described by considering checkpoint coordination and exploiting piecewise determinism as mechanisms for bounding rollback propagation, and the applicability of the optimal garbage collection algorithm to domino-free recovery protocols is demonstrated.

Wang, Yi-Min↗

Motion-base simulator study of control of an externally blown flap STOL transport aircraft after failure of an outboard engine during landing approach

A moving-base simulator investigation of the problems of recovery and landing of a STOL aircraft after failure of an outboard engine during final approach was made. The approaches were made at 75 knots along a 6 deg glide slope. The engine was failed at low altitude and the option to go around was not allowed. The aircraft was simulated with each of three control systems, and it had four high-bypass-ratio fan-jet engines exhausting against large triple-slotted wing flaps to produce additional lift. A virtual-image out-the-window television display of a simulated STOL airport was operating during part of the investigation. Also, a simple heads-up flight director display superimposed on the airport landing scene was used by the pilots to make some of the recoveries following an engine failure. The results of the study indicated that the variation in visual cues and/or motion cues had little effect on the outcome of a recovery, but they did have some effect on the pilot's response and control patterns.

Middleton, D. B.↗

Compiler-assisted multiple instruction rollback recovery using a read buffer

Multiple instruction rollback (MIR) is a technique that has been implemented in mainframe computers to provide rapid recovery from transient processor failures. Hardware-based MIR designs eliminate rollback data hazards by providing data redundancy implemented in hardware. Compiler-based MIR designs have also been developed which remove rollback data hazards directly with data-flow transformations. This paper describes compiler-assisted techniques to achieve multiple instruction rollback recovery. We observe that some data hazards resulting from instruction rollback can be resolved efficiently by providing an operand read buffer while others are resolved more efficiently with compiler transformations. The compiler-assisted scheme presented consists of hardware that is less complex than shadow files, history files, history buffers, or delayed write buffers, while experimental evaluation indicates performance improvement over compiler-based schemes.

Alewine, Neal J.↗

Compiler-assisted multiple instruction rollback recovery using a read buffer

Multiple instruction rollback (MIR) is a technique that has been implemented in mainframe computers to provide rapid recovery from transient processor failures. Hardware-based MIR designs eliminate rollback data hazards by providing data redundancy implemented in hardware. Compiler-based MIR designs have also been developed which remove rollback data hazards directly with data-flow transformations. This paper focuses on compiler-assisted techniques to achieve multiple instruction rollback recovery. We observe that some data hazards resulting from instruction rollback can be resolved efficiently by providing an operand read buffer while others are resolved more efficiently with compiler transformations. A compiler-assisted multiple instruction rollback scheme is developed which combines hardware-implemented data redundancy with compiler-driven hazard removal transformations. Experimental performance evaluations indicate improved efficiency over previous hardware-based and compiler-based schemes.

Alewine, N. J.↗

Compiler-Assisted Multiple Instruction Rollback Recovery Using a Read Buffer

Multiple instruction rollback (MIR) is a technique to provide rapid recovery from transient processor failures and was implemented in hardware by researchers and slow in mainframe computers. Hardware-based MIR designs eliminate rollback data hazards by providing data redundancy implemented in hardware. Compiler-based MIR designs were also developed which remove rollback data hazards directly with data flow manipulations, thus eliminating the need for most data redundancy hardware. Compiler-assisted techniques to achieve multiple instruction rollback recovery are addressed. It is observed that data some hazards resulting from instruction rollback can be resolved more efficiently by providing hardware redundancy while others are resolved more efficiently with compiler transformations. A compiler-assisted multiple instruction rollback scheme is developed which combines hardware-implemented data redundancy with compiler-driven hazard removal transformations. Experimental performance evaluations were conducted which indicate improved efficiency over previous hardware-based and compiler-based schemes. Various enhancements to the compiler transformations and to the data redundancy hardware developed for the compiler-assisted MIR scheme are described and evaluated. The final topic deals with the application of compiler-assisted MIR techniques to aid in exception repair and branch repair in a speculative execution architecture.

Alewine, Neal Jon↗

Simulation results of automatic restructurable flight control system concepts

The restructurable flight control system (RFCS) described by Weiss et al. (1986) is reviewed, and several results of an extensive six degrees of freedom nonlinear simulation of several aspects of this system are reported. It is concluded that the nontraditional use of standard control surfaces in a nominal feedback control system to spread control authority among many redundant control elements provides a significant amount of fault tolerance without any use of restructuring techniques. The use of new feedback gains alone following a failure can provide significantly improved recovery as long as the control elements remain within their travel limits and as long as uncertainty about the failure identity is properly handled. The use of the feed-forward trim solution in conjunction with redesigned feedback gains allows recovery to take place even when significant control saturation occurs.

Weiss, J. L.↗

Holistic Measurement Driven Resilience: Combining Operational Fault and Failure Measurements and Fault Injection for Quantifying Fault Detection, Propagation and Impact. Final report

For HPC systems to date, application resilience to faults and failures has been accomplished by the brute- force method of checkpoint/restart, which allows an application to make forward progress in the face of system and application faults, errors, and failures independent of root cause or end result. It has remained the primary resilience mechanism because we lack a way to identify faults and anticipate consequences early enough to take meaningful mitigating action. However, checkpoint/restart implementations put a tremendous burden on system resources and on the applications themselves and is becoming less feasible at scale. Because we have not yet operated at scales at which checkpoint/restart fails to provide forward progress, despite increasing costs, vendors have had little motivation to provide the instrumentation necessary for early identification of faults and failures. However, as we move from petascale to exascale, component mean time to failure (MTTF) will render the existing techniques ineffectual and/or too expensive. Furthermore, fault recovery mechanisms such as failover and/or error correction introduce performance inconsistency. Instrumentation allowing early indication of problems and tools to enable use of such information by systems, operating systems, and applications offer an alternative, more scalable and less costly solution. In the HMDR project, we built on our experience and expertise developed and accumulated over years of research on design, monitoring, measurement, and assessment of resilient computing systems. Analysis of field data on the current and past generations of extreme-scale systems revealed several challenges that, if not addressed in increasingly larger and more complex systems, may hinder the effectiveness of future exascale computing systems. Specifically, i) file systems and interconnects in current-generation large-scale systems already operate at the margins of resiliency, including consistent performance, and may not scale to larger deployments; ii) automated, software-based failover mechanisms are frequently inadequate and can introduce wider failures, such that failures during recovery may lead to system/application failures, including system-wide outages; and iii) silent data corruption represents a critical fault mode and will require efficient detection mechanisms if next-generation applications are to take full advantage of exascale hardware. To address the above challenges, we assembled a team of world-renowned experts in resilient extreme- scale computing from the University of Illinois (Electrical and Computer Engineering, Computer Science, and NCSA), SNL, LANL, NERSC, and Cray. Our team includes representatives from centers that house many of the largest HPC resources in the world, both today and over the coming years. The team has a unique track record of research in i) system and application failure characterization based on the analysis of field data, ii) data-driven design of fault/error detection mechanisms, and iii) experimental characterization of system/application resiliency. The team includes system owners/operators who provide continuous data collection and access and ensure installation of appropriate analysis tools.

97 MATHEMATICS AND COMPUTING↗

A dual-mode generalized likelihood ratio approach to self-reorganizing digital flight control system design

Analytic techniques have been developed for detecting and identifying abrupt changes in dynamic systems. The GLR technique monitors the output of the Kalman filter and searches for the time that the failure occured, thus allowing it to be sensitive to new data and consequently increasing the chances for fast system recovery following detection of a failure. All failure detections are based on functional redundancy. Performance tests of the F-8 aircraft flight control system and computerized modelling of the technique are presented.

Source record↗

Probing the evolution of fault properties during the seismic cycle with deep learning

We use seismic waves that pass through the hypocentral region of the 2016 M6.5 Norcia earthquake together with Deep Learning (DL) to distinguish between foreshocks, aftershocks and time-to-failure (TTF). Binary and N-class models defined by TTF correctly identify seismograms in test with > 90% accuracy. We use raw seismic records as input to a 7 layer CNN model to perform the classification. Here we show that DL models successfully distinguish seismic waves pre/post mainshock in accord with lab and theoretical expectations of progressive changes in crack density prior to abrupt change at failure and gradual postseismic recovery. Performance is lower for band-pass filtered seismograms (below 10 Hz) suggesting that DL models learn from the evolution of subtle changes in elastic wave attenuation. Tests to verify that our results indeed provide a proxy for fault properties included DL models trained with the wrong mainshock time and those using seismic waves far from the Norcia mainshock; both show degraded performance. Our results demonstrate that DL models have the potential to track the evolution of fault zone properties during the seismic cycle. If this result is generalizable it could improve earthquake early warning and seismic hazard analysis.

58 GEOSCIENCES↗

Steady state analysis of Energy Transfer Control (ETC) and compressor bleed concepts of remote lift fan control

Two new concepts of control for VTOL aircraft remote lift fans are analyzed and discussed. Both concepts employ flow transfer among lift fans located in separate parts of the aircraft in order to obtain attitude control moments in hover and low-speed flight. The results presented are from a digital computer analysis of the YJ97/LF460 remote drive turbofan, although qualitative conclusions should apply to any remote fan system. The basic operating principles of the two systems are described, and their steady-state moment generating capabilities are discussed. Lift-recovery strategies for component failures are also analyzed.

Sellers, J. F.↗

A High Temperature Vapor Phase Lubrication Study Utilizing a Thioether Liquid Lubricant

Much of the experimental work on vapor phase lubrication has employed certain organo phosphorous compounds as the vapor phase lubricant. Graham and Klaus, for instance, used tricresyl phosphate (TCP) and tributyl phosphate to vapor phase lubricate a four-ball wear tester using M50 steel balls at 370 C. Makki and Graham were able to vapor phase lubricate a reciprocating pin on plate tribometer using 1018 steel at 280 C with TCP vapor. Although a few organo phosphorous compounds, such as TCP, have been successfully used as vapor phase lubricants in many laboratory experiments, many problems remain unsolved. Two areas of concern relate to the 'durability' of phosphate deposited films and to the ability of the lubricating system to "self-recover" when vapor phase lubricated with an organo phosphorous compound. Durability refers to the ability of the deposited film to provide effective lubrication, for a period of time, after the vapor flow to the lubricating surfaces has been interrupted. Vapor phase lubrication tests, conducted at Cleveland State University with their high temperature tribometer, revealed that when TCP vapor flow to the lubricating surfaces was interrupted the frictional coefficient of the system rapidly increased from a value less than 0.1 to a value of 0.3 which was selected as our failure point. Self-recovery means the ability of the vapor phase lubricant to reduce the frictional coefficient of the lubricating system back down to value less than 0.1 after startup of the interrupted vapor flow. Lubrication tests conducted at Cleveland State University revealed that the high temperature tribometer could not self-recover after startup of the interrupted TCP vapor flow.

Morales, Wilfredo↗

Ground System for Solar Dynamics Observatory (SDO) Mission

NASA s Goddard Space Flight Center (GSFC) has recently completed its Critical Design Review (CDR) of a new dual Ka and S-band ground system for the Solar Dynamics Observatory (SDO) Mission. SDO, the flagship mission under the new Living with a Star Program Office, is one of GSFC s most recent large-scale in-house missions. The observatory is scheduled for launch in August 2008 from the Kennedy Space Center aboard an Atlas-5 expendable launch vehicle. Unique to this mission is an extremely challenging science data capture requirement. The mission is required to capture 99.99% of available science over 95% of all observation opportunities. Due to the continuous, high volume (150 Mbps) science data rate, no on-board storage of science data will be implemented on this mission. With the observatory placed in a geo-synchronous orbit at 36,000 kilometers within view of dedicated ground stations, the ground system will in effect implement a "real-time" science data pipeline with appropriate data accounting, data storage, data distribution, data recovery, and automated system failure detection and correction to keep the science data flowing continuously to three separate Science Operations Centers (SOCs). Data storage rates of approx. 45 Tera-bytes per month are expected. The Mission Operations Center (MOC) will be based at GSFC and is designed to be highly automated. Three SOCs will share in the observatory operations, each operating their own instrument. Remote operations of a multi-antenna ground station in White Sands, New Mexico from the MOC is part of the design baseline.

Tann, Hun K.↗