Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “failure recovery”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 217 records · Page 12
All sourcesNASA NTRSDOE OSTI
NASA NTRS1989

An experimental investigation of fault tolerant software structures in an avionics application

The objective of this experimental investigation is to compare the functional performance and software reliability of competing fault tolerant software structures utilizing software diversity. In this experiment, three versions of the redundancy management software for a skewed sensor array have been developed using three diverse failure detection and isolation algorithms and incorporated into various N-version, recovery block and hybrid software structures. The empirical results show that, for maximum functional performance improvement in the selected application domain, the results of diverse algorithms should be voted before being processed by multiple versions without enforced diversity. Results also suggest that when the reliability gain with an N-version structure is modest, recovery block structures are more feasible since higher reliability can be obtained using an acceptance check with a modest reliability.

Caglayan, Alper K.↗
NASA NTRS1993

A fault-tolerant intelligent robotic control system

This paper describes the concept, design, and features of a fault-tolerant intelligent robotic control system being developed for space and commercial applications that require high dependability. The comprehensive strategy integrates system level hardware/software fault tolerance with task level handling of uncertainties and unexpected events for robotic control. The underlying architecture for system level fault tolerance is the distributed recovery block which protects against application software, system software, hardware, and network failures. Task level fault tolerance provisions are implemented in a knowledge-based system which utilizes advanced automation techniques such as rule-based and model-based reasoning to monitor, diagnose, and recover from unexpected events. The two level design provides tolerance of two or more faults occurring serially at any level of command, control, sensing, or actuation. The potential benefits of such a fault tolerant robotic control system include: (1) a minimized potential for damage to humans, the work site, and the robot itself; (2) continuous operation with a minimum of uncommanded motion in the presence of failures; and (3) more reliable autonomous operation providing increased efficiency in the execution of robotic tasks and decreased demand on human operators for controlling and monitoring the robotic servicing routines.

Marzwell, Neville I.↗
NASA NTRS2015

Failure Analysis Results and Corrective Actions Implemented for the EMU 3011 Water in the Helmet Mishap

During EVA (Extravehicular Activity) No. 23 aboard the ISS (International Space Station) on 07/16/2013 water entered the EMU (Extravehicular Mobility Unit) helmet resulting in the termination of the EVA (Extravehicular Activity) approximately 1-hour after it began. It was estimated that 1.5-L of water had migrated up the ventilation loop into the helmet, adversely impacting the astronauts hearing, vision and verbal communication. Subsequent on-board testing and ground-based TT and E (Test, Tear-down and Evaluation) of the affected EMU hardware components led to the determination that the proximate cause of the mishap was blockage of all water separator drum holes with a mixture of silica and silicates. The blockages caused a failure of the water separator function which resulted in EMU cooling water spilling into the ventilation loop, around the circulating fan, and ultimately pushing into the helmet. The root cause of the failure was determined to be ground-processing short-comings of the ALCLR (Airlock Cooling Loop Recovery) Ion Filter Beds which led to various levels of contaminants being introduced into the Filters before they left the ground. Those contaminants were thereafter introduced into the EMU hardware on-orbit during ALCLR scrubbing operations. This paper summarizes the failure analysis results along with identified process, hardware and operational corrective actions that were implemented as a result of findings from this investigation.

Steele, John↗
DOE OSTI2020

Leveraging Flexible Smart Manufacturing to Accelerate Industrial Supply Chain Recovery

Any major crisis such as the latest coronavirus disease pandemic will have a monumental effect on the worldwide economy and on the international supply chain. Much of the global supply chain relies on China, Germany, and the United States for manufacturing and distribution; this fragile system is subject to failure. Climactic changes in demand and rapid shortages in necessities are the results of the pandemic’s disruption. The recovery forecast for the ongoing pandemic is uncertain and, therefore, any recovery effort will need to be adaptable to build the supply chain resiliency. A wide adoption of flexible smart automated technologies in the manufacturing sector are helping to build robust supply chains and assisting in recovery. Here, a brief discussion of these smart manufacturing technologies is presented, along with a list of potential supply chain issues and corresponding solutions. Such smart manufacturing technologies possess the potential to empower and revolutionize the traditional manufacturing environment by enhancing its resiliency and flexibility. These smart technologies will play a crucial role in accelerating the worldwide supply chain recovery in and after a pandemic.

99 GENERAL AND MISCELLANEOUS↗
DOE OSTI2022

Towards Low-Overhead Resilience for Data Parallel Deep Learning

Data parallel techniques have been widely adopted both in academia and industry as a tool to enable scalable training of deep learning models. At scale, DL training jobs can fail due to software or hardware bugs, may need to be preempted or terminated due to unexpected events, or may perform suboptimally because they were misconfigured. Under such circumstances, there is a need to recover and/or reconfigure data-parallel DL training jobs on-the-fly, while minimizing the impact on the accuracy of the DNN model and the runtime overhead. In this regard, state-of-art techniques adopted by the HPC community mostly rely on checkpoint-restart, which inevitably leads to loss of progress, thus increasing the runtime overhead. In this paper we explore alternative techniques that exploit the properties of modern deep learning frameworks (overlapping of gradient averaging and weight updates with local gradient computations through pipeline parallelism) to reduce the overhead of resilience/elasticity. To this end we introduce a failure simulation framework and two resilience strategies (immediate mini-batch rollback and lossy forward recovery), which we study compared with checkpoint-restart approaches in a variety of settings in order to understand the trade-offs between the accuracy loss of the DNN model and the runtime overhead.

data-parallel training↗
NASA NTRS2007

Deep Space Network Antenna Logic Controller

The Antenna Logic Controller (ALC) software controls and monitors the motion control equipment of the 4,000-metric-ton structure of the Deep Space Network 70-meter antenna. This program coordinates the control of 42 hydraulic pumps, while monitoring several interlocks for personnel and equipment safety. Remote operation of the ALC runs via the Antenna Monitor & Control (AMC) computer, which orchestrates the tracking functions of the entire antenna. This software provides a graphical user interface for local control, monitoring, and identification of faults as well as, at a high level, providing for the digital control of the axis brakes so that the servo of the AMC may control the motion of the antenna. Specific functions of the ALC also include routines for startup in cold weather, controlled shutdown for both normal and fault situations, and pump switching on failure. The increased monitoring, the ability to trend key performance characteristics, the improved fault detection and recovery, the centralization of all control at a single panel, and the simplification of the user interface have all reduced the required workforce to run 70-meter antennas. The ALC also increases the antenna availability by reducing the time required to start up the antenna, to diagnose faults, and by providing additional insight into the performance of key parameters that aid in preventive maintenance to avoid key element failure. The ALC User Display (AUD) is a graphical user interface with hierarchical display structure, which provides high-level status information to the operation of the ALC, as well as detailed information for virtually all aspects of the ALC via drill-down displays. The operational status of an item, be it a function or assembly, is shown in the higher-level display. By pressing the item on the display screen, a new screen opens to show more detail of the function/assembly. Navigation tools and the map button allow immediate access to all screens.

Ahlstrom, Harlow↗
NASA NTRS1979

Fault-tolerant software

Limitations in the current capabilities for verifying programs by formal proof or by exhaustive testing have led to the investigation of fault-tolerance techniques for applications where the consequence of failure is particularly severe. Two current approaches, N-version programming and the recovery block, are described. A critical feature in the latter is the acceptance test, and a number of useful techniques for constructing these are presented. A system reliability model for the recovery block is introduced, and conclusions derived from this model that affect the design of fault-tolerant software are discussed.

Hecht, H.↗
NASA NTRS2009

Verification and Validation of Adaptive and Intelligent Systems with Flight Test Results

F-15 IFCS project goals are: a) Demonstrate Control Approaches that can Efficiently Optimize Aircraft Performance in both Normal and Failure Conditions [A] & [B] failures. b) Advance Neural Network-Based Flight Control Technology for New Aerospace Systems Designs with a Pilot in the Loop. Gen II objectives include; a) Implement and Fly a Direct Adaptive Neural Network Based Flight Controller; b) Demonstrate the Ability of the System to Adapt to Simulated System Failures: 1) Suppress Transients Associated with Failure; 2) Re-Establish Sufficient Control and Handling of Vehicle for Safe Recovery. c) Provide Flight Experience for Development of Verification and Validation Processes for Flight Critical Neural Network Software.

Burken, John J.↗
NASA NTRS1987

An evaluation of a real-time fault diagnosis expert system for aircraft applications

A fault monitoring and diagnosis expert system called Faultfinder was conceived and developed to detect and diagnose in-flight failures in an aircraft. Faultfinder is an automated intelligent aid whose purpose is to assist the flight crew in fault monitoring, fault diagnosis, and recovery planning. The present implementation of this concept performs monitoring and diagnosis for a generic aircraft's propulsion and hydraulic subsystems. This implementation is capable of detecting and diagnosing failures of known and unknown (i.e., unforseeable) type in a real-time environment. Faultfinder uses both rule-based and model-based reasoning strategies which operate on causal, temporal, and qualitative information. A preliminary evaluation is made of the diagnostic concepts implemented in Faultfinder. The evaluation used actual aircraft accident and incident cases which were simulated to assess the effectiveness of Faultfinder in detecting and diagnosing failures. Results of this evaluation, together with the description of the current Faultfinder implementation, are presented.

Schutte, Paul C.↗
NASA NTRS1973

Phenomena associated with bench and thermal-vacuum testing of super conductors - Heat pipes.

Test failures of heat pipes occur when the functional performance is unable to match the expected design limits or when the power applied to the heat pipe (in the form of heat) is distributed unevenly through the system, yielding a large thermal gradient. When a thermal gradient larger than expected is measured, it normally occurs in the evaporator or condenser sections of the pipe. Common causes include evaporator overheating, condenser dropout, noncondensable gas formation, surge and partial recovery of evaporator temperatures, masking of thermal profiles, and simple malfunctions due to leaks and mechanical failures or flaws. Examples of each of these phenomena are described along with corresponding failure analyses and corrective measures.

Marshburn, J. P.↗
DOE OSTI2025

Resilient Entanglement Distribution in a Multihop Quantum Network

The evolution of quantum networking requires architectures capable of dynamically reconfigurable entanglement distribution to meet diverse user needs and ensure tolerance against transmission disruptions. We introduce multihop quantum networks to improve network reach and resilience by enabling quantum communications across intermediate nodes, thus broadening network connectivity and increasing scalability. We present multihop two-qubit polarization-entanglement distribution within a quantum network at the Oak Ridge National Laboratory campus. Our system uses wavelength-selective switches for adaptive bandwidth management on a software-defined quantum network that integrates a quantum data plane with classical data and control planes, creating a flexible, reconfigurable mesh. Our network distributes entanglement across six nodes within three subnetworks, each located in a separate building, optimizing quantum state fidelity and transmission rate through adaptive resource management. Additionally, we demonstrate the network's resilience by implementing a link recovery approach that monitors and reroutes quantum resources to maintain service continuity despite link failures—paving the way for scalable and reliable quantum networking infrastructures.

Alshowkan, Muneer [Oak Ridge National Laboratory (↗
DOE OSTI2022

Stress-Controlled Creep–Fatigue of an Advanced Austenitic Stainless Steel at Elevated Temperatures

Creep–fatigue interaction occurs in many structural components of high-temperature systems operating under cyclic and steady-state service conditions, such as in nuclear power plants, aerospace, naval, and other industrial applications. Thus, understanding micromechanisms governing high-temperature creep–fatigue behavior is essential for safety and design considerations. In this work, stress-controlled creep–fatigue tests of advanced austenitic stainless steel (Alloy 709) were performed at a 400 MPa stress range and 750 °C with tensile hold times of 0, 60, 600, 1800, and 3600 s, followed by microstructural examinations. The creep–fatigue lifetime of the Alloy 709 was found to decrease with increasing hold time until reaching a saturation level where the number of cycles to failure did not exhibit a significant decrease. Softening behavior was observed at the beginning of the test, possibly due to the recovery of entangled dislocations and de-twining. In addition, hysteresis loops showed ratcheting behavior, although the mean stress was zero during creep–fatigue cycling, which was attributed to activity of partial dislocations. Microstructural examination of the fracture surfaces showed that fatigue failure dominated at small hold times where the cracks initiated at the surface of the sample. Larger creep cracks were found for longer hold times with a lower probability of dimpled cavities, indicating the dominance of creep deformation. The results were compared with other commonly used stainless steels, and plausible reasons for the observed responses were described.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
NASA NTRS1990

Failure analysis and modeling of a multicomputer system

This thesis describes the results of an extensive measurement-based analysis of real error data collected from a 7-machine DEC VaxCluster multicomputer system. In addition to evaluating basic system error and failure characteristics, we develop reward models to analyze the impact of failures and errors on the system. The results show that, although 98 percent of errors in the shared resources recover, they result in 48 percent of all system failures. The analysis of rewards shows that the expected reward rate for the VaxCluster decreases to 0.5 in 100 days for a 3 out of 7 model, which is well over a 100 times that for a 7-out-of-7 model. A comparison of the reward rates for a range of k-out-of-n models indicates that the maximum increase in reward rate (0.25) occurs in going from the 6-out-of-7 model to the 5-out-of-7 model. The analysis also shows that software errors have the lowest reward (0.2 vs. 0.91 for network errors). The large loss in reward rate for software errors is due to the fact that a large proportion (94 percent) of software errors lead to failure. In comparison, the high reward rate for network errors is due to fast recovery from a majority of these errors (median recovery duration is 0 seconds).

Subramani, Sujatha Srinivasan↗
NASA NTRS1995

Simulator Investigation of Pilot Aids for Helicopter Terminal Area Operations with One Engine Inoperative

Two recent piloted simulation experiments have investigated advanced display concepts applied to civil transport helicopter terminal area operations. Civil Category A helicopter operations apply to multi-engine helicopters wherein a safe recovery (land or fly out) is required in the event of a single engine failure. The investigation used the NASA Ames Research Center Vertical Motion Simulator, which has a full six degrees of freedom, to simulate the flight task as closely as possible. The goal of these experiments was to use advanced cockpit displays to improve flight safety and enhance the mission performance of Category A terminal area operations in confined areas. The first experiment investigated the use of military display formats to assist civil rotorcraft in performing a Category A takeoff in confined terminal areas. Specifically, it addressed how well a difficult hovering backup path could be followed using conventional instruments in comparison to panel mounted integrated displays. The hovering backup takeoff, which enables pilots to land back to the confined area pad in the event of an engine failure, was chosen since it is a difficult task to perform. Seven NASA and Army test pilots participated in the experiment. Evaluations, based on task performance and pilot workload, showed that an integrated display enabled the pilot to consistently achieve adequate or desired performance with reasonable pilot workload. Use of conventional instruments, however, frequently resulted in unacceptable performance (poor flight path tracking), higher pilot workload, and poor situational awareness. Although OEI landbacks were considered a visual task, the improved performance on the backup portion, in conjunction with increased situational awareness resulting from use of integrated displays, enabled the pilots to handle an engine failure and land back safely. In contrast, use of conventional instruments frequently led to excessive rates of sink at touchdown. A second simulation (in progress - July - August) is being conducted to investigate the use of advanced displays to perform vertical and short takeoffs and landings. One Engine Inoperative trajectories, which were optimized based on safety of flight restrictions, are utilized. Based on comments from the first experiment and further analytic development, appropriate fly out and approach guidance was added. Displays include conventional instruments with raw data, and the following integrated displays: multi-view and side-view hover displays based on the Apache Pilot Night Vision System, and variations of the pathway-in-the-sky displays with a flight-path-vector, a leader and flight director modifications. Panel mounted and head-up displays are being evaluated. Engine modifications have been incorporated to simulate 30 second and 2 minute contingency power ratings. Evaluations are based on task performance and pilot workload. NASA, Army, FAA, and industry test pilots participated. Details concerning the design, conduct, and the results of the experiment will be reported in the proposed paper.

Iseler, Laura↗
NASA NTRS1993

Magellan Recorder Data Recovery Algorithms

This paper describes algorithms implemented by the Magellan High Rate Processor to recover radar data corrupted by the failure of an onboard tape recorder that dropped bits. For data with error correction coding, an algorithm was developed that decodes data in the presence of bit errors and missing bits.

Error↗
NASA NTRS2006

Catastrophic fault recovery with self-reconfigurable chips

Mission critical systems typically employ multi-string redundancy to cope with possible hardware failure. Such systems are only as fault tolerant as there are many redundant strings. Once a particular critical component exhausts its redundant spares, the multi-string architecture cannot tolerate any further hardware failure. This paper aims at addressing such catastrophic faults through the use of "Self-Reconfigurable Chips" as a last resort effort to "repair" a faulty critical component.

Chau, Savio N.↗
NASA NTRS2006

Dynamics and Adaptive Control for Stability Recovery of Damaged Aircraft

This paper presents a recent study of a damaged generic transport model as part of a NASA research project to investigate adaptive control methods for stability recovery of damaged aircraft operating in off-nominal flight conditions under damage and or failures. Aerodynamic modeling of damage effects is performed using an aerodynamic code to assess changes in the stability and control derivatives of a generic transport aircraft. Certain types of damage such as damage to one of the wings or horizontal stabilizers can cause the aircraft to become asymmetric, thus resulting in a coupling between the longitudinal and lateral motions. Flight dynamics for a general asymmetric aircraft is derived to account for changes in the center of gravity that can compromise the stability of the damaged aircraft. An iterative trim analysis for the translational motion is developed to refine the trim procedure by accounting for the effects of the control surface deflection. A hybrid direct-indirect neural network, adaptive flight control is proposed as an adaptive law for stabilizing the rotational motion of the damaged aircraft. The indirect adaptation is designed to estimate the plant dynamics of the damaged aircraft in conjunction with the direct adaptation that computes the control augmentation. Two approaches are presented 1) an adaptive law derived from the Lyapunov stability theory to ensure that the signals are bounded, and 2) a recursive least-square method for parameter identification. A hardware-in-the-loop simulation is conducted and demonstrates the effectiveness of the direct neural network adaptive flight control in the stability recovery of the damaged aircraft. A preliminary simulation of the hybrid adaptive flight control has been performed and initial data have shown the effectiveness of the proposed hybrid approach. Future work will include further investigations and high-fidelity simulations of the proposed hybrid adaptive Bight control approach.

Nguyen, Nhan↗
NASA NTRS2008

Di-Isocyanate Crosslinked Aerogels with 1, 6-Bis (Trimethoxysilyl) Hexane Incorporated in Silica Backbone

Silica aerogels are desirable materials for many applications that take advantage of their light weight and low thermal conductivity. Addition of a conformal polymer coating which bonds with the amine decorated surface of the silica network improves the strength of the aerogels by as much as 200 times. Even with vast improvement in strength they still tend to undergo brittle failure due to the rigid silica backbone. We hope to increase the flexibility and elastic recovery of the silica based aerogel by altering the silica back-bone by incorporation of more flexible hexane links. To this end, we investigated the use of 1,6-bis(trimethoxysilyl)hexane (BTMSH), a polysilsesquioxane precursor3, as an additional co-reactant to prepare silica gels which were subsequently cross-linked with di-isocyanate. Previously, this approach of adding flexibility by BTMSH incorporation was demonstrated with styrene cross-linked aerogels. In our study, we varied silane concentration, mol % of silicon from BTMSH and di-isocyanate concentration by weight percent to attempt to optimize both the flexibility and the strength of the aerogels.

Vivod, Stephanie L.↗