Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “software fault tolerance”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 379 records · Page 21

Fault Tolerant Communication Runtime Support for Data-Centric Programming Models

The largest supercomputers in the world today consist of hundreds of thousands of processing cores and many more other hardware components. At such scales, hardware faults are a common place necessitating the need for fault-resilient software systems. While different fault resilient models are available today, most of them focus on allowing the computational processes to survive faults. On the other hand, we have recently started investigating fault resilience techniques for data-centric programming models such as the partitioned global address space (PGAS) models. The primary difference in data-centric models is the decoupling of computation and data locality. That is, data placement is decoupled from the executing processes allowing us to view process failure (a physical node hosting a process is dead) separately from data failure (a physical node hosting data is dead). In this paper, we take a first step towards data centric fault resilience by designing and implementing a fault resilient one-sided communication runtime framework using Global Arrays and its communication system, ARMCI. The framework consists of a fault resilient process manager, low overhead and network assisted remote node fault detection module, non-data moving collective communication primitives and providing failure semantics and error codes for one-sided communication runtime systems. Our performance evaluation indicates that our framework incurs little overhead compared to state of the art designs and provides a fundamental framework of fault resiliency for PGAS models. Keywords

Vishnu, Abhinav↗

A checkpoint compression study for high-performance computing systems

As high-performance computing systems continue to increase in size and complexity, higher failure rates and increased overheads for checkpoint/restart (CR) protocols have raised concerns about the practical viability of CR protocols for future systems. Previously, compression has proven to be a viable approach for reducing checkpoint data volumes and, thereby, reducing CR protocol overhead leading to improved application performance. In this article, we further explore compression-based CR optimization by exploring its baseline performance and scaling properties, evaluating whether improved compression algorithms might lead to even better application performance and comparing checkpoint compression against and alongside other software- and hardware-based optimizations. Our results highlights are: (1) compression is a very viable CR optimization; (2) generic, text-based compression algorithms appear to perform near optimally for checkpoint data compression and faster compression algorithms will not lead to better application performance; (3) compression-based optimizations fare well against and alongside other software-based optimizations; and (4) while hardware-based optimizations outperform software-based ones, they are not as cost effective.

97 MATHEMATICS AND COMPUTING↗

Fault detection, isolation and reconfiguration in FTMP Methods and experimental results

The Fault-Tolerant Multiprocessor (FTMP) is a highly reliable computer designed to meet a goal of 10 to the -10th failures per hour and built with the objective of flying an active-control transport aircraft. Fault detection, identification, and recovery software is described, and experimental results obtained by injecting faults in the pin level in the FTMP are presented. Over 21,000 faults were injected in the CPU, memory, bus interface circuits, and error detection, masking, and error reporting circuits of one LRU of the multiprocessor. Detection, isolation, and reconfiguration times were recorded for each fault, and the results were found to agree well with earlier assumptions made in reliability modeling.

Lala, J. H.↗

The implementation and use of Ada on distributed systems with high reliability requirements

The general inadequacy of Ada for programming systems that must survive processor loss was shown. A solution to the problem was proposed in which there are no syntatic changes to Ada. The approach was evaluated using a full-scale, realistic application. The application used was the Advanced Transport Operating System (ATOPS), an experimental computer control system developed for a modified Boeing 737 aircraft. The ATOPS system is a full authority, real-time avionics system providing a large variety of advanced features. Methods of building fault tolerance into concurrent systems were explored. A set of criteria by which the proposed method will be judged was examined. Extensive interaction with personnel from Computer Sciences Corporation and NASA Langley occurred to determine the requirements of the ATOPS software. Backward error recovery in concurrent systems was assessed.

Knight, J. C.↗

Data Management Working Group report

The current flight qualification program lags technology insertion by 6 to 10 years. The objective is to develop an integrated software engineering and development environment assisted by an expert system technology. An operating system needs to be developed which is portable to the on-board computers of the year 2000. The use of ADA verses a High-Order Language; fault tolerance; fiber optics networks; communication protocols; and security are also examined and outlined.

Filardo, Edward J.↗

Expert System for UNIX System Reliability and Availability Enhancement

Highly reliable and available systems are critical to the airline industry. However, most off-the-shelf computer operating systems and hardware do not have built-in fault tolerant mechanisms, the UNIX workstation is one example. In this research effort, we have developed a rule-based Expert System (ES) to monitor, command, and control a UNIX workstation system with hot-standby redundancy. The ES on each workstation acts as an on-line system administrator to diagnose, report, correct, and prevent certain types of hardware and software failures. If a primary station is approaching failure, the ES coordinates the switch-over to a hot-standby secondary workstation. The goal is to discover and solve certain fatal problems early enough to prevent complete system failure from occurring and therefore to enhance system reliability and availability. Test results show that the ES can diagnose all targeted faulty scenarios and take desired actions in a consistent manner regardless of the sequence of the faults. The ES can perform designated system administration tasks about ten times faster than an experienced human operator. Compared with a single workstation system, our hot-standby redundancy system downtime is predicted to be reduced by more than 50 percent by using the ES to command and control the system.

Xu, Catherine Q.↗

The environmental control and life support system advanced automation project. Phase 1: Application evaluation

The Environmental Control and Life Support System (ECLSS) is a Freedom Station distributed system with inherent applicability to advanced automation primarily due to the comparatively large reaction times of its subsystem processes. This allows longer contemplation times in which to form a more intelligent control strategy and to detect or prevent faults. The objective of the ECLSS Advanced Automation Project is to reduce the flight and ground manpower needed to support the initial and evolutionary ECLS system. The approach is to search out and make apparent those processes in the baseline system which are in need of more automatic control and fault detection strategies, to influence the ECLSS design by suggesting software hooks and hardware scars which will allow easy adaptation to advanced algorithms, and to develop complex software prototypes which fit into the ECLSS software architecture and will be shown in an ECLSS hardware testbed to increase the autonomy of the system. Covered here are the preliminary investigation and evaluation process, aimed at searching the ECLSS for candidate functions for automation and providing a software hooks and hardware scars analysis. This analysis shows changes needed in the baselined system for easy accommodation of knowledge-based or other complex implementations which, when integrated in flight or ground sustaining engineering architectures, will produce a more autonomous and fault tolerant Environmental Control and Life Support System.

Dewberry, Brandon S.↗

Development and evaluation of a fault-tolerant multiprocessor (FTMP) computer. Volume 4: FTMP executive summary

The FTMP architecture is a high reliability computer concept modeled after a homogeneous multiprocessor architecture. Elements of the FTMP are operated in tight synchronism with one another and hardware fault-detection and fault-masking is provided which is transparent to the software. Operating system design and user software design is thus greatly simplified. Performance of the FTMP is also comparable to that of a simplex equivalent due to the efficiency of fault handling hardware. The FTMP project constructed an engineering module of the FTMP, programmed the machine and extensively tested the architecture through fault injection and other stress testing. This testing confirmed the soundness of the FTMP concepts.

Smith, T. B., III↗

Using Ada for a distributed, fault tolerant system

It is pointed out that advanced avionics applications increasingly require underlying machine architectures which are damage and fault tolerant, and which provide access to distributed sensors, effectors and high-throughput computational resources. The Advanced Information Processing System (AIPS), sponsored by NASA, is to provide an architecture which can meet the considered requirements. Ada was selected for implementing the AIPS system software. Advantages of Ada are related to its provisions for real-time programming, error detection, modularity and separate compilation, and standardization and portability. Chief drawbacks of this language are currently limited availability and maturity of language implementations, and limited experience in applying the language to real-time applications. The present investigation is concerned with current plans for employing Ada in the design of the software for AIPS. Attention is given to an overview of AIPS, AIPS software services, and representative design issues in each of four major software categories.

Dewolf, J. B.↗

Abnormal fault-recovery characteristics of the fault-tolerant multiprocessor uncovered using a new fault-injection methodology

An investigation was made in AIRLAB of the fault handling performance of the Fault Tolerant MultiProcessor (FTMP). Fault handling errors detected during fault injection experiments were characterized. In these fault injection experiments, the FTMP disabled a working unit instead of the faulted unit once in every 500 faults, on the average. System design weaknesses allow active faults to exercise a part of the fault management software that handles Byzantine or lying faults. Byzantine faults behave such that the faulted unit points to a working unit as the source of errors. The design's problems involve: (1) the design and interface between the simplex error detection hardware and the error processing software, (2) the functional capabilities of the FTMP system bus, and (3) the communication requirements of a multiprocessor architecture. These weak areas in the FTMP's design increase the probability that, for any hardware fault, a good line replacement unit (LRU) is mistakenly disabled by the fault management software.

Padilla, Peter A.↗

Simulating and Detecting Radiation-Induced Errors for Onboard Machine Learning

Spacecraft processors and memory are subjected to high radiation doses and therefore employ radiation-hardened components. However, these components are orders of magnitude more expensive than typical desktop components, and they lag years behind in terms of speed and size. We have integrated algorithm-based fault tolerance (ABFT) methods into onboard data analysis algorithms to detect radiation-induced errors, which ultimately may permit the use of spacecraft memory that need not be fully hardened, reducing cost and increasing capability at the same time. We have also developed a lightweight software radiation simulator, BITFLIPS, that permits evaluation of error detection strategies in a controlled fashion, including the specification of the radiation rate and selective exposure of individual data structures. Using BITFLIPS, we evaluated our error detection methods when using a support vector machine to analyze data collected by the Mars Odyssey spacecraft. We found ABFT error detection for matrix multiplication is very successful, while error detection for Gaussian kernel computation still has room for improvement.

data analysis↗

NASA Tech Briefs, April 2005

Gas-Tolerant Device Senses Electrical Conductivity of Liquid Nanoactuators Based on Electrostatic Forces on Dielectrics Replaceable Microfluidic Cartridges for a PCR Biosensor CdZnTe Image Detectors for Hard-X-Ray Telescopes High-Aperture-Efficiency Horn Antenna Full-Circle Resolver-to-Linear-Analog Converter Continuous, Full-Circle Arctangent Circuit Advanced Three-Dimensional Display System Automatic Focus Adjustment of a Microscope Topics covered include: FastScript3D - A Companion to Java 3D; Generating Mosaics of Astronomical Images; Simulating Descent and Landing of a Spacecraft; Simulating Vibrations in a Complex Loaded Structure; Rover Sequencing and Visualization Program; Software Template for Instruction in Mathematics; Support for User Interfaces for Distributed Systems; Nanostructured MnO2-Based Cathodes for Li-Ion/Polymer Cells; Multi-Layer Laminated Thin Films for Inflatable Structures; Two-Step Laser Ranging for Precise Tracking of a Spacecraft; Growing Aligned Carbon Nanotubes for Interconnections in ICs; Multilayer Composite Pressure Vessels; Texturing Blood-Glucose-Monitoring Optics Using Oxygen Beams; Fault-Tolerant Heat Exchanger; Atomic Clock Based on Opto-Electronic Oscillator; Microfocus/Polycapillary-Optic Crystallographic X-Ray Sys; Depth-Penetrating Luminescence Thermography of Thermal- Barrier Coatings; One-Dimensional Photonic Crystal Superprisms; Measuring Low-Order Aberrations in a Segmented Telescope; Mapping From an Instrumented Glove to a Robot Hand; Application of the Hilbert-Huang Transform to Financial Data; Optimizing Parameters for Deep-Space Optical Communication; and Low-Shear Microencapsulation and Electrostatic Coating.

Source record↗

Fault Propagation, EMI Propagation, and Fault Containment in Aerospace Systems

The occurrence of faults in aerospace system hardware and software have consequences ranging from minor effects to catastrophic effects, and such faults can directly affect the safety of hardware and personnel. There are many origins to fault conditions, and the hardware that is capable of still meeting its performance requirements after experiencing itself a fault is said to be fault tolerant. A fault tolerant hardware is capable of detecting, isolating, and recovering from a fault condition; and this is a subfield of control engineering. An aerospace system that has been shown to have electromagnetic compatibility (EMC) in all its subsystems and systems cannot induced faults caused by electromagnetic interference (EMI). It can be proposed that the presence of EMI (or lack of EMC) is analogous to a potential fault initiator and the effects can likewise range from minor to severe. This paper starts by addressing the consequences of hardware failure in aerospace systems from a fault perspective, because the design of fault tolerant system is a major endeavor in aerospace. To arrive to this goal the paper starts with the concepts of fault, fault propagation, and a new concept called fault containment region. The paper then proceeds to provide two very recent examples in the aircraft industry of fault propagation with catastrophic effects. The paper proceeds to introduce the concept of EMI fault containment and a brief introduction to another new concept called the EMI containment region. The paper proceeds with an example of EMI fault containment region. The paper ends with a lesson learned conclusions.

Perez, Reinaldo↗

A study of the relationship between the performance and dependability of a fault-tolerant computer

This thesis studies the relationship by creating a tool (FTAPE) that integrates a high stress workload generator with fault injection and by using the tool to evaluate system performance under error conditions. The workloads are comprised of processes which are formed from atomic components that represent CPU, memory, and I/O activity. The fault injector is software-implemented and is capable of injecting any memory addressable location, including special registers and caches. This tool has been used to study a Tandem Integrity S2 Computer. Workloads with varying numbers of processes and varying compositions of CPU, memory, and I/O activity are first characterized in terms of performance. Then faults are injected into these workloads. The results show that as the number of concurrent processes increases, the mean fault latency initially increases due to increased contention for the CPU. However, for even higher numbers of processes (less than 3 processes), the mean latency decreases because long latency faults are paged out before they can be activated.

Goswami, Kumar K.↗

Formal methods for achieving reliable software

Requirements for reliable avionic systems are discussed in terms of the effectiveness of programming methodology. The need for methods to cope with the complexity of critical real-time systems is emphasized. Some general concepts about formal methods are presented and an example is given of the SRI hierarchical development methodology taken from the executive system of the SIFT fault tolerant computer. Formal methods with alternatives are compared and the prospects for introducing formal methods into practice are considered.

Goldberg, J.↗

Validation of a fault-tolerant multiprocessor: Baseline experiments and workload implementation

In the future, aircraft must employ highly reliable multiprocessors in order to achieve flight safety. Such computers must be experimentally validated before they are deployed. This project outlines a methodology for validating reliable multiprocessors. The methodology begins with baseline experiments, which tests a single phenomenon. As experiments progress, tools for performance testing are developed. The methodology is used, in part, on the Fault Tolerant Multiprocessor (FTMP) at NASA-Langley's AIRLAB facility. Experiments are designed to evaluate the fault-free performance of the system. Presented are the results of interrupt baseline experiments performed on FTMP. Interrupt causing exception conditions were tested, and several were found to have unimplemented interrupt handling software while one had an unimplemented interrupt vector. A synthetic workload model for realtime multiprocessors is then developed as an application level performance analysis tool. Details of the workload implementation and calibration are presented. Both the experimental methodology and the synthetic workload model are general enough to be applicable to reliable multiprocessors beside FTMP.

Feather, Frank↗

Improved Re-Configurable Sliding Mode Controller for Reusable Launch Vehicle of Second Generation Addressing Aerodynamic Surface Failures and Thrust Deficiencies

In this report we present a time-varying sliding mode control (TV-SMC) technique for reusable launch vehicle (RLV) attitude control in ascent and entry flight phases. In ascent flight the guidance commands Euler roll, pitch and yaw angles, and in entry flight it commands the aerodynamic angles of bank, attack and sideslip. The controller employs a body rate inner loop and the attitude outer loop, which are separated in time-scale by the singular perturbation principle. The novelty of the TVSMC is that both the sliding surface and the boundary layer dynamics can be varied in real time using the PD-eigenvalue assignment technique. This salient feature is used to cope with control command saturation and integrator windup in the presence of severe disturbance or control effector failure, which enhances the robustness and fault tolerance of the controller. The TV-SMC is developed and tuned up for the X-33 sub-orbital technology demonstration vehicle in launch and re-entry modes. A variety of nominal, dispersion and failure scenarios have tested via high fidelity 6DOF simulations using MAVERIC/SLIM simulation software.

Shtessel, Yuri B.↗

Middleware Architecture Evaluation for Dependable Self-managing Systems

Middleware provides infrastructure support for creating dependable software systems. A specific middleware implementation plays a critical role in determining the quality attributes that satisfy a system’s dependability requirements. Evaluating a middleware architecture at an early development stage can help to pinpoint critical architectural challenges and optimize design decisions. In this paper, we present a method and its application to evaluate middleware architectures, driven by emerging architecture patterns for developing self-managing systems. Our approach focuses on two key attributes of dependability, reliability and maintainability by means of fault tolerance and fault prevention. We identify the architectural design patterns necessary to build an adaptive self-managing architecture that is capable of preventing or recovering from failures. These architectural patterns and their impacts on quality attributes create the context for middleware evaluation. Our approach is demonstrated by an example application -- failover control of a financial application on an enterprise service bus.

97 MATHEMATICS AND COMPUTING↗