Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “time to failure”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 829 records · Page 46

Archetype-based Redshift Estimation for the Dark Energy Spectroscopic Instrument Survey

We present a computationally efficient galaxy archetype-based redshift estimation and spectral classification method for the Dark Energy Survey Instrument (DESI) survey. The DESI survey currently relies on a redshift fitter and spectral classifier using a linear combination of principal component analysis–derived templates, which is very efficient in processing large volumes of DESI spectra within a short time frame. However, this method occasionally yields unphysical model fits for galaxies and fails to adequately absorb calibration errors that may still be occasionally visible in the reduced spectra. Our proposed approach improves upon this existing method by refitting the spectra with carefully generated physical galaxy archetypes combined with additional terms designed to absorb data reduction defects and provide more physical models to the DESI spectra. We test our method on an extensive data set derived from the survey validation (SV) and Year 1 (Y1) data of DESI. Our findings indicate that the new method delivers marginally better redshift success for SV tiles while reducing catastrophic redshift failure by 10%–30%. At the same time, results from millions of targets from the main survey show that our model has relatively higher redshift success and purity rates (0.5%–0.8% higher) for galaxy targets while having similar success for QSOs. These improvements also demonstrate that the main DESI redshift pipeline is generally robust. Additionally, it reduces the false-positive redshift estimation by 5%–40% for sky fibers. We also discuss the generic nature of our method and how it can be extended to other large spectroscopic surveys, along with possible future improvements.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Unravelling electro-chemo-mechanical interplay in layered oxide cathode degradation in solid-state batteries

Solid-state batteries (SSBs) hold notable promise for advancing energy storage technologies. However, their commercial viability is limited by the poor cycle stability and complex degradation mechanism. This study underscores the pivotal role of electro-chemo-mechanical interactions in driving the failure of SSBs. Leveraging advanced x-ray imaging and spectroscopy techniques, we analyzed LiNi 0.8 Mn 0.1 Co 0.1 O 2 (NMC811) cathodes from cycled Li x In||Li 6 PS 5 Cl (LPSC)||NMC811 SSBs, uncovering the interplay between microstructure, chemical heterogeneity, mechanical characteristics, and electrochemical performance. Our results show that revealing electro-chemo-mechanical interactions is essential to develop strategies to suppress the degradation of SSBs. Particularly, we revisit a LiNbO 3 (LNO) coating layer to mitigate electrochemical degradation. The LNO@NMC811 cathode retains 116 milliampere-hours per gram after 200 cycles, showing excellent stability, while the uncoated NMC811 cathode keeps degrading over time, with suppressed chemical heterogeneity and mechanical failure. This work highlights the importance of synergizing advanced material design with coating techniques, ensuring uniform lithium flux and improving mechanical properties to achieve stable, high-performance SSBs.

Zheng, Xueli [SLAC National Accelerator Laboratory↗

Design of Critical Components

Critical component design is based on minimizing product failures that results in loss of life. Potential catastrophic failures are reduced to secondary failures where components removed for cause or operating time in the system. Issues of liability and cost of component removal become of paramount importance. Deterministic design with factors of safety and probabilistic design address but lack the essential characteristics for the design of critical components. In deterministic design and fabrication there are heuristic rules and safety factors developed over time for large sets of structural/material components. These factors did not come without cost. Many designs failed and many rules (codes) have standing committees to oversee their proper usage and enforcement. In probabilistic design, not only are failures a given, the failures are calculated; an element of risk is assumed based on empirical failure data for large classes of component operations. Failure of a class of components can be predicted, yet one can not predict when a specific component will fail. The analogy is to the life insurance industry where very careful statistics are book-kept on classes of individuals. For a specific class, life span can be predicted within statistical limits, yet life-span of a specific element of that class can not be predicted.

Hendricks, Robert C.↗

Lessons Learned in Systems Engineering Availability and Recommendations for Mission Technical Leaders

In spaceflight missions at Goddard Space Flight Center (GSFC), the Mission System Engineer (MSE) is the technical leader of the overall engineering team and also is the Independent Engineering Technical Authority. The responsibility of this role includes the definition of the mission design architecture, concept of operations, and mission requirements, management of risk throughout the development, and verification and validation of the final system performance and function amongst other duties. This responsibility inherently requires time management, enabling focus on a balanced development with appropriate risk. Time is the most valuable resource of the MSE. The system engineer’s availability to interact with the development team (often product or component design leads and technicians, often in different worksites) to discover and mitigate mission risks during development is key to mission success. This paper presents examples from Lunar Reconnaissance Orbiter, Landsat 9, and Neutron star Interior Composition ExploreR (NICER) which represent in-house and out-of-house hardware builds. These examples demonstrate how interactions between the Mission Systems Engineers and other project and partner engineers result in discovery of critical risks, leading to early mitigation with significant cost and performance savings. These three missions would have suffered test failures or on-orbit failures had their MSEs not set aside time to visit engineers and technicians that were working on key pieces of space flight hardware. Availability is more than just time; it is openness to listen to concerns and questions. It begins by building a level of trust in the team that it is safe to ask questions or share concerns without the fear of blame or additional workload. It also requires enabling informal conversations (over lunch, coffee, in the clean room, or at the team members desk, etc.) where key information can be exchanged, and team members may even provide an easy-to-implement mitigation idea for another subsystem. Availability is a highly valuable commodity and completely non-obvious to protect and optimize. The natural tendency of engineers is to keep themselves busy with solving problems that they know about. This paper is encouraging MSEs to resist this tendency to try and solve all the complex problems themselves and actively devote daily time to learning and solving problems that are found with informal communications with other team members.

Lessons Learned↗

Prognostic model and failure mechanisms of steam generators in Sodium-Cooled fast reactors

This paper presents a prognostic model for sodium-cooled fast reactor (SFR) steam generators (SGs). Here, the purpose of the model is to estimate the remaining useful life of SFR SGs and thus to support the decision-making of autonomous control. SFR SGs are of great interest for plant integrity due to their harsh operation environment. They operate at higher temperatures and higher coolant-to-steam pressure differences than those of current light water reactors (LWR). The severity of the SFR SG failure consequences, which include water-sodium contact, is another reason. Understanding its failure mechanisms is important for the development of its prognostic model. Based on our literature review and physics-based analysis, we concluded that creep would be a dominant degradation mode of SFR SGs due to SFR’s high-temperature and high-pressure environment. Thus, creep is the focus of the prognostic model. Various other failure modes were also investigated in this study. The mechanical fatigue due to flow-induced vibration is usually observed in early developmental phases and would not be an important issue during normal operation. The thermal fatigue due to thermal stripping is occasionally observed in other components in SFRs but does not affect SG integrity. Pure water stress corrosive crack, fretting, and so on are commonly observed in LWR SGs but are significantly less important in SFR SGs because of the high temperatures, high pressure differences, and chemical properties of liquid sodium. Based on these investigations, a prognostic model focusing on creep failures was developed. It estimates the failure probability profile by sampling the Larson-Miller parameter (LMP) and temperature and associated uncertainties through Monte Carlo methods. Two case studies were presented. The first one demonstrated the model’s capability to calculate the failure probability for a new specimen within a given time. If the tube is working under temperatures of 500C ± 3 and pressures of 15.2 MPa, which leads to LMP of 20,100 ± 50, the failure probabilities within 50, 70, and 90 years are approximately 0.1 %, 3.2 %, and 17.5 %. The second demonstrated how adjusting the workload can help to protect the integrity of the component. For an old tube reaching 99 % of its lifespan, continuing to run at temperatures of 500C ± 3 and LMP of 20,100 ± 50 leads to a failure probability of about 29.0 % within a year. If the temperature is reduced by 5C, the failure probability can be reduced to 2.5 %, and, if the pressure is also reduced such that the LMP is increased by 100, the failure probability can further be reduced to 0.12 %.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

Failure of Nd:YVO4 Amplifier Crystals

Brittle single crystals are used in NASA applications ranging from lenses to centrally heated laser slabs. Despite standard procedures to design such components, unexpected failures occasionally occur. The ICEsat-2 (Ices, cloud, and elevation satellite) employs Yttrium orthovanadate single crystals in laser amplifiers and oscillators. Although the systems are currently flying and successfully operating, some unexpected crystal fractures occurred just prior to flight. The failures were traced to poor crystal quality along with time depend chemical reaction within the system assembly that increased stress beyond expectations and promoted both fast fracture and stress corrosion. This presentation will discuss failure analysis of the crystals and the chemical reactions that promoted failures, along with corrective actions taken to reduce likelihood of fracture over the three year mission. Re-designed lasers were stored for 12 months and operated for a total of 1,000 hours without signs of degradation.

Salem, Jon↗

STS-51 pad abort. OV103-engine 2033 (ME-2) fuel flowmeter sensor open circuit

The STS-51 initial launch attempt of Discovery (OV-103) was terminated on KSC launch pad 39B on 12 Aug. 1993 at 9:12 AM E.S.T. due to a sensor redundancy failure in the liquid hydrogen system of ME-2 (Engine 2033). The event description and time line are summarized. Propellant loading was initiated on 12 Aug. 1993 at 12:00 AM EST. All space shuttle main engine (SSME) chill parameters and Launch Commit Criteria (LCC) were nominal. At engine start plus 1.34 seconds a Failure Identification (FID) was posted against Engine 2033 for exceeding the 1800 spin intra-channel (A1-A2) Fuel Flowrate sensor channel qualification limit. The engine was shut down at 1.50 seconds followed by Engines 2032 and 2030. All shut down sequences were nominal and the mission was safely aborted. SSME Avionics hardware and software performed nominally during the incident. A review of vehicle data table (VDT) data and controller software logic revealed no failure indications other than the single FID 111-101, Fuel Flowrate Intra-Channel Test Channel A disqualification. Software logic was executed according to requirements and there was no anomalous controller software operation. Immediately following the abort, a Rocketdyne/NASA failure investigation team was assembled. The team successfully isolated the failure cause to an open circuit in a Fuel Flowrate Sensor. This type of failure has occurred eight previous times in ground testing. The sensor had performed acceptably on three previous flights of the engine and SSME flight history shows 684 combined fuel flow rate sensor channel flights without failure. The disqualification of an Engine 2 (SSME No. 2033) Fuel Flowrate sensor channel was a result of an instrumentation failure and not engine performance. All other engine operations were nominal. This disqualification resulted in an engine shutdown and safe sequential shutdown of all three engines prior to ignition of the solid boosters.

Source record↗

Multifidelity Active Learning for Failure Estimation of TRISO Nuclear Fuel

The Tristructural isotropic (TRISO)-coated particle fuel is a robust nuclear fuel proposed to be used for multiple modern nuclear technologies. Therefore, characterizing its safety is vital for the reliable operation of nuclear technologies. However, the TRISO fuel failure probabilities are small and the computational model is time consuming to evaluate them using traditional Monte Carlo-type approaches. In the paper, we present a multifidelity active learning approach to efficiently estimate small failure probabilities given an expensive computational model. Active learning suggests the next best training set for optimal subsequent predictive performance and multifidelity modeling uses cheaper low-fidelity models to approximate the high-fidelity model output. After presenting the multifidelity active learning approach, we apply it to efficiently predict TRISO failure probability and make comparisons to the reference results.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

Monitoring Damage Accumulation in Ceramic Matrix Composites Using Electrical Resistivity

The electric resistance of woven SiC fiber reinforced SiC matrix composites were measured under tensile loading conditions. The results show that the electrical resistance is closely related to damage and that real-time information about the damage state can be obtained through monitoring of the resistance. Such self-sensing capability provides the possibility of on-board/in-situ damage detection and accurate life prediction for high-temperature ceramic matrix composites. Woven silicon carbide fiber-reinforced silicon carbide (SiC/SiC) ceramic matrix composites (CMC) possess unique properties such as high thermal conductivity, excellent creep resistance, improved toughness, and good environmental stability (oxidation resistance), making them particularly suitable for hot structure applications. In specific, CMCs could be applied to hot section components of gas turbines [1], aerojet engines [2], thermal protection systems [3], and hot control surfaces [4]. The benefits of implementing these materials include reduced cooling air requirements, lower weight, simpler component design, longer service life, and higher thrust [5]. It has been identified in NASA High Speed Research (HSR) program that the SiC/SiC CMC has the most promise for high temperature, high oxidation applications [6]. One of the critical issues in the successful application of CMCs is on-board or insitu assessment of the damage state and an accurate prediction of the remaining service life of a particular component. This is of great concern, since most CMC components envisioned for aerospace applications will be exposed to harsh environments and play a key role in the vehicle s safety. On-line health monitoring can enable prediction of remaining life; thus resulting in improved safety and reliability of structural components. Monitoring can also allow for appropriate corrections to be made in real time, therefore leading to the prevention of catastrophic failures. Most conventional nondestructive evaluation (NDE) techniques such as ultrasonic C-scan, x-ray, thermography, and eddy current are limited since they require structural components of complex geometry to be taken out of service for a substantial length of time for post-damage inspection and assessment. Furthermore, the typical NDE techniques are useful for identifying large interlaminar flaws, but insensitive to CMC materials flaws developed perpendicular to the surface under tensile creep conditions. There are techniques such as piezoelectric sensor [7,8], and optical fiber [9,10] that could be used for on-line health monitoring of CMC structures. However, these systems involve attaching an external sensor or putting special fibers in CMC composites, which would be problematic at high temperature applications.

Smith, Craig E.↗

Creep/Rupture Behavior of Melt-Infiltrated SiC/SiC Composites Being Investigated

The failure behavior of melt-infiltrated SiC/SiC ceramic matrix composites is under investigation at the NASA Glenn Research Center as part of NASA's Ultra-Efficient Engine Technology Program. This material was originally developed under the High Speed Research Office's Enabling Propulsion Materials Program. Creep and rupture data provide accelerated testing information to predict material behavior under engine use situations (1500 to 2400 F). This information gives insights into various material development paths to improve composites as well as improve understanding of failure mechanisms. The left figure shows the fracture surface of a CMC material following over 200 hr of testing at 2400 F. This surface demonstrates the kind of fibrous pullout desirable for maximum crack deflection, hence non-brittle failure. Microscopy suggests that creep and rupture of these materials can best be considered as a probabilistic property, rather than a material property. Fiber failure occurs first in isolated regions, while stronger adjacent fibers remain intact. The right figure shows a region where oxide deposits blur and round the fiber images. Because the oxidation kinetics of SiC are well understood, this oxide scale can be used as a measure of the length of time various regions of the composites have been exposed to the environment, hence providing vital information regarding the sequence of failure. The oxide scale in the right figure indicates an early failure of this tow of fibers, whereas adjacent tows remain oxide free, suggesting failure much later in time. The path of various cracks can be followed throughout the composite in this manner, suggesting failure mechanisms.

Hurst, Janet B.↗

MECHANISTIC MULTIPHYSICS MODELING OF CLADDING RUPTURE IN NUCLEAR FUEL RODS DURING LOSS-OF-COOLANT ACCIDENT CONDITIONS

The Loss of Coolant Accident (LOCA) is a design basis accident that is included as part of the safety analysis of nuclear power plants. As the nuclear industry desires to increase the cycle length and discharge burnup of existing nuclear power plants they must demonstrate safe operation during a LOCA on high burnup fuel. During a LOCA transient on high burnup fuel rods, the rods may undergo a process known as fuel fragmentation, relocation, and dispersal (FFRD). To permit dispersal, the cladding encapsulating the fuel must rupture with an opening size large enough to allow the fragmented fuel particles to release. Current licensing tools used by industry and the United States Nuclear Regulatory Commission are limited in geometric fidelity and materials that can be analyzed. These simulation tools generally employ a quasi-two-dimensional (1.5D or Layered1D) or 2D-RZ axisymmetric geometric representations exclusively. While a valid approach under some instances, there are times when important physics have an asymmetric behavior in the fuel rod. Examples include fuel fragmentation, thermal-hydraulic boundary conditions, and cladding rupture, all of which are important for LOCA analysis. As industry pursues burnup extensions it must be demonstrated that fuel dispersal can be mitigated or eliminated. To do this, an understanding of the rupture opening after cladding failure is required. This work presents the development of a model for predicting the size and location of the rupture opening in failed fuel rods during LOCA conditions using advanced modeling and simulation tools. In order to supply the rupture model with appropriate boundary conditions, improvements to fuel fragmentation, axial relocation and oxidation modeling were required. First, the eXtended Finite Element Method (XFEM) is used to mechanistically predict the number of fuel fragments that form due to material strength randomization, criteria for strength randomization, mesh density, power ramping rates and irradiation effects. These predictions with associated uncertainty were compared to empirical correlations developed for UO2 verifying that they can be used with increased confidence in subsequent axial relocation analyses. Secondly, a new first-of-its-kind Layered2D computational framework was developed that provides the ability to apply azimuthally varying boundary conditions while providing discrete layers to track fuel movement during the LOCA. An existing fuel axial relocation model developed for Layered1D was extended to work within the Layered2D framework. A large sensitivity study was performed on the initial version of the model to identify modeling parameters of particular importance, with the emissivity used for radiation after blowdown being the primary source of uncertainty. Then, a simplistic approach to incorporate mechanical degradation of the cladding due to oxidation was also developed to investigate the impact of reduced cladding thickness on predictions of the time to failure of cladding tubes. It was found that the cladding will typically rupture prior to a reduction in thickness that is sufficient to impact the rupture behavior. A model was then developed for predicting cladding rupture that transfers the cladding surface temperatures, rod internal and external pressures, fast neutron fluence, and fast neutron flux from a more detailed Layered1D, Layered2D, or 2D-RZ analysis to a 3D cladding only analysis. Comparisons of the rupture model to a few experiments indicate reasonable predictions. The rupture model was then applied to two accident tolerant fuel concepts (FeCrAl and Cr-coated Zircaloy) where it predicted that both ATF concepts would have smaller rupture openings and delayed rupture times than the standard Zircaloy-4 cladding material under identical loading conditions.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

The implementation and use of Ada on distributed systems with reliability requirements

The issues involved in the use of the programming language Ada on distributed systems are discussed. The effects of Ada programs on hardware failures such as loss of a processor are emphasized. It is shown that many Ada language elements are not well suited to this environment. Processor failure can easily lead to difficulties on those processors which remain. As an example, the calling task in a rendezvous may be suspended forever if the processor executing the serving task fails. A mechanism for detecting failure is proposed and changes to the Ada run time support system are suggested which avoid most of the difficulties. Ada program structures are defined which allow programs to reconfigure and continue to provide service following processor failure.

Reynolds, P. F.↗

International Space Station Mechanisms and Maintenance Flight Control Documentation and Training Development

International Space Station (ISS) crew and flight controller training documentation is used to aid in training operations. The Generic Simulations References SharePoint (Gen Sim) site is a database used as an aid during flight simulations. The Gen Sim site is used to make individual mission segment timelines, data, and flight information easily accessible to instructors. The Waste and Hygiene Compartment (WHC) training schematic includes simple and complex fluid schematics, as well as overall hardware locations. It is used as a teaching aid during WHC lessons for both ISS crew and flight controllers. ISS flight control documentation is used to support all aspects of ISS mission operations. The Quick Look Database and Consolidated Tool Page are imagery-based references used in real-time to help the Operations Support Officer (OSO) find data faster and improve discussions with the Flight Director and Capsule Communicator (CAPCOM). A Quick Look page was created for the Permanent Multipurpose Module (PMM) by locating photos of the module interior, labeling specific hardware, and organizing them in schematic form to match the layout of the PMM interior. A Tool Page was created for the Maintenance Work Area (MWA) by gathering images, detailed drawings, safety information, procedures, certifications, demonstration videos, and general facts of each MWA component and displaying them in an easily accessible and consistent format. Participation in ISS mechanisms and maintenance lessons, mission simulation On-the-Job Training (OJT), and real-time flight OJT was used as an opportunity to train for day-to-day operations as an OSO, as well as learn how to effectively respond to failures and emergencies during mission simulations and real-time flight operations.

Daugherty, Colin C.↗

Signature analysis of acoustic emission from graphite/epoxy composites

Acoustic emissions were monitored for crack extension across and parallel to the fibers in a single ply and multiply laminates of graphite epoxy composites. Spectrum analysis was performed on the transient signal to ascertain if the fracture mode can be characterized by a particular spectral pattern. The specimens were loaded to failure quasistatically in a tensile machine. Visual observations were made via either an optical microscope or a television camera. The results indicate that several types of characteristics in the time and frequency domain correspond to different types of failure.

Russell, S. S.↗

Effect of strain isolator pad modulus on inplane strain in Shuttle Orbiter thermal protection system tiles

The thermal protection system used on the Space Shuttle orbiter to determine strains in the reusable surface insulation tiles under simulated flight loads was investigated. The effects of changes in the strain isolator pad (SIP) moduli on the strains in the tile were evaluated. To analyze the SIP/tile system, it was necessary to conduct tests to determine inplane tension and compression modulus and inplane failure strain for the densified layer of the tiles. It is shown that densification of the LI-900 tile material increases the modulus by a factor of 6 to 10 and reduces the failure strain by about 50%. It is indicated that the inplane strain levels in the Shuttle tiles in the highly loaded regions are approximately 2 orders of magnitude lower than the failure strain of the material. It is concluded that most of the LI-900 tiles on the Shuttle could be mounted on a SIP with tensile and shear stiffnesses 10 times those of the present SIP without inplane strain failure in the tile.

Sawyer, J. W.↗

Characterization of Ceramic Matrix Composite Fasteners Exposed in a Combustion Liner Rig Test

Combustion tests on SiC/SiC CMC (Ceramic Matrix Composite) components were performed in an aircraft combustion environment using the Rich-burn, Quick-quench, Lean-burn (RQL) sector rig. SiC/SiC fasteners were used to attach several of these components to the metallic rig structure. The effect of combustion exposure on the fastener material was characterized via microstructural examination. Fasteners were also destructively tested, after combustion exposure, and the failure loads of fasteners exposed in the sector rig were compared to those of as-manufactured fasteners. Combustion exposure reduced the fastener failure load by 50% relative to the as-manufactured fasteners for exposure times ranging from 50 to 260 hours. The fasteners exposed in the combustion environment demonstrated failure loads that varied with failure mode. Fasteners that had the highest average failure load, failed in the same manner as the unexposed fasteners.

Verrilli, Michael J.↗

Test-to-Failure of a Two-Grid, 30-cm-dia. Ion Accelerator System

To determine the failure mechanism and erosion characteristics of an ion accelerator system due to erosion by charge-exchange ions a test was performed in which a 30-cm-diameter, 2-grid ion accelerator system was tested to failure. The erosion charcteristics observed in this test, however, imply significantly shorter accelerator grid life times than typically stated in the literature. Finally, the test suggests that structural failure is probably not the most likely first failure mechanism for the accelerator grid.

distortion↗

SMC 2021 : Analyzing Resource Utilization and User Behavior on Titan Supercomputer

Resource utilization statistics of submitted jobs on a supercomputer can help us understand how users from various scientific domains use HPC platforms and better design a job scheduler. We explore to generate insight regarding workload distribution and usage pattern domains from job scheduler trace, GPU failure information, and project-specific information collected from Titan supercomputer. Furthermore, we want to know how the scheduler performance varies over time and how the users' scheduling behavior changes following a system failure. These observations have the potential to provide valuable insight, which is helpful to prepare for system failures. These practices will help us develop and apply novel machine learning algorithms in understanding system behavior, requirement, and better scheduling of HPC systems. There are two datasets, RUR and GPU. RUR: This dataset is the job scheduler traces collected from the Titan supercomputerfrom 01/01/2015 to 07/31/2019 (2015.csv - 2019.csv). These were collected usingResource Utilization Report (RUR), a Cray-developed resource-usage data collectionand reporting system. It contains the usage information of its critical resources (CPU,Memory, GPU, and I/O) of each running job on Titan during that period [2]. ProjectAreas: Every job is associated with a project ID. TheProjectAreas.csvdatasetprovides a mapping of the project ID to its domain science. GPU: There have been some hardware-related issues in the GPUs in Titan that caused some GPUs to fail, sometimes irrecoverably during some job runs. This dataset provides information regarding these failures during the execution of the submitted jobs. GPUs on Titan are uniquely identified by a serial number (SN), and they are installed in a location. A GPU can be installed in a location, then removed from that location following a failure, and then re-installed in a different location after fixing the problem. If the failure can't be recovered, the GPU might be removed entirely from Titan. There are two prominent types of failures that resulted in the removal of GPUs from Titan: Double Bit Error (DBE) and Out of the Bus (OTB). The dataset (gc_full.csv) has the following fields: 1. SN : Serial number of a GPU 2. location : The location where it is installed 3. insert : The time when it was inserted into that location 4. remove : The time when it was removed from that location 5. duration : Amount of time the GPU spent in this location 6. out : If the device was taken out entirely w/o a re-installment into a new location. 7. event : If the GPU was taken out entirely, the reason for its removal. To learn more about this dataset, please refer to the git repositoryhttps://github.com/olcf/TitanGPULifeand the related publication [1]. References [1] George Ostrouchov, Don Maxwell, Rizwan A Ashraf, Christian Engelmann, MallikarjunShankar, and James H Rogers. Gpu lifetimes on titan supercomputer: Survival analysisand reliability. InSC20: International Conference for High Performance Computing,Networking, Storage and Analysis, pages 1-14. IEEE, 2020. [2] Feiyi Wang, Sarp Oral, Satyabrata Sen, and Neena Imam. Learning from five-yearresource-utilization data of titan system. In2019 IEEE International Conference onCluster Computing (CLUSTER), pages 1-6. IEEE, 2019.

42 ENGINEERING↗