Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Disk failure”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records

Alpine disk failure dataset

This dataset consists of 4114 disk failure events collected from Alpine from 07/01/2019 to 04/18/2022. Individual events are identified by their timestamp in ISO 8601 format, rack and enclosure where the failed disk is located.

97 MATHEMATICS AND COMPUTING↗

Collection of Disk Failure Events from Alpine, the Parallel File System for Summit Supercomputer

This dataset contains disk (HDD) failure events collected from the Alpine storage system of the Summit supercomputer, hosted at OLCF, spanning from January 4, 2019, to December 21, 2023 (a total of 4 years, 11 months, and 18 days), covering 89% of its operational lifetime. It includes 3,766 disk failure events, each recorded with its detection timestamp (in ISO 8601 format) and detailed by its location within the storage system - rack, enclosure, and drive slot number.

97 MATHEMATICS AND COMPUTING↗

Disk Failure Dataset from the Campaign Storage System

This dataset consists of 1,389 disk (HDD) failure events collected from the Campaign storage system at LANL. The Campaign system supported various compute platforms throughout its lifespan, including Cielo, Fire, Ice, and notably, the Trinity supercomputer. Each recorded event includes its detection timestamp (in ISO 8601 format) and details such as its location within the storage system—rack, enclosure, and drive slot number. The data, spanning from May 4, 2021, to July 25, 2023 (2 years, 2 months, and 22 days), represents failure events from the terminal years of Campaign's operational period, accounting for 26% of its total operational time.

97 MATHEMATICS AND COMPUTING↗

From Failure to Insight: Analyzing Disk Breakdowns in Large-Scale HPC Environments

Disk failure data provides valuable insights for preventing failures, enhancing storage robustness, guiding system design and deployment, and ensuring reliable operations at data centers. This paper introduces two disk failure datasets collected from large-scale HPC production environments over the past five years, comprising over 5,000 failure records from more than 40,000 disks. We analyzed these datasets across multiple dimensions, including temporal, spatial, and relational trends, and performed a comprehensive reliability assessment. Our analysis yielded numerous observations and insights that influence various operational aspects of HPC storage systems. We believe this study offers a holistic understanding of disk failure trends likely to interest the HPC storage community.

George, Anjus↗

Predicting Computer System Failures Using Support Vector Machines

Mitigating the impact of computer failure is possible if accurate failure predictions are provided. Resources, applications, and services can be scheduled around predicted failure and limit the impact. Such strategies are especially important for multi-computer systems, such as compute clusters, that experience a higher rate failure due to the large number of components. However providing accurate predictions with sufficient lead time remains a challenging problem. This paper describes a new spectrum-kernel Support Vector Machine (SVM) approach to predict failure events based on system log files. These files contain messages that represent a change of system state. While a single message in the file may not be sufficient for predicting failure, a sequence or pattern of messages may be. The approach described in this paper will use a sliding window (sub-sequence) of messages to predict the likelihood of failure. The frequency representation of the message sub-sequences observed are then used as input to the SVM that associates the messages to a class of failed or non-failed system. Experimental results using actual system log files from a Linux-based compute cluster indicate the proposed SVM approach can predict hard disk failure with an accuracy of 76% one day in advance.

99 GENERAL AND MISCELLANEOUS↗

Integrated Monitoring of Macroalgae Farms Using Acoustics and UUV Sensing

The vision of this project was to develop an integrated system for autonomous underwater vehicle (AUV) monitoring of offshore kelp farms using acoustic, environmental, and optical sensors. This project supports the overall MARINER goals of developing an offshore kelp aquaculture industry to produce low-carbon or carbon-neutral biofuels. The project commenced in the spring of 2018 and used laboratory experiments to test the efficacy of acoustic sensors for monitoring kelp farm lines and growing kelp biomass. Sensors were then integrated onto two AUVs as well as an autonomous surface vehicle in order to establish the optimal type, price point, and vehicle to most efficiently monitor kelp farm structures, kelp biomass, and the surrounding environment. Multiple field deployments of these vehicles and sensors in Massachusetts, New Hampshire, and Maine confirmed that farm lines and kelp could be visualized and that monitoring the spatial patterns of environmental variables was possible. The COVID-19 pandemic severely limited fieldwork activities and laboratory testing, with deployments around kelp farms not occurring again until January 2021. Time spent away from the field focused on data visualization and the development of a low-cost, vessel-based sensor system. Unfortunately, the low-cost system experienced a disk failure during its first deployment and due to multiple resignations from the project team further engineering and development was not possible. Additionally, final results from acoustic sensor testing in 2021 were also not able to be completed due to the resignation of the postdoc leading the analysis. Despite these setbacks, multiple avenues for further development of optical imagery processing from the 360-degree Kelpcam camera and testing of the low-cost sensor system may be possible.

09 BIOMASS FUELS↗

Improving the Availability of Supercomputer Job Input Data Using Temporal Replication

Storage systems in supercomputers are a major reason for service interruptions. RAID solutions alone cannot provide sufficient protection as (1) growing average disk recovery times make RAID groups increasingly vulnerable to disk failures during reconstruction, and (2) RAID does not help with higher-level faults such failed I/O nodes. This paper presents a complementary approach based on the observation that files in the supercomputer scratch space are typically accessed by batch jobs whose execution can be anticipated. Therefore, we propose to transparently, selectively, and temporarily replicate 'active' job input data by coordinating the parallel file system with the batch job scheduler. We have implemented the temporal replication scheme in the popular Lustre parallel file system and evaluated it with real-cluster experiments. Our results show that the scheme allows for fast online data reconstruction, with a reasonably low overall space and I/O bandwidth overhead.

99 GENERAL AND MISCELLANEOUS↗

MLEC-Sim: A Simulator for Evaluating Multi-Level Erasure Coding

We present MLEC-Sim, a sophisticated simulator for Multi-Level Erasure Coding (MLEC), developed in approximately 13 KLOC. The simulator is engineered to analyze the impact of various system configurations and erasure coding policies on system durability and network overhead. It supports a comprehensive range of parameters including disk capacity, disk I/O bandwidth, failure rates, network bandwidth, and system scale, accommodating various erasure coding approaches such as Single-Level Erasure Coding (SLEC), Multi-Level Erasure Coding (MLEC), and Local Reconstruction Codes (LRC). MLEC-Sim provides support for multiple chunk placement policies, including clustered parity and declustered parity, and encompasses a variety of repair methods like Repair-ALL, Repair-FCO, Repair-HYB, and Repair-MIN. It is capable of simulating disk failures through a variety of means, including distribution-based or trace-based mechanisms, and can handle complex multi-level (de)clustered placements and repair processes. A key feature of MLEC-Sim is its adoption of the splitting simulation method for evaluating system durabilities at extremely high levels, which are challenging to assess with traditional simulation approaches. This feature allows for a detailed evaluation of system resilience under a range of conditions, aiding in the selection of appropriate erasure coding solutions for enhancing system durability. MLEC-Sim contributes to the field of data storage and reliability by providing a tool for the detailed evaluation of the durability and efficiency of erasure coding configurations, intended for use by researchers and practitioners in the design and optimization of storage systems.

Wang, Meng↗

Triggering collapse of the presolar dense cloud core and injecting short-lived radioisotopes with a shock wave. III. Rotating three-dimensional cloud cores

A key test of the supernova triggering and injection hypothesis for the origin of the solar system's short-lived radioisotopes is to reproduce the inferred initial abundances of these isotopes. We present here the most detailed models to date of the shock wave triggering and injection process, where shock waves with varied properties strike fully three-dimensional, rotating, dense cloud cores. The models are calculated with the FLASH adaptive mesh hydrodynamics code. Three different outcomes can result: triggered collapse leading to fragmentation into a multiple protostar system; triggered collapse leading to a single protostar embedded in a protostellar disk; or failure to undergo dynamic collapse. Shock wave material is injected into the collapsing clouds through Rayleigh-Taylor fingers, resulting in initially inhomogeneous distributions in the protostars and protostellar disks. Cloud rotation about an axis aligned with the shock propagation direction does not increase the injection efficiency appreciably, as the shock parameters were chosen to be optimal for injection even in the absence of rotation. For a shock wave from a core-collapse supernova, the dilution factors for supernova material are in the range of ∼10{sup –4} to ∼3 × 10{sup –4}, in agreement with recent laboratory estimates of the required amount of dilution for {sup 60}Fe and {sup 26}Al. We conclude that a type II supernova remains as a promising candidate for synthesizing the solar system's short-lived radioisotopes shortly before their injection into the presolar cloud core by the supernova's remnant shock wave.

79 ASTRONOMY AND ASTROPHYSICS↗

In situ environmental scanning electron microscope observations of colloidal failure mechanisms using the Brazilian disk test

This paper presents data produced using a novel test fixture to study the crack initiation and fracture mechanisms in brittle colloidal systems, in situ in an Environmental Scanning Electron Microscope (ESEM), based on the Brazilian disk test. The test is based on diametral compression loading of a disk of material, which causes a tensile stress in the centre of the disk perpendicular to the loading direction. Key advantages over commercial tensile tests are simplicity of specimen preparation, especially important for fragile and brittle materials, which because of their nature in many cases cannot be microscopically characterised after testing. The test was successful in creating the desired perpendicular tensile stresses and yielded diametral cracks across the latex specimens. It was found that when end crushing was minimal, multiple cracks initiated at several points on the disk and a number of factors appeared to influence the propagation of cracks through the latex, these include: particle packing arrangements, surface defects, and the presence of other cracks. Following the onset of fracture, it was observed that the cracks connected to form a single failure route, here described as the path of least resistance. This, in essence, resulted in the crack taking a route around stronger, more ordered regions of the latex (be they “crystals” or surface defects). The experimental results presented and discussed here represent the first attempt to utilise this loading arrangement in the study of crack propagation in colloidal systems.

42 ENGINEERING↗

A Layered Solution for Supercomputing Storage

To solve the supercomputing challenge of memory keeping up with processing speed, a team at Los Alamos National Laboratory developed two innovative memory management and storage technologies. Burst buffers peel off data onto flash memory to support the checkpoint/restart paradigm of large simulations. MarFS adds a thin software layer enabling a new tier for campaign storage—based on inexpensive, failure-prone disk drives—between disk drives and tape archives.

97 MATHEMATICS AND COMPUTING↗

The Chemical Abundance Structure of the Inner Milky Way: A Signature of “Upside-down” Disk Formation

We present a model for the [α/Fe]--[Fe/H] distribution of stars in the inner Galaxy, 3 kpc<R<5 kpc, measured as a function of vertical distance |z| from the midplane by Hayden et al. (H15). Motivated by an “upside-down” scenario for thick disk formation, in which the thickness of the star-forming gas layer contracts as the stellar mass of the disk grows, we combine one-zone chemical evolution with a simple prescription in which the scale-height of the stellar distribution drops linearly from z{sub h}=0.8 kpc to z{sub h}=0.2 kpc over a timescale t {sub c}, remaining constant thereafter. We assume a linear-exponential star formation history, M-dot {sub ∗}(t)∝te{sup −t/t{sub s}{sub f}}. With a star formation efficiency timescale τ{sub ∗}=M{sub g}(t)/ M-dot {sub ∗}(t)=2 Gyr, an outflow mass-loading factor η= M-dot {sub out}(t)/ M-dot {sub ∗}(t)=1.5, t{sub sf}=3 Gyr, and t{sub c}=2.5 Gyr, the model reproduces the observed locus of inner disk stars in [α/Fe]--[Fe/H] and the metallicity distribution functions (MDFs) measured by H15 at |z|=0--0.5 kpc, 0.5--1 kpc, and 1--2 kpc. Substantial changes to model parameters lead to disagreement with the H15 data; for example, models with t{sub c}=1 Gyr or t{sub sf}=1 Gyr fail to match the observed MDF at high-|z| and low-|z|, respectively. The inferred scale-height evolution, with z {sub h}(t) dropping on a timescale t{sub c}∼t{sub sf} at large lookback times, favors upside-down formation over dynamical heating of an initially thin stellar population as the primary mechanism regulating disk thickness. The failure of our short-t {sub c} models suggests that any model in which thick disk formation is a discrete event will not reproduce the continuous dependence of the MDF on |z| found by H15. Our scenario for the evolution of the inner disk can be tested by future measurements of the |z|-distribution and the age–metallicity distribution at R=3--5 kpc.

79 ASTRONOMY AND ASTROPHYSICS↗

Failing in place for low-serviceability storage infrastructure using high-parity GPU-based RAID.

In order to provide large quantities of high-reliability disk-based storage, it has become necessary to aggregate disks into fault-tolerant groups based on the RAID methodology. Most RAID levels do provide some fault tolerance, but there are certain classes of applications that require increased levels of fault tolerance within an array. Some of these applications include embedded systems in harsh environments that have a low level of serviceability, or uninhabited data centers servicing cloud computing. When describing RAID reliability, the Mean Time To Data Loss (MTTDL) calculations will often assume that the time to replace a failed disk is relatively low, or even negligible compared to rebuild time. For platforms that are in remote areas collecting and processing data, it may be impossible to access the system to perform system maintenance for long periods. A disk may fail early in a platform's life, but not be replaceable for much longer than typical for RAID arrays. Service periods may be scheduled at intervals on the order of months, or the platform may not be serviced until the end of a mission in progress. Further, this platform may be subject to extreme conditions that can accelerate wear and tear on a disk, requiring even more protection from failures. We have created a high parity RAID implementation that uses a Graphics Processing Unit (GPU) to compute more than two blocks of parity information per stripe, allowing extra parity to eliminate or reduce the requirement for rebuilding data between service periods. While this type of controller is highly effective for RAID 6 systems, an important benefit is the ability to incorporate more parity into a RAID storage system. Such RAID levels, as yet unnamed, can tolerate the failure of three or more disks (depending on configuration) without data loss. While this RAID system certainly has applications in embedded systems running applications in the field, similar benefits can be obtained for servers that are engineered for storage density, with less regard for serviceability or maintainability. A storage brick can be designed to have a MTTDL that extends well beyond the useful lifetime of the hardware used, allowing the disk subsystem to require less service throughout the lifetime of a compute resource. This approach is similar to the Xiotech ISE. Such a design can be deliberately placed remotely (without frequent support) in order to provide colocation, or meet cost goals. For workloads where reliability is key, but conditions are sub-optimal for routine serviceability, a high-parity RAID can provide extra reliability in extraordinary situations. For example, for installations requiring very high Mean Time To Repair, the extra parity can eliminate certain problems with maintaining hot spares, increasing overall reliability. Furthermore, in situations where disk reliability is reduced because of harsh conditions, extra parity can guard against early data loss due to lowered Mean Time To Failure. If used through an iSCSI interface with a streaming workload, it is possible to gain all of these benefits without impacting performance.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Mean PB To Failure - Initial results from a long-term study of disk storage patterns at the RACF

The RACF (RHIC-ATLAS Computing Facility) has operated a large, multi-purpose dedicated computing facility since the mid-l 990's, serving a worldwide, geographically diverse scientific community that is a major contributor to various HEPN projects. A central component of the RACF is the Linux-based worker node cluster that is used for both computing and data storage purposes. It currently has nearly 50,000 computing cores and over 23 PB of storage capacity distributed over 12,000+ (non-SSD) disk drives. The majority of the 12,000+ disk drives provide a cost-effective solution for dCache/XRootD-managed storage, and a key concern is the reliability of this solution over the lifetime of the hardware, particularly as the number of disk drives and the storage capacity of individual drives grow. We report initial results of a long-term study to measure lifetime PB read/written to disk drives in the worker node cluster. We discuss the historical disk drive mortality rate, disk drive manufacturers' published MPTF (Mean PB to Failure) data and how they are correlated to our results. The results help the RACF understand the productivity and reliability of its storage solutions and have implications for other highly-available storage systems (NFS, GPFS, CVMFS, etc) with large I/0 requirements.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Dynamic File Striping and Data Layout Transformation on Parallel System with Fluctuating I/O Workload

As the number of compute cores on modern parallel machines increases to more than hundreds of thousands, scalable and consistent I/O performance is becoming hard to obtain due to fluctuating file system performance. This fluctuation is often caused by rebuilding RAID disk from hardware failures or concurrent jobs competing for I/O. We present a mechanism that stripes across a dynamically-selected subset of I/O servers with the lightest workload to achieve the best I/O bandwidth available from the system. We implement this mechanism into an I/O software layer that enables memory-to-file data layout transformation and allows transparent file partitioning. File partitioning is a technique that divides data among a set of files and manages file access, making data appear as a single file to users. Experimental results on NERSC’s Hopper indicate that our approach effectively isolates I/O variation on shared systems and improves overall I/O performance significantly.

Son, Seung Woo↗