Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Parallel file systems”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2

Evaluation of Active Storage Strategies for the Lustre Parallel File System

Active Storage provides an opportunity for reducing the amount of data movement between storage and compute nodes of a parallel filesystem such as Lustre, PVFS, etc. It allows certain types of data processing operations to be performed directly on the storage nodes of modern parallel filesystems, near the data that they manage. This is possible by exploiting the underutilized processor and memory resources of storage nodes that are implemented using general purpose servers and operating systems. In this paper, we present a novel user-space implementation of Active Storage for Lustre, and compare it to the traditional kernel-based implementation. Based on microbenchmark and application level evaluation, we show that both approaches can reduce the network traffic, and take advantage of the extra computing capacity offered by the storage nodes at the same time. However, our user-space approach has proved to be faster, more flexible, portable, and readily deployable than the kernel-space version.

99 GENERAL AND MISCELLANEOUS↗

Enhancing I/O Throughput via Efficient Routing and Placement for Large-scale Parallel File Systems

As storage systems get larger to meet the demands of petascale systems, careful planning must be applied to avoid congestion points and extract the maximum performance. In addition, the large data sets generated by such systems makes it desirable for all compute resources to have common access to this data without needing to copy it to each machine. This paper describes a method of placing I/O close to the storage nodes to minimize contention on Cray s SeaStar2+ network, and extends it to a routed Lustre configuration to gain the same benefits when running against a center-wide file system. Our experiments using half of the resources of Spider the center-wide file system at the Oak Ridge Leadership Computing Facility show that I/O write bandwidth can be improved by up to 45% (from 71.9 to 104 GB/s) for a direct-attached configuration and by 137% (47.6 GB/s to 115 GB/s) for a routed configuration. We demonstrated up to 20.7% reduction in run-time for production scientific applications. With the full Spider system, we demonstrated over 240 GB/s of aggregate bandwidth using our techniques.

99 GENERAL AND MISCELLANEOUS↗

Symmetric Active/Active Metadata Service for High Availability Parallel File Systems

High availability data storage systems are critical for many applications as research and business become more data-driven. Since metadata management is essential to system availability, multiple metadata services are used to improve the availability of distributed storage systems. Past research focused on the active/standby model, where each active service has at least one redundant idle backup. However, interruption of service and even some loss of service state may occur during a fail-over depending on the used replication technique. In addition, the replication overhead for multiple metadata services can be very high. The research in this paper targets the symmetric active/active replication model, which uses multiple redundant service nodes running in virtual synchrony. In this model, service node failures do not cause a fail-over to a backup and there is no disruption of service or loss of service state. We further discuss a fast delivery protocol to reduce the latency of the needed total order broadcast. Our prototype implementation shows that metadata service high availability can be achieved with an acceptable performance trade-off using our symmetric active/active metadata service solution.

99 GENERAL AND MISCELLANEOUS↗

Implementation and Evaluation of Active Storage in Modern Parallel File Systems

Active Storage is a technology aimed at reducing the bandwidth requirements of current supercomputing systems, and leveraging the processing power of the storage nodes used by some modern file systems. Active Storage achieves both objectives by moving certain processing tasks to the storage nodes, near the data they manage. Our proposal for Active Storage has several key features: user-space implementation which facilitates the port to different file systems, analytical model to anticipate the performance of Active Storage with respect to a traditional system, support for striped files and complex format files such as netCDF, scientific-friendly programming and runtime environment, etc.

99 GENERAL AND MISCELLANEOUS↗

Ceph Parallel File System Evaluation Report

We used Data Direct Networks’ (DDN) SFA10K as the storage backend during this evaluation. It consists of 200 SAS drives and 280 SATA drives, organized into various RAID levels by two active-active RAID controllers. The exported RAID groups by these controllers are driven by four hosts. Each host has two InfiniBand (IB) QDR connections to the storage backend. We used a single dualport Mellanox connectX IB card per host. By our calculation, this setup can saturate SFA10K’s maximum theoretical throughput (~12 GB/s). Our Ceph testbed employs a collection of testing nodes. These nodes and their roles are summarized in Table 1. In the following discussion, we use “servers”, “osd servers”, “server hosts” interchangeably. We will emphasize with “client” prefix when we want to distinguish it from above. All hosts (client and servers) were configured with Redhat 6.3 and kernel version 3.5.1 initially, and later upgraded to 3.9 (rhl-ceph image), Glibc 2.12 with syncfs support, locally patched. We used the Ceph 0.48 and 0.55 release in the initial tests, upgraded to 0.64 and then to 0.67RC for a final round of tests.

Wang, Feiyi↗

FCP: A Fast and Scalable Data Copy Tool for High Performance Parallel File Systems

The design of HPC file and storage systems has largely been driven by the requirements on capability, reliability, and capacity. However, the convergence of large-scale simulations with big data analytics have put the data, its usability, and management back on the front and center position. One of the most common and time consuming data management tasks is the transfer of very large datasets within and across file systems.In this paper, we are introducing the FCP tool, a file system agnostic copy tool designed at the OLCF for scalable and high-performance data transfers between two file system endpoints. It provides an array of interesting features such as adaptive chunking, checksumming on the fly, checkpoint and resume capabilities to handle failures, and preserving stripe information for Lustre file system among others. It is currently available on the Titan supercomputer at the OLCF. Initial tests have shown that FCP has much better and scalable performance than traditional data copy tools and it was capable of transferring petabyte-scale datasets between two Lustre file systems.

Wang, Feiyi↗

Machine Learning Based Parallel I/O Predictive Modeling: A Case Study on Lustre File Systems

Parallel I/O hardware and software infrastructure is a key contributor to performance variability for applications running on large-scale HPC systems. This variability confounds efforts to predict application performance for characterization, modeling, optimization, and job scheduling. We propose a modeling approach that improves predictive ability by explicitly treating the variability and by leveraging the sensitivity of application parameters on performance to group applications with similar characteristics. We develop a Gaussian process-based machine learning algorithm to model I/O performance and its variability as a function of application and file system characteristics. We demonstrate the effectiveness of the proposed approach using data collected from the Edison system at the National Energy Research Scientific Computing Center. The results show that the proposed sensitivity-based models are better at prediction when compared with application-partitioned or unpartitioned models. We highlight modeling techniques that are robust to the outliers that can occur in production parallel file systems. Using the developed metrics and modeling approach, we provide insights into the file system metrics that have a significant impact on I/O performance.

I/O performance variability↗

BurstMem: A high-performance burst buffer system for scientific applications

The growth of computing power on large-scale systems requires commensurate high-bandwidth I/O systems. Many parallel file systems are designed to provide fast sustainable I/O in response to applications' soaring requirements. To meet this need, a novel system is imperative to temporarily buffer the bursty I/O and gradually flush datasets to long-term parallel file systems. In this paper, we introduce the design of BurstMem, a high-performance burst buffer system. BurstMem provides a storage framework with efficient storage and communication management strategies. Our experiments demonstrate that BurstMem is able to speed up the I/O performance of scientific applications by up to 8.5× on leadership computer systems.

Wang, Teng↗