Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Disk storage”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 343 records · Page 19

CMS Data transfer operations after the first years of LHC collisions

CMS experiment utilizes distributed computing infrastructure and its performance heavily depends on the fast and smooth distribution of data between different CMS sites. Data must be transferred from the Tier-0 (CERN) to the Tier-1s for processing, storing and archiving, and time and good quality are vital to avoid overflowing CERN storage buffers. At the same time, processed data has to be distributed from Tier-1 sites to all Tier-2 sites for physics analysis while Monte Carlo simulations sent back to Tier-1 sites for further archival. At the core of all transferring machinery is PhEDEx (Physics Experiment Data Export) data transfer system. It is very important to ensure reliable operation of the system, and the operational tasks comprise monitoring and debugging all transfer issues. Based on transfer quality information Site Readiness tool is used to create plans for resources utilization in the future. We review the operational procedures created to enforce reliable data delivery to CMS distributed sites all over the world. Additionally, we need to keep data and meta-data consistent at all sites and both on disk and on tape. In this presentation, we describe the principles and actions taken to keep data consistent on sites storage systems and central CMS Data Replication Database (TMDB/DBS) while ensuring fast and reliable data samples delivery of hundreds of terabytes to the entire CMS physics community.

Kaselis, Rapolas↗

An emerging network storage management standard: Media error monitoring and reporting information (MEMRI) - to determine optical tape data integrity

Sophisticated network storage management applications are rapidly evolving to satisfy a market demand for highly reliable data storage systems with large data storage capacities and performance requirements. To preserve a high degree of data integrity, these applications must rely on intelligent data storage devices that can provide reliable indicators of data degradation. Error correction activity generally occurs within storage devices without notification to the host. Early indicators of degradation and media error monitoring 333 and reporting (MEMR) techniques implemented in data storage devices allow network storage management applications to notify system administrators of these events and to take appropriate corrective actions before catastrophic errors occur. Although MEMR techniques have been implemented in data storage devices for many years, until 1996 no MEMR standards existed. In 1996 the American National Standards Institute (ANSI) approved the only known (world-wide) industry standard specifying MEMR techniques to verify stored data on optical disks. This industry standard was developed under the auspices of the Association for Information and Image Management (AIIM). A recently formed AIIM Optical Tape Subcommittee initiated the development of another data integrity standard specifying a set of media error monitoring tools and media error monitoring information (MEMRI) to verify stored data on optical tape media. This paper discusses the need for intelligent storage devices that can provide data integrity metadata, the content of the existing data integrity standard for optical disks, and the content of the MEMRI standard being developed by the AIIM Optical Tape Subcommittee.

Podio, Fernando↗

dCache: from Resilience to Quality of Service

A major goal of future dCache development will be to allow users to define file Quality of Service (QoS) in a more flexible way than currently available. This will mean implementing what might be called a QoS rule engine responsible for registering and managing time-bound QoS transitions for files or storage units. In anticipation of this extension to existing dCache capabilities, the Resilience service, which maintains on-disk replica state, needs to undergo both structural modification and generalization. This paper describes ongoing work to transform Resilience into the new architecture which will eventually support a more broadly defined file QoS.

97 MATHEMATICS AND COMPUTING↗

A File Archival System

ARCH, file archival system for DEC VAX, provides for easy offline storage and retrieval of arbitrary files on DEC VAX system. System designed to eliminate situations that tie up disk space and lead to confusion when different programers develop different versions of same programs and associated files.

Fanselow, J. L.↗

Packet Capture Solutions: PcapDB Benchmark for High-Bandwidth Capture, Storage, and Searching

PcapDB stands alone when looking at the overall field of competitors, from the cost-effective COTS hardware, to the efficient utilization of disk space that enables a longer packet history. A scalable, 100GbE-enabled system that indexes every packet and indexes flow data without complicated load-balancing requirements. The Transport Layer search and indexing approach led to patent-pending flow indexing technology, providing a specialized database system specifically optimized around providing fast flow searches. While there are a plethora of options in network packet capture, there are very few that are able to effectively manage capture rates of more than 10 Gb/s, distributed capture and querying, and a responsive user interface. By far, the primary competitor in the market place is Endace and DeepSee; in addition to meeting the technical requirements we set out in this document, they provide technical support and a fully 'appliance like' system. In terms of cost, however, our experience has been that the yearly maintenance charges alone outstrip the entire hardware cost of solutions like PcapDB. Investment in cyber security research and development is a large part of what has enabled us to build the base of knowlegable workers needed to defend government resources in the rapidly evolving cyber security landscape. We believe projects like Bro, WireCap, and Farm do more than just fill temporary gaps in our capabilities. They give allow us to build the firm foundation needed to tackle the next generation of cyber challenges. PcapDB was built with loftier ambitions than simply solving the packet capture of a single lab site, but instead to provide a robust, scaleable packet capture solution to the DOE complex and beyond.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

Multi-level Hybrid Cache: Impact and Feasibility

Storage class memories, including flash, has been attracting much attention as promising candidates fitting into in today's enterprise storage systems. In particular, since the cost and performance characteristics of flash are in-between those of DRAM and hard disks, it has been considered by many studies as an secondary caching layer underneath main memory cache. However, there has been a lack of studies of correlation and interdependency between DRAM and flash caching. This paper views this problem as a special form of multi-level caching, and tries to understand the benefits of this multi-level hybrid cache hierarchy. We reveal that significant costs could be saved by using Flash to reduce the size of DRAM cache, while maintaing the same performance. We also discuss design challenges of using flash in the caching hierarchy and present potential solutions.

99 GENERAL AND MISCELLANEOUS↗

Ultra-high density recording technologies

The Engineering Research Center in Data Storage Systems at Carnegie Mellon University in cooperation with the National Storage Industry Consortium has selected goals of achieving 10 Gbit/sq in. recording density in magnetic and magneto-optic disk recording and 1 terabyte/cubic in. magnetic tape recording technologies. This talk describes the approaches being taken and the status of research leading to these goals.

Kryder, Mark H.↗

Optimizing raid performance with cache

We live in a world of increasingly complex applications and operating systems. Information is increasing at a mind-boggling rate. The consolidation of text, voice, and imaging represents an even greater challenge for our information systems. Which forced us to address three important questions: Where do we store all this information? How do we access it? And, how do we protect it against the threat of loss or damage? Introduced in the 1980s, RAID (Redundant Arrays of Independent Disks) represents a cost-effective solution to the needs of the information age. While fulfilling expectations for high storage, and reliability, RAID is sometimes subject to criticisms in the area of performance. However, there are design elements that can significantly enhance performance. They can be subdivided into two areas: (1) RAID levels or basic architecture. And, (2) enhancement schemes such as intelligent caching, support of tagged command queuing, and use of SCSI-2 Fast and Wide features.

Bouzari, Alex↗

Horizontally scaling dChache SRM with the Terracotta platform

The dCache disk caching file system has been chosen by a majority of LHC experiments Tier 1 centers for their data storage needs. It is also deployed at many Tier 2 centers. The Storage Resource Manager (SRM) is a standardized grid storage interface and a single point of remote entry into dCache, and hence is a critical component. SRM must scale to increasing transaction rates and remain resilient against changing usage patterns. The initial implementation of the SRM service in dCache suffered from an inability to support clustered deployment, and its performance was limited by the hardware of a single node. Using the Terracotta platform, we added the ability to horizontally scale the dCache SRM service to run on multiple nodes in a cluster configuration, coupled with network load balancing. This gives site administrators the ability to increase the performance and reliability of SRM service to face the ever-increasing requirements of LHC data handling. In this paper we will describe the previous limitations of the architecture SRM server and how the Terracotta platform allowed us to readily convert single node service into a highly scalable clustered application.

99 GENERAL AND MISCELLANEOUS↗

Optimized self-designing key-value storage engine

Embodiments of the invention utilize an optimized key-value storage engine to strike the optimal balance between cloud-cost and performance and supports queries, including updates, lookups, range queries, inserts, and read-modify-writes. Cloud cost is manifested in purchasing both storage and processing resources. The improved approach has the ability to self-design and instantiate holistic configurations given a workload, a cloud budget, and optionally performance goals and a set of Service Level Agreement (SLA) specifications. A configuration reflects an optimized storage engine design in terms of, for example, the individual data structures design (in-memory and on-disk) in the engine as well as their algorithms and interactions, a cloud provider, and the exact virtual machines to be used.

Idreos, Stratos↗

An Exact Algorithm for the Linear Tape Scheduling Problem

Magnetic tapes are often considered as an outdated storage technology, yet they are still used to store huge amounts of data. Their main interests are a large capacity and a low price per gigabyte, which come at the cost of a much larger file access time than on disks. With tapes, finding the right ordering of multiple file accesses is thus key to performance. Moving the reading head back and forth along a kilometer long tape has a non-negligible cost and unnecessary movements thus have to be avoided. However, the optimization of tape request ordering has rarely been studied in the scheduling literature, much less than I/O scheduling on disks. For instance, minimizing the average service time for several read requests on a linear tape remains an open question. Therefore, in this paper, we aim at improving the quality of service experienced by users of tape storage systems, and not only the peak performance of such systems. To this end, we propose a reasonable polynomial-time exact algorithm while this problem and simpler variants have been conjectured NP-hard. We also refine the proposed model by considering U-turn penalty costs accounting for inherent mechanical accelerations. Then, we propose a low-cost variant of our optimal algorithm by restricting the solution space, yet still yielding an accurate suboptimal solution. Finally, we compare our algorithms to existing solutions from the literature on logs of the mass storage management system of a major datacenter. This allows us to assess the quality of previous solutions and the improvement achieved by our low-cost algorithm. Aiming for reproducibility, we make available the complete implementation of the algorithms used in our evaluation, alongside the dataset of tape requests that is, to the best of our knowledge, the first of its kind to be publicly released.

Honoré, Valentin↗

Composition and Realization of Source-to-Sink High-Performance Flows: File Systems, Storage, Hosts, LAN and WAN

A number of Department of Energy (DOE) science applications, involving exascale computing systems and large experimental facilities, are expected to generate large volumes of data, in the range of petabytes to exabytes, which will be transported over wide-area networks for the purpose of storage, visualization, and analysis. To support such capabilities, significant progress has been made in various components including the deployment of 100 Gbps networks with future 1 Tbps bandwidth, increases in end-host capabilities with multiple cores and buses, capacity improvements in large disk arrays, and deployment of parallel file systems such as Lustre and GPFS. High-performance source-to-sink data flows must be composed of these component systems, which requires significant optimizations of the storage-to-host data and execution paths to match the edge and long-haul network connections. In particular, end systems are currently supported by 10-40 Gbps Network Interface Cards (NIC) and 8-32 Gbps storage Host Channel Adapters (HCAs), which carry the individual flows that collectively must reach network speeds of 100 Gbps and higher. Indeed, such data flows must be synthesized using multicore, multibus hosts connected to high-performance storage systems on one side and to the network on the other side. Current experimental results show that the constituent flows must be optimally composed and preserved from storage systems, across the hosts and the networks with minimal interference. Furthermore, such a capability must be made available transparently to the science users without placing undue demands on them to account for the details of underlying systems and networks. And, this task is expected to become even more complex in the future due to the increasing sophistication of hosts, storage systems, and networks that constitute the high-performance flows. The objectives of this proposal are to (1) develop and test the component technologies and their synthesis methods to achieve source-to-sink high-performance flows, and (2) develop tools that provide these capabilities through simple interfaces to users and applications. In terms of the former, we propose to develop (1) optimization methods that align and transition multiple storage flows to multiple network flows on multicore, multibus hosts; and (2) edge and long-haul network path realization and maintenance using advanced provisioning methods including OSCARS and OpenFlow. We also propose synthesis methods that combine these individual technologies to compose high-performance flows using a collection of constituent storage-network flows, and realize them across the storage and local network connections as well as long-haul connections. We propose to develop automated user tools that profile the hosts, storage systems, and network connections; compose the source-to-sink complex flows; and set up and maintain the needed network connections. These solutions will be tested using (1) 100 Gbps connection(s) between Oak Ridge National Laboratory (ORNL) and Argonne National Laboratory (ANL) with storage systems supported by Lustre and GPFS file systems with an asymmetric connection to University of Memphis (UM); (2) ORNL testbed with multicore and multibus hosts, switches with OpenFlow capabilities, and network emulators; and (3) 100 Gbps connections from ESnet and their Openflow testbed, and other experimental connections. This proposal brings together the expertise and facilities of the two national laboratories, ORNL and ANL, and UM. It also represents a collaboration between DOE and the Department of Defense (DOD) projects at ORNL by sharing technical expertise and personnel costs, and leveraging the existing DOD Extreme Scale Systems Center (ESSC) facilities at ORNL.

97 MATHEMATICS AND COMPUTING↗

Improving the Availability of Supercomputer Job Input Data Using Temporal Replication

Storage systems in supercomputers are a major reason for service interruptions. RAID solutions alone cannot provide sufficient protection as (1) growing average disk recovery times make RAID groups increasingly vulnerable to disk failures during reconstruction, and (2) RAID does not help with higher-level faults such failed I/O nodes. This paper presents a complementary approach based on the observation that files in the supercomputer scratch space are typically accessed by batch jobs whose execution can be anticipated. Therefore, we propose to transparently, selectively, and temporarily replicate 'active' job input data by coordinating the parallel file system with the batch job scheduler. We have implemented the temporal replication scheme in the popular Lustre parallel file system and evaluated it with real-cluster experiments. Our results show that the scheme allows for fast online data reconstruction, with a reasonably low overall space and I/O bandwidth overhead.

99 GENERAL AND MISCELLANEOUS↗

Oralloy (93.2 235 U) Bare Metal Annuli And Disks

A multitude of critical experiments with highly enriched uranium metal were conducted in the 1960s and 1970s at the Oak Ridge Critical Experiments Facility (ORCEF) in support of criticality safety operations at the Y-12 Plant. These experiments served to evaluate the storage, casting, and handling limits for the Y-12 Plant while also providing data for verification of different calculation methods and associated cross-sections for nuclear criticality safety applications. These included both solid cylinders and annuli of various diameters, interacting cylinders of various diameters, parallelepipeds, and reflected cylinders and annuli. The experiments described here involve a series of delayed critical stacks of bare oralloy HEU annuli and disks. Three of these experiments consist of stacking bare HEU annuli of varying diameters to obtain critical configurations. These annuli have nominal inner and outer diameters (ID/OD) including: 7 inches (") ID – 9" OD, 9" ID – 11" OD, 11" ID – 13" OD, and 13? ID – 15" OD. The nominal heights range from 0.125" to 1.5". The three experiments themselves range from 7" – 13", 7" – 15", and 9" – 15" in diameter, respectively. The fourth experiment ranges from 7" – 11", and along with different annuli, it also includes an 11" disk and several 7" diameter disks. All four delayed critical experiments were configured and evaluated by J. T. Mihalczo, J. J. Lynn, and D. E. McCarty from December of 1962 to February 1963 with additional information in their corresponding logbook.

42 ENGINEERING↗

Permanent-File-Validation Utility Computer Program

Errors in files detected and corrected during operation. Permanent File Validation (PFVAL) utility computer program provides CDC CYBER NOS sites with mechanism to verify integrity of permanent file base. Locates and identifies permanent file errors in Mass Storage Table (MST) and Track Reservation Table (TRT), in permanent file catalog entries (PFC's) in permit sectors, and in disk sector linkage. All detected errors written to listing file and system and job day files. Program operates by reading system tables , catalog track, permit sectors, and disk linkage bytes to vaidate expected and actual file linkages. Used extensively to identify and locate errors in permanent files and enable online correction, reducing computer-system downtime.

Derry, Stephen D.↗

ICI optical data storage tape

Optical data storage tape is now a commercial reality. The world's first successful development of a digital optical tape system is complete. This is based on the Creo 1003 optical tape recorder with ICI 1012 write-once optical tape media. Several other optical tape drive development programs are underway, including one using the IBM 3480 style cartridge at LaserTape Systems. In order to understand the significance and potential of this step change in recording technology, it is useful to review the historical progress of optical storage. This has been slow to encroach on magnetic storage, and has not made any serious dent on the world's mountains of paper and microfilm. Some of the reasons for this are the long time needed for applications developers, systems integrators, and end users to take advantage of the potential storage capacity; access time and data transfer rate have traditionally been too slow for high-performance applications; and optical disk media has been expensive compared with magnetic tape. ICI's strategy in response to these concerns was to concentrate its efforts on flexible optical media; in particular optical tape. The manufacturing achievements, media characteristics, and media lifetime of optical media are discussed.

Mclean, Robert A.↗

XDD (ORNL extensions)

XDD is a disk benchmarking tool and data transfer tool. It is well known to move data particularly efficiently both to and from storage devices.

Settlemyer, Bradley W.↗

Data storage technology comparisons

The role of data storage and data storage technology is an integral, though conceptually often underestimated, portion of data processing technology. Data storage is important in the mass storage mode in which generated data is buffered for later use. But data storage technology is also important in the data flow mode when data are manipulated and hence required to flow between databases, datasets and processors. This latter mode is commonly associated with memory hierarchies which support computation. VLSI devices can reasonably be defined as electronic circuit devices such as channel and control electronics as well as highly integrated, solid-state devices that are fabricated using thin film deposition technology. VLSI devices in both capacities play an important role in data storage technology. In addition to random access memories (RAM), read-only memories (ROM), and other silicon-based variations such as PROM's, EPROM's, and EEPROM's, integrated devices find their way into a variety of memory technologies which offer significant performance advantages. These memory technologies include magnetic tape, magnetic disk, magneto-optic disk, and vertical Bloch line memory. In this paper, some comparison between selected technologies will be made to demonstrate why more than one memory technology exists today, based for example on access time and storage density at the active bit and system levels.

Katti, Romney R.↗