Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Disk storage”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 271 records · Page 15

EOS at the Fermilab LHC Physics Center(LPC)

The Fermilab CMS LPC center has operated an EOS instance since initial testing began in June 2012, transitioning to production storage in 2013 with an initial capacity of 600\,TB. Today, the system provides approximately 16\,PB of storage to support the 4{,}500-core LPC user analysis cluster, which serves several hundred active CMS users at any given time. In this talk, we will provide an update on our system setup and outline near-term plans for the CMS LPC production EOS system. We will share operational experiences from running our EOS system over the past year. Additionally, we will briefly discuss our recent efforts within the Fermilab Disk Evolution Project to consider the future architecture of HEP data storage at Fermilab for the HL-LHC era.

Wu, Yujun↗

Scale tests of the new DUNE data pipeline

In preparation for the second runs of the ProtoDUNE detectors at CERN (NP02 and NP04)[1], DUNE has established a new data pipeline for bringing the data from the EHN-1 experimental hall at CERN to primary tape storage at Fermilab and CERN, and then spreading it out to a distributed disk data store at many locations around the world. This system includes a new Ingest Daemon and a new Declaration Daemon. The Rucio[2] replica catalog, and FTS3 transport are used to transport all files. All file metadata is declared to the new MetaCat[3] metadata service. All of these new components have been successfully tested at a scale equal to the expected output of the detector data acquisition system (~2-4 GB/s), and the expected network bandwidth out of the experimental hall. We present the procedure that was used to test and the results of the test.

Timm, Steven↗

NASA Earth Science Data Rescue Efforts

Historically, at the end of a NASA mission, earth and space science data were stored at NASA's National Space Science Data Center (NSSDC). The original data archive consisted of both magnetic tapes and film media. As data storage technology improved, data from later missions were stored on disks and platters and higher capacity magnetic media for online accessibility. To conserve physical space at NASA archive sites and to meet disaster recovery guidelines, historical data originally stored on magnetic tapes and film were moved to the Federal Archives and Record Center (FRC) as a temporary holding area until its long-term value was determined by NASA. All records at the FRC are controlled by the NASA Records Retention Schedule (NRRS) which determines the disposal date for each record. On that date, responsible NASA parties are notified that all scheduled records should be reviewed and assessed to determine if they continue to hold significant historical, scientific or administrative value. For Earth Science data records being held at FRC, the Earth Science Data and Information System (ESDIS) Project office is the party responsible for making the value assessment that determines which records warrant preservation and which are ready for proper disposal according to NASA guidelines. Once the data's long-term value is determined, ESDIS takes definitive steps to preserve this data for future discovery and access. Deteriorating media containing historic data of value are recalled from FRC and brought back to ESDIS. Through a tedious, laborious process, digital data are recovered and restored to modern formats with improved metadata and documentation to aid discovery. The restored digital products are then incorporated into our modern online archive, and made immediately accessible to the public. In this paper, we will discuss how we identify data-at-risk, ways to minimize data loss, how we plan for recovery, how we delegate recovery activities to our archive facilities, and how we make recovered data more accessible.

Data Systems; Social and Information sciences↗

System and method for manipulating domain pinning and reversal in ferromagnetic materials

A method for manipulating domain pinning and reversal in a ferromagnetic material comprises applying an external magnetic field to a uniaxial ferromagnetic material comprising a plurality of magnetic domains, where each domain has an easy axis oriented along a predetermined direction. The external magnetic field is applied transverse to the predetermined direction and at a predetermined temperature. The strength of the magnetic field is varied at the predetermined temperature, thereby isothermally regulating pinning of the domains. A magnetic storage device for controlling domain dynamics includes a magnetic hard disk comprising a uniaxial ferromagnetic material, a magnetic recording head including a first magnet, and a second magnet. The ferromagnetic material includes a plurality of magnetic domains each having an easy axis oriented along a predetermined direction. The second magnet is positioned adjacent to the magnetic hard disk and is configured to apply a magnetic field transverse to the predetermined direction.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

Managing People's Data

Just imagine a mass storage system that consists of a machine with 2 CPUs, 1 Gigabyte (GB) of memory, 400 GB of disk space, 16800 cartridge tapes in the automated tape silos, 88,000 tapes located in the vault, and the software to manage the system. This system is designed to be a data repository; it will always have disk space to store all the incoming data. Currently 9.14 GB of new data per day enters the system with this rate doubling each year. To assure there is always disk space available for new data, the system. has to move data reside from the expensive disk to a much less expensive medium such as the 3480 cartridge tapes. Once the data is archived to tape, it should be able to move back to disk when someone wants to access it and the data movement should be transparent to the user. Now imagine all the tasks that a system administrator must perform to keep this system running 24 hour a day, 7 days a week. Since the filesystem maintains the illusion of unlimited disk space, data that comes to the system must get moved to tapes in an efficient manner. This paper will describe the mass storage system running at the Numerical Aerodynamic Simulation (NAS) at NASA Ames Research Center in both software and hardware aspects, then it will describe all of the tasks the system administrator has to perform on this system.

Le, Diana↗

Software for Optical Archive and Retrieval (SOAR) user's guide, version 4.2

The optical disk is an emerging technology. Because it is not a magnetic medium, it offers a number of distinct advantages over the established form of storage, advantages that make it extremely attractive. They are as follows: (1) the ability to store much more data within the same space; (2) the random access characteristics of the Write Once Read Many optical disk; (3) a much longer life than that of traditional storage media; and (4) much greater data access rate. Software for Optical Archive and Retrieval (SOAR) user's guide is presented.

Davis, Charles↗

Integration experiences and performance studies of A COTS parallel archive systems

Current and future Archive Storage Systems have been asked to (a) scale to very high bandwidths, (b) scale in metadata performance, (c) support policy-based hierarchical storage management capability, (d) scale in supporting changing needs of very large data sets, (e) support standard interface, and (f) utilize commercial-off-the-shelf(COTS) hardware. Parallel file systems have been asked to do the same thing but at one or more orders of magnitude faster in performance. Archive systems continue to move closer to file systems in their design due to the need for speed and bandwidth, especially metadata searching speeds such as more caching and less robust semantics. Currently the number of extreme highly scalable parallel archive solutions is very small especially those that will move a single large striped parallel disk file onto many tapes in parallel. We believe that a hybrid storage approach of using COTS components and innovative software technology can bring new capabilities into a production environment for the HPC community much faster than the approach of creating and maintaining a complete end-to-end unique parallel archive software solution. In this paper, we relay our experience of integrating a global parallel file system and a standard backup/archive product with a very small amount of additional code to provide a scalable, parallel archive. Our solution has a high degree of overlap with current parallel archive products including (a) doing parallel movement to/from tape for a single large parallel file, (b) hierarchical storage management, (c) ILM features, (d) high volume (non-single parallel file) archives for backup/archive/content management, and (e) leveraging all free file movement tools in Linux such as copy, move, ls, tar, etc. We have successfully applied our working COTS Parallel Archive System to the current world's first petaflop/s computing system, LANL's Roadrunner, and demonstrated its capability to address requirements of future archival storage systems.

99 GENERAL AND MISCELLANEOUS↗

Integration experiments and performance studies of a COTS parallel archive system

Current and future Archive Storage Systems have been asked to (a) scale to very high bandwidths, (b) scale in metadata performance, (c) support policy-based hierarchical storage management capability, (d) scale in supporting changing needs of very large data sets, (e) support standard interface, and (f) utilize commercial-off-the-shelf (COTS) hardware. Parallel file systems have been asked to do the same thing but at one or more orders of magnitude faster in performance. Archive systems continue to move closer to file systems in their design due to the need for speed and bandwidth, especially metadata searching speeds such as more caching and less robust semantics. Currently the number of extreme highly scalable parallel archive solutions is very small especially those that will move a single large striped parallel disk file onto many tapes in parallel. We believe that a hybrid storage approach of using COTS components and innovative software technology can bring new capabilities into a production environment for the HPC community much faster than the approach of creating and maintaining a complete end-to-end unique parallel archive software solution. In this paper, we relay our experience of integrating a global parallel file system and a standard backup/archive product with a very small amount of additional code to provide a scalable, parallel archive. Our solution has a high degree of overlap with current parallel archive products including (a) doing parallel movement to/from tape for a single large parallel file, (b) hierarchical storage management, (c) ILM features, (d) high volume (non-single parallel file) archives for backup/archive/content management, and (e) leveraging all free file movement tools in Linux such as copy, move, Is, tar, etc. We have successfully applied our working COTS Parallel Archive System to the current world's first petafiop/s computing system, LANL's Roadrunner machine, and demonstrated its capability to address requirements of future archival storage systems.

99 GENERAL AND MISCELLANEOUS↗

The role of non-volatile memory from an application perspective

Current, emerging, and future NVM (non-volatile memory) technologies give us hope that we will be able to architect HPC (high performance computing) systems that initially use them in a memory and storage hierarchy, and eventually use them as the memory and storage for the system, complete with ownership and protections as a HDD-based (hard-disk-drive-based) file system provides today.

97 MATHEMATICS AND COMPUTING↗

A frontal procedure using skyline storage

A frontal-skyline method is presented which allows for compact storage while using the frontal method. It is shown that the method requires the same minimum core storage as the frontal method and fewer transfers to and from disk than the blocked-skyline method. Subroutines necessary for its implementation with finite element codes are given.

Thompson, E.↗

CCD use at Lick Observatory

The CCD detector and data handling system in regular use at Lick Observatory are described. A grism system has been installed on the automated Cassegrain spectrograph at the Shane 3-meter telescope. A very compact CCD cooling system has been developed using a commercial gas expansion refrigerator; the dewar is a cylinder 15 cm in diameter and 6 cm high. The data acquisition computer, an LSI 11/23 with 256 kbyte RAM, 160 Mbyte Winchester disk, and color video display, provides for FITS format magnetic tape storage as well as preliminary analysis of images and spectra. The 3-meter telescope spectrographic system uses a 500 x 500 pixel thinned CCD. Design information and operational experience for these detector systems are presented along with some results to illustrate the quality of the data being obtained and the current limitations of the CCD detectors and data system.

Lauer, T. R.↗

Spaceborne optical disk controller development

The current status and potential applications of an optical-disk buffer (ODB) memory system being developed by an interagency consortium including NASA and the USAF are reviewed. The design goals for the ODB include usable capacity 1 Tb, maximum data rate 1.6 Gb/s, read error rate less than 10 to the -12th, time to initial access less than 100 ms, and unlimited read/write cycles. Present efforts focus on a brassboard ODB which employs 12 14-inch magnetooptic disks and 24 nine-diode read/write heads. A typical space application of an optical disk mass memory system (ODMMS) is discussed: as communications buffer, temporary storage, and/or multiuser I/O buffer for data management on the Space Station Earth Observing System. Environmental, operational, system-architecture, and functional-separation factors; critical design issues; and standardization questions for spaceborne ODMMSs are examined in detail.

Shull, Thomas A.↗

An overview of the education and training component of RICIS

Research in education and training according to RICIS (Research Institute for Computing and Information Systems) program focuses on means to disseminate knowledge, skills, and technological advances rapidly, accurately, and effectively. A range of areas for study include: artificial intelligence, hypermedia and full-text retrieval strategies, use of mass storage and retrieval options such as CD-ROM and laser disks, and interactive video and interactive media presentations.

Freedman, Glenn B.↗

An interactive environment for the analysis of large Earth observation and model data sets

We propose to develop an interactive environment for the analysis of large Earth science observation and model data sets. We will use a standard scientific data storage format and a large capacity (greater than 20 GB) optical disk system for data management; develop libraries for coordinate transformation and regridding of data sets; modify the NCSA X Image and X Data Slice software for typical Earth observation data sets by including map transformations and missing data handling; develop analysis tools for common mathematical and statistical operations; integrate the components described above into a system for the analysis and comparison of observations and model results; and distribute software and documentation to the scientific community.

Bowman, Kenneth P.↗

An interactive environment for the analysis of large Earth observation and model data sets

We propose to develop an interactive environment for the analysis of large Earth science observation and model data sets. We will use a standard scientific data storage format and a large capacity (greater than 20 GB) optical disk system for data management; develop libraries for coordinate transformation and regridding of data sets; modify the NCSA X Image and X DataSlice software for typical Earth observation data sets by including map transformations and missing data handling; develop analysis tools for common mathematical and statistical operations; integrate the components described above into a system for the analysis and comparison of observations and model results; and distribute software and documentation to the scientific community.

Bowman, Kenneth P.↗

Test methods for optical disk media characteristics (for 356 mm ruggedized magneto-optic media)

Standard test methods for computer storage media characteristics are essential and allow for conformance to media interchange standards. The test methods were developed for 356 mm two-sided laminated glass substrate with a magneto-optic active layer media technology. These test methods may be used for testing other media types, but in each case their applicability must be evaluated. Test methods are included for a series of different media characteristics, including operational, nonoperational, and storage environments; mechanical and physical characteristics; and substrate, recording layer, and preformat characteristics. Tests for environmental qualification and media lifetimes are also included. The best methods include testing conditions, testing procedures, a description of the testing setup, and the required calibration procedures.

Podio, Fernando L.↗

RAID Disk Arrays for High Bandwidth Applications

High bandwidth applications require large amounts of data transferred to/from storage devices at extremely high data rates. Further, these applications often are 'real time' in which access to the storage device must take place on the schedule of the data source, not the storage. A good example is a satellite downlink - the volume of data is quite large and the data rates quite high (dozens of MB/sec). Further, a telemetry downlink must take place while the satellite is overhead. A storage technology which is ideally suited to these types of applications is redundant arrays of independent discs (RAID). Raid storage technology, while offering differing methodologies for a variety of applications, supports the performance and redundancy required in real-time applications. Of the various RAID levels, RAID-3 is the only one which provides high data transfer rates under all operating conditions, including after a drive failure.

Moren, Bill↗