Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Disk storage”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 289 records · Page 16

SAM-FS: LSC's New Solaris-Based Storage Management Product

SAM-FS is a full featured hierarchical storage management (HSM) device that operates as a file system on Solaris-based machines. The SAM-FS file system provides the user with all of the standard UNIX system utilities and calls, and adds some new commands, i.e. archive, release, stage, sls, sfind, and a family of maintenance commands. The system also offers enhancements such as high performance virtual disk read and write, control of the disk through an extent array, and the ability to dynamically allocate block size. SAM-FS provides 'archive sets' which are groupings of data to be copied to secondary storage. In practice, as soon as a file is written to disk, SAM-FS will make copies onto secondary media. SAM-FS is a scalable storage management system. The system can manage millions of files per system, though this is limited today by the speed of UNIX and its utilities. In the future, a new search algorithm will be implemented that will remove logical and performance restrictions on the number of files managed.

Angell, Kent↗

Theoretical and Experimental Study of Current from Non-Disintegrable Suspended Particles at a Rotating Disk Electrode

Understanding the current response at an electrode from suspended solid particles in an electrolyte is crucial for developing materials to be used in semi-solid electrodes for energy storage applications. Here, an analytical model is proposed to predict and understand the current response from non-disintegrable solid particles at a rotating disk electrode. The current is shown to be limited by a combination of ion diffusion within the solid particle and the mean residence time of the particle at the rotating disk electrode. This results in a relationship between current and angular frequency of I ∝ ω 3 / 4 , instead of the classical I ∝ ω 1 / 2 predicted by Levich theory. Specifically, the current response of Li 4 Ti 5 O 12 (LTO) microparticles suspended in a non-aqueous electrolyte of lithium hexafluorophosphate (LiPF 6 ) in ethylene carbonate: diethyl carbonate (EC:DEC) was determined experimentally and compared favorably with predictions from the proposed analytical model using fitting parameters consistent with the experimental conditions.

25 ENERGY STORAGE↗

Investigation of storage options for scientific computing on Grid and Cloud facilities

In recent years, several new storage technologies, such as Lustre, Hadoop, OrangeFS, and BlueArc, have emerged. While several groups have run benchmarks to characterize them under a variety of configurations, more work is needed to evaluate these technologies for the use cases of scientific computing on Grid clusters and Cloud facilities. This paper discusses our evaluation of the technologies as deployed on a test bed at FermiCloud, one of the Fermilab infrastructure-as-a-service Cloud facilities. The test bed consists of 4 server-class nodes with 40 TB of disk space and up to 50 virtual machine clients, some running on the storage server nodes themselves. With this configuration, the evaluation compares the performance of some of these technologies when deployed on virtual machines and on bare metal nodes. In addition to running standard benchmarks such as IOZone to check the sanity of our installation, we have run I/O intensive tests using physics-analysis applications. This paper presents how the storage solutions perform in a variety of realistic use cases of scientific computing. One interesting difference among the storage systems tested is found in a decrease in total read throughput with increasing number of client processes, which occurs in some implementations but not others.

Garzoglio, Gabriele↗

A distributed parallel storage architecture and its potential application within EOSDIS

We describe the architecture, implementation, use of a scalable, high performance, distributed-parallel data storage system developed in the ARPA funded MAGIC gigabit testbed. A collection of wide area distributed disk servers operate in parallel to provide logical block level access to large data sets. Operated primarily as a network-based cache, the architecture supports cooperation among independently owned resources to provide fast, large-scale, on-demand storage to support data handling, simulation, and computation.

Johnston, William E.↗

RAID Unbound: Storage Fault Tolerance in a Distributed Environment

Mirroring, data replication, backup, and more recently, redundant arrays of independent disks (RAID) are all technologies used to protect and ensure access to critical company data. A new set of problems has arisen as data becomes more and more geographically distributed. Each of the technologies listed above provides important benefits; but each has failed to adapt fully to the realities of distributed computing. The key to data high availability and protection is to take the technologies' strengths and 'virtualize' them across a distributed network. RAID and mirroring offer high data availability, which data replication and backup provide strong data protection. If we take these concepts at a very granular level (defining user, record, block, file, or directory types) and them liberate them from the physical subsystems with which they have traditionally been associated, we have the opportunity to create a highly scalable network wide storage fault tolerance. The network becomes the virtual storage space in which the traditional concepts of data high availability and protection are implemented without their corresponding physical constraints.

Ritchie, Brian↗

Deployment of neural-network-based neutron microscopic cross sections in the Griffin reactor physics application

The capability to utilize neural networks to predict macroscopic and microscopic cross section parametric spaces has been developed for the Griffin reactor physics application. The LibTorch interface enables Griffin's MOOSE-based materials to interact with LibTorch-trained models, allowing for the evaluation of complex macroscopic or microscopic cross section spaces, which are then used to evaluate the neutronic properties of the Griffin finite element model. This study benchmarks traditional ISOXML-formatted tabulation libraries against neural network-based models for 279 nuclides on 20,160 grid points for zero-dimensional and two-dimensional reactor models. Benchmark metrics include the fundamental mode eigenvalue, fission and absorption rates, and various temperature coefficients of reactivity (isothermal, fuel, and moderator). From the perspective of storage space, the complete set of LibTorch models uses 11 MB on disk, compared to the 10 GB for the ISOXML multigroup library that covers the same grid space. For the two-dimensional performance case considered in Griffin, the Torch model uses 97% less RAM than the reference ISOXML dataset while runtime increases by a factor of 3 when using the LibTorch model compared to the ISOXML dataset with multi-linear interpolation. The LibTorch model consistently yields errors within 0.01% for most analyzed quantities except for the temperature coefficients of reactivity where the maximum discrepancies are up to 0.3 $\frac{pcm}{K}$. Due to the neural network attempting to best predict quantities with no regard for a positive or negative bias for any given quantity, predictions may experience random fluctuations, resulting in both positive and negative errors. Future work will entail both depletion and coupled transient analysis to determine the predictive capabilities of Griffin with neural network-based cross sections.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Viewing spin structures with soft x-ray microscopy

The spin of the electron and its associated magnetic moment marks the basic unit for magnetic properties of matter. Magnetism, in particular ferromagnetism and antiferromagnetism is described by a collective order of these spins, where the interaction between individual spins reflects a competition between exchange, anisotropy and dipolar energy terms. As a result the energetically favored ground state of a ferromagnetic system is a rather complex spin configuration, the magnetic domain structure. Magnetism is one of the eldest scientific phenomena, yet it is one of the most powerful and versatile utilized physical effects in modern technologies, such as in magnetic storage and sensor devices. To achieve highest storage density, the relevant length scales, such as the bit size in disk drives is now approaching the nanoscale and as such further developments have to deal with nanoscience phenomena. Advanced characterization tools are required to fully understand the underlying physical principles. Magnetic microscopes using polarized soft X-rays offer a close-up view into magnetism with unique features, these include elemental sensitivity due to X-ray magnetic dichroism effects as contrast mechanism, high spatial resolution provided by state-of-the-art X-ray optics and fast time resolution limited by the inherent time structure of current X-ray sources, which will be overcome with the introduction of ultrafast and high brilliant X-ray sources.

36 MATERIALS SCIENCE↗

Magnetically suspended flywheel system study

A program to study the application of a graphite/epoxy, magnetically suspended, pierced disk flywheel for the combined function of spacecraft attitude control and energy storage (ACES) is described. Past achievements of the program include design and analysis computer codes for the flywheel rotor, a magnetically suspended flywheel model, and graphite/epoxy rotor rings that were successfully prestressed via interference assembly. All hardware successfully demonstrated operation of the necessary subsystems which form a complete ACES design. Areas of future work include additional rotor design research, system definition and control strategies, prototype development, and design/construction of a UM/GSFC spin test facility. The results of applying design and analysis computer codes to a magnetically suspended interference assembled rotor show specific energy densities of 42 Wh/lb (92.4 Wh/kg) are obtained for a 1.6 kWh system.

Kirk, J. A.↗

Optical archiving for scientific data

Write-once-read-many (WORM) optical disk technology is now practical for use as (1) a near online storage media, (2) long-term storage in a passive repository, and (3) under some circumstances, transporting from and to data generators and data users at remote sites. Optical technology provides several significant advantages over magnetic tapes as a data storage media because it is more compact, provides random access, has a longer lifetime, at least 30 years, and is less sensitive to environmental factors such as temperature and humidity. Two major archival projects using optical technology underway are discussed: the Coastal Zone Color Scanner data and the Dynamics Explorer data. Both projects have used the software for optical archive and retrieval (SOAR) to transfer data in a volume format that can be read transparently by VAX/VMS. Through the use of this software technique, (1) data generators and data users do not have to rewrite extant software and (2) the VAX/VMS Files-11 volume format is in wide use, so it can serve as an interim standard and will be accessible in the long-term.

Lowrey, Barbara E.↗

SANs and Large Scale Data Migration at the NASA Center for Computational Sciences

Evolution and migration are a way of life for provisioners of high-performance mass storage systems that serve high-end computers used by climate and Earth and space science researchers: the compute engines come and go, but the data remains. At the NASA Center for Computational Sciences (NCCS), disk and tape SANs are deployed to provide high-speed I/O for the compute engines and the hierarchical storage management systems. Along with gigabit Ethernet, they also enable the NCCS's latest significant migration: the transparent transfer of 300 Til3 of legacy HSM data into the new Sun SAM-QFS cluster.

Salmon, Ellen M.↗

Ultrahigh resolution photographic films for X-ray/EUV/FUV astronomy

The quest for ultrahigh resolution full-disk images of the sun at soft X-ray/EUV/FUV wavelengths has increased the demand for photographic films with broad spectral sensitivity, high spatial resolution, and wide dynamic range. These requirements were made more stringent by the recent development of multilayer telescopes and coronagraphs capable of operating at normal incidence at soft X-ray/EUV wavelengths. Photographic films are the only detectors now available with the information storage capacity and dynamic range such as is required for recording images of the solar disk and corona simultaneously with sub arc second spatial resolution. During the Stanford/MSFC/LLNL Rocket X-Ray Spectroheliograph and Multi-Spectral Solar Telescope Array (MSSTA) programs, we utilized photographic films to obtain high resolution full-disk images of the sun at selected soft X-ray/EUV/FUV wavelengths. In order to calibrate our instrumentation for quantitative analysis of our solar data and to select the best emulsions and processing conditions for the MSSTA reflight, we recently tested several photographic films. These studies were carried out at the NIST SURF II synchrotron and the Stanford Synchrotron Radiation Laboratory. In this paper, we provide the results of those investigations.

Hoover, Richard B.↗

Up-to-date state of storage techniques used for large numerical data files

Methods for data storage and output in data banks and memory files are discussed along with a survey of equipment available for this. Topics discussed include magnetic tapes, magnetic disks, Terabit magnetic tape memory, Unicon 690 laser memory, IBM 1360 photostore, microfilm recording equipment, holographic recording, film readers, optical character readers, digital data storage techniques, and photographic recording. The individual types of equipment are summarized in tables giving the basic technical parameters.

Chlouba, V.↗

The Science of Computing: Virtual Memory

In the March-April issue, I described how a computer's storage system is organized as a hierarchy consisting of cache, main memory, and secondary memory (e.g., disk). The cache and main memory form a subsystem that functions like main memory but attains speeds approaching cache. What happens if a program and its data are too large for the main memory? This is not a frivolous question. Every generation of computer users has been frustrated by insufficient memory. A new line of computers may have sufficient storage for the computations of its predecessor, but new programs will soon exhaust its capacity. In 1960, a longrange planning committee at MIT dared to dream of a computer with 1 million words of main memory. In 1985, the Cray-2 was delivered with 256 million words. Computational physicists dream of computers with 1 billion words. Computer architects have done an outstanding job of enlarging main memories yet they have never kept up with demand. Only the shortsighted believe they can.

Denning, Peter J.↗

ROOT: A C++ framework for petabyte data storage, statistical analysis and visualization

ROOT is an object-oriented C++ framework conceived in the high-energy physics (HEP) community, designed for storing and analyzing petabytes of data in an efficient way. Any instance of a C++ class can be stored into a ROOT file in a machine-independent compressed binary format. In ROOT the TTree object container is optimized for statistical data analysis over very large data sets by using vertical data storage techniques. These containers can span a large number of files on local disks, the web or a number of different shared file systems. In order to analyze this data, the user can chose out of a wide set of mathematical and statistical functions, including linear algebra classes, numerical algorithms such as integration and minimization, and various methods for performing regression analysis (fitting). In particular, the RooFit package allows the user to perform complex data modeling and fitting while the RooStats library provides abstractions and implementations for advanced statistical tools. Multivariate classification methods based on machine learning techniques are available via the TMVA package. A central piece in these analysis tools are the histogram classes which provide binning of one- and multi-dimensional data. Results can be saved in high-quality graphical formats like Postscript and PDF or in bitmap formats like JPG or GIF. The result can also be stored into ROOT macros that allow a full recreation and rework of the graphics. Users typically create their analysis macros step by step, making use of the interactive C++ interpreter CINT, while running over small data samples. Once the development is finished, they can run these macros at full compiled speed over large data sets, using on-the-fly compilation, or by creating a stand-alone batch program. Finally, if processing farms are available, the user can reduce the execution time of intrinsically parallel tasks - e.g. data mining in HEP - by using PROOF, which will take care of optimally distributing the work over the available resources in a transparent way.

99 GENERAL AND MISCELLANEOUS↗

Electron trapping optical data storage system and applications

A new technology developed at Optex Corporation out-performs all other existing data storage technologies. The Electron Trapping Optical Memory (ETOM) media stores 14 gigabytes of uncompressed data on a single, double-sided 130 mm disk with a data transfer rate of up to 120 megabits per second. The disk is removable, compact, lightweight, environmentally stable, and robust. Since the Write/Read/Erase (W/R/E) processes are carried out photonically, no heating of the recording media is required. Therefore, the storage media suffers no deleterious effects from repeated W/R/E cycling. This rewritable data storage technology has been developed for use as a basis for numerous data storage products. Industries that can benefit from the ETOM data storage technologies include: satellite data and information systems, broadcasting, video distribution, image processing and enhancement, and telecommunications. Products developed for these industries are well suited for the demanding store-and-forward buffer systems, data storage, and digital video systems needed for these applications.

Brower, Daniel↗

Reliable and Efficient Distributed Checkpointing System for Grid Environments

In Fine-Grained Cycle Sharing (FGCS) systems, machine owners voluntarily share their unused CPU cycles with guest jobs, as long as their performance degradation is tolerable. However, unpredictable evictions of guest jobs lead to fluctuating completion times. Checkpoint-recovery is an attractive mechanism for recovering from such “failures”. Today’s FGCS systems often use expensive, high-performance dedicated checkpoint servers. However, in geographically distributed clusters, this may incur high checkpoint transfer latencies. Here we present a distributed checkpointing system called FALCON that uses available disk resources of the FGCS machines as shared checkpoint repositories. However, an unavailable storage host may lead to loss of checkpoint data. Therefore, we model the failures of a storage host and develop a prediction algorithm for choosing reliable checkpoint repositories. We experiment with FALCON in the university-wide Condor testbed at Purdue and show improved and consistent performance for guest jobs in the presence of irregular resource availability.

checkpoint↗

ATLAS Data Carousel

The ATLAS experiment at CERN's LHC stores detector and simulation data in raw and derived data formats across more than 150 Grid sites world-wide, currently in total about 200PB on disk and 250PB on tape. Data have different access characteristics due to various computational workflows, and can be accessed from different media, such as remote I/O, disk cache on hard disk drives or SSDs. Also, larger data centers provide the majority of offline storage capability via tape systems. For the High-Luminosity LHC (HL-LHC), the estimated data storage requirements are several factors bigger than the present forecast of available resources, based on a flat budget assumption. On the computing side, ATLAS Distributed Computing was very successful in the last years with high performance and high throughput computing integration and in using opportunistic computing resources for the Monte Carlo simulation. On the other hand, equivalent opportunistic storage does not exist. ATLAS started the "Data Carousel" project to increase the usage of less expensive storage, i.e. tapes or even commercial storage, so it is not limited to tape technologies exclusively. Data Carousel orchestrates data processing between workload management, data management, and storage services with the bulk data resident on offline storage. The processing is executed by staging and promptly processing a sliding window of inputs onto faster buffer storage, such that only a small percentage of input data are available at any one time. With this project, we aim to demonstrate that this is the natural way to dramatically reduce our storage cost. The first phase of the project was started in the fall of 2018 and was related to I/O tests of the sites archiving systems. Phase II now requires a tight integration of the workload and data management systems. Additionally, the Data Carousel studies the feasibility to run multiple computing workflows from tape. The project is progressing very well and the results presented in this document will be used before the LHC Run 3.

97 MATHEMATICS AND COMPUTING↗

A Performance Model of In-Situ Techniques

The computational capacity of High-Performance Computing (HPC) systems increases continuously with the rapid development of central processing units (CPUs) and graphic processing units (GPUs), while the in-/output (IO) subsystem develops relatively slowly and storage capacity is also limited. Data-intensive applications, which are designed to leverage the high computational capacity of HPC resources, typically generate a considerable amount of data for post-processing visualizations and data analytics. The limited IO speed and storage space could lead to constraints in the actual performance of these applications and, therefore, scientific discovery. In-situ techniques, where data is visualized/analysed while still in memory rather than through disk, can contribute to alleviating these problems as they can reduce or even fully avoid data writing/reading through the IO subsystem to/from storage. However, the overall efficiency of insitu techniques crucially depends on the characteristics of both the in-situ tasks and the applications, and the resource distribution among them. Therefore, choosing the right in-situ approach (synchronous, asynchronous, or hybrid) and resource allocation is essential to minimize overhead and maximize the benefits of concurrent execution. In this paper, we present a performance model of in-situ techniques to find the most beneficial in-situ approach and the preferred resource configuration. We verify the high accuracy of our approach with over 6800 measurements and provide use cases with different applications.

Ju, Yi [Max Planck Computing and Data Facility, Ga↗