Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Disk storage”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3
All sourcesNASA NTRSDOE OSTI
NASA NTRS1994

Coupled-Cluster Theory Employing Approximate Integrals: An Approach to Avoid the Input/Output and Storage Bottlenecks

By representing orbital products in an expansion basis, certain classes of two-electron integrals are approximated for use in CCSD(T) calculations (singles and doubles coupled-cluster plus a perturbational estimate of the effects of connected triple excitations). This leads to a very large reduction in disk storage and input/output requirements, with usually only a modest increase in computational effort. The new procedure will allow very large CCSD(T) calculations to be undertaken, limited only by available processor time. Using the molecular basis as the expansion basis, explicit numerical comparisons of equilibrium geometries, harmonic frequencies, and energy differences indicate that the error due to the use of approximate integrals is less than the error associated with truncation of the molecular basis set.

Rendell, Alistair P.↗
DOE OSTI2024

PHYSLITE - A new reduced common data format for ATLAS

The High Luminosity LHC (HL-LHC) era brings unprecedented computing challenges that call for novel approaches to reduce the amount of real and Monte Carlo-simulated data that is stored, while continuing to support the rich physics program of the ATLAS experiment. With the beginning of LHC Run 3, ATLAS introduced a new common data format, PHYS, that replaces most of the analysis-specific formats that were used in Run 2, and therefore reduces the disk storage significantly. ATLAS also launched the prototype of another common format, PHYSLITE, that is about a third of the size of PHYS. PHYSLITE will be the main format for ATLAS at the HL-LHC and aims to serve 80% of all physics analyses. To simplify analysis workloads and further reduce disk usage it is designed to largely replace user-defined analysis n-tuples and consequently contains pre-calibrated objects. Various forms of validations are in place to ensure correct functionality for users. Developments continue towards HL-LHC to improve the PHYSLITE format further.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2015

Data preservation at the Fermilab Tevatron

The Fermilab Tevatron collider's data-taking run ended in September 2011, yielding a dataset with rich scientific potential. The CDF and DO experiments each have nearly 9 PB of collider and simulated data stored on tape. A large computing infrastructure consisting of tape storage, disk cache, and distributed grid computing for physics analysis with the Tevatron data is present at Fermilab. The Fermilab Run II data preservation project intends to keep this analysis capability sustained through the year 2020 or beyond. To achieve this, we are implementing a system that utilizes virtualization, automated validation, and migration to new standards in both software and data storage technology as well as leveraging resources available from currently-running experiments at Fermilab. Furthermore, these efforts will provide useful lessons in ensuring long-term data access for numerous experiments throughout high-energy physics, and provide a roadmap for high-quality scientific output for years to come.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
DOE OSTI2017

Data preservation at the Fermilab Tevatron

The Fermilab Tevatron collider's data-taking run ended in September 2011, yielding a dataset with rich scientific potential. The CDF and D0 experiments each have approximately 9 PB of collider and simulated data stored on tape. A large computing infrastructure consisting of tape storage, disk cache, and distributed grid computing for physics analysis with the Tevatron data is present at Fermilab. The Fermilab Run II data preservation project intends to keep this analysis capability sustained through the year 2020 and beyond. To achieve this goal, we have implemented a system that utilizes virtualization, automated validation, and migration to new standards in both software and data storage technology and leverages resources available from currently-running experiments at Fermilab. Lastly, these efforts have also provided useful lessons in ensuring long-term data access for numerous experiments, and enable high-quality scientific output for years to come.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2013

SIPMaP: A Tool for Modeling Irregular Parallel Computations in the Super Instruction Architecture

Performance modeling is becoming an increasingly important part of the parallel application development process, particularly for expensive computations that will be run on very high-end systems where resources are scarce. We describe a performance modeling tool SIPMaP (Super Instruction Processor Modeling and Prediction) developed for the Super-Instruction Architecture (SIA). The SIA is designed for applications where the dominant data structures are large multi-dimensional arrays and it comprises a DSL, the Super-Instruction Assembly Language (SIAL) that supports expressing algorithms in terms of blocks (tiles), and its runtime system Super Instruction Processor (SIP) that manages distribution and disk storage of the arrays. SIPMaP generates performance models from the SIAL source code. In comparison with many applications where useful performance models have been developed and reported, these programs are irregular and have other difficult to model characteristics such as extensive overlapping of communication and computation.

Computer Science↗
DOE OSTI2022

A Scalable Pipeline for Gigapixel Whole Slide Imaging Analysis on Leadership Class HPC Systems

Whole Slide Imaging (WSI) captures microscopic details of a patient's histopathological features at multiple res-olutions organized across different levels. Images produced by WSI are gigapixel-sized, and saving a single image in memory requires a few gigabytes which is scarce since a complicated model occupies tens of gigabytes. Performing a simple met-ric operation on these large images is also expensive. High-performance computing (HPC) can help us quickly analyze such large images using distributed training of complex deep learning models. One popular approach in analyzing these images is to divide a WSI image into smaller tiles (patches) and then train a simpler model with these reduced-sized but large numbers of patches. However, we need to solve three pre-processing challenges efficiently for pursuing this patch-based approach. 1) Creating small patches from a high-resolution image can result in a high number (hundreds of thousands per image) of patches. Storing and processing these images can be challenging due to a large number of I/O and arithmetic operations. To reduce I/Oand memory accesses, an optimal balance between the size and number of patches must exist to reduce I/O and memory accesses. 2) WSI images may have tiny annotated regions for cancer tissue and a significant portion with normal and fatty tissues; correct patch sampling should avoid dataset imbalance. 3) storing and retrieving many patches to and from disk storage might incur I/O latency while training a deep learning model. An efficient distributed data loader should reduce I/O latency during the training and inference steps. This paper explores these three challenges and provides empirical and algorithmic solutions deployed on the Summit supercomputer hosted at the Oak Ridge Leadership Computing Facility.

Dash, Sajal↗
DOE OSTI2017

Parallel peak pruning for scalable SMP contour tree computation

As data sets grow to exascale, automated data analysis and visualisation are increasingly important, to intermediate human understanding and to reduce demands on disk storage via in situ analysis. Trends in architecture of high performance computing systems necessitate analysis algorithms to make effective use of combinations of massively multicore and distributed systems. One of the principal analytic tools is the contour tree, which analyses relationships between contours to identify features of more than local importance. Unfortunately, the predominant algorithms for computing the contour tree are explicitly serial, and founded on serial metaphors, which has limited the scalability of this form of analysis. While there is some work on distributed contour tree computation, and separately on hybrid GPU-CPU computation, there is no efficient algorithm with strong formal guarantees on performance allied with fast practical performance. Here in this paper, we report the first shared SMP algorithm for fully parallel contour tree computation, withfor-mal guarantees of O(lgnlgt) parallel steps and O(n lgn) work, and implementations with up to 10x parallel speed up in OpenMP and up to 50x speed up in NVIDIA Thrust.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2019

Scalable Contour Tree Computation by Data Parallel Peak Pruning

As data sets grow to exascale, automated data analysis and visualization are increasingly important, to intermediate human understanding and to reduce demands on disk storage via in situ analysis. Trends in architecture of high performance computing systems necessitate analysis algorithms to make effective use of combinations of massively multicore and distributed systems. One of the principal analytic tools is the contour tree, which analyses relationships between contours to identify features of more than local importance. Unfortunately, the predominant algorithms for computing the contour tree are explicitly serial, and founded on serial metaphors, which has limited the scalability of this form of analysis. While there is some work on distributed contour tree computation, and separately on hybrid GPU-CPU computation, there is no efficient algorithm with strong formal guarantees on performance allied with fast practical performance. Furthermore, we report the first shared SMP algorithm for fully parallel contour tree computation, with formal guarantees of O(lg V lg t) parallel steps and O(V lg V) work for data with V samples and t contour tree supernodes, and implementations with more than 30× parallel speed up on both CPU using TBB and GPU using Thrust and up 70× speed up compared to the serial sweep and merge algorithm.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Bloom Filter framework for Web Archives

The software provides a framework to build a fast look up layer that reflects the holdings of an archive or database and operates between search/discovery system and disk storage. The software utilizes Bloom filter (BF) data structure. The discovery service powered by the Bloom filter layer comes with a very high level of accuracy yet space-saving, since BF compresses data to fit into RAM.

Balakireva, Lyudmila↗
DOE OSTI2015

Scientific Computing Strategic Plan for the Idaho National Laboratory

Scientific computing is a critical foundation of modern science. Without innovations in the field of computational science, the essential missions of the Department of Energy (DOE) would go unrealized. Taking a leadership role in such innovations is Idaho National Laboratory’s (INL’s) challenge and charge, and is central to INL’s ongoing success. Computing is an essential part of INL’s future. DOE science and technology missions rely firmly on computing capabilities in various forms. Modeling and simulation, fueled by innovations in computational science and validated through experiment, are a critical foundation of science and engineering. Big data analytics from an increasing number of widely varied sources is opening new windows of insight and discovery. Computing is a critical tool in education, science, engineering, and experiments. Advanced computing capabilities in the form of people, tools, computers, and facilities, will position INL competitively to deliver results and solutions on important national science and engineering challenges. A computing strategy must include much more than simply computers. The foundational enabling component of computing at many DOE national laboratories is the combination of a showcase like data center facility coupled with a very capable supercomputer. In addition, network connectivity, disk storage systems, and visualization hardware are critical and generally tightly coupled to the computer system and co located in the same facility. The existence of these resources in a single data center facility opens the doors to many opportunities that would not otherwise be possible.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Scalable Skipper-CCD Analog Readout Electronics (SSCARE) for the Massive Multichannel OSCURA Experiment

We present a multiplexed analog readout electronics system for Skipper-CCDs based on the MIDNA ASIC. It allows for subelectron noise-level operation while maintaining a minimal number of acquisition channels. In addition, it requires low-disk storage and low-bandwidth data transfer with zero added multiplexing time during the simultaneous operation of thousands of channels. We describe the implementation of such a system in a new instrument composed by 16 sensors operated with a two-stage analog multiplexed readout scheme. The instrument is a part of the R&D effort of the OSCURA experiment.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2025

Continuing Development of the Nuclear Data Processing Code AMPX [Poster]

The ENDF/B-VIII.1 evaluation library has seen a great growth in the thermal neutron scattering sub-library. The SCALE code system has traditionally approached CE transport by assuming that the CE library on disk represented the fully expanded cumulative probability distributions, conditional on exiting angle and marginal on exiting energy. While this is a complete description of the data, it comes at the potential cost of large amounts of on-disk storage. This approach was strained by several TSL files in ENDF/B-VIII.1, such as graphite, which contained data for a large number of Bragg edges. In the fully expanded probability distributions, this was found to be a disproportionately large fraction of the SCALE CE library.

GNDS↗
DOE OSTI2022

Multiplexed Readout for an Experiment with a Large Number of Channels Using Single-Electron Sensitivity Skipper-CCDs

This paper presents the implementation of a multiplexed analog readout electronics system that can achieve single-electron counting using Skipper-CCDs with non-destructive readout. The proposed system allows the best performance of the sensors to be maintained, with sub-electron noise-level operation, while maintaining low-bandwidth data transfer, a minimum number of analog-to-digital converters (ADC) and low disk storage requirement with zero added multiplexing time, even for the simultaneous operation of thousands of channels. These features are possible with a combination of analog charge pile-up, sample and hold circuits and analog multiplexing. The implementation also aims to use the minimum number of components in circuits to keep compatibility with high-channel-density experiments using Skipper-CCDs for low-threshold particle detection applications. Performance details and experimental results using a sensor with 16 output stages are presented along with a review of the circuit design considerations.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
NASA NTRS1973

Analyses of requirements for computer control and data processing experiment subsystems: Image data processing system (IDAPS) software description (7094 version), volume 2

A description of each of the software modules of the Image Data Processing System (IDAPS) is presented. The changes in the software modules are the result of additions to the application software of the system and an upgrade of the IBM 7094 Mod(1) computer to a 1301 disk storage configuration. Necessary information about IDAPS sofware is supplied to the computer programmer who desires to make changes in the software system or who desires to use portions of the software outside of the IDAPS system. Each software module is documented with: module name, purpose, usage, common block(s) description, method (algorithm of subroutine) flow diagram (if needed), subroutines called, and storage requirements.

Source record↗
NASA NTRS1975

Geophysical data base

A general data-management system that provides a random-access capability for large amounts of data is described. The system operates on a CDC 6400 computer using a combination of magnetic tape and disk storage. A FORTRAN subroutine package is provided to simplify the maintenance and use of the data.

Williamson, M. R.↗
NASA NTRS1975

Upgrading of AMTRAN on the datacraft DC 6024

A number of new software extensions and modifications to the AMTRAN system were completed. These modifications include input/output alterations, one-dimensional and two-dimensional array improvements, and disk storage of data enhancements. In order to promote a better understanding of the AMTRAN system software structure, a general description of the AMTRAN modules and their interrelationships was provided. A few sample statements have also been traced through the compilation and execution stages in order to illustrate the flow of system logic. In addition, the previous documentation was updated to reflect these modifications.

Beasley, P. E.↗
NASA NTRS1978

The IMSL environment for software development

The international mathematical and statistical library (IMSL) developed a set of macros and a file naming convention that automates the subroutine development and testing process over ten computer types. The IMSL software development system is implemented on a Data General Eclipse C330 computer with 256K bytes of central memory and 192M bytes of disk storage using the AOS Operating System. RJE activity is handled by a Data 100 communications computer. The system allows the programmer to work with basis decks. Distribution decks are generated, by the IMSL FORTRAN converter, as they are needed for testing and whenever the basis deck has been modified.

Aird, T. J.↗