Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data storage”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,747 records · Page 97

Tenth Goddard Conference on Mass Storage Systems and Technologies in Cooperation with the Nineteenth IEEE Symposium on Mass Storage Systems

This document contains copies of those technical papers received in time for publication prior to the Tenth Goddard Conference on Mass Storage Systems and Technologies which is being held in cooperation with the Nineteenth IEEE Symposium on Mass Storage Systems at the University of Maryland University College Inn and Conference Center April 15-18, 2002. As one of an ongoing series, this Conference continues to provide a forum for discussion of issues relevant to the ingest, storage, and management of large volumes of data. The Conference encourages all interested organizations to discuss long-term mass storage requirements and experiences in fielding solutions. Emphasis is on current and future practical solutions addressing issues in data management, storage systems and media, data acquisition, long-term retention of data, and data distribution. This year's discussion topics include architecture, future of current technology, storage networking with emphasis on IP storage, performance, standards, site reports, and vendor solutions. Tutorials will be available on perpendicular magnetic recording, object based storage, storage virtualization and IP storage.

Kobler, Benjamin↗

Long-Term file activity patterns in a UNIX workstation environment

As mass storage technology becomes more affordable for sites smaller than supercomputer centers, understanding their file access patterns becomes crucial for developing systems to store rarely used data on tertiary storage devices such as tapes and optical disks. This paper presents a new way to collect and analyze file system statistics for UNIX-based file systems. The collection system runs in user-space and requires no modification of the operating system kernel. The statistics package provides details about file system operations at the file level: creations, deletions, modifications, etc. The paper analyzes four months of file system activity on a university file system. The results confirm previously published results gathered from supercomputer file systems, but differ in several important areas. Files in this study were considerably smaller than those at supercomputer centers, and they were accessed less frequently. Additionally, the long-term creation rate on workstation file systems is sufficiently low so that all data more than a day old could be cheaply saved on a mass storage device, allowing the integration of time travel into every file system.

Gibson, Timothy J.↗

Development of medical data information systems

Computerized storage and retrieval of medical information is discussed. Tasks which were performed in support of the project are: (1) flight crew health stabilization computer system, (2) medical data input system, (3) graphic software development, (4) lunar receiving laboratory support, and (5) Statos V printer/plotter software development.

Anderson, J.↗

Scalable Incremental Checkpointing using GPU-Accelerated De-Duplication

Writing large amounts of data concurrently to stable storage is a typical I/O pattern of many HPC workflows. This pattern introduces high I/O overheads and results in increased storage space utilization especially for workflows that need to capture the evolution of data structures with high frequency as checkpoints. In this context, many applications, such as graph pattern matching, perform sparse updates to large data structures between checkpoints. For these applications, incremental checkpointing techniques that save only the differences from one checkpoint to another can dramatically reduce the checkpoint sizes, I/O bottlenecks, and storage space utilization. However, such techniques are not without challenges: it is non-trivial to transparently determine what data has changed since a previous checkpoint and assemble the differences in a compact fashion that does not result in excessive metadata. State-of-art data reduction techniques (e.g., compression and de-duplication) have significant limitations when applied to modern HPC applications that leverage GPUs: slow at detecting the differences, generate a large amount of metadata to keep track of the differences, and ignore crucial spatiotemporal checkpoint data redundancy. This paper addresses these challenges by proposing a Merkle tree-based incremental checkpointing method to exploit GPUs' high memory bandwidth and massive parallelism. Experimental results at scale show a significant reduction of the I/O overhead and space utilization of checkpointing compared with state-of-the-art incremental checkpointing and compression techniques.

Tan, Nigel↗

Smart CO2 Transport-Route Planning Tool: Providing Data and Insights for Accelerating Carbon Transport & Storage Deployment

Overview presentation given at the 2024 FECM / NETL Carbon Management Research Project Review Meeting on NETL's Bipartisan Infrastructure Law-funded Smart CO2 Transport-Route Planning Tool and associated geodatabase. This machine learning informed, data-driven public resource was designed to inform regulators, industry, and researchers plan and develop safe and efficient transport routes across the country.

Romeo, Lucy↗

FIRM: federated image reconstruction using multimodal tomographic data

Here, we propose a federated algorithm for reconstructing images using multimodal tomographic data sourced from dispersed locations, addressing the challenges of traditional unimodal approaches that are prone to noise and reduced image quality, as well as the limitations of centralized multimodal approaches that require extensive data transfer, leading to significant communication overhead, storage demands, and potential data privacy concerns. Our approach formulates a joint inverse optimization problem incorporating multimodality constraints and solves it in a federated framework through local gradient computations complemented by lightweight central operations, thereby ensuring data decentralization. Leveraging the connection between our federated algorithm and the quadratic penalty method, we introduce an adaptive step-size rule with guaranteed sublinear convergence. Numerical results demonstrate superior computational efficiency and improved image reconstruction quality compared to existing approaches.

federated algorithm↗

Implementation and Evaluation of Active Storage in Modern Parallel File Systems

Active Storage is a technology aimed at reducing the bandwidth requirements of current supercomputing systems, and leveraging the processing power of the storage nodes used by some modern file systems. Active Storage achieves both objectives by moving certain processing tasks to the storage nodes, near the data they manage. Our proposal for Active Storage has several key features: user-space implementation which facilitates the port to different file systems, analytical model to anticipate the performance of Active Storage with respect to a traditional system, support for striped files and complex format files such as netCDF, scientific-friendly programming and runtime environment, etc.

99 GENERAL AND MISCELLANEOUS↗

ndi

The Nuclear Data Interface (NDI) is an application programming interface (API) that allows access to standard nuclear data parameters while hiding the underlying details of the data libraries and their storage. It allows access to multigroup transport data (neutron and gamma), thermonuclear burn data, dosimetry data, production/depletion chain data, radiochemistry data, and secondary neutron multiplicity data. The name NDI refers to both the code and data formats supported by the code.

Saller, Thomas↗

Integrated Energy-Water Data for Cross-Sector Resilience

This white paper focuses on the “energy-for-water” domain, addressing the urgent need for integrated, empirical data to support regional management, benchmarking, and research on improving efficiency and developing technologies for water and wastewater management systems. The costs and energy required for the supply, treatment, and distribution of water and wastewater lack a standard data collection mechanism and centralized database or storage infrastructure, limiting data-driven decision-making across interdependent infrastructure systems.

42 ENGINEERING↗

SPAR data handling utilities

The SPAR computer software system is a collection of processors that perform particular steps in the finite-element structural analysis procedure. The data generated by each processor are stored on a data base complex residing on an auxiliary storage device, and these data are then used by subsequent processors. The SPAR data handling utilities use routines to transfer data between the processors and the data base complex. A detailed description of the data base complex organization is presented. A discussion of how these SPAR data handling utilities are used in an application program to perform desired user functions is given with the steps necessary to convert an existing program to a SPAR processor by incorporating these utilities. Finally, a sample SPAR processor is included to illustrate the use of the data handling utilities.

Giles, G. L.↗

Data movement engine

A memory system of a data processing system includes one or more storage devices and a data rearrangement engine for moving data between memory regions of the plurality of memory regions. The data rearrangement engine is configured to rearrange data stored at non-contiguous addresses in a source memory region into contiguous address in a destination region responsive to a rearrangement specified by a host processing unit of the data processing system. A description of the rearranged data is maintained in a metadata memory region. Rearranged data may be accessed by one or more host processing units. Write-back of data from the destination to the source region may be reduced by use of Bloom filter or the like.

Beard, Jonathan Curtis↗

Development of an end state vision to implement digital monitoring in nuclear plants

Transitioning from an onsite Maintenance & Diagnostics Center to cloud-based services offers many new opportunities with computing power and storage, but also new challenges in terms of networking and security. This report will cover everything required for that transition including data processing and uploading to cloud services, feature selection, model creation, and result visualization for decision making. Although there are several other cloud-based services (e.g. Amazon Web Services and Google Cloud), this report explores Microsoft Azure to simplify nomenclature and maintain a consistent focus. Many of the services offered by Microsoft Azure are also available in the other cloud-based services, and their differences have been recorded in other literature. The Azure services most important to a nuclear power plant including networking & security, storage & databases, and Artificial Intelligence (AI) are reviewed here. Networking covers all aspects related to communication to Azure resources including security, privacy, and redundancy. Storage & databases includes data storage, upgrading, patching, backups, and monitoring. The AI services allows the user access to the machine learning (ML) techniques developed with Azure including automated ML, anomaly detection, computer vision, and natural language processing. This report summaries the features, capabilities, and challenges when using cloud-based services in a user-friendly manner.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Real-Time Operating System/360

RTOS has a cost savings advantage for real-time applications, such as those with random inputs requiring a flexible data routing facility, display systems simplified by a device independent interface language, and complex applications needing added storage protection and data queuing.

Hoffman, R. L.↗

Opportunities and data requirements for data-driven prognostics and health management in liquid hydrogen storage systems

During the past decade, Prognostics and Health Management (PHM) has become an important set of tools in various areas of industry and academic reliability engineering. PHM consists of a variety of mathematical and computational methods used to support data-driven decision-making to increase the safety, availability, and reliability of complex engineering systems. In particular, PHM can provide crucial insight into reliability and safety design improvements for developing technologies where historical performance and failure data are limited. This is the case of hydrogen fueling and storage technologies. This work presents a high-level approach for designing data-driven PHM applications for bulk liquid hydrogen (LH 2 ) storage systems for hydrogen fueling stations. This paper addresses core aspects of the design, development, and implementation of data-driven PHM applications that can improve the reliability assessment of hydrogen components. The analysis focuses on the relationship between data availability and diagnostic/prognostic capabilities; potential challenges; and integration schemes for current risk mitigation measures. We identify potential condition-monitoring data sources for key components in an LH 2 storage system, including storage tanks, piping, and pumps. Further, we determine that the short-term goals for the implementation of data-driven models in PHM frameworks in hydrogen systems should focus on developing adequate data collection and analysis strategies, as well as exploring the effect on reliability, safety, and regulations for hydrogen systems.

08 HYDROGEN↗

Cooperative high-performance storage in the accelerated strategic computing initiative

The use and acceptance of new high-performance, parallel computing platforms will be impeded by the absence of an infrastructure capable of supporting orders-of-magnitude improvement in hierarchical storage and high-speed I/O (Input/Output). The distribution of these high-performance platforms and supporting infrastructures across a wide-area network further compounds this problem. We describe an architectural design and phased implementation plan for a distributed, Cooperative Storage Environment (CSE) to achieve the necessary performance, user transparency, site autonomy, communication, and security features needed to support the Accelerated Strategic Computing Initiative (ASCI). ASCI is a Department of Energy (DOE) program attempting to apply terascale platforms and Problem-Solving Environments (PSEs) toward real-world computational modeling and simulation problems. The ASCI mission must be carried out through a unified, multilaboratory effort, and will require highly secure, efficient access to vast amounts of data. The CSE provides a logically simple, geographically distributed, storage infrastructure of semi-autonomous cooperating sites to meet the strategic ASCI PSE goal of highperformance data storage and access at the user desktop.

Gary, Mark↗

Data co-processing for extreme scale analysis level II ASC milestone (4745).

Exascale supercomputing will embody many revolutionary changes in the hardware and software of high-performance computing. A particularly pressing issue is gaining insight into the science behind the exascale computations. Power and I/O speed con- straints will fundamentally change current visualization and analysis work ows. A traditional post-processing work ow involves storing simulation results to disk and later retrieving them for visualization and data analysis. However, at exascale, scien- tists and analysts will need a range of options for moving data to persistent storage, as the current o ine or post-processing pipelines will not be able to capture the data necessary for data analysis of these extreme scale simulations. This Milestone explores two alternate work ows, characterized as in situ and in transit, and compares them. We nd each to have its own merits and faults, and we provide information to help pick the best option for a particular use.

Rogers, David↗

The Prompt Processing System and Data Quality Monitoring in the ProtoDUNE-SP Experiment

The DUNE Collaboration is conducting an experimental program (named protoDUNE) which involves a beam test of two large-scale prototypes of the DUNE Far Detector at CERN operating in 2018-2019. The volume of data to be collected by the protoDUNE-SP (the single-phase detector) will amount to a few petabytes and the sustained rate of data sent to mass storage is in the range of a few hundred MB per second. After collection the data is committed to storage at CERN and transmitted to Fermi National Accelerator Laboratory in the US for processing, analysis and long-term preservation. The protoDUNE experiment requires substantial Data Quality Monitoring capabilities in order to ascertain the condition of the detector and its various subsystems. We present the design of the protoDUNE Prompt Processing System, its deployment at CERN and its performance during the data challenges and actual data taking.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗