Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “DATA STORAGE”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,297 records · Page 72
All sourcesNASA NTRSDOE OSTI
DOE OSTI2016

The Medical Science DMZ

Objective We describe use cases and an institutional reference architecture for maintaining high-capacity, data-intensive network flows (e.g., 10, 40, 100 Gbpsþ) in a scientific, medical context while still adhering to security and privacy laws and regulations. Materials and Methods High-end networking, packet filter firewalls, network intrusion detection systems. Results We describe a “Medical Science DMZ” concept as an option for secure, high-volume transport of large, sensitive data sets between research institutions over national research networks. Discussion The exponentially increasing amounts of “omics” data, the rapid increase of high-quality imaging, and other rapidly growing clinical data sets have resulted in the rise of biomedical research “big data.” The storage, analysis, and network resources required to process these data and integrate them into patient diagnoses and treatments have grown to scales that strain the capabilities of academic health centers. Some data are not generated locally and cannot be sustained locally, and shared data repositories such as those provided by the National Library of Medicine, the National Cancer Institute, and international partners such as the European Bioinformatics Institute are rapidly growing. The ability to store and compute using these data must therefore be addressed by a combination of local, national, and industry resources that exchange large data sets. Maintaining data-intensive flows that comply with HIPAA and other regulations presents a new challenge for biomedical research. Recognizing this, we describe a strategy that marries performance and security by borrowing from and redefining the concept of a “Science DMZ”—a framework that is used in physical sciences and engineering research to manage high-capacity data flows. Conclusion By implementing a Medical Science DMZ architecture, biomedical researchers can leverage the scale provided by high-performance computer and cloud storage facilities and national high-speed research networks while preserving privacy and meeting regulatory requirements.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2017

A Comparative Study of Multi-material Data Structures for Computational Physics Applications

The data structures used to represent the multi-material state of a computational physics application can have a drastic impact on the performance of the application. We look at efficient data structures for sparse applications where there may be many materials, but only one or few in most computational cells. We develop simple performance models for use in selecting possible data structures and programming patterns. We verify the analytic models of performance through a small test program of the representative cases.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Object storage model for CMS data

In CMS, data access and management is organized around the data-tier model: a static definition of what subset of event information is available in a particular dataset, realized as a collection of files. In previous work, we have proposed a novel data management model that obviates the need for data tiers by exploding files into individual event data product objects. In this work, we estimate the potential savings in data volume based on user analysis patterns.

Smith, Nick↗
NASA NTRS1975

Automated storage and retrieval of data obtained in the Interkosmos project

The formation of a data bank and information retrieval system for scientific data is described. The stored data can be digital or documentation data. Data classification methods are discussed along with definition and compilation of the dictionary utilized, definition of the indexing scheme, and definition of the principles used in constructing a file for documents, data blocks, and tapes. Operating principles are also presented.

Ziolkovski, K.↗
NASA NTRS1994

System considerations for efficient communication and storage of MSTI image data

The Ballistic Missile Defense Organization has been developing the capability to evaluate one or more high-rate sensor/hardware combinations by incorporating them as payloads on a series of Miniature Seeker Technology Insertion (MSTI) flights. This publication represents the final report of a 1993 study to analyze the potential impact f data compression and of related communication system technologies on post-MSTI 3 flights. Lossless compression is considered alone and in conjunction with various spatial editing modes. Additionally, JPEG and Fractal algorithms are examined in order to bound the potential gains from the use of lossy compression. but lossless compression is clearly shown to better fit the goals of the MSTI investigations. Lossless compression factors of between 2:1 and 6:1 would provide significant benefits to both on-board mass memory and the downlink. for on-board mass memory, the savings could range from $5 million to $9 million. Such benefits should be possible by direct application of recently developed NASA VLSI microcircuits. It is shown that further downlink enhancements of 2:1 to 3:1 should be feasible thorough use of practical modifications to the existing modulation system and incorporation of Reed-Solomon channel coding. The latter enhancement could also be achieved by applying recently developed VLSI microcircuits.

Rice, Robert F.↗
DOE OSTI2021

A deep learning-accelerated data assimilation and forecasting workflow for commercial-scale geologic carbon storage

Fast assimilation of monitoring data to update forecasts of pressure buildup and carbon dioxide (CO 2 ) plume migration under geologic uncertainties is a challenging problem in geologic carbon storage. The high computational cost of data assimilation with a high-dimensional parameter space impedes fast decision-making for commercial-scale reservoir management. We propose to leverage physical understandings of porous medium flow behavior with deep learning techniques to develop a fast data assimilation-reservoir response forecasting workflow. Applying an Ensemble Smoother Multiple Data Assimilation (ES-MDA) framework, the workflow updates geologic properties and predicts reservoir performance with quantified uncertainty from pressure history and CO 2 plumes interpreted through seismic inversion. As the most computationally expensive component in such a workflow is reservoir simulation, we developed surrogate models to predict dynamic pressure and CO 2 plume extents under multi-well injection. The surrogate models employ deep convolutional neural networks, specifically, a wide residual network and a residual U-Net. The workflow is validated against a flat threedimensional reservoir model representative of a clastic shelf depositional environment. Intelligent treatments are applied to bridge between quantities in a true-3D reservoir model and those in a single-layer reservoir model. The workflow can complete history matching and reservoir forecasting with uncertainty quantification in less than one hour on a mainstream personal workstation.

25 ENERGY STORAGE↗
NASA NTRS2019

Application of GRACE to the Estimation of Groundwater Storage Change in a Data-poor Region: A Case Study of Ngadda Catchment in the Lake Chad Basin

The present study is to explore the feasibility of GRACE-based estimation of a groundwater storage change in a data-poor region using a case study of the Ngadda catchment in the Lake Chad Basin. Although the Ngadda catchment has only one set of in situ time series data of groundwater from 2006 to 2009 and a limited number of groundwater measurements in 2005 and 2009, GRACE-based groundwater storage change can be evaluated against thein situ groundwater measurements combined with specific yield data. The cross-correlation analysis in the Ngadda catchment shows that maximum rainfall reached in July and August, whereas both the maximum total water storage anomaly and the maximum groundwater storage anomaly occurred 2months later. Whereas the mean annual amplitude of total water storage anomaly is about 17cm from both the average total water storage anomaly from three mascon products and the one from three spherical harmonic products, the mean annual amplitude of soil moisture storage anomaly is substantially varied from5.58cm for CLM to about 14cm for NOAH and Mosaic. The goodness-of-fit tests show that CLM soil moisture produces the closest estimation of groundwater storage anomaly to thein situ groundwater measurements. The present study shows that GRACE-based estimation for groundwater storage anomaly can be a cost-effective and alternative tool to observe how groundwater changes in a basin scale under the limitation of modelling and in situ data availability.

GRACE↗
NASA NTRS1994

The design of a petabyte archive and distribution system for the NASA ECS project

The NASA EOS Data and Information System (EOSDIS) Core System (ECS) will contain one of the largest data management systems ever built - the ECS Science and Data Processing System (SDPS). SDPS is designed to support long term Global Change Research by acquiring, producing, and storing earth science data, and by providing efficient means for accessing and manipulating that data. The first two releases of SDPS, Release A and Release B, will be operational in 1997 and 1998, respectively. Release B will be deployed at eight Distributed Active Archiving Centers (DAAC's). Individual DAAC's will archive different collections of earth science data, and will vary in archive capacity. The storage and management of these data collections is the responsibility of the SDPS Data Server subsystem. It is anticipated that by the year 2001, the Data Server subsystem at the Goddard DAAC must support a near-line data storage capacity of one petabyte. The development of SDPS is a system integration effort in which COTS products will be used in favor of custom components in very possible way. Some software and hardware capabilities required to meet ECS data volume and storage management requirements beyond 1999 are not yet supported by available COTS products. The ECS project will not undertake major custom development efforts to provide these capabilities. Instead, SDPS and its Data Server subsystem are designed to support initial implementations with current products, and provide an evolutionary framework that facilitates the introduction of advanced COTS products as they become available. This paper provides a high-level description of the Data Server subsystem design from a COTS integration standpoint, and discussed some of the major issues driving the design. The paper focuses on features of the design that will make the system scalable and adaptable to changing technologies.

Caulk, Parris M.↗
DOE OSTI2018

Development of Intelligent Monitoring System (IMS) Modules for the Aquistore CO 2 Storage Project

The monitoring data generated at a CO 2 storage site can quickly result in large amounts of data, often requiring significant labor- and time-intensive efforts to process and analyze before being able to make operational decisions. Implementing an intelligent monitoring system (IMS) can reduce the associated data-processing and analysis costs, which can thereby improve the efficiency of the monitoring process and enable real-time operational decision-making. The Energy & Environmental Research Center (EERC) has completed a 3-year project focused on developing an IMS comprising new workflows, algorithms, and a user interface for the automation and integration of geologic carbon dioxide (CO 2 ) storage site-monitoring and simulation data. The design for the IMS was based on using injection and monitoring data acquired from Aquistore, an operating CO 2 storage project. Managed by the Petroleum Technology Research Centre, the Aquistore Project is injecting CO 2 captured from SaskPower’s coal-fired Boundary Dam Power Station into a deep saline formation for geologic storage. Using data from the Aquistore Project provided a real-world basis to develop an IMS.

47 OTHER INSTRUMENTATION↗
DOE OSTI2026

Storing Affordability: Battery Storage as an Asset to Reduce Data Center Cost Shifts

This report examines how battery energy storage systems (BESS) can help utilities accommodate large load growth while protecting affordability for existing ratepayers. Rapid growth in electricity demand from artificial intelligence (AI) data centers is straining the U.S. grid. Furthermore, many new data centers are entering rural markets, which could offer economic benefits but may also pose implementation challenges for smaller utilities. At the same time, retail electricity prices are increasing faster than inflation, elevating customer affordability as a key challenge. While data centers have not been the primary driver of increases in residential prices to date, they have pushed wholesale energy and capacity prices higher in several markets. Fundamental utility cost-allocation principles show that data center growth can be rate-positive for existing customers only if new peak demand grows faster than the costs a utility must incur to serve it. Several factors, including a utility’s degree of wholesale market exposure, forecast uncertainty and stranded-asset risk, and tariff design can determine the outcome of load growth on retail rates. Energy storage can make several affordability contributions in the face of this landscape of uncertainty and market volatility, including deferral of higher-cost grid investments through improved utilization of existing assets and flexibility of new large loads, insulation from volatile wholesale prices through peak shaving, and reliability support to address grid risks stemming from the behavior of AI data center loads. Different potential BESS deployment pathways—utility-scale front-of-the-meter systems, aggregated small-scale storage installations, and data center-sited behind-the-meter storage—are compared against each other and against conventional capacity alternatives. This framework is intended as a conceptual resource to utilities, particularly smaller public utilities with rural service territories, who may be considering the role that energy storage can play in insulating existing ratepayers from data center cost shifts.

25 ENERGY STORAGE↗
NASA NTRS1996

RAID Disk Arrays for High Bandwidth Applications

High bandwidth applications require large amounts of data transferred to/from storage devices at extremely high data rates. Further, these applications often are 'real time' in which access to the storage device must take place on the schedule of the data source, not the storage. A good example is a satellite downlink - the volume of data is quite large and the data rates quite high (dozens of MB/sec). Further, a telemetry downlink must take place while the satellite is overhead. A storage technology which is ideally suited to these types of applications is redundant arrays of independent discs (RAID). Raid storage technology, while offering differing methodologies for a variety of applications, supports the performance and redundancy required in real-time applications. Of the various RAID levels, RAID-3 is the only one which provides high data transfer rates under all operating conditions, including after a drive failure.

Moren, Bill↗
DOE OSTI2013

Asynchronous Object Storage with QoS for Scientific and Commercial Big Data

This paper presents our design for an asynchronous object storage system intended for use in scientific and commercial big data workloads. Use cases from the target workload do- mains are used to motivate the key abstractions used in the application programming interface (API). The architecture of the Scalable Object Store (SOS), a prototype object stor- age system that supports the API s facilities, is presented. The SOS serves as a vehicle for future research into scalable and resilient big data object storage. We briefly review our research into providing efficient storage servers capable of providing quality of service (QoS) contracts relevant for big data use cases.

Cloud Storage↗
DOE OSTI2016

Storage and Read-Optimized Data Placement Structures for High-Performance Analysis [Book Chapter]

Here, we present state-of-art structures and methods for efficient data preparation and representation for analysis. Our intent is to introduce the data science and analytics communities to open source data placements, structures, and methods. These practices can make the foundational processes of data preparation and access dramatically more efficient than typical raw file or database representations and use more conservative storage. To illustrate this, we introduce two highly efficient data placement structures. We then present a tutorial, supported by step-by-step examples, of how to create, use, and access data, structured by Parquet or ORC (Optimized Row Columnar), using Apache Spark. Finally, we illustrate the benefits of using these structures with computational and storage volume benchmarks.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2015

Minimally buffered data transfers between nodes in a data communications network

Methods, apparatus, and products for minimally buffered data transfers between nodes in a data communications network are disclosed that include: receiving, by a messaging module on an origin node, a storage identifier, a origin data type, and a target data type, the storage identifier specifying application storage containing data, the origin data type describing a data subset contained in the origin application storage, the target data type describing an arrangement of the data subset in application storage on a target node; creating, by the messaging module, origin metadata describing the origin data type; selecting, by the messaging module from the origin application storage in dependence upon the origin metadata and the storage identifier, the data subset; and transmitting, by the messaging module to the target node, the selected data subset for storing in the target application storage in dependence upon the target data type without temporarily buffering the data subset.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2017

BASTet: Shareable and Reproducible Analysis and Visualization of Mass Spectrometry Imaging Data via OpenMSI

Mass spectrometry imaging (MSI) is a transformative imaging method that supports the untargeted, quantitative measurement of the chemical composition and spatial heterogeneity of complex samples with broad applications in life sciences, bioenergy, and health. While MSI data can be routinely collected, its broad application is currently limited by the lack of easily accessible analysis methods that can process data of the size, volume, diversity, and complexity generated by MSI experiments. The development and application of cutting-edge analytical methods is a core driver in MSI research for new scientific discoveries, medical diagnostics, and commercial-innovation. However, the lack of means to share, apply, and reproduce analyses hinders the broad application, validation, and use of novel MSI analysis methods. To address this central challenge, we introduce the Berkeley Analysis and Storage Toolkit (BASTet), a novel framework for shareable and reproducible data analysis that supports standardized data and analysis interfaces, integrated data storage, data provenance, workflow management, and a broad set of integrated tools. Based on BASTet, we describe the extension of the OpenMSI mass spectrometry imaging science gateway to enable web-based sharing, reuse, analysis, and visualization of data analyses and derived data products. Here, we demonstrate the application of BASTet and OpenMSI in practice to identify and compare characteristic substructures in the mouse brain based on their chemical composition measured via MSI.

97 MATHEMATICS AND COMPUTING↗