Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data storage”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,333 records · Page 74

A deep learning-accelerated data assimilation and forecasting workflow for commercial-scale geologic carbon storage

Fast assimilation of monitoring data to update forecasts of pressure buildup and carbon dioxide (CO 2 ) plume migration under geologic uncertainties is a challenging problem in geologic carbon storage. The high computational cost of data assimilation with a high-dimensional parameter space impedes fast decision-making for commercial-scale reservoir management. We propose to leverage physical understandings of porous medium flow behavior with deep learning techniques to develop a fast data assimilation-reservoir response forecasting workflow. Applying an Ensemble Smoother Multiple Data Assimilation (ES-MDA) framework, the workflow updates geologic properties and predicts reservoir performance with quantified uncertainty from pressure history and CO 2 plumes interpreted through seismic inversion. As the most computationally expensive component in such a workflow is reservoir simulation, we developed surrogate models to predict dynamic pressure and CO 2 plume extents under multi-well injection. The surrogate models employ deep convolutional neural networks, specifically, a wide residual network and a residual U-Net. The workflow is validated against a flat threedimensional reservoir model representative of a clastic shelf depositional environment. Intelligent treatments are applied to bridge between quantities in a true-3D reservoir model and those in a single-layer reservoir model. The workflow can complete history matching and reservoir forecasting with uncertainty quantification in less than one hour on a mainstream personal workstation.

25 ENERGY STORAGE↗

Application of GRACE to the Estimation of Groundwater Storage Change in a Data-poor Region: A Case Study of Ngadda Catchment in the Lake Chad Basin

The present study is to explore the feasibility of GRACE-based estimation of a groundwater storage change in a data-poor region using a case study of the Ngadda catchment in the Lake Chad Basin. Although the Ngadda catchment has only one set of in situ time series data of groundwater from 2006 to 2009 and a limited number of groundwater measurements in 2005 and 2009, GRACE-based groundwater storage change can be evaluated against thein situ groundwater measurements combined with specific yield data. The cross-correlation analysis in the Ngadda catchment shows that maximum rainfall reached in July and August, whereas both the maximum total water storage anomaly and the maximum groundwater storage anomaly occurred 2months later. Whereas the mean annual amplitude of total water storage anomaly is about 17cm from both the average total water storage anomaly from three mascon products and the one from three spherical harmonic products, the mean annual amplitude of soil moisture storage anomaly is substantially varied from5.58cm for CLM to about 14cm for NOAH and Mosaic. The goodness-of-fit tests show that CLM soil moisture produces the closest estimation of groundwater storage anomaly to thein situ groundwater measurements. The present study shows that GRACE-based estimation for groundwater storage anomaly can be a cost-effective and alternative tool to observe how groundwater changes in a basin scale under the limitation of modelling and in situ data availability.

GRACE↗

The design of a petabyte archive and distribution system for the NASA ECS project

The NASA EOS Data and Information System (EOSDIS) Core System (ECS) will contain one of the largest data management systems ever built - the ECS Science and Data Processing System (SDPS). SDPS is designed to support long term Global Change Research by acquiring, producing, and storing earth science data, and by providing efficient means for accessing and manipulating that data. The first two releases of SDPS, Release A and Release B, will be operational in 1997 and 1998, respectively. Release B will be deployed at eight Distributed Active Archiving Centers (DAAC's). Individual DAAC's will archive different collections of earth science data, and will vary in archive capacity. The storage and management of these data collections is the responsibility of the SDPS Data Server subsystem. It is anticipated that by the year 2001, the Data Server subsystem at the Goddard DAAC must support a near-line data storage capacity of one petabyte. The development of SDPS is a system integration effort in which COTS products will be used in favor of custom components in very possible way. Some software and hardware capabilities required to meet ECS data volume and storage management requirements beyond 1999 are not yet supported by available COTS products. The ECS project will not undertake major custom development efforts to provide these capabilities. Instead, SDPS and its Data Server subsystem are designed to support initial implementations with current products, and provide an evolutionary framework that facilitates the introduction of advanced COTS products as they become available. This paper provides a high-level description of the Data Server subsystem design from a COTS integration standpoint, and discussed some of the major issues driving the design. The paper focuses on features of the design that will make the system scalable and adaptable to changing technologies.

Caulk, Parris M.↗

Development of Intelligent Monitoring System (IMS) Modules for the Aquistore CO 2 Storage Project

The monitoring data generated at a CO 2 storage site can quickly result in large amounts of data, often requiring significant labor- and time-intensive efforts to process and analyze before being able to make operational decisions. Implementing an intelligent monitoring system (IMS) can reduce the associated data-processing and analysis costs, which can thereby improve the efficiency of the monitoring process and enable real-time operational decision-making. The Energy & Environmental Research Center (EERC) has completed a 3-year project focused on developing an IMS comprising new workflows, algorithms, and a user interface for the automation and integration of geologic carbon dioxide (CO 2 ) storage site-monitoring and simulation data. The design for the IMS was based on using injection and monitoring data acquired from Aquistore, an operating CO 2 storage project. Managed by the Petroleum Technology Research Centre, the Aquistore Project is injecting CO 2 captured from SaskPower’s coal-fired Boundary Dam Power Station into a deep saline formation for geologic storage. Using data from the Aquistore Project provided a real-world basis to develop an IMS.

47 OTHER INSTRUMENTATION↗

Storing Affordability: Battery Storage as an Asset to Reduce Data Center Cost Shifts

This report examines how battery energy storage systems (BESS) can help utilities accommodate large load growth while protecting affordability for existing ratepayers. Rapid growth in electricity demand from artificial intelligence (AI) data centers is straining the U.S. grid. Furthermore, many new data centers are entering rural markets, which could offer economic benefits but may also pose implementation challenges for smaller utilities. At the same time, retail electricity prices are increasing faster than inflation, elevating customer affordability as a key challenge. While data centers have not been the primary driver of increases in residential prices to date, they have pushed wholesale energy and capacity prices higher in several markets. Fundamental utility cost-allocation principles show that data center growth can be rate-positive for existing customers only if new peak demand grows faster than the costs a utility must incur to serve it. Several factors, including a utility’s degree of wholesale market exposure, forecast uncertainty and stranded-asset risk, and tariff design can determine the outcome of load growth on retail rates. Energy storage can make several affordability contributions in the face of this landscape of uncertainty and market volatility, including deferral of higher-cost grid investments through improved utilization of existing assets and flexibility of new large loads, insulation from volatile wholesale prices through peak shaving, and reliability support to address grid risks stemming from the behavior of AI data center loads. Different potential BESS deployment pathways—utility-scale front-of-the-meter systems, aggregated small-scale storage installations, and data center-sited behind-the-meter storage—are compared against each other and against conventional capacity alternatives. This framework is intended as a conceptual resource to utilities, particularly smaller public utilities with rural service territories, who may be considering the role that energy storage can play in insulating existing ratepayers from data center cost shifts.

25 ENERGY STORAGE↗

RAID Disk Arrays for High Bandwidth Applications

High bandwidth applications require large amounts of data transferred to/from storage devices at extremely high data rates. Further, these applications often are 'real time' in which access to the storage device must take place on the schedule of the data source, not the storage. A good example is a satellite downlink - the volume of data is quite large and the data rates quite high (dozens of MB/sec). Further, a telemetry downlink must take place while the satellite is overhead. A storage technology which is ideally suited to these types of applications is redundant arrays of independent discs (RAID). Raid storage technology, while offering differing methodologies for a variety of applications, supports the performance and redundancy required in real-time applications. Of the various RAID levels, RAID-3 is the only one which provides high data transfer rates under all operating conditions, including after a drive failure.

Moren, Bill↗

Asynchronous Object Storage with QoS for Scientific and Commercial Big Data

This paper presents our design for an asynchronous object storage system intended for use in scientific and commercial big data workloads. Use cases from the target workload do- mains are used to motivate the key abstractions used in the application programming interface (API). The architecture of the Scalable Object Store (SOS), a prototype object stor- age system that supports the API s facilities, is presented. The SOS serves as a vehicle for future research into scalable and resilient big data object storage. We briefly review our research into providing efficient storage servers capable of providing quality of service (QoS) contracts relevant for big data use cases.

Cloud Storage↗

Storage and Read-Optimized Data Placement Structures for High-Performance Analysis [Book Chapter]

Here, we present state-of-art structures and methods for efficient data preparation and representation for analysis. Our intent is to introduce the data science and analytics communities to open source data placements, structures, and methods. These practices can make the foundational processes of data preparation and access dramatically more efficient than typical raw file or database representations and use more conservative storage. To illustrate this, we introduce two highly efficient data placement structures. We then present a tutorial, supported by step-by-step examples, of how to create, use, and access data, structured by Parquet or ORC (Optimized Row Columnar), using Apache Spark. Finally, we illustrate the benefits of using these structures with computational and storage volume benchmarks.

97 MATHEMATICS AND COMPUTING↗

Minimally buffered data transfers between nodes in a data communications network

Methods, apparatus, and products for minimally buffered data transfers between nodes in a data communications network are disclosed that include: receiving, by a messaging module on an origin node, a storage identifier, a origin data type, and a target data type, the storage identifier specifying application storage containing data, the origin data type describing a data subset contained in the origin application storage, the target data type describing an arrangement of the data subset in application storage on a target node; creating, by the messaging module, origin metadata describing the origin data type; selecting, by the messaging module from the origin application storage in dependence upon the origin metadata and the storage identifier, the data subset; and transmitting, by the messaging module to the target node, the selected data subset for storing in the target application storage in dependence upon the target data type without temporarily buffering the data subset.

97 MATHEMATICS AND COMPUTING↗

BASTet: Shareable and Reproducible Analysis and Visualization of Mass Spectrometry Imaging Data via OpenMSI

Mass spectrometry imaging (MSI) is a transformative imaging method that supports the untargeted, quantitative measurement of the chemical composition and spatial heterogeneity of complex samples with broad applications in life sciences, bioenergy, and health. While MSI data can be routinely collected, its broad application is currently limited by the lack of easily accessible analysis methods that can process data of the size, volume, diversity, and complexity generated by MSI experiments. The development and application of cutting-edge analytical methods is a core driver in MSI research for new scientific discoveries, medical diagnostics, and commercial-innovation. However, the lack of means to share, apply, and reproduce analyses hinders the broad application, validation, and use of novel MSI analysis methods. To address this central challenge, we introduce the Berkeley Analysis and Storage Toolkit (BASTet), a novel framework for shareable and reproducible data analysis that supports standardized data and analysis interfaces, integrated data storage, data provenance, workflow management, and a broad set of integrated tools. Based on BASTet, we describe the extension of the OpenMSI mass spectrometry imaging science gateway to enable web-based sharing, reuse, analysis, and visualization of data analyses and derived data products. Here, we demonstrate the application of BASTet and OpenMSI in practice to identify and compare characteristic substructures in the mouse brain based on their chemical composition measured via MSI.

97 MATHEMATICS AND COMPUTING↗

Dynamic Testing of a Commercial FRAM Device Under Gamma Ray Dose and Neutron Beam

Here, this work presents dose rate and neutron testing on a ferroelectric random-access memory (FRAM) device in dynamic operation during radiation pulses. Radiation failure modes are shown to be unique. Dose rate failure is shown to be based primarily on integrated dose in the radiation pulse and terminates the FRAM operation and can alter up to 8 bytes of data in storage. Neutron failures are attributed to a shifting of data like a clocking error.

Harris, Nathan↗

Quality Data Essential for Modeling Water Cycles Effectively

The experimental and observational data is the backbone of effective modeling of all aspects of water cycles. As the computer technology used for these data gathering efforts has improved dramatically in the recent decades, the environmental conditions these instruments are subjected to remain pretty much the same. These measuring instruments are exposed to the sun light, wind, rain, snow and ice; and they are immersed in dirt, water and mud. All of these are damaging to the electronics components used for data gathering, storage, and transmission, which creates many different kinds of data quality issues that affect the quality and reliability of modeling and prediction efforts.

54 ENVIRONMENTAL SCIENCES↗

Correlation Analysis Of Vibration Data From Rotary Pumps

Correlation technique represents vibration data from rotary pumps in compact form convenient for storage in data base. Analysis determines coefficients of Fourier series representing typical response of pump for one revolution of pump shaft. Changes in harmonic spectrum of pump indicates changes in performance caused by internal wear, rubbing of seals, and the like.

Fenwick, James R.↗

Deep learning-accelerated 3D carbon storage reservoir pressure forecasting based on data assimilation using surface displacement from InSAR

Fast forecasting of the reservoir pressure distribution during geologic carbon storage (GCS) by assimilating monitoring data is a challenging problem. Due to high drilling cost, GCS projects usually have spatially sparse measurements from few wells, leading to high uncertainties in reservoir pressure prediction. To address this challenge, we use low-cost Interferometric Synthetic-Aperture Radar (InSAR) data as monitoring data to infer reservoir pressure build up. We develop a deep learning-accelerated workflow to assimilate surface displacement maps interpreted from InSAR and to forecast dynamic reservoir pressure. Employing an Ensemble Smoother Multiple Data Assimilation (ES-MDA) framework, the workflow updates three-dimensional (3D) geologic properties and predicts reservoir pressure with quantified uncertainties. We use a synthetic commercial-scale GCS model with bimodally distributed permeability and porosity to demonstrate the efficacy of the workflow. A two-step CNN-PCA approach is employed to parameterize the bimodal fields. The computational efficiency of the workflow is boosted by two residual U-Net based surrogate models for surface displacement and reservoir pressure predictions, respectively. The workflow can complete data assimilation and reservoir pressure forecasting in half an hour on a personal computer.

25 ENERGY STORAGE↗

Science-Driven Data Management for Multi-Tiered Storage (Final Report)

Scientific discovery at the exascale will not be possible without significant new research in the management, storage and retrieval over the long lifespan of the extreme amounts of data that will be produced. Our thesis is that adding application level knowledge about data to guide the actions of the storage system provides substantial benefits to the organization, storage, and access to extreme scale data, resulting in improved productivity for computational science. In this project we will demonstrate novel techniques to facilitate efficient mapping of data objects, even partitioning individual variables, from the user space onto multiple storage tiers, and enable application-guided data reductions and transformations to address capacity and bandwidth bottlenecks. Our goal is to address the associated Input/ Output (I/O) and storage challenges in the context of current and emerging storage landscapes, and expedite insights into mission critical scientific processes.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

CARETS: A prototype regional environmental information system. Volume 3: Toward a national land use information system

It is recommended that a national land use information system be established by an agency of the Federal Government. This recommendation comes at a time of increasing demand for scientific information in support of environmentally relevant land use planning and management at all levels of government. It is also a time when new airborne and spaceborne remote sensors, tested in cooperation with the National Aeronautics and Space Administration (NASA) and the Earth Resources Observation Systems (EROS) Program of the Department of the Interior, make possible the gathering of land use information rapidly and on an unprecedented scale. Furthermore, information handling technology is developing toward a capability to receive, store, and disseminate the huge quantities of data that would be involved. The recommendation for the national land use information system is based upon careful analysis of the results of remote sensing experiments funded by NASA, EROS, and the Geography Program of the Geological Survey, with specific examples drawn from the demonstration project known as the Central Atlantic Regional Ecological Test Site (CARETS). CARETS is cast in the framework of a regional land use information system, channeling the flow bf information generated in response to users' declaration of their needs, through stages dealing with remote sensing data gathering systems, data processing and land mensuration, calibration in-terms of environmental impact, and evaluation with feedback from users. The proposed system would develop and implement a unified approach to the description and interpretation of the changing uses of the nation's land resources, building upon the base of interagency and intergovernmental cooperation already achieved in the experimental work to date. The land use data base that is being derived from high-altitude aerial color infrared photography would be the initial component of the recommended system. High-altitude photographic coverage would immediately be extended to as much of the nation as possible as technological developments and economic considerations permit. The system would later expand to include multiple-sensor, multiple-platform data sources. Six system characteristics are recommended: (1) High capacity storage of data available for quick retrieval, inexpensive processing, and update, (2) provision of accuracy appropriate to the scale of survey or to the level of detail dictated by different types of management and decision requirements; (3) permanent, publicly accessible sensor records for historical interpretation; (4) compatibility of the recording, storage, and retrieval system with all types of inputs, from ground observer to satellite; (5) products of diverse formats and scales, responsive to user feedback; (6) and standardization of formats, scales, and storage inputs to permit nationwide comparability.

Land use mapping↗

DE-FE0029488 - North Dakota Integrated Carbon Capture and Storage Complex Feasibility Study Public Data

Data from award DE-FE0029488 - North Dakota Integrated Carbon Capture and Storage Complex Feasibility Study performed by the Energy & Environmental Research Center including the following: - 2D Seismic {Input data, sgy files, maps, logs, and descriptors} - Core Petrophysics {Core analysis of plugs from the two stratigraphic test wells (Flemmer-1 [API 33-057-00039] and BNI-1 [API 33-065-00018])} - North Dakota Oil and Gas File No 37380 Files - North Dakota Oil and Gas File No 37672 Files - Well Testing Data {Summary of well testing methods and results from the stratigraphic test wells (Flemmer-1 and BNI-1)} Additional References: https://www.netl.doe.gov/sites/default/files/2017-12/Wesley-Peck-_Mastering-the-Subsurface_CarbonSAFE-Phase-II_August-2017-final.pdf Peck, W.D., Ayash, S.C., Klapperich, R.J., Gorecki, C.D. (2019) The North Dakota integrated carbon storage complex feasibility study, International Journal of Greenhouse Gas Control, Volume 84, 2019, Pages 47-53, https://doi.org/10.1016/j.ijggc.2019.03.001

Carbon Storage↗