Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data storage”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,441 records · Page 80

NATCARB

The National Carbon Sequestration Database (NATCARB) is a geographic information system (GIS)-based tool for viewing carbon capture, use and storage (CCUS) potential across the United States. Through a graphical user interface, NATCARB displays locations and associated metadata about stationary CO2 sources and potential geologic sinks. Users can access, query and model, analyze, display, and download CO2 storage resource data.

Atlas↗

NASDA's earth observation satellite data archive policy for the earth observation data and information system (EOIS)

NASDA's new Advanced Earth Observing Satellite (ADEOS) is scheduled for launch in August, 1996. ADEOS carries 8 sensors to observe earth environmental phenomena and sends their data to NASDA, NASA, and other foreign ground stations around the world. The downlink data bit rate for ADEOS is 126 MB/s and the total volume of data is about 100 GB per day. To archive and manage such a large quantity of data with high reliability and easy accessibility it was necessary to develop a new mass storage system with a catalogue information database using advanced database management technology. The data will be archived and maintained in the Master Data Storage Subsystem (MDSS) which is one subsystem in NASDA's new Earth Observation data and Information System (EOIS). The MDSS is based on a SONY ID1 digital tape robotics system. This paper provides an overview of the EOIS system, with a focus on the Master Data Storage Subsystem and the NASDA Earth Observation Center (EOC) archive policy for earth observation satellite data.

Sobue, Shin-ichi↗

Evaluate data lake design for the accelerator control system

Increasing precision in automation for modern particle accelerators not only creates a requirement to gather data from all devices but also demands scalable and high-performance data infrastructure with the capability of handling vast incoming device data. A well architected data lake is suitable for such a system which integrates real-time data acquisition, transient data caching, and long-term storage. This paper evaluates data lake architecture for an Accelerator Control System (ACS), focusing on two critical components of a data lake, data cache and long-term storage.

Jaikar, Amol [Fermilab]↗

Resonant structure of low-energy H{sub 3}{sup +} dissociative recombination

High-resolution dissociative recombination rate coefficients of rotationally cool and hot H{sub 3}{sup +} in the vibrational ground state have been measured with a 22-pole trap setup and a Penning ion source, respectively, at the ion storage-ring TSR. The experimental results are compared with theoretical calculations to explore the dependence of the rate coefficient on ion temperature and to study the contributions of different symmetries to probe the rich predicted resonance spectrum. The kinetic energy release was investigated by fragment imaging to derive internal temperatures of the stored parent ions under differing experimental conditions. A systematic experimental assessment of heating effects is performed which, together with a survey of other recent storage-ring data, suggests that the present rotationally cool rate-coefficient measurement was performed at 380{sub -130}{sup +50} K and that this is the lowest rotational temperature so far realized in storage-ring rate-coefficient measurements on H{sub 3}{sup +}. This partially supports the theoretical suggestion that temperatures higher than assumed in earlier experiments are the main cause for the large gap between the experimental and the theoretical rate coefficients. For the rotationally hot rate-coefficient measurement a temperature of below 3250 K is derived. From these higher-temperature results it is found that increasing the rotational ion temperature in the calculations cannot fully close the gap between the theoretical and the experimental rate coefficients.

43 PARTICLE ACCELERATORS↗

Model-Based Clustering of Regression Time Series Data via APECM -- An AECM Algorithm Sung to an Even Faster Beat

We propose a model-based approach for clustering time series regression data in an unsupervised machine learning framework to identify groups under the assumption that each mixture component follows a Gaussian autoregressive regression model of order p. Given the number of groups, the traditional maximum likelihood approach of estimating the parameters using the expectation-maximization (EM) algorithm can be employed, although it is computationally demanding. The somewhat fast tune to the EM folk song provided by the Alternating Expectation Conditional Maximization (AECM) algorithm can alleviate the problem to some extent. In this article, we develop an alternative partial expectation conditional maximization algorithm (APECM) that uses an additional data augmentation storage step to efficiently implement AECM for finite mixture models. Results on our simulation experiments show improved performance in both fewer numbers of iterations and computation time. The methodology is applied to the problem of clustering mutual funds data on the basis of their average annual per cent returns and in the presence of economic indicators.

99 GENERAL AND MISCELLANEOUS↗

Methods for Specifying Scientific Data Standards and Modeling Relationships with Applications to Neuroscience [Plus Supplementary Data]

Neuroscience continues to experience a tremendous growth in data; in terms of the volume and variety of data, the velocity at which data is acquired, and in turn the veracity of data. These challenges are a serious impediment to sharing of data, analyses, and tools within and across labs. Here, we introduce BRAINformat, a novel data standardization framework for the design and management of scientific data formats. The BRAINformat library defines application-independent design concepts and modules that together create a general framework for standardization of scientific data. We describe the formal specification of scientific data standards, which facilitates sharing and verification of data and formats. We introduce the concept of Managed Objects, enabling semantic components of data formats to be specified as self-contained units, supporting modular and reusable design of data format components and file storage. We also introduce the novel concept of Relationship Attributes for modeling and use of semantic relationships between data objects. Based on these concepts we demonstrate the application of our framework to design and implement a standard format for electrophysiology data and show how data standardization and relationship-modeling facilitate data analysis and sharing. The format uses HDF5, enabling portable, scalable, and self-describing data storage and integration with modern high-performance computing for data-driven discovery. The BRAINformat library is open source, easy-to-use, and provides detailed user and developer documentation and is freely available at: https://bitbucket.org/oruebel/brainformat.

60 APPLIED LIFE SCIENCES↗

Could Blobs Fuel Storage-Based Convergence between HPC and Big Data?

The increasingly growing data sets processed on HPC platforms raise major challenges for the underlying storage layer. A promising alternative to POSIX-IO- compliant file systems are simpler blobs (binary large objects), or object storage systems. Such systems offer lower overhead and better performance at the cost of largely unused features such as file hierarchies or permissions. Similarly, blobs are increasingly considered for replacing distributed file systems for big data analytics or as a base for storage abstractions such as key-value stores or time-series databases. This growing interest in such object storage on HPC and big data platforms raises the question: Are blobs the right level of abstraction to enable storage-based convergence between HPC and Big Data? In this paper we study the impact of blob-based storage for real-world applications on HPC and cloud environments. The results show that blobbased storage convergence is possible, leading to a significant performance improvement on both platforms

Matri, Pierre↗

Interactive Visualization of Near Real Time and Production Global Precipitation Measurement (GPM) Mission Data Online Using CesiumJS

Advancements in the capabilities of JavaScript frameworks and web browsing technology make online visualization of large geospatial datasets viable. Commonly this is done using static image overlays, prerendered animations, or cumbersome geoservers. These methods can limit interactivity andor place a large burden on server-side post-processing and storage of data. Geospatial data, and satellite data specifically, benefit from being visualized both on and above a three-dimensional surface. The open-source JavaScript framework CesiumJS, developed by Analytical Graphics, Inc., leverages the WebGL protocol to do just that. It has entered the void left by the abandonment of the Google Earth Web API, and it serves as a capable and well-maintained platform upon which data can be displayed. This paper will describe the technology behind the two primary products developed as part of the NASA Precipitation Processing System STORM website: GPM Near Real Time Viewer (GPMNRTView) and STORM Virtual Globe (STORM VG). GPMNRTView reads small post-processed CZML files derived from various Level 1 through 3 near real-time products. For swath-based products, several brightness temperature channels or precipitation-related variables are available for animating in virtual real-time as the satellite-observed them on and above the Earths surface. With grid-based products, only precipitation rates are available, but the grid points are visualized in such a way that they can be interactively examined to explore raw values. STORM VG reads values directly off the HDF5 files, converting the information into JSON on the fly. All data points both on and above the surface can be examined here as well. Both the raw values and, if relevant, elevations are displayed. Surface and above-ground precipitation rates from select Level 2 and 3 products are shown. Examples from both products will be shown, including visuals from high impact events observed by GPM constellation satellites.

satellite precipitation measurement↗

Mission Operations Center (MOC) - Precipitation Processing System (PPS) Interface Software System (MPISS)

MPISS is an automatic file transfer system that implements a combination of standard and mission-unique transfer protocols required by the Global Precipitation Measurement Mission (GPM) Precipitation Processing System (PPS) to control the flow of data between the MOC and the PPS. The primary features of MPISS are file transfers (both with and without PPS specific protocols), logging of file transfer and system events to local files and a standard messaging bus, short term storage of data files to facilitate retransmissions, and generation of file transfer accounting reports. The system includes a graphical user interface (GUI) to control the system, allow manual operations, and to display events in real time. The PPS specific protocols are an enhanced version of those that were developed for the Tropical Rainfall Measuring Mission (TRMM). All file transfers between the MOC and the PPS use the SSH File Transfer Protocol (SFTP). For reports and data files generated within the MOC, no additional protocols are used when transferring files to the PPS. For observatory data files, an additional handshaking protocol of data notices and data receipts is used. MPISS generates and sends to the PPS data notices containing data start and stop times along with a checksum for the file for each observatory data file transmitted. MPISS retrieves the PPS generated data receipts that indicate the success or failure of the PPS to ingest the data file and/or notice. MPISS retransmits the appropriate files as indicated in the receipt when required. MPISS also automatically retrieves files from the PPS. The unique feature of this software is the use of both standard and PPS specific protocols in parallel. The advantage of this capability is that it supports users that require the PPS protocol as well as those that do not require it. The system is highly configurable to accommodate the needs of future users.

Ferrara, Jeffrey↗

The Global Trigger Processor: A VXS Switch Module for Triggering Large Scale Data Acquisition Systems

The 12 GeV upgrade for Jefferson Lab's Continuous Electron Beam Accelerator Facility requires the development of a new data acquisition system to accommodate the proposed 200 kHz Level 1 trigger rates expected for fixed target experiments at 12 GeV. As part of a suite of trigger electronics comprised of VXS switch and payload modules, the Global Trigger Processor (GTP) will handle up to 32,768 channels of preprocessed trigger information data from the multiple detector systems that surround the beam target at a system clock rate of 250 MHz. The GTP is configured with user programmable Physics trigger equations and when trigger conditions are satisfied, the GTP will activate the storage of data for subsequent analysis. The GTP features an Altera Stratix IV GX FPGA allowing interface to 16 Sub-System Processor modules via 32 5-Gbps links, DDR2 and flash memory devices, two gigabit Ethernet interfaces using Nios II embedded processors, fiber optic transceivers, and trigger output signals. The GTP's high-bandwidth interconnect with the payload modules in the VXS crate, the Ethernet interface for parameter control, status monitoring, and remote update, and the inherent nature of its FPGA give it the flexibility to be used large variety of tasks and adapt to future needs. This paper details the responsibilities of the GTP, the hardware's role in meeting those requirements, and elements of the VXS architecture that facilitated the design of the trigger system. Also presented will be the current status of development including significant milestones and challenges.

Kaneta, Scott↗

DeFault: DEep‐Learning‐Based FAULT Delineation Using the IBDP Passive Seismic Data at the Decatur CO2 Storage Site

Abstract The carbon capture, utilization, and storage (CCUS) framework is an essential component in reducing greenhouse gas emissions, with its success hinging on the comprehensive knowledge of subsurface geology and geomechanics. Passive seismic event relocation and fault detection offer vital insights into subsurface structures and the ability to monitor fluid migration pathways. Accurate identification and localization of seismic events, however, face significant challenges, including the necessity for high‐quality seismic data and advanced computational methods. To address these challenges, we introduce a novel deep learning method, , specifically designed for passive seismic source relocation and fault delineating for passive seismic monitoring projects. By leveraging data domain‐adaptation, allows us to train a neural network with labeled synthetic data and apply it directly to field data. Using , the passive seismic sources are automatically clustered based on their recording time and spatial locations, and subsequently, faults and fractures are delineated accordingly. We demonstrate the efficacy of on a field case study involving injection related microseismic data from Decatur, Illinois area. Our approach accurately and efficiently relocated passive seismic events, identified faults and could aid in potential damage induced by seismicity. Our results highlight the potential of as a valuable tool for passive seismic monitoring, emphasizing its role in ensuring CCUS project safety. This research bolsters the understanding of subsurface characterization in CCUS, illustrating machine learning’s capacity to refine these methods. Ultimately, our work has significant implications for CCUS technology deployment, an essential strategy in combating climate change. Plain Language Summary In our quest to tackle climate change, we use a strategy known as carbon capture, utilization, and storage (CCUS) to keep greenhouse gases out of the atmosphere. This strategy relies heavily on our ability to understand what's happening deep under the earth's surface. To make sure we store super critical safely, we need to accurately map out the geological structure, especially faults, but this is tough without high‐quality data and complex computer programs. We've developed a new tool called “DeFault,” which uses advanced machine learning to improve how we find and map these underground features. “DeFault” is smart enough to learn from numerically simulated data and then apply what it’s learned to real‐world situations. It groups together seismic activity—tiny tremors and shifts in the earth—based on when and where they happen, which helps us spot where there might be cracks or faults. We tested “DeFault” in Illinois, where CO 2 is injected underground, and it successfully pinpointed where these tremors occurred and mapped out the faults, helping to prevent accidents accurately in the future. Our study shows that “DeFault” will be a powerful ally in making CCUS safer and more effective, especially for the Illinois Basin Decatur Project. Key Points Faults and fractures introduced by carbon storage can be monitored by passive seismicity DeFault algorithm enables an automatic process for accurate and efficient passive seismic event locating and clustering

58 GEOSCIENCES↗

Data from: "Towards CONUS-Wide ML-Augmented Conceptually-Interpretable Modeling of Catchment-Scale Precipitation-Storage-Runoff Dynamics"

This data package was generated to support the manuscript “Towards CONUS-Wide Machine Learning-Augmented Conceptually Interpretable Modeling of Catchment-Scale Precipitation-Storage-Runoff Dynamics.” It provides input files, model outputs, plotting data, scripts, notebooks, and documentation used to develop, evaluate, and reproduce Mass-Conserving Perceptron (MCP)-based hydrologic modeling experiments across 513 selected Catchment Attributes and Meteorology for Large-sample Studies in the United States (CAMELS-US) basins. The files are organized by modeling component and analysis purpose, including rainfall–runoff experiments, snow module experiments, coupled hydrologic-snow experiments, Long Short-Term Memory (LSTM) benchmark results, model skill metrics, initialization and epoch records, cell-state normalization files, Akaike Information Criterion (AIC)-based model comparison files, and data used to generate manuscript figures. Tabular files can be opened using standard spreadsheet software or Python/R data-analysis tools. Python scripts, Jupyter notebooks, and selected MATLAB scripts are included for model execution, postprocessing, plotting, and statistical analysis. Quality assurance and quality control were conducted through the source-data selection and modeling workflow. Meteorological forcing, streamflow, and static catchment attributes were derived from the CAMELS-US dataset, and snow water equivalent data were derived from the University of Arizona (UA) Snow Water Equivalent dataset. Selected basins and time periods were screened during the associated research workflow to avoid missing observations or poor-quality cases. Static geospatial features were processed primarily using Quantum Geographic Information System (QGIS) and Geospatial Data Abstraction Library (GDAL) workflows. Additional details are provided in the associated manuscript and documentation.

ESS-DIVE CSV File Formatting Guidelines Reporting ↗

A modular architecture for multi-channel external cavity quantum cascade laser-based chemical sensors: a systems approach

A multi-channel laser-based chemical sensor platform is presented, in which a modular architecture allows the exchange of complete sensor channels without disruption to overall operation. Each sensor channel contains custom optical and electronics packages, which can be selected to access laser wavelengths, interaction path lengths and modulation techniques optimal for a given application or mission. Although intended primarily to accommodate mid-infrared (MIR) external cavity quantum cascade lasers (ECQCLs)and astigmatic Herriott cells, channels using visible or near infrared (NIR) lasers or other gas cell architectures can also be used, making this a truly versatile platform. Analog and digital resources have been carefully chosen to facilitate small footprint, rapid spectral scanning, ow-noise signal recovery, failsafe autonomous operation, and in-situ chemometric data analysis, storage and transmission. Results from the demonstration of a two-channel version of this platform are also presented.

99 GENERAL AND MISCELLANEOUS↗

DOECGF 2010 Site Report

The Data group provides data analysis and visualization support to its customers. This consists primarily of the development and support of VisIt, a data analysis and visualization tool. Support ranges from answering questions about the tool, providing classes on how to use the tool, and performing data analysis and visualization for customers. The Information Management and Graphics Group supports and develops tools that enhance our ability to access, display, and understand large, complex data sets. Activities include applying visualization software for large scale data exploration; running video production labs on two networks; supporting graphics libraries and tools for end users; maintaining PowerWalls and assorted other displays; and developing software for searching and managing scientific data. Researchers in the Center for Applied Scientific Computing (CASC) work on various projects including the development of visualization techniques for large scale data exploration that are funded by the ASC program, among others. The researchers also have LDRD projects and collaborations with other lab researchers, academia, and industry. The IMG group is located in the Terascale Simulation Facility, home to Dawn, Atlas, BGL, and others, which includes both classified and unclassified visualization theaters, a visualization computer floor and deployment workshop, and video production labs. We continued to provide the traditional graphics group consulting and video production support. We maintained five PowerWalls and many other displays. We deployed a 576-node Opteron/IB cluster with 72 TB of memory providing a visualization production server on our classified network. We continue to support a 128-node Opteron/IB cluster providing a visualization production server for our unclassified systems and an older 256-node Opteron/IB cluster for the classified systems, as well as several smaller clusters to drive the PowerWalls. The visualization production systems includes NFS servers to provide dedicated storage for data analysis and visualization. The ASC projects have delivered new versions of visualization and scientific data management tools to end users and continue to refine them. VisIt had 4 releases during the past year, ending with VisIt 2.0. We released version 2.4 of Hopper, a Java application for managing and transferring files. This release included a graphical disk usage view which works on all types of connections and an aggregated copy feature for quickly transferring massive datasets quickly and efficiently to HPSS. We continue to use and develop Blockbuster and Telepath. Both the VisIt and IMG teams were engaged in a variety of movie production efforts during the past year in addition to the development tasks.

97 MATHEMATICS AND COMPUTING↗

Assessment of CO 2 Storage Resources in Depleted Oil and Gas Fields in the Ship Shoal Area, Gulf of Mexico

The Gulf of Mexico is one of the most important regions in the United States for energy resources and infrastructure. Gulf of Mexico federal offshore oil and gas production accounts for 17% of total U.S. crude oil production and 5% of total U.S. gas production (EIA Gulf of Mexico Fact Sheet). This region presents an excellent combination of high need and significant opportunity for large scale geologic storage of CO 2 . The Ship Shoal Area is located about 20 miles offshore Louisiana within the Gulf Coast federal waters. Miocene and Pliocene sediments in the Ship Shoal Area are proven to provide excellent and secure traps for oil and gas. The Ship Shoal Area contains a large number of depleted oil and gas fields either currently abandoned, or planned for abandonment by 2025, which may provide very significant potential CO 2 storage capacity. GeoMechanics Technologies conducted a comprehensive research project to better characterize Neogene sediments in the Ship Shoal Area for high volume CO 2 storage. The data generated from literature, well data, well log, and formation evaluation were input into Rockwork 16 geologic software to create a detailed geologic model for the Ship Shoal study area. The geologic grids were then fed into TOUGH2, the gas migration model software and FLAC 3D, the geomechanical model software. The fluid flow modeling for both the Miocene and Pliocene simulation from both Block 107 and Block 84 indicated that Pliocene and Miocene are a good reservoir for the CO 2 sequestration. Thirty years of CO 2 injection (at a rate of 1 million metric tons CO 2 per year) and 30 years of observation simulations were run with the fluid flow modeling. The 3D geomechanical modeling for Ship Shoal Block 107 and Block 84 were developed to evaluate induced surface deformation and potential fault reactivation after 30 years CO 2 injection at base of Pliocene and at upper Miocene injection locations. Evaluation results indicate low to no risks for fault slips or fault reactivation. Twelve well bores and 76 well bores for Ship Shoal Blocks 84 and 107 respectively were reviewed for their cement history. Most wells (57 out of 76 and 7 out of 12 wells in Ship Shoal Block 107 and 84 respectively) have good integrity. Nineteen and 5 wells (in Ship Shoal Block 107 and 84 respectively) with no top plug, incomplete cement or Plug and Abandonment information are given yellow cautionary indicators. These cautionary wells may provide leakage paths of CO 2 through the well bores to the USDWs. Using our Quantitative Risk & Decision Analysis Tool (QRDAT) for caprock integrity evaluation, we compared Ship Shoal’s risk to that of In Salah, Sleipner, Kevin Dome, Loudon, Illinois Industrial CCS and Wilmington Graben. We found the risk at the Ship Shoal Blocks 84 and 107 fields are similar to the known CO 2 active sequestration sites, but lower than the Wilmington Graben turbidities offshore California studied site. The risk of natural seismicity in the Gulf of Mexico is relatively low. Geomechanics Technologies has documented the top 25 CO 2 emission sources within the close proximity of the Ship Shoal Block 84 and 107 fields. All the offshore and onshore pipelines have been digitized and can be viewed in an interactive website (http://www.geomechanicstech.com/shipshoal.html). We also performed a feasibility study on the potential for converting existing oil or gas pipelines for CO 2 transport. There are abandoned, idled and retired onshore and offshore pipelines. However, there is no standard specification for maximum pipeline pressures needed for CO 2 transport. It is a function of design, materials, and testing provided; thus it will be the responsibility of the pipeline operator to correctly determine, maintain and operate within the limits of the pipelines. There are a few transit corridors extending from onshore Louisiana (Cailou Bay) to offshore trunk-lines. The cost for constructing a pipeline has increased about 46% from 2015 to an average cost of $5,064,046 per mile in 2016. We use the NETL CO 2 storage resource mass estimate to calculate the potential resources for Ship Shoal Block 84 and 107 fields using the depleted oil and gas reservoir volume and sand volume generated from our geologic modeling. The estimated storage resource results are greater when using the sand volumes derived through geologic modeling than the oil and gas reservoir data. The difference is due to the depleted oil and gas reservoir information not accounting for the water-flooded sand located below the oil/gas-water contact or unproductive sand units. Resource calculation using the sand volume obtained through geologic modeling overestimate the storage capacity as the model accounts for all sand within the formation, not just the interconnected sand.

02 PETROLEUM↗

Application of a Prize Mechanism to Address Data Utilization Challenges at Utilities

The electric industry sector is facing an “explosion” of data from a variety of sources. Electric sector stakeholders need to define how to capitalize on large datasets, both those they create and those from other sources (like data on weather, buildings, electric vehicles, etc.), to improve reliability and resilience and meet the changing system dynamics from renewable integration. For the electricity sector to fully utilize these vast new datasets, it must undergo a transformation in how it manages data quality, storage, and processing. The U.S. Department of Energy (DOE) Office of Electricity (OE) is committed to accelerating research, development, and demonstration of new technologies and tools within the electricity sector to advance reliability, resilience, and affordable operation of the power system. Through the prize mechanism, OE identified two widespread data-related challenges for utilities—load modeling and data analysis automation—and offered an opportunity for utilities and teams of software engineers to identify additional challenges faced by utilities. After completing one round of the American-Made Digitizing Utilities Prize, OE, the National Renewable Energy Laboratory (NREL) as the prize administrator, and Pacific Northwest National Laboratory (PNNL) as the domain experts have compiled the results and lessons learned to feed into the second round of the prize.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

Volume server: A scalable high speed and high capacity magnetic tape archive architecture with concurrent multi-host access

A major challenge facing data processing centers today is data management. This includes the storage of large volumes of data and access to it. Current media storage for large data volumes is typically off line and frequently off site in warehouses. Access to data archived in this fashion can be subject to long delays, errors in media selection and retrieval, and even loss of data through misplacement or damage to the media. Similarly, designers responsible for architecting systems capable of continuous high-speed recording of large volumes of digital data are faced with the challenge of identifying technologies and configurations that meet their requirements. Past approaches have tended to evaluate the combination of the fastest tape recorders with the highest capacity tape media and then to compromise technology selection as a consequence of cost. This paper discusses an architecture that addresses both of these challenges and proposes a cost effective solution based on robots, high speed helical scan tape drives, and large-capacity media.

Rybczynski, Fred↗