Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “DATA STORAGE”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,765 records · Page 98
All sourcesNASA NTRSDOE OSTI
NASA NTRS1993

Storage system architectures and their characteristics

Not all users storage requirements call for 20 MBS data transfer rates, multi-tier file or data migration schemes, or even automated retrieval of data. The number of available storage solutions reflects the broad range of user requirements. It is foolish to think that any one solution can address the complete range of requirements. For users with simple off-line storage requirements, the cost and complexity of high end solutions would provide no advantage over a more simple solution. The correct answer is to match the requirements of a particular storage need to the various attributes of the available solutions. The goal of this paper is to introduce basic concepts of archiving and storage management in combination with the most common architectures and to provide some insight into how these concepts and architectures address various storage problems. The intent is to provide potential consumers of storage technology with a framework within which to begin the hunt for a solution which meets their particular needs. This paper is not intended to be an exhaustive study or to address all possible solutions or new technologies, but is intended to be a more practical treatment of todays storage system alternatives. Since most commercial storage systems today are built on Open Systems concepts, the majority of these solutions are hosted on the UNIX operating system. For this reason, some of the architectural issues discussed focus around specific UNIX architectural concepts. However, most of the architectures are operating system independent and the conclusions are applicable to such architectures on any operating system.

Sarandrea, Bryan M.↗
NASA NTRS2012

NAND flash screening and qualification guideline for space application

All space missions have a need for nonvolatile memory (NVM), which maintains data integrity when unpowered. Types of NVM include PROM, EEPROM, NOR Flash, and NAND Flash. PROMs, NOR Flash, and EEPROMs are good choices for storing smaller file size data such as boot code or FPGA configurations. These products have excellent data retention characteristics and can reliably store data for years, unpowered, without any data corruption. They can also be read many times without disturbing the data. Another application of NVM is storage of science and engineering data, which requires large of amounts of memory. The highest density memories available today are SDRAM and NAND Flash. However, the power required to operate and store data in NAND Flash is far less than SDRAM. Wherever high density and low power is required, NAND Flash is very attractive.

Heidecker, Jason↗
NASA NTRSDate not supplied

Opening Historical Airborne Data to Present Day Researchers

For more than 50 years, NASA has flown airborne sensors to carry out research, validate satellite sensors, and test new instrument capabilities. Data collected prior to 2000 are typically analog and difficult to locate and use. The Airborne Data Management Group (ADMG) facilitates rescue of these valuable data to ensure easier discovery, access, and use. But opening historical data comes at a cost of both time and money. Careful decisions are required in assessing the return on investment. - Is there interest in the science community? - Are there government data requirements? - What is the temporal / spatial value of the data? - Can data be transformed to a digital format? - What is cost of transformation? - What time period is needed for rescue? Converting the data to today’s digital storage standards increases value and provides data access. The addition of metadata makes the data easier to search for.

Deborah Smith↗
DOE OSTI2017

Retrofitting a Geothermal Plant with Solar and Storage to Increase Power Generation

Solar hybridization using concentrating solar power (CSP) can be an effective approach to augment the power generation and power cycle efficiency of a geothermal power plant with a declining resource. Thermal storage can further increase the dispatchability of a geothermal/solar hybrid system, which is particularly valued for a national grid with high renewable penetration. In this paper, a hybrid plant design with thermal storage is proposed based on the requirements of the Coso geothermal field in China Lake, California. The objective is to increase the power production by 4 MWe. In this system, a portion of the injection brine is recirculated through a heat exchanger with the solar heat transfer fluid, before being mixed with the production well brine. In the solar heating loop the brine should be heated to at least 155 degrees C to increase the net power. The solar field and storage were sized based on solar data for China Lake. Thermal storage is used to store excess power at the high-solar-irradiation hours and generate additional power during the evenings. The solar field size, the type and capacity of thermal storage and the operating temperatures are critical factors in determining the most economic hybrid system. Further investigations are required to optimize the hybrid system and evaluate its economic feasibility.

15 GEOTHERMAL ENERGY↗
DOE OSTI2023

HERMES: A cyber-energy SAAS platform to provide real-time 2d and 3d interactive visualizations [SWR-23-15]

HERMES is a novel visualization SAAS platform for achieving real-time visualization of large-scale environments involving cyber-energy devices. The platform receives data from emulated environments which include physical hardware and network devices that represent a complex IT / OT system. The platform is capable of streaming, filtering, storing, and visualizing all data within the environment and scaling vertically and horizontally. This capability enables high fidelity visual analysis of events to be performed in real time as well as the collection and storage of historical data for forensic analysis.

Van Natta, Joshua↗
NASA NTRS1981

Science and Applications Space Platform (SASP) End-to-End Data System Study

The capability of present technology and the Tracking and Data Relay Satellite System (TDRSS) to accommodate Science and Applications Space Platforms (SASP) payload user's requirements, maximum service to the user through optimization of the SASP Onboard Command and Data Management System, and the ability and availability of new technology to accommodate the evolution of SASP payloads were assessed. Key technology items identified to accommodate payloads on a SASP were onboard storage devices, multiplexers, and onboard data processors. The primary driver is the limited access to TDRSS for single access channels due to sharing with all the low Earth orbit spacecraft plus shuttle. Advantages of onboard data processing include long term storage of processed data until TRDSS is accessible, thus reducing the loss of data, eliminating large data processing tasks at the ground stations, and providing a more timely access to the data.

Crawford, P. R.↗
NASA NTRS1993

Mass optical storage-tape (MOST)

In today's large mainframe and supercomputer environment there exists a continuous demand for increased performance in digital storage systems. The user need for near-line storage capacity is currently doubling every four years. In addition to higher capacity, a desire exists for higher data transfer rates, and longer term database archivability, at lower, and lower cost. Each component of this quartet of demands appears to be insatiable. Magnetic tape technology presently dominates the digital mass storage markets, but the continuous growth of requirements is drawing attention to the limitations of the technology as an archival mass storage medium. The lowest cost option currently available for long term data storage is to use magnetic tape, although it is not well suited to meeting the need for many tens of years of reliable storage. Today, the majority of magnetic tape mass storage systems are based on the IBM 3480/3490 (or compatible) tape drives. These drives offer only moderate transfer rates and relatively small increments of storage, both of which create a logistics problem due to the large numbers of cartridges necessary in a typical system and the time taken to transfer data. Both higher cartridge capacity and data transfer rates are available in some helical scan magnetic tape systems; however, these command a substantially higher price, exacerbating the cost problem, and are not compatible with most installed systems or tape databases.

Oakley, William S.↗
DOE OSTI2026

REV-INR: Regularized Evidential Implicit Neural Representation for Uncertainty-Aware Volume Visualization

Applications of Implicit Neural Representations (INRs) have emerged as a promising deep learning approach for compactly representing large volumetric datasets. These models can act as surrogates for volume data, enabling efficient storage and on-demand reconstruction via model predictions. However, conventional deterministic INRs only provide value predictions without insights into the model’s prediction uncertainty or the impact of inherent noisiness in the data. This limitation can lead to unreliable data interpretation and visualization due to prediction inaccuracies in the reconstructed volume. Identifying erroneous results extracted from model-predicted data may be infeasible, as raw data may be unavailable due to its large size. To address this challenge, we introduce REV-INR, Regularized Evidential Implicit Neural Representation, which learns to predict data values accurately along with the associated coordinate-level data uncertainty and model uncertainty using only a single forward pass of the trained REV-INR during inference. By comprehensively comparing and contrasting REV-INR with existing well-established deep uncertainty estimation methods, we show that REV-INR achieves the best volume reconstruction quality with robust data (aleatoric) and model (epistemic) uncertainty estimates using the fastest inference time. Consequently, we demonstrate that REV-INR facilitates assessment of the reliability and trustworthiness of the extracted isosurfaces and volume visualization results, enabling analyses to be solely driven by model-predicted data.

Saklani, Shanu [Indian Institute of Technology, Ka↗
NASA NTRS1978

A conceptual design for an integrated data base management system for remote sensing data

The requirements of potential users were considered in the design of an integrated data base management system, developed to be independent of any specific computer or operating system, and to be used to support investigations in weather and climate. Ultimately, the system would expand to include data from the agriculture, hydrology, and related Earth resources disciplines. An overview of the system and its capabilities is presented. Aspects discussed cover the proposed interactive command language; the application program command language; storage and tabular data maintained by the regional data base management system; the handling of data files and the use of system standard formats; various control structures required to support the internal architecture of the system; and the actual system architecture with the various modules needed to implement the system. The concepts on which the relational data model is based; data integrity, consistency, and quality; and provisions for supporting concurrent access to data within the system are covered in the appendices.

Maresca, P. A.↗
DOE OSTI2026

Project Phase 1 Report: Reducing Data Center Peak Cooling Demand and Energy Costs With Cold Underground Thermal Energy Storage (Cold UTES)

Cold Underground Thermal Energy Storage (Cold UTES) is an ultra-long duration grid energy storage technology. With Cold-UTES, low-cost grid power is converted to cold thermal energy and stored in the native subsurface rock at the point of use. Cold UTES is one approach within the general category of engineered geothermal systems. Cold UTES for peak-hour cooling of data centers (DCs) was studied for deployment in Maricopa County, Arizona and Loudoun County, Virgina using thermal storage capacities from 4 GWh-th to over 1,000 GWh-th (>1 Terawatt-hour). The two sites have different power and transmission systems, available grid energy resources, daily and seasonal load profiles, weather conditions, and grid regulatory requirements. The study results indicate high value for both locations and because of this, likely indicates value across most of the US and the world. The basis of the study was a 1,000 MW-e hourly electric use of DC computing and auxiliary loads, which was modeled as 1 GW-th of thermal load to a dry cooled heat rejection system - i.e. a cooling system that does not consume water. The electric power required for cooling the DC varies as the air temperature changes. In cold weather only the dry-coolers are used, with an electrical load for cooling load as low as 10 MW-e. In hot summer hours, chillers and dry- coolers are required, which raises the electrical load for cooling load to as much as 300 MW-e. The continuous and peak cooling electrical loads result in a grid interconnection requirement of no less than 1,300 MW-e. From both a grid and thermal design modeling perspective the 1.3 GW-e could either be a single facility or result from the total load at multiple sites.

15 GEOTHERMAL ENERGY↗
NASA NTRS2019

Harmony: Bringing Data Together by Bringing Services Together

As Earth observation data volume, velocity, and variety continues to accelerate, so do the challenges of working with those data. The computation, storage, and network transfer requirements for multi-petabyte datasets demand solutions that allow users to process, store, and move fewer bytes while still finding the valuable insights they seek. Our cloud-based data transformation framework, Harmony, tackles these challenges by providing consistent, standards-based interface to multiple disparate backend processors collocated with NASA EOSDIS data. It allows data scientists to perform data processing and harmonization activities across many datasets through familiar tooling. Simultaneously, it provides a clear, consistent interface for data providers to expose new or customized backend services for their data. This talk will discuss Harmony, its architecture and interface, and the path forward for cloud-based NASA EOSDIS data harmonization services.

Quinn, Patrick↗
NASA NTRSDate not supplied

Microbial Vessel for Impedance Spectroscopy and Electrochemistry (Mvise): an Extensible, Interoperable Data Acquisition Platform for Liquid Culture Studies in Space Biology Research

The White House Office of Science and Technology Policy (OSTP) has declared 2023 to be the Year of Open Science following an initiative to democratize scientific knowledge. Simultaneously, new sensor technologies have broadened the experimental space available to bioastronautics research. With these open-science goals and technological advances in mind, we have designed and constructed a data acquisition platform for high-precision, real-time monitoring of liquid culture systems. The vessel rig is fitted with six Atlas Scientific probes (micro pH, electrical conductivity, dissolved oxygen, oxidation-reduction potential, liquid temperature, air CO2) and a custom optical density probe similar to the one on BioSentinel’s BioSensor payload. A custom dielectric spectroscopy probe is also planned. The structure of the vessel is resin 3-D printed on a hobbyist-level machine, reducing the production cost and iteration time by over 60% each while increasing extensibility. Data acquisition and storage is controlled with a standalone C state machine-based program running on a Raspberry Pi 3 Model B. When not running headless, an additional program automatically generates and updates plots for live data visualization. Validation of the rig as a data collection system was performed with a yeast liquid culture experiment. While the vessel rig is currently used for standalone experiments, it can also be used as the base perception unit in a self-driving laboratory (SDL). SDLs are high-throughput data collection systems that employ automation and artificial intelligence to conduct and manage routine experiments. Here, we envision an SDL driven by several vessel rigs in which an automated script compares key results, informing the design of future experiments. A vessel rig SDL would streamline many operations, including 1) strain selection for the Lunar Explorer Instrument for space biology Applications (LEIA) investigation and 2) the study of bioregenerative life support systems (BLSS). Ultimately, the datasets that can now be acquired will provide crucial information for accelerating bioastronautics application development in the era of commercial space.

Stephen Lantin↗
DOE OSTI2017

Storage 2020: A Vision for the Future of HPC Storage

The explosive growth in data over the next five years that will accompany exascale simulations and new experimental detectors will enable new data-driven science across virtually every domain. At the same time, new nonvolatile storage technologies will enter the market in volume and upend long-held principles used to design the storage hierarchy. The disruption that these forces will bring to bear on high-performance computing (HPC) will also create significant opportunities to innovate and accelerate scientific discovery. To ensure that NERSC fully capitalizes on these opportunities, we have developed a comprehensive vision for the future of storage in HPC and identified short- and long-term strategic goals to effectively realize this vision. This report presents the results of this effort and offers a blueprint for designing a storage infrastructure for supporting HPC through 2025 and beyond.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2014

Checksumming strategies for data in volatile memories

The increase in the number of processors needed to build exascale systems implies that the mean time between failure will further decrease, making it increasingly important to develop scalable techniques for fault tolerance. In this paper we develop an efficient checksum-based approach to fault tolerance for data in volatile memory systems, i.e., an approach without the need to save any data on stable persistent storage. The developed scheme is applicable in multiple scenarios, including: 1) online recovery of large read-only data structures from the memory of failed nodes, with very low storage overhead 2) online recovery from soft errors in blocked data, and 3) online recovery of read/write data via in-memory check-pointing. The approach uses a logical multi-dimension view of the data to be protected. Changing the dimensionality of the data view enables a trade-off between multiple factors, including the storage overheads, the checksum generation time, the failure recovery time, and the number of faults that can be tolerated. Experimental results demonstrating effectiveness are presented on a Cray XE6 system.

Arafat, Md H.↗
DOE OSTI2020

Ad Hoc File Systems for High-Performance Computing

Storage backends of parallel compute clusters are still based mostly on magnetic disks, while newer and faster storage technologies such as flash-based SSDs or non-volatile random access memory (NVRAM) are deployed within compute nodes. Including these new storage technologies into scientific workflows is unfortunately today a mostly manual task, and most scientists therefore do not take advantage of the faster storage media. One approach to systematically include nodelocal SSDs or NVRAMs into scientific workflows is to deploy ad hoc file systems over a set of compute nodes, which serve as temporary storage systems for single applications or longer-running campaigns. This paper presents results from the Dagstuhl Seminar 17202 “Challenges and Opportunities of User-Level File Systems for HPC” and discusses application scenarios as well as design strategies for ad hoc file systems using node-local storage media. The discussion includes open research questions, such as how to couple ad hoc file systems with the batch scheduling environment and how to schedule stage-in and stage-out processes of data between the storage backend and the ad hoc file systems. Also presented are strategies to build ad hoc file systems by using reusable components for networking and how to improve storage device compatibility. Various interfaces and semantics are presented, for example those used by the three ad hoc file systems BeeOND, GekkoFS, and BurstFS. Their presentation covers a range from file systems running in production to cutting-edge research focusing on reaching the performance limits of the underlying devices.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS2017

Ecological Study of Lagoons Surrounding the John F. Kennedy Space Center, Brevard County, Florida : Theses and Project Reports - Volume 2

Many of the detailed studies done in connection with Ecological Study of the Lagoons surrounding the John F. Kennedy Space Port were performed as Master's Thesis investigations by various graduate students enrolled at F. I. T. during and subsequent to the report. The scope and purpose of what we came to call "the KSC Baseline Study" caught the imagination and interest of our student body. Many who were not financially or otherwise connected with the project found their inspiration in studies that were directly connected with project, and thus added materially to the totality of the knowledge gained. An example of one such study is the first article included in this Volume, A Master's Thesis study performed by Shen Phillip Chen, who chose the site for his investigation so that his results would correlate with and extend the results of others. In addition to the Master's Theses contained in this Volume, six other graduate studies must be acknowledged here as contributing to this Report, although they have not reached the stage of final publication. Ms. Sandra Fettes has completed a study of the amounts of five trace metals in mangrove leaves from plants at various locations around the Kennedy Space Center. Mr. Bernard Cohenour has isolated and identified a number of oil consuming bacteria endemic in the waters of the Indian Rivers. Mr. Charles Waterhouse has analyzed historic data of tidal gauges in the lagoonal area and correlated it with wind field records. Mr. Renkert Meyer has measured the vertical and horizontal currents of the lagoons and is attempting an interpretation of them in terms of the wind field as a driving force. Mr. Richard Campbell has measured the rate of nitrogen fixation in both the water columns and the sediments under them in the lagoons. Mr. Craig Weiderhold has measured thl3 annual variations in the populations of benthic invertebrates in the lagoons. An integral part of the F. I. T. curriculum is a requirement that each undergraduate student perform an independent study during his Senior year and submit his results in the form and format of a professional scientific report. While none of these senior Project Reports have been included in this Volume because of space limitations, many of them aided significantly in the overall study. Typical of these were: light and dark bottle studies of the blue-green algae respiration; writing and executing computer studies comparing the water chemistry data from various sites; detection and identification of nitrogen fixation bacteria in the Indian River; measurement of dissolved oils and greases in the river, photographically recording the benthic life-forms found in the river bottom; measuring the vertical structure of drift and slope currents in too lagoons; and measurement of water level variations along the long axis of a lagoon as a function of wind stress. These articles are published as a part of this Final Report in an effort to make available for the record as much of the basic data as possible. The few articles published on the Indian River have in general not contained much detailed data, so that comparisons of current status to conditions of the past are difficult or impossible. A major effort of both the Oceanographic and Ecological communities for the past decade has been the retrieval and storage of basic data in order to permit determinations of trends or changes in baseline conditions. It is believed that the information compiled here will be of great value in any future investigations.

Source record↗
DOE OSTI2014

The 'last mile' of data handling: Fermilab's IFDH tools

IFDH (Intensity Frontier Data Handling), is a suite of tools for data movement tasks for Fermilab experiments and is an important part of the FIFE[2] (Fabric for Intensity Frontier [1] Experiments) initiative described at this conference. IFDH encompasses moving input data from caches or storage elements to compute nodes (the 'last mile' of data movement) and moving output data potentially to those caches as part of the journey back to the user. IFDH also involves throttling and locking to ensure that large numbers of jobs do not cause data movement bottlenecks. IFDH is realized as an easy to use layer that users call in their job scripts (e.g. 'ifdh cp'), hiding the low level data movement tools. One advantage of this layer is that the underlying low level tools can be selected or changed without the need for the user to alter their scripts. Logging and performance monitoring can also be added easily. This system will be presented in detail as well as its impact on the ease of data handling at Fermilab experiments.

Lyon, Adam L.↗
DOE OSTI2022

A dictionary learning algorithm for compression and reconstruction of streaming data in preset order

There has been an emerging interest in developing and applying dictionary learning (DL) to process massive datasets in the last decade. Many of these efforts, however, focus on employing DL to compress and extract a set of important features from data, while considering restoring the original data from this set a secondary goal. On the other hand, although several methods are able to process streaming data by updating the dictionary incrementally as new snapshots pass by, most of those algorithms are designed for the setting where the snapshots are randomly drawn from a probability distribution. In this paper, we present a new DL approach to compress and denoise massive dataset in real time, in which the data are streamed through in a preset order (instances are videos and temporal experimental data), so at any time, we can only observe a biased sample set of the whole data. Here, our approach incrementally builds up the dictionary in a relatively simple manner: if the new snapshot is adequately explained by the current dictionary, we perform a sparse coding to find its sparse representation; otherwise, we add the new snapshot to the dictionary, with a Gram-Schmidt process to maintain the orthogonality. To compress and denoise noisy datasets, we apply the denoising to the snapshot directly before sparse coding, which deviates from traditional dictionary learning approach that achieves denoising via sparse coding. Compared to full-batch matrix decomposition methods, where the whole data is kept in memory, and other mini-batch approaches, where unbiased sampling is often assumed, our approach has minimal requirement in data sampling and storage: i) each snapshot is only seen once then discarded, and ii) the snapshots are drawn in a preset order, so can be highly biased. Through experiments on climate simulations and scanning transmission electron microscopy (STEM) data, we demonstrate that the proposed approach performs competitively to those methods in data reconstruction and denoising.

97 MATHEMATICS AND COMPUTING↗