Engineering Papers⌕ Search
Home/Archive/DOE OSTI

DOE OSTI · 1760648

Data Storage and Visualization Solutions for Real Time Simulations and Experiments

Prabakar, Kumaraguru (ORCID:0000000229620743)·Brunhart-Lupo, Nicholas·Wunder, Nick·Lunacek, Monte·Potter, Kristi·Pailing, Courtney·Munch, Kristin (ORCID:0000000349965172)·Pratt, Annabelle
View original source ↗Read source document ↗Export metadata ↓

Abstract

There are multiple challenges in performing real time simulations and real time experiments. Data storage and visualization are challenges that could be solved easily and when solved provide rich and valuable insights into the experiments. In this presentation, we will talk about the National Renewable Energy Laboratory solutions to tackle these issues while running real time experiments and simulations. The NREL team has made these open-source for the community to utilize with real time simulations and experiments.

Explore related subjects

data storageMATHEMATICS AND COMPUTING,SOLAR ENERGYreal time experimentsreal time simulationsvisualization

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Prabakar, Kumaraguru (ORCID:0000000229620743), Brunhart-Lupo, Nicholas, Wunder, Nick, Lunacek, Monte, Potter, Kristi, Pailing, Courtney, Munch, Kristin (ORCID:0000000349965172), Pratt, Annabelle. 2021-01-04. Data Storage and Visualization Solutions for Real Time Simulations and Experiments. https://www.osti.gov/biblio/1760648

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related reports

NASA NTRS2017

Re-Organizing Earth Observation Data Storage to Support Temporal Analysis of Big Data

The Earth Observing System Data and Information System archives many datasets that are critical to understanding long-term variations in Earth science properties. Thus, some of these are large, multi-decadal datasets. Yet the challenge in long time series analysis comes less from the sheer volume than the data organization, which is typically one (or a small number of) time steps per file. The overhead of opening and inventorying complex, API-driven data formats such as Hierarchical Data Format introduces a small latency at each time step, which nonetheless adds up for datasets with O(10^6) single-timestep files. Several approaches to reorganizing the data can mitigate this overhead by an order of magnitude: pre-aggregating data along the time axis (time-chunking); storing the data in a highly distributed file system; or storing data in distributed columnar databases. Storing a second copy of the data incurs extra costs, so some selection criteria must be employed, which would be driven by expected or actual usage by the end user community, balanced against the extra cost.

data storage↗
NASA NTRS2004

Performance of multi level error correction in binary holographic memory

At the Optical Computing Lab in the Jet Propulsion Laboratory (JPL) a binary holographic data storage system was designed and tested with methods of recording and retrieving the binary information. Levels of error correction were introduced to the system including pixel averaging, thresholding, and parity checks. Errors were artificially introduced into the binary holographic data storage system and were monitored as a function of the defect area fraction, which showed a strong influence on data integrity. Average area fractions exceeding one quarter of the bit area caused unrecoverable errors. Efficient use of the available data density was discussed. .

data storage↗