SEARCH · Engineering Papers

Results for “DATA STORAGE”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,369 records · Page 76

All sources NASA NTRS DOE OSTI

DOE OSTI2026

Vanderbilt CMS Heavy-Ion Tier-2 Facility (Final Report)

The CMS experiment at the LHC at CERN in Geneva, Switzerland spends a portion of its running time colliding heavy ions. These heavy ion collisions are a key probe of a state of matter known as a Quark Gluon Plasma, which is a state of matter found in the very early universe 10s of microseconds after the big bang. These collision data are recorded by the CMS experiment and then undergo several stages of refinement to extract physics properties of the collisions from the raw data recorded by the various detector subsystems in CMS. In addition, copies of these datasets are stored on tape archival storage for data preservation. This award funds the deployment, maintenance, and operation of a computing facility at Vanderbilt University which processes, stores, and analyzes these collision data.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

NASA NTRS1975

Computer program compatible with a laser nephelometer

The laser nephelometer data system was updated to provide magnetic tape recording of data, and real time or near real time processing of data to provide particle size distribution and liquid water content. Digital circuits were provided to interface the laser nephelometer to a Data General Nova 1200 minicomputer. Communications are via a teletypewriter. A dual Linc Magnetic Tape System is used for program storage and data recording. Operational programs utilize the Data General Real-Time Operating System (RTOS) and the ERT AIRMAP Real-Time Operating System (ARTS). The programs provide for acquiring data from the laser nephelometer, acquiring data from auxiliary sources, keeping time, performing real time calculations, recording data and communicating with the teletypewriter.

Paroskie, R. M.↗

NASA NTRS1975

Data management, chapter 5, part C

The data management for a spacecraft radar was defined in terms of an end-to-end data system, which performs the following three functions: (1) sampling and compaction of data onboard the spacecraft, (2) manipulation of radar data on the ground and (3) conversion of radar measurements to geophysical quantities by means of pattern recognition and other machine techniques. Data processing for imaging radar onboard the spacecraft was examined with the conclusion that several techniques can be used to compact the data before storage. It is recommended that compaction techniques be studied further and that existing aircraft radars be modified to provide digital data so that these compaction techniques can be tested.

Source record↗

DOE OSTI2018

Toward Transparent Data Management in Multi-layer Storage Hierarchy for HPC Systems

Upcoming exascale high performance computing (HPC) systems are expected to comprise multi-tier storage hierarchy, and thus will necessitate innovative storage and I/O mechanisms. Traditional disk and block-based interfaces and file systems face severe challenges in utilizing capabilities of storage hierarchies due to the lack of hierarchy support and semantic interfaces. Object-based and semantically-rich data abstractions for scientific data management on large scale systems offer a sustainable solution to these challenges. Such data abstractions can also simplify users involvement in data movement. Here, we take the first steps of realizing such an object abstraction and explore storage mechanisms for these objects to enhance I/O performance, especially for scientific applications. We explore how an object-based interface can facilitate next generation scalable computing systems by presenting the mapping of data I/O from two real world HPC scientific use cases: a plasma physics simulation code (VPIC) and a cosmology simulation code (HACC). Our storage model stores data objects in different physical organizations to support data movement across layers of memory/storage hierarchy. Our implementation sclaes well to 16K parallel processes, and compared to the state of the art, such as MPI-IO and HDF5, our object-based data abstractions and data placement strategy in multi-level storage hierarchy achieves up to 7 X I/O performance improvement for scientific data.

97 MATHEMATICS AND COMPUTING↗

NASA NTRSDate not supplied

Big Data Challenges at CCMC

Like other research centers, the Community Coordinated Modeling Center (CCMC, https://ccmc.gsfc.nasa.gov) at NASA Goddard Space Flight Center (GSFC) is also experiencing the big data challenges. CCMC hosts over 80 space weather models for Runs On Request (ROR), Continuous Runs and Instant Runs simulation services for the research community. In addition, CCMC has started to support simulation output onboarding in response to the Open Science initiative. Overall, we have accumulated over petabytes of simulation output data and are rapidly growing. In this presentation, we will discuss our data and storage challenges. We will present our attempts to address our challenges and any associated lessons learned. CCMC uses Apache Airflow to ensure data transfer is consistent. We will give a brief overview on how we leverage Apache Airflow to enhance our environment.

space weather↗

NASA NTRSDate not supplied

Big Data Challenges at the Community Coordinated Modeling Center (CCMC)

space weather↗

DOE OSTI2016

Prospective CO 2 saline resource estimation methodology: Refinement of existing US-DOE-NETL methods based on data availability

Carbon storage resource estimation in subsurface saline formations plays an important role in establishing the scale of carbon capture and storage activities for governmental policy and commercial project decision-making. Prospective CO 2 resource estimation of large regions or subregions, such as a basin, occurs at the initial screening stages of a project using only limited publicly available geophysical data, i.e. prior to project-specific site selection data generation. As the scale of investigation is narrowed and selected areas and formations are identified, prospective CO 2 resource estimation can be refined and uncertainty narrowed when site-specific geophysical data are available. Here, we refine the United States Department of Energy – National Energy Technology Laboratory (US-DOE-NETL) methodology as the scale of investigation is narrowed from very large regional assessments down to selected areas and formations that may be developed for commercial storage. In addition, we present a new notation that explicitly identifies differences between data availability and data sources used for geologic parameters and efficiency factors as the scale of investigation is narrowed. This CO 2 resource estimation method is available for screening formations in a tool called CO 2 -SCREEN.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2015

AnalyzeThis: An Analysis Workflow-Aware Storage System

The need for novel data analysis is urgent in the face of a data deluge from modern applications. Traditional approaches to data analysis incur significant data movement costs, moving data back and forth between the storage system and the processor. Emerging Active Flash devices enable processing on the flash, where the data already resides. An array of such Active Flash devices allows us to revisit how analysis workflows interact with storage systems. By seamlessly blending together the flash storage and data analysis, we create an analysis workflow-aware storage system, AnalyzeThis. Our guiding principle is that analysis-awareness be deeply ingrained in each and every layer of the storage, elevating data analyses as first-class citizens, and transforming AnalyzeThis into a potent analytics-aware appliance. We implement the AnalyzeThis storage system atop an emulation platform of the Active Flash array. Our results indicate that AnalyzeThis is viable, expediting workflow execution and minimizing data movement.

Data Analytics↗

NASA NTRS1979

Fuel spray data with LDV

Droplet size and two component velocities in the severe environment of an operating gas turbine combustor system can be measured simultaneously using the solar laser morphokinetomer (SLM) which incorporates the following capabilities: (1) measurement of a true two-dimensional velocity vector with a range of + or - (0.01-200 m/sec); (2) measurement of particle size (range 5 to 300 micron m) simultaneously with the measurement of velocity; (3) specification of probe volume position coordinates with a high degree of accuracy (+ or - 0.5 mm); (4) immediate on-line data checks; and (5) rapid computer storage of acquired data. The optical system of the SLM incorporates an ultrasonic beam splitter to allow the measurement of a two-dimensional velocity vector simultaneously with particle size. A microprocessor with a limited storage capability permits immediate analysis of test data in the test cell.

Rohy, D. A.↗

DOE OSTI2016

Greenhouse gas Laser Imaging Tomography Experiment (GreenLITE)

This report describes the development and testing of a novel system, the Greenhouse gas Laser Imaging Tomography Experiment (GreenLITE), for Monitoring, Reporting and Verification (MRV) of CO 2 at Geological Carbon Storage (GCS) sites. The system consists of a pair of laser based transceivers, a number of retroreflectors, and a set of cloud based data processing, storage and dissemination tools, which enable 2-D mapping of the CO 2 in near real time. A system was built, tested locally in New Haven, Indiana, and then deployed to the Zero Emissions Research and Technology (ZERT) facility in Bozeman, MT. Testing at ZERT demonstrated the ability of the GreenLITE system to identify and map small underground leaks, in the presence of other biological sources and with widely varying background concentrations. The system was then ruggedized and tested at the Harris test site in New Haven, IN, during winter time while exposed to temperatures as low as -15 °CºC. Additional testing was conducted using simulated concentration enhancements to validate the 2-D retrieval accuracy. This test resulted in a high confidence in the reconstruction ability to identify sources to tens of meters resolution in this configuration. Finally, the system was deployed for a period of approximately 6 months to an active industrial site, Illinois Basin – Decatur Project (IBDP), where >1M metric tons of CO 2 had been injected into an underground sandstone basin. The main objective of this final deployment was to demonstrate autonomous operation over a wide range of environmental conditions with very little human interaction, and to demonstrate the feasibility of the system for long term deployment in a GCS environment.

20 FOSSIL-FUELED POWER PLANTS↗

DOE OSTI2021

Hydrogen-Based Energy Storage Systems for Large-Scale Data Center Applications

Global demand for data and data access has spurred the rapid growth of the data center industry. To meet demands, data centers must provide uninterrupted service even during the loss of primary power. Service providers seeking ways to eliminate their carbon footprint are increasingly looking to clean and sustainable energy solutions, such as hydrogen technologies, as alternatives to traditional backup generators. In this viewpoint, a survey of the current state of data centers and hydrogen-based technologies is provided along with a discussion of the hydrogen storage and infrastructure requirements needed for large-scale backup power applications at data centers.

08 HYDROGEN↗

NASA NTRS2017

Recent Advancements in the Infrared Flow Visualization System for the NASA Ames Unitary Plan Wind Tunnels

The following details recent efforts undertaken at the NASA Ames Unitary Plan wind tunnels to design and deploy an advanced, production-level infrared (IR) flow visualization data system. Highly sensitive IR cameras, coupled with in-line image processing, have enabled the visualization of wind tunnel model surface flow features as they develop in real-time. Boundary layer transition, shock impingement, junction flow, vortex dynamics, and buffet are routinely observed in both transonic and supersonic flow regimes all without the need of dedicated ramps in test section total temperature. Successful measurements have been performed on wing-body sting mounted test articles, semi-span floor mounted aircraft models, and sting mounted launch vehicle configurations. The unique requirements of imaging in production wind tunnel testing has led to advancements in the deployment of advanced IR cameras in a harsh test environment, robust data acquisition storage and workflow, real-time image processing algorithms, and evaluation of optimal surface treatments. The addition of a multi-camera IR flow visualization data system to the Ames UPWT has demonstrated itself to be a valuable analyses tool in the study of new and old aircraft/launch vehicle aerodynamics and has provided new insight for the evaluation of computational techniques.

Infrared Thermography↗

NASA NTRS2018

Data Assimilation of Terrestrial Water Storage to Adjust Precipitation Fluxes

The Gravity Recovery and Climate Experiment (GRACE) mission has provided unprecedented observations of terrestrial water storage (TWS) dynamics at basin to continental scales. TWS is defined as the sum of groundwater, soil moisture, snow, surface water, ice and biomass water. Data assimilation of GRACE TWS observations has been shown to improve simulation of groundwater, streamflow, and snow water equivalent, and has also proven useful for drought monitoring and identifying human impacts on the water cycle. From a modeling perspective, the TWS components are defined as "prognostic hydrological states". Existing GRACE data assimilation schemes update these prognostic states directly. In this work, we propose an alternate approach in which precipitation fluxes are adjusted in order to achieve the desired change in the hydrological prognostic states. Limitations of such an approach include the assumption that all errors in TWS originate from errors in precipitation. Nonetheless, benefits comprise (1) the water balance is maintained, as opposed to having to add increments to the water budget components, (2) the model automatically determines how to distribute the updates among the TWS prognostic states, and (3) it is not necessary to know the exact time of the observation TWS, because the TWS change timing is determined by the precipitation forcing.

Girotto, Manuela↗

DOE OSTI2018

Storage Systems and Input/Output: Organizing, Storing, and Accessing Data for Scientific Discovery. Report for the DOE ASCR Workshop on Storage Systems and I/O. [Full Workshop Report]

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2019

Storage Systems and Input/Output: Organizing, Storing, and Accessing Data for Scientific Discovery. Report for the DOE ASCR Workshop on Storage Systems and I/O. [Summary Brief]

In September, 2018, the Department of Energy, Office of Science, Advanced Scientific Computing Research Program convened a workshop to identify key challenges and define research directions that will advance the field of storage systems and I/O over the next 5-7 years.The workshop concluded that addressing these combined challenges and opportunities requires tools and techniques that greatly extend traditional approaches and require new research directions. Key research opportunities were identified. This summary brief highlights three areas: The Impact of Storage and I/O, Research Opportunities, and, A new and productive Storage and I/O Environment. The full report from the workshop can be viewed at DOI: 10.2172/1491994.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2015

High Performance Data Analytics on a Commodity Private Cloud

Sandia is researching novel methods of analysis for large datasets. The term “Big Data” has been used frequently in recent years to describe a discipline of computer science that attempts to distil an increasing quantity of data into a result or meaningful subset that can be easily visualized and acted upon by human consumers. These data are frequently characterized by one or more of the following: high volume, velocity, variety, and veracity. Sandia scientists and researchers are no strangers to large datasets, which often typify the result of high-performance computing workloads. The challenge for Sandia data scientists, however, comes at the arrival of exascale computing, which has become mandated by executive order for national computing centers. Because of power and networking limitations, exascale computing precludes the movement of datasets at exascale from data production to data analysis. Such analysis must be done in-situ, to mitigate the time and power costs of network transport. Currently, many of Sandia’s data analytics environments exist as independent entities, often specialized for individual use cases, interconnected to other data analytics, high-performance computing, and storage systems. Sandia is procuring, installing, and researching future platforms for such data analytics. A memory footprint of 256 GB per node, fourteen data rate (FDR) InfiniBand networking, PCIe connected non-volatile memory (NVMe) data storage, and an object data store will characterize the coming institutional data analytics platforms. The combined systems will contain 300 compute nodes. A cloud-inspired software stack resulting from collaboration between the Sandia centers will allow the systems to support a flexible mix of user and system-facing workloads and use cases including code development and testing in R and Python, Apache Hadoop and Spark, distributed in-memory databases, graph analytics, emulytics, system log and predictive analytics, and data visualization for institutional and mission customers. These systems will arrive at the end of FY 2015 and production is planned by the spring of FY 2016.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2018

System, method, and computer program product for warming a cache for a task launch

A system, method, and computer program product for warming a cache for a task launch is described. The method includes the steps of receiving a task data structure that defines a processing task, extracting information stored in a cache warming field of the task data structure, and, prior to executing the processing task, generating a cache warming instruction that is configured to load one or more entries of a cache storage with data fetched from a memory.

Ricketts, Scott↗

NASA NTRS2007

Processing Raw HST Data With Up-to-Date Calibration Data

On-the-Fly Reprocessing (OTFR) is a collection of data-processing routines that work within the context of the Hubble Space Telescope (HST) pipeline data-flow system. The purpose served by OTFR is to generate, on demand, scientifically useful data products from raw HST data stored in an archive. First, on the basis of the requested final data products, OTFR retrieves the corresponding sets of raw data from the archives. Next, OTFR processes the raw data sets to remove artifacts and to establish proper header and other template information. Finally, the calibration routines appropriate to the specific data sets are invoked to produce the requested data products, and the data products are released to an archive distribution system for transmission to the requesting party. OTFR offers two notable advantages: (1) Inasmuch as calibrated data occupy about 8 times as much storage space as do raw data, by obviating storage of calibrated data, OTFR reduces the storage capacity needed by the archive; and (2) the calibration routines can be updated to give requesters the benefit of the most recent calibrations.

Miller, Warren↗