Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data storage”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,243 records · Page 69
All sourcesNASA NTRSDOE OSTI
DOE OSTI2012

System and method for identifying, reporting, and evaluating presence of substance

A system and method for identifying, reporting, and evaluating a presence of a solid, liquid, gas, or other substance of interest, particularly a dangerous, hazardous, or otherwise threatening chemical, biological, or radioactive substance. The system comprises one or more substantially automated, location self-aware remote sensing units; a control unit; and one or more data processing and storage servers. Data is collected by the remote sensing units and transmitted to the control unit; the control unit generates and uploads a report incorporating the data to the servers; and thereafter the report is available for review by a hierarchy of responsive and evaluative authorities via a wide area network. The evaluative authorities include a group of relevant experts who may be widely or even globally distributed.

47 OTHER INSTRUMENTATION↗
DOE OSTI2016

Supporting Building Portfolio Investment and Policy Decision Making through an Integrated Building Utility Data Platform

The American Recovery and Reinvestment Act stimulus funding of 2009 for smart grid projects resulted in the tripling of smart meters deployment. In 2012, the Green Button initiative provided utility customers with access to their real-time1 energy usage. The availability of finely granular data provides an enormous potential for energy data analytics and energy benchmarking. The sheer volume of time-series utility data from a large number of buildings also poses challenges in data collection, quality control, and database management for rigorous and meaningful analyses. In this paper, we will describe a building portfolio-level data analytics tool for operational optimization, business investment and policy assessment using 15-minute to monthly intervals utility data. The analytics tool is developed on top of the U.S. Department of Energy’s Standard Energy Efficiency Data (SEED) platform, an open source software application that manages energy performance data of large groups of buildings. To support the significantly large volume of granular interval data, we integrated a parallel time-series database to the existing relational database. The time-series database improves on the current utility data input, focusing on real-time data collection, storage, analytics and data quality control. The fully integrated data platform supports APIs for utility apps development by third party software developers. These apps will provide actionable intelligence for building owners and facilities managers. Unlike a commercial system, this platform is an open source platform funded by the U.S. Government, accessible to the public, researchers and other developers, to support initiatives in reducing building energy consumption.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2015

BASTet (Berkeley Analysis and Storage Toolkit)

BASTet is an advanced software library written in Python. BASTet serves as the analysis and storage library for the OpenMSI project. BASTet is an integrate framework for: i) storage of spectral imaging data, ii) storage of derived analysis data, iii) provenance of analyses, iv) integration and execution of analyses via complex workflows. BASTet implements the API for the HDF5 storage format used by OpenMSI. Analyses that are developed using BASTet benefit from direct integration with storage format, automatic tracking of provenance, and direct integration with command-line and workflow execution tools. BASTet also defines interfaces to enable developers to directly integrate their analysis with OpenMSI's web-based viewing infrastruture without having to know OpenMSI. BASTet also provides numerous helper classes and tools to assist with the conversion of data files, ease parallel implementation of analysis algorithms, ease interaction with web-based functions, description methods for data reduction. BASTet also includes detailed developer documentation, user tutorials, iPython notebooks, and other supporting documents.

Bowen, Benjamin↗
NASA NTRS2013

Towards Efficient Scientific Data Management Using Cloud Storage

A software prototype allows users to backup and restore data to/from both public and private cloud storage such as Amazon's S3 and NASA's Nebula. Unlike other off-the-shelf tools, this software ensures user data security in the cloud (through encryption), and minimizes users operating costs by using space- and bandwidth-efficient compression and incremental backup. Parallel data processing utilities have also been developed by using massively scalable cloud computing in conjunction with cloud storage. One of the innovations in this software is using modified open source components to work with a private cloud like NASA Nebula. Another innovation is porting the complex backup to- cloud software to embedded Linux, running on the home networking devices, in order to benefit more users.

He, Qiming↗
DOE OSTI2018

Broom: application for non-redundant storage of high throughput sequencing data

Abstract Motivation The data generation capabilities of high throughput sequencing (HTS) instruments have exponentially increased over the last few years, while the cost of sequencing has dramatically decreased allowing this technology to become widely used in biomedical studies. For small labs and individual researchers, however, storage and transfer of large amounts of HTS data present a significant challenge. The recent trends in increased sequencing quality and genome coverage can be used to reconsider HTS data storage strategies. Results We present Broom, a stand-alone application designed to select and store only high-quality sequencing reads at extremely high compression rates. Written in C++, the application accepts single and paired-end reads in FASTQ and FASTA formats and decompresses data in FASTA format. Availability and implementation C++ code available at https://scsb.utmb.edu/labgroups/fofanov/broom.asp. Supplementary information Supplementary data are available at Bioinformatics online.

Albayrak, Levent↗
DOE OSTI2026

Data for The Value of Reversible Carbon Storage in a Zero-Emissions World

Atmospheric carbon dioxide removal (CDR) is required to stabilize global temperature. CDR can be achieved via ecosystem-based approaches that are cost-effective but reversible (e.g., soil and forest management) or by more durable but expensive approaches (e.g., direct air capture coupled with geologic storage). Here, we examine trade-offs between these approaches, focusing on timing, climate impacts, and cost. We simulated reversible carbon accrual for a range of CDR contract structures using a general minimalist model of ecosystem carbon cycling, and parameterized it to simulate US agricultural soil management─specifically cover cropping─as a case study. We then quantified the resulting impact on atmospheric carbon and global temperature using a climate model emulator. We find that maintaining a patchwork of reversible CDR projects by replacing lapsed projects with new projects can reduce warming by 22–195 μ°C in 2100 and that the magnitude of this cooling effect depends on how effectively the patchwork is maintained. Long-term maintenance of reversible CDR projects requires institutional stability that cannot be guaranteed over multiple decades. Consequently, effective CDR ultimately requires replacing reversible projects with durable projects. To address this problem, we modeled the cost of replacing reversible agricultural soil CDR with geologic CDR. We found that using reversible CDR as a bridge to durable CDR is potentially more cost-effective as a global cooling strategy (0.20–0.81 billion USD per μ°C avoided) than perpetual maintenance of reversible CDR (0.32–1.31 billion USD per μ°C avoided) or an immediate transition to durable CDR (1.37–2.19 billion USD per μ°C avoided). However, we emphasize that institutional commitments to maintain reversible CDR projects cannot be guaranteed. Reliance on reversible CDR as a bridge to durable CDR therefore carries an unknown amount of risk and will only function if efforts to maintain reversible CDR are robust.

Carbon↗
DOE OSTI2014

An Advanced Network and distributed Storage Laboratory (ANDSL) for Data Intensive Science (Final Report)

The original intent of this project was to build and operate an Advanced Network and Distributed Storage Laboratory (ANDSL) for Data Intensive Science that will prepare the Open Science Grid (OSG) community for a new generation of wide area communication capabilities operating at a 100Gb rate. The plan was to engage a team of four dedicated scientists that will design, develop and operate a state of the art laboratory that will be fully integrated into the organizational and operational infrastructure of the OSG. The ANDSL was planned to include host-based hardware and software required to profile, emulate and study real-life distributed storage workloads at rates of 100 Gbps. The laboratory was designed to utilize the Department Of Energy’s Advanced Network Initiative (ANI) 100GbTestbed as connections to the fabric become available so that OSG stakeholders can use it to profile real-life data intensive science applications as well as experiment with advanced storage and data management technologies. The goal of the original program of work was to close the adoption gap, decreasing the time between availability of production 100Gbps links and the end-to-end use of such a network to enable scientific discovery.

100Gb networks↗
DOE OSTI2015

Cloud object store for archive storage of high performance computing data using decoupling middleware

Cloud object storage is enabled for archived data, such as checkpoints and results, of high performance computing applications using a middleware process. A plurality of archived files, such as checkpoint files and results, generated by a plurality of processes in a parallel computing system are stored by obtaining the plurality of archived files from the parallel computing system; converting the plurality of archived files to objects using a log structured file system middleware process; and providing the objects for storage in a cloud object storage system. The plurality of processes may run, for example, on a plurality of compute nodes. The log structured file system middleware process may be embodied, for example, as a Parallel Log-Structured File System (PLFS). The log structured file system middleware process optionally executes on a burst buffer node.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1991

Data compression: The end-to-end information systems perspective for NASA space science missions

The unique characteristics of compressed data have important implications to the design of space science data systems, science applications, and data compression techniques. The sequential nature or data dependence between each of the sample values within a block of compressed data introduces an error multiplication or propagation factor which compounds the effects of communication errors. The data communication characteristics of the onboard data acquisition, storage, and telecommunication channels may influence the size of the compressed blocks and the frequency of included re-initialization points. The organization of the compressed data are continually changing depending on the entropy of the input data. This also results in a variable output rate from the instrument which may require buffering to interface with the spacecraft data system. On the ground, there exist key tradeoff issues associated with the distribution and management of the science data products when data compression techniques are applied in order to alleviate the constraints imposed by ground communication bandwidth and data storage capacity.

Tai, Wallace↗