Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “DATA STORAGE”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 361 records · Page 20
All sourcesNASA NTRSDOE OSTI
NASA NTRS2002

Adapting Digital Libraries to Continual Evolution

In this paper, we describe five investment streams (data storage infrastructure, knowledge management, data production control, data transport and security, and personnel skill mix) that need to be balanced against short-term operating demands in order to maximize the probability of long-term viability of a digital library. Because of the rapid pace of information technology change, a digital library cannot be a static institution. Rather, it has to become a flexible organization adapted to continuous evolution of its infrastructure.

Barkstrom, Bruce R.↗
NASA NTRS2019

New Era, New Opportunity, Is GES DISC Ready for Big Data Challenge?

The new era of Big Data has opened doors for many new opportunities, as well as new challenges, for both Earth science research/application and data communities. As one of the twelve NASA data centers - Goddard Earth Sciences Data and Information Services Center (GES DISC), one of our great challenges has been how to help research/application community efficiently (quickly and properly) accessing, visualizing and analyzing the massive and diverse data in natural hazard research, management, or even prediction. GES DISC has archived over 2000 TB data on premises and distributed over 23,000 TB of data since 2010. Our data has been widely used in every phase of natural hazard management and research, i.e. long term risk assessment and reduction, forecasting and predicting, monitoring and detection, early warning, damage assessment and response. The big data challenge is not just about data storage, but also about data discoverability and accessibility, and even more, about data migration/mirroring in the cloud. This paper is going to demonstrate GES DISC’s efforts and approaches of evolving our overall Web services and powerful Giovanni (Geospatial Interactive Online Visualization ANd aNalysis Infrastructure) tool into further improving data discoverability and accessibility. Prototype works will also be presented.

Li, A.↗
NASA NTRSDate not supplied

Open Source Application of Fusing Aerosol Products from GEO and LEO Satellites

Retrieving aerosol optical depths (AODs) from sun-synchronous polar orbiting (aka low earth orbit, LEO) satellites, such as MODISs, and VIIRSs, OMI, TROPOMI, etc, has become well-established as a tool for extracting information on particulate matter (PM) and related processes in the atmosphere. However, with recently launched geostationary satellites (GEO), such as GOES-16/17/18, and Himawari-8/9, and Meteosat Third Generation (MTG) they provide a much higher temporal resolution (order of 10 minutes), typically an image once or more per hour during daylight compared to LEO once per day. By combining these observations, we may be able to characterize the diurnal cycle of global AOD at the local, regional and global scale. While the science community is still exploring the new data from GEO observations, we have been thinking about how to properly combine/merge/fuse those data considering differences in their spatial and temporal resolutions. However, this poses a “Big Data” challenge. The big data challenge is not just about data storage, but also about data discoverability, and accessibility, and even more, about data migration/mirroring in the cloud-computing environment. This paper is merely showing some of the efforts and approaches we have attempted in fusing six satellites’ Level 2 aerosol data (three are from GEO (GOES-16/17 and Himawari-8), and the other three are from LEO (TERRA/MODIS, AQUA/MODIS, SNPP-VIIRS) from Dark Target (DT) aerosol retrieval algorithm. Having the on-demand capability of fusing remote sensing products onto the desired temporal and spatial domain enables researchers and application practitioners to better manipulate and work with satellite and sensor data. It is our hopeWe hope that by making such an open-source package, and the accompanying functionality, the scientific community will be granted easier access to aerosol data processing resources. The MEaSUREs Program (Making Earth System Data Records for Use in Research Environments) expands our understanding of the Earth's current system through atmospheric and surface measurements. In an effort to aid the scientific research component and improve open source methods, this project developed Python code for fusing six satellite Level 2 aerosol data (three are from geostationary satellites (GEO), and the other three are from low earth orbital satellites (LEO)) from Dark Target Aerosol Retrieval Algorithm.

Jennifer Wei↗
NASA NTRSDate not supplied

Breaking Barriers: Integrating Geo-Leo Aerosol Data with an Open-Source Approach

The scientific community is still examining the novel data from geostationary satellite observations and evaluating methods for effectively fusing the polar observations with various spatial and temporal resolutions. However, the merged data will present a significant ""Big Data"" challenge, including processing, storage, data discoverability, accessibility, and migration within cloud computing environments. We have developed an open-source package to fuse aerosol optical depths (AOD) products from six satellite sensors in the past four years (2019~2023), and this presentation will update our recent progress. Using this Python-based package, we produced a level 3 global (AOD) product in a quarter-degree spatial resolution every half-hour, fusing the Level 2 AOD data with the Dark Target aerosol retrieval algorithm from six satellites: three geostationary (GOES-16/17 and Himawari-8) with high temporal resolution, and three polar orbiting (TERRA/MODIS, AQUA/MODIS, and SNPP-VIIRS) with global coverage. By integrating these observations, the diurnal cycle of global AOD in this fused product can be characterized at local, regional, and global scales. Furthermore, we are committed to openness and transparency by providing our package and its associated functionalities as open-source. Our dedication to adhering to the FAIR, CARE, and TRUST principles ensures that our users can rely on the integrity and ethical standards of our work. For instance of Interoperability, this package fuses remote sensing products on demand into desired temporal and spatial domains. It can be run in a central processing unit (CPU) or a Graphics processing unit (GPU) mode. This package will empower researchers and practitioners to use satellite and sensor data efficiently in various applications and research.

Xiaohua Pan↗
DOE OSTI2017

Systems and methods for rapid processing and storage of data

Systems and methods of building massively parallel computing systems using low power computing complexes in accordance with embodiments of the invention are disclosed. A massively parallel computing system in accordance with one embodiment of the invention includes at least one Solid State Blade configured to communicate via a high performance network fabric. In addition, each Solid State Blade includes a processor configured to communicate with a plurality of low power computing complexes interconnected by a router, and each low power computing complex includes at least one general processing core, an accelerator, an I/O interface, and cache memory and is configured to communicate with non-volatile solid state memory.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2019

Systems and methods for rapid processing and storage of data

Systems and methods of building massively parallel computing systems using low power computing complexes in accordance with embodiments of the invention are disclosed. A massively parallel computing system in accordance with one embodiment of the invention includes at least one Solid State Blade configured to communicate via a high performance network fabric. In addition, each Solid State Blade includes a processor configured to communicate with a plurality of low power computing complexes interconnected by a router, and each low power computing complex includes at least one general processing core, an accelerator, an I/O interface, and cache memory and is configured to communicate with non-volatile solid state memory.

Stalzer, Mark A.↗
DOE OSTI2021

Systems and methods for rapid processing and storage of data

Systems and methods of building massively parallel computing systems using low power computing complexes in accordance with embodiments of the invention are disclosed. A massively parallel computing system in accordance with one embodiment of the invention includes at least one Solid State Blade configured to communicate via a high performance network fabric. In addition, each Solid State Blade includes a processor configured to communicate with a plurality of low power computing complexes interconnected by a router, and each low power computing complex includes at least one general processing core, an accelerator, an I/O interface, and cache memory and is configured to communicate with non-volatile solid state memory.

Stalzer, Mark A.↗
DOE OSTI2025

Dani Sleight Intern Poster

The NRDS Portal is a login-based data storage solution and science data gateway for researchers to centralize and analyze data before publication. Users can upload, edit, review, and approve their own datasets within the site to eventually be published for public use on the main NRDS site. More development was needed to extend NRDS Portal with new Artificial Intelligence features.

99 - GENERAL AND MISCELLANEOUS↗
NASA NTRS1991

Balloon borne optical disk mass storage system

An on-board data recording system for balloon-borne interferometer using a vacuum operable, ruggedized WORM optical drive is presented. This system, as presently under development, provides 320 Mbytes of data storage (or approximately 11 hrs at the 64 kbits/sec telemetry rate of the experiment). It has the capability of recording the unmodified telemetry bit system as transmitted or doing some preprocessing of the data onboard. The system is compact and requires less than 28 watts of battery power to operate.

Vanek, M. D.↗
DOE OSTI2021

SIRIUS: Science-Driven Data Management for Multi-Tiered Storage

The data sets being generated by large applications on very large-scale systems are increasing in both size and complexity. At the same time, there are new ways available to store and access these data sets. The goal in this project is to develop software that applications can use to make use of new and existing storage technologies in more sophisticated ways. One challenge in scientific data management is handling ‘hot’ vs ‘cold’ data. Data that is hot is data that is needed (or will be needed soon) in order for the program to continue progressing, while cold data is either output (and so will not be need further during the life of the program) or will not be needed until significantly later in the program’s run. Hot data should be stored in a way that allows fast access. On most systems, economic factors lead to an inverse relationship between storage performance and storage capacity and so fast access storage is limited. This makes it important to correctly place hot and cold data and avoid cold data unnecessarily consuming precious resources. In this reporting period, we addressed this challenge in various ways and at various levels. Data management frameworks offer only limited control to applications in how data is stored. We have added software capabilities for seamlessly moving data between layers of the storage technology using promote and demote functions to existing software frameworks. This gives direct control to applications in deciding what priority data receives. Additionally, we integrated different storage layer management frameworks in order to allow data to be exchanged and moved between storage layers in a consistent way across the application. Further, applications are not always able to directly decide what storage level makes sense for a given piece of data without an understanding of the underlying storage technologies. Data storage frameworks are often positioned to make these sorts of decisions in service of the application. We have added machine-learning based capabilities to data staging frameworks in order to make intelligent decisions about where data should be stored given learning about patterns in previous usage of similar data.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1976

Photorefractive materials for optical storage and display

Real-time data storage and processing using optical techniques have been considered in recent years. Of particular interest are photosensitive electro-optic crystals which permit volume storage in the form of phase holograms, by means of a charge transfer process. A survey of the state of the art of such holographic memories is presented. The physical mechanism responsible for the formation of phase holograms in such crystals is discussed. Attention is focused on various aspects of materials characterization, development and utilization. Experimental reversible holographic read-write memory systems with fast random access and high storage capacity employing this new class of photosensitive materials have already been demonstrated.

Shah, R. R.↗
NASA NTRS1988

Ocean Data Acquisition System

The Ocean Data Acquisition System (ODAS) is a low cost instrument with potential commercial application. It is easily mounted on a small aircraft and flown over the coastal zone ocean to remotely measure sea surface temperature and three channels of ocean color information. From this data, chlorophyll levels can be derived for use by ocean scientists, fisheries, and environmental offices. Data can be transmitted to shipboard for real-time use with sea truth measurements, ocean productivity estimates and fishing fleet direction. The aircraft portion of the system has two primary instruments: an IR radiometer to measure sea surface temperature and a three channel visible spectro-radiometer for 460, 490, and 520 nm wavelength measurements from which chlorophyll concentration can be derived. The aircraft package contains a LORAN-C unit for aircraft location information, clock, on-board data processor and formatter, digital data storage, packet radio terminal controller, and radio transceiver for data transmission to a ship. The shipboard package contains a transceiver, packet terminal controller, data processing and storage capability, and printer. Both raw data and chlorophyll concentrations are available for real-time analysis.

Johnson, B.↗
DOE OSTI2020

Livewire Data Platform - A Solution for Energy Efficient Mobility Systems (EEMS) Data Sharing

The Livewire Data Platform is a data storage and exchange platform. It provides: a platform, allowing data sharing and discovery by linking disparate data sources to a single website via application programming interfaces (APIs); a community, building partnerships and collaboration rather than competition; a system, allowing shared data to increase in size and complexity as EEMS evolves.

47 OTHER INSTRUMENTATION↗
DOE OSTI2025

VA EDH Advanced Software Pipeline Framework Report: Enhancing Automation and Scalability

The VA Environmental Determinants of Health (EDH) Advanced Software Pipeline Framework is designed to enhance the efficiency, scalability, and security of geospatial data processing workflows. This framework integrates modern data orchestration and containerization technologies, including Prefect for workflow automation, Docker for containerization, and PostgreSQL/PostGIS for geospatial data storage and analysis. It ensures standardized, reproducible, and automated data processing, supporting VA objectives related to substance use risk assessment and recovery research. The pipeline addresses key scalability and performance challenges through horizontal and vertical scaling, high-performance computing (HPC) integration, parallel processing, task caching, and dynamic resource allocation. These optimizations improve throughput and reduce latency, allowing the system to efficiently manage large and complex datasets. Additionally, security and compliance measures—such as data encryption (SSL), Role-Based Access Control (RBAC), and adherence to GDPR and HIPAA standards—safeguard sensitive information throughout data transmission and storage. A key implementation of this framework includes the automation of shelter list geolocation workflows, ensuring that up-to-date data is readily available for VA decision-making. Lessons learned from this project include the transition from in-memory processing to incremental storage writes, improving resource management and reliability. Future enhancements aim to expand automation, integrate AI-driven anomaly detection, and incorporate high-performance computing resources. This framework provides a scalable, secure, and adaptable solution for managing geospatial datasets, reinforcing the VA’s ability to support clinical and strategic initiatives through data-driven decision-making.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Carbon Storage Technical Viability Approach and National Data Assessment

Carbon Storage Technical Viability Approach is a comprehensive evaluation of potential geologic carbon storage sites that includes classic subsurface resource assessment components such as reservoir suitability in addition to hazard, environmental, social, regulatory, and jurisdiction components that may enhance or hinder successful carbon storage. Data for assessments of technically viable carbon storage span many categories, types; are disparate and numerous. Variables and types of data required to support CS assessments were previously not clearly defined and documented. Contextualizing available data indicates their utility, potential uncertainty, and gaps. This presentation details the Carbon Storage Technical Viability Approach, the associated matrix and database, and initial results from a portion of the national data availability assessment that contextualizes the data available to support carbon storage projects. Presented at the American Association of Petroleum Geologists' Carbon Capture Utilization and Storage (SPE-AAPG-SEG CCUS 2024) conference in Houston, TX, March 11-13, 2024.

Mark-Moser, Mackenzie K.↗
DOE OSTI2025

Securing 3D NAND Without Density Loss via In-Situ Encryption Using a Single Transistor XOR Cell

In this article, we push lightweight XOR-based in-situ encryption to extreme density by proposing a singletransistor XOR memory cell and applying it to 3D NAND, enabling secure data storage without density loss. Using a ferroelectric field-effect transistor (FeFET) as an example technology, we demonstrate that: i) a single-transistor memory can realize the XOR function by exploiting the ability to charge the source and drain separately and control current flow direction, eliminating the need for conventional encrypted cells that rely on complementary devices; ii) with a XOR-based cipher, encryption and decryption can be mapped to in-situ array operations, where ciphertext is stored as the threshold voltage (VTH) states of FeFETs in a NAND string, and decryption is achieved through read operations using key-dependent complementary source/drain bias; iii) the proposed technique is scalable to multi-level cell (MLC) storage by encrypting and decrypting data bit by bit; iv) using an integrated NAND FeFET array, we experimentally demonstrate encryption and decryption operations for both single-level cell (SLC) and MLC storage; v) systemlevel benchmarking shows that the proposed technique achieves 48× and 278× improvements in encryption and decryption throughput, respectively, compared to AES.

36 MATERIALS SCIENCE↗