SEARCH · Engineering Papers

Results for “DATA STORAGE”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 271 records · Page 15

All sources NASA NTRS DOE OSTI

NASA NTRS2013

Usage of Data-Encoded Web Maps with Client Side Color Rendering for Combined Data Access, Visualization and Modeling Purposes

Current approaches to satellite observation data storage and distribution implement separate visualization and data access methodologies which often leads to the need in time consuming data ordering and coding for applications requiring both visual representation as well as data handling and modeling capabilities. We describe an approach we implemented for a data-encoded web map service based on storing numerical data within server map tiles and subsequent client side data manipulation and map color rendering. The approach relies on storing data using the lossless compression Portable Network Graphics (PNG) image data format which is natively supported by web-browsers allowing on-the-fly browser rendering and modification of the map tiles. The method is easy to implement using existing software libraries and has the advantage of easy client side map color modifications, as well as spatial subsetting with physical parameter range filtering. This method is demonstrated for the ASTER-GDEM elevation model and selected MODIS data products and represents an alternative to the currently used storage and data access methods. One additional benefit includes providing multiple levels of averaging due to the need in generating map tiles at varying resolutions for various map magnification levels. We suggest that such merged data and mapping approach may be a viable alternative to existing static storage and data access methods for a wide array of combined simulation, data access and visualization purposes.

Pliutau, Denis↗

NASA NTRS1977

SPAR: Structural-performance analysis and redesign

System of processor programs performs stress, buckling, and vibrational analysis of large linear finite element systems in excess of 50,000 degrees of freedom, while minimizing processing cost, execution time, central memory storage, and secondary data storage requirements. Programs use sparse matrix solution techniques and other computational and data management procedures.

Whetstone, W. D.↗

DOE OSTI2020

Striped Data Analysis Framework

A columnar data representation is known to be an efficient way for data storage, specifically in cases when the analysis is often done based only on a small fragment of the available data structures. A data representation like Apache Parquet is a step forward from a columnar representation, which splits data horizontally to allow for easy parallelization of data analysis. Based on the general idea of columnar data storage, working on the [LDRD Project], we have developed a striped data representation, which, we believe, is better suited to the needs of High Energy Physics data analysis. A traditional columnar approach allows for efficient data analysis of complex structures. While keeping all the benefits of columnar data representations, the striped mechanism goes further by enabling easy parallelization of computations without requiring special hardware. We will present an implementation and some performance characteristics of such a data representation mechanism using a distributed no-SQL database or a local file system, unified under the same API and data representation model. The representation is efficient and at the same time simple so that it allows for a common data model and APIs for wide range of underlying storage mechanisms such as distributed no-SQL databases and local file systems. Striped storage adopts Numpy arrays as its basic data representation format, which makes it easy and efficient to use in Python applications. The Striped Data Server is a web service, which allows to hide the server implementation details from the end user, easily exposes data to WAN users, and allows to utilize well known and developed data caching solutions to further increase data access efficiency. We are considering the Striped Data Server as the core of an enterprise scale data analysis platform for High Energy Physics and similar areas of data processing. We have been testing this architecture with a 2TB dataset from a CMS dark matter search and plan to expand it to multiple 100 TB or even PB scale. We will present the striped format, Striped Data Server architecture and performance test results.

Gutsche, Oliver↗

DOE OSTI2018

Striped Data Server for Scalable Parallel Data Analysis

A columnar data representation is known to be an efficient way for data storage, specifically in cases when the analysis is often done based only on a small fragment of the available data structures. A data representation like Apache Parquet is a step forward from a columnar representation, which splits data horizontally to allow for easy parallelization of data analysis. Based on the general idea of columnar data storage, working on the [LDRD Project], we have developed a striped data representation, which, we believe, is better suited to the needs of High Energy Physics data analysis. A traditional columnar approach allows for efficient data analysis of complex structures. While keeping all the benefits of columnar data representations, the striped mechanism goes further by enabling easy parallelization of computations without requiring special hardware. We will present an implementation and some performance characteristics of such a data representation mechanism using a distributed no-SQL database or a local file system, unified under the same API and data representation model. The representation is efficient and at the same time simple so that it allows for a common data model and APIs for wide range of underlying storage mechanisms such as distributed no-SQL databases and local file systems. Striped storage adopts Numpy arrays as its basic data representation format, which makes it easy and efficient to use in Python applications. The Striped Data Server is a web service, which allows to hide the server implementation details from the end user, easily exposes data to WAN users, and allows to utilize well known and developed data caching solutions to further increase data access efficiency. We are considering the Striped Data Server as the core of an enterprise scale data analysis platform for High Energy Physics and similar areas of data processing. As a result, we have been testing this architecture with a 2TB dataset from a CMS dark matter search and plan to expand it to multiple 100 TB or even PB scale. We will present the striped format, Striped Data Server architecture and performance test results.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

NASA NTRS1993

The implementation of a lossless data compression module in an advanced orbiting system: Analysis and development

Data compression has been proposed for several flight missions as a means of either reducing on board mass data storage, increasing science data return through a bandwidth constrained channel, reducing TDRSS access time, or easing ground archival mass storage requirement. Several issues arise with the implementation of this technology. These include the requirement of a clean channel, onboard smoothing buffer, onboard processing hardware and on the algorithm itself, the adaptability to scene changes and maybe even versatility to the various mission types. This paper gives an overview of an ongoing effort being performed at Goddard Space Flight Center for implementing a lossless data compression scheme for space flight. We will provide analysis results on several data systems issues, the performance of the selected lossless compression scheme, the status of the hardware processor and current development plan.

Yeh, Pen-Shu↗

NASA NTRS1985

Bubble memory module for spacecraft application

Bubble domain technology offers an all-solid-state alternative for data storage in onboard data systems. A versatile modular bubble memory concept was developed. The key module is the bubble memory module which contains all of the storage devices and circuitry for accessing these devices. This report documents the bubble memory module design and preliminary hardware designs aimed at memory module functional demonstration with available commercial bubble devices. The system architecture provides simultaneous operation of bubble devices to attain high data rates. Banks of bubble devices are accessed by a given bubble controller to minimize controller parts. A power strobing technique is discussed which could minimize the average system power dissipation. A fast initialization method using EEPROM (electrically erasable, programmable read-only memory) devices promotes fast access. Noise and crosstalk problems and implementations to minimize these are discussed. Flight memory systems which incorporate the concepts and techniques of this work could now be developed for applications.

Hayes, P. J.↗

NASA NTRS2000

The Mission Transcript Collection: U.S. Human Spaceflight Missions from Mercury Redstone 3 to Apollo 17

Aboard every U.S. piloted spacecraft, from Mercury through Apollo, NASA installed tape recorders that captured nearly every word spoken by the astronauts during their history-making flights into space. For the first time ever, NASA has digitally scanned all of the transcripts made from both the onboard tapes and those tape recordings made on the ground from the air-to-ground transmissions and placed them on this two CD-ROM set. Gathered in this special collection are 80 transcripts totaling nearly 45,000 pages of text that cover every US human spaceflight from the first human Mercury mission through the last lunar landing flight of Apollo 17. Users of this CD will note that the quantity and type of transcripts made for each mission vary. For example, the Mercury flights each had one transcript whereas the Gemini missions produced several. Starting with the Gemini flights, NASA produced a Public Affairs Office (PAO) commentary version, as well as at least one "technical" air-to-ground transcript version, per mission. Most of the Apollo missions produced four transcripts per flight. These included the onboard voice data recorder transcripts made from the Data Storage Equipment (DSE) on the Command Module (CM), and the Data Storage Electronics Assembly (DSEA) onboard the Lunar Module (LM), in addition to the PAO commentary and air-to-ground technical transcripts. The CD set includes an index listing each transcript file by name. Some of the transcripts include a detailed explanation of their contents and how they were made. Also included in this collection is a listing of all the original air-to-ground audiotapes housed in NASA's archives from which many of these transcripts were made. We hope you find this collection of transcripts interesting and useful.

Source record↗

NASA NTRS1996

Storage and network bandwidth requirements through the year 2000 for the NASA Center for Computational Sciences

The data storage and retrieval demands of space and Earth sciences researchers have made the NASA Center for Computational Sciences (NCCS) Mass Data Storage and Delivery System (MDSDS) one of the world's most active Convex UniTree systems. Science researchers formed the NCCS's Computer Environments and Research Requirements Committee (CERRC) to relate their projected supercomputing and mass storage requirements through the year 2000. Using the CERRC guidelines and observations of current usage, some detailed projections of requirements for MDSDS network bandwidth and mass storage capacity and performance are presented.

Salmon, Ellen↗

NASA NTRS1992

Transfer function characteristics of super resolving systems

Signal quality in an optical storage device greatly depends on the optical system transfer function used to write and read data patterns. The problem is similar to analysis of scanning optical microscopes. Hopkins and Braat have analyzed write-once-read-many (WORM) optical data storage devices. Herein, transfer function analysis of magnetooptic (MO) data storage devices is discussed with respect to improving transfer-function characteristics. Several authors have described improving the transfer function as super resolution. However, none have thoroughly analyzed the MO optical system and effects of the medium. Both the optical system transfer function and effects of the medium of this development are discussed.

Milster, Tom D.↗

NASA NTRS2004

Performance of multi level error correction in binary holographic memory

At the Optical Computing Lab in the Jet Propulsion Laboratory (JPL) a binary holographic data storage system was designed and tested with methods of recording and retrieving the binary information. Levels of error correction were introduced to the system including pixel averaging, thresholding, and parity checks. Errors were artificially introduced into the binary holographic data storage system and were monitored as a function of the defect area fraction, which showed a strong influence on data integrity. Average area fractions exceeding one quarter of the bit area caused unrecoverable errors. Efficient use of the available data density was discussed. .

data storage↗

NASA NTRS1993

The modern high rate digital cassette recorder

The magnetic tape recorder has played an essential role in the capture and storage of instrumentation data for more than thirty years. During this time, data recording technology has steadily progressed to meet user demands for more channels, wider bandwidths, and longer recording durations. When acquisition and processing moved from analog to digital techniques, so recorder design followed suit. Milestones marking the evolution of the data recorder through these various stages - multi-track analog, high density longitudinal digital, and more recently rotary digital - have often represented important breakthroughs in the handling of ever-greater quantities of data. Throughout this period there has been a very clear line of demarcation between data storage methods in the 'instrumentation world' on the one hand and the 'computer peripheral world' on the other. This is despite the fact that instrumentation data, whether analog or digital at the point of acquisition, is now likely to be processed on a digital computer at some stage. Regardless of whether the processing device is a small personal computer, a workstation, or the largest supercomputer, system integrators have traditionally been faced with the same basic problem - how to interface what is essentially a manually controlled, continuously running device (the tape recorder) into the fast start/stop computer environment without resorting to an excessive amount of complex custom interfacing and performance compromise. The increasing availability of affordable high power processing equipment throughout the scientific world is forcing recorder manufacturers to make their latest and perhaps most important breakthrough - the computer-friendly data recorder. The operating characteristics of such recorders are discussed and the resultant impact on both data acquisition and data analysis elements of system configuration are considered.

Clemow, Martin↗

DOE OSTI2016

Mechanism of supporting sub-communicator collectives with o(64) counters as opposed to one counter for each sub-communicator

A system and method for enhancing barrier collective synchronization on a computer system comprises a computer system including a data storage device. The computer system includes a program stored in the data storage device and steps of the program being executed by a processor. The system includes providing a plurality of communicators for storing state information for a barrier algorithm. Each communicator designates a master core in a multi-processor environment of the computer system. The system allocates or designates one counter for each of a plurality of threads. The system configures a table with a number of entries equal to the maximum number of threads. The system sets a table entry with an ID associated with a communicator when a process thread initiates a collective. The system determines an allocated or designated counter by searching entries in the table.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2023

Level-conversion circuits utilizing level-dependent inverter supply voltages

Voltage level conversion circuits include PMOS pull-down devices or NMOS pull-up devices, and inverters with outputs that determine gate voltages of these devices. The inverters are powered by moving supply voltages, for example complementary supply voltages generated from a pair of cross-coupled inverters. The cross-coupled inverters may implement a data storage latch with the moving supply voltages generated from the internal data storage nodes of the latch.

Turner, Walker Joseph↗

NASA NTRS1975

On-line mass storage system functional design document

A functional system definition for an on-line high density magnetic tape data storage system is provided. This system can be implemented in a multi-purpose, multi-host environment, and satisfy the requirements of economical data storage in the range of 2 to 50 billion bytes.

Earnest, D.↗

DOE OSTI2015

LLNL Data Disk Evaluation Report and Information Gathering Document #449.R1.3

This is a report on Data Storage Disk Evaluation and characterization. The purpose of this memo is to capture new recipients due to some recent characterization issues with the Hammer Mill process. The Data Storage Disk Evaluation report was generated utilizing data acquired during 2009 and 2010 from submitted storage media.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2023

Global inventory and meta-analysis of offshore geologic carbon storage efforts

Poster for presentation at the American Geophysical Union Fall Meeting 2023 detailing work conducted for the Carbon Storage Data Field Work Proposal. This poster presents an inventory and meta-analysis conducted for Carbon Storage Data Task 4, which includes a review of offshore geologic carbon storage projects worldwide including site characterization, resource estimates, and transport information.

Mark-Moser, Mackenzie K.↗

DOE OSTI2020

Policy Considerations When Federating Facilities for Experimental and Observational Data Analysis

Today’s computational, experimental, and observational facilities afford us tremendous opportunities to couple theory and experiment at increasingly large scales. Empirical sensing capabilities are growing dramatically with beam line and detector improvements, and with advances in our ability to deploy large-scale data gathering observations of the natural world. The coupling of computational simulations and analysis to process the data from experimental and observational facilities is giving rise to cross-facility workflows. Such federations of facilities are in fact becoming an explicit requirement for large-scale scientific discovery. As we scale up these pipelines of scientific discovery, each participating facility needs to establish and align policies so that the federation can work seamlessly in an end-to-end manner. This chapter outlines specific policy considerations in enabling the federation of facilities for data analysis. Design choices and vital policy decisions cover the areas of data acquisition and storage, data transfer, computational resource allocation and co-scheduling, seamless federated user access, and cross-cutting governance. By highlighting the explicit and implicit interdependencies between facilities, we aim to provide facility designers and policymakers the information on policy issues to address early in a facility’s operations, thus enabling successful cross-facility federation and improved experimental and observational data analysis outcomes.

Shankar, Mallikarjun (Arjun)↗

DOE OSTI2017

Data systems for the Linac coherent light source

Abstract The data systems for X-ray free-electron laser (FEL) experiments at the Linac coherent light source (LCLS) are described. These systems are designed to acquire and to reliably transport shot-by-shot data at a peak throughput of 5 GB/s to the offline data storage where experimental data and the relevant metadata are archived and made available for user analysis. The analysis and monitoring implementation (AMI) and Photon Science ANAlysis (psana) software packages are described. Psana is open source and freely available.

97 MATHEMATICS AND COMPUTING↗