Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data compression techniques”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3
All sourcesNASA NTRSDOE OSTI
NASA NTRS1990

Data compression applied to HHVT

A task order was written by the High Resolution, High Frame Rate Video Technology (HHVT) project engineers to study data compression techniques that could be applied to the HHVT system. Specifically, the goals of the HHVT data compression study are to accomplish the following: (1) Determine the downlink capabilities of the Space Shuttle and Space Station Freedom to support HHVT data (i.e., determine the maximum data rates and link availability); (2) Determine current and projected capabilities of high speed storage media to support HHVT data by determining their maximum data acquisition/transmission rates and volumes; (3) Identify which experiment in the HHVT Users' Requirement data base need data compression, based on the experiments' imaging requirements; (4) Select the best data compression technique for each of these users by identifying a technique that provides compression but minimizes distortion; and (5) Investigate state-of-the-art technologies for possible implementation of selected data compression techniques. Data compression will be needed because of the high data rates and larger volumes of data that will result from the use of digitized video onboard the Space Shuttle and Space Station Freedom.

Thompson, William K.↗
NASA NTRS1991

Data compression: The end-to-end information systems perspective for NASA space science missions

The unique characteristics of compressed data have important implications to the design of space science data systems, science applications, and data compression techniques. The sequential nature or data dependence between each of the sample values within a block of compressed data introduces an error multiplication or propagation factor which compounds the effects of communication errors. The data communication characteristics of the onboard data acquisition, storage, and telecommunication channels may influence the size of the compressed blocks and the frequency of included re-initialization points. The organization of the compressed data are continually changing depending on the entropy of the input data. This also results in a variable output rate from the instrument which may require buffering to interface with the spacecraft data system. On the ground, there exist key tradeoff issues associated with the distribution and management of the science data products when data compression techniques are applied in order to alleviate the constraints imposed by ground communication bandwidth and data storage capacity.

Tai, Wallace↗
NASA NTRS1995

Subband Coding Methods for Seismic Data Compression

This paper presents a study of seismic data compression techniques and a compression algorithm based on subband coding. The compression technique described could be used as a progressive transmission system, where successive refinements of the data can be requested by the user. This allows seismologists to first examine a coarse version of waveforms with minimal usage of the channel and then decide where refinements are required. Rate-distortion performance results are presented and comparisons are made with two block transform methods.

subband coding seismic data compression techniques↗
DOE OSTI2022

Data Compression in the NEXT-100 Data Acquisition System

NEXT collaboration detectors are based on energy measured by an array of photomultipliers (PMT) and topological event filtering based on an array of silicon photomultipliers (SiPMs). The readout of the PMT sensors for low-frequency noise effects and detector safety issues requires a grounded cathode connection that makes the readout AC-couple with variations in the signal baseline. Strict detector requirements of energy resolution better than 1% FWHM require a precise baseline reconstruction that is performed offline for data analysis and detector performance characterization. Baseline variations make it inefficient to apply traditional lossy data compression techniques, such as zero-suppression, that help to minimize data throughput and, therefore, the dead time of the system. However, for the readout of the SiPM sensors with less demanding requirements in terms of accuracy, a traditional zero-suppression is currently applied with a configuration that allows for a compression ratio of around 71%. The third stage in the NEXT detectors program, the NEXT-100 detector, is a 100 kg detector that instruments approximately five times more PMT sensors and twice the number of SiPM sensors than its predecessor, the NEXT-White detector, putting more pressure in the DAQ throughput, expected to be over 900 MB/s with the current configuration, which will worsen the dead time of the acquisition data system. This paper describes the data compression techniques applied to the sensor data in the NEXT-100 detector, which reduces data throughput and minimizes dead time while maintaining the event rate to the level of its predecessor, around 50 Hz.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2025

Scalable Hybrid Learning Techniques for Scientific Data Compression

Data compression is becoming critical for storing scientific data because many scientific applications need to store large amounts of data and post process this data for scientific discovery. Unlike image and video compression algorithms that limit errors to primary data (PD), scientists require compression techniques that accurately preserve derived quantities of interest (QoIs). Here, this article presents a physics-informed compression technique implemented as an end-to-end, scalable, GPU-based pipeline for data compression that addresses this requirement. Our hybrid compression technique combines machine learning techniques and standard compression methods. Specifically, we combine an autoencoder, an error-bounded lossy compressor to provide guarantees on raw data error, and a constraint satisfaction post-processing step to preserve the QoIs within a minimal error (generally less than floating point error). The effectiveness of the data compression pipeline is demonstrated by compressing nuclear fusion simulation data generated by a large-scale fusion code, XGC, which produces hundreds of terabytes of data in a single day. Our approach works within the ADIOS framework and results in compression by a factor of more than 150 while requiring only a few percent of the computational resources necessary for generating the data, making the overall approach highly effective for practical scenarios.

ITER↗
NASA NTRS1996

Electromagnetic Scattered Field Evaluation and Data Compression Using Imaging Techniques

This is the final report on Project #727625 between The Ohio State University and NASA, Lewis Research Center, Cleveland, Ohio. Under this project, a data compression technique for scattered field data of electrically large targets is developed. The technique was applied to the scattered fields of two targets of interest. The backscattered fields of the scale models of these targets were measured in a ra compact range. For one of the targets, the backscattered fields were also calculated using XPATCH computer code. Using the technique all scattered field data sets were compressed successfully. A compression ratio of the order 40 was achieved. In this report, the technique is described briefly and some sample results are included.

Gupta, I. J.↗
NASA NTRS1993

An investigative study of multispectral data compression for remotely-sensed images using vector quantization and difference-mapped shift-coding

A study is conducted to investigate the effects and advantages of data compression techniques on multispectral imagery data acquired by NASA's airborne scanners at the Stennis Space Center. The first technique used was vector quantization. The vector is defined in the multispectral imagery context as an array of pixels from the same location from each channel. The error obtained in substituting the reconstructed images for the original set is compared for different compression ratios. Also, the eigenvalues of the covariance matrix obtained from the reconstructed data set are compared with the eigenvalues of the original set. The effects of varying the size of the vector codebook on the quality of the compression and on subsequent classification are also presented. The output data from the Vector Quantization algorithm was further compressed by a lossless technique called Difference-mapped Shift-extended Huffman coding. The overall compression for 7 channels of data acquired by the Calibrated Airborne Multispectral Scanner (CAMS), with an RMS error of 15.8 pixels was 195:1 (0.41 bpp) and with an RMS error of 3.6 pixels was 18:1 (.447 bpp). The algorithms were implemented in software and interfaced with the help of dedicated image processing boards to an 80386 PC compatible computer. Modules were developed for the task of image compression and image analysis. Also, supporting software to perform image processing for visual display and interpretation of the compressed/classified images was developed.

Jaggi, S.↗
NASA NTRS2004

An adaptable recursive binary entropy coding technique

We present a novel data compression technique, called recursive interleaved entropy coding, that is based on recursive interleaving of variable-to-variable length binary source codes.

data compression entrophy coding biliary adaptive ↗
NASA NTRS1991

SAR data compression: Application, requirements, and designs

The feasibility of reducing data volume and data rate is evaluated for the Earth Observing System (EOS) Synthetic Aperture Radar (SAR). All elements of data stream from the sensor downlink data stream to electronic delivery of browse data products are explored. The factors influencing design of a data compression system are analyzed, including the signal data characteristics, the image quality requirements, and the throughput requirements. The conclusion is that little or no reduction can be achieved in the raw signal data using traditional data compression techniques (e.g., vector quantization, adaptive discrete cosine transform) due to the induced phase errors in the output image. However, after image formation, a number of techniques are effective for data compression.

Curlander, John C.↗
NASA NTRS1981

Compressing TV-image data

Compressing technique calculates activity estimator for each segment of image line. Estimator is used in conjunction with allowable bits per line, N, to determine number of bits necessary to code each segment and which segments can tolerate truncation. Preprocessed line data are then passed to adaptive variable-length coder, which selects optimum transmission code. Method increases capacity of broadcast and cable television transmissions and helps reduce size of storage medium for video and digital audio recordings.

Hilbert, E. E.↗
NASA NTRS1989

Image data compression investigation

NASA continuous communications systems growth has increased the demand for image transmission and storage. Research and analysis was conducted on various lossy and lossless advanced data compression techniques or approaches used to improve the efficiency of transmission and storage of high volume stellite image data such as pulse code modulation (PCM), differential PCM (DPCM), transform coding, hybrid coding, interframe coding, and adaptive technique. In this presentation, the fundamentals of image data compression utilizing two techniques which are pulse code modulation (PCM) and differential PCM (DPCM) are presented along with an application utilizing these two coding techniques.

Myrie, Carlos↗
NASA NTRS1990

Phase Calibration Of Radar Polarimetric Data

Technique for phase calibration of data acquired by airborne imaging radar polarimeter based on extraction of calibration parameters from data themselves. Enables use of data-compression technique to reduce volume of data in synthetic-aperture-radar correlator. Typical radar polarimeter includes transmitting and receiving channels for horizontally and vertically polarized signals. Phase delay in each channel usually known only approximately if at all. Consequently, necessary to phase-calibrate radar return signals.

Zebker, Howard A.↗
NASA NTRS1971

A real time programmable data compression system for video data.

Description of the implementation of a data compression system for the real-time operational transmission (through microwave links) of ATS satellite pictures between the command and data acquisition station and a central location for computer processing. The system features the use of general-purpose minicomputers for encoding and decoding; this makes it possible to vary the employed data compression technique and to make simultaneous statistical calculations on the data. Data compression and expansion is accomplished in a manner that does not lower data quality.

Kutz, R. L.↗
NASA NTRS1991

Scientific requirements for space science data systems

In the 1990's space plasma physics studies will increasingly involve correlative analysis of observations from multiple instruments and multiple spacecraft. The solar terrestrial physics missions in the 1990's will be designed around simultaneous observations from spacecraft monitoring the solar wind, the polar magnetosphere and the near and distant magnetotail. Within these regions, clusters of spacecraft flying in formation will increasingly involve comparative magnetospheric studies. No single lab will have the expertise to process and analyze all of the different types of data so the data repositories will be distributed. Catalog and browse systems will be required to help select events for study. Data compression techniques may be useful in designing the data bases used for selecting events for study. Data compression onboard the spacecraft will be necessary, since instrument data rates will be much larger than available telemetry rates. However, considerable care will be necessary to avoid losing valuable data when applying data compression algorithms.

Walker, Raymond J.↗
NASA NTRS1993

Low-Complexity Progressive Image-Transmission Schemes For Space Applications

This paper describes the use of combination of multiresolution image representation and data compression techniques to reduce transmission time and permit early recognition of images. Data compression removes the inherent redundancies in the image data to reduce the overal data volume.

Image Transmission↗
DOE OSTI2026

Scientific Data Compression for Large Scale Computational Fluid Dynamics (CFD) Simulations

This Cooperative Research and Development Agreement (CRADA) between Oak Ridge National Laboratory (ORNL) and General Electric (GE) investigated methods for reducing the size of large computational fluid dynamics (CFD) simulation datasets using scientific data compression techniques. The work focused on adapting the MultiGrid Adaptive Reduction of Data (MGARD) compression framework and integrating it with high-performance I/O and visualization tools used in CFD workflows. MGARD uses hierarchical multilevel decomposition to enable error-controlled compression of floating-point scientific data while preserving quantities of interest. During the project, MGARD compression was integrated with the ADIOS I/O framework and visualization tools such as ParaView to enable efficient storage, transfer, and analysis of simulation data. The collaboration also explored approaches for improving compression performance for CFD data defined on unstructured meshes. Results demonstrate that scientific data compression can significantly reduce storage requirements and improve data management for large-scale CFD simulations.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1987

JPL multipolarization workstation - Hardware, software and examples of data analysis

A low-cost stand-alone interactive image processing workstation has been developed for operations on multipolarization JPL aircraft SAR data, as well as data from future spaceborne imaging radars. A recently developed data compression technique is used to reduce the data volume to 10 Mbytes, for a typical data set, so that interactive analysis may be accomplished in a timely and efficient manner on a supermicrocomputer. In addition to presenting a hardware description of the work station, attention is given to the software that has been developed. Three illustrative examples of data analysis are presented.

Burnette, Fred↗