Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Data Compression”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 415 records · Page 23

Less is More: Bigger Data from Compressive Measurements

Compressive sensing approaches are beginning to take hold in (scanning) transmission electron microscopy (S/TEM) [1,2,3]. Compressive sensing is a mathematical theory about acquiring signals in a compressed form (measurements) and the probability of recovering the original signal by solving an inverse problem [4]. The inverse problem is underdetermined (more unknowns than measurements), so it is not obvious that recovery is possible. Compression is achieved by taking inner products of the signal with measurement weight vectors. Both Gaussian random weights and Bernoulli (0,1) random weights form a large class of measurement vectors for which recovery is possible. The measurements can also be designed through an optimization process. The key insight for electron microscopists is that compressive sensing can be used to increase acquisition speed and reduce dose. Building on work initially developed for optical cameras, this new paradigm will allow electron microscopists to solve more problems in the engineering and life sciences. We will be collecting orders of magnitude more data than previously possible. The reason that we will have more data is because we will have increased temporal/spatial/spectral sampling rates, and we will be able ability to interrogate larger classes of samples that were previously too beam sensitive to survive the experiment. For example consider an in-situ experiment that takes 1 minute. With traditional sensing, we might collect 5 images per second for a total of 300 images. With compressive sensing, each of those 300 images can be expanded into 10 more images, making the collection rate 50 images per second, and the decompressed data a total of 3000 images [3]. But, what are the implications, in terms of data, for this new methodology? Acquisition of compressed data will require downstream reconstruction to be useful. The reconstructed data will be much larger than traditional data, we will need space to store the reconstructions during analysis, and the computational demands for analysis will be higher. Moreover, there will be time costs associated with reconstruction. Deep learning [5] is an approach to address these problems. Deep learning is a hierarchical approach to find useful (for a particular task) representations of data. Each layer of the hierarchy is intended to represent higher levels of abstraction. For example, a deep model of faces might have sinusoids, edges and gradients in the first layer; eyes, noses, and mouths in the second layer, and faces in the third layer. There has been significant effort recently in deep learning algorithms for tasks beyond image classification such as compressive reconstruction [6] and image segmentation [7]. A drawback of deep learning, however, is that training the model requires large datasets and dedicated computational resources (to reduce training time to a few days). A second issue is that deep learning is not user-friendly and the meaning behind the results is usually not interpretable. We have shown it is possible to reduce the data set size while maintaining model quality [8] and developed interpretable models for image classification [9], but the demands are still significant. The key to addressing these problems is to NOT reconstruct the data. Instead, we should design computational sensors that give answers to specific problems. A simple version of this idea is compressive classification [10], where the goal is to classify signal type from a small number of compressed measurements. Classification is a much simpler problem than reconstruction, so 1) much fewer measurements will be necessary, and 2) these measurements will probably not be useful for reconstruction. Other simple examples of computational sensing include determining object volume or the number of objects present in the field of view [11].

Stevens, Andrew↗

A modified adaptive delta modulator

This paper presents the results of a study in data compression of adaptive delta modulated video signals. The Song mode ADM is first investigated at a sampling rate of 16 Mbits/s and shown not to produce enough redundancy to warrant entropy encoding. A modified adaptive delta modulator algorithm is presented that operates at a sampling rate of 16 Mbits/s and does produce sufficient redundancy to yield a 40-50 percent data compression by using a simple code on 4 bit data blocks. Other techniques such as field interpolation and direct substitution are shown to increase the possible data compression further without noticeable degradation in the two input images used in this investigation. The effects of channel errors in the transmission of packet video over a computer network are considered. A leaky integrator is used to reduce the effects of channel errors in the data bits. It is shown that the effects of channel errors can be reduced by field interpolating those packets that can be shown to contain errors.

Barba, J.↗

Time-temperature-stress superposition of PBX 9502 compressive creep data for lifetime predictions

The plastic-bonded explosive (PBX) 9502 is a viscoelastic, high-solids loaded polymer bound composite comprised of 95 weight% (wt%) TATB explosive crystals and 5 wt% FK-800 polymer binder. The uniaxial quasi-static mechanical properties have been studied and characterized extensively over 25+ years, including creep. Creep is the strain evolution over time as a result of an asymmetric load typically at temperatures above ambient. Creep measurements to failure can take excessively long periods of time depending on various factors including applied stress, temperature, and loading rate that render a single test to failure impossible. Using the principles of time-temperature-stress superposition (TTSSP), short-term creep tests at various applied stresses and temperatures can be used for long-term creep predictions. The TTSSP model shifts short-term test data using two factors, one for temperature and stress, creating a master curve at a reference test condition. The master curve can be shifted to predict the creep response at any desired test condition. PBX 9502 compression specimens were tested at three temperatures and various applied stresses. This work discusses the TTSSP analysis of PBX 9502 compressive creep data to create a single comprehensive master curve.

36 MATERIALS SCIENCE↗

Time-Temperature-Stress Superposition of PBX 9502 Compressive Creep Data for Lifetime Predictions

The plastic-bonded explosive (PBX) 9502 is a viscoelastic, high-solids loaded polymer bound composite comprised of 95 weight% (wt%) TATB explosive crystals and 5 wt% FK-800 polymer binder. The uniaxial quasi-static mechanical properties have been studied and characterized extensively over 25+ years, including creep. Creep is the strain evolution over time as a result of an asymmetric load typically at temperatures above ambient. Creep measurements to failure can take excessively long periods of time depending on various factors including applied stress, temperature, and loading rate that render a single test to failure impossible. Using the principles of time-temperature-stress superposition (TTSSP), short-term creep tests at various applied stresses and temperatures can be used for long-term creep predictions. The TTSSP model shifts short-term test data using two factors, one for temperature and stress, creating a master curve at a reference test condition. The master curve can be shifted to predict the creep response at any desired test condition. PBX 9502 compression specimens were tested at three temperatures and various applied stresses. This work discusses the TTSSP analysis of PBX 9502 compressive creep data to create a single comprehensive master curve.

36 MATERIALS SCIENCE↗

A Novel Event Detection Method Using PMU Data with High Precision

To take full advantage of the considerably high reporting rate of phasor measurement units (PMU) data, this paper develops a novel PMU-based event detection methodology. Considering the huge amount of streaming PMU data, a data compression algorithm, swinging door trending (SDT), is first used to compress the PMU data and generate multiple compression intervals. Then dynamic programming is utilized to solve the optimization problem, which is recursively constituted by a score function. Based on predefined PMU event rules, dynamic programming merges adjacent compression intervals with the same slope direction. Finally, all the PMU event features are characterized. A conventional wavelet-based event detection method is compared with the developed dynamic programming based SDT (DPSDT) method. Numerical simulations on the real-time and synthetic PMU data show that the DPSDT method can accurately detect the start-time of an event and the event placement with relatively high precision. Furthermore, the PMU event features, including the magnitude and duration of strokes, are characterized.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Streaming Compression of Scientific Data via Weak-SINDy

Here, in this paper, a streaming weak-SINDy algorithm is developed specifically for compressing streaming scientific data. The production of scientific data, either via simulation or experiments, is undergoing a stage of exponential growth, which makes data compression important and often necessary for storing and utilizing large scientific data sets. As opposed to classical “offline” compression algorithms that perform compression on a readily available data set, streaming compression algorithms compress data “online” while the data generated from simulation or experiments is still flowing through the system. This feature makes streaming compression algorithms well suited for scientific data compression, where storing the full data set offline is often infeasible. This work proposes a new streaming compression algorithm, streaming weak-SINDy, which takes advantage of the underlying data characteristics during compression. The streaming weak-SINDy algorithm constructs feature matrices and target vectors in the online stage via a streaming integration method in a memory efficient manner. The feature matrices and target vectors are then used in the offline stage to build a model through a regression process that aims to recover equations that govern the evolution of the data. For compressing high-dimensional streaming data, we adopt a streaming proper orthogonal decomposition (POD) process to reduce the data dimension and then use the streaming weak-SINDy algorithm to compress the temporal data of the POD expansion. We propose modifications to the streaming weak-SINDy algorithm to accommodate the dynamically updated POD basis. By combining the built model from the streaming weak-SINDy algorithm and a small amount of data samples, the full data flow could be reconstructed accurately at a low memory cost, as shown in the numerical tests.

97 MATHEMATICS AND COMPUTING↗

Compressing Image Data While Limiting the Effects of Data Losses

ICER is computer software that can perform both lossless and lossy compression and decompression of gray-scale-image data using discrete wavelet transforms. Designed for primary use in transmitting scientific image data from distant spacecraft to Earth, ICER incorporates an error-containment scheme that limits the adverse effects of loss of data and is well suited to the data packets transmitted by deep-space probes. The error-containment scheme includes utilization of the algorithm described in "Partitioning a Gridded Rectangle Into Smaller Rectangles " (NPO-30479), NASA Tech Briefs, Vol. 28, No. 7 (July 2004), page 56. ICER has performed well in onboard compression of thousands of images transmitted from the Mars Exploration Rovers.

Kiely, Aaron↗

Some aspects of adaptive transform coding of multispectral data

This paper concerns a data compression study pertaining to multi-spectral scanner (MSS) data. The motivation for this undertaking is the need for securing data compression of images obtained in connection with the Landsat Follow-On Mission, where a compression of at least 6:1 is required. The MSS data used in this study consisted of four scenes: Tristate, consisting of 256 pels per row and a total of 512 rows - i.e., (256x512), (2) Sacramento (256x512), (3) Portland (256x512), and (4) Bald Knob (200x256). All these scenes were on digital tape at 6 bits/pel. The corresponding reconstructed scenes of 1 bit/pel (i.e., a 6:1 compression) are included.

Ahmed, N.↗

Data-storage compression scheme

System uses scheme which does not respond to redundant data. Encoded sensor output signals are transferred to central processing unit only when change occurs in encoded 12-bit word.

Salomon, P. M.↗

Prior data assisted compressed sensing: A novel MR imaging strategy for real time tracking of lung tumors

Purpose: Hybrid radiotherapy-MRI devices promise real time tracking of moving tumors to focus the radiation portals to the tumor during irradiation. This approach will benefit from the increased temporal resolution of MRI's data acquisition and reconstruction. In this work, the authors propose a novel spatial-temporal compressed sensing (CS) imaging strategy for the real time MRI–-prior data assisted compressed sensing (PDACS), which aims to improve the image quality of the conventional CS without significantly increasing reconstruction times. Methods: Conventional 2D CS requires a random sampling of partial k-space data, as well as an iterative reconstruction that simultaneously enforces the image's sparsity in a transform domain as well as maintains the fidelity to the acquired k-space. PDACS method requires the additional acquisition of the prior data, and for reconstruction, it additionally enforces fidelity to the prior k-space domain similar to viewsharing. In this work, the authors evaluated the proposed PDACS method by comparing its results to those obtained from the 2D CS and viewsharing methods when performed individually. All three methods are used to reconstruct images from lung cancer patients whose tumors move and who are likely to benefit from lung tumor tracking. The patients are scanned, using a 3T MRI, under free breathing using the fully sampled k-space with 2D dynamic bSSFP sequence in a sagittal plane containing lung tumor. These images form a reference set for the evaluation of the partial k-space methods. To create partial k-space, the fully sampled k-space is retrospectively undersampled to obtain a range of acquisition acceleration factors, and reconstructed with 2D-CS, PDACS, and viewshare methods. For evaluation, metrics assessing global image artifacts as well as tumor contour shape fidelity are determined from the reconstructed images. These analyses are performed both for the original 3T images and those at a simulated 0.5T equivalent noise level. Results: In the 3.0T images, the PDACS strategy is shown to give superior results compared to viewshare and conventional 2D CS using all metrics. The 2D-CS tends to perform better than viewshare at the low acceleration factors, while the opposite is true at the high acceleration factors. At simulated 0.5T images, PDACS method performs only marginally better than the viewsharing method, both of which are superior compared to 2D CS. The PDACS image reconstruction time (0.3 s/image) is similar to that of the conventional 2D CS. Conclusions: The PDACS method can potentially improve the real time tracking of moving tumors by significantly increasing MRI's data acquisition speeds. In 3T images, the PDACS method does provide a benefit over the other two methods in terms of both the overall image quality and the ability to accurately and automatically contour the tumor shape. MRI's data acquisition may be accelerated using the simpler viewsharing strategy at the lower, 0.5T magnetic field, as the marginal benefit of the PDACS method may not justify its additional reconstruction times.

60 APPLIED LIFE SCIENCES↗

Freeing Space for NASA: Incorporating a Lossless Compression Algorithm into NASA's FOSS System

NASA's Fiber Optic Strain Sensing (FOSS) system can gather and store up to 1,536,000 bytes (1.46 megabytes) per second. Since the FOSS system typically acquires hours - or even days - of data, the system can gather hundreds of gigabytes of data for a given test event. To store such large quantities of data more effectively, NASA is modifying a Lempel-Ziv-Oberhumer (LZO) lossless data compression program to compress data as it is being acquired in real time. After proving that the algorithm is capable of compressing the data from the FOSS system, the LZO program will be modified and incorporated into the FOSS system. Implementing an LZO compression algorithm will instantly free up memory space without compromising any data obtained. With the availability of memory space, the FOSS system can be used more efficiently on test specimens, such as Unmanned Aerial Vehicles (UAVs) that can be in flight for days. By integrating the compression algorithm, the FOSS system can continue gathering data, even on longer flights.

Fiechtner, Kaitlyn↗

Adaptive wavelet compression of large additive manufacturing experimental and simulation datasets

New manufacturing technologies such as additive manufacturing require research and development to minimize the uncertainties in the produced parts. The research involves experimental measurements and large simulations, which result in huge quantities of data to store and analyze. We address this challenge by alleviating the data storage requirements using lossy data compression. We select wavelet bases as the mathematical tool for compression. Unlike images, additive manufacturing data is often represented on irregular geometries and unstructured meshes. Thus, we use Alpert tree-wavelets as bases for our data compression method. We first analyze different basis functions for the wavelets and find the one that results in maximal compression and miminal error in the reconstructed data. We then devise a new adaptive thresholding method that is data-agnostic and allows a priori estimation of the reconstruction error. Finally, we propose metrics to quantify the global and local errors in the reconstructed data. One of the error metrics addresses the preservation of physical constraints in reconstructed data fields, such as divergence-free stress field in structural simulations. While our compression and decompression method is general, we apply it to both experimental and computational data obtained from measurements and thermal/structural modeling of the sintering of a hollow cylinder from metal powders using a Laser Engineered Net Shape process. The results show that monomials achieve optimal compression performance when used as wavelet bases. The new thresholding method results in compression ratios that are two to seven times larger than the ones obtained with commonly used thresholds. Overall, adaptive Alpert tree-wavelets can achieve compression ratios between one and three orders of magnitude depending on the features in the data that are required to preserve. Furthermore, these results show that Alpert tree-wavelet compression is a viable and promising technique to reduce the size of large data structures found in both experiments and simulations.

42 ENGINEERING↗

Lossless Coding Standards for Space Data Systems

The International Consultative Committee for Space Data Systems (CCSDS) is preparing to issue its first recommendation for a digital data compression standard. Because the space data systems of primary interest are employed to support scientific investigations requiring accurate representation, this initial standard will be restricted to lossless compression.

digital data compression digital data compression ↗

Basic cluster compression algorithm

Feature extraction and data compression of LANDSAT data is accomplished by BCCA program which reduces costs associated with transmitting, storing, distributing, and interpreting multispectral image data. Algorithm uses spatially local clustering to extract features from image data to describe spectral characteristics of data set. Approach requires only simple repetitive computations, and parallel processing can be used for very high data rates. Program is written in FORTRAN IV for batch execution and has been implemented on SEL 32/55.

Hilbert, E. E.↗

Noiseless Coding Of Magnetometer Signals

Report discusses application of noiseless data-compression coding to digitized readings of spaceborne magnetometers for transmission back to Earth. Objective of such coding to increase efficiency by decreasing rate of transmission without sacrificing integrity of data. Adaptive coding compresses data by factors ranging from 2 to 6.

Rice, Robert F.↗

The Exploitation of Data Reduction for Visualization

The disparity between the computational speed and storage bandwidth, as demonstrated in Figure 1, is a well known problem that grows with each successive generation. The visualization community is principally responding to this issue by using in situ to reduce which data must be written to storage. However, other communities are taking different, possibly complementary approaches. In particular, data compression is a common general approach to reduce storage demands. Data compression technologies are typically not designed with post processing in mind. The principal metrics measured are compression ratio, the improved bandwidth to storage, and the error introduced. It is assumed that data is inflated to its full size before any post processing can happen. Although when talking about bandwidth disparities, HPC’s dirty little secret is that no part of the memory nor interconnect hardware is increasing at the rate of computation. For example, the Summit supercomputer has a peak computation rate almost 10 times its predecessor, Titan, but only about 4 times the memory, less than twice the aggregate memory bandwidth, and almost no improvement in the interconnect bisection bandwidth. Naively inflating data for post processing does not help with limitations in the memory and interconnect systems.

97 MATHEMATICS AND COMPUTING↗