Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Data Compression”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6

Fast algorithms for nonconvex compression sensing: MRI reconstruction from very few data

Compressive sensing is the reconstruction of sparse images or signals from very few samples, by means of solving a tractable optimization problem. In the context of MRI, this can allow reconstruction from many fewer k-space samples, thereby reducing scanning time. Previous work has shown that nonconvex optimization reduces still further the number of samples required for reconstruction, while still being tractable. In this work, we extend recent Fourier-based algorithms for convex optimization to the nonconvex setting, and obtain methods that combine the reconstruction abilities of previous nonconvex approaches with the computational speed of state-of-the-art convex methods.

99 GENERAL AND MISCELLANEOUS↗

Parallel compression of data chunks of a shared data object using a log-structured file system

Techniques are provided for parallel compression of data chunks being written to a shared object. A client executing on a compute node or a burst buffer node in a parallel computing system stores a data chunk generated by the parallel computing system to a shared data object on a storage node by compressing the data chunk; and providing the data compressed data chunk to the storage node that stores the shared object. The client and storage node may employ Log-Structured File techniques. The compressed data chunk can be de-compressed by the client when the data chunk is read. A storage node stores a data chunk as part of a shared object by receiving a compressed version of the data chunk from a compute node; and storing the compressed version of the data chunk to the shared data object on the storage node.

97 MATHEMATICS AND COMPUTING↗

TTHRESH: Tensor Compression for Multidimensional Visual Data

Memory and network bandwidth are decisive bottlenecks when handling high-resolution multidimensional data sets in visualization applications, and they increasingly demand suitable data compression strategies. In this paper, we introduce a novel lossy compression algorithm for multidimensional data over regular grids. It leverages the higher-order singular value decomposition (HOSVD), a generalization of the SVD to three dimensions and higher, together with bit-plane, run-length and arithmetic coding to compress the HOSVD transform coefficients. Our scheme degrades the data particularly smoothly and achieves lower mean squared error than other state-of-the-art algorithms at low-to-medium bit rates, as it is required in data archiving and management for visualization purposes. Further advantages of the proposed algorithm include very fine bit rate selection granularity and the ability to manipulate data at very small cost in the compression domain, for example to reconstruct filtered and/or subsampled versions of all (or selected parts) of the data set.

97 MATHEMATICS AND COMPUTING↗

Classical and quantum compression for edge computing: the ubiquitous data dimensionality reduction

Edge computing aims to address the challenges associated with communicating and transferring large amounts of data generated remotely to a data center in a timely and efficient manner. A central pillar of edge computing is local (i.e., at- or near-source) data processing capability so that data transfer to a data center for processing can be minimized. Data compression at the edge is therefore a natural component of edge workflows. Here we present a survey of data compression algorithms with a focus on edge computing. Not all compression algorithms can accommodate the data type heterogeneity, tight processing and communication time constraints, or energy efficiency requirement characteristics of edge computing. We discuss specific examples of compression algorithms that are being explored in the context of edge computing. We end our review with a brief survey of emerging quantum compression techniques that are of importance in quantum information processing, including the proposed concept of quantum edge computing.

97 MATHEMATICS AND COMPUTING↗

Extreme Compression for Large Scale Data Store

For the last 5 years Accelogic pioneered and perfected a radically new theory of numerical computing codenamed “Compressive Computing”, which has an extremely profound impact on real-world computer science [1]. At the core of this new theory is the discovery of one of its fundamental theorems which states that, under very general conditions, the vast majority (typically between 70% and 80%) of the bits used in modern large-scale numerical computations are absolutely irrelevant for the accuracy of the end result. This theory of Compressive Computing provides mechanisms able to identify (with high intelligence and surgical accuracy) the number of bits (i.e., the precision) that can be used to represent numbers without affecting the substance of the end results, as they are computed and vary in real time. The bottom line outcome would be to provide a state-of-the-art compression algorithm that surpasses those currently available in the ROOT framework, with the purpose of enabling substantial economic and operational gains (including speedup) for High Energy and Nuclear Physics data storage/analysis. In our initial studies, a factor of nearly x4 (3.9) compression was achieved with RHIC/STAR data where ROOT compression managed only x1.4.In this contribution, we will present our concepts of “functionally lossless compression”, have a glance at examples and achievements in other communities, present the results and outcome of our current, ongoing R&D, as well as present a high-level view of our plan to move forward with a ROOT implementation that would deliver a basic solution readily integrated into HENP applications. As a collaboration of experimental scientists, private industry, and the ROOT Team, our aim is to capitalize on the substantial success delivered by the initial effort and produce a robust technology properly packaged as an open-source tool that could be used by virtually every experiment around the world as means for improving data management and accessibility.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

About a method for compressing X-ray computed microtomography data

The management of scientific data is of high importance especially for experimental techniques that produce big data volumes. Such a technique is x-ray computed tomography (CT) and its community has introduced advanced data formats which allow for better management of experimental data. Rather than the organization of the data and the associated meta-data, the main topic on this work is data compression and its applicability to experimental data collected from a synchrotron-based CT beamline at the Elettra-Sincrotrone Trieste facility (Italy) and studies images acquired from various types of samples. This study covers parallel beam geometry, but it could be easily extended to a cone-beam one. The reconstruction workflow used is the one currently in operation at the beamline. Contrary to standard image compression studies, this manuscript proposes a systematic framework and workflow for the critical examination of different compression techniques and does so by applying it to experimental data. Beyond the methodology framework, this study presents and examines the use of JPEG-XR in combination with HDF5 and TIFF formats providing insights and strategies on data compression and image quality issues that can be used and implemented at other synchrotron facilities and laboratory systems. In conclusion, projection data compression using JPEG-XR appears as a promising, efficient method to reduce data file size and thus to facilitate data handling and image reconstruction

data compression↗

Graphics processing unit-assisted lossless decompression

Systems and methods for decompressing compressed data that has been compressed by way of a lossless compression algorithm are described herein. In a general embodiment, a graphics processing unit (GPU) is programmed to receive compressed data packets and decompress such packets in parallel. The compressed data packets are compressed representations of an image, and the lossless compression algorithm is a Rice compression algorithm.

97 MATHEMATICS AND COMPUTING↗

Enhancing ZFP: A Statistical Approach to Understanding and Reducing Error Bias in a Lossy Floating-Point Compression Algorithm

The amount of data generated and gathered in scientific simulations and data collection applications is continuously growing, putting mounting pressure on storage and bandwidth concerns. A means of reducing such issues is data compression; but, lossless data compression is typically ineffective when applied to floating-point data. Thus, users tend to apply a lossy data compressor, which allows for small deviations from the original data. It is essential to understand how the error from lossy compression impacts the accuracy of the data analytics. Thus, we must analyze not only the compression properties but the error as well. In this paper, we provide a statistical analysis of the error caused by ZFP compression, a state-of-the-art, lossy compression algorithm explicitly designed for floating-point data. We show that the error is indeed biased and propose simple modifications to the algorithm to neutralize the bias and further reduce the resulting error.

97 MATHEMATICS AND COMPUTING↗

Electronic device and method for compressing sampled data

An electronic device for compressing sampled data comprises a memory element and a processing element. The memory element is configured to store sampled data points and sampled times. The processing element is in electronic communication with the memory element and is configured to receive a plurality of sampled data points, a slope for each sampled data point in succession, the slope being a value of a change between the sampled data point and its successive sampled data point, and store the sampled data point in the memory element when the slope changes in value from a previous sampled data point.

97 MATHEMATICS AND COMPUTING↗

Electronic device and method for compressing sampled data

An electronic device for compressing sampled data comprises a memory element and a processing element. The memory element is configured to store sampled data points and sampled times. The processing element is in electronic communication with the memory element and is configured to receive a plurality of sampled data points, a slope for each sampled data point in succession, the slope being a value of a change between the sampled data point and its successive sampled data point, and store the sampled data point in the memory element when the slope changes in value from a previous sampled data point.

Morrow, Mitchell Hedges↗

Electronic device and method for compressing sampled data

An electronic device for compressing sampled data comprises a memory element and a processing element. The memory element is configured to store sampled data points and sampled times. The processing element is in electronic communication with the memory element and is configured to receive a plurality of sampled data points, a slope for each sampled data point in succession, the slope being a value of a change between the sampled data point and its successive sampled data point, and store the sampled data point in the memory element when the slope changes in value from a previous sampled data point.

Tohlen, Michael Aaron↗

Electronic device and method for compressing sampled data

An electronic device for compressing sampled data comprises a memory element and a processing element. The memory element is configured to store sampled data points and sampled times. The processing element is in electronic communication with the memory element and is configured to receive a plurality of sampled data points, a slope for each sampled data point in succession, the slope being a value of a change between the sampled data point and its successive sampled data point, and store the sampled data point in the memory element when the slope changes in value from a previous sampled data point.

Morrow, Mitchell Hedges↗

Fast 2D Bicephalous Convolutional Autoencoder for Compressing 3D Time Projection Chamber Data

High-energy large-scale particle colliders produce data at high speed in the order of 1 terabytes per second in nuclear physics and petabytes per second in high energy physics. Developing real-time data compression algorithms to reduce such data at high throughput to fit permanent storage has drawn increasing attention. Specifically, at the newly constructed sPHENIX experiment at the Relativistic Heavy Ion Collider (RHIC), a time projection chamber is used as the main tracking detector, which records particle trajectories in a volume of three-dimensional (3D) cylinder. The resulting data are usually very sparse with occupancy around 10.8%. Such sparsity presents a challenge to conventional learning-free lossy compression algorithms, such as SZ, ZFP, and MGARD. The 3D convolutional neural network (CNN)-based approach, Bicephalous Convolutional Autoencoder (BCAE), outperforms traditional methods both in compression rate and reconstruction accuracy. BCAE can also utilize the computation power of graphical processing units suitable for deployment in a modern heterogeneous highperformance computing environment. This work introduces two BCAE variants: BCAE++ and BCAE-2D. BCAE++ achieves a 15% better compression ratio and a 77% better reconstruction accuracy measured in mean absolute error compared with BCAE. BCAE-2D treats the radial direction as the channel dimension of an image, resulting in a 3× speedup in compression throughput. In addition, we demonstrate an unbalanced autoencoder with a larger decoder can improve reconstruction accuracy without significantly sacrificing throughput. Lastly, we observe both the BCAE++ and BCAE-2D can benefit more from using half-precision mode in throughput (76 - 79% increase) without loss in reconstruction accuracy. The source code and links to data and pretrained models can be found at https://github.com/BNL-DAQ-LDRD/NeuralCompression_v2

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Compression of tokamak boundary plasma simulation data using a maximum volume algorithm for matrix skeleton decomposition

This report demonstrates satisfactory data compression of SOLPS-ITER simulation output ranging from 2D fields, 1D profiles, and 0D scalar variables with a novel matrix decomposition approach. The singular value decomposition (SVD) scales poorly for large matrix sizes and is unsuited to the application on high dimensional data common to fusion plasma physics simulation. In this work, we employ the columns-submatrix-rows (CUR) matrix factorization technique in order to compute a low-rank approximation up to two orders of magnitude faster than the SVD, but within a nominal L2-norm relative error of ε = 10 –2 . In addition, the CUR approach maintains the original format of the data, in its extracted columns and rows, allowing for interpretable data storage at the original resolution of the simulation. We utilize an iterative algorithm to compute the CUR decomposition of simulation output by maximizing the volume, or linearly independent information content, of a low-rank submatrix contained within the data. Experiments over $\textit{n} × \textit{n}$ randomized test matrices with embedded rank-deficient features show that this maximum volume implementation of CUR matrix approximation has reduced asymptotic computational complexity on the order of n compared to the SVD, which scales approximately as $n^3$. These results show that the CUR technique can be used to effectively select time step snapshots (columns) of over 140 SOLPS-ITER output variables and the associated discretized coordinate timeseries (rows) allowing for reconstruction of the complete simulation dynamics.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Fast digital lossy compression for X-ray ptychographic data

Increases in X-ray brightness from synchrotron light sources lead to a requirement for higher frame rates from hybrid pixel array detectors (HPADs), while also favoring charge integration over photon counting. However, transfer of the full uncompressed data will begin to constrain detector design, as well as limit the achievable continuous frame rate. Here a data compression scheme that is easy to implement in a HPAD's application-specific integrated circuit (ASIC) is described, and how different degrees of compression affect image quality in ptychography, a commonly employed coherent imaging method, is examined. Using adaptive encoding quantization, it is shown in simulations that one can digitize signals up to 16383 photons per pixel (corresponding to 14 bits of information) using only 8 or 9 bits for data transfer, with negligible effect on the reconstructed image.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

SU-G-JeP1-15: Sliding Window Prior Data Assisted Compressed Sensing for MRI Lung Tumor Tracking

Purpose: Prior Data Assisted Compressed Sensing (PDACS) is a partial k-space acquisition and reconstruction method for mobile tumour (i.e. lung) tracking using on-line MRI in radiotherapy. PDACS partially relies on prior data acquired at the beginning of dynamic scans, and is therefore susceptible to artifacts in longer duration scan due to slow drifts in MR signal. A novel sliding window strategy is presented to mitigate this effect. Methods: MRI acceleration is simulated by retrospective removal of data from the fully sampled sets. Six lung cancer patients were scanned (clinical 3T MRI) using a balanced steady state free precession (bSSFP) sequence for 3 minutes at approximately 4 frames per second, for a total of 650 dynamics. PDACS acceleration is achieved by undersampling of k-space in a single pseudo-random pattern. Reconstruction iteratively minimizes the total variations while constraining the images to satisfy both the currently acquired data and the prior data in missing k-space. Our novel sliding window technique (SW-PDACS), uses a series of distinct pseudo-random under-sampling patterns of partial k-space – with the prior data drawn from a sliding window of the most recent data available. Under-sampled data, simulating 2 – 5x acceleration are reconstructed using PDACS and SW-PDACS. Three quantitative metrics: artifact power, centroid error and Dice’s coefficient are computed for comparison. Results: Quantitively metric values from all 6 patients are averaged in 3 bins, each containing approximately one minute of dynamic data. For the first minute bin, PDACS and SW-PDACS give comparable results. Progressive decline in image quality metrics in bins 2 and 3 are observed for PDACS. No decline in image quality is observed for SW-PDACS. Conclusion: The novel approach presented (SW-PDACS) is a more robust for accelerating longer duration (>1 minute) dynamic MRI scans for tracking lung tumour motion using on-line MRI in radiotherapy. B.G. Fallone is a co-founder and CEO of MagnetTx Oncology Solutions (under discussions to license Alberta bi-planar linac MR for commercialization).

60 APPLIED LIFE SCIENCES↗

Monitoring of Optical Emission from High Temperature Plasma Based on Chromatic Modulation

An integrated experimental approach is presented for processing the optical emission produced from electric arc plasma. The method is based on chromatic modulation techniques to provide a holistic measurement of the persistence of particle decays within the environment of high power circuit breakers. Chromaticity changes in a number of chromatic parameters are related to changes in physical electric arc plasma environment (e.g. particle concentration). The results are in the form of chromatic maps which show how the overall electric arc plasma and its environment behave and respond. Such maps show the totality of information which can be accessed about the arcing event and the level of monitoring discrimination which is achievable with the chromatic methodology in a simple and easy to understand manner. The suggested method provides easier data analysis and high levels of data compression.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Hyperspectral image reconstruction for x-ray fluorescence tomography

A penalized maximum-likelihood estimation is proposed to perform hyperspectral (spatio-spectral) image reconstruction for X-ray fluorescence tomography. The approach minimizes a Poisson-based negative log-likelihood of the observed photon counts, and uses a penalty term that has the effect of encouraging local continuity of model parameter estimates in both spatial and spectral dimensions simultaneously. The performance of the reconstruction method is demonstrated with experimental data acquired from a seed of arabidopsis thaliana collected at the 13-ID-E microprobe beamline at the Advanced Photon Source. The resulting element distribution estimates with the proposed approach show significantly better reconstruction quality than the conventional analytical inversion approaches, and allows for a high data compression factor which can reduce data acquisition times remarkably. In particular, this technique provides the capability to tomographically reconstruct full energy dispersive spectra without compromising reconstruction artifacts that impact the interpretation of results.

Fluorescence microscopy↗