Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Data Compression”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 469 records · Page 26

Pre-coding method and apparatus for multiple source or time-shifted single source data and corresponding inverse post-decoding method and apparatus

A pre-coding method and device for improving data compression performance by removing correlation between a first original data set and a second original data set, each having M members, respectively. The pre-coding method produces a compression-efficiency-enhancing double-difference data set. The method and device produce a double-difference data set, i.e., an adjacent-delta calculation performed on a cross-delta data set or a cross-delta calculation performed on two adjacent-delta data sets, from either one of (1) two adjacent spectral bands coming from two discrete sources, respectively, or (2) two time-shifted data sets coming from a single source. The resulting double-difference data set is then coded using either a distortionless data encoding scheme (entropy encoding) or a lossy data compression scheme. Also, a post-decoding method and device for recovering a second original data set having been represented by such a double-difference data set.

Yeh, Pen-Shu↗

Application specific compression : final report.

With the continuing development of more capable data gathering sensors, comes an increased demand on the bandwidth for transmitting larger quantities of data. To help counteract that trend, a study was undertaken to determine appropriate lossy data compression strategies for minimizing their impact on target detection and characterization. The survey of current compression techniques led us to the conclusion that wavelet compression was well suited for this purpose. Wavelet analysis essentially applies a low-pass and high-pass filter to the data, converting the data into the related coefficients that maintain spatial information as well as frequency information. Wavelet compression is achieved by zeroing the coefficients that pertain to the noise in the signal, i.e. the high frequency, low amplitude portion. This approach is well suited for our goal because it reduces the noise in the signal with only minimal impact on the larger, lower frequency target signatures. The resulting coefficients can then be encoded using lossless techniques with higher compression levels because of the lower entropy and significant number of zeros. No significant signal degradation or difficulties in target characterization or detection were observed or measured when wavelet compression was applied to simulated and real data, even when over 80% of the coefficients were zeroed. While the exact level of compression will be data set dependent, for the data sets we studied, compression factors over 10 were found to be satisfactory where conventional lossless techniques achieved levels of less than 3.

99 GENERAL AND MISCELLANEOUS↗

Compression research on the REINAS Project

We present approaches to integrating data compression technology into a database system designed to support research of air, sea, and land phenomena of interest to meteorology, oceanography, and earth science. A key element of the Real-Time Environmental Information Network and Analysis System (REINAS) system is the real-time component: to provide data as soon as acquired. Compression approaches being considered for REINAS include compression of raw data on the way into the database, compression of data produced by scientific visualization on the way out of the database, compression of modeling results, and compression of database query results. These compression needs are being incorporated through client-server, API, utility, and application code development.

Rosen, Eric↗

Organizing Compression of Hyperspectral Imagery to Allow Efficient Parallel Decompression

family of schemes has been devised for organizing the output of an algorithm for predictive data compression of hyperspectral imagery so as to allow efficient parallelization in both the compressor and decompressor. In these schemes, the compressor performs a number of iterations, during each of which a portion of the data is compressed via parallel threads operating on independent portions of the data. The general idea is that for each iteration it is predetermined how much compressed data will be produced from each thread.

Klimesh, Matthew A.↗

Determining Biosignatures by Complexity Analysis in Antarctic Cryptoendolithic Communities

One of the most difficult problems of life detection is that of identifying biosignatures across a wide range of scales using multiple co-registered probes. The technique should be of equal utility across a wide range of search spaces from remote sensors probing volumes of space or planetary surfaces, visual eye or camera searches across the surface of a rock in Antarctica, low resolution microscopic scanning of a rock or a space craft in situ, or high resolution electron microscope and computerized tomography scanning of geobiological samples. We describe here an approach to this problem which derives in large part from past work done in the area of astrophysics - namely the analysis of complexity in galactic signals by data compression methods. This approach is a radically new one for geobiology and astrobiology, and allows us to assess the complexity (and thus potential biogenicity) of an object being examined. This is done by considering the information within pixels of an image (regardless the sensor used to gather the information) as an energetic system capable of description in terms of classical thermodynamics. The image data space is searched by an algorithm that judges complexity via data compression (e.g., the more compressible it is, the less complex, and vice versa) and maximum entropy as originally outlined by Shannon. At present we are implementing methods to utilize images from multiple sensors gathering different kinds of information (e.g., visible gray-scale data, color analyses, UV fluorescence, chemical information, etc). We present here preliminary data from deep UV fluorescence and ESEM (Environmental Scanning Electron Microscope) images from a layered cryptoendolithic community of an Antarctic rock.

Storrie-Lombardi, M. C.↗

A computer program for plotting stress-strain data from compression, tension, and torsion tests of materials

A computer program for plotting stress-strain curves obtained from compression and tension tests on rectangular (flat) specimens and circular-cross-section specimens (rods and tubes) and both stress-strain and torque-twist curves obtained from torsion tests on tubes is presented in detail. The program is written in FORTRAN 4 language for the Control Data 6000 series digital computer with the SCOPE 3.0 operating system and requires approximately 110000 octal locations of core storage. The program has the capability of plotting individual strain-gage outputs and/or the average output of several strain gages and the capability of computing the slope of a straight line which provides a least-squares fit to a specified section of the plotted curve. In addition, the program can compute the slope of the stress-strain curve at any point along the curve. The computer program input and output for three sample problems are presented.

Greenbaum, A.↗

Experimental verification of the high pressure crystal structures in NH{sub 3}BH{sub 3}

A detailed high-pressure study on NH{sub 3}BH{sub 3} has been carried out using in situ synchrotron X-ray diffraction (XRD) and Raman scattering with a diamond anvil cell up to 20 and 33 GPa, respectively. The Rietveld refinement based on the XRD pattern and analysis of Raman data indicate two first-order phase transitions from the ambient pressure I4 mm structure (α-NH{sub 3}BH{sub 3}) to a high pressure Cmc2{sub 1} phase (β-NH{sub 3}BH{sub 3}) at 2.14 GPa, and further into a monoclinic P2{sub 1} (Z = 2) phase (γ-NH{sub 3}BH{sub 3}) at 9.67 GPa. Fitting the measured volumetric compression data to the third order Birch-Murnaghan equation of state reveals a bulk modulus of B{sub 0} = 9.9 ± 0.5 and 17.0 ± 3.0 GPa (with fixed B{sub 0}{sup ′} = 4) for the β-NH{sub 3}BH{sub 3} below and above 5 GPa, respectively. Still, with the splitting of the NBH rock mode in Raman experiment, it is concluded that a second-order isostructural phase transition occurs at 5 GPa. By analyzing the dihydrogen bonding framework, the origin of the isostructural phase transition is attributed to the number of dihydrogen bondings per molecule in the Cmc2{sub 1} phase increasing from 12 to 14 at 5 GPa.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

JPEG 2000 Encoding with Perceptual Distortion Control

An alternative approach has been devised for encoding image data in compliance with JPEG 2000, the most recent still-image data-compression standard of the Joint Photographic Experts Group. Heretofore, JPEG 2000 encoding has been implemented by several related schemes classified as rate-based distortion-minimization encoding. In each of these schemes, the end user specifies a desired bit rate and the encoding algorithm strives to attain that rate while minimizing a mean squared error (MSE). While rate-based distortion minimization is appropriate for transmitting data over a limited-bandwidth channel, it is not the best approach for applications in which the perceptual quality of reconstructed images is a major consideration. A better approach for such applications is the present alternative one, denoted perceptual distortion control, in which the encoding algorithm strives to compress data to the lowest bit rate that yields at least a specified level of perceptual image quality. Some additional background information on JPEG 2000 is prerequisite to a meaningful summary of JPEG encoding with perceptual distortion control. The JPEG 2000 encoding process includes two subprocesses known as tier-1 and tier-2 coding. In order to minimize the MSE for the desired bit rate, a rate-distortion- optimization subprocess is introduced between the tier-1 and tier-2 subprocesses. In tier-1 coding, each coding block is independently bit-plane coded from the most-significant-bit (MSB) plane to the least-significant-bit (LSB) plane, using three coding passes (except for the MSB plane, which is coded using only one "clean up" coding pass). For M bit planes, this subprocess involves a total number of (3M - 2) coding passes. An embedded bit stream is then generated for each coding block. Information on the reduction in distortion and the increase in the bit rate associated with each coding pass is collected. This information is then used in a rate-control procedure to determine the contribution of each coding block to the output compressed bit stream.

Watson, Andrew B.↗

Femtosecond diffraction studies of the sodium chloride phase diagram under laser shock compression

The phase diagram of sodium chloride (NaCl) under laser shock compression has been studied at Linac Coherent Light Source (LCLS) at the x-ray free-electron laser facility. Both solid–solid (B1 → B2) and solid–liquid (B2 → liquid) transitions have been observed along the Hugoniot over nanosecond time scales. By combining structural measurements through in situ x-ray diffraction, pressure determination through velocimetry, and a thermal equation-of-state, the shock-compressed data are used to constrain the phase diagram of NaCl. Transformation into the B2 phase is found to occur at 28(2) GPa, and B2–liquid coexistence is observed between 54(4) and 66(6) GPa, with near full melt at 66(6) GPa. Late-time pressure release from an initial shocked B2-state results in a B2 → B1 back transformation. Our results show agreement with previous static compression data, suggesting that the time scale for melting is very rapid and that equilibrium states in NaCl are being accessed over nanosecond time scales. Finally, a multiphase equation-of-state description of NaCl incorporated into a one-dimensional hydrocode is used to interpret pressure and temperature evolution over these rapid time scales.

36 MATERIALS SCIENCE↗

Dark Energy Survey Year 3 results: optimized $w$CDM simulation-based inference with weak lensing map-level hybrid statistics

We present cosmological constraints from the Dark Energy Survey Year 3 (DES Y3) weak lensing data using hierarchical hybrid statistics within a Bayesian simulation-based inference framework that is based on the Gower Street simulations. To maximize the precision of the inference, we have developed a new, information-theory based, data compression of the weak lensing maps to just seven highly informative summary statistics. The hybrid scheme exploits the high information content of the power spectrum, compressing both the power spectrum and neural-based summaries that are designed to extract further information. Our simulation-based approach enables principled forward modelling of all major sources of systematic uncertainty and survey properties into realistic mock observations, including the survey mask, photometric redshift uncertainties, intrinsic galaxy alignments, multiplicative shear calibration bias, source galaxy clustering, non-Gaussian shape noise, and non-linear structure formation. The summary statistics are then used in a Bayesian simulation-based inference pipeline. The inference is validated through coverage tests and checks for robustness against baryonic feedback. Assuming a $w$CDM cosmology, our analysis yields $S_8 = 0.808 \pm 0.017$, $Ω_{\rm m} = 0.325 \pm 0.024$, and $w < -0.766$ (marginalized posterior 68 per cent credible intervals). This rigorous combination of information theory, physics- and neural network-based extreme data compression, and principled Bayesian analysis improves the figure of merit for $(Ω_{\rm m}, S_8, w)$ by 60 per cent over the previous state-of-the-art, and by almost a factor of 3 over two-point analyses of the same data. They are the most precise joint constraints on $(Ω_{\rm m}, S_8, w)$ from weak gravitational lensing data alone of any survey to date. We intend to apply this analysis to the more recent DES Y6 data.

Williamson, J. [University Coll. London]↗

Tensor-decomposition techniques for ab initio nuclear structure calculations: From chiral nuclear potentials to ground-state energies

The computational resources needed to generate the ab initio solution of the nuclear many-body problem for increasing mass number and/or accuracy necessitates innovative developments to improve upon (i) the storage of many-body operators and (ii) the scaling of many-body methods used to evaluate nuclear observables. The storing and efficient handling of many-body operators with high particle ranks is currently one of the major bottlenecks limiting the applicability range of ab initio studies with respect to mass number and accuracy. Recently, the application of tensor decomposition techniques to many-body tensors has proven highly beneficial to reduce the computational cost of ab initio calculations in quantum chemistry and solid-state physics. The impact of applying state-of-the-art tensor factorization techniques to modern nuclear Hamiltonians derived from chiral effective field theory is investigated. Subsequently, the error induced by the tensor decomposition of the input Hamiltonian on ground-state energies of closed-shell nuclei calculated via second-order many-body perturbation theory is benchmarked. The first proof-of-principles application of tensor-decomposition techniques to the nuclear Hamiltonian is performed. Two different tensor formats are investigated by systematically benchmarking the approximation error on matrix elements stored in various bases of interest. The analysis is achieved while including normal-ordered three-nucleon interactions that are nowadays used as input to the most advanced ab initio calculations in medium-mass nuclei. With the aid of the factorized Hamiltonian, the second-order perturbative correction to ground-state energies is decomposed and the scaling properties of the underlying tensor network are discussed. The employed tensor formats are found to lead to an efficient data compression of two-body matrix elements of the nuclear Hamiltonian. In particular, the sophisticated tensor hypercontraction scheme yields low tensor ranks with respect to both harmonic-oscillator and Hartree-Fock single-particle bases. It is found that the tensor rank depends on the two-body total angular momentum J for which one performs the decomposition, which is itself directly related to the sparsity of the corresponding tensor. Furthermore, including normal-ordered two-body contributions originating from three-body interactions does not compromise the efficient data compression. Ultimately, the use of factorized matrix elements authorizes controlled approximations of the exact second-order ground-state energy corrections. In particular, a small enough error is obtained from low-rank factorizations in He4,O16, and Ca40. It is presently demonstrated that tensor-decomposition techniques can be efficiently applied to systematically approximate the nuclear many-body Hamiltonian in terms of lower-rank tensors. Beyond the input Hamiltonian, tensor-decomposition techniques can be envisioned to scale down the cost of state-of-the-art nonperturbative many-body methods in order to extend ab initio studies to (i) higher precisions, (ii) larger masses, and (iii) nuclei of doubly open-shell character.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

High Performance Compression of Science Data

Two papers make up the body of this report. One presents a single-pass adaptive vector quantization algorithm that learns a codebook of variable size and shape entries; the authors present experiments on a set of test images showing that with no training or prior knowledge of the data, for a given fidelity, the compression achieved typically equals or exceeds that of the JPEG standard. The second paper addresses motion compensation, one of the most effective techniques used in interframe data compression. A parallel block-matching algorithm for estimating interframe displacement of blocks with minimum error is presented. The algorithm is designed for a simple parallel architecture to process video in real time.

Storer, James A.↗

Compression behavior of dense H 2 -He mixtures up to 160 GPa

We have studied the compression behavior of H 2 -He mixtures in comparison with pure H 2 and He using powder synchrotron x-ray diffraction and present the pressure-volume (PV) compression data of H 2 -He mixtures to 160 GPa. The results indicate that both H 2 and He in H 2 -He mixtures remain in hcp to the maximum pressure studied, yet develop a substantial level of lattice distortion in the (100) plane, most profound in He-rich solids and below 66 GPa. The measured PV data also indicate softening of He (or H 2 )-rich lattice upon increasing the level of the guest H 2 (or He) concentration. We suggest that the observed softening and lattice distortion are due to a substitutional incorporation of H2 (guest) molecules into the basal plane of hcp-He (host) lattice and, thereby, reflect the miscibility between H 2 and He in H 2 -He mixtures. Interestingly, solid He exhibits a lesser degree of preferred orientation in H2-He mixtures than in pure He, likely due to the presence of solid H 2 disturbing the crystalline ordering of He-rich solids. Finally, the present PV compression data of H 2 -rich and He-rich solids to 160 GPa deviate from those of pure H 2 and pure He above ~70 and 45 GPa respectively, providing new constraints for development of the EOS for H 2 -He mixtures for planetary models.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

High performance compression of science data

Two papers make up the body of this report. One presents a single-pass adaptive vector quantization algorithm that learns a codebook of variable size and shape entries; the authors present experiments on a set of test images showing that with no training or prior knowledge of the data, for a given fidelity, the compression achieved typically equals or exceeds that of the JPEG standard. The second paper addresses motion compensation, one of the most effective techniques used in the interframe data compression. A parallel block-matching algorithm for estimating interframe displacement of blocks with minimum error is presented. The algorithm is designed for a simple parallel architecture to process video in real time.

Storer, James A.↗

Compression and error correction for TV

Data compression and error correcting codes applied to digital transmission of real time, standard format TV, along with voice and other data from Apollo spacecraft

Blizard, R. B.↗

GRUMDN: A Multi-Task Model for Predicting Human Patterns-of-Life from Stay Transition Data

Understanding human patterns-of-life (PoL) is essential towards ensuring safe and secure indoor facility environment as well as outdoor urban environment. Prediction of human movement in between places of interest is vital in understanding human PoL. Movement between spaces maybe represented and detected in one of the two forms: 1) trajectories: locations measured at regular time intervals by mobile sensors, bluetooth or GPS sensors; or 2) stay transitions: semantic PoI (points of interest) and stay duration data measurable by eventbased sensors that collect data when a check-in or check-out event is detected. Stay transition data provides a more compressed data format compared to trajectories data, especially in situations with longer stay durations, while preserving the information necessary for PoL analysis. Now as introduced briefly in the paper, our deployed end application (Digital Twin of a facility with non-player characters, besides the interactive user in virtual reality) needed a well-performing and validated AI/ML model for simulating high quality stay transitions behavior. In this study we thus primarily present our findings with developing and validating that model, which is a multi-task neural network for stay transition prediction. The neural network consists of two heads, for corresponding two tasks of stay category prediction and stay duration prediction. We evaluated gated recurrent units and multi-layer perceptrons of varying network sizes for stay category prediction; while mixture density networks, noisy generator-only networks, and generative adversarial networks of varying network sizes for stay duration prediction. We have then evaluated four multi-task models, constructed by combining these specialized models, on their ability to predict stay transition data. We tested our models on datasets from two different cases: 1) a simulation-generated dataset of indoor movement within the HFIR (high flux isotope reactor) nuclear reactor facility at Oak Ridge National Laboratory (ORNL); and 2) the GeoLife human mobility dataset of outdoor urban movement available in literature. Our results indicate that GRUMDN, which combines gated recurrent units (GRU) for stay category prediction task, and mixture density networks (MDN) for stay duration prediction task, did overall outperform other multitask models and the current state-of-the-art.

Gunaratne, Chathika [ORNL] (ORCID:0000000225088745↗

HQ-Sim: High-performance State Vector Simulation of Quantum Circuits on Heterogeneous HPC Systems

Quantum circuit simulations are applied in more and more circum- stances as the quantum computing community becomes broader. It helps researchers to evaluate quantum algorithms and relieve the burden of limited quantum computing resources. However, most of the state-of-the-art quantum simulators utilize either CPU or GPU to store and calculate the state vector, which results in resource starvation. Moreover, the maximum number of qubits supported by the simulator is bounded by the memory, since the memory utilization increases exponentially with the number of qubits. In this study, we leverage Heterogeneous computing to utilize both CPU and GPU to store and update state vectors. We also integrate lossy data compression to reduce memory requirements. Specifically, we develop a heterogeneous framework that has a dynamic scheduler to fully utilize the computing resources. We apply lossy compression to chunked state vector to make the maximum number of qubits higher than the regular simulators, the compression also benefits the data movement between CPU and GPU.

Zhang, Boyuan↗