Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Data Compression”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 703 records · Page 39

QProR: An Efficient Framework for Quantity-of-Interest Based Progressive Retrieval with Guaranteed Error Control

Scientific applications generate an unprecedented volume of data, overwhelming the network and file systems’ bandwidth and posing challenges for efficient and scalable data retrieval and analysis. Progressive data compression offers a promising solution by enabling on-demand retrieval at reduced size. However, existing progressive methods either fail to bound the errors in essential quantities of interest (QoIs) derived from raw data or suffer from suboptimal retrieval efficiency. In this work, we propose QProR, an efficient QoI-based progressive framework that optimizes progressive retrieval for target QoIs. Our key contributions include: (1) a systematic framework that integrates error-controlled lossy compressors with bitplane encoding while decoupling the two processes for high flexibility and adaptability; (2) a novel weighted bitplane encoding method which incorperates QoI knowledge into data refactoring to enhance retrieval efficiency; (3) an optimized retrieval strategy that accounts for the varying impacts of different variables on multivariate QoIs; (4) comprehensive evaluations using six real-world datasets from multiple scientific applications and thorough comparisons against state of the arts. Experimental results demonstrate that QProR achieves up to 80.38% reduction in the retrieval size under the same requested QoI error tolerance, when compared with the best-performing existing methods. When transferring 384 GB of scientific data to remote sites, QProR delivers up to 1.68 × speedup in the end-to-end data transfer performance.

Li, Wenbo [University of Kentucky]↗

Information preserving coding for multispectral data

A general formulation of the data compression system is presented. A method of instantaneous expansion of quantization levels by reserving two codewords in the codebook to perform a folding over in quantization is implemented for error free coding of data with incomplete knowledge of the probability density function. Results for simple DPCM with folding and an adaptive transform coding technique followed by a DPCM technique are compared using ERTS-1 data.

Duan, J. R.↗

Source encoding for orbiter communications links

The feasibility of using data compression to improve link efficiency as an alternative to increased transmitter power, reducing receiver noise figures, increasing antenna gain through more stringent orbiter attitude constraints, etc. is studied. A method of encoding digital data is developed which permits low band-width encoding as well as a unique system of adaptive run length encoding. The effectiveness of these techniques for the air-to-ground link and for the bandwidth-limited ground-to-ground data link used for the orbiter downlink data is evaluated. Results are presented.

Source record↗

Image recovery from edge primitives

A method for extracting edge primitives from Mach-band patterns is presented together with a method for recovering image representations of features outlined by the edge boundaries. The accuracy, stability, and resolution of these representations are assessed. Since these representations are most commonly used in characterizing targets, this method of low-level processing offers new opportunities for computer vision and high data-compressing coding. Two bandpass filters are considered, the spatially invariant Laplacian of Gaussian filter and spatially variant intensity-dependent spatial (IDS) summation. It is shown that the recovery from the IDS bandpass data is particularly advantageous in applications for which robustness to local and temporal variations in illumination is important. It is concluded that the edge primitives extracted from bandpassed images can be an efficient way to store, transmit, and represent images.

Alter-Gartenberg, Rachel↗

SAMPA Based Streaming Readout Data Acquisition Prototype

We have assembled a small-scale streaming data acquisition system based on the SAMPA front-end ASIC. The 32-channel SAMPA chip was designed for the high-luminosity upgrade of the ALICE Time Projection Chamber (TPC) detector at the CERN Large Hadron Collider. The goals of the prototype system are to determine if the SAMPA chip is appropriate for use in detector systems at Jefferson Lab, and to gain experience with the hardware and software required to deploy streaming data acquisition systems in nuclear physics experiments. The 800 channel system is composed of components used in the ALICE TPC data acquisition upgrade. Five front-end cards (FEC) support five SAMPA chips each. SAMPA data streams on an FEC are concentrated into two high-speed (4.48 Gb/s) serial data streams by a pair Gigabit Transceiver ASICs (GBTx). These ten streams (44.8 Gb/s) are transmitted from the FECs over fibers to a PCIe based readout unit. The FPGA engine on the readout unit compresses data for transmission to a server via 100 Gb ethernet. Components on the FECs are radiation tolerant. High data rates can be handled. The system is by design scalable and thus provides a functional prototype for high-rate streaming readout at Jefferson Lab and the future Electron Ion Collider. We have made fundamental measurements (noise, linearity, time resolution) on the SAMPA ASIC. We have also coupled the readout system to a small Gas Electron Multiplier (GEM) detector and have studied its response to cosmic rays. A beam test of the system is planned.

ABBOTT, David↗

Digital signal processor and processing method for GPS receivers

A digital signal processor and processing method therefor for use in receivers of the NAVSTAR/GLOBAL POSITIONING SYSTEM (GPS) employs a digital carrier down-converter, digital code correlator and digital tracking processor. The digital carrier down-converter and code correlator consists of an all-digital, minimum bit implementation that utilizes digital chip and phase advancers, providing exceptional control and accuracy in feedback phase and in feedback delay. Roundoff and commensurability errors can be reduced to extremely small values (e.g., less than 100 nanochips and 100 nanocycles roundoff errors and 0.1 millichip and 1 millicycle commensurability errors). The digital tracking processor bases the fast feedback for phase and for group delay in the C/A, P.sub.1, and P.sub.2 channels on the L.sub.1 C/A carrier phase thereby maintaining lock at lower signal-to-noise ratios, reducing errors in feedback delays, reducing the frequency of cycle slips and in some cases obviating the need for quadrature processing in the P channels. Simple and reliable methods are employed for data bit synchronization, data bit removal and cycle counting. Improved precision in averaged output delay values is provided by carrier-aided data-compression techniques. The signal processor employs purely digital operations in the sense that exactly the same carrier phase and group delay measurements are obtained, to the last decimal place, every time the same sampled data (i.e., exactly the same bits) are processed.

Thomas, Jr., Jess B.↗

STZ: A High Quality and High Speed Streaming Lossy Compression Framework for Scientific Data

Error-bounded lossy compression is one of the most efficient solutions to reduce the volume of scientific data. For lossy compression, progressive decompression and random-access decompression are critical features that enable on-demand data access and flexible analysis workflows. However, these features can severely degrade compression quality and speed. To address these limitations, we propose a novel streaming compression framework that supports both progressive decompression and random-access decompression while maintaining high compression quality and speed. Our contributions are three-fold: (1) we design the first compression framework that simultaneously enables both progressive decompression and random-access decompression; (2) we introduce a hierarchical partitioning strategy to enable both streaming features, along with a hierarchical prediction mechanism that mitigates the impact of partitioning and achieves high compression quality—even comparable to state-of-the-art (SOTA) non-streaming compressor SZ3; and (3) our framework delivers high compression and decompression speed, up to 6.7 × faster than SZ3.

Wang, Daoce [University of Nebraska, Omaha]↗

Developing a radiation-aging model for polymeric foam cushions

Motivation: A major component of the A&L constitutive modeling effort over the last couple of years has involved developing a thermal aging model for foam cushions. Through extensive studies of uniaxial compression data, we have established that the Ogden N = 2 Hyperfoam in conjunction with the Tobolsky two-network model [Ref. 4] provides an accurate theoretical framework to describe the results, both for the traditional (stochastic) as well as the AM (3D-printed) foam pads. The results display numerical accuracy over a wide range of compression levels, from low to moderate up to slightly over 90% of lock-up. Over the last few weeks we have been exploring radiation aging data on a specific 3D printed foam, i.e., SE1700 AM FCT foam, to see if the Ogden + Tobolsky model can explain such data as well, and if so, how the various model parameters change as a function of dosage. As this report shows, the answer is indeed yes. We find that the modulus of the second (i.e., induced network) increases nearly proportionally with the radiation dosage, with a smaller sub-linear decrease in the primary shear modulus of the original network. This is consistent with the interpretation that exposure to energetic gamma-radiation is dominated by the formation of new cross-links. Significance: An important goal of A&L modeling efforts is to develop a fully age-aware materials model that can describe the joint effects of commonly present aging drivers like temperature, radiation, humidity, etc. Such a model is necessary to assess the lifetime of essential components like polymeric foams and rubber pads.

36 MATERIALS SCIENCE↗

Reconfigurable Hardware for Compressing Hyperspectral Image Data

High-speed, low-power, reconfigurable electronic hardware has been developed to implement ICER-3D, an algorithm for compressing hyperspectral-image data. The algorithm and parts thereof have been the topics of several NASA Tech Briefs articles, including Context Modeler for Wavelet Compression of Hyperspectral Images (NPO-43239) and ICER-3D Hyperspectral Image Compression Software (NPO-43238), which appear elsewhere in this issue of NASA Tech Briefs. As described in more detail in those articles, the algorithm includes three main subalgorithms: one for computing wavelet transforms, one for context modeling, and one for entropy encoding. For the purpose of designing the hardware, these subalgorithms are treated as modules to be implemented efficiently in field-programmable gate arrays (FPGAs). The design takes advantage of industry- standard, commercially available FPGAs. The implementation targets the Xilinx Virtex II pro architecture, which has embedded PowerPC processor cores with flexible on-chip bus architecture. It incorporates an efficient parallel and pipelined architecture to compress the three-dimensional image data. The design provides for internal buffering to minimize intensive input/output operations while making efficient use of offchip memory. The design is scalable in that the subalgorithms are implemented as independent hardware modules that can be combined in parallel to increase throughput. The on-chip processor manages the overall operation of the compression system, including execution of the top-level control functions as well as scheduling, initiating, and monitoring processes. The design prototype has been demonstrated to be capable of compressing hyperspectral data at a rate of 4.5 megasamples per second at a conservative clock frequency of 50 MHz, with a potential for substantially greater throughput at a higher clock frequency. The power consumption of the prototype is less than 6.5 W. The reconfigurability (by means of reprogramming) of the FPGAs makes it possible to effectively alter the design to some extent to satisfy different requirements without adding hardware. The implementation could be easily propagated to future FPGA generations and/or to custom application-specific integrated circuits.

Aranki, Nazeeh↗

Alternate physical formats for storing data in HDF

Since its inception HDF has evolved to meet new demands by the scientific community to support new kinds of data and data structures, larger data sets, and larger numbers of data sets. The first generation of HDF supported simple objects and simple storage schemes. These objects were used to build more complex objects such as raster images and scientific data sets. The second generation of HDF provided alternate methods of storing data elements, making it possible to do such things as store extendible objects with in HDF, to store data externally from HDF files, and support data compression effectively. As we look to the next generation of HDF, we are considering fundamental changes to HDF, including a redefinition of the basic HDF object from a simple object to a more general, higher-level scientific data object that has certain characteristics, such as dimensionality, a more general atomic number type, and attributes. These changes suggest corresponding changes to the HDF file format itself.

Folk, Mike↗

A comparison between ShapeFit compression and Full-Modelling method with PyBird for DESI 2024 and beyond

DESI aims to provide one of the tightest constraints on cosmological parameters by analysing the clustering of more than thirty million galaxies. However, obtaining such constraints requires special care in validating the methodology and efforts to reduce the computational time required through data compression and emulation techniques. In this work, we perform a rigorous validation of the PyBird power spectrum modelling code with both a traditional emulated Full-Modelling approach and the model-independent ShapeFit compression approach. By using cubic box simulations that accurately reproduce the clustering and precision of the DESI survey, we find that the cosmological constraints from ShapeFit and Full-Modelling are consistent with each other at the ∼ 0.5σ level for the ΛCDM model. Both ShapeFit and Full-Modelling are also consistent with the true ΛCDM simulation cosmology down to a scale of k max = 0.20 hMpc -1 even after including the hexadecapole. For extended models such as the wCDM and the oCDM models, we find that including the hexadecapole can significantly improve the constraints and reduce the modelling errors with the same k max . While their discrepancies between the constraints from ShapeFit and Full-Modelling are more significant than ΛCDM, they remain consistent within 0.7σ. Lastly, we also show that the constraints on cosmological parameters with the correlation function evaluated from PyBird down to s min = 30h -1 Mpc are unbiased and consistent with the constraints from the power spectrum.

79 ASTRONOMY AND ASTROPHYSICS↗

Data transmission networks

A task order was written by the High Resolution, High Frame Rate Video Technology (HHVT) project engineers to investigate data compression techniques that could be applied to the HHVT system, and both existing and planned downlink/uplink capabilities of the Space Shuttle and Space Station Freedom. The following tasks were included: (1) Investigate signal channel availability and determine both the maximum possible data rate and the average data rate; (2) Identify time blocks for HHVT video transmission assuming time sharing and interruptions in the communication links; (3) Determine the bit error rates to be expected; and (4) Define the transmit and receive interfaces. A summary chart of the data transmission capabilities for Tracking and Data Relay Satellite System (TDRSS), the Space Shuttle, Space Station Freedom, Spacelab, and USLab are also presented.

Alexovich, Robert↗

TRMM .25 deg x .25 deg Gridded Precipitation Text Product

Since the launch of the Tropical Rainfall Measuring Mission (TRMM), the Precipitation Measurement Missions science team has endeavored to provide TRMM precipitation retrievals in a variety of formats that are more easily usable by the broad science community than the standard Hierarchical Data Format (HDF) in which TRMM data is produced and archived. At the request of users, the Precipitation Processing System (PPS) has developed a .25 x .25 gridded product in an easily used ASCII text format. The entire TRMM mission data has been made available in this format. The paper provides the details of this new precipitation product that is designated with the TRMM designator 3G68.25. The format is packaged into daily files. It provides hourly precipitation information from the TRMM microwave imager (TMI), precipitation radar (PR), and TMI/PR combined rain retrievals. A major advantage of this approach is the inclusion only of rain data, compression when a particular grid has no rain from the PR or combined, and its direct ASCII text format. For those interested only in rain retrievals and whether rain is convection or stratiform, these products provide a huge reduction in the data volume inherent in the standard TRMM products. This paper provides examples of the 3G68 data products and their uses. It also provides information about C tools that can be used to aggregate daily files into larger time samples. In addition, it describes the possibilities inherent in the spatial sampling which allows resampling into coarser spatial sampling. The paper concludes with information about downloading the gridded text data products.

Stocker, Erich↗

A user's guide for the signal processing software for image and speech compression developed in the Communications and Signal Processing Laboratory (CSPL), version 1

A complete documentation of the software developed in the Communication and Signal Processing Laboratory (CSPL) during the period of July 1985 to March 1986 is provided. Utility programs and subroutines that were developed for a user-friendly image and speech processing environment are described. Additional programs for data compression of image and speech type signals are included. Also, programs for the zero-memory and block transform quantization in the presence of channel noise are described. Finally, several routines for simulating the perfromance of image compression algorithms are included.

Kumar, P.↗

Reduced order models for thermal radiative transfer problems based on moment equations and data-driven approximations of the Eddington tensor

Here a new group of structure and asymptotic preserving reduced-order models (ROMs) for multidimensional nonlinear thermal radiative transfer (TRT) problems is presented. They are formulated by means of the nonlinear projective approach and data compression techniques. The nonlinear projection is applied to the Boltzmann transport equation (BTE) to derive a hierarchy of low-order moment equations. Approximation of the Eddington tensor that provides exact closure for the system of moment equations is found with projection-based data-driven methodologies. These include the (i) proper orthogonal decomposition (POD), (ii) dynamic mode decomposition (DMD) and (iii) a variant of the DMD. A parameterization is derived for this ROM for the temperature of radiation incoming to the problem domain (the radiation drive temperature). This parameterization is informed from results of a dimensionless study of the TRT problem. Analysis of the ROMs is performed on the classical Fleck-Cummings TRT multigroup test problem in 2D geometry with a radiation-driven Marshak wave. Numerical results are presented to demonstrate the performance of these ROMs for the simulation of evolving radiation and heat waves. Results show these models to be sufficiently accurate for practical computations with rather low-rank representations of the Eddington tensor. As the rank of the approximation is increased, the errors of solutions generated by the ROMs gradually decreases.

42 ENGINEERING↗

Isentropic compression studies using the NHMFL single turn

Magnetic isentropic compression experiments (ICE) provide the most accurate shock free compression data for materials at megabar stresses. Recent ICE experiments performed on the Sandia Z-machine (Asay, 1999) and at the Los Alamos High Explosive Pulsed Power facility (Tasker, 2006) are providing our nation with data on material properties in extreme dynamic high stress environments. The LANL National High Magnetic Field Laboratory (NHMFL) can offer a less complex ICE experiment at high stresses (up to {approx}1Mbar) with a high sample throughput and relatively low cost. This is not to say that the NHMFL technique will replace the other methods but rather complement them. For example, NHMFL-ICE is ideal for the development of advanced diagnostics, e.g., to detect phase changes. We will discuss the physics of the NHMFL-ICE experiments and present data from the first proof-of-principle experiments that were performed in September 2010.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Archival of Poker Flat MST radar data

The Poker Flat MST (mesosphere stratosphere troposphere) radar has operated from early 1979 to 1985. The data recorded during that time resides on some 1100 magnetic tapes. A second set containing only the derived parameters of velocity, width and signal to noise of the primary echo at each height, plus the noise spectra, occupies another 250 tapes. While processing to generate the compressed data set does correct some known errors, no attempt has been made to identify or remove spurious echoes. Because the Poker Flat data set is such a unique and valuable resource, a proposal is made to archive the data in forms more useful for analysis. The archived data set would contain only the parameters for significant echoes with contamination from airplanes, unwanted ionospheric returns, frequently aliased Doppler signals and other sources removed. The archived set should be as good or better than the example shown and may occupy only 25 to 50 tapes.

Riddle, A. C.↗

ICASE/LaRC Symposium on Visualizing Time-Varying Data

Time-varying datasets present difficult problems for both analysis and visualization. For example, the data may be terabytes in size, distributed across mass storage systems at several sites, with time scales ranging from femtoseconds to eons. In response to these challenges, ICASE and NASA Langley Research Center, in cooperation with ACM SIGGRAPH, organized the first symposium on visualizing time-varying data. The purpose was to bring the producers of time-varying data together with visualization specialists to assess open issues in the field, present new solutions, and encourage collaborative problem-solving. These proceedings contain the peer-reviewed papers which were presented at the symposium. They cover a broad range of topics, from methods for modeling and compressing data to systems for visualizing CFD simulations and World Wide Web traffic. Because the subject matter is inherently dynamic, a paper proceedings cannot adequately convey all aspects of the work. The accompanying video proceedings provide additional context for several of the papers.

Banks, D. C.↗