Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Data compression”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 847 records · Page 47

Selection of a Pair of Experiments to Optimally Reduce Uncertainty in Targeted Nuclear Data

We propose a novel process to select a pair of differential and integral experiments that best reduce uncertainties in targeted 239 ⁢Pu nuclear data while compressing the current nuclear data pipeline from 20 to 3 years. 239⁢ Pu nuclear data are poorly understood for neutrons in the intermediate energy range due to sparsity and uncertainty in historical experiments. New experiments targeting this range will enable better understanding of these nuclear data, but choosing the ideal experiments to conduct is challenging. Beginning with a prior distribution represented by samples of nuclear data generated from theory, generalized least squares adjustments are made to incorporate data from historical experiments. To quantify potential uncertainty reduction obtainable from a pair of candidate experiments, we compute the D-optimality criterion of the posterior covariance of intermediate energy range nuclear data compared to the equivalent covariance after additional adjustment to the pair of candidate experiments. Repeating the process for each of many candidate pairs facilitates the final selection. Results support 63⁢ Cu total cross section measurements for differential experiments and alumina and alumina/graphite configurations for integral experiments. This analysis enables choosing differential and integral experiments to be executed concurrently while shortening decision times relative to the current nuclear data pipeline.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Readout of DSN Monitor Data

DSN Monitor Data Reader is a computer program that, as its name suggests, reads file of monitor data from the Deep Space Network (DSN). The monitor data constitute information on the status and performance of tracking, telemetry, command, and pointing equipment at the DSN antennas. The DSN has recently introduced a new, more advanced monitor data format, denoted 0158-Mon, that is based on the standard formatted data unit (SFDU) and compressed header data objects (CHDO) of the Consultative Committee for Space Data Systems (CCSDS). The 0158-Mon data format is a very flexible generic format that provides for specific variable-length formats and for self-identifying parameters that obviate the proprietary NASA Communications (NASCOM) bit-packed formats of the past. The monitor data SFDUs are also encapsulated in Standard DSN Blocks and routed to DSN customers for processing at their local mission control centers. This program helps a DSN customer to read and parse the monitor data to assess the statuses of the DSN stations in support of spacecraft flight operations.

Levister, Katherine↗

Multilevel Techniques for Compression and Reduction of Scientific Data---The Unstructured Case

Previous work on multilevel techniques for compression and reduction of scientific data is extended to the case of data given on unstructured meshes in two and three dimensions. The centerpiece of the work is a decomposition algorithm which is shown to be optimal, in terms of both storage and operational complexity, applicable to unstructured grids in both two and three dimensions, and which implicitly gives a Riesz basis that can be exploited to reduce the data while maintaining rigorous bounds on the loss incurred. The flexibility of the approach is illustrated by applications to potential flow around an airfoil and the effect of compression on quantities of interest relevant to airfoil design; compression of computational simulation of a nonlinear reaction-diffusion system with special attention given to the problem of time series reduction; and, data from a simulation of magnetically confined plasma in a fusion reactor reduced so as to preserve the electric field computed from the data.

97 MATHEMATICS AND COMPUTING↗

Information content of polarimetric SAR data

The information content of the compressed Stoke's matrix data from the Airborne Synthetic Aperture Radar (AIRSAR) is examined in two ways - by measuring how each feature separates classes of terrain in an image, and by measuring how well a classifier performs with and without each feature. In this way, the features may then be ranked in order of information content (or in order of utility to the classifier). Suggestions are made regarding those variables that can be omitted in a data compression scheme or in a future simplified radar system.

Cumming, Ian G.↗

Coding For Compression Of Low-Entropy Data

Improved method of encoding digital data provides for efficient lossless compression of partially or even mostly redundant data from low-information-content source. Method of coding implemented in relatively simple, high-speed arithmetic and logic circuits. Also increases coding efficiency beyond that of established Huffman coding method in that average number of bits per code symbol can be less than 1, which is the lower bound for Huffman code.

Yeh, Pen-Shu↗

MER Telemetry Processor

MERTELEMPROC processes telemetered data in data product format and generates Experiment Data Records (EDRs) for many instruments (HAZCAM, NAVCAM, PANCAM, microscopic imager, Moessbauer spectrometer, APXS, RAT, and EDLCAM) on the Mars Exploration Rover (MER). If the data is compressed, then MERTELEMPROC decompresses the data with an appropriate decompression algorithm. There are two compression algorithms (ICER and LOCO) used in MER. This program fulfills a MER specific need to generate Level 1 products within a 60-second time requirement. EDRs generated by this program are used by merinverter, marscahv, marsrad, and marsjplstereo to generate higher-level products for the mission operations. MERTELEPROC was the first GDS program to process the data product. Metadata of the data product is in XML format. The software allows user-configurable input parameters, per-product processing (not streambased processing), and fail-over is allowed if the leading image header is corrupted. It is used within the MER automated pipeline. MERTELEMPROC is part of the OPGS (Operational Product Generation Subsystem) automated pipeline, which analyzes images returned by in situ spacecraft and creates level 1 products to assist in operations, science, and outreach.

Lee, Hyun H.↗

Multiresolution With Super-Compact Wavelets

The solution data computed from large scale simulations are sometimes too big for main memory, for local disks, and possibly even for a remote storage disk, creating tremendous processing time as well as technical difficulties in analyzing the data. The excessive storage demands a corresponding huge penalty in I/O time, rendering time and transmission time between different computer systems. In this paper, a multiresolution scheme is proposed to compress field simulation or experimental data without much loss of important information in the representation. Originally, the wavelet based multiresolution scheme was introduced in image processing, for the purposes of data compression and feature extraction. Unlike photographic image data which has rather simple settings, computational field simulation data needs more careful treatment in applying the multiresolution technique. While the image data sits on a regular spaced grid, the simulation data usually resides on a structured curvilinear grid or unstructured grid. In addition to the irregularity in grid spacing, the other difficulty is that the solutions consist of vectors instead of scalar values. The data characteristics demand more restrictive conditions. In general, the photographic images have very little inherent smoothness with discontinuities almost everywhere. On the other hand, the numerical solutions have smoothness almost everywhere and discontinuities in local areas (shock, vortices, and shear layers). The wavelet bases should be amenable to the solution of the problem at hand and applicable to constraints such as numerical accuracy and boundary conditions. In choosing a suitable wavelet basis for simulation data among a variety of wavelet families, the supercompact wavelets designed by Beam and Warming provide one of the most effective multiresolution schemes. Supercompact multi-wavelets retain the compactness of Haar wavelets, are piecewise polynomial and orthogonal, and can have arbitrary order of approximation. The advantages of the multiresolution algorithm are that no special treatment is required at the boundaries of the interval, and that the application to functions which are only piecewise continuous (internal boundaries) can be efficiently implemented. In this presentation, Beam's supercompact wavelets are generalized to higher dimensions using multidimensional scaling and wavelet functions rather than alternating the directions as in the 1D version. As a demonstration of actual 3D data compression, supercompact wavelet transforms are applied to a 3D data set for wing tip vortex flow solutions (2.5 million grid points). It is shown that high data compression ratio can be achieved (around 50:1 ratio) in both vector and scalar data set.

Lee, Dohyung↗

Poly-Pattern Compressive Segmentation of ASTER Data for GIS

Pattern-based segmentation of multi-band image data, such as ASTER, produces one-byte and two-byte approximate compressions. This is a dual segmentation consisting of nested coarser and finer level pattern mappings called poly-patterns. The coarser A-level version is structured for direct incorporation into geographic information systems in the manner of a raster map. GIs renderings of this A-level approximation are called pattern pictures which have the appearance of color enhanced images. The two-byte version consisting of thousands of B-level segments provides a capability for approximate restoration of the multi-band data in selected areas or entire scenes. Poly-patterns are especially useful for purposes of change detection and landscape analysis at multiple scales. The primary author has implemented the segmentation methodology in a public domain software suite.

Myers, Wayne↗

Progressive Tree-Based Compression of Large-Scale Particle Data

Scientific simulations and observations using particles have been creating large datasets that require effective and efficient data reduction to store, transfer, and analyze. However, current approaches either compress only small data well while being inefficient for large data, or handle large data but with insufficient compression. Toward effective and scalable compression/decompression of particle positions, we introduce new kinds of particle hierarchies and corresponding traversal orders that quickly reduce reconstruction error while being fast and low in memory footprint. Our solution to compression of large-scale particle data is a flexible block-based hierarchy that supports progressive, random-access, and error-driven decoding, where error estimation heuristics can be supplied by the user. For low-level node encoding, we introduce new schemes that effectively compress both uniform and densely structured particle distributions. Our proposed methods thus target all three phases of a tree-based particle compression pipeline, namely tree construction, tree traversal, and node encoding. In conclusion, the improved efficacy and flexibility of these methods over existing compressors are demonstrated through extensive experimentation, using a wide range of scientific particle datasets.

97 MATHEMATICS AND COMPUTING↗

HPDR: High-Performance Portable Scientific Data Reduction Framework

The rapid growth in scientific data generation is outpacing advancements in computing systems necessary for efficient storage, transfer, and analysis, particularly in the context of exascale computing. With the deployment of first-generation exascale computing systems and next-generation experimental facilities, this gap is widening and necessitates effective data reduction techniques to manage enormous data volumes. Over the past decade, various data reduction methods, including lossless compression, error-controlled lossy compression, and data refactoring, have been developed to accelerate I/O in scientific workflows. Despite significant reductions in data volume, these methods introduce considerable computational overhead, which can become the new bottleneck in data processing. To mitigate this, GPU-accelerated data reduction algorithms have been introduced. However, challenges remain in their integration into exascale workflows, including limited portability across different GPU architectures, substantial memory transfer overhead, and reduced scalability on dense multi-GPU systems. To address these challenges, we propose HPDR, a high-performance and portable data reduction framework. HPDR is designed to enable the execution of state-of-the-art reduction algorithms across diverse processor architectures while reducing memory transfer overhead to 2.3 % of the original, resulting in up to 3.5× faster throughput compared to existing solutions. It also achieves up to 96% of the theoretical speedup in multi-GPU settings. In addition, evaluations on accelerating I/O operations at scale up to 1,024 nodes of the Frontier supercomputer demonstrate that HPDR can achieve up to 103 TB/s reduction throughput, providing up to 4× acceleration in parallel I/O performance compared to existing data reduction routines. This work highlights the potential of HPDR to significantly enhance data reduction efficiency in exascale computing environments.

Chen, Jieyang [University of Oregon]↗

Comparison of Estimated with Measured Maximum Instantaneous Distortion Using Flight Data from an Axisymmetric Mixed Compression Inlet

YF-12C flight-measured inlet dynamic distortion data are compared with predictions made on the basis of the method reported by Melick et al. (1976). The YF-12C aircraft is a twin engine aircraft capable of speeds above 3. The inlets have a translating spike to control the inlet throat area. A bypass system is used to control the terminal shock of the inlet for operation in the mixed compression mode. The dynamic data were obtained with the aid of 24 high frequency response total pressure sensors. The model of Melick et al. is discussed along with the computer program used to implement the model. It is found that the predictions of maximum instantaneous distortion are within 20 percent of the measured values, which had been obtained at Mach numbers of 1.8, 2.1, 2.5, and 3.0.

Brilliant, H. M.↗

Band-Moment Compression of AVIRIS Hyperspectral Data and its Use in the Detection of Vegetation Stress

A remote sensing campaign was conducted over a U.S. Department of Agriculture test farm at Shelton, Nebraska. An experimental field was set off in plots that were differentially treated with anhydrous ammonia. Four replicates of 0-kg/ha to 200-kg/ha plots, in 50-kg/ha increments, were set out in a random block design. Low-altitude (GSD of 3 m) Airborne Visible/Infrared Imaging Spectrometer (AVIRIS) hyperspectral data were collected over the site in 224 bands. Simultaneously, ground data were collected to support the airborne imagery. In an effort to reduce data load while maintaining or enhancing algorithm performance for vegetation stress detection, band-moment compression and analysis was applied to the AVIRIS image cube. The results indicated that band-moment techniques compress the AVIRIS dataset significantly while retaining the capability of detecting environmentally induced vegetation stress.

Estep, L.↗

Orbit Determination of LEO Satellites for a Single Pass through a Radar: Comparison of Methods

The problem of determining the orbit of a space object from measurements based on one pass through the field of view of a radar is not a new one. Extensive research in this area has been carried out in the USA and Russia since the late 50s when these countries started the development of ballistic missile defense (BMD) and Early Warning systems. In Russia these investigations got additional stimulation in the early 60s after the decision to create a Space Surveillance System, whose primary task would be the maintenance of the satellite catalog. These problems were the focus of research interest until the middle 70s when the appropriate techniques and software were implemented for all radars. Then for more than 20 years no new research papers appeared on this subject. This produced an impression that all the problems of track determination based on one pass had been solved and there was no need for further research. In the late 90s interest in this problem arose again in relation to the following. It was estimated that there would be greater than 100,000 objects with size greater than 1-2 cm and collision of an operational spacecraft with any of these objects could have catastrophic results. Thus, for prevention of hazardous approaches and collisions with valuable spacecraft the existing satellite catalog should be extended by at least an order of magnitude This is a very difficult scientific and engineering task. One of the issues is the development of data fusion procedures and the software capable of maintaining such a huge catalog in near real time. The number of daily processed measurements (of all types, radar and optical) for such a system may constitute millions, thus increasing the number of measurements by at least an order of magnitude. Since we will have ten times more satellites and measurements the computer effort required for the correlation of measurements will be two orders of magnitude greater. This could create significant problems for processing data close to real time even for modern computers. Preliminary "compression" of data for one pass through the field of view of a sensor can significantly reduce the requirements to computers and data communication. This compression will occur when all the single measurements of the sensor are replaced by the orbit determined on their basis. The single measurement here means the radar parameters (range, azimuth, elevation, and in some cases range rate) measured by a single pulse.

Khutorovsky, Z.↗

A strategy for compression and analysis of massive geophysical data sets

This paper describes a method for summaraizing data in a way that approximately preserves high-resolution data structure while reducing data volume and maintaining global integrity of very large, remote sensing data sets. The method is under development for one of Terra's instruments, the Multi-angle Imaging SpectroRadiometer (MISR).

geophysical data sets compression↗