Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Data compression”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 613 records · Page 34
All sourcesNASA NTRSDOE OSTI
NASA NTRS1986

A user's guide for the signal processing software for image and speech compression developed in the Communications and Signal Processing Laboratory (CSPL), version 1

A complete documentation of the software developed in the Communication and Signal Processing Laboratory (CSPL) during the period of July 1985 to March 1986 is provided. Utility programs and subroutines that were developed for a user-friendly image and speech processing environment are described. Additional programs for data compression of image and speech type signals are included. Also, programs for the zero-memory and block transform quantization in the presence of channel noise are described. Finally, several routines for simulating the perfromance of image compression algorithms are included.

Kumar, P.↗
DOE OSTI2022

Reduced order models for thermal radiative transfer problems based on moment equations and data-driven approximations of the Eddington tensor

Here a new group of structure and asymptotic preserving reduced-order models (ROMs) for multidimensional nonlinear thermal radiative transfer (TRT) problems is presented. They are formulated by means of the nonlinear projective approach and data compression techniques. The nonlinear projection is applied to the Boltzmann transport equation (BTE) to derive a hierarchy of low-order moment equations. Approximation of the Eddington tensor that provides exact closure for the system of moment equations is found with projection-based data-driven methodologies. These include the (i) proper orthogonal decomposition (POD), (ii) dynamic mode decomposition (DMD) and (iii) a variant of the DMD. A parameterization is derived for this ROM for the temperature of radiation incoming to the problem domain (the radiation drive temperature). This parameterization is informed from results of a dimensionless study of the TRT problem. Analysis of the ROMs is performed on the classical Fleck-Cummings TRT multigroup test problem in 2D geometry with a radiation-driven Marshak wave. Numerical results are presented to demonstrate the performance of these ROMs for the simulation of evolving radiation and heat waves. Results show these models to be sufficiently accurate for practical computations with rather low-rank representations of the Eddington tensor. As the rank of the approximation is increased, the errors of solutions generated by the ROMs gradually decreases.

42 ENGINEERING↗
NASA NTRS1986

Archival of Poker Flat MST radar data

The Poker Flat MST (mesosphere stratosphere troposphere) radar has operated from early 1979 to 1985. The data recorded during that time resides on some 1100 magnetic tapes. A second set containing only the derived parameters of velocity, width and signal to noise of the primary echo at each height, plus the noise spectra, occupies another 250 tapes. While processing to generate the compressed data set does correct some known errors, no attempt has been made to identify or remove spurious echoes. Because the Poker Flat data set is such a unique and valuable resource, a proposal is made to archive the data in forms more useful for analysis. The archived data set would contain only the parameters for significant echoes with contamination from airplanes, unwanted ionospheric returns, frequently aliased Doppler signals and other sources removed. The archived set should be as good or better than the example shown and may occupy only 25 to 50 tapes.

Riddle, A. C.↗
NASA NTRS1996

ICASE/LaRC Symposium on Visualizing Time-Varying Data

Time-varying datasets present difficult problems for both analysis and visualization. For example, the data may be terabytes in size, distributed across mass storage systems at several sites, with time scales ranging from femtoseconds to eons. In response to these challenges, ICASE and NASA Langley Research Center, in cooperation with ACM SIGGRAPH, organized the first symposium on visualizing time-varying data. The purpose was to bring the producers of time-varying data together with visualization specialists to assess open issues in the field, present new solutions, and encourage collaborative problem-solving. These proceedings contain the peer-reviewed papers which were presented at the symposium. They cover a broad range of topics, from methods for modeling and compressing data to systems for visualizing CFD simulations and World Wide Web traffic. Because the subject matter is inherently dynamic, a paper proceedings cannot adequately convey all aspects of the work. The accompanying video proceedings provide additional context for several of the papers.

Banks, D. C.↗
DOE OSTI2021

A Preprocessing Tool for Enhanced Ion Mobility–Mass Spectrometry-Based Omics Workflows

The ability to improve the data quality of ion mobility–mass spectrometry (IM-MS) measurements is of great importance for enabling modular and efficient computational workflows and gaining better qualitative and quantitative insights from complex biological and environmental samples. We developed the PNNL PreProcessor, a standalone and user-friendly software housing various algorithmic implementations to generate new MS-files with enhanced signal quality and in the same instrument format. Different experimental approaches are supported for IM-MS based on Drift-Tube (DT) and Structures for Lossless Ion Manipulations (SLIM), including liquid chromatography (LC) and infusion analyses. The algorithms extend the dynamic range of the detection system, while reducing file sizes for faster and memory-efficient downstream processing. Specifically, multidimensional smoothing improves peak shapes of poorly defined low-abundance signals, and saturation repair reconstructs the intensity profile of high-abundance peaks from various analyte types. Further, other functionalities are data compression and interpolation, IM demultiplexing, noise filtering by low intensity threshold and spike removal, and exporting of acquisition metadata. Several advantages of the tool are illustrated, including an increase of 19.4% in lipid annotations and a two-times faster processing of LC-DT IM-MS data-independent acquisition spectra from a complex lipid extract of a standard human plasma sample. The software is freely available at https://omics.pnl.gov/software/pnnl-preprocessor.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2021

LANL-UW Collaboration on LC-QTOF Datasets (Annual Report)

Prior to the application of non-targeted chemometric techniques (i.e., Fisher ratio analysis), pre-processing steps must be carried out on the data collected using liquid chromatography-quadrupole time of flight (LC-QTOF) mass spectrometry. Data compression is the most important pre-processing step due to the vast number of high-resolution mass channels (m/z; 0.0001 Da) collected. Our first approach was to bin the mass spectral dimension to 0.01 Da; however, this approach was computationally expensive and did not preserve the high-resolution data.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2021

Real-time data reduction at 100 Tbps: Challenge and opportunity for AI-based data reduction for next-generation large-scale nuclear physics collider experiment

The modern large-scale nuclear physics (NP) experiments in high-energy particle colliders utilize streaming-readout electronics to digitize detector response at O(100) Tbps bandwidth. Prominent examples at Brookhaven National Lab (BNL) include the sPHENIX experiment at Relativistic Heavy Ion Collider (RHIC), which is under construction, and the experiments proposed for the Electron-Ion Collider (EIC), planned for the 2030s . One of the main challenges for these streaming readout systems is to manage the data rate with sufficient data reduction in real time so the end-data fit persistent storage for offline analysis, which is typically at O(1000) times smaller and O(100) Gbps. Such data reduction traditionally is achieved via real-time high level triggers, which select and save a small subset of collisions of interest. Although triggering is applicable to high energy collider experiments such as those at the Large Hardron Collider at CERN, it is insufficient for these nuclear physics experiments which study diverse collision topologies. And traditional triggering approach is inefficient to preserve the max information harvested from the operation of colliders that costs O(100)M per year to DOE. Meanwhile, in recent years, ML-based high-throughput data reduction has emerged as a promising approach to efficiently preserve max information for a given space of persistent storage, e.g. via AI data compression, feature extraction, and noise filtering.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2026

Ensemble Kalman filter for data assimilation coupled with low-resolution computations techniques applied in fluid dynamics

This paper presents an innovative Reduced-order model (ROM) for merging experimental and simulation data using data assimilation (DA) to estimate the "True" state of a fluid dynamics system, leading to more accurate predictions. Our methodology introduces a novel approach by implementing the ensemble Kalman filter (EnKF) within a reduced-dimensional framework, grounded in a robust theoretical foundation and applied to fluid dynamics. To address the substantial computational demands of DA, the proposed ROM employs low-resolution (LR) techniques to drastically reduce computational costs. This innovative approach involves downsampling datasets for DA computations, followed by an advanced reconstruction technique based on low-cost singular value decomposition (lcSVD). The lcSVD method, a key innovation in this paper, has never been applied to DA before and offers a highly efficient way to enhance resolution with minimal computational resources. Our results demonstrate significant reductions in both computation time and RAM usage through these LR techniques without compromising the accuracy of the estimations. For instance, in a turbulent test case, for a data compression rate of 15.9, the LR approach can achieve a speed-up of 13.7 and a RAM compression of 90.9% while maintaining a low relative root mean square error (RRMSE) of 2.6%, compared to 0.8% in the high-resolution (HR) reference. Furthermore, we highlight the effectiveness of the EnKF in estimating and predicting the state of fluid flow systems based on limited observations and given low-fidelity numerical data. This paper highlights the potential of the proposed DA method in fluid dynamics applications, particularly for improving computational efficiency in CFD and related fields. Its ability to balance accuracy with low computational and memory costs makes it especially suitable for large-scale and real-time applications, such as environmental monitoring or engineering design. This method will be incorporated into ModelFLOWs-app.

Data Assimilation↗
NASA NTRS1990

Learning and diagnosing faults using neural networks

Neural networks have been employed for learning fault behavior from rocket engine simulator parameters and for diagnosing faults on the basis of the learned behavior. Two problems in applying neural networks to learning and diagnosing faults are (1) the complexity of the sensor data to fault mapping to be modeled by the neural network, which implies difficult and lengthy training procedures; and (2) the lack of sufficient training data to adequately represent the very large number of different types of faults which might occur. Methods are derived and tested in an architecture which addresses these two problems. First, the sensor data to fault mapping is decomposed into three simpler mappings which perform sensor data compression, hypothesis generation, and sensor fusion. Efficient training is performed for each mapping separately. Secondly, the neural network which performs sensor fusion is structured to detect new unknown faults for which training examples were not presented during training. These methods were tested on a task of fault diagnosis by employing rocket engine simulator data. Results indicate that the decomposed neural network architecture can be trained efficiently, can identify faults for which it has been trained, and can detect the occurrence of faults for which it has not been trained.

Whitehead, Bruce A.↗
NASA NTRS1981

Entropy encoding applied to a video DM bit stream

This paper investigates bandwidth compression of delta modulated video signals using a variety of novel techniques. The video signal is encoded at 16 MHz with an adaptive delta modulator (ADM) using the Song video mode algorithm. One method of achieving bandwidth compression is through the detection and reduction of steady state patterns in the DM bit stream. Further reduction is obtained by deleting the horizontal and vertical drive information. Finally, a modified ADM algorithm is introduced to increase the frequency of steady state patterns. When the modified ADM is combined with entropy coding a 30-50% data compression can be realized.

Barba, J.↗
NASA NTRS1989

Deep space communications, weather effects, and error control

Deep space telemetry is and will remain signal-to-noise limited and vulnerable to interference. A need exists to increase received signal power and decrease noise. This includes going to Ka-band in the mid-1990's to increase directivity. The effects of a wet atmosphere can increase the noise temperature by a factor of 5 or more, even at X-band, but the order of magnitude increase in average data rate obtainable at Ka-band relative to X-band makes the increased uncertainty a good trade. Lowbit error probabilities required by data compression are available both theoretically and practically with coding, at an infinitesimal power penalty rather than the 10 to 15 dB more power required to reduce error probabilities without coding. Advances are coming rapidly in coding, as with the new constraint-length 15 rate 1/4 convolutional code concatenated with the already existing Reed-Solomon code to be demonstrated on Galileo. In addition, high density spacecraft data storage will allow selective retransmissions, even from the edge of the Solar System, to overcome weather effects. In general, deep space communication was able to operate, and will continue to operate, closer to theoretical limits than any other form of communication. These include limits in antenna area and directivity, system noise temperature, coding efficiency, and everything else. The deep space communication links of the mid-90's and beyond will be compatible with new instruments and compression algorithms and represent a sensible investment in an overall end-to-end information system design.

Posner, Edward C.↗
NASA NTRS1995

Method for coding low entrophy data

A method of lossless data compression for efficient coding of an electronic signal of information sources of very low information rate is disclosed. In this method, S represents a non-negative source symbol set, (s(sub 0), s(sub 1), s(sub 2), ..., s(sub N-1)) of N symbols with s(sub i) = i. The difference between binary digital data is mapped into symbol set S. Consecutive symbols in symbol set S are then paired into a new symbol set Gamma which defines a non-negative symbol set containing the symbols (gamma(sub m)) obtained as the extension of the original symbol set S. These pairs are then mapped into a comma code which is defined as a coding scheme in which every codeword is terminated with the same comma pattern, such as a 1. This allows a direct coding and decoding of the n-bit positive integer digital data differences without the use of codebooks.

Yeh, Pen-Shu↗
NASA NTRS1985

An algorithm for computing the number of distinct spectral vectors in thematic mapper data

A computationally efficient method was developed to compute the number of distinct spectral vectors and their frequency of occurrence in Landsat-4 Thematic Mapper (TM) data. The algorithm first partitions the image into spectrally disjoint subsets and then computes the frequency distribution of distinct spectral vectors within each subset from a multidimensional histogram. The overall frequency distribution is tabulated by accumulating the results from each subset. The number of distinct spectral vectors could be used as a measure of potential storage compaction of alternate data representations for data compression, or as a measure of information content in the comparison of spectral band combinations and/or spatial resolutions for an image. Results from processing three 512 x 512 pixel Landsat-4 TM images and one Landsat-4 Multispectral Scanner (MSS) image are presented as examples. An algorithm for computing the frequency distribution of distinct spectral vectors in MSS data is given in the Appendix.

Wharton, S. W.↗
NASA NTRS1988

Identification of visual evoked response parameters sensitive to pilot mental state

Systems analysis techniques were developed and demonstrated for modeling the electroencephalographic (EEG) steady state visual evoked response (ssVER), for use in EEG data compression and as an indicator of mental workload. The study focused on steady state frequency domain stimulation and response analysis, implemented with a sum-of-sines (SOS) stimulus generator and an off-line describing function response analyzer. Three major tasks were conducted: (1) VER related systems identification material was reviewed; (2) Software for experiment control and data analysis was developed and implemented; and (3) ssVER identification and modeling was demonstrated, via a mental loading experiment. It was found that a systems approach to ssVER functional modeling can serve as the basis for eventual development of a mental workload indicator. The review showed how transient visual evoked response (tVER) and ssVER research are related at the functional level, the software development showed how systems techniques can be used for ssVER characterization, and the pilot experiment showed how a simple model can be used to capture the basic dynamic response of the ssVER, under varying loads.

Zacharias, G. L.↗
NASA NTRS1973

Image processing techniques and applications to the Earth Resources Technology Satellite program

The Earth Resources Technology Satellite system is studied, with emphasis on sensors, data processing requirements, and image data compression using the Fast Fourier and Hadamard transforms. The ERTS-A system and the fundamentals of remote sensing are discussed. Three user applications (forestry, crops, and rangelands) are selected and their spectral signatures are described. It is shown that additional sensors are needed for rangeland management. An on-board information processing system is recommended to reduce the amount of data transmitted.

Polge, R. J.↗
NASA NTRS1991

A neural net based architecture for the segmentation of mixed gray-level and binary pictures

A neural-net-based architecture is proposed to perform segmentation in real time for mixed gray-level and binary pictures. In this approach, the composite picture is divided into 16 x 16 pixel blocks, which are identified as character blocks or image blocks on the basis of a dichotomy measure computed by an adaptive 16 x 16 neural net. For compression purposes, each image block is further divided into 4 x 4 subblocks; a one-bit nonparametric quantizer is used to encode 16 x 16 character and 4 x 4 image blocks; and the binary map and quantizer levels are obtained through a neural net segmentor over each block. The efficiency of the neural segmentation in terms of computational speed, data compression, and quality of the compressed picture is demonstrated. The effect of weight quantization is also discussed. VLSI implementations of such adaptive neural nets in CMOS technology are described and simulated in real time for a maximum block size of 256 pixels.

Tabatabai, Ali↗
NASA NTRS1975

Digital command system

The correct processing of the Digital Command System (DCS), which provides a limited real-time means of controlling specific flight program functions, was verified. The ability of the flight program to correctly read and process DCS commands was verified. Tests made by the flight program after reading the contents of the command decoder register to establish the validity of the received data were verified through the following: DCS mode command verification, DCS data command verification, DCS data validation, and DCS error message. The operation of the following DCS commands accepted and processed by the flight program was tested: time base update, navigation update, generalized switch selector, memory dump, terminate, execute generalized maneuver, return to nominal timeline, ECS water control valve logic inhibit, execute maneuver, execute alternate sequence, targeting load, ladder magnitude limit, S-IVB/IU de-orbit, compressed data dump, and remove inhibit on the extraction maneuver.

Source record↗
DOE OSTI2025

Continuum shock mixture models for Ni+Al multilayers: Individual layers and bulk equations of state

Continuum shock mixture models are reviewed and applied to determine the equations of state for five different compositions of Ni x Al y ⁠, as well as bulk Ni+Al reactive multilayers, by combining the fundamental property data for elemental nickel and aluminum. From the literature, we down-select and evaluate two analytical models for the mixture Hugoniot, i.e., the well-known method of kinetic energy averaging (KEA) and a recent model proposed by Jordan and Baer [J. Appl. Phys. 111, 083516 (2012)]. Fundamentally, the former method assumes pressure equilibrium, whereas the latter assumes a common particle velocity and mixture sound speed from compressible two-phase cavitating flows. Additionally, we construct thermodynamically complete equations of state by fitting Einstein oscillator series models for the specific heat at constant volume. Finally, the solid solution approximation is invoked for intermetallic compositions, which are not strictly physical mixtures. Overall, the KEA model provides a better fit to the available Ni x Al y and Ni+Al multilayer shock compression data; however, there are combinations of material properties where the performance of these two models is thought to be reversed. Moreover, the results of this work include the first analytical solution of Jordan–Baer that does not require numerical root finding, as well as proposed modifications to the Einstein oscillator series to incorporate some effects of local pressure–temperature equilibrium and reaction–diffusion. Future work is planned that will use these equations of state in mesoscale simulations to study shock-induced reaction in Ni+Al multilayers, and the intended application is illustrated with a brief 2D hydrocode example.

36 MATERIALS SCIENCE↗