Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Data compression”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,063 records · Page 59
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Data Science and Computation for Rapid and Dynamic Compression Experiment Workflows at Experimental Facilities, September 8-11, 2020. Workshop Report

The application of high pressure to materials has enabled discoveries in scientific fields such as planetary science, materials science, and materials synthesis. Recent advances in X-ray user light sources and other facilities, co-location and integration of user facilities with high-pressure drivers, availability of high-performance computing (HPC) platforms, and the development of new data science techniques have created opportunities for, and challenges in, advancing data analytics for rapid and dynamic compression experiments. To address these challenges, harness the emerging technology now available, and expedite scientific discovery, Los Alamos National Laboratory (LANL) hosted a virtual workshop entitled “Data Science and Computation for Rapid and Dynamic Compression Workflows at Experimental Facilities” from September 8 to 11, 2020. The workshop included 95 registered scientists and analytics experts from 15 universities, 9 United States (US) national laboratories, 5 US and European X-ray light sources, neutron sources such as the Los Alamos Neutron Science Center (LANSCE), other big science facilities such as the National Ignition Facility (NIF), and an industry representative. The workshop included 31 invited talks and 4 lightning talks by students and postdocs.

36 MATERIALS SCIENCE↗
NASA NTRS1993

A study of the compressible flow through a diffusing S-duct

Benchmark aerodynamic data are presented for compressible flow through a representative S-duct configuration. A numerical prediction of the S-duct flow field, obtained from a subsonic parabolized Navier-Stokes algorithm, is also shown. The experimental and numerical results are compared. Measurements of the three-dimensional velocity field, total pressures, and static pressures were obtained at five cross-sectional planes. Aerodynamic data were gathered with calibrated pneumatic probes. Surface static pressure and surface flow visualization data were also acquired. All reported tests were conducted with an inlet centerline Mach number of 0.6. The Reynolds number, based on the inlet centerline velocity and duct inlet diameter, was 2.6 x 10(exp 6). Thin inlet turbulent boundary layers existed. The collected data should be beneficial to aircraft inlet designers and the measurements are suitable for the validation of computational codes. The results show that a region of streamwise flow separation occurred within the duct. Details about the separated flow region, including mechanisms which drive this complicated flow phenomenon, are discussed. Results also indicate that the duct curvature induces strong pressure driven secondary flows. The cross flows evolve into counter-rotating vortices. These vortices convect low momentum fluid of the boundary layer toward the center of the duct, degrading both the uniformity and magnitude of the total pressure profile.

Wellborn, Steven R.↗
DOE OSTI2022

Ephemeral Learning - Augmenting Triggers with Online-Trained Normalizing Flows

The large data rates at the LHC require an online trigger system to select relevant collisions. Rather than compressing individual events, we propose to compress an entire data set at once. We use a normalizing flow as a deep generative model to learn the probability density of the data online. The events are then represented by the generative neural network and can be inspected offline for anomalies or used for other analysis purposes. We demonstrate our new approach for a toy model and a correlation-enhanced bump hunt.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1993

On the performance of Trellis coded modulation with octal phase shift keying over the TDRSS channel

As the National Aeronautics and Space Administration moves into the 21st century with programs like Space Station Freedom, a manned mission to Mars, and the new Landsat mission, transmission demands on the Tracking and Data Relay Satellite System (TDRSS) will very likely exceed the available bandwidth. The Manual Lujan, Jr. Center for Space Telemetering and Telecommunications Systems (CSTTS) at New Mexico State University (NMSU) is studying techniques for increasing the data rate capabilities of TDRSS. These techniques include the use of advanced bandwidth efficient modulation formats to increase the data rate that can be sustained in a TDRSS transponder and the use of lossless bandwidth compression of the data to be transmitted to lower the data rate required from the user spacecraft. Based upon current technology the most promising bandwidth efficient modulation technique is Trellis Coded Modulation (TCM) operating with Octal Phase shift Keying (8PSK). Trellis Coded Modulation coding with 8PSK carrier modulation has the capability to increase the data rate which can be transmitted through the TDRSS spacecraft by a factor of 2 to 2.5 times that available with todays coded QPSK systems with only a small penalty in link performance relative to the existing systems. However, before NASA can safely employ TCM coding it is necessary to prove that this complex format can perform on the real TDRSS link as it does in labs and simulation studies. This proof-of-concept test over a live satellite channel was the objective of the construction and testing performed under this task of the NMSU NASA grant referenced above. In conjunction with NASA, NMSU's CSTTS has constructed a system to test a new candidate TDRSS modulation scheme, TCM 8PSK, that can enhance the information throughput of the TDRSS spacecraft. The test system for this project which was constructed over a period of 18 months by NMSU consisted of two racks of commercial and univeristy-designed and -built equipment. This project has included modifications of an existing White Sands Ground Terminal (WSGT) High Rate QPSK Demodulator to demodulate 8PSK as well as the construction of other support hardware. Also, two TCM codecs (coder/decoders) have been constructed to implement two levels of bandwidth efficiency. One was designed and built by the research team at NMSU while the other was created by the University of Notre Dame with the University of South Australia. The NMSU codec achieves a 2-to-1 increase in data rate per unit bandwidth with a coding gain relative to QPSK of about 3dB. The Notre Dame/South Australia codec achieves a 2.5-to-1 increase in data rate per unit of occupied bandwidth and a coding gain of about 2dB.

Osborne, William P.↗
NASA NTRS1987

A multiprocessing architecture for real-time monitoring

A multiprocessing architecture for performing real time monitoring and analysis using knowledge-based problem solving techniques is discussed. To handle asynchronous inputs and perform in real time, the system consists of three or more separate processes which run concurrently on one or more processors and communicate via a message passing scheme. The Data Management Process gathers, compresses, scales and sends the incoming telemetry data to other tasks. The Inference Process consists of a proprietary high performance inference engine that runs at 1000 rules per second using telemetry data to perform real time analysis on the state and health of the Space Telescope. The multiprocessing architecture has been interfaced to a simulator and is able to process the incoming telemetry in real time.

Laffey, Thomas J.↗
NASA NTRS1978

Applications of dynamic measurement techniques to wind-tunnel testing

The uses of laser Doppler velocimeter, hot wire, and surface hot film techniques in the study of turbulent flows are described, and data obtained in compressible flows are discussed. Applications are illustrated with measurements of wind tunnel freestream turbulence characteristics and with data obtained in transitional, turbulent, and separated shear flows. A new method which was developed for the study of time dependent and unsteady turbulent flows is also presented.

Owen, F. K.↗
NASA NTRS2005

Low-Complexity Lossless Compression of Hyperspectral Imagery via Adaptive Filtering

A low-complexity, adaptive predictive technique for lossless compression of hyperspectral data is presented. The technique relies on the sign algorithm from the repertoire of adaptive filtering. The compression effectiveness obtained with the technique is competitive with that of the best of previously described techniques with similar complexity.

Klimesh, M.↗
DOE OSTI2020

Toxicology and Biodegradability of Blendstocks for Mixing Controlled Compression Ignition Combustion: Literature Review of Available Data

The goal of the U.S. Department of Energy’s Co-Optima project is to develop an understanding of potential biomass-derived blendstocks for use in the transportation sector. This report documents the estimated environmental and toxicological impact of potential bioblendstocks for mixing controlled compression ignition (MCCI) engines. The candidate molecules were: 2-nonanol; n-undecane; 2,6,10-trimethyldodecane; 5-ethyl-4-propylnonane; hexylhexanoate; methyldecanoate; dibutoxymethane; 4-butoxyheptane; dipentylether; renewable diesel; and soy biodiesel. To provide a robust comparison, diesel surrogate molecules were also considered: α-methylnaphthalene; decahydronaphthalene; 2,2,4,4,6,8,8-heptamethylnonane; n-butylcyclohexane; n-hexadecane; tetralin; and n-dodecylbenzene. The intent of this work was not to provide absolute answers and remove any molecules from consideration, but to provide more robust data for each molecule that can be used in subsequent evaluations. Little literature data was available for many of the molecules, both potential blendstocks and diesel surrogates, so a quantitative structural activity relationship (QSAR) approach was used to provide input into the relevant models. This assessment included an evaluation of compartment partitioning in the event of an accidental chemical spill, fate and transport indicators, estimated biodegradability, and human and environmental toxicology. The higher molecular weight and longer hydrocarbon chain lengths of these MCCI molecules reduce water solubility and mobility. Generally, the oxygenated molecules improved biodegradability metrics. The hydrocarbons farnesane and 5-Et-4-PrNonane are not predicted to be biodegradable. Human acute toxicity was found to be slightly to non-toxic for the MCCI blendstocks and the diesel surrogates. 2-Nonanol, undecane, MeDecanoate, dibutoxymethane, and dipentylether were likely developmental toxicants. Although poorly water soluble, all the compounds, whether Co-Optima blendstocks or diesel surrogates, showed significant toxicity to D. magna. The oxygenated blendstocks were less likely to bioaccumulate than the non-oxygenated blendstocks, which behaved similar to the diesel surrogates. Overall, this predicted analysis showed that the MCCI blendstocks are very similar to their diesel counterparts. No significant showstoppers were found that would indicate concern moving forward with additional research into these potential fuels. The results presented were based solely on predictive models and serve to provide information going forward for continued evaluation of these molecules.

09 BIOMASS FUELS↗
DOE OSTI2025

TensorID v1.0

This Python software package includes new and efficient algorithms for satellite and core interpolative decomposition of tensor data. In general, these algorithms target high-dimensional data reduction and compression. The software is purely numerical and can be applied by others to many important sources of tensor data generated by computation or experiment.

Zhang, Yifan [Lawrence Berkeley National Laborator↗
DOE OSTI2020

Seismology in the Cloud: A New Streaming Workflow

Data-intensive research in seismology is experiencing a recent boom, driven in part by large volumes of available data and advances in the growing field of data science. However, there are significant barriers to processing large data volumes, such as long retrieval times from data repositories, complex data management, and limited computational resources. New tools and platforms have reduced the barriers to entry for scientific cluster computing, including the maturation of the commercial cloud as an accessible instrument for research. Here, we build a customized research cluster in the cloud to test a new workflow for large-scale seismic analysis, in which data are processed as a stream (retrieved on-the-fly and acted upon without storing), with data from the Incorporated Research Institutions for Seismology Data Management Center. We use this workflow to deploy a spectral peak detection algorithm over 5.6 TB of compressed continuous seismic data from 2074 stations of the USArray Transportable Array EarthScope network. Using a 50-node cluster in the cloud, we completed the noise survey in 80 hr, with an average data throughput of 1.7 GB per minute. By varying cluster sizes, we find the scaling of our analysis to be sublinear, due to a combination of algorithmic limitations and data center response times. The cloud-based streaming workflow represents an order-of-magnitude increase in acquisition and processing speed compared to a traditional download-store-process workflow, and offers the additional benefits of employing a flexible, accessible, and widely used computing architecture. It is limited, however, due to its reliance on Internet transfer speeds and data center service capacity, and may not work well for repeated analyses or those for which even higher data throughputs are needed. These research applications will require a new class of cloud-native approaches in which both data and analysis are in the cloud.

58 GEOSCIENCES↗
DOE OSTI2021

A Platform-Independent X-ray Diffraction Diagnostic for Phase Transition Kinetics in Traditional and Synthetic Microstructure Materials (LDRD Project 213088 Final Report)

Pulsed-power generators using the magnetic loading technique are able to produce well-controlled continuous ramp compression of condensed matter for high-pressure equation-of-state studies. X-ray diffraction (XRD) data from dynamically compressed samples provide direct measurements of the elastic compression of the crystal lattice, onset of plastic flow, strength-strain rate dependence, structural phase transitions, and density of crystal defects such as dislocations. Here, we present a cost effective, compact X-ray source for XRD measurements on pulsed-power-driven ramp-loaded samples. This combination of magnetically-driven ramp compression of materials with single, short-pulse XRD diagnostic will be a powerful capability for the dynamic materials community. The success in fielding this new XRD diagnostic dramatically improves our predictive capability and understanding of rate-dependent behavior at or near phase transition. As Sandia plans the next-generation pulse-power driver platform, a key element needed to deliver new state-of-the-art experiments will be having the necessary diagnostic tools to probe new regimes and phenomena. These diagnostics need to be as versatile, compact, and portable as they are powerful. The development of a platform-independent XRD diagnostic gives Sandia researchers a new window to study the microstructure and phase dynamics of materials under load. This project has paved the way for phase transition research in a variety of materials with mission interest.

36 MATERIALS SCIENCE↗
NASA NTRS2009

Improving 3D Wavelet-Based Compression of Hyperspectral Images

Two methods of increasing the effectiveness of three-dimensional (3D) wavelet-based compression of hyperspectral images have been developed. (As used here, images signifies both images and digital data representing images.) The methods are oriented toward reducing or eliminating detrimental effects of a phenomenon, referred to as spectral ringing, that is described below. In 3D wavelet-based compression, an image is represented by a multiresolution wavelet decomposition consisting of several subbands obtained by applying wavelet transforms in the two spatial dimensions corresponding to the two spatial coordinate axes of the image plane, and by applying wavelet transforms in the spectral dimension. Spectral ringing is named after the more familiar spatial ringing (spurious spatial oscillations) that can be seen parallel to and near edges in ordinary images reconstructed from compressed data. These ringing phenomena are attributable to effects of quantization. In hyperspectral data, the individual spectral bands play the role of edges, causing spurious oscillations to occur in the spectral dimension. In the absence of such corrective measures as the present two methods, spectral ringing can manifest itself as systematic biases in some reconstructed spectral bands and can reduce the effectiveness of compression of spatially-low-pass subbands. One of the two methods is denoted mean subtraction. The basic idea of this method is to subtract mean values from spatial planes of spatially low-pass subbands prior to encoding, because (a) such spatial planes often have mean values that are far from zero and (b) zero-mean data are better suited for compression by methods that are effective for subbands of two-dimensional (2D) images. In this method, after the 3D wavelet decomposition is performed, mean values are computed for and subtracted from each spatial plane of each spatially-low-pass subband. The resulting data are converted to sign-magnitude form and compressed in a manner similar to that of a baseline hyperspectral- image-compression method. The mean values are encoded in the compressed bit stream and added back to the data at the appropriate decompression step. The overhead incurred by encoding the mean values only a few bits per spectral band is negligible with respect to the huge size of a typical hyperspectral data set. The other method is denoted modified decomposition. This method is so named because it involves a modified version of a commonly used multiresolution wavelet decomposition, known in the art as the 3D Mallat decomposition, in which (a) the first of multiple stages of a 3D wavelet transform is applied to the entire dataset and (b) subsequent stages are applied only to the horizontally-, vertically-, and spectrally-low-pass subband from the preceding stage. In the modified decomposition, in stages after the first, not only is the spatially-low-pass, spectrally-low-pass subband further decomposed, but also spatially-low-pass, spectrally-high-pass subbands are further decomposed spatially. Either method can be used alone to improve the quality of a reconstructed image (see figure). Alternatively, the two methods can be combined by first performing modified decomposition, then subtracting the mean values from spatial planes of spatially-low-pass subbands.

Klimesh, Matthew↗
DOE OSTI2020

Efficient and Flexible Hierarchical Data Layouts for a Unified Encoding of Scalar Field Precision and Resolution

To address the problem of ever-growing scientific data sizes making data movement a major hindrance to analysis, we introduce a novel encoding for scalar fields: a unified tree of resolution and precision, specifically constructed so that valid cuts correspond to sensible approximations of the original field in the precision-resolution space. Furthermore, we introduce a highly flexible encoding of such trees that forms a parameterized family of data hierarchies. We discuss how different parameter choices lead to different trade-offs in practice, and show how specific choices result in known data representation schemes such as zfp[52], idx[58], and jpeg2000 [76]. Lastly, we provide system-level details and empirical evidence on how such hierarchies facilitate common approximate queries with minimal data movement and time, using real-world data sets ranging from a few gigabytes to nearly a terabyte in size. Experiments suggest that our new strategy of combining reductions in resolution and precision is competitive with state-of-the-art compression techniques with respect to data quality, while being significantly more flexible and orders of magnitude faster, and requiring significantly reduced resources.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1995

An object-oriented, technology-adaptive information model

The primary objective was to develop a computer information system for effectively presenting NASA's technologies to American industries, for appropriate commercialization. To this end a comprehensive information management model, applicable to a wide variety of situations, and immune to computer software/hardware technological gyrations, was developed. The model consists of four main elements: a DATA_STORE, a data PRODUCER/UPDATER_CLIENT and a data PRESENTATION_CLIENT, anchored to a central object-oriented SERVER engine. This server engine facilitates exchanges among the other model elements and safeguards the integrity of the DATA_STORE element. It is designed to support new technologies, as they become available, such as Object Linking and Embedding (OLE), on-demand audio-video data streaming with compression (such as is required for video conferencing), Worldwide Web (WWW) and other information services and browsing, fax-back data requests, presentation of information on CD-ROM, and regular in-house database management, regardless of the data model in place. The four components of this information model interact through a system of intelligent message agents which are customized to specific information exchange needs. This model is at the leading edge of modern information management models. It is independent of technological changes and can be implemented in a variety of ways to meet the specific needs of any communications situation. This summer a partial implementation of the model has been achieved. The structure of the DATA_STORE has been fully specified and successfully tested using Microsoft's FoxPro 2.6 database management system. Data PRODUCER/UPDATER and PRESENTATION architectures have been developed and also successfully implemented in FoxPro; and work has started on a full implementation of the SERVER engine. The model has also been successfully applied to a CD-ROM presentation of NASA's technologies in support of Langley Research Center's TAG efforts.

Anyiwo, Joshua C.↗
DOE OSTI2021

DNTTD (Distributed Non-Negative Tensor Train Decomposition)

The era of exascale computing opens new venues for innovations and discoveries in many scientific, engineering, and commercial fields. However, with the exa flops also come the extra-large high-dimensional data generated by high performance computing. High-dimensional data is presented as multidimensional arrays, aka tensors. The presence of latent (not directly observable) structures in the tensor allows a unique representation and compression of the data by classical tensor factorization techniques. However, the classical tensor methods are not always stable or they can be exponential in their memory requirements, which makes them not suitable for high-dimensional tensors. Tensor train (TT) is a state-of-the-art tensor network introduced for factorization of high-dimensional tensors. TT transforms the initial high-dimensional tensor in a network of three-dimensional tensors that requires only a linear storage. Many real-world data, such as, density, temperature, population, probability, etc., are non-negative and for an easy interpretation, the algorithms preserving non-negativity are preferred. Here, we introduce a distributed non-negative tensor-train and demonstrate its scalability and the compression on synthetic and real world big datasets.

Bhattarai, Manish↗
DOE OSTI2022

A compact x-ray diffraction system for dynamic compression experiments on pulsed-power generators

Pulsed-power generators can produce well-controlled continuous ramp compression of condensed matter for high-pressure equation-of-state studies using the magnetic loading technique. X-ray diffraction (XRD) data from dynamically compressed samples provide direct measurements of the elastic compression of the crystal lattice, onset of plastic flow, strength–strain rate dependence, structural phase transitions, and density of crystal defects, such as dislocations. Here, we present a cost-effective, compact, pulsed x-ray source for XRD measurements on pulsed-power-driven ramp-loaded samples. This combination of magnetically driven ramp compression of materials with a single, short-pulse XRD diagnostic will be a powerful capability for the dynamic materials’ community to investigate in situ dynamic phase transitions critical to equation of states. Finally, we present results using this new diagnostic to evaluate lattice compression in Zr and Al and to capture signatures of phase transitions in CdS.

47 OTHER INSTRUMENTATION↗
DOE OSTI2020

Efficient DNA sequence compression with neural networks

Abstract Background The increasing production of genomic data has led to an intensified need for models that can cope efficiently with the lossless compression of DNA sequences. Important applications include long-term storage and compression-based data analysis. In the literature, only a few recent articles propose the use of neural networks for DNA sequence compression. However, they fall short when compared with specific DNA compression tools, such as GeCo2. This limitation is due to the absence of models specifically designed for DNA sequences. In this work, we combine the power of neural networks with specific DNA models. For this purpose, we created GeCo3, a new genomic sequence compressor that uses neural networks for mixing multiple context and substitution-tolerant context models. Findings We benchmark GeCo3 as a reference-free DNA compressor in 5 datasets, including a balanced and comprehensive dataset of DNA sequences, the Y-chromosome and human mitogenome, 2 compilations of archaeal and virus genomes, 4 whole genomes, and 2 collections of FASTQ data of a human virome and ancient DNA. GeCo3 achieves a solid improvement in compression over the previous version (GeCo2) of $2.4\%$, $7.1\%$, $6.1\%$, $5.8\%$, and $6.0\%$, respectively. To test its performance as a reference-based DNA compressor, we benchmark GeCo3 in 4 datasets constituted by the pairwise compression of the chromosomes of the genomes of several primates. GeCo3 improves the compression in $12.4\%$, $11.7\%$, $10.8\%$, and $10.1\%$ over the state of the art. The cost of this compression improvement is some additional computational time (1.7–3 times slower than GeCo2). The RAM use is constant, and the tool scales efficiently, independently of the sequence size. Overall, these values outperform the state of the art. Conclusions GeCo3 is a genomic sequence compressor with a neural network mixing approach that provides additional gains over top specific genomic compressors. The proposed mixing method is portable, requiring only the probabilities of the models as inputs, providing easy adaptation to other data compressors or compression-based data analysis tools. GeCo3 is released under GPLv3 and is available for free download at https://github.com/cobilab/geco3.

Silva, Milton↗
DOE OSTI2024

Sound speed and Grüneisen parameter up to three terapascal in shock-compressed iron

This paper presents the first sound speed and Grüneisen parameter data for fluid iron compressed to 3 TPa (30 million atmospheres) and 20 g/cm 3 on the Hugoniot. Both the sound speed and Grüneisen parameter are derivatives of the equation of state (EOS), and thus tightly constrain the contours of the EOS surface. The sound speed data are systematically lower than expected from a simple extrapolation of previous data. The Grüneisen parameter shows a 30% drop at pressures and temperatures above the melt transition. Furthermore, while some models compare well with either the sound speed or Grüneisen parameter, none of today’s state-of-the-art models can explain both sets of data. Furthermore these new data will provide pivotal benchmarks for both future theoretical EOSs of warm dense iron and modeling planetary states and processes.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗