Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Data compression”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 505 records · Page 28

Deep compressed seismic learning for fast location and moment tensor inferences with natural and induced seismicity

Fast detection and characterization of seismic sources is crucial for decision-making and warning systems that monitor natural and induced seismicity. However, besides the laying out of ever denser monitoring networks of seismic instruments, the incorporation of new sensor technologies such as Distributed Acoustic Sensing (DAS) further challenges our processing capabilities to deliver short turnaround answers from seismic monitoring. In response, this work describes a methodology for the learning of the seismological parameters: location and moment tensor from compressed seismic records. In this method, data dimensionality is reduced by applying a general encoding protocol derived from the principles of compressive sensing. The data in compressed form is then fed directly to a convolutional neural network that outputs fast predictions of the seismic source parameters. Thus, the proposed methodology can not only expedite data transmission from the field to the processing center, but also remove the decompression overhead that would be required for the application of traditional processing methods. An autoencoder is also explored as an equivalent alternative to perform the same job. We observe that the CS-based compression requires only a fraction of the computing power, time, data and expertise required to design and train an autoencoder to perform the same task. Implementation of the CS-method with a continuous flow of data together with generalization of the principles to other applications such as classification are also discussed.

54 ENVIRONMENTAL SCIENCES↗

Big Data and Deep data in scanning and electron microscopies: functionality from multidimensional data sets

The development of electron, and scanning probe microscopies in the second half of the twentieth century have produced spectacular images of internal structure and composition of matter with, at nanometer, molecular, and atomic resolution. Largely, this progress was enabled by computer-assisted methods of microscope operation, data acquisition and analysis. The progress in imaging technologies in the beginning of the twenty first century has opened the proverbial floodgates of high-veracity information on structure and functionality. High resolution imaging now allows information on atomic positions with picometer precision, allowing for quantitative measurements of individual bond length and angles. Functional imaging often leads to multidimensional data sets containing partial or full information on properties of interest, acquired as a function of multiple parameters (time, temperature, or other external stimuli). Here, we review several recent applications of the big and deep data analysis methods to visualize, compress, and translate this data into physically and chemically relevant information from imaging data.

36 MATERIALS SCIENCE↗

Lifting MGARD: Construction of (pre)wavelets on the interval using polynomial predictors of arbitrary order

MGARD (MultiGrid Adaptive Reduction of Data) is an algorithm for compressing and refactoring scientific data, based on the theory of multigrid methods. The core algorithm is built around stable multilevel decompositions of conforming piecewise linear $C^0$ finite element spaces, enabling accurate error control in various norms and derived quantities of interest. In this work, we extend this construction to arbitrary order Lagrange finite elements $\mathbb{Q}_p$, $p \geq 0$, and propose a reformulation of the algorithm as a lifting scheme with polynomial predictors of arbitrary order. Additionally, a new formulation using a compactly supported wavelet basis is discussed, and an explicit construction of the proposed wavelet transform for uniform dyadic grids is described.

Reshniak, Viktor [Oak Ridge National Laboratory (O↗

Byte select cache compression

Techniques are disclosed for designing cache compression algorithms that control how data in caches are compressed. The techniques generate a custom “byte select algorithm” by applying repeated transforms applied to an initial compression algorithm until a set of suitability criteria is met. The suitability criteria include that the “cost” is below a threshold and that a metadata constraint is met. The “cost” is the number of blocks that can be compressed by an algorithm as compared with the “ideal” algorithm. The metadata constraint is the number of bits required for metadata.

Das, Shomit N.↗

Identifying and discriminating phase transitions along decaying shocks with line imaging Doppler interferometric velocimetry and streaked optical pyrometry [Identify phase transition along decaying shocks with line imaging VISAR and streaked optical pyrometry]

Ultrafast line-imaging velocity interferometer system for any reflector and streaked optical pyrometry are now commonly used to obtain high precision equation of state and electronic transport data under dynamic compression at major high energy density science facilities. In this work, we describe a simple way to improve distinguishing phase transformation signatures from other signals when monitoring decaying shock waves. The line-imaging capability of these optical diagnostics offers additional supporting evidence to the assignment of particular anomalies—such as plateaus or reversals—to the occurrence of a phase transition along the Hugoniot. Finally, we illustrate the discussion with two example datasets collected during laser driven shock compression of quartz and stishovite.

47 OTHER INSTRUMENTATION↗

Compressed Natural Gas Component Leak Frequency Estimation

The frequency of unintended releases in a compressed natural gas system is an important aspect of the system quantitative risk assessment. The frequencies for possible release scenarios, along with engineering models, are utilized to quantify the risks for compressed natural gas facilities. This report documents component leakage frequencies representative of compressed natural gas components that were estimated as a function of the normalized leak size. A Bayesian statistical method was used which results in leak frequency distributions for each component which represent variation and uncertainty in the leak frequency. The analysis shows that there is high uncertainty in the estimated leak frequencies due to sparsity in compressed natural gas data. These leak frequencies may still be useful in compressed natural gas system risk assessments, as long as this high uncertainty is acknowledged and considered appropriately.

03 NATURAL GAS↗

Complete EOS for PBX 9502

PBX 9502 is an insensitive plastic-bonded explosive based on triamino-trinitrobenzene (TATB). A complete equation of state (EOS) is constructed for unreacted PBX 9502 suitable for reactive burn models, i.e., high pressure regime in which material strength is unimportant. The PBX EOS is composed of two parts: a complete EOS for TATB and a porosity model which allows for variations in the initial PBX density. The TATB EOS is based on a cold curve and a thermal model for lattice vibrations. The heat capacity, and hence thermal model, is determined by the vibrational spectrum from Raman scattering. The cold curve is calibrated to diamond anvil cell data for isothermal compression using a two-piece Keane fitting form. Hugoniot data for PBX 9502 is used as a consistency check.

45 MILITARY TECHNOLOGY, WEAPONRY, AND NATIONAL DEF↗

FPGA-based compression of streaming x-ray photon correlation spectroscopy data

A data acquisition system to perform real-time background subtraction and lower-level-discrimination-based compression of streaming x-ray photon correlation spectroscopy (XPCS) data from a fast charge-coupled device (CCD) area detector has been built and put into service at the Advanced Photon source (APS) at Argonne National Laboratory. A commercial frame grabber with on-board field-programmable gate array (FPGA) was used in the design, and continuously processes 60 frames per second each consisting of 1,024 x 1,024 pixels with up to 64512 photon hits per frame.

43 PARTICLE ACCELERATORS↗

A checkpoint compression study for high-performance computing systems

As high-performance computing systems continue to increase in size and complexity, higher failure rates and increased overheads for checkpoint/restart (CR) protocols have raised concerns about the practical viability of CR protocols for future systems. Previously, compression has proven to be a viable approach for reducing checkpoint data volumes and, thereby, reducing CR protocol overhead leading to improved application performance. In this article, we further explore compression-based CR optimization by exploring its baseline performance and scaling properties, evaluating whether improved compression algorithms might lead to even better application performance and comparing checkpoint compression against and alongside other software- and hardware-based optimizations. Our results highlights are: (1) compression is a very viable CR optimization; (2) generic, text-based compression algorithms appear to perform near optimally for checkpoint data compression and faster compression algorithms will not lead to better application performance; (3) compression-based optimizations fare well against and alongside other software-based optimizations; and (4) while hardware-based optimizations outperform software-based ones, they are not as cost effective.

97 MATHEMATICS AND COMPUTING↗

Community detection in hypergraphs via mutual information maximization

Abstract The hypergraph community detection problem seeks to identify groups of related vertices in hypergraph data. We propose an information-theoretic hypergraph community detection algorithm which compresses the observed data in terms of community labels and community-edge intersections. This algorithm can also be viewed as maximum-likelihood inference in a degree-corrected microcanonical stochastic blockmodel. We perform the compression/inference step via simulated annealing. Unlike several recent algorithms based on canonical models, our microcanonical algorithm does not require inference of statistical parameters such as vertex degrees or pairwise group connection rates. Through synthetic experiments, we find that our algorithm succeeds down to recently-conjectured thresholds for sparse random hypergraphs. We also find competitive performance in cluster recovery tasks on several hypergraph data sets.

97 MATHEMATICS AND COMPUTING↗

Leaf spectral reflectance and transmittance, San Lorenzo, Panama, 2020

This data package contains full-range (350 – 2500 nm) leaf reflectance and transmittance spectra of leaves collected at the San Lorenzo forest canopy crane site in Panama (PA-SLZ) from January to March 2020. Around 1000 leaves were measured from 50 species and various leaf phenological stages. Leaves were sampled from both the top of the canopy and multiple heights within the canopy from 10 vertical profiles and for six core species, from the top of the canopy. This data package includes processed leaf spectra (*.csv), and raw data from the spectroradiometer (compressed as .zip). Metadata files include data descriptions (_dd.csv) for tabular data and sample information and a description of the field campaign protocol (PA-SLZ_2020_Protocol.pdf). In addition to the leaf spectral measurements included here, these samples were also used for measurement of leaf gas exchange, carbon and nitrogen content and leaf mass per unit leaf area (LMA) contained in related data packages.

54 ENVIRONMENTAL SCIENCES↗

Comprehensive, Multi-Source Cyber-Security Events Data Set

This data set represents 58 consecutive days of de-identified event data collected from five sources within Los Alamos National Laboratory’s corporate, internal computer network. The data sources include Windows-based authentication events from both individual computers and centralized Active Directory domain controller servers; process start and stop events from individual Windows computers; Domain Name Service (DNS) lookups as collected on internal DNS servers; network flow data as collected on at several key router locations; and a set of well-defined red teaming events that present bad behavior within the 58 days. In total, the data set is approximately 12 gigabytes compressed across the five data elements and presents 1,648,275,307 events in total for 12,425 users, 17,684 computers, and 62,974 processes. Specific users that are well known system related (SYSTEM, Local Service) were not de-identified though any well-known administrators account were still de-identified. In the network flow data, well-known ports (e.g. 80, 443, etc) were not de-identified. All other users, computers, process, ports, times, and other details were de-identified as a unified set across all the data elements (e.g. U1 is the same U1 in all of the data). The specific timeframe used is not disclosed for security purposes. In addition, no data that allows association outside of LANL’s network is included. All data starts with a time epoch of 1 using a time resolution of 1 second. In the authentication data, failed authentication events are only included for users that had a successful authentication event somewhere within the data set.

97 MATHEMATICS AND COMPUTING↗

What to Support When You’re Compressing

Over the last nearly 20 years, lossy compression has become an essential aspect of HPC applications’ data pipelines, allowing them to overcome limitations in storage capacity and bandwidth and, in some cases, increase computational throughput and capacity. However, with the adoption of lossy compression comes the requirement to assess and control the impact lossy compression has on scientific outcomes. In this work, we take a major step forward in describing the state of practice and by characterizing workloads. We examine applications’ needs and compressors’ capabilities across 9 different supercomputing application domains. We present 24 takeaways that provide best practices for applications, operational impacts for facilities achieving compressed data, and gaps in application needs not addressed by production compressors that point towards opportunities for future compression research.

Error-Bounded Lossy Compression↗

Equation-of-state, sound speed, and reshock of shock-compressed fluid carbon dioxide

Mechanical equation-of-state data of initially liquid and solid CO2 shock-compressed to terapascal conditions are reported. Diamond-sapphire anvil cells were used to vary the initial density and state of CO2 samples that were then further compressed with laser-driven shock waves, resulting in a data set from which precise derivative quantities, including Grüneisen parameter and sound speed, are determined. Reshock states are measured to 800 GPa and map the same pressure-density conditions as the single shock using different thermodynamic paths. The compressibility data reported here do not support current density-functional-theory calculations, but are better represented by tabular equation-of-state models.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗