Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “compression techniques”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Diamond precipitation dynamics from hydrocarbons at icy planet interior conditions

The pressure and temperature conditions at which precipitation of diamond occurs from hydrocarbon mixtures is important for modelling the interior dynamics of icy planets. However, there is substantial disagreement from laboratory experiments, with those using dynamic compression techniques finding much more extreme conditions are required than in static compression. Here we report the time-resolved observation of diamond formation from statically compressed polystyrene, (C 8 H 8 ) n , heated using the 4.5 MHz X-ray pulse trains at the European X-ray Free Electron Laser facility. Diamond formation is observed above 2,500 K from 19 GPa to 27 GPa, conditions representative of Uranus’s and Neptune’s shallow interiors, on 30 μs to 40 μs timescales. This is much slower than may be observed during the ~10 ns duration of typical dynamic compression experiments, revealing reaction kinetics to be the reason for the discrepancy. In conclusion, reduced pressure and temperature conditions for diamond formation has implications for icy planetary interiors, where diamond subduction leads to heating and could drive convection in the conductive ice layer that has a role in their magnetic fields.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2026

Scientific Data Compression for Large Scale Computational Fluid Dynamics (CFD) Simulations

This Cooperative Research and Development Agreement (CRADA) between Oak Ridge National Laboratory (ORNL) and General Electric (GE) investigated methods for reducing the size of large computational fluid dynamics (CFD) simulation datasets using scientific data compression techniques. The work focused on adapting the MultiGrid Adaptive Reduction of Data (MGARD) compression framework and integrating it with high-performance I/O and visualization tools used in CFD workflows. MGARD uses hierarchical multilevel decomposition to enable error-controlled compression of floating-point scientific data while preserving quantities of interest. During the project, MGARD compression was integrated with the ADIOS I/O framework and visualization tools such as ParaView to enable efficient storage, transfer, and analysis of simulation data. The collaboration also explored approaches for improving compression performance for CFD data defined on unstructured meshes. Results demonstrate that scientific data compression can significantly reduce storage requirements and improve data management for large-scale CFD simulations.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Lion Cub: Minimizing Communication Overhead in Distributed Lion

Communication overhead is a key challenge in distributed deep learning, especially on slower Ethernet intercon nects, and given current hardware trends, communication is likely to become a major bottleneck. While gradient compression techniques have been explored for SGD and Adam, the Lion optimizer has the distinct advantage that its update vectors are the output of a sign operation, enabling straightforward quantization. However, simply compressing updates for communication and using techniques like majority voting fails to lead to end-to-end speedups due to inefficient communication algorithms and reduced convergence. We analyze three factors critical to distributed learning with Lion: optimizing communication methods, identifying effective quantization methods, and assessing the necessity of momentum synchronization. Our findings show that quantization techniques adapted to Lion and selective momentum synchronization can significantly reduce communication costs while maintaining convergence. We combine these into Lion Cub, which enables up to 5x speedups in end-to-end training compared to Lion. This highlights Lion’s potential as a communication-efficient solution for distributed training.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Unbalanced Parallel I/O: An Often-Neglected Side Effect of Lossy Scientific Data Compression

Lossy compression techniques have demonstrated promising results in significantly reducing the scientific data size while guaranteeing the compression error bounds. However, one important yet often neglected side effect of lossy scientific data compression is its impact on the performance of parallel I/O. Our key observation is that the compressed data size is often highly skewed across processes in lossy scientific compression. To understand this behavior, we conduct extensive experiments where we apply three lossy compressors MGARD, ZFP, and SZ, which are specifically designed and optimized for scientific data, to three real-world scientific applications Gray-Scott simulation, WarpX, and XGC. Our analysis result demonstrates that the size of the compressed data is always skewed even if the original data is evenly decomposed among processes. Such skewness widely exists in different scientific applications using different compressors as long as the information density of the data varies across processes. We then systematically study how this side effect of lossy scientific data compression impacts the performance of parallel I/O. We observe that the skewness in the sizes of the compressed data often leads to I/O imbalance, which can significantly reduce the efficiency of I/O bandwidth utilization if not properly handled. In addition, writing data concurrently to a single shared file through MPI-IO library is more sensitive to the unbalanced I/O loads. Therefore, we believe our research community should pay more attention to the unbalanced parallel I/O caused by lossy scientific data compression.

Wang, Xinying↗
DOE OSTI2023

zPerf: A Statistical Gray-Box Approach to Performance Modeling and Extrapolation for Scientific Lossy Compression

With the scaling up of simulation-based scientific discovery on high-performance computing systems, the disparity between compute and I/O has increased, forcing domain scientists to save only a small amount of simulation data to persistent storage. This can result in the loss of essential physics fields that are needed for data analysis. While error-bounded lossy compression has made tremendous progress in bridging the gap between compute and I/O, the lack of understanding of compression performance remains a key hurdle to its wide adoption. Here, in this work, we present zPerf, a statistical gray-box performance modeling approach for scientific lossy compression. Our contributions are threefold: 1) We develop zPerf to estimate the performance of lossy compression techniques, based on in-depth understanding and statistical modeling for data features and core compression metrics; 2) We demonstrate the in-detailed implementation of zPerf using two case studies, where we derive the performance modeling for SZ and ZFP, two leading lossy compressors; 3) We evaluate the effectiveness of zPerf on real-world datasets across various domains. Based on the evaluation, we demonstrate the efficacy of the zPerf performance model; 4) We further discuss three case studies where zPerf is applied to extrapolate the compression ratio of SZ and ZFP with alternative encoding schemes as well as ZFP with an alternative transform scheme. Through the case studies, we demonstrate the potential of zPerf for exploring the design space of lossy compression, which has hardly been studied in the literature.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Measurements of K-edge and L-edge extended x-ray absorption fine structure at the national ignition facility (invited)

High-energy-density laser facilities and advances in dynamic compression techniques have expanded access to material states in the Terapascal regime relevant to inertial confinement fusion, planetary science, and geophysics. However, experimentally determining the material temperature in these extreme conditions has remained a difficult challenge. Extended X-ray Absorption Fine Structure (EXAFS), referring to the modulations in x-ray absorption above an absorption edge from photoelectrons’ interactions with neighboring atoms, has proven to be a versatile and robust technique for probing material temperature and density for mid-to-high Z elements under dynamic compression. The current platform at the National Ignition Facility has developed six configurations for EXAFS measurements between 7 and 18 keV for different absorption edges (Fe K, Co K, Cu K, Ta L3, Pb L3, and Zr K) using a curved-crystal spectrometer and a bright, continuum foil x-ray source. Here, in this work, we describe the platform geometry, x-ray source performance, spectrometer resolution and throughput, design considerations, and data in ambient and dynamic-compression conditions.

36 MATERIALS SCIENCE↗
DOE OSTI2023

Quantum-enhanced Imaging and Spectroscopy and their Relevance to International Safeguards

As a follow-up to our previous report on quantum sensing for safeguards, here we delve deeper into quantum-enhanced imaging & spectroscopy and address their relevance to international safeguards. Much of the approaches rely on entangled photons, a quantum phenomenon not possible with classical physics, although just correlated photons will work for some applications, such as ghost imaging. We provide a comprehensive survey of quantum approaches, including multiple entangled photon ghost imaging and spectroscopy techniques. Entangled photons for noise reduction are also described, as well as Non-Line-Of-Sight imaging, compressive techniques, and squeezed light. Of particular interest is the generation of entangled photons with large wavelength separation, such as infrared/visible entangled photon pairs. Such entangled pairs would allow interaction with objects in the IR, such as in the molecular “fingerprint” wavelength region, while the recording device captures the visible photons, thus leveraging the high efficiency and lower cost of visible detectors. Unfortunately, entangled x-ray photons are not practical, which would have been useful for safeguards to interrogate shielded materials. Entangled gamma rays are even further beyond reason. We provide our assessment for application of quantum-enhanced imaging & spectroscopy for international safeguards, including suggested improvements to existing IAEA instruments and destructive assay measurements that are done at IAEA lab facilities.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2020

Deep learning for in situ data compression of large turbulent flow simulations

As the size of turbulent flow simulations continues to grow, in situ data compression is becoming increasingly important for visualization, analysis, and restart checkpointing. For these applications, single-pass compression techniques with low computational and communication overhead are crucial. In this paper we present a deep-learning approach to in situ compression using an autoencoder architecture that is customized for three-dimensional turbulent flows and is well suited for contemporary heterogeneous computing resources. The autoencoder is compared against a recently introduced randomized single-pass singular value decomposition (SVD) for three different canonical turbulent flows: decaying homogeneous isotropic turbulence, a Taylor-Green vortex, and turbulent channel flow. Our proposed fully convolutional autoencoder architecture compresses turbulent flow snapshots by a factor of 64 with a single pass, allows for arbitrarily sized input fields, is cheaper to compute than the randomized single-pass SVD for typical simulation sizes, performs well on unseen flow configurations, and has been made publicly available. The results reported here show that the autoencoder dramatically outperforms a randomized single-pass SVD with similar compression ratio and yields comparable performance to a higher-rank decomposition with an order of magnitude less compression in regard to preserving a number of important statistical quantities such as turbulent kinetic energy, enstrophy, and Reynolds stresses.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

AEflow (Autoencoder fluid flow compression network) [SWR-22-29]

As the size of turbulent flow simulations continues to grow, in situ data compression is becoming increasingly important for visualization, analysis, and restart checkpointing. For these applications, single-pass compression techniques with low computational and communication overhead are crucial. In this paper we present a deep-learning approach to in situ compression using an autoencoder architecture that is customized for three-dimensional turbulent flows and is well suited for contemporary heterogeneous computing resources. The autoencoder is compared against a recently introduced randomized single-pass singular value decomposition (SVD) for three different canonical turbulent flows: decaying homogeneous isotropic turbulence, a Taylor-Green vortex, and turbulent channel flow. Our proposed fully convolutional autoencoder architecture compresses turbulent flow snapshots by a factor of 64 with a single pass, allows for arbitrarily sized input fields, is cheaper to compute than the randomized single-pass SVD for typical simulation sizes, performs well on unseen flow configurations, and has been made publicly available. The results reported here show that the autoencoder dramatically outperforms a randomized single-pass SVD with similar compression ratio and yields comparable performance to a higher-rank decomposition with an order of magnitude less compression in regard to preserving a number of important statistical quantities such as turbulent kinetic energy, enstrophy, and Reynolds stresses.

King, Ryan↗
DOE OSTI2025

Overcoming energy limitation in post-compression of picosecond pulses by free beam propagation in air and temporal pulse splitting and recombining

Cryogenically cooled diode-pumped Yb:YAG has been demonstrated to efficiently generate pulses of >1 J energy and several picoseconds duration at a kilohertz repetition rate. Pulse broadening and post-compression techniques capable of operating at these high pulse energies will expand the use of these lasers in applications requiring shorter pulses. Towards this goal, we demonstrate the post-compression of 5.5 ps pulses with energy in the 0.1 J range by spectral broadening of self-collimated large area beams in air. In proof-of-principle experiments, pulses of 65 mJ energy from a cryogenically cooled Yb:YAG laser were compressed to 1.1 ps after 65 m propagation in a compact setup in which the beam was folded using flat mirrors. Further energy scaling was demonstrated by combining this spectral broadening technique with temporal pulse splitting and recombining. 120 mJ pulses were split into two pulses and coherently recombined with 85% efficiency after 50 m of propagation in air to yield 1.4 ps pulses.

High Energy Laser↗
DOE OSTI2025

FedCSpc: A Cross-Silo Federated Learning System With Error-Bounded Lossy Parameter Compression

Cross-Silo federated learning is widely used for scaling deep neural network (DNN) training over data silos from different locations worldwide while guaranteeing data privacy. Communication has been identified as the main bottleneck when training large-scale models due to large-volume model parameters and gradient transmission across public networks with limited bandwidth. Most previous works focus on gradient compression, while limited work tries to compress parameters that can not be ignored and extremely affect communication performance during the training. Here, to bridge this gap, we propose FedCSpc: an efficient cross-silo federated learning system with an XAI-driven adaptive parameter compression strategy for large-scale model training. Our work substantially differs from existing gradient compression techniques due to the distinct data features of gradient and parameter. The key contributions of this paper are fourfold. (1) Our designed FedCSpc proposes to compress the parameter during the training using the state-of-the-art error-bounded lossy compressor – SZ3. (2) We develop an adaptive compression error bound adjustment algorithm to guarantee the model accuracy effectively. (3) We exploit an efficient approach to utilize the idle CPU resources of clients to compress the parameters. (4) We perform a comprehensive evaluation with a wide range of models and benchmarks on a GPU cluster with 65 GPUs. Results show that FedCSpc can achieve the same model accuracy as FedAvg while reducing the data volume of parameters and gradients in communication by up to 7.39× and 288×, respectively. With 32 clients on a 4 Gb size model, FedCSpc significantly outperforms FedAvg in wall-clock time in the emulated WAN environment (at the bandwidth of 1 Gbps or lower without loss of generality).

SZ3↗
DOE OSTI2024

Phase transition kinetics revealed by in situ x-ray diffraction in laser-heated dynamic diamond anvil cells

We report successful coupling of dynamic loading in a diamond anvil cell and stable laser heating, which enables compression rates up to 500 GPa/s along high-temperature isotherms. Dynamic loading in a diamond-anvil cell allows exploration of a wider range of pathways in the pressure-temperature space compared to conventional dynamic compression techniques. By x-ray diffraction, we are able to characterize and monitor the structural transitions with the appropriate time resolution i.e., millisecond timescales. Using this method, we investigate the γ − ε phase transition of iron under dynamic compression, reaching compression rates of hundreds of GPa/s and temperatures of 2000 K. Our results demonstrate a distinct response of the γ − ε and α − ε transitions to the high compression rates achieved, possibly due to the different transition mechanisms. These findings open up new avenues to study tailored dynamic compression pathways in the pressure-temperature space and highlight the potential of this platform to capture kinetic effects (over ms time scales) in a diamond anvil cell. Published by the American Physical Society 2024

Ricks, Matthew (ORCID:000000015315612X)↗
DOE OSTI2024

Hybrid Approaches for Data Reduction of Spatiotemporal Scientific Applications

Scientists conduct large-scale simulations to compute derived quantities from primary data. Thus, it is crucial that data compression techniques maintain bounded errors on these derived quantities or quantities of interest (QOI). For many spatiotemporal applications, these QOIs are binary in nature and represent presence or absence of a physical phenomenon. In this work, we propose to use a hybrid approah for differential compression for such applications. We use a neural network (NN) approach to determine regions-of-interest (ROIs) where the binary QOIs are going to be prevalent. This is then used with traditional approaches that compress at a lower level (and higher accuracy) for these ROIs as compared to other regions.

Li, Xiao↗
DOE OSTI2023

Classical and quantum compression for edge computing: the ubiquitous data dimensionality reduction

Edge computing aims to address the challenges associated with communicating and transferring large amounts of data generated remotely to a data center in a timely and efficient manner. A central pillar of edge computing is local (i.e., at- or near-source) data processing capability so that data transfer to a data center for processing can be minimized. Data compression at the edge is therefore a natural component of edge workflows. Here we present a survey of data compression algorithms with a focus on edge computing. Not all compression algorithms can accommodate the data type heterogeneity, tight processing and communication time constraints, or energy efficiency requirement characteristics of edge computing. We discuss specific examples of compression algorithms that are being explored in the context of edge computing. We end our review with a brief survey of emerging quantum compression techniques that are of importance in quantum information processing, including the proposed concept of quantum edge computing.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Error-Bounded Learned Scientific Data Compression with Preservation of Derived Quantities

Scientific applications continue to grow and produce extremely large amounts of data, which require efficient compression algorithms for long-term storage. Compression errors in scientific applications can have a deleterious impact on downstream processing. Thus, it is crucial to preserve all the “known” Quantities of Interest (QoI) during compression. To address this issue, most existing approaches guarantee the reconstruction error of the original data or primary data (PD), but cannot directly control the problem of preserving the QoI. In this work, we propose a physics-informed compression technique that is composed of two parts: (i) reduction of the PD with bounded errors and (ii) preservation of the QoI. In the first step, we combine tensor decompositions, autoencoders, product quantizers, and error-bounded lossy compressors to bound the reconstruction error at high levels of compression. In the second step, we use constraint satisfaction post-processing followed by quantization to preserve the QoI. To illustrate the challenges of reducing the reconstruction errors of the PD and QoI, we focus on simulation data generated by a large-scale fusion code, XGC, which can produce tens of petabytes in a single day. The results show that our approach can achieve a high compression amount while accurately preserving the QoI within scientifically acceptable bounds.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Online and Scalable Data Compression Pipeline with Guarantees on Quantities of Interest

Data compression is becoming critical for data-intensive scientific applications. Scientists require compression techniques that accurately preserve derived quantities of interest (QoIs). Prior work has shown that a pipeline can be built to guarantee error on the primary data (PD) within user-defined bounds and achieve near-floating point QoI errors. In this paper, we present novel computational approaches for accelerating the pipeline and demonstrate results that enable concurrent execution of compression in parallel with the simulation nodes. This allows compression, including the writing of the required compression data, for the previous time step to be completed while the simulation proceeds with the current time step. Overall, the approach presented in this paper results in a 6–8 times improvement in computational overhead compared to previous work. These results were obtained using data generated by a large-scale fusion code called XGC, which produces hundreds of terabytes of data in a single day.

Banerjee, Tania↗
DOE OSTI2024

Synthesis, microstructure and micro-mechanical characterization of metal (Nb, Ti) – MAX phase (Ti 2 AlC) nanolaminates

We utilize elevated temperature physical vapor deposition (PVD) techniques to design metal/MAX multilayered nanocomposite thin films with alternating nanoscale metallic (Nb, Ti) and MAX phase (Ti 2 AlC) layer thicknesses. These metal/MAX nanolaminate architectures attempt to exploit a unique hierarchical topology – as interfaces between the layers are expected to be in direct competition with the internal interfaces within the MAX layers, to drive their tunable macroscopic mechanical behavior. Two metal/MAX nanolaminates – Nb/Ti 2 AlC and Ti/Ti 2 AlC – were deposited. The Nb/Ti 2 AlC metal/MAX system showed highly diffused layer interfaces with distinct Ti – rich and Nb–Al – rich layers, with the presence of MAX phase alongside TiC and other Ti–Al and Nb–Al intermetallic phases. The Nb/Ti 2 AlC system possessed a layered architecture, though the MAX phases were not found to be continuously present in each alternating layer. The second Ti/Ti 2 AlC system showed a non-lamellar nanocomposite microstructure and the formation of mixed Ti n+1 AlC n phases (a mix of n = 1, 2), and no indication of layering. Diffusion occurring between the metal/MAX layers in both cases, likely due to the elevated temperatures during the deposition process, is speculated as the likely cause of these resultant microstructures. The mechanical properties of both systems were evaluated using micromechanical (nanoindentation and micro-pillar compression) techniques, which demonstrated high strengths for both systems (Nb system: yield and instability strengths of 4.88 ± 0.1 GPa and 5.57 ± 0.03 GPa, Ti system: yield and instability strength of 5.61 ± 0.28 GPa and 6.21 ± 0.25 GPa). In conclusion, this work highlights the promising mechanical properties of metal/MAX multilayered depositions and summarizes the challenges in PVD synthesis of metal/MAX multilayered nanolaminates.

MAX phase↗
DOE OSTI2026

Calculating shock Hugoniot and isentropes using multiphase equation of state tables and application to shock and release of diamond ablators in inertial confinement fusion implosions

Advances in shock and ramp compression techniques now allow experimental access to unprecedented extreme conditions of pressure and temperature, providing a means to test theoretical models. Here, we describe a simple methodology to compute multi-phase shock Hugoniot and isentropes using multiphase equation of state tables. We treat explicitly the phase coexistence along the phase boundary to reveal the evolution of the sample as it undergoes the phase transformation in adiabatic conditions. We illustrate the method by calculating the predicted shock and shock-and-release behavior of diamond at conditions relevant for the initial stage of inertial confinement fusion implosions.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗