Mesh Optimization and Basis Compression]{Mesh Optimization and Basis Compression for Extreme-Scale Solution of Partial Differential Equations
Explore the source record for details and available documents.
Harper, Graham Bennett↗
SEARCH · Engineering Papers
Results for “compressing”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Finite Element Basis Compression for Extreme-Scale Simulations and R-Adaptive Mesh Optimization to Enable Compression
Explore the source record for details and available documents.
Harper, Graham Bennett↗
What to Support when You're Compressing: The State of Practice, Gaps, and Opportunities for Scientific Data Compression
Explore the source record for details and available documents.
Computer science↗
Streaming Compression of Scientific Data via Weak-SINDy
Here, in this paper, a streaming weak-SINDy algorithm is developed specifically for compressing streaming scientific data. The production of scientific data, either via simulation or experiments, is undergoing a stage of exponential growth, which makes data compression important and often necessary for storing and utilizing large scientific data sets. As opposed to classical “offline” compression algorithms that perform compression on a readily available data set, streaming compression algorithms compress data “online” while the data generated from simulation or experiments is still flowing through the system. This feature makes streaming compression algorithms well suited for scientific data compression, where storing the full data set offline is often infeasible. This work proposes a new streaming compression algorithm, streaming weak-SINDy, which takes advantage of the underlying data characteristics during compression. The streaming weak-SINDy algorithm constructs feature matrices and target vectors in the online stage via a streaming integration method in a memory efficient manner. The feature matrices and target vectors are then used in the offline stage to build a model through a regression process that aims to recover equations that govern the evolution of the data. For compressing high-dimensional streaming data, we adopt a streaming proper orthogonal decomposition (POD) process to reduce the data dimension and then use the streaming weak-SINDy algorithm to compress the temporal data of the POD expansion. We propose modifications to the streaming weak-SINDy algorithm to accommodate the dynamically updated POD basis. By combining the built model from the streaming weak-SINDy algorithm and a small amount of data samples, the full data flow could be reconstructed accurately at a low memory cost, as shown in the numerical tests.
97 MATHEMATICS AND COMPUTING↗
Optimising the processing and storage of visibilities using lossy compression
The next-generation radio astronomy instruments are providing a massive increase in sensitivity and coverage, largely through increasing the number of stations in the array and the frequency span sampled. The two primary problems encountered when processing the resultant avalanche of data are the need for abundant storage and the constraints imposed by I/O, as I/O bandwidths drop significantly on cold storage. An example of this is the data deluge expected from the SKA Telescopes of more than 60 PB per day, all to be stored on the buffer filesystem. While compressing the data is an obvious solution, the impacts on the final data products are hard to predict. In this paper, we chose an error-controlled compressor – MGARD – and applied it to simulated SKA-Mid and real pathfinder visibility data, in noise-free and noise-dominated regimes. As the data have an implicit error level in the system temperature, using an error bound in compression provides a natural metric for compression. MGARD ensures the compression incurred errors adhere to the user-prescribed tolerance. To measure the degradation of images reconstructed using the lossy compressed data, we proposed a list of diagnostic measures, exploring the trade-off between these error bounds and the corresponding compression ratios, as well as the impact on science quality derived from the lossy compressed data products through a series of experiments. We studied the global and local impacts on the output images for continuum and spectral line examples. We found relative error bounds of as much as 10%, which provide compression ratios of about 20, have a limited impact on the continuum imaging as the increased noise is less than the image RMS, whereas a 1% error bound (compression ratio of 8) introduces an increase in noise of about an order of magnitude less than the image RMS. For extremely sensitive observations and for very precious data, we would recommend a 0.1% error bound with compression ratios of about 4. These have noise impacts two orders of magnitude less than the image RMS levels. At these levels, the limits are due to instabilities in the deconvolution methods. We compared the results to the alternative compression tool DYSCO, in both the impacts on the images and in the relative flexibility. MGARD provides better compression for similar error bounds and has a host of potentially powerful additional features.
Techniques: interferometric↗
Exploring Autoencoder-based Error-bounded Compression for Scientific Data
Error-bounded lossy compression is becoming an indispensable technique for the success of today's scientific projects with vast volumes of data produced during the simulations or instrument data acquisitions. Not only can it significantly reduce data size, but it also can control the compression errors based on user-specified error bounds. Autoencoder (AE) models have been widely used in image compression, but few AE-based compression approaches support error-bounding features, which are highly required by scientific applications. To address this issue, we explore using convolutional autoencoders to improve error-bounded lossy compression for scientific data, with the following three key contributions. (1) We provide an in-depth investigation of the characteristics of various autoencoder models and develop an error-bounded autoencoder-based framework in terms of the SZ model. (2) We optimize the compression quality for main stages in our designed AE-based error-bounded compression framework, fine-tuning the block sizes and latent sizes and also optimizing the compression efficiency of latent vectors. (3) We evaluate our proposed solution using five real-world scientific datasets and comparing them with six other related works. Experiments show that our solution exhibits a very competitive compression quality from among all the compressors in our tests. In absolute terms, it can obtain a much better compression quality (100%similar to 800% improvement in compression ratio with the same data distortion) compared with SZ2.1 and ZFP in cases with a high compression ratio.
Liu, Jinyang↗
Compressing data for storage in cache memories in a hierarchy of cache memories
An electronic device includes at least one compression-decompression functional block and a hierarchy of cache memories with a first cache memory and a second cache memory. The at least one compression-decompression functional block receives data in an uncompressed state, compresses the data using one of a first compression or a second compression, and, after compressing the data, provides the data to the first cache memory for storage therein. When the data is retrieved from the first cache memory to be stored in the second cache memory, when the data is compressed using the first compression, the compression-decompression functional block decompresses the data to reverse effects of the first compression on the data, thereby restoring the data to the uncompressed state and provides the data compressed using the second compression or in the uncompressed state to the second cache memory for storage therein.
97 MATHEMATICS AND COMPUTING↗
MHD simulation on magnetic compression of field reversed configurations with NIMROD
Abstract Magnetic compression has long been proposed a promising method for plasma heating in a field reversed configuration (FRC). However, it remains a challenge to fully understand the physical mechanisms underlying the compression process, due to its highly dynamic nature beyond the one-dimensional (1D) adiabatic theory model (Spencer et al 1983 Phys. Fluids 26 1564). In this work, magnetohydrodynamics simulations on the magnetic compression of FRCs using the NIMROD code (Sovinec et al 2004 J. Comput. Phys. 195 355) and their comparisons with the 1D theory have been performed. The effects of the assumptions of the theory on the compression process have been explored, and the detailed profiles of the FRC during compression have been investigated. The pressure evolution agrees with the theoretical prediction under various initial conditions. The axial contraction of the FRC can be affected by the initial density profile and the ramping rate of the compression magnetic field, but the theoretical predictions on the FRC’s length in general and the relation r s = 2 r o in particular hold approximately well during the whole compression process, where r s is the major radius of FRC separatrix and r o is that of the magnetic axis. The evolutions of the density and temperature can be affected significantly by the initial equilibrium profile and the ramping rate of the compression magnetic field. During the compression, the major radius of the FRC is another parameter that is susceptible to the ramping rate of the compression field. Basically, for the same magnetic compression ratio, the peak density is higher and the FRC’s radius r s is smaller than the theoretical predictions.
70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
Temperature-induced densification in compressed basaltic glass revealed by in-situ ultrasonic measurements
Abstract Acoustic velocities of a model basalt glass (64 mol% CaMgSi2O6 + 36 mol% CaAl2Si2O8) were measured along different pressure-temperature (P-T) paths. One set of experiments involved isothermal compression-decompression cycles, performed at temperatures of 300, 641, 823, and 1006 K and pressures up to 12.2 GPa. The other set of experiments involved constant-load heating-cooling cycles at temperatures up to 823 K and pressures up to 7.5 GPa. Both sets of experiments were performed in a multi-anvil apparatus using a synchrotron-based ultrasonic technique. Our results show that the glass compressed isothermally at 300 K (cold-compression) displays anomalously decreasing compressional (VP) and shear (VS) wave velocities with increasing pressure until ~8 GPa. Beyond 8 GPa, both VP and VS start to increase sharply with pressure and irreversible densification of the glass occurred, producing large hysteresis loops of velocities upon decompression. However, for the glass compressed isothermally at increasingly higher temperatures (hot-compression), the velocity minima gradually shift to lower pressures. At temperature close to the glass transition temperature Tg, the velocity minima disappear completely, displaying a monotonic increase of velocities during compression and higher VP and VS during decompression. In addition, constant-load heating-cooling experiments show that velocities generally decrease slightly with increasing temperature, but start to increase once heated above a threshold temperature (~650 K). During cooling the velocities increase almost linearly with decreasing temperature, resulting in higher velocities (~1.5–2.5% higher) when returned to 300 K. This implies that a temperature-induced densification may have occurred in the glass at high pressures. Raman spectra on recovered samples show that the hot-compressed and high-P heated glasses contain distinctly densified and depolymerized structural signatures compared to the initial glass and the cold-compressed glass below the velocity transition pressure PT (~8 GPa). Such densification may be attributed to the breaking of bridging oxygen bonds and compaction in the intermediate-range structure. Our results demonstrate that temperature can facilitate glass densification at high pressures and point out the importance of P-T history in understanding the elastic properties of silicate glasses. Comparison with melt velocity suggests that hot-compressed glasses may better resemble the pressure dependence of velocity of silicate melts than cold-compressed glasses, but still show significantly higher velocities than melts. If the abnormal acoustic behaviors of cold-compressed glasses were used to constrain melt fractions in the mantle low-velocity regions, the melt fractions needed to explain a given velocity reduction would be significantly underestimated at high pressures.
Geochemistry & Geophysics↗
Localized material compression to correct distortion in wire arc additive manufacturing
Wire Arc Additive Manufacturing (WAAM) is an advanced manufacturing technology which utilizes welding systems to generate three dimensional geometries in a layer-by-layer fashion. Distortion or warping of a print substrate and WAAM components due to thermally induced residual stresses is an ongoing challenge limiting the widespread adoption of WAAM technologies for producing components. In this manuscript, a novel approach is described to address thermal distortion in deposited components by applying lateral compressions along the length of the deposited material. To demonstrate this method, a series of single-track walls were printed and compressed at evenly spaced intervals using a modified hydraulic cutter tool. The jaws of the tool were modified to compress material rather than to shear it. A mathematical model was developed to relate the curvature of the deposited material to the volume of compression required to eliminate this distortion. Validation of this model was performed using 3D scan data to compare the change in wall curvature induced by compression to the volume of the applied compressions. Substrate deflection was also compared against a control wall, and implementation of wall compression reduced maximum deflections by 93% across a series of four depositions and subsequent compressions. Wall cross sections were also analyzed to determine the impact of compression on material hardness and grain structure. The results demonstrate that successively placed lateral compressions can effectively control and potentially eliminate bending distortion in printed parts. This methodology can be further developed to form a robust model for correction of thermally-induced distortion in WAAM components.
Additive manufacturing↗
Scalable Hybrid Learning Techniques for Scientific Data Compression
Data compression is becoming critical for storing scientific data because many scientific applications need to store large amounts of data and post process this data for scientific discovery. Unlike image and video compression algorithms that limit errors to primary data (PD), scientists require compression techniques that accurately preserve derived quantities of interest (QoIs). Here, this article presents a physics-informed compression technique implemented as an end-to-end, scalable, GPU-based pipeline for data compression that addresses this requirement. Our hybrid compression technique combines machine learning techniques and standard compression methods. Specifically, we combine an autoencoder, an error-bounded lossy compressor to provide guarantees on raw data error, and a constraint satisfaction post-processing step to preserve the QoIs within a minimal error (generally less than floating point error). The effectiveness of the data compression pipeline is demonstrated by compressing nuclear fusion simulation data generated by a large-scale fusion code, XGC, which produces hundreds of terabytes of data in a single day. Our approach works within the ADIOS framework and results in compression by a factor of more than 150 while requiring only a few percent of the computational resources necessary for generating the data, making the overall approach highly effective for practical scenarios.
ITER↗
Model of ramp compression of diamond from ab initio simulations
Ramp compression experiments characterize high-pressure states of matter at temperatures well below those present in shock compression. However, because temperature is typically not directly measured during ramp compression, it is uncertain how much heating occurs under these shock-free conditions. Here, we performed a series of ab initio simulations on carbon in order to match the density-stress measurements of Smith et al. [Smith et al., Nature (London) 511, 330 (2014)]. We considered isotropically as well as uniaxially compressed solid carbon in the diamond and BC8 phases, with and without defects, as well as liquid carbon. Our idealized model ascribes heating during ramp compression to an initially uniaxially compressed cell transforming isochorically into an isotropically (hydrostatic equivalent) compressed state having lower internal energy, hence higher temperature so as to conserve energy. Multiple such heating events can occur during a single ramp experiment, leading to higher temperatures than with isentropic compression. Comparison with experiments shows that heating alone does not explain the equation of state measurements on diamond, instead implying that a significant uniaxial stress component remains present at high compression. The temperature predictions of our ramp compression model remain to be verified with future laboratory measurements.
58 GEOSCIENCES↗
An Algorithmic and Software Pipeline for Very Large Scale Scientific Data Compression with Error Guarantees
Efficient data compression is becoming increasingly critical for storing scientific data because many scientific applications produce vast amounts of data. This paper presents an end-to-end algorithmic and software pipeline for data compression that guarantees both error bounds on primary data (PD) and derived data, known as Quantities of Interest (QoI).We demonstrate the effectiveness of the pipeline by compressing fusion data generated by a large-scale fusion code, XGC, which produces tens of petabytes of data in a single day. We demonstrate that the compression is conducted by setting aside computational resources known as staging nodes, and does not impact the simulation performance. For efficient parallel I/O, the pipeline uses ADIOS2, which many codes such as XGC already use for their parallel I/O. We show that our approach can compress the data by two orders of magnitude while guaranteeing high accuracy on both the PD and the QoIs. Further, the amount of resources required by compression is a few percent of the resources required by simulation while ensuring that the compression time for each stage is less than the corresponding simulation time.This pipeline consists of three main steps. The first step decomposes the data using domain decomposition into small subdomains. Each subdomain is then compressed independently to achieve a high level of parallelism. The second step uses existing techniques that guarantee error bounds on the primary data for each subdomain. The third step uses a post-processing optimization technique based on Lagrange multipliers to reduce the QoI errors for data corresponding to each subdomain. The Lagrange multipliers generated can be further quantized or truncated to increase the compression level. All of the above characteristics of our approach make it highly practical to apply on-the-fly compression while guaranteeing errors on QoIs that are critical to the scientists.
Banerjee, Tania↗
In situ compression artifact removal in scientific data using deep transfer learning and experience replay
The massive amount of data produced during simulation on high-performance computers has grown exponentially over the past decade, exacerbating the need for streaming compression and decompression methods for efficient storage and transfer of this data---key to realizing the full potential of large-scale computational science. Lossy compression approaches such as JPEG when applied to scientific simulation data realized as a stream of images can achieve good compression rates but at the cost of introducing compression artifacts and loss of information. This paper develops a unified framework for in situ compression artifact removal in which the fully convolutional neural network architectures are combined with scalable training, transfer learning, and experience replay to achieve superior accuracy and efficiency while significantly decreasing the storage footprint as compared with the traditional optimization-based approaches. We demonstrate the proposed approach and compare it with compressed sensing postprocessing and other baseline deep learning models using climate simulations and nuclear reactor simulations, both of which are driven by hyperbolic partial differential equations. Our approach when applied to remove the compression artifacts on the JPEG-compressed nuclear reactor simulation data (using a transfer-trained model that was pretrained on the climate simulation data and updated incrementally as the nuclear reactor simulation progressed), achieved a significant improvement---mean peak signal-to-noise ratio of 42.438 as compared with 27.725 obtained with the compressed sensing approach.
97 MATHEMATICS AND COMPUTING↗
A Comparative Study of Compressive Effects on the Morphology and Performance of Carbon Paper and Cloth Electrodes in Redox Flow Batteries
Compressing porous carbon electrodes is a common approach to improve flow battery performance, but the resulting impact on electrode structure, fluid dynamics, and cell performance is not well understood. Herein, microtomographic imaging, load cell testing, and flow cell diagnostics are employed to characterize how compression‐induced changes impact pressure drop, polarization, and mass‐transfer scaling. Five different compressions are tested, spanning ranges typically used in literature, for AvCarb 1071 cloth (0%, 9%, 20%, 25%, 32%) and Freudenberg H23 paper (0%, 8%, 12%, 17%, 22%). It is found that the two electrode structures have distinct responses to compression, resulting in differing optimal conditions identified for each material; specifically, the Freudenberg H23 exhibits lower combined ohmic, charge‐transfer, and mass‐transport values at 8% compression, resulting in improved electrochemical performance across all compressive values, as compared to the optimal AvCarb 1071 compression (20%). Overall, Freudenberg H23 exhibits a greater sensitivity to compression with peak electrochemical activity correlating with increased permeability, whereas AvCarb 1071 is insensitive to compressive loads but produces lower electrochemical performance. Herein, the trade‐offs of mechanical robustness on fluid‐dynamic and electrochemical performance between the two electrodes are demonstrated by the aforementioned findings, suggesting each could be used for specific operating environments.
25 ENERGY STORAGE↗
Anisotropy evolution of elastomeric foams during uniaxial compression measured via in-situ X-ray computed tomography
We have characterized the three-dimensional evolution of microstructural anisotropy of a family of elastomeric foams during uniaxial compression via in-situ X-ray computed tomography. Flexible polyurethane foam specimens with densities of 136, 160 and 240 kg/m 3 were compressed in uniaxial stress tests both parallel and perpendicular to the foam rise direction, to engineering strains exceeding 70%. The uncompressed microstructures show slightly elongated ellipsoidal pores, with elongation aligned parallel to the foam rise direction. The evolution of this microstructural anisotropy during deformation is quantified based on the autocorrelation of the image intensity, and verified via the mean intercept length as well as the shape of individual pores. Trends are consistent across all three methods. In the rise direction, the material remains transversely anisotropic throughout compression. Anisotropy initially decreases with compression, reaches a minimum, then increases up to large strains, followed by a small decrease in anisotropy at the largest strains as pores collapse. Compression perpendicular to the foam rise direction induces secondary anisotropy with respect to the compression axis, in addition to primary anisotropy associated with the foam rise direction. In contrast to compression in the rise direction, primary anisotropy initially increases with compression, and shows a slight decrease at large strains. These surprising non-monotonic trends and qualitative differences in rise and transverse loading are explained based on the compression of initially ellipsoidal pores. Microstructural anisotropy trends reflect macroscopic stress-strain and lateral strain response. These findings provide novel quantitative connections between three-dimensional microstructure and anisotropy in moderate density polymer foams up to large deformation, with important implications for understanding complex three-dimensional states of deformation.
36 MATERIALS SCIENCE↗
A mechanism for reduced compression in indirectly driven layered capsule implosions
High-yield implosions on the National Ignition Facility rely on maintaining low entropy in the deuterium–tritium fuel, quantified by its adiabat, in order to efficiently couple energy to the hot spot through high compression of the fuel layer. We present very-high-resolution xRAGE simulation results that study the impacts of interfacial mixing and the jetting of materials due to surface defects, defects on internal interfaces, voids, and engineering features on fuel layer compression. Defects and voids are typically neglected in implosion simulations due to their small size and three-dimensional geometry. Our results showed that supersonic jets of material arise through weak spots in the shell at peak implosion velocity that prevent uniform compression of the fuel layer even when they do not introduce contaminant into the hot spot. This occurs despite maintaining low fuel entropy, since the formation of the weak spots involves nonradial displacement of fuel mass. In contrast, simulations show that fuel–ablator mixing due to interfacial instabilities has a much smaller impact on compression. We show that defects on interior interfaces of plastic capsules decrease compression by 15% to 25% and interfacial mixing between the ablator and fuel decreases compression by less than 1% for implosions with plastic or high-density carbon (HDC) ablators. For low adiabat implosions, the impact of jetting seeded by the support tent can also decrease the compression by 25%. We demonstrate that the inclusion of interior defects in simulations can explain the inferred compression in two fielded plastic capsule implosions and that the inclusion of voids, for which available characterization has large uncertainties, in simulations of HDC capsule implosions has a qualitatively consistent impact. This mechanism offers a potential explanation for persistently overestimated fuel compression in design simulations of layered implosions on the National Ignition Facility.
70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
Efficient DNA sequence compression with neural networks
Abstract Background The increasing production of genomic data has led to an intensified need for models that can cope efficiently with the lossless compression of DNA sequences. Important applications include long-term storage and compression-based data analysis. In the literature, only a few recent articles propose the use of neural networks for DNA sequence compression. However, they fall short when compared with specific DNA compression tools, such as GeCo2. This limitation is due to the absence of models specifically designed for DNA sequences. In this work, we combine the power of neural networks with specific DNA models. For this purpose, we created GeCo3, a new genomic sequence compressor that uses neural networks for mixing multiple context and substitution-tolerant context models. Findings We benchmark GeCo3 as a reference-free DNA compressor in 5 datasets, including a balanced and comprehensive dataset of DNA sequences, the Y-chromosome and human mitogenome, 2 compilations of archaeal and virus genomes, 4 whole genomes, and 2 collections of FASTQ data of a human virome and ancient DNA. GeCo3 achieves a solid improvement in compression over the previous version (GeCo2) of $2.4\%$, $7.1\%$, $6.1\%$, $5.8\%$, and $6.0\%$, respectively. To test its performance as a reference-based DNA compressor, we benchmark GeCo3 in 4 datasets constituted by the pairwise compression of the chromosomes of the genomes of several primates. GeCo3 improves the compression in $12.4\%$, $11.7\%$, $10.8\%$, and $10.1\%$ over the state of the art. The cost of this compression improvement is some additional computational time (1.7–3 times slower than GeCo2). The RAM use is constant, and the tool scales efficiently, independently of the sequence size. Overall, these values outperform the state of the art. Conclusions GeCo3 is a genomic sequence compressor with a neural network mixing approach that provides additional gains over top specific genomic compressors. The proposed mixing method is portable, requiring only the probabilities of the models as inputs, providing easy adaptation to other data compressors or compression-based data analysis tools. GeCo3 is released under GPLv3 and is available for free download at https://github.com/cobilab/geco3.
Silva, Milton↗