Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Data Compression”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 325 records · Page 18

Data Movement Categories

We have endeavored to classify the commonly seen data movement needs, as observed in data-intensive institutions (both commercial and non-profit), into four categories. Knowing how to map a data movement task into one of the four categories helps select proper data mover tools. For each category, how the data storage is involved, high-level examples and the nature of typical solutions are described. Finally, some general remarks are provided to help further orient readers new to this field - the 4th IT pillar.

99 GENERAL AND MISCELLANEOUS↗

Variable Precision Computing (VPC) (Final Report)

The overall goal of the Variable Precision Computing project is exploring and identifying ways to reduce internodal communication cost in production physics simulation experiments running on a distributed-memory systems by compressing data, either in a lossy or lossless way, thus resulting in a significant computational performance increase. Among the several approaches proposed, the one discussed here has been proposed, designed and implemented by me. It concerns the application of information theory-derived metrics to classical molecular dynamics and computational fluid dynamics simulation experiments.

97 MATHEMATICS AND COMPUTING↗

Characterization of Tri-lab β-Tin (Sn)

This report documents details of the microstructure and mechanical properties of -tin (Sn), that is used in the Tri-lab (Los Alamos National Laboratory (LANL), Lawrence Livermore National Laboratory (LLNL), Sandia National Laboratories (SNL)) collaboration project on Multi-phase Tin Strength. We report microstructural features detailing the crystallographic texture and grain morphology of as-received -tin from electron back scatter diffraction (EBSD). Temperature and strain rate dependent mechanical behavior was investigated by multiple compression tests at temperatures of 200K to 400K and strain rates of 0.0001 /s to 100 /s. Tri-lab tin showed significant temperature and strain rate dependent strength with no significant plastic anisotropy. A sample to sample material variation was observed from duplicate compression tests and texture measurements. Compression data was used to calibrate model parameters for temperature and rate dependent strength models, Johnson-Cook (JC), Zerilli-Armstrong (ZA) and Preston-Tonks-Wallace (PTW) strength models.

36 MATERIALS SCIENCE↗

New Sesame Equations of State for Polypropylene, Polyvinylchloride (PVC), and Nylon 66

I built new Sesame equations of state (EOS) for polypropylene (PP), polyvinylchloride (PVC), and nylon 66. PP has the same stoichiometry as polyethylene and TPX but with a density intermediate between the two. Nylon is a polyamide manufactured chiefly in the form of fibers. PP and PVC are the second- and third-most widely produced polymers in the world. All three are thermoplastics. Their monomer units are depicted in Figure 1. I discuss the theory on which the EOS are based in the following section, then compare results to thermal and shock compression data in Section III.

36 MATERIALS SCIENCE↗

LLNL FESP Theory Highlights: October 2024

I. Novikau, I. Y. Dodin, E. A. Startsev, I. Joseph, Quantum algorithms for simulating dissipative linear and nonlinear dynamics of plasmas. Invited talk at the 66th Annual Meeting of the APS Division of Plasma Physics, Atlanta, Georgia. Novikau I., Dodin I.Y., Startsev E.A., Encoding of linear kinetic plasma problems in quantum circuits via data compression, Journal of Plasma Physics. 2024;90(4):805900401, doi:10.1017/S0022377824000795. We propose an algorithm for encoding linear kinetic plasma problems in quantum circuits. The focus is on modelling electrostatic linear waves in a one-dimensional Maxwellian electron plasma. The waves are described by the linearized Vlasov–Ampère system with a spatially localized external current that drives plasma oscillations. This system is formulated as a boundary-value problem and cast in the form of a linear vector equation to be solved by using the quantum signal processing algorithm. The latter requires encoding of a matrix in a quantum circuit as a sub-block of a unitary matrix. We propose how to encode in a circuit in a compressed form and discuss how the resulting circuit scales with the problem size and the desired precision.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Can An Evolutionary Process Create English Text?

Critics of the conventional theory of biological evolution have asserted that while natural processes might result in some limited diversity, nothing fundamentally new can arise from 'random' evolution. In response, biologists such as Richard Dawkins have demonstrated that a computer program can generate a specific short phrase via evolution-like iterations starting with random gibberish. While such demonstrations are intriguing, they are flawed in that they have a fixed, pre-specified future target, whereas in real biological evolution there is no fixed future target, but only a complicated 'fitness landscape'. In this study, a significantly more sophisticated evolutionary scheme is employed to produce text segments reminiscent of a Charles Dickens novel. The aggregate size of these segments is larger than the computer program and the input Dickens text, even when comparing compressed data (as a measure of information content).

97 MATHEMATICS AND COMPUTING↗

Control of Weld Residual Stress in a Thin Steel Plate through Low Transformation Temperature Welding Consumables

Low transformation temperature welding (LTTW) consumables are characterized by a low martensite start temperature and a large fraction of martensite forming in the weld. It can efficiently reduce the tensile residual stress because the volume expansion associated with the martensitic transformation compensates for the thermal contraction during cooling. In this work, a LTTW wire, designated as EH200B, was created for the arc welding of advanced high-strength steel thin plates. In comparison to conventional ER70S-3 wires, this LTTW wire generated an opposite distortion pattern. Neutron diffraction measurements along the center thickness of the welded plates showed the maximum residual stress along the longitudinal direction (LD) in the weld region, and the heat-affected zone (HAZ) immediately adjacent to the weld region was reduced from ~330 MPa to below 240 MPa by using the LTTW wire. A finite element (FE) model was developed to predict the residual stress distributions of the plates welded under these two wires. The simulation results showed reasonable agreement with the volume-average neutron diffraction data. Compressive residual stress in the weld region using the LTTW wire was predicted by the FE method. Electron backscattered diffraction and x-ray diffraction measurements confirmed ~90% martensite was present in the LTTW weld. The fatigue life of DP980 steel lap joint panels using EH200B wire nearly doubled that of ER70S-3 wire. This improvement was attributed to the high strength and low LD residual stress in the weld and HAZ immediately adjacent to the weld.

36 MATERIALS SCIENCE↗

Friendly Extensible Transfer Tool Beta Version

Often data transfer software is designed to meet specific requirements or apply to specific environments. Frequently, this requires source code integration for added functionality. An extensible data transfer framework is needed to more easily incorporate new capabilities, in modular fashion. Using FrETT framework, functionality may be incorporated (in many cases without need of source code) to handle new platform capabilities: I/O methods (e.g., platform specific data access), network transport methods, data processing (e.g., data compression.).

Collins, William P.↗

Probabilistic Swinging Door Algorithm as Applied to Photovoltaic Power Ramping Event Detection

Photovoltaic (PV) power generation experiences power ramping events due to cloud interference. Depending on the extent of PV aggregation and local grid features, such power variability can be constructive or destructive to measures of uncertainty regarding renewable power generation; however, it directly influences contingency planning, production costs, and the overall reliable operation of power systems. For enhanced power system flexibility, and to help mitigate the negative impacts of power ramping, it is desirable to analyze events in a probabilistic fashion so degrees of beliefs concerning system states and forecastability are better captured and uncertainty is explicitly quantified. A probabilistic swinging door algorithm is developed and presented in this paper. It is then applied to a solar data set of PV power generation. The probabilistic swinging door algorithm builds on results from the original swinging door algorithm, first used for data compression in trend logging, and it is described by two uncertain parameters: (i) e, the threshold sensitivity to a given ramp, and (ii) s, the residual of the piecewise linear ramps. These two parameters determine the distribution of ramps and capture the uncertainty in PV power generation.

14 SOLAR ENERGY↗

Dynamic Modeling and Characterization of Nuclear-grade Graphite

Idaho National Labs serves as the spearhead for many innovative energy solutions to the world's energy crisis. One such solution is the INL's Microreactor which is designed to deploy to extreme/remote environments where other sources of power are either unavailable or unreliable. In order to best design these energy solutions for their operational environments, it is crucial to understand how the design, components, and materials will respond to the environmental conditions. One key material in these innovative designs is a nuclear-grade graphite known as PCEA. This study examines the behavior of PCEA graphite under dynamic loading, similar to that which may occur in extreme environments. The objective is to characterize the dynamic behavior and produce an accurate, reliable constitutive model suitable for use in simulation tools such as INL's MOOSE. Graphite specimens were tested using a Split Hopkinson Pressure Bar (SHPB) to administer the dynamic compressive load. The SHPB was charged at various pressures to produce a range of strain rates on the material in compression. Data was acquired via strain gauges on the SHPB setup, from which stress, strain, and time data were collected. Analysis revealed the stress-strain behavior of the material as well as insights into the material behavior's relationship to strain rate. Further work must continue to characterize the various other dynamic behaviors of the material which will combine to create a substantially trustworthy constitutive model for this grade of nuclear-grade graphite. Ultimately, this will allow for realistic simulation of the material in reactor designs, allowing for prediction of design weaknesses and leading to improved designs for increased resilience, security, and reliability.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Online randomized interpolative decomposition with a posteriori error estimator for temporal PDE data reduction

Traditional low-rank approximation is a powerful tool for compressing large data matrices that arise in simulations of partial differential equations (PDEs), but suffers from high computational cost and requires several passes over the PDE data. The compressed data may also lack interpretability thus making it difficult to identify feature patterns from the original data. Here, to address these issues, we present an online randomized algorithm to compute the interpolative decomposition (ID) of large-scale data matrices in situ. Compared to previous randomized IDs that used the QR decomposition to determine the column basis, we adopt a streaming ridge leverage score-based column subset selection algorithm that dynamically selects proper basis columns from the data and thus avoids an extra pass over the data to compute the coefficient matrix of the ID. In particular, we adopt a single-pass error estimator based on the non-adaptive Hutch++ algorithm to provide real-time error approximation for determining the best coefficients. As a result, our approach only needs a single pass over the original data and thus is suitable for large and high-dimensional matrices stored outside of core memory or generated in PDE simulations. A strategy to improve the accuracy of the reconstructed data gradient, when desired, within the ID framework is also presented. We provide numerical experiments on turbulent channel flow and ignition simulations, and on the NSTX Gas Puff Image dataset, comparing our algorithm with the offline ID algorithm to demonstrate its utility in real-world applications.

Column subset selection↗

Optimizing Error-Bounded Lossy Compression for Scientific Data by Dynamic Spline Interpolation

Today's scientific simulations are producing vast volumes of data that cannot be stored and transferred efficiently because of limited storage capacity, parallel I/O bandwidth, and network bandwidth. The situation is getting worse over time because of the ever-increasing gap between relatively slow data transfer speed and fast-growing computation power in modern supercomputers. Error-bounded lossy compression is becoming one of the most critical techniques for resolving the big scientific data issue, in that it can significantly reduce the scientific data volume while guaranteeing that the reconstructed data is valid for users because of its compression-error-bounding feature. In this paper, we present a novel error-bounded lossy compressor based on a state-of-the-art prediction-based compression framework. Our solution exhibits substantially better compression quality than all of the existing error-bounded lossy compressors, with comparable compression speed. Specifically, our contribution is threefold. (1) We provide an in-depth analysis of why the best-existing prediction-based lossy compressor can only minimally improve the compression quality. (2) We propose a dynamic spline interpolation approach with a series of optimization strategies that can significantly improve the data prediction accuracy, substantially improving the compression quality in turn. (3) We perform a thorough evaluation using six real-world scientific simulation datasets across different science domains to evaluate our solution vs. all other related works. Experiments show that the compression ratio of our solution is higher than that of the second-best lossy compressor by 20%similar to 460% with the same error bound in most of the cases.

Zhao, Kai↗

Massive compression for high data rate macromolecular crystallography (HDRMX): impact on diffraction data and subsequent structural analysis

New higher-count-rate, integrating, large-area X-ray detectors with framing rates as high as 17400 images per second are beginning to be available. These will soon be used for specialized macromolecular crystallography experiments but will require optimal lossy compression algorithms to enable systems to keep up with data throughput. Some information may be lost. Can we minimize this loss with acceptable impact on structural information? To explore this question, we have considered several approaches: summing short sequences of images, binning to create the effect of larger pixels, use of JPEG-2000 lossy wavelet-based compression, and use of Hcompress, which is a Haar-wavelet-based lossy compression borrowed from astronomy. We also explore the effect of the combination of summing, binning, and Hcompress or JPEG-2000. In each of these last two methods one can specify approximately how much one wants the result to be compressed from the starting file size. These provide particularly effective lossy compressions that retain essential information for structure solution from Bragg reflections.

47 OTHER INSTRUMENTATION↗

Automated pipeline processing X-ray diffraction data from dynamic compression experiments on the Extreme Conditions Beamline of PETRA III

Presented and discussed here is the implementation of a software solution that provides prompt X-ray diffraction data analysis during fast dynamic compression experiments conducted within the dynamic diamond anvil cell technique. It includes efficient data collection, streaming of data and metadata to a high-performance cluster (HPC), fast azimuthal data integration on the cluster, and tools for controlling the data processing steps and visualizing the data using the DIOPTAS software package. This data processing pipeline is invaluable for a great number of studies. The potential of the pipeline is illustrated with two examples of data collected on ammonia–water mixtures and multiphase mineral assemblies under high pressure. The pipeline is designed to be generic in nature and could be readily adapted to provide rapid feedback for many other X-ray diffraction techniques, e.g. large-volume press studies, in situ stress/strain studies, phase transformation studies, chemical reactions studied with high-resolution diffraction etc.

97 MATHEMATICS AND COMPUTING↗

Optimizing Error-Bounded Lossy Compression for Scientific Data With Diverse Constraints

Vast volumes of data are produced by today's scientific simulations and advanced instruments. These data cannot be stored and transferred efficiently because of limited I/O bandwidth, network speed, and storage capacity. Error-bounded lossy compression can be an effective method for addressing these issues: not only can it significantly reduce data size, but it can also control the data distortion based on user-defined error bounds. In practice, many scientific applications have specific requirements or constraints for lossy compression, in order to guarantee that the reconstructed data are valid for post hoc analysis. For example, some datasets contain irrelevant data that should be isolated in particular and users often have intuition regarding value ranges, geospatial regions, and other data subsets that are crucial for subsequent analysis. Existing state-of-the-art error-bounded lossy compressors, however, do not consider these constraints during compression, resulting in inferior compression ratios with respect to user's post hoc analysis, due to the fact that the data itself provides little or no value for post hoc analysis. In this work we address this issue by proposing an optimized framework that can preserve diverse constraints during the error-bounded lossy compression, e.g., cleaning the irrelevant data, efficiently preserving different precision for multiple value intervals, and allowing users to set diverse precision over both regular and irregular regions. We perform our evaluation on a supercomputer with up to 2,100 cores. Experiments with six real-world applications show that our proposed diverse constraints based error-bounded lossy compressor can obtain a higher visual quality or data fidelity on reconstructed data with the same or even higher compression ratios compared with the traditional state-of-the-art compressor SZ. Furthermore, our experiments also demonstrate very good scalability in compression performance compared with the I/O throughput of the parallel file system.

97 MATHEMATICS AND COMPUTING↗

Compression-based integral curve data reuse framework for flow visualization

Currently, by default, integral curves are repeatedly re-computed in different flow visualization applications, such as FTLE field computation, source-destination queries, etc., leading to unnecessary resource cost. We present a compression-based data reuse framework for integral curves, to greatly reduce their retrieval cost, especially in a resource-limited environment. In our design, a hierarchical and hybrid compression scheme is proposed to balance three objectives, including high compression ratio, controllable error, and low decompression cost. Specifically, we use and combine digitized curve sparse representation, floating-point data compression, and octree space partitioning to adaptively achieve the objectives. Results have shown that our data reuse framework could acquire tens of times acceleration in the resource-limited environment compared to on-the-fly particle tracing, and keep controllable information loss. Moreover, our method could provide fast integral curve retrieval for more complex data, such as unstructured mesh data.

97 MATHEMATICS AND COMPUTING↗