Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Compressor”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 541 records · Page 30

Improving I/O Throughput with PRIMACY: Preconditioning ID-Mapper for Compressing Incompressibility

The ability to efficiently handle massive amounts of data is necessary for the continuing development towards exascale scientific data-mining applications and database systems. Unfortunately, recent years have shown a growing gap between the size and complexity of data produced from scientific applications and the limited I/O bandwidth available on modern high-performance computing systems. Utilizing data compression in order to lower the degree of I/O activity offers a promising means to addressing this problem. However, the standard compression algorithms previously explored for such use offer limited gains on both the end-to-end throughput and storage fronts. In this paper, we introduce an in-situ compression scheme aimed at improving end-to-end I/O throughput as well as reduction of dataset size. Our technique, PRIMACY (Preconditioning Id-MApper for Compressing incompressibility), acts as a preconditioner for standard compression libraries by modifying representation of original floating-point scientific data to increase byte-level repeatability, allowing standard loss less compressors to take advantage of their entropy-based byte-level encoding schemes. We additionally present a theoretical model for compression efficiency in high-performance computing environments and evaluate the efficiency of our approach via comparative analysis. Based on our evaluations on 20 real-world scientific datasets, PRIMACY achieved up to 38% and 22% improvements upon standard end-to-end write and read throughputs respectively in addition to a 25% increase in compression ratios paired with 3-to-4-fold improvement in both compression and decompression throughput over general purpose compressors.

Computer Science↗

Unbalanced Parallel I/O: An Often-Neglected Side Effect of Lossy Scientific Data Compression

Lossy compression techniques have demonstrated promising results in significantly reducing the scientific data size while guaranteeing the compression error bounds. However, one important yet often neglected side effect of lossy scientific data compression is its impact on the performance of parallel I/O. Our key observation is that the compressed data size is often highly skewed across processes in lossy scientific compression. To understand this behavior, we conduct extensive experiments where we apply three lossy compressors MGARD, ZFP, and SZ, which are specifically designed and optimized for scientific data, to three real-world scientific applications Gray-Scott simulation, WarpX, and XGC. Our analysis result demonstrates that the size of the compressed data is always skewed even if the original data is evenly decomposed among processes. Such skewness widely exists in different scientific applications using different compressors as long as the information density of the data varies across processes. We then systematically study how this side effect of lossy scientific data compression impacts the performance of parallel I/O. We observe that the skewness in the sizes of the compressed data often leads to I/O imbalance, which can significantly reduce the efficiency of I/O bandwidth utilization if not properly handled. In addition, writing data concurrently to a single shared file through MPI-IO library is more sensitive to the unbalanced I/O loads. Therefore, we believe our research community should pay more attention to the unbalanced parallel I/O caused by lossy scientific data compression.

Wang, Xinying↗

Spatiotemporally Adaptive Compression for Scientific Dataset with Feature Preservation – A Case Study on Simulation Data with Extreme Climate Events Analysis

Scientific discoveries are increasingly constrained by limited storage space and I/O capacities. For time-series simulations and experiments, their data often need to be decimated over timesteps to accommodate storage and I/O limitations. In this paper, we propose a technique that addresses storage costs while improving post-analysis accuracy through spatiotemporal adaptive, error-controlled lossy compression. We investigate the trade-off between data precision and temporal output rates, revealing that reducing data precision and increasing timestep frequency lead to more accurate analysis outcomes. Additionally, we integrate spatiotemporal feature detection with data compression and demonstrate that performing adaptive error-bounded compression in higher dimensional space enables greater compression ratios, leveraging the error propagation theory of a transformation-based compressor. To evaluate our approach, we conduct experiments using the well-known E3SM climate simulation code and apply our method to compress variables used for cyclone tracking. Our results show a significant reduction in storage size while enhancing the quality of cyclone tracking analysis, both quantitatively and qualitatively, in comparison to the prevalent timestep decimation approach. Compared to three state-of-the-art lossy compressors lacking feature preservation capabilities, our adaptive compression framework improves perfectly matched cases in TC tracking by 26.4-51.3% at medium compression ratios and by 77.3-571.1% at large compression ratios, with a merely 5–11% computational overhead.

Gong, Qian↗

A General Framework for Error-controlled Unstructured Scientific Data Compression

Data compression plays a key role in reducing storage and I/O costs. Traditional lossy methods primarily target data on rectilinear grids and cannot leverage the spatial coherence in unstructured mesh data, leading to suboptimal compression ratios. We present a multi-component, error-bounded compression framework designed to enhance the compression of floating-point unstructured mesh data, which is common in scientific applications. Our approach involves interpolating mesh data onto a rectilinear grid and then separately compressing the grid interpolation and the interpolation residuals. This method is general, independent of mesh types and typologies, and can be seamlessly integrated with existing lossy compressors for improved performance. We evaluated our framework across twelve variables from two synthetic datasets and two real-world simulation datasets. The results indicate that the multi-component framework consistently outperforms state-of-the-art lossy compressors on unstructured data, achieving, on average, a 2.3 − 3.5× improvement in compression ratios, with error bounds ranging from 1 × 10 the −6 to 1×10−2. We further investigate impact of hyperparameters, such as grid spacing and error allocation, to deliver optimal compression ratios in diverse datasets.

Gong, Qian↗

Efficient I/O for Neural Network Training with Compressed Data

FanStore is a shared object store that enables efficient and scalable neural network training on supercomputers. By providing a global cache layer on node-local burst buffers using a compressed representation, it significantly enhances the processing capability of deep learning (DL) applications on existing hardware. In addition, FanStore allows POSIX-compliant file access to the compressed data in user space. We investigate the tradeoff between runtime overhead and data compression ratio using real-world datasets and applications, and propose a compressor selection algorithm to maximize storage capacity given performance constraints. We consider both asynchronous (i.e., with prefetching) and synchronous I/O strategies, and propose mechanisms for selecting compressors for both approaches. Using FanStore, the same storage hardware can host 2-13x more data for example applications without significant runtime overhead. Empirically, our experiments show that FanStore scales to 512 compute nodes with near linear performance scalability.

Zhang, Zhao↗

Diagnostics, Control and Performance Parameters for the BELLA High Repetition Rate Petawatt Class Laser

A laser system producing controllable and stable pulses with high power and ultrashort duration at high repetition rate is a key component of a high energy laser-plasma accelerator (LPA). Precise characterization and control of laser properties are essential to understanding laser-plasma interactions required to build a 10-GeV class LPA. This study discusses the diagnostics, control and performance parameters of a 1 Hz, 1 petawatt (PW) class laser at the Berkeley Lab Laser Accelerator (BELLA) facility. The BELLA PW laser provided up to 46 J on target with a 1% level energy fluctuation and 1.3-μrad pointing stability. The spatial profile was measured and optimized by using a camera, wavefront sensor, and deformable mirror (ILAO system). The focus waist was measured to be r 0 = 53 μm and the fraction of energy within the circular area defined by the first minimum of the diffraction pattern (r = 67 μm) was 0.75. The temporal profile was controlled via the angle of incidence on a stretcher and a compressor, as well as an acousto-optic programmable dispersive. The temporal pulse shape was measured to be about 33 fs in full width at half maximum (WIZZLER and GRENOUILLE diagnostics). In order to accurately evaluate peak intensity, the energy-normalized peak fluence, and energy-normalized peak power were analyzed for the measured spatial and temporal mode profiles, and were found to be 15 kJ/(cm 2 J) with 6% fluctuation (standard deviation) and 25 TW/J with 5% fluctuation for 46-J on-target energy, respectively. This yielded a peak power of 1.2 PW and a peak intensity of 17×10 18 W/cm 2 with 8% fluctuation. A method to model the pulse shape for arbitrary compressor grating distance with high accuracy was developed. The pulse contrast above the amplified spontaneous emission pedestal was measured by SEQUOIA and found to be better than 10 9 . The first order spatiotemporal couplings (STCs) were measured with GRENOUILLE, and a simulation of the pulse's evolution at the vicinity of the target was presented. A maximum pulse front tilt angle of less than 7 mrad was achieved. The reduction of the peak power caused by the first order STCs was estimated to be less than 1%. Finally, the capabilities described in this paper are essential for generation of high quality electron beams.

47 OTHER INSTRUMENTATION↗

DuoModel: Leveraging Reduced Model for Data Reduction and Re-Computation on HPC Storage

High-performance computing (HPC) applications generate large amounts of floating-point data that need to be stored and analyzed efficiently to extract the insights and advance knowledge discovery. With the growing disparities between compute and I/O, optimizing the storage stack alone may not suffice to cure the I/O problem. There has been a strong push in the HPC communities to perform data reduction before data is transmitted to storage in order to lower the I/O cost. However, as of now, neither lossless nor lossy compressors can achieve the adequate reduction ratio that is desired by applications. This paper proposes DuoModel, a new approach that leverages the similarity between the full and reduced application models, and further improve the data reduction ratio. DouModel further improves the compression ratio of state-of-the-art compressors via compressing the differences (termed as delta) between the data products of the two models. For data analytics, the high fidelity data can be re-computed by launching the reduced model and applying the compressed delta. Our evaluations confirm that DuoModel can further push the limit of data reduction while the high fidelity of data is maintained.

Luo, Huizhang (ORCID:0000000323920267)↗

Towards Feature-Preserving 2D and 3D Vector Field Compression

The objective of this work is to develop error-bounded lossy compression methods to preserve topological features in 2D and 3D vector fields. Specifically, we explore the preservation of critical points in piecewise linear vector fields. We define the preservation of critical points as, without any false positive, false negative, or false type change in the decompressed data, (1) keeping each critical point in its original cell and (2) retaining the type of each critical point (e.g., saddle and attracting node). The key to our method is to adapt a vertex-wise error bound for each grid point and to compress input data together with the error bound field using a modified lossy compressor. Our compression algorithm can be also embarrassingly parallelized for large data handling and in situ processing. We benchmark our method by comparing it with existing lossy compressors in terms of false positive/negative/type rates, compression ratio, and various vector field visualizations with several scientific applications.

critical points↗

A High-Quality Workflow for Multi-Resolution Scientific Data Reduction and Visualization

Multi-resolution methods such as Adaptive Mesh Refinement (AMR) can enhance storage efficiency for HPC applications generating vast volumes of data. However, their applicability is limited and cannot be universally deployed across all applications. Furthermore, integrating lossy compression with multi-resolution techniques to further boost storage efficiency encounters significant barriers. To this end, we introduce an innovative workflow that facilitates high-quality multi-resolution data compression for both uniform and AMR simulations. Initially, to extend the usability of multi-resolution techniques, our workflow employs a compression-oriented Region of Interest (ROI) extraction method, transforming uniform data into a multi-resolution format. Subsequently, to bridge the gap between multi-resolution techniques and lossy compressors, we optimize three distinct compressors, ensuring their optimal performance on multi-resolution data. These optimizations can improve the compression ratio of SOTA approaches by up to 3.3× under the same data quality loss. Lastly, we incorporate an advanced uncertainty visualization method into our workflow to understand the potential impacts of lossy compression. Experimental evaluation demonstrates that our workflow achieves significant compression quality improvements.

Wang, Daoce↗

Operation of Natural Gas Pipeline Networks With Storage Under Transient Flow Conditions

Here, we formulate a nonlinear optimal control problem for intraday operation of a natural gas pipeline network that includes storage reservoirs. The dynamics of compressible gas flow through pipes, compressors, reservoirs, and wells are considered. In particular, a reservoir is modeled as a rigid, hollow container that stores gas under isothermal conditions and uniform density, and a well is modeled as a vertical pipe. For each pipe, flow dynamics are described by a coupled partial differential equation (PDE) system in density and mass flux variables, with momentum dissipation modeled using the Darcy–Wiesbach friction approximation. Compressors are modeled as scaling up the pressure of gas between the inlet and outlet. The governing equations for all network components are spatially discretized and assembled into a nonlinear differential-algebraic equation (DAE) system, which synthesizes above-ground pipeline and subsurface reservoir dynamics into a single reduced-order model. We seek to maximize an objective function that quantifies economic profit and network efficiency subject to the flow equations and inequalities that represent operating limitations. The problem is solved using a primal–dual interior point solver, and the solutions are validated in computational experiments and simulations on several pipeline test networks to demonstrate the effectiveness of the proposed methodology.

03 NATURAL GAS↗

waveSZ: A Hardware-Algorithm Co-Design of Efficient Lossy Compression for Scientific Data

Error-bounded lossy compression is critical to the success of extreme-scale scientific research because of ever-increasing volumes of data produced by today's high-performance computing (HPC) applications. Not only can error-controlled lossy compressors significantly reduce the I/O and storage burden but they can retain high data fidelity for post analysis. Existing state-of-the-art lossy compressors, however, generally suffer from relatively low compression and decompression throughput (up to hundreds of megabytes per second on a single CPU core), which considerably restrict the adoption of lossy compression by many HPC applications especially those with a fairly high data production rate. In this paper, we propose a highly efficient lossy compression approach based on field programmable gate arrays (FPGAs) under the state-of-the-art lossy compression model SZ. Our contributions are fourfold. (1) We adopt a wavefront memory layout to alleviate the data dependency during the prediction for higher-dimensional predictors, such as the Lorenzo predictor. (2) We propose a co-design framework named WAVESZ based on the wavefront memory layout and the characteristics of SZ algorithm and carefully implement it by using high-level synthesis. (3) We propose a hardware-algorithm co-optimization method to improve the performance. (4) We evaluate our proposed WAVESZ on three real-world HPC simulation datasets from the Scientific Data Reduction Benchmarks and compare it with other state-of-the-art methods on both CPUs and FPGAs. Experiments show that our WAVESZ can improve SZ's compression throughput by 6.9x similar to 8.7x over the production version running on a state-of-the-art CPU and improve the compression ratio and throughput by 2.1x and 5.8x on average, respectively, respectively, compared with the state-of-the-art FPGA design.

Compression Ratio↗

What to Support When You’re Compressing

Over the last nearly 20 years, lossy compression has become an essential aspect of HPC applications’ data pipelines, allowing them to overcome limitations in storage capacity and bandwidth and, in some cases, increase computational throughput and capacity. However, with the adoption of lossy compression comes the requirement to assess and control the impact lossy compression has on scientific outcomes. In this work, we take a major step forward in describing the state of practice and by characterizing workloads. We examine applications’ needs and compressors’ capabilities across 9 different supercomputing application domains. We present 24 takeaways that provide best practices for applications, operational impacts for facilities achieving compressed data, and gaps in application needs not addressed by production compressors that point towards opportunities for future compression research.

Error-Bounded Lossy Compression↗

Gas Reliability Analysis Integrated Library (GRAIL)

The Gas Reliability Analysis Integrated Library (GRAIL) is a prototype computational tool for solving a class of intra-day model-predictive control problems for a simplified network model of midstream natural gas pipeline systems. The model accurately resolves dynamic gas flows through long pipes using a non-ideal gas equation of state. It can be used to provide optimal intra-day flow schedules as well as spatiotemporal economic values of natural gas while ensuring that pipeline hydraulic limitations, compressor station constraints, operational factors, and pre-existing shipping contracts are satisfied. The inputs to the code are a model of the pipeline system as well as time-series data that specify boundary conditions on the network and time-dependent constraints on the optimal control problem. The outputs are time-series data that provide an approximately optimal solution of flows, pressures, compressor settings, and sensitivities to the objective function.

Zlotnik, Anatoly↗

Pipeline Hydrogen Decarbonization and Repurposing Analyzers (P-HyDRAs)

The Pipeline Hydrogen Decarbonization and Repurposing Analyzers (P-HyDRAs) are a set of prototype computational tools for simulating and optimizing midstream natural gas pipeline system operations subject to location and time-dependent hydrogen blending. The models can accurately resolve dynamic gas flows through large-scale pipeline networks using non-ideal gas equations of state. The codes can be used as decision support for planning and design decisions involving intra-day energy flow schedules as well as spatiotemporal economic values of natural gas, hydrogen, and net energy delivered to consumers while ensuring that pipeline hydraulic limitations, gas compressor station constraints, operational factors, and pre-existing shipping contracts are satisfied. The inputs to the codes are a model of the pipeline system as well as time-series data that specify boundary conditions on the network. For optimization, the code module requires price and quantity offers for natural gas and hydrogen and price and quantity bids for energy, which are used as time-dependent constraints in an optimal control problem. The outputs are time-series data that provide a predictive simulation of gas flows, mass fractions, and pressures, or with additional degrees of freedom give an approximately optimal solution for gas injections/withdrawals, compressor settings, and sensitivities to the objective function that provide locational values of energy.

Zlotnik, Anatoly↗

Control of hybrid boosting in highly diluted internal combustion engines

A novel hybrid system enabling both flexible supercharging and limited torque assist is studied to mitigate transient response challenges of a turbocharged spark ignition engine equipped with low pressure exhaust gas recirculation. The hybrid system, called a power split supercharger (SC), is configured with a planetary gear set that splits the supercharging power between a small low-voltage electric motor and the engine crankshaft. The air path controller relies on the coordination of four actuators on the engine side and three low-level actuators on the hybrid boosting device. A decentralized control scheme is used in this work, in which the master–slave structure of the boost pressure controller decreases the turbo-lag associated with exhaust gas recirculation, while minimizing the SC operation for fuel economy. A vector reference governor is used to prevent compressor surge during engine tip-outs. In addition, the desired intake manifold pressure is modified to provide time for evacuating the exhaust gas recirculation and avoiding misfires when transitioning to low load, while the hybrid capability of the power split SC is used to recuperate the unwanted generated power. All controllers are validated on both a mean value engine model and a high-fidelity engine model. Practical challenges of implementing the vector reference governor to the highly nonlinear engine air path with pressure pulsations originating from the engine reciprocation are discussed and some solutions are proposed.

Engineering↗

Recent advances in SRS on hydrogen isotope separation using thermal cycling absorption process

TCAP (Thermal Cycling Absorption Process) is a gas chromatograph in principle using palladium in the column packing, but it is unique in the fact that the carrier gas, hydrogen, is being isotopically separated and the system is operated in a semi-continuous manner. TCAP units are used to purify tritium. The recent TCAP advances at Savannah River Site (SRS) include compressor-free concept for heating/cooling, push and pull separation using an active inverse column, and compact column design. The new developments allow significantly higher throughput and better reliability from 1/10 of the current production system's footprint while consuming 60% less energy. Various versions are derived in the meantime for external customers to be used in fusion energy projects.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Methodology for designing grism stretchers for idler-based optical parametric chirped-pulse-amplification systems

Powerful ultrafast lasers with wavelengths above 1 mm are in demand to create attosecond pulses and ultra-intense laser sources. A promising technology to achieve these laser sources uses the by-product from an optical parametric chirped-pulse amplifier, known as the idler. Compression of the idler to high peak powers requires a grism stretcher and a grating compressor. Grism stretchers have many more design parameters than conventional stretchers, complicating the design process. This paper presents a method for designing a grism stretcher and grating compressor that logically navigates the design process. This method was applied to design a grism stretcher for the optical parametric amplifier line that is under construction at the Laboratory for Laser Energetics to produce a 50-mJ, 115-fs pulse at 1170 nm.

47 OTHER INSTRUMENTATION↗

Chirped-pulse amplification with narrowband pulses

In this study, we demonstrate a compact hyperdispersion stretcher and compressor pair that permit chirped-pulse amplification in Nd:YAG. We generate 750mJ, 0.2nm FWHM, 10Hz pulses recompressed to an 8ps near-transform-limited duration. The dispersion-matched pulse compressor and stretcher impart a chirp of 7300ps/nm, in a 3m×1m footprint.

36 MATERIALS SCIENCE↗