Engineering Papers⌕ Search

Engineering topics

Kolla, Hemanth

Publications and source records attributed to Kolla, Hemanth.

180 records · Page 10

ALACRITY: Analytics-Driven Lossless Data Compression for Rapid In-Situ Indexing, Storing, and Querying. In: Transactions on Large-Scale Data- and Knowledge-Centered Systems X. Lecture Notes in Computer Science, vol 8220

High-performance computing architectures face nontrivial data processing challenges, as computational and I/O components further diverge in performance trajectories. For scientific data analysis in particular, methods based on generating heavyweight access acceleration structures, e.g. indexes, are becoming less feasible for ever-increasing dataset sizes. We present ALACRITY, demonstrating the effectiveness of a fused data and index encoding of scientific, floating-point data in generating lightweight data structures amenable to common types of queries used in scientific data analysis. We exploit the representation of floating-point values by extracting significant bytes, using the resulting unique values to bin the remaining data along fixed-precision boundaries. To optimize query processing, we use an inverted index, mapping each generated bin to a list of records contained within, allowing us to optimize query processing with attribute range constraints. Overall, the storage footprint for both index and data is shown to be below numerous configurations of bitmap indexing, while matching or outperforming query performance.

Compression Ratio↗

Exploring power behaviors and trade-offs of in-situ data analytics, In: SC '13 Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis

As scientific applications target exascale, challenges related to data and energy are becoming dominating concerns. For example, coupled simulation workflows are increasingly adopting in-situ data processing and analysis techniques to address costs and overheads due to data movement and I/O. However it is also critical to understand these overheads and associated trade-offs from an energy perspective. The goal of this paper is exploring data-related energy/performance trade-offs for end-to-end simulation workflows running at scale on current high-end computing systems. Specifically, this paper presents: (1) an analysis of the data-related behaviors of a combustion simulation workflow with an in-situ data analytics pipeline, running on the Titan system at ORNL; (2) a power model based on system power and data exchange patterns, which is empirically validated; and (3) the use of the model to characterize the energy behavior of the workflow and to explore energy/performance trade-offs on current as well as emerging systems.

Computer Science↗

Combining In-situ and In-transit Processing to Enable Extreme-Scale Scientific Analysis

With the onset of extreme-scale computing, I/O constraints make it increasingly difficult for scientists to save a sufficient amount of raw simulation data to persistent storage. One potential solution is to change the data analysis pipeline from a post-process centric to a concurrent approach based on either in-situ or in-transit processing. In this context computations are considered in-situ if they utilize the primary compute resources, while in-transit processing refers to offloading computations toa set of secondary resources using asynchronous data transfers. In this paper we explore the design and implementation of three common analysis techniques typically performed on large-scale scientific simulations: topological analysis, descriptive statistics, and visualization. We summarize algorithmic developments, describe a resource scheduling system to coordinate the execution ofvarious analysis workflows, and discuss our implementation using the DataSpaces and ADIOS frameworks that support efficient data movement between in-situ and in-transit computations. Wedemonstrate the efficiency of our lightweight, flexible framework by deploying it on the Jaguar XK6 to analyze data generated by S3D, a massively parallel turbulent combustion code. Our framework allows scientists dealing with the data deluge at extreme scale to perform analyses at increased temporal resolutions, mitigate I/O costs, and significantly improve the time to insight.

Bennett, Janine C.↗

A petascale direct numerical simulation study of the modelling of flame wrinkling for large-eddy simulations in intense turbulence

A new set of petascale direct numerical simulations (DNS) modelling lean hydrogen combustion with detailed chemistry in a temporally evolving slot-jet configuration is presented as a database for the development and validation of models for premixed turbulent combustion. The jet Reynolds number is 10,000, requiring grid numbers up to nearly seven billion, which was achieved by computation on 120,000 processor cores. In contrast to many prior DNS studies, a mean shear exists that drives strong turbulent mixing within the flame structure. Three cases are simulated with different Damköhler numbers, while Reynolds number is held fixed. Basic statistics are presented showing that integrated burning rates up to approximately six times the laminar burning rate are obtained. It is shown that increased flame surface area accounts for most of the enhanced burning while increases in the burning rate per unit area also play an important contribution. The database is then used to assess a new model of flame wrinkling intended for large-eddy simulations (LES). The approach draws on concepts from fractal geometry, requiring the modelling of an inner cut-off scale representing the smallest scale of flame wrinkling, and the fractal dimension controlling the resolution dependence of the unresolved flame surface area. In contrast to previous modelling, it is argued that the inner cut-off should be filter-size invariant in an inertial range. Then, dimensional and physical reasoning together with Damköhler’s limiting scaling laws for the turbulent flame speed are used to infer the cut-off and fractal dimension in limiting regimes. Two methods of determining the fractal dimension are proposed: a static, algebraic expression or a dynamic approach exploiting a Germano-type identity. Finally the model is compared against the DNS in a priori tests and is found to give excellent results, quantitatively capturing the trends with time, space, filter size and Damköhler number.

Energy & Fuels↗

Multi-level Layout Optimization for Efficient Spatio-temporal Queries on ISABELA-compressed Data

The size and scope of cutting-edge scientific simulations are growing much faster than the I/O subsystems of their runtime environments, not only making I/O the primary bottleneck, but also consuming space that pushes the storage capacities of many computing facilities. These problems are exacerbated by the need to perform data-intensive analytics applications, such as querying the dataset by variable and spatio-temporal constraints, for what current database technologies commonly build query indices of size greater than that of the raw data. To help solve these problems, we present a parallel query-processing engine that can handle both range queries and queries with spatio-temporal constraints, on B-spline compressed data with user-controlled accuracy. Our method adapts to widening gaps between computation and I/O performance by querying on compressed metadata separated into bins by variable values, utilizing Hilbert space-filling curves to optimize for spatial constraints and aggregating data access to improve locality of per-bin stored data, reducing the false positive rate and latency bound I/O operations (such as seek) substantially. We show our method to be efficient with respect to storage, computation, and I/O compared to existing database technologies optimized for query processing on scientific data.

Computer Science↗

S-Preconditioner for Multi-fold Data Reduction with Guaranteed User-Controlled Accuracy

The growing gap between the massive amounts of data generated by petascale scientific simulation codes and the capability of system hardware and software to effectively analyze this data necessitates data reduction. Yet, the increasing data complexity challenges most, if not all, of the existing data compression methods. In fact, lossless compression techniques offer no more than 10% reduction on scientific data that we have experience with, which is widely regarded as effectively incompressible. To bridge this gap, in this paper, we advocate a transformative strategy that enables fast, accurate, and multi-fold reduction of double-precision floating-point scientific data. The intuition behind our method is inspired by an effective use of preconditioners for linear algebra solvers optimized for a particular class of computational dwarfs (e.g., dense or sparse matrices). Focusing on a commonly used multi-resolution wavelet compression technique as the underlying solver for data reduction we propose the S-preconditioner, which transforms scientific data into a form with high global regularity to ensure a significant decrease in the number of wavelet coefficients stored for a segment of data. Combined with the subsequent EQ-calibrator, our resultant method (called S-Preconditioned EQ-Calibrated Wavelets (SPEQC-WAVELETS)), robustly achieved a 4- to 5- fold data reduction while guaranteeing user-defined accuracy of reconstructed data to be within 1% point-by-point relative error, lower than 0:01 Normalized RMSE, and higher than 0:99 Pearson Correlation. In this paper, we show the results we obtained by testing our method on six petascale simulation codes including fusion, combustion, climate, astrophysics, and subsurface groundwater in addition to 13 publicly available scientific datasets. We also demonstrate that application-driven data mining tasks performed on decompressed variables or their derived quantities produce results of comparable quality with the ones for the original data.

Jin, Ye↗