Engineering Papers⌕ Search

Engineering topics

Kim, Jinsung

Publications and source records attributed to Kim, Jinsung.

TAMM: Tensor algebra for many-body methods

Tensor algebra operations such as contractions in computational chemistry consume a significant fraction of the computing time on large-scale computing platforms. The widespread use of tensor contractions between large multi-dimensional tensors in describing electronic structure theory has motivated the development of multiple tensor algebra frameworks targeting heterogeneous computing platforms. In this paper, we present Tensor Algebra for Many-body Methods (TAMM), a framework for productive and performance-portable development of scalable computational chemistry methods. TAMM decouples the specification of the computation from the execution of these operations on available high-performance computing systems. With this design choice, the scientific application developers (domain scientists) can focus on the algorithmic requirements using the tensor algebra interface provided by TAMM, whereas high-performance computing developers can direct their attention to various optimizations on the underlying constructs, such as efficient data distribution, optimized scheduling algorithms, and efficient use of intra-node resources (e.g., graphics processing units). The modular structure of TAMM allows it to support different hardware architectures and incorporate new algorithmic advances. We describe the TAMM framework and our approach to the sustainable development of scalable ground- and excited-state electronic structure methods. We present case studies highlighting the ease of use, including the performance and productivity gains compared to other frameworks.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Scalable Heterogeneous Execution of a Coupled-Cluster Model with Perturbative Triples

The CCSD(T) coupled-cluster model with perturbative triples is considered a gold standard for computational modeling of the correlated behavior of electrons in molecular systems. A fundamental constraint is the relatively small global-memory capacity in GPUs compared to the main-memory capacity on host nodes, necessitating relatively smaller tile sizes for high-dimensional tensor contractions in NWChem's GPU-accelerated implementation of the CCSD(T) method. A coordinated redesign is described to address this limitation and associated data movement overheads, including a novel fused GPU kernel for a set of tensor contractions, along with inter-node communication optimization and data caching. The new implementation of GPU-accelerated CCSD(T) improves overall performance by 3.4x. Finally, we discuss the trade-offs in using this fused algorithm on current and future supercomputing platforms.

Kim, Jinsung↗

A Code Generator for High-Performance Tensor Contractions on GPUs

Tensor contractions are higher dimensional generalizations of matrix-matrix multiplication. They form the compute-intensive core for many applications in computational science and data science. Although high-performance library implementations are widely available for dense matrix-matrix multiplication on GPUs, the same is not true for contraction of dense tensors. In this paper, we describe a high- performance GPU code generator for arbitrary tensor contractions. It exploits domain-specific properties about data reuse in tensor contractions to devise an effective code generation schema, coupled with an effective model-driven search to determine parameters for mapping of computation to threads and staging of data through the GPU memory hierarchy. Experimental evaluation using a set of tensor contraction benchmarks demonstrates performance improvement and/or significantly reduced code generation time over other state-of-the-art tensor contraction libraries and code generators.

Kim, Jinsung↗

Optimizing Tensor Contractions in CCSD(T) for Efficient Execution on GPUs

Tensor contractions are higher dimensional analogs of matrix multiplications, used in many computational contexts such as high order models in quantum chemistry, deep learning, nite element methods etc. In contrast to the wide availability of high-performance libraries for matrix multiplication on GPUs, the same is not true for tensor contractions. In this paper, we address the optimization of a set of symmetrized tensor contractions that form the computational bottleneck in the CCSD(T) coupled-cluster method in computational chemistry suites like NWChem. Some of the challenges in optimizing tensor contractions that arise in practice from the variety of dimensionalities and shapes for tensors include effective mapping of the high-dimensional iteration space to threads, choice of data buffering in shared-memory and registers, and tile sizes for multi-level tiling. Furthermore, in the case of symmetrized tensor contractions in CCSD(T), it is also a challenge to fuse contractions to reduce data movement cost by exploiting reuse of intermediate tensors. In this paper, we developed an efficient GPU implementation of the tensor contractions in CCSD(T) using shared-memory buffering, register tiling, loop fusion and register transpose. Experimental results demonstrate significant improvement over the current state-of-the-art.

Kim, Jinsung↗

GPU code optimization using abstract kernel emulation and sensitivity analysis

In this paper, we develop a novel approach to GPU kernel optimization by focusing on identification of bottleneck resources and determining optimization parameters that can alleviate the bottleneck. Performance modeling for GPUs is done by abstract kernel emulation along with latency/gap modeling of resources. Sensitivity analysis with respect to resource latency/gap parameters is used to predict the bottleneck resource for a given kernel's execution. Here, the utility of the bottleneck analysis is demonstrated in two contexts: 1) Enhancing the OpenTuner auto-tuner with the new bottleneck-driven optimization strategy. Experimental results on all kernels from the Rodinia suite and GPU tensor contraction kernels from the NWChem computational chemistry suite demonstrate effectiveness. 2) Manual code optimization: two case studies illustrate the use of the bottleneck analysis to iteratively improve performance of code from state-of-the-art DSL code generators.

97 MATHEMATICS AND COMPUTING↗

A domain-specific compiler for a parallel multiresolution adaptive numerical simulation environment

This paper describes the design and implementation of a layered domain-specific compiler to support MADNESS---Multiresolution ADaptive Numerical Environment for Scientific Simulation. MADNESS is a high-level software environment for the solution of integral and differential equations in many dimensions, using adaptive and fast harmonic analysis methods with guaranteed precision. MADNESS uses k-d trees to represent spatial functions and implements operators like addition, multiplication, differentiation, and integration on the numerical representation of functions. The MADNESS runtime system provides global namespace support and a task-based execution model including futures. MADNESS is currently deployed on massively parallel supercomputers and has enabled many science advances. Due to the highly irregular and statically unpredictable structure of the k-d trees representing the spatial functions encountered in MADNESS applications, only purely runtime approaches to optimization have previously been implemented in the MADNESS framework. This paper describes a layered domain-specific compiler developed to address some performance bottlenecks in MADNESS. The newly developed static compile-time optimizations, in conjunction with the MADNESS runtime support, enable significant performance improvement for the MADNESS framework.

Rajbhandari, Samyam↗

On Fusing Recursive Traversals of K-d Trees

Loop fusion is a key program transformation for data locality optimization that is implemented in production compilers. But optimizing compilers currently cannot exploit fusion opportunities across a set of recursive tree traversal computations with producer-consumer relationships. In this paper, we develop a compile-time approach to dependence characterization and program transformation to enable fusion across recursively specified traversals over k-ary trees. We present the FuseT source-to-source code transformation framework to automatically generate fused composite recursive operators from an input program containing a sequence of primitive recursive operators. We use our framework to implement fused operators for MADNESS, Multiresolution Adaptive Numerical Environment for Scientific Simulation. We show that locality optimization through fusion can offer more than an order of magnitude performance improvement.

data locality↗

Development of a 3D optical scanning-based automatic quality assurance system for proton range compensators

Purpose: A new automatic quality assurance (AutoRCQA) system using a three-dimensional scanner (3DS) with system automation was developed to improve the accuracy and efficiency of the quality assurance (QA) procedure for proton range compensators (RCs). The system performance was evaluated for clinical implementation. Methods: The AutoRCQA system consists of a three-dimensional measurement system (3DMS) based on 3DS and in-house developed verification software (3DVS). To verify the geometrical accuracy, the planned RC data (PRC), calculated with the treatment planning system (TPS), were reconstructed and coregistered with the measured RC data (MRC) based on the beam isocenter. The PRC and MRC inner surfaces were compared with composite analysis (CA) using 3DVS, using the CA pass rate for quantitative analysis. To evaluate the detection accuracy of the system, the authors designed a fake PRC by artificially adding small cubic islands with side lengths of 1.5, 2.5, and 3.5 mm on the inner surface of the PRC and performed CA with the depth difference and distance-to-agreement tolerances of [1 mm, 1 mm], [2 mm, 2 mm], and [3 mm, 3 mm]. In addition, the authors performed clinical tests using seven RCs [computerized milling machine (CMM)-RCs] manufactured by CMM, which were designed for treating various disease sites. The systematic offsets of the seven CMM-RCs were evaluated through the automatic registration function of AutoRCQA. For comparison with conventional technique, the authors measured the thickness at three points in each of the seven CMM-RCs using a manual depth measurement device and calculated thickness difference based on the TPS data (TPS-manual measurement). These results were compared with data obtained from 3DVS. The geometrical accuracy of each CMM-RC inner surface was investigated using the TPS data by performing CA with the same criteria. The authors also measured the net processing time, including the scan and analysis time. Results: The AutoRCQA system accurately detected all fake objects in accordance with the given criteria. The median systematic offset of the seven CMM-RCs was 0.08 mm (interquartile range: −0.25 to 0.37 mm) and −0.08 mm (−0.58 to 0.01 mm) in the X- and Y-directions, respectively, while the median distance difference was 0.37 mm (0.23–0.94 mm). The median thickness difference of the TPS-manual measurement at points 1, 2, and 3 was −0.4 mm (−0.4 to −0.2 mm), −0.2 mm (−0.3 to 0.0 mm), and −0.3 mm (−0.6 to −0.1 mm), respectively, while the median difference of 3DMS was 0.0 mm (−0.1 to 0.2 mm), 0.0 mm (−0.1 to 0.3 mm), and 0.1 mm (−0.1 to 0.2 mm), respectively. Thus, 3DMS showed slightly better values compared to the manual measurements for points 1 and 3 in statistical analysis (p < 0.05). The average pass rate of the seven CMM-RCs was 97.97% ± 1.68% for 1-mm CA conditions, increasing to 99.98% ± 0.03% and 100% ± 0.00% for 2- and 3-mm CA conditions, respectively. The average net analysis time was 18.01 ± 1.65 min. Conclusions: The authors have developed an automated 3DS-based proton RC QA system and verified its performance. The AutoRCQA system may improve the accuracy and efficiency of QA for RCs.

07 ISOTOPE AND RADIATION SOURCES↗

Craniospinal Irradiation Techniques: A Dosimetric Comparison of Proton Beams With Standard and Advanced Photon Radiotherapy

Purpose: To evaluate the dosimetric benefits of advanced radiotherapy techniques for craniospinal irradiation in cancer in children. Methods and Materials: Craniospinal irradiation (CSI) using three-dimensional conformal radiotherapy (3D-CRT), tomotherapy (TOMO), and proton beam treatment (PBT) in the scattering mode was planned for each of 10 patients at our institution. Dosimetric benefits and organ-specific radiation-induced cancer risks were based on comparisons of dose-volume histograms (DVHs) and on the application of organ equivalent doses (OEDs), respectively. Results: When we analyzed the organ-at-risk volumes that received 30%, 60%, and 90% of the prescribed dose (PD), we found that PBT was superior to TOMO and 3D-CRT. On average, the doses delivered by PBT to the esophagus, stomach, liver, lung, pancreas, and kidney were 19.4 Gy, 0.6 Gy, 0.3 Gy, 2.5 Gy, 0.2 Gy, and 2.2 Gy for the PD of 36 Gy, respectively, which were significantly lower than the doses delivered by TOMO (22.9 Gy, 4.5 Gy, 6.1 Gy, 4.0 Gy, 13.3 Gy, and 4.9 Gy, respectively) and 3D-CRT (34.6 Gy, 3.6 Gy, 8.0 Gy, 4.6 Gy, 22.9 Gy, and 4.3 Gy, respectively). Although the average doses delivered by PBT to the chest and abdomen were significantly lower than those of 3D-CRT or TOMO, these differences were reduced in the head-and-neck region. OED calculations showed that the risk of secondary cancers in organs such as the stomach, lungs, thyroid, and pancreas was much higher when 3D-CRT or TOMO was used than when PBT was used. Conclusions: Compared with photon techniques, PBT showed improvements in most dosimetric parameters for CSI patients, with lower OEDs to organs at risk.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

Radiation-Induced Cancers From Modern Radiotherapy Techniques: Intensity-Modulated Radiotherapy Versus Proton Therapy

Purpose: To assess and compare secondary cancer risk resulting from intensity-modulated radiotherapy (IMRT) and proton therapy in patients with prostate and head-and-neck cancer. Methods and Materials: Intensity-modulated radiotherapy and proton therapy in the scattering mode were planned for 5 prostate caner patients and 5 head-and-neck cancer patients. The secondary doses during irradiation were measured using ion chamber and CR-39 detectors for IMRT and proton therapy, respectively. Organ-specific radiation-induced cancer risk was estimated by applying organ equivalent dose to dose distributions. Results: The average secondary doses of proton therapy for prostate cancer patients, measured 20-60cm from the isocenter, ranged from 0.4 mSv/Gy to 0.1 mSv/Gy. The average secondary doses of IMRT for prostate patients, however, ranged between 3 mSv/Gy and 1 mSv/Gy, approximately one order of magnitude higher than for proton therapy. Although the average secondary doses of IMRT were higher than those of proton therapy for head-and-neck cancers, these differences were not significant. Organ equivalent dose calculations showed that, for prostate cancer patients, the risk of secondary cancers in out-of-field organs, such as the stomach, lungs, and thyroid, was at least 5 times higher for IMRT than for proton therapy, whereas the difference was lower for head-and-neck cancer patients. Conclusions: Comparisons of organ-specific organ equivalent dose showed that the estimated secondary cancer risk using scattering mode in proton therapy is either significantly lower than the cases in IMRT treatment or, at least, does not exceed the risk induced by conventional IMRT treatment.

62 RADIOLOGY AND NUCLEAR MEDICINE↗