Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Parallel Supercomputers”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 199 records · Page 11

Scalable Parallel Methods for Analyzing Metagenomics Data at Extreme Scale

The field of bioinformatics and computational biology is currently experiencing a data revolution. The exciting prospect of making fundamental biological discoveries is fueling the rapid development and deployment of numerous cost-effective, high-throughput next-generation sequencing technologies. The result is that the DNA and protein sequence repositories are being bombarded with new sequence information. Databases are continuing to report a Moore’s law-like growth trajectory in their database sizes, roughly doubling every 18 months. In what seems to be a paradigm-shift, individual projects are now capable of generating billions of raw sequence data that need to be analyzed in the presence of already annotated sequence information. While it is clear that data-driven methods, such as sequencing homology detection, are becoming the mainstay in the field of computational life sciences, the algorithmic advancements essential for implementing complex data analytics at scale have mostly lagged behind. Sequence homology detection is central to a number of bioinformatics applications including genome sequencing and protein family characterization. Given millions of sequences, the goal is to identify all pairs of sequences that are highly similar (or “homologous”) on the basis of alignment criteria. While there are optimal alignment algorithms to compute pairwise homology, their deployment for large-scale is currently not feasible; instead, heuristic methods are used at the expense of quality. In this dissertation, we present the design and evaluation of a parallel implementation for conducting optimal homology detection on distributed memory supercomputers. Our approach uses a combination of techniques from asynchronous load balancing (viz. work stealing, dynamic task counters), data replication, and exact-matching filters to achieve homology detection at scale. Results for a collection of 2.56M sequences show parallel efficiencies of ~75-100% on up to 8K cores, representing a time-to-solution of 33 seconds. We extend this work with a detailed analysis of single-node sequence alignment performance using the latest CPU vector instruction set extensions. Preliminary results reveal that current sequence alignment algorithms are unable to fully utilize widening vector registers.

59 BASIC BIOLOGICAL SCIENCES↗

Enabling Low-Overhead HT-HPC Workflows at Extreme Scale using GNU Parallel

GNU Parallel is a versatile and powerful tool for process parallelization widely used in scientific computing. This paper demonstrates its effective application in high-performance computing (HPC) environments, particularly focusing on its scalability and efficiency in executing large-scale high-throughput high-performance computing (HT-HPC) workflows. Through real-world examples, we highlight GNU Parallel’s performance across various HPC workloads, including GPU computing, container-based workloads, and node-local NVMe storage. Our results on two leading supercomputers, OLCF’s Frontier and NERSC’s Perlmutter, showcase GNU Parallel’s rapid process dispatching ability and its capacity to maintain low overhead even at extreme scales. We explore GNU Parallel’s application in massive parallel file transfers using a scheduled Data Transfer Node (DTN) cluster, emphasizing its broad utility in diverse scientific workflows. Beyond its direct application as a viable workflow manager, GNU Parallel can be employed in conjunction with other workflow systems as a "last-mile" parallelizing driver and as a quick prototyping tool to design and extract parallel profiles from application executions. We then argue that the potential for GNU Parallel to transform workflow management at extreme scales is substantial, paving the way for more efficient and effective scientific discoveries.

Maheshwari, Ketan↗

Framework Application for Core Edge Transport Simulations (FACETS) Performance Analysis (Final Report)

The goal of the FACETS project (Framework Application for Core-Edge Transport Simulations) was to provide a multiphysics, parallel framework application (FACETS) that will enable whole-device modeling for the U.S. fusion program, to provide the modeling infrastructure needed for ITER, the next step fusion confinement device. Through use of modern computational methods, including component technology and object oriented design, FACETS is able to switch from one model to another for a given aspect of the physics in a flexible manner. This enables use of simplified models for rapid turnaround or high-fidelity models that can take advantage of the largest supercomputer hardware. FACETS does so in a heterogeneous parallel context, where different parts of the application execute in parallel by utilizing task farming, domain decomposition, and/or pipelining as needed and applicable. ParaTools, Inc. was tasked with supporting the performance analysis and tuning of the FACETS components and framework in order to achieve the parallel scaling goals of the project. The TAU Performance System® was used for instrumentation, measurement, archiving, and profile / tracing analysis. ParaTools, Inc. also assisted in FACETS performance engineering efforts. Through the use of the TAU Performance System, ParaTools provided instrumentation, measurement, analysis and archival support for the FACETS project. Performance optimization of key components has yielded significant performance speedups. TAU was integrated into the FACETS build for both the full coupled application and the UEDGE component. The performance database provided archival storage of the performance regression testing data generated by the project, and helped to track improvements in the software development.

97 MATHEMATICS AND COMPUTING↗

Toward Large-Scale Image Segmentation on Summit

Semantic segmentation of images is an important computer vision task that emerges in a variety of application domains such as medical imaging, robotic vision and autonomous vehicles to name a few. While these domain-specific image analysis tasks involve relatively small image sizes (~ 10 2 × 10 2 ), there are many applications that need to train machine learning models on image data with extents that are orders of magnitude larger (~10 4 × 10 4 ). Training deep neural network (DNN) models on large extent images is extremely memory-intensive and often exceeds the memory limitations of a single graphical processing unit, a hardware accelerator of choice for computer vision workloads. Here, an efficient, sample parallel approach to train U-Net models on large extent image data sets is presented. Its advantages and limitations are analyzed and near-linear strong-scaling speedup demonstrated on 256 nodes (1536 GPUs) of the Summit supercomputer. Using a single node of the Summit supercomputer, an early evaluation of a recently released model parallel framework called GPipe is demonstrated to deliver ~ 2X speedup in executing a U-Net model with an order of magnitude larger number of trainable parameters than reported before. Performance bottlenecks for pipelined training of U-Net models are identified and mitigation strategies to improve the speedups are discussed. Together, these results open up the possibility of combining both approaches into a unified scalable pipelined and data parallel algorithm to efficiently train U-Net models with very large receptive fields on data sets of ultra-large extent images.

Seal, Sudip↗

Final Report for Project "Framework Application for Core-Edge Transport Simulations (FACETS)"

This is the final report for the Colorado State University Component of the FACETS Project. FACETS was focused on the development of a multiphysics, parallel framework application that could provide the capability to enable whole-device fusion reactor modeling and, in the process, the development of the modeling infrastructure and computational understanding needed for ITER. It was intended that FACETS be highly flexible, through the use of modern computational methods, including component technology and object oriented design, to facilitate switching from one model to another for a given aspect of the physics, and making it possible to use simplified models for rapid turnaround or high-fidelity models that will take advantage of the largest supercomputer hardware. FACETS was designed in a heterogeneous parallel context, where different parts of the application can take advantage through parallelism based on task farming, domain decomposition, and/or pipelining as needed and applicable. As with all fusion simulations, an integral part of the FACETS project was treatment of the coupling of different physical processes at different scales interacting closely. A primary example for the FACETS project is the coupling of existing core and edge simulations, with the transport and wall interactions described by reduced models. However, core and edge simulations themselves involve significant coupling of different processes with large scale differences. Numerical treatment of coupling is impacted by a number of factors including, scale differences, form of information transferred between processes, implementation of solvers for different codes, and high performance computing concerns. Operator decomposition involving the computation of the individual processes individually using appropriate simulation codes and then linking/synchronizing the component simulations at regular points in space and time, is the defacto approach to high performance simulation of multiphysics, multiscale systems. Various forms of operator decomposition are used in nearly all fusion simulations. However, operator decomposition generally has a complex effect on accuracy and stability of numerical simulations. Yet, these effects can be difficult to detect. The Colorado State University component of the FACETS team led by P.I.D. Estep was focused on analyzing the effects of operator decomposition on fusion simulations. The approach was based on a posteriori error analysis employing adjoint problems, computable residuals, and variational analysis to produce accurate computational error estimates for quantities of interest. Computable residuals are used to quantify the effects of various discretization choices. The generalized Greens function satisfying the adjoint problem quantities the effects of stability. Technical issues to be addressed included: (1) defining appropriate adjoint operators for operator decomposition discretizations; (2) determining the appropriate residuals for the multifaceted aspects involved with multiphysics discretizations; (3) producing the estimates within the computational framework of existing fusion codes; (4) carrying out the analysis for discretizations used in fusion simulations; and (5) devising efficient approaches to mitigating the effects of discretization. This report provides a summary of the accomplished research and a detailed description of personnel, activities, outcomes and achievements.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Multithreaded parallelization of the energy and analytic gradient in the fragment molecular orbital method

The fragment molecular orbital method in GAMESS is parallelized in a multithreaded OpenMP implementation combined with the MPI version of the two-level generalized distributed data interface. The energy and analytic gradient in gas phase and the polarizable continuum model of solvation are parallelized in this hybrid three-level scheme, achieving a large memory footprint reduction and a high parallel efficiency on Intel Xeon Phi processors. The parallel efficiency is demonstrated on the Stampede2 and Theta supercomputers using up to 2048 nodes (262 144 threads).

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Counting Kmers for Biological Sequences at Large Scale

Counting the abundance of all the distinct kmers in biological sequence data is a fundamental step in bioinformatics. These applications include de novo genome assembly, error correction, etc. With the development of sequencing technology, the sequence data in a single project can reach Petabyte-scale or Terabyte-scale nucleotides. Counting demand for the abundance of these sequencing data is beyond the memory and computing capacity of single computing node, and how to process it efficiently is a challenge on a high-performance computing cluster. As such, we propose SWAPCounter, a highly scalable distributed approach for kmer counting. This approach is embedded with an MPI streaming I/O module for loading huge data set at high speed, and a counting bloom filter module for both memory and communication efficiency. By overlapping all the counting steps, SWAPCounter achieves high scalability with high parallel efficiency. In this work, the experimental results indicate that SWAPCounter has competitive performance with two other tools on shared memory environment, KMC 2 , and MSPKmerCounter. Moreover, SWAPCounter also shows the highest scalability under strong scaling experiments. In our experiment on Cetus supercomputer, SWAPCounter scales to 32,768 cores with 79% parallel efficiency (using 2048 cores as baseline) when processing 4 TB sequence data of 1000 Genomes. The source code of SWAPCounter is publicly available at https://github.com/mengjintao/SWAPCounter.

59 BASIC BIOLOGICAL SCIENCES↗

Evaluation of Low Power ARM Processors for Monte Carlo Particle Transport: MCNP6 on the ARM Cortex-A8 - Paper 83

In the push towards exascale computing, the limiting factors are power requirements and heat dissipation. Using current technology, the power required to drive an exascale supercomputer would near 400-MW. Clearly, future computing hardware will have to be more energy efficient in order to achieve exascale performance. Most of the current research in advanced computing architectures is in the use of standard CPUs with accelerators. However, these accelerators are not generally supported by the traditional scientific programming languages (C, C++, and Fortran) and the scientific software must be tailored specifically to take advantage of the accelerators. Large scale scientific software (hundreds-of-thousand to millions of lines of code) can not be easily ported to the computing platform du jour. The widespread use of low power ARM based processors in the mobile computing market offer a low power alternative to accelerators that are also supported by the traditional scientific programming languages and programming paradigms such as MPI and OpenMP. Computing architectures for scientific computing are often assessed using standard benchmarks such as LINPACK benchmark or using suites of mini-applications that are greatly simplified stand-ins for scientific software. Often the real applications that will deployed on the scientific supercomputers are not ported or tested until delivery of the machines. In order to evaluate the use of ARM processors for scientific computing the Monte Carlo particle transport code, MCNP version 6 was compiled on the ARM Cortex-A8 and a set of benchmark problems were used to assess performance in comparison to an Intel Xeon E5-2670 and the low power Intel Atom N270. MCNP is a continuous energy Monte Carlo particle transport code in development at Los Alamos National Laboratory since 1977. MCNP is used to transport neutrons, photon, electrons, and charged particles using continuous energy interaction data. It is commonly used in fields of nuclear power, nuclear medicine, radiation protection, and high-energy physics through-out the world. MCNP version 6 is approximately 450 thousand lines of code, written primary in Fortran90, with some C and C++. MCNP supports distributed multiprocessing using MPI and threading with OpenMP. The ARM Cortex-A8 executes MCNP6 using about twice as much power as the Intel Xeon E5-2670 and about the same amount of power as the Intel Atom N270. However, these results are encouraging for the use of low-power ARM chips in future supercomputers. The ARM Cortex-A8 is a 32-bit CPU and it's NEON general-purpose SIMD (single-instruction multiple data) engine does not support double-precision floating-point arithmetic. More recent ARM processors (ARM Cortex-A53 and Cortex-A57) support 64-bit addressing and have NEON co-processors that support double-precision floating-point arithmetic. ARM claims that the A50 series delivers a performance increase of 15 times the Cortex-A8 performance for similar power usage. (authors)

61 RADIATION PROTECTION AND DOSIMETRY↗

GASNet-EX RMA Communication Performance on Recent Supercomputing Systems

Partitioned Global Address Space (PGAS) programming models, typified by systems such as Unified Parallel C (UPC) and Fortran coarrays, expose one-sided Remote Memory Access (RMA) communication as a key building block for High Performance Computing (HPC) applications. Architectural trends in supercomputing make such programming models increasingly attractive, and newer, more sophisticated models such as UPC++, Legion and Chapel that rely upon similar communication paradigms are gaining popularity. GASNet-EX is a portable, open-source, high-performance communication library designed to efficiently support the networking requirements of PGAS runtime systems and other alternative models in emerging exascale machines. The library is an evolution of the popular GASNet communication system, building upon 20 years of lessons learned. We present microbenchmark results which demonstrate the RMA performance of GASNet-EX is competitive with MPI implementations on four recent, high-impact, production HPC systems. These results are an update relative to previously published results on older systems. The networks measured here are representative of hardware currently used in six of the top ten fastest supercomputers in the world, and all of the exascale systems on the U.S. DOE road map.

Hargrove, Paul H↗

Reconciling Scratch Space Consumption, Exposure, and Volatility to Achieve Timely Staging of Job Input Data

Innovative scientific applications and emerging dense data sources are creating a data deluge for high-end computing systems. Processing such large input data typically involves copying (or staging) onto the supercomputer's specialized high-speed storage, scratch space, for sustained high I/O throughput. The current practice of conservatively staging data as early as possible makes the data vulnerable to storage failures, which may entail re-staging and consequently reduced job throughput. To address this, we present a timely staging framework that uses a combination of job startup time predictions, user-specified intermediate nodes, and decentralized data delivery to coincide input data staging with job start-up. By delaying staging to when it is necessary, the exposure to failures and its effects can be reduced. Evaluation using both PlanetLab and simulations based on three years of Jaguar (No. 1 in Top500) job logs show as much as 85.9% reduction in staging times compared to direct transfers, 75.2% reduction in wait time on scratch, and 2.4% reduction in usage/hour.

99 GENERAL AND MISCELLANEOUS↗

Massively parallel quantum chemical density matrix renormalization group method

Here, we present, to the best of our knowledge, the first attempt to exploit the supercomputer platform for quantum chemical density matrix renormalization group (QCDMRG) calculations. We have developed the parallel scheme based on the in-house MPI global memory library, which combines operator and symmetry sector parallelisms, and tested its performance on three different molecules, all typical candidates for QC-DMRG calculations. In case of the largest calculation, which is the nitrogenase FeMo cofactor cluster with the active space comprising 113 electrons in 76 orbitals and bond dimension equal to 6000, our parallel approach scales up to approximately 2000 CPU cores.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

fftMPI, a library for performing 2d and 3d FFTs in parallel

The fftMPI library enables two and three dimensional Fast Fourier Transforms (FFTs) to be performed efficiently in parallel on a variety of platforms from desktop machines to supercomputers. It allows for flexibility in the initial distribution of the FFT grid across processors, as well as the final output. It handles the data communication and reordering necessary to perform multi-dimensional FFTs in parallel. It relies on open-source or vendor-provided libraries to perform one dimensional FFTs on a single processor. It uses MPI for inter-processor communication, which is a standard distributed memory message-passing library.

Plimpton, Steven↗

Towards reversible basic linear algebra subprograms: A performance study

Problems such as fault tolerance and scalable synchronization can be efficiently solved using reversibility of applications. Making applications reversible by relying on computation rather than on memory is ideal for large scale parallel computing, especially for the next generation of supercomputers in which memory is expensive in terms of latency, energy, and price. In this direction, a case study is presented here in reversing a computational core, namely, Basic Linear Algebra Subprograms, which is widely used in scientific applications. A new Reversible BLAS (RBLAS) library interface has been designed, and a prototype has been implemented with two modes: (1) a memory-mode in which reversibility is obtained by checkpointing to memory in forward and restoring from memory in reverse, and (2) a computational-mode in which nothing is saved in the forward, but restoration is done entirely via inverse computation in reverse. The article is focused on detailed performance benchmarking to evaluate the runtime dynamics and performance effects, comparing reversible computation with checkpointing on both traditional CPU platforms and recent GPU accelerator platforms. For BLAS Level-1 subprograms, data indicates over an order of magnitude better speed of reversible computation compared to checkpointing. For BLAS Level-2 and Level-3, a more complex tradeoff is observed between reversible computation and checkpointing, depending on computational and memory complexities of the subprograms.

97 MATHEMATICS AND COMPUTING↗

Development of mpi_EPIC model for global agroecosystem modeling

Models that address policy-maker concerns about multi-scale effects of food and bioenergy production systems are computationally demanding. We integrated the message passing interface algorithm into the process-based EPIC model to accelerate computation of ecosystem effects. Simulation performance was further enhanced by applying the Vampir framework. When this enhanced mpi_EPIC model was tested, total execution time for a global 30-year simulation of a switchgrass cropping system was shortened to less than 0.5 hours on a supercomputer. The results illustrate that mpi_EPIC using parallel design can balance simulation workloads and facilitate large-scale, high-resolution analysis of agricultural production systems, management alternatives and environmental effects.

54 ENVIRONMENTAL SCIENCES↗

Porting Ordinary Applications to Blue Gene/Q Supercomputers

Efficiently porting ordinary applications to Blue Gene/Q supercomputers is a significant challenge. Codes are often originally developed without considering advanced architectures and related tool chains. Science needs frequently lead users to want to run large numbers of relatively small jobs (often called many-task computing, an ensemble, or a workflow), which can conflict with supercomputer configurations. In this paper, we discuss techniques developed to execute ordinary applications over leadership class supercomputers. We use the high-performance Swift parallel scripting framework and build two workflow execution techniques-sub-jobs and main-wrap. The sub-jobs technique, built on top of the IBM Blue Gene/Q resource manager Cobalt's sub-block jobs, lets users submit multiple, independent, repeated smaller jobs within a single larger resource block. The main-wrap technique is a scheme that enables C/C++ programs to be defined as functions that are wrapped by a high-performance Swift wrapper and that are invoked as a Swift script. We discuss the needs, benefits, technicalities, and current limitations of these techniques. We further discuss the real-world science enabled by these techniques and the results obtained.

Maheshwari, Ketan C.↗

Community Detection on the GPU

We present and evaluate a new GPU algorithm based on the Louvain method for community detection. Our algorithm is the first for this problem that parallelizes the access to individual edges. In this way we can fine tune the load balance when processing networks with nodes of highly varying degrees. This is achieved by scaling the number of threads assigned to each node according to its degree. Extensive experiments show that we obtain speedups up to a factor of 270 compared to the sequential algorithm. The algorithm consistently outperforms other recent shared memory implementations and is only one order of magnitude slower than the current fastest parallel Louvain method running on a Blue Gene/Q supercomputer using more than 500K threads.

community detection↗

Cyber Dumpster Diving: Creating New Software Systems for Less

Modern scientific software is daunting in its breadth and complexity. From massively parallel simulations running on the world’s largest supercomputers to visualizations and user support environments that manage ever-growing, complex data collections, the challenges for software engineers are plentiful indeed.

software architecture↗

ExaMPM

ExaMPM is a mini-application for the Material Point Method (MPM) for studying the application of MPM to future exascale computing systems. MPM is a general method for computational mechanics and fluids and is used in a wide variety of science and engineering disciplines to study problems with large deformations, phase change, fracture, and other phenomena. ExaMPM provides a reference implementation of MPM as described in the 1994 work of Sulsky et.al. (Sulsky, Deborah, Zhen Chen, and Howard L. Schreyer. "A particle method for history-dependent materials." Computer methods in applied mechanics and engineering 118.1-2 (1994): 179-196.). The software can solve basic MPM problems in solid mechanics using the original algorithm of Sulsky with explicit time integration, basic geometries, and free-slip and no-slip boundary conditions as described in the reference. ExaMPM is intended to be used as a starting point to design new parallel algorithms for the next generation of DOE supercomputers.

Slattery, StuartR↗