Engineering Papers⌕ Search

Engineering topics

Jun, Soon Yung

Publications and source records attributed to Jun, Soon Yung.

Celeritas: Accelerating Geant4 with GPUs

Celeritas [1] is a new Monte Carlo (MC) detector simulation code designed for computationally intensive applications (specifically, High Lumi- nosity Large Hadron Collider (HL-LHC) simulation) on high-performance heterogeneous architectures. In the past two years Celeritas has advanced from prototyping a GPU-based single physics model in infinite medium to implementing a full set of electromagnetic (EM) physics processes in complex geometries. The current release of Celeritas, version 0.3, has incorporated full device-based navigation, an event loop in the presence of magnetic fields, and detector hit scoring. New functionality incorporates a scheduler to offload electromagnetic physics to the GPU within a Geant4-driven simulation, enabling integration of Celeritas into high energy physics (HEP) experimental frameworks such as CMSSW. On the Summit supercomputer, Celeritas performs EM physics between 6 and 32 faster using the machine’s Nvidia GPUs compared to using only CPUs. When running a multithreaded Geant4 ATLAS test beam application with full hadronic physics, using Celeritas to accelerate the EM physics results in an overall simulation speedup of 1.8–2.3× on GPU and 1.2× on CPU.

Johnson, Seth R.↗

Celeritas R&D Report: Accelerating Geant4

Celeritas is a new Monte Carlo (MC) detector simulation code designed for computationally intensive applications on high-performance heterogeneous architectures. In the past two years Celeritas has advanced from prototyping a Graphics Processing Unit (GPU)-based single physics model in infinite medium to implementing a full set of electromagnetic (EM) physics processes in complex geometries. The current release of Celeritas, version 0.4, has incorporated full device-based navigation, an event loop in the presence of magnetic fields, and detector hit scoring. New functionality incorporates a scheduler to offload electromagnetic physics to the GPU within a Geant4-driven simulation, enabling straightforward integration of Celeritas into the high energy physics (HEP) experimental frameworks CMSSW and ATLAS FullSimLight. On the Perlmutter supercomputer, Celeritas performs EM physics between 3× and 18× faster using the machine’s Nvidia GPUs compared to using only CPUs, corresponding to an electrical power efficiency up to a factor of 5. When running a multithreaded Geant4 ATLAS test beam application with full hadronic physics, using Celeritas to accelerate the EM physics results in an overall simulation speedup of 1.7–2.2× on GPU and 1.2× on CPU. In a CMS test application using tt¯ events and the prototype Run 4 configuration, compared to Geant4 CPU, Celeritas with a Nvidia A100 improves overall throughput up to a factor of 2.7× but cannot be efficiently shared with more than 8 cores.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

CaTS: Integration of Geant4 and Opticks

CaTS [6]is an advanced example that is part of Geant4 since version 11.0. It demonstrates the use of Opticks to offload the simulation of optical photons to GPUs. Opticks interfaces with the Geant4 toolkit to collect all the necessary information to generate and trace optical photons, re-implements the optical physics processes to be run on the GPU, and automatically translates the Geant4 geometry into a GPU appropriate format. To trace the photons, Opticks uses NVIDIA OptiX®. In this report, we describe CaTS and the integration of Opticks with Geant4. We demonstrate that the generation and tracing of optical photons represents an ideal application to be offloaded to GPUs, fully utilizing the high degree of available parallelism. In a typical liquid argon TPC simulation, a speedup of several hundred times is observed compared to an equivalent simulation using single threaded Geant4.

Wenzel, Hans↗

Optimizing Geant4 Hadronic Models

Geant4, the leading detector simulation toolkit used in high energy physics, employs a set of physics models to simulate interactions of particles with matter across a wide range of energies. These models, especially the hadronic ones, rely largely on directly measured cross-sections and inclusive characteristics, and use physically motivated parameters. However, they generally aim to cover a broad range of possible simulation tasks and may not always be optimized for a particular process or a given material. The Geant4 collaboration recently made many parameters of the models accessible via a configuration interface. This opens a possibility to fit simulated distributions to the thin target experimental datasets and extract optimal values of the model parameters and the associated uncertainties. Such efforts are currently undertaken by the Geant4 collaboration with the goal of offering alternative sets of model parameters, also known as "tunes", for certain applications. The effort should subsequently lead to more accurate estimates of the systematic errors in physics measurements given the detector simulation role in performing the physics measurements. Results of the study are presented to illustrate how Geant4 model parameters can be optimized through applying fitting techniques, to improve the agreement between the Geant4 and the experimental data.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Celeritas

Celeritas is a new Monte Carlo transport code designed for high-performance simulation of high-energy physics detectors.

Johnson, Seth R.↗

Novel features and GPU performance analysis for EM particle transport in the Celeritas code

Celeritas is a new computational transport code designed for high-performance simulation of high-energy physics detectors. This work describes some of its current capabilities and the design choices that enable the rapid development of efficient on-device physics. The abstractions that underpin the code design facilitate low-level performance tweaks that require no changes to the higher-level physics code. We evaluate a set of independent changes that together yield an almost 40% speedup over the original GPU code for a net performance increase of 220× for a single GPU over a single CPU running 8.4M tracks on a small demonstration physics app.

Johnson, Seth R.↗

GQLink: an implementation of Quantized State Systems (QSS) methods in Geant4

Simulations in high energy physics (HEP) often require the numerical solution of ordinary differential equations (ODE) to determine the trajectories of charged particles in a magnetic field when particles move throughout detector volumes. Each crossing of a volume interrupts the underlying numerical method that solves the equations of motion, triggering iterative algorithms to estimate the intersection point within a given accuracy. The computational cost of this procedure can grow significantly depending on the application at hand. Quantized State System (QSS) is a recent family of discrete-event driven numerical methods exhibiting attractive features for this type of problems, such as native dense output (sequences of polynomial segments updated only by accuracy-driven events) and lightweight detection and handling of volume crossings. In this work we present GQLink, a proof-of-concept integration of QSS with the Geant4 simulation toolkit which stands as an interface for co-simulation that orchestrates robustly and transparently the interaction between the QSS simulation engine and aspects such as geometry definition and physics processes controlled by Geant4. In conclusion, we validate the accuracy and study the performance of the method in simple geometries (subject to intense volume crossing activity) and then in a realistic HEP application using a full CMS detector configuration.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Application of State Quantization-Based Methods in HEP Particle Transport Simulation

Simulation of particle-matter interactions in complex geometries is one of the main tasks in high energy physics (HEP) research. An essential aspect of it is an accurate and efficient particle transportation in a non-uniform magnetic field, which includes the handling of volume crossings within a predefined 3D geometry. Quantized State Systems (QSS) is a family of numerical methods that provides attractive features for particle transportation processes, such as dense output (sequences of polynomial segments changing only according to accuracy-driven discrete events) and lightweight detection and handling of volume crossings (based on simple root-finding of polynomial functions). In this work we present a proof-of-concept performance comparison between a QSS-based standalone numerical solver and an application based on the Geant4 simulation toolkit, with its default Runge-Kutta based adaptive step method. In a case study with a charged particle circulating in a vacuum (with interactions with matter turned off), in a uniform magnetic field, and crossing up to 200 volume boundaries twice per turn, simulation results showed speedups of up to 6 times in favor of QSS while it being 10 times slower in the case with zero volume boundaries.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗