Experimental Physics Software and Computing Infrastructure
EPSCI Group activities including Hydra, AIEC, AIOP, AIOPP, EJFAT, ERSAP, and JANA.
Lawrence, David↗
Engineering topics
Brei, Nathan
Publications and source records attributed to Brei, Nathan.
Towards High-Performance AI4NP Applications on Modern GPU Platforms
The evolution of modern heterogeneous accelerators, such as GPUs, has significantly advanced the landscape of artificial intelligence (AI). There is a notable surge to adopt AI within the nuclear physics domain (AI4NP). While most AI4NP studies focus on feasibility analysis, our attention is directed towards evaluating their performance on contemporary GPUs that integrate tensor cores. We first benchmark the throughput of hyperparameterized multi-layer perceptron (MLP) models. We then examine the performance of an AI4NP application: Hydra. We assess the performance gain and accuracy loss caused by the tensor cores for low-precision floating-point operations. Our experiments encompass the PyTorch and TensorFlow Keras frameworks on NVIDIA’s T4 and A100 GPUs. We explore the behavior of different GPU hardware platforms and AI software tools. This study can be a valuable resource for guiding the performance optimization of larger-scale deployments of AI4NP applications.
Mei, Xinxin↗
PHASM: A Toolkit for Creating AI Surrogate Models within Legacy Codebases
PHASM (“Parallel Hardware viA Surrogate Models”) is a software toolkit for creating AI-based surrogate models of scientific code. AI-based surrogate models are widely used for creating fast and inverse simulations. PHASM anticipates an additional future use case: adapting legacy code to modern hardware. While data centers are investing in heterogeneous hardware such as GPUs and FPGAs, many established scientific codebases remain unable to take advantage of the hardware’s higher parallelism without undergoing a costly rewrite. An alternative is to train a AI-based surrogate model to mimic computationally intensive functions in the code, and run the surrogate instead. PHASM formalizes a development lifecycle for such surrogate models, including discovering functions amenable to replacement with a surrogate model, predicting the resulting performance, identifying the function’s space of inputs and outputs, binding the model to the code, and managing model versions. A suite of software tools for facilitating these steps was written and validated against a set of model problems.
97 MATHEMATICS AND COMPUTING↗
Streaming readout for next generation electron scattering experiments
Current and future experiments at the high-intensity frontier are expected to produce an enormous amount of data that needs to be collected and stored for offline analysis. Thanks to the continuous progress in computing and networking technology, it is now possible to replace the standard ‘triggered’ data acquisition systems with a new, simplified and outperforming scheme. ‘Streaming readout’ (SRO) DAQ aims to replace the hardware-based trigger with a much more powerful and flexible software-based one, that considers the whole detector information for efficient real-time data tagging and selection. Considering the crucial role of DAQ in an experiment, validation with on-field tests is required to demonstrate SRO performance. In this paper, we report results of the on-beam validation of the Jefferson Lab SRO framework. In this work, we exposed different detectors (PbWO-based electromagnetic calorimeters and a plastic scintillator hodoscope) to the Hall-D electron-positron secondary beam and to the Hall-B production electron beam, with increasingly complex experimental conditions. By comparing the data collected with the SRO system against the traditional DAQ, we demonstrate that the SRO performs as expected. Furthermore, we provide evidence of its superiority in implementing sophisticated AI-supported algorithms for real-time data analysis and reconstruction.
47 OTHER INSTRUMENTATION↗
Streaming Readout of the CLAS12 Forward Tagger Using TriDAS and JANA2
An effort is underway to develop streaming readout data acquisition system for the CLAS12 detector in Jefferson Lab’s experimental Hall-B. Successful beam tests were performed in the spring and summer of 2020 using a 10GeV electron beam from Jefferson Lab’s CEBAF accelerator. The prototype system combined elements of the TriDAS and CODA data acquisition systems with the JANA2 analysis/reconstruction framework. This successfully merged components that included an FPGA stream source, a distributed hit processing system, and software plugins that allowed offline analysis written in C++ to be used for online event filtering. Details of the system design and performance are presented.
Ameli, Fabrizio↗
JANA2 LDRD (Final Project Report)
The project sought to develop a second generation of the JANA reconstruction framework that leverages modern programming features (C++11 and C++14) while maintaining its basic user centric design. The goals included making the project easy to deploy to large HPC facilities such as NERSC and to include detailed documentation to allow scientists outside of Jefferson Lab to use it at such a facility. This would position JLab to play a key role in the larger Laboratory based computing ecosystem under the ASCR initiative. The experience gained will help inform the design of Exascale class applications for experimental nuclear physics. Developing a next generation JANA provides an advantage over older serial frameworks or ones retro-fitted to provide some multi-threading support. That technique requires significantly more memory than multi-threading which leads to compute jobs being limited by available RAM, and not fully utilizing the available CPU. The smaller memory footprint of multi-threaded programs also leads to increased efficiency due to fewer cache misses. It is for these and other reasons that multi-threading is the more accepted industry standard in the modern era.
73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
JANA2 Framework for Event Based and Triggerless Data Processing
Development of the second generation JANA2 multi-threaded event processing framework is ongoing through an LDRD initiative grant at Jefferson Lab. The framework is designed to take full advantage of all cores on modern many-core compute nodes. JANA2 efficiently handles both traditional hardware triggered event data and streaming data in online triggerless environments. Development is being done in conjunction with the Electron Ion Collider development. Anticipated to be the next large scale Nuclear Physics facility constructed. The core framework is written in modern C++ but includes an integrated Python interface. The status of development and summary of the more interesting features are presented.
Lawrence, David↗
Multi-threaded Event Reconstruction with JANA
JANA2 is a multi-threaded event reconstruction framework being developed for Experimental Nuclear Physics. It is an LDRD funded project that will be the successor of the original JANA framework. JANA2 is a near complete rewrite emphasizing C++ language features that have only become available since the C++11 standard. features such as shared pointers, language native threading, and atomics. This paper outlines the status of the project.
Lawrence, David↗