Design and Evaluation of GPU-FPX: A Low-Overhead tool forFloating-Point Exception Detection in NVIDIA GPUs
Explore the source record for details and available documents.
Computer science↗
SEARCH · Engineering Papers
Results for “hardware accelerators”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Machine learning for collective variable discovery and enhanced sampling in biomolecular simulation
Classical molecular dynamics simulates the time evolution of molecular systems through the phase space spanned by the positions and velocities of the constituent atoms. Molecular-level thermodynamic, kinetic, and structural data extracted from the resulting trajectories provide valuable information for the understanding, engineering, and design of biological and molecular materials. The cost of simulating many-body atomic systems makes simulations of large molecules prohibitively expensive, and the high-dimensionality of the resulting trajectories presents a challenge for analysis. Driven by advances in algorithms, hardware, and data availability, there has been a flare of interest in recent years in the applications of machine learning – especially deep learning – to molecular simulation. Furthermore, these techniques have demonstrated great power and flexibility in both extracting mechanistic understanding of the important nonlinear collective variables governing the dynamics of a molecular system, and in furnishing good low-dimensional system representations with which to perform enhanced sampling or develop long-timescale dynamical models. It is the purpose of this article to introduce the key machine learning approaches, describe how they are married with statistical mechanical theory into domain-specific tools, and detail applications of these approaches in understanding and accelerating biomolecular simulation.
74 ATOMIC AND MOLECULAR PHYSICS↗
MIND-MAC: Multi-Level In-memory Quasi Non-Destructive MAC Operation in Compact 2T-nC FeRAM for Efficient DNN Accelerator
We present MIND-MAC, a compact 2T-nC FeRAM architecture that performs multi-level, quasi-non-destructive in-memory multiply–accumulate (MAC) for deep neural networks. By exploiting voltage-controlled partial domain switching in MFM capacitors and read-transistor amplification, the cell stores multi-bit weights and gates bit-serial inputs to produce an accumulated current on shared lines. We combine TCAD-extracted parasitics with experimentally calibrated ferroelectric models in SPICE to validate device-/circuit-level behavior, and validate multi-level sensing and QNRO with measurements on a fabricated 2T-3C test vehicle. An analytical system model maps MIND-MAC to a 6-GB main-memory in-memory compute (IMC) architecture and benchmarks VGG13 inference in 61.08 ms at 964.99 mJ. Results indicate high density, reduced rewrite overhead, and energy efficiency, positioning 2T-nC FeRAM as a promising IMC candidate for next-generation AI hardware.
36 MATERIALS SCIENCE↗
EXAGRAPH: Graph and combinatorial methods for enabling exascale applications
Combinatorial algorithms in general and graph algorithms in particular play a critical enabling role in numerous scientific applications. However, the irregular memory access nature of these algorithms makes them one of the hardest algorithmic kernels to implement on parallel systems. With tens of billions of hardware threads and deep memory hierarchies, the exascale computing systems in particular pose extreme challenges in scaling graph algorithms. The codesign center on combinatorial algorithms, ExaGraph, was established to design and develop methods and techniques for efficient implementation of key combinatorial (graph) algorithms chosen from a diverse set of exascale applications. Algebraic and combinatorial methods have a complementary role in the advancement of computational science and engineering, including playing an enabling role on each other. In this paper, we survey the algorithmic and software development activities performed under the auspices of ExaGraph from both a combinatorial and an algebraic perspective. In particular, we detail our recent efforts in porting the algorithms to manycore accelerator (GPU) architectures. We also provide a brief survey of the applications that have benefited from the scalable implementations of different combinatorial algorithms to enable scientific discovery at scale. We believe that several applications will benefit from the algorithmic and software tools developed by the ExaGraph team.
97 MATHEMATICS AND COMPUTING↗
OpenACC offloading of the MFC compressible multiphase flow solver on AMD and NVIDIA GPUs
GPUs are the heart of the latest generations of supercomputers. We efficiently accelerate a compressible multiphase flow solver via OpenACC on NVIDIA and AMD Instinct GPUs. Optimization is accomplished by specifying the directive clauses gang vector and collapse. Further speedups of six and ten times are achieved by packing user-defined types into coalesced multidimensional arrays and manual inlining via metaprogramming. Additional optimizations yield seven-times speedup of array packing and thirty-times speedup of select kernels on Frontier. Weak scaling efficiencies of 97% and 95% are observed when scaling to 50% of Summit and 87% of Frontier. Strong scaling efficiencies of 84% and 81% are observed when increasing the device count by a factor of 8 and 16 on V100 and MI250X hardware. The strong scaling efficiency of AMD’s MI250X increases to 92% when increasing the device count by a factor of 16 when GPU-aware MPI is used for communication.
Wilfong, Benjamin↗
Dish-STARS Commercialization (Final Report)
The goal of this project is to aggressively support the near-term commercialization of a new technology platform – based on the integration of solar concentrators and micro- and meso-channel process technology (MMPT) – that was evaluated and identified as a strong candidate for near-term commercialization at EERE’s inaugural Lab-Corps program during early FY2016. Known as STARS, for Solar Thermochemical Advanced Reactor System, or Dish-STARS TM when paired with parabolic dish concentrators, STARS is a promising energy-related technology developed at the Pacific Northwest National Laboratory (PNNL) that efficiently converts solar energy into chemical energy. Combined with economies through hardware mass production, the efficiency of Dish-STARS TM provides a near-term opportunity for the production of renewable electricity, fuels and chemicals. The project supported the cooperative development of Dish-STARSTM by PNNL and industry partners including California Gas Company (SoCalGas) and the startup company, STARS Technology Corporation (STC), which was founded by the PNNL Lab-Corps team that evaluated STARS on behalf of EERE. Under this project, the team advanced the Technology Readiness Level 6 (TRL 6) STARS reaction system, developed under the previous DOE SunShot project, to TRL 7 through on-sun testing in California by PNNL. The advances made in this project enabled STC to accelerate commercial development and initiate work towards a major technology demonstration in California for a hydrogen filling station application.
14 SOLAR ENERGY↗
Design and Characterization of the 162.5 MHz RF Component Layout for Fermilab's PIP-II Reference Line
The Proton Improvement Plan II (PIP-II) Reference Line at Fermi National Accelerator Laboratory distributes phase-stable radio-frequency (RF) signals throughout the accelerator. PIP-II requires stable timing and phase reference signals so its accelerating cavities transfer energy to the particle beam at the correct point in each RF cycle. The full system includes 162.5, 325, and 650 MHz sections corresponding to the frequency sections of the PIP-II Linac, with this project focusing on the 162.5 MHz section. The Reference Line must provide a phase stable source signal while responding to phase changes caused by environmental conditions or system drift. Its RF components will be mounted on aluminum heat plates inside a temperature controlled enclosure to further limit temperature-driven phase changes. To prepare the system for manufacture, the project reviewed component functions and dimensions, developed a computer-aided design (CAD) model, and arranged the hardware to support short cable paths, grounding, fastener access, and maintenance. Several full scale, three dimensional printed prototypes allowed the available components to be mounted and inspected. These fit checks revealed mechanical conflicts and guided revisions to component placement, countersink geometry, labeling, and plate thickness. Electrical characterization was also performed on selected RF hardware to compare its measured behavior with the performance metrics that were set for our design. Overall, the project produced a manufacturable 162.5 MHz layout, physical fit-check prototypes, and documented electrical measurements that support review before metal fabrication. The 325 and 650 MHz layouts remain future work because they require additional minor mechanical changes.
Subedi, Harsheet [Unlisted, US, CA] (ORCID:0009000↗
Towards High-Performance AI4NP Applications on Modern GPU Platforms
The evolution of modern heterogeneous accelerators, such as GPUs, has significantly advanced the landscape of artificial intelligence (AI). There is a notable surge to adopt AI within the nuclear physics domain (AI4NP). While most AI4NP studies focus on feasibility analysis, our attention is directed towards evaluating their performance on contemporary GPUs that integrate tensor cores. We first benchmark the throughput of hyperparameterized multi-layer perceptron (MLP) models. We then examine the performance of an AI4NP application: Hydra. We assess the performance gain and accuracy loss caused by the tensor cores for low-precision floating-point operations. Our experiments encompass the PyTorch and TensorFlow Keras frameworks on NVIDIA’s T4 and A100 GPUs. We explore the behavior of different GPU hardware platforms and AI software tools. This study can be a valuable resource for guiding the performance optimization of larger-scale deployments of AI4NP applications.
Mei, Xinxin↗
Using a Large Language Model as a Building Block to Generate Usable Validation and Verification Suite for OpenMP
In the HPC area, both hardware and software move quickly. Often new hardware is developed and deployed, the corresponding software stack, including compilers and other tools, are under active development while leading edge software developers are working to port and tune their applications, all at the same time. While the software ecosystem is in flux, one of the key challenges for users is obtaining insight into the state of implementation of key features in the programming languages and models their applications are using – whether they have been implemented, and whether the implementation conforms to the specification, especially for newly implemented features (less tested by widespread use). OpenMP is one of the most prominent shared memory programming models used for on-node programming in HPC. With the shift towards accelerators (such as GPUs and FPGAs) and heterogeneous programming OpenMP features are getting more complex. It is natural to ask whether generative AI approaches, and large language models (LLMs) in particular, can help in producing validation and verification test suites to allow users better and faster insights into the availability and correctness of OpenMP features of interest. In this work, we explore the use of ChatGPT-4 to generate a suite of tests for OpenMP features. We have chosen a set of directives and clauses, a total of 78 combinations, which first appeared in OpenMP 3.0 (released in May 2008) but are also relevant for accelerators. We prompted ChatGPT to generate tests in the C and Fortran languages, for both host (CPU) and device (accelerator). On the Summit super-computer using the GNU implementation, we found that, of the 78 generated tests 67 C tests and 43 Fortran tests compiled successfully and fewer than those executed to completion. On further analysis we show that not all generated tests are valid. We document the process, results, and provide detailed analysis regarding the quality of tests generated. With the aim of providing input to a production quality validation and verification suite, we manually implement the corrections required to make the tests valid according to the current OpenMP specification. We quantify this effort as small, medium, or large, and record the lines of code changed to correct the invalid tests. With the corrected tests we validate recent implementations from HPE, AMD, and GNU on the Frontier supercomputer. Our experiment and subsequent analysis show that although LLMs are capable of producing HPC specific codes, they are limited by their understanding of the deeper semantics and restrictions of programming models such as OpenMP. Unsurprisingly more commonly used features have better support, while some OpenMP 3.0 directives such as sections and tasking are not universally supported on accelerators. We demonstrate that successful compilation and execution to completion are inadequate metrics for evaluating generated code and that, at this time, commodity LLMs require expert intervention for code verification. This points to gaps in the training data that is currently available for HPC. We demonstrate that with "small" effort 37% of generated invalid C tests and 63% of generated invalid Fortran tests could be corrected. This improves productivity of test generation as we circumvent writing from scratch and the common programming errors associated with it.
Pophale, Swaroop [ORNL] (ORCID:0000000185446367)↗
Quantum Computing for Neutrino-Nucleus Scattering
Neutrino-nucleus cross section uncertainties are expected to be a dominant systematic in future accelerator neutrino experiments. The cross sections are determined by the linear response of the nucleus to the weak interactions of the neutrino, and are dominated by energy and distance scales of the order of the separation between nucleons in the nucleus. These response functions are potentially an important early physics application of quantum computers. Here we present an analysis of the resources required and their expected scaling for scattering cross section calculations. The current estimates of Trotter steps needed to achieve an energy resolution of 10 MeV and the number of CNOT gates for analyzing Ar40 highlights the need for significant improvements in algorithms. We also examine simple small-scale neutrino-nucleus models on modern quantum hardware. In this paper, we use variational methods to obtain the ground state of a three nucleon system (the triton) and then implement the relevant time evolution. To tame the errors in present-day NISQ devices, we explore the use of different error-mitigation techniques to increase the fidelity of the calculations.
72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
NISQ+: Boosting quantum computing power by approximating quantum error correction
Quantum computers are growing in size, and design decisions are being made now that attempt to squeeze more computation out of these machines. In this spirit, we design a method to boost the computational power of near-term quantum computers by adapting protocols used in quantum error correction to implement "Approximate Quantum Error Correction (AQEC)." By approximating fully-fledged error correction mechanisms, we can increase the compute volume (qubits × gates, or "Simple Quantum Volume (SQV)") of near-term machines. The crux of our design is a fast hardware decoder that can approximately decode detected error syndromes rapidly. Specifically, we demonstrate a proof-of-concept that approximate error decoding can be accomplished online in near-term quantum systems by designing and implementing a novel algorithm in Single-Flux Quantum (SFQ) superconducting logic technology. This avoids a critical decoding backlog, hidden in all offline decoding schemes, that leads to idle time exponential in the number of T gates in a program. Our design utilizes one SFQ processing module per physical qubit. Employing state-of-the-art SFQ synthesis tools, we show that the circuit area, power, and latency are within the constraints of contemporary quantum system designs. Under pure dephasing error models, the proposed accelerator and AQEC solution is able to expand SQV by factors between 3,402 and 11,163 on expected near-term machines. The decoder achieves a 5% accuracy-threshold and pseudo-thresholds of ~ 5%,4.75%,4.5%, and 3.5% physical error-rates for code distances 3,5,7, and 9. Decoding solutions are achieved in a maximum of ~20 nanoseconds on the largest code distances studied. By avoiding the exponential idle time in offline decoders, we achieve a 10x reduction in required code distances to achieve the same logical performance as alternative designs.
97 MATHEMATICS AND COMPUTING↗
Performance Portable Graphics Processing Unit Acceleration of a High-Order Finite Element Multiphysics Application
The Lawrence Livermore National Laboratory (LLNL) will soon have in place the El Capitan exascale supercomputer, based on advanced micro devices (AMD) graphics processing units (GPUs). As part of a multiyear effort under the National Nuclear Security Administration (NNSA) Advanced Simulation and Computing (ASC) program, we have been developing marbl, a next generation, performance portable multiphysics application based on high-order finite elements. In previous years, we successfully ported the Arbitrary Lagrangian–Eulerian (ALE), multimaterial, compressible flow capabilities of marbl to nvidia GPUs as described in Vargas et al. Here, in this paper, we describe our ongoing effort in extending marbl's GPU capabilities with additional physics, including multigroup radiation diffusion and thermonuclear burn for high energy density physics (HEDP) and fusion modeling. We also describe how our portability abstraction approach based on the raja Portability Suite and the mfem finite element discretization library has enabled us to achieve high performance on AMD based GPUs with minimal effort in hardware-specific porting. Throughout this work, we highlight numerical and algorithmic developments that were required to achieve GPU performance.
70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
Maximizing RHIC Deliverable Luminosity with Dynamic Telescopic Beta* Squeeze
To benefit fully from the exceptional performance of the Relativistic Heavy Ions Collider (RHIC) Stochastic Cooling system during heavy ions physics store, the betatron amplitude β ∗ at a given col lision point (IP) can be squeezed from its design value by using the available space in the quadrupole triplets from smaller beam emittances. This should allow reaching new heights in deliverable lumi nosity for both STAR and PHENIX experiments. Prior to 2014, RHIC lattice designs only squeezed the interaction regions (IR’s) down to β ∗ = 0.70 m. To get past this limitation, one can use a telescopic scheme where the non-experimental insertions surrounding STAR and PHENIX are used to generate and close a β-beat wave contributing to the final squeeze in the two IP’s of interest. The following presents the various steps of the implementation in RHIC of this telescopic scheme and the corresponding changes in collision rates and specific luminosity. A full feasibility study with a review of hardware and beam dynamics limitations is also included.
43 PARTICLE ACCELERATORS↗
Flexible Pilot Jobs Framework for Distributed High Throughput Computing
Experimental particle physics has been at the forefront of analyzing the world’s largest datasets for decades. The high-energy physics (HEP) community was among the first to develop suitable software and computing tools for this purpose. GlideinWMS is a Glidein-based workload management system whose purpose is to provide experiments like CMS at CERN, DUNE at Fermilab, and others, a way to access and efficiently use vast amounts of computing resources. This system wants to provide a simple way to submit jobs to a set of computing resources, that will be provided to users behind the scenes. Glideins are the pilot jobs executed on the worker nodes at the grid sites, performing operations such as hardware detection, environment setup, and error handling. After all these operations, they will launch the actual user job. Many grid sites are supported, such as shared clusters, Google CE, and AWS. My internship aimed to design and code a flexible pilot jobs framework that will replace the one used by GlideinWMS, developing a modular and flexible skeleton of the Glidein and adding further functionalities. My project also focused on the application of machine learning techniques as support to this management system.
72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
Real-Time Data Acquisition and Processing System for MHz Repetition Rate Image Sensors
An electro-optic detector is one of the diagnostic setups used in particle accelerators. It employs an electro-optic crystal to encode the longitudinal beam charge profile in the spectrum of a light pulse. The charge distribution is then reconstructed using data captured by a fast spectrometer. The measurement repetition rate should match or exceed the machine bunching frequency, which is often in the range of several MHz. A high-speed optical line detector (HOLD) is a linear camera designed for easy integration with scientific experiments. The use of modern FPGA circuits helps in the efficient collection and processing of data. The solution is based on Xilinx 7-Series FPGA circuits and implements a custom latency-optimized architecture utilizing the AXI4 family of interfaces. HOLD is one of the fastest line cameras in the world. Thanks to its hardware architecture and a powerful KALYPSO sensor from KIT, it outperforms the fastest comparable commercial devices.
MicroTCA↗
Design of Controller Hardware-In-the-Loop Model of Microgrid with Modular Building Blocks and Automated Design Script
The scalability of controller hardware-in-the-loop (CHIL) simulation is critical for validating control coordination and energy management in microgrids with distributed energy resources, especially as these modern systems become more complex and decentralized. This paper presents a CHIL modeling methodology that combines modular building blocks with an automated design script to streamline the development of high-fidelity microgrid models. Standardized subsystem templates for resources, converters, and buses are integrated with a Python-based script that compiles structured JSON configuration files into simulation-ready initialization code. The proposed approach reduces development time, improves model consistency, and enhances simulation fidelity. The methodology is validated on a Typhoon HIL604 platform and is broadly applicable to real-time simulation of complex, networked microgrid systems. This framework establishes a foundation for automated, scalable CHIL validation and accelerates the design of next-generation distributed energy systems.
Kim, Namwon [ORNL] (ORCID:0000000200438489)↗
Reducing module soiling with scalable and robust photocatalytic coatings
The air-glass interface at the front of a photovoltaic (PV) module reflects approximately 4% of incident light, decreasing the potential power output of the module by the same amount. Today’s modules reduce this loss by adding a low-refractive-index (1.25-1.30) SiO2 coating to the sunward side of the module glass; this antireflection coating recovers approximately 3% of the 4% light that would otherwise be lost. While such antireflection coatings work very well on clean, new modules, they do not inhibit soiling—the accumulation of soilants such as dust, pollen, soot, or other foreign material—on the module glass, and soilants reflect and scatter incident light. An improved coating would serve provide not only an antireflection effect, but also an anti-soiling effect. The goal of this project was to develop such a coating and provide a path for it to be manufactured in the U.S. The project successfully designed and fabricated coatings that provided >3% transmittance gain compared to bare glass (matching the performance of commercial antireflection coatings) and displayed anti-soiling behavior in standard laboratory soiling effects. This was achieved by using a Swift Coat proprietary coating deposition technique, aerosol impact-driven assembly (AIDA), to control the porosity and thus refractive index of coatings of photocatalytic materials—such as TiO2—that would otherwise increase (instead of decrease) reflection. These combined antireflection/anti-soiling coatings passed PV industry standard module reliability tests as well as coating-specific abrasion tests, showing that they have the durability needed for decades in the field. Swift Coat scaled the AIDA hardware and deposition process to make mini-modules that were monitored for nearly two years during field tests administered by a third party, as well as demonstrated scaling to the widths of full-sized modules. The fielded mini-modules outperformed reference modules (with commercial antireflection coatings) in two locations, providing a 1% absolute average performance boost and larger increases during periods of heavier soiling. Swift Coat’s cost analysis indicated a coating manufacturing cost below the sales price of today’s antireflection coatings. More than five module manufacturers sampled and assessed the coatings, and three provided letters of support. The coating developed in this project increases the energy output of PV modules, thereby decreasing the cost per kilowatt-hour of solar energy generated. Cheaper solar electricity benefits the public by accelerating the transition to a stable, affordable, carbon-free energy economy. In addition, for select applications in which PV modules are highly visible—such as on residential rooftops—the coating provides an aesthetic benefit because it stays cleaner than today’s modules. Finally, Swift Coat and its prospective customers are U.S. companies, and successful commercialization of this technology will provide U.S. jobs and a secure solar supply chain.
14 SOLAR ENERGY↗
GPU-enabled extreme-scale turbulence simulations: Fourier pseudo-spectral algorithms at the exascale using OpenMP offloading
Fourier pseudo-spectral methods for nonlinear partial differential equations are of wide interest in many areas of advanced computational science, including direct numerical simulation of three-dimensional (3-D) turbulence governed by the Navier-Stokes equations in fluid dynamics. This paper presents a new capability for simulating turbulence at a new record resolution up to 35 trillion grid points, on the world's first exascale computer, Frontier, comprising AMD MI250x GPUs with HPE's Slingshot interconnect and operated by the US Department of Energy's Oak Ridge Leadership Computing Facility (OLCF). Key programming strategies designed to take maximum advantage of the machine architecture involve performing almost all computations on the GPU which has the same memory capacity as the CPU, performing all-to-all communication among sets of parallel processes directly on the GPU, and targeting GPUs efficiently using OpenMP offloading for intensive number-crunching including 1-D Fast Fourier Transforms (FFT) performed using AMD ROCm library calls. With 99% of computing power on Frontier being on the GPU, leaving the CPU idle leads to a net performance gain via avoiding the overhead of data movement between host and device except when needed for some I/O purposes. Memory footprint including the size of communication buffers for MPI_ALLTOALL is managed carefully to maximize the largest problem size possible for a given node count. Detailed performance data including separate contributions from different categories of operations to the elapsed wall time per step are reported for five grid resolutions, from 2048 3 on a single node to 32768 3 on 4096 or 8192 nodes out of 9408 on the system. Both 1D and 2D domain decompositions which divide a 3D periodic domain into slabs and pencils respectively are implemented. The present code suite (labeled by the acronym GESTS, GPUs for Extreme Scale Turbulence Simulations) achieves a figure of merit (in grid points per second) exceeding goals set in the Center for Accelerated Application Readiness (CAAR) program for Frontier. The performance attained is highly favorable in both weak scaling and strong scaling, with notable departures only for 2048 3 where communication is entirely intra-node, and for 32768 3 , where a challenge due to small message sizes does arise. Communication performance is addressed further using a lightweight test code that performs all-to-all communication in a manner matching the full turbulence simulation code. Performance at large problem sizes is affected by both small message size due to high node counts as well as dragonfly network topology features on the machine, but is consistent with official expectations of sustained performance on Frontier. Overall, although not perfect, the scalability achieved at the extreme problem size of 32768 3 (and up to 8192 nodes — which corresponds to hardware rated at just under 1 exaflop/sec of theoretical peak computational performance) is arguably better than the scalability observed using prior state-of-the-art algorithms on Frontier's predecessor machine (Summit) at OLCF. New science results for the study of intermittency in turbulence enabled by this code and its extensions are to be reported separately in the near future.
3D fast Fourier transform↗