Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “parallelize algorithm computation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,585 records · Page 88

Cardinal: A Lower Length-scale Simulator for Fluoride Cooled High Temperature Reactors

The new DOE-NE “Center of Excellence for Thermal-fluids applications in Nuclear Energy” inaugurated in April 2018 considers and researches novel new solution strategies for historically challenging flow issues that still plague the current fleet of deployed Light Water Reactor (LWR) nuclear reactors as well as predicting various fluid flow and fluid related issues with advanced reactor technologies. Our advanced thermal-fluids research and development approach synergistically combines three natural, though overlapping, length and time scales in a hierarchal multi-scale approach to avoid the temptation and pitfalls of attempting to develop a single “solve all” algorithm for physical fluid flow problems that will span 109 in spatial and temporal scales. The demonstrate the multi-scale philosophy of the center we focus first on Fluoride Cooled High Temperature Reactors (FHRs), and in particular on the Berkeley’s PB-FHR design Mark-I design. The Fluoride salt cooled High temperature Reactor (FHR) is a class of advanced nuclear reactors that combine the robust coated particle fuel form from high temperature gas cooled reactors, direct reactor auxiliary cooling system (DRACS) passive decay removal of liquid metal fast reactors, and the transparent, high volumetric heat capacitance liquid fluoride salt working fluids - Flibe - from molten salt reactors. This combination of fuel and coolant enables FHRs to operate in a high-temperature low-pressure design space that has beneficial safety and economic implications. The PB-FHR reactor relies on a pebble bed approach and pebble bed reactors are in a sense the poster child for multiscale analysis. The lower length-scale simulator for pebble reactor cores comprises three physics: neutronics (OpenMC), thermal-fluids (Nek5000) and fuel performance (BISON). As part of center of excellence ongoing research efforts we have developed “Cardinal”, a new tool platform for lower length-scale simulation. Cardinal tightly couples all three physics and leverages advances in MOOSE such as the MultiApp system and the concept of MOOSE-wrapped Apps. Moreover it is designed from the ground-up to scale on massively parallel architectures and perform well on world-class super-computing architectures. In this report we describe the design of Cardinal and present a demo of the application of Cardinal to an FHR.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

Randomized Preconditioned Solvers for Strong Constraint 4D-Var Data Assimilation

The Strong Constraint 4D Variational (SC-4DVAR) data assimilation method is widely used in climate and weather applications. SC-4DVAR involves solving a minimization problem to compute the maximum a posteriori estimate, which we tackle using the Gauss-Newton method. The computation of the descent direction is expensive since it involves the solution of a large-scale and potentially ill-conditioned linear system, solved using the preconditioned conjugate gradient (PCG) method. Here, to address this cost, we efficiently construct scalable preconditioners using three different randomization techniques, which all rely on a certain low-rank structure involving the Gauss-Newton Hessian. The proposed techniques come with theoretical guarantees on the condition number, and at the same time, are amenable to parallelization. We also develop an adaptive approach to estimate the sketch size and choose between the reuse or recomputation of the preconditioner. We demonstrate the performance and effectiveness of our methodology on two representative model problems—the Burgers and barotropic vorticity equation—showing a drastic reduction in both the number of PCG iterations and the number of Gauss-Newton Hessian products after including the preconditioner construction cost.

Gauss-Newton↗

Theoretical Plasma Physics

Lattice Boltzmann algorithms are a mesoscopic method to solve problems in nonlinear physics which are highly parallelized – unlike the direction solution of the original problem. These methods are applied to both fluid and magnetohydrodynamic turbulence. By introducing entropic constraints one can enforce the positive definiteness of the distribution functions and so be able to simulate fluids at high Reynolds numbers without numerical instabilities. By introducing a vector distribution function for the magnetic field one can enforce the divergence free condition on the magnetic field automatically, without the need of divergence cleaning as needed in most direct numerical solutions of the resistive magnetohydrodynamic equations. The principal reason for the high parallelization of lattice Boltzmann codes is that they consist of a kinetic collisional relaxation step (which is purely local) followed by a simple shift of the relaxed data to neighboring lattice sites. In large eddy simulations, the closure schemes are highly nonlocal – the most famous of these schemes is that due to Smagorinsky. Under a lattice Boltzmann representation the Smagorinsky closure is purely local – being simply a particular moment on the perturbed distribution fucntions. After nonlocal fluid moment models were discovered to represent Landau damping, it was found possible to model these fluid models using an appropriate lattice Boltzmann algorithm. The close to ideal parallelization of the lattice Boltzmann codes permitted us to be Gordon Bell finalists on using the Earth Simulation in Japan. We have also been involved in the radio frequency propagation of waves into a tokamak and into a spherical overdense tokamak plasma. Initially we investigated the use of a quasi-optical grill for the launching of lower hybrid waves into a tokamak. It was found that the conducting walls do not prevent the rods from being properly irradiated, the overloading of the quasi-optical grill is not severe with the electric field only being about three times higher than in the ideal case. Moreover, the quasi-optical grill was significantly fewer structural elements that the multijunction grill. Nevertheless there has not been much interest from experimental fusion groups to implementing these structures. Hence we have returned to optimizing the multijunction grill so that the large number of coupling matrix elements can be efficiently evaluated using symmetry arguments. In overdense plasmas, the standard electromagnetic waves cannot propagate into the plasma center, but are reflected at the plasma edge. By optimizing mode conversion processes (in particular, the O-X-B wave propagation of Ordinary Mode converting to an Extraordinary mode which then converts into an electrostatic Bernstein wave) one can excite within the plasma an electrostatic Bernstein wave that does not suffer density cutoffs and is absorbed on the electron cyclotron harmonics. Finally we have started looking at other mesoscopic lattice algorithms that involve unitary collision and streaming steps. Because these algorithms are unitary they can be run on quantum computers when they become available – unlike their computational cousin of lattice Boltzmann which is a purely classical code. These quantum lattice gas algorithms have been tested successfully on exact analytic soliton collision solution. These calculations are hoped to be able to study Bose Einstein condensed atomic gases and their ground states in an optical lattice.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Development Status of PROTEUS-MOC

The PROTEUS-MOC solver is intended to resolve the outstanding computational challenges that the PROTEUS-SN is limited by and bridge the gap between PROTEUS-SN and existing homogenized structured grid solvers. The primary push behind all of this work is to carry out coupled thermal-mechanical, fluid dynamics, and neutronics physics problems. The PROTEUS-MOC development has taken advantage of the combined experience with PROTEUS-SN and DeCART with the goal of producing a tool which combines the best features of both tools. Taking the DeCART tool as the model, it merges a subgroup cross section interpolation technique with a 2D method of characteristics approach and a 1D SN formulation. DeCART has been specifically demonstrated to be accurate on PWR and VHTR problems, but it has displayed several issues with regard to accuracy on thin axial mesh sizes which are attributable to the 2D1D scheme it is built upon. There are also notable problems with performance and scalability all of which we hope to resolve in PROTEUS-MOC. PROTEUS-MOC is at its core, based upon the success achieved with MOCFE which demonstrated excellent parallelization with respect to angle and good parallelization with respect to space. More recent efforts have indicated parallelization with respect to energy is also feasible. The work completed yielded a novel methodology where the axial basis functions are treated separately from the radial ones, but the geometry is extruded. This eliminates the need to carry out and store 3D ray tracing data and simultaneously allows the coarse axial node sizes typical with DeCART. To rapidly build a prototype, a standard power iteration algorithm was constructed using an unpreconditioned GMRES solution algorithm for solving the flux vector in each fission source iteration. The new PROTEUS-MOC solver was deployed on two test problems: Takeda and a heterogeneous cylindrical heavy water system. The Takeda benchmark problem is a standard homogeneous test problem which was previously found to be too difficult for the 3D MOCFE code to solve. Using the PROTEUS-MOC we were able to show space-angle convergence although full convergence requires further improvements in performance. The second benchmark problem was constructed more for future work than this year, but the results generated are promising indicating that it will be very useful in highlighting the weaknesses and strengths of the PROTEUS-MOC tool. While we are far from a production tool, the results included in this report give us confidence that we are on the right path to constructing an interceding code between the plethora of structured grid homogenized assembly solvers and the supercomputer consuming PROTEUS-SN code. With time, the creation of a suitable preconditioner along with typical optimization should reduce the performance shortcomings that are observable today.

Marin-Lafleche, A.↗

Visualization at exascale: Making it all work with VTK-m

The VTK-m software library enables scientific visualization on exascale-class supercomputers. Exascale machines are particularly challenging for software development in part because they use GPU accelerators to provide the vast majority of their computational throughput. Algorithmic designs for GPUs and GPU-centric computing often deviate from those that worked well on previous generations of high-performance computers that relied on traditional CPUs. Fortunately, VTK-m provides scientific visualization algorithms for GPUs and other accelerators. VTK-m also provides a framework that simplifies the implementation of new algorithms and adds a porting layer to work across multiple processor types. This paper describes the main challenges encountered when making scientific visualization available at exascale. Here, we document the surprises and obstacles faced when moving from pre-exascale platforms to the final exascale designs and the performance on those systems including scaling studies on Frontier, an exascale machine with over 37,000 AMD GPUs. We also report on the integration of VTK-m with other exascale software technologies. Finally, we show how VTK-m helps scientific discovery for applications such as fusion and particle acceleration that leverage an exascale supercomputer.

97 MATHEMATICS AND COMPUTING↗

A Performance Portable, Fully Implicit Landau Collision Operator with Batched Linear Solvers

Modern accelerators use hierarchical parallel programming models that enable massive multithreading within a processing element (PE), with multiple PEs per device driven by traditional processes. Batching is a technique for exposing PE-level parallelism in algorithms that have traditionally run on MPI processes or multiple threads within a single process. Opportunities for batching arise in, for example, kinetic discretizations of magnetized plasmas where collisions are advanced in velocity space at each spatial point independently. This paper builds on previous work on a high-performance, fully nonlinear, Landau collision operator by batching the linear solver, as well as batching the spatial point problems and adding new support for multiple grids for multiscale, multispecies problems. An anisotropic relaxation verification test that agrees well with previously published results and analytical models is presented. The performance results from NVIDIA A100 and AMD MI250X nodes are presented with hardware utilization analysis for each architecture. Finally, the entire implicit Landau operator time advance is implemented in Kokkos for performance portability, running entirely on the device and is available in the PETSc numerical library.

97 MATHEMATICS AND COMPUTING↗

Communication Characterization and Optimization of Applications Using Topology-Aware Task Mapping on Large Supercomputers

On large supercomputers, the job scheduling systems may assign a non-contiguous node allocation for user applications depending on available resources. With parallel applications using MPI (Message Passing Interface), the default process ordering does not take into account the actual physical node layout available to the application. This contributes to non-locality in terms of physical network topology and impacts communication performance of the application. In order to mitigate such performance penalties, this work describes techniques to identify suitable task mapping that takes the layout of the allocated nodes as well as the application's communication behavior into account. During the first phase of this research, we instrumented and collected performance data to characterize communication behavior of critical US DOE (United States - Department of Energy) applications using an augmented version of the mpiP tool. Subsequently, we developed several reordering methods (spectral bisection, neighbor join tree etc.) to combine node layout and application communication data for optimized task placement. We developed a tool called mpiAproxy to facilitate detailed evaluation of the various reordering algorithms without requiring full application executions. This work presents a comprehensive performance evaluation (14,000 experiments) of the various task mapping techniques in lowering communication costs on Titan, the leadership class supercomputer at Oak Ridge National Laboratory.

97 MATHEMATICS AND COMPUTING↗

ELSI: A unified software interface for Kohn–Sham electronic structure solvers

Solving the electronic structure from a generalized or standard eigenproblem is often the bottleneck in large scale calculations based on Kohn-Sham density-functional theory. This problem must be addressed by essentially all current electronic structure codes, based on similar matrix expressions, and by high-performance computation. We here present a unified software interface, ELSI, to access different strategies that address the Kohn-Sham eigenvalue problem. Currently supported algorithms include the dense generalized eigensolver library ELPA, the orbital minimization method implemented in libOMM, and the pole expansion and selected inversion (PEXSI) approach with lower computational complexity for semilocal density functionals. The ELSI interface aims to simplify the implementation and optimal use of the different strategies, by offering (a) a unified software framework designed for the electronic structure solvers in Kohn-Sham density-functional theory; (b) reasonable default parameters for a chosen solver; (c) automatic conversion between input and internal working matrix formats, and in the future (d) recommendation of the optimal solver depending on the specific problem. As a result, comparative benchmarks are shown for system sizes up to 11,520 atoms (172,800 basis functions) on distributed memory supercomputing architectures.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Two-Stage Gauss-Seidel Preconditioners and Smoothers for Krylov Solvers on a GPU Cluster: Preprint

Gauss-Seidel (GS) relaxation is often employed as a preconditioner for a Krylov solver or as a smoother for Algebraic Multigrid (AMG). However, the requisite sparse triangular solve is difficult to parallelize on many-core architectures such as graphics processing units (GPUs). In the present study, the performance of the sequential GS relaxation based on a triangular solve is compared with two-stage variants, replacing the direct triangular solve with a fixed number of inner Jacobi-Richardson (JR) iterations. When a small number of inner iterations is sufficient to maintain the Krylov convergence rate, the two-stage GS (GS2) often outperforms the sequential algorithm on many-core architectures. The GS2 algorithm is also compared with JR. When they perform the same number of ops for SpMV (e.g. three JR sweeps compared to two GS sweeps with one inner JR sweep), the GS2 iterations, and the Krylov solver preconditioned with GS2, may converge faster than the JR iterations. Moreover, for some problems (e.g. elasticity), it was found that JR may diverge with a damping factor of one, whereas two-stage GS may improve the convergence with more inner iterations. Finally, to study the performance of the two-stage smoother and preconditioner for a practical problem, these were applied to incompressible uid ow simulations on GPUs.

algebraic multigrid↗

Multi-GPU immersed boundary method hemodynamics simulations

Large-scale simulations of blood flow that resolve the 3D deformation of each comprising cell are increasingly popular owing to algorithmic developments in conjunction with advances in compute capability. Among different approaches for modeling cell-resolved hemodynamics, fluid structure interaction (FSI) algorithms based on the immersed boundary method are frequently employed for coupling separate solvers for the background fluid and the cells within one framework. GPUs can accelerate these simulations; however, both current pre-exascale and future exascale CPU-GPU heterogeneous systems face communication challenges critical to performance and scalability. In this paper, we describe, to our knowledge, the largest distributed GPU-accelerated FSI simulations of high hematocrit cell-resolved flows with over 17 million red blood cells. We compare scaling on a fat node system with six GPUs per node and on a system with a single GPU per node. Through comparison between the CPU- and GPU-based implementations, we identify the costs of data movement in multiscale multi-grid FSI simulations on heterogeneous systems and show it to be the greatest performance bottleneck on the GPU.

97 MATHEMATICS AND COMPUTING↗

A communication-avoiding 3D algorithm for sparse LU factorization on heterogeneous systems

We propose a new algorithm to improve the strong scalability of right-looking sparse LU factorization on distributed memory systems. Our 3D algorithm for sparse LU uses a three-dimensional MPI process grid, exploits elimination tree parallelism, and trades off increased memory for reduced per-process communication. We also analyze the asymptotic improvements for planar graphs (e.g., those arising from 2D grid or mesh discretizations) and certain non-planar graphs (specifically for 3D grids and meshes). For a planar graph with $n$ vertices, our algorithm reduces communication volume asymptotically in $n$ by a factor of $\mathscr{O}$ $\Big(\sqrt{log \ n}\Big)$ and latency by a factor of $\mathscr{O}$ $(log \ n)$. For nonplanar cases, our algorithm can reduce the per-process communication volume by 3× and latency by $\mathscr{O}$ $\Big(n^\frac13\Big)$ times. In all cases, the memory needed to achieve these gains is a constant factor. We implemented our algorithm by extending the 2D data structure used in SuperLU_DIST. Our new 3D code achieves empirical speedups up to 27× for planar graphs and up to 3.3× for non-planar graphs over the baseline 2D SuperLU_DIST when run on 24,000 cores of a Cray XC30. We extend the 3D algorithm for heterogeneous architectures by adding the Highly Asynchronous Lazy Offload (Halo) algorithm for co-processor offload. On 4096 nodes of a Cray XK7 with 32,768 CPU cores and 4096 Nvidia K20x GPUs, the 3D algorithm achieves empirical speedups up to 24× for planar graphs and 3.5× for non-planar graphs over the baseline 2D SuperLU_DIST with co-processor acceleration.

97 MATHEMATICS AND COMPUTING↗

QCOR; A Language Extension Specification for the Heterogeneous Quantum-Classical Model of Computation

Quantum computing (QC) is an emerging computational paradigm that leverages the laws of quantum mechanics to perform elementary logic operations. Existing programming models for QC were designed with fault-tolerant hardware in mind, envisioning stand-alone applications. However, the susceptibility of near-term quantum computers to noise limits their stand-alone utility. To better leverage limited computational strengths of noisy quantum devices, hybrid algorithms have been suggested whereby quantum computers are used in tandem with their classical counterparts in a heterogeneous fashion. This modus operandi calls out for a programming model and a high-level programming language that natively and seamlessly supports heterogeneous quantum-classical hardware architectures in a single-source-code paradigm. Motivated by the lack of such a model, we introduce a language extension specification, called QCOR, which enables single-source quantum-classical programming. Programs written using the QCOR library–based language extensions can be compiled to produce functional hybrid binary executables. After defining QCOR’s programming model, memory model, and execution model, we discuss how QCOR enables variational, iterative, and feed-forward QC. Additionally, QCOR approaches quantum-classical computation in a hardware-agnostic heterogeneous fashion and strives to build on best practices of high-performance computing. The high level of abstraction in the language extension is intended to accelerate the adoption of QC by researchers familiar with classical high-performance computing.

97 MATHEMATICS AND COMPUTING↗

Addressing Global Data Dependencies in Heterogeneous Asynchronous Runtime Systems on GPUs

Large-scale parallel applications with complex global data dependencies beyond those of reductions pose significant scalability challenges in an asynchronous runtime system. Internodal challenges include identifying the all-to-all communication of data dependencies among the nodes. Intranodal challenges include gathering together these data dependencies into usable data objects while avoiding data duplication. This paper addresses these challenges within the context of a large-scale, industrial coal boiler simulation using the Uintah asynchronous many-task runtime system on GPU architectures. We show significant reduction in time spent analyzing data dependencies through refinements in our dependency search algorithm. Multiple task graphs are used to eliminate subsequent analysis when task graphs change in predictable and repeatable ways. Using a combined data store and task scheduler redesign reduces data dependency duplication ensuring that problems fit within host and GPU memory. Furthermore, these modifications did not require any changes to application code or sweeping changes to the Uintah runtime system. We report results running on the DOE Titan system on 119K CPU cores and 7.5K GPUs simultaneously. Our solutions can be generalized to other task dependency problems with global dependencies among thousands of nodes which must be processed efficiently at large scale.

97 MATHEMATICS AND COMPUTING↗

Distributed out-of-memory NMF on CPU/GPU architectures

We propose an efficient distributed out-of-memory implementation of the non-negative matrix factorization (NMF) algorithm for heterogeneous high-performance-computing systems. The proposed implementation is based on prior work on NMFk, which can perform automatic model selection and extract latent variables and patterns from data. In this work, we extend NMFk by adding support for dense and sparse matrix operation on multi-node, multi-GPU systems. The resulting algorithm is optimized for out-of-memory problems where the memory required to factorize a given matrix is greater than the available GPU memory. Memory complexity is reduced by batching/tiling strategies, and sparse and dense matrix operations are significantly accelerated with GPU cores (or tensor cores when available). Input/output latency associated with batch copies between host and device is hidden using CUDA streams to overlap data transfers and compute asynchronously, and latency associated with collective communications (both intra-node and inter-node) is reduced using optimized NVIDIA Collective Communication Library (NCCL) based communicators. Benchmark results show significant improvement, from 32X to 76x speedup, with the new implementation using GPUs over the CPU-based NMFk. Good weak scaling was demonstrated on up to 4096 multi-GPU cluster nodes with approximately 25,000 GPUs when decomposing a dense 340 Terabyte-size matrix and an 11 Exabyte-size sparse matrix of density 10 -6 .

97 MATHEMATICS AND COMPUTING↗

MAESTROeX: A Massively Parallel Low Mach Number Astrophysical Solver

@e present MAESTROeX, a massively parallel solver for low Mach number astrophysical flows. The underlying low Mach number equation set allows for efficient, long-time integration for highly subsonic flows compared to compressible approaches. MAESTROeX is suitable for modeling full spherical stars as well as well as planar simulations of dynamics within localized regions of a star, and can robustly handle several orders of magnitude of density and pressure stratification. Previously, we have described the development of the predecessor of MAESTROeX, called MAESTRO, in a series of papers. Here, we present a new, greatly simplified temporal integration scheme that retains the same order of accuracy as our previous approaches. We also explore the use of alternative spatial mapping of the one-dimensional base state onto the full Cartesian grid. The code leverages the new AMReX software framework for block-structured adaptive mesh refinement (AMR) applications, allowing for scalability to large fractions of leadership-class machines. Finally, using our previous studies on the convective phase of single-degenerate progenitor models of SNe Ia as a guide, we characterize the performance of the code and validate the new algorithmic features. Like MAESTRO, MAESTROeX is fully open source.

79 ASTRONOMY AND ASTROPHYSICS↗

An asynchronous traversal engine for graph-based rich metadata management

Rich metadata in high-performance computing (HPC) systems contains extended information about users, jobs, data files, and their relationships. Property graphs are a promising data model to represent heterogeneous rich metadata flexibly. Specifically, a property graph can use vertices to represent different entities and edges to record the relationships between vertices with unique annotations. The high-volume HPC use case, with millions of entities and relationships, naturally requires an out-of-core distributed property graph database, which must support live updates (to ingest production information in real time), low-latency point queries (for frequent metadata operations such as permission checking), and large-scale traversals (for provenance data mining). Among these needs, large-scale property graph traversals are particularly challenging for distributed graph storage systems. Most existing graph systems implement a "level synchronous" breadth-first search algorithm that relies on global synchronization in each traversal step. This performs well in many problem domains; but a rich metadata management system is characterized by imbalanced graphs, long traversal lengths, and concurrent workloads, each of which has the potential to introduce or exacerbate stragglers (i.e., abnormally slow steps or servers in a graph traversal) that lead to low overall throughput for synchronous traversal algorithms. Previous research indicated that the straggler problem can be mitigated by using asynchronous traversal algorithms, and many graph-processing frameworks have successfully demonstrated this approach. Such systems require the graph to be loaded into a separate batch-processing framework instead of being iteratively accessed, however. In this work, we investigate a general asynchronous graph traversal engine that can operate atop a rich metadata graph in its native format. We outline a traversal-aware query language and key optimizations (traversal-affiliate caching and execution merging) necessary for efficient performance. We further explore the effect of different graph partitioning strategies on the traversal performance for both synchronous and asynchronous traversal engines. Our experiments show that the asynchronous graph traversal engine is more efficient than its synchronous counterpart in the case of HPC rich metadata processing, where more servers are involved and larger traversals are needed. Furthermore, the asynchronous traversal engine is more adaptive to different graph partitioning strategies.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

Supervised enhancer prediction with epigenetic pattern recognition and targeted validation

Enhancers are important non-coding elements, but they have traditionally been hard to characterize experimentally. The development of massively parallel assays allows the characterization of large numbers of enhancers for the first time. Here, we developed a framework using Drosophila STARR-seq to create shape-matching filters based on meta-profiles of epigenetic features. We integrated these features with supervised machine-learning algorithms to predict enhancers. We further demonstrated that our model could be transferred to predict enhancers in mammals. We comprehensively validated the predictions using a combination of in vivo and in vitro approaches, involving transgenic assays in mice and transduction-based reporter assays in human cell lines (153 enhancers in total). The results confirmed that our model can accurately predict enhancers in different species without re-parameterization. Finally, we examined the transcription factor binding patterns at predicted enhancers versus promoters. Here, we demonstrated that these patterns enable the construction of a secondary model that effectively distinguishes enhancers and promoters.

59 BASIC BIOLOGICAL SCIENCES↗

Performance-Aligned LLMs for Generating Fast HPC Code

Optimizing scientific software is a difficult task because codebases are often large and complex, and performance can depend upon several factors including the algorithm, its implementation, and hardware among others. Causes of poor performance can originate from disparate sources and be difficult to diagnose. Recent years have seen a multitude of work that use large language models (LLMs) to assist in software development tasks. However, these tools are trained to model the distribution of code as text, and are not specifically designed to understand performance aspects of code. In this work, we introduce a reinforcement learning based methodology to align the outputs of code LLMs with performance. This allows us to build upon the current code modeling capabilities of LLMs and extend them to generate better performing code. Here, we demonstrate that our fine-tuned model improves the expected speedup of generated code over base models for a set of benchmark tasks from 0.9 to 1.6 for serial code and 1.9 to 4.5 for OpenMP parallel code.

Computer science↗