Toward Agentic HPC: Serving and Evaluating LLM-Powered Agents for Scientific Applications On Leadership-Class Platforms
Explore the source record for details and available documents.
Yin, Junqi [ORNL] (ORCID:0000000338435520)↗
SEARCH · Engineering Papers
Results for “HPC applications”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
User-Oriented Improvements in the MOOSE framework in support of Multiphysics Simulation
The MOOSE Framework is a foundational capability used by NEAMS to create over 15 different simulation tools for advanced nuclear reactors. Due to this ubiquity, improvements to the framework in support of modeling and simulation goals are critical to the program. These improvements can take many forms including optimization, improved user experience, streamlined APIs, parallelism, and new capability. The work transcribed in the report was in direct support of the simulation tools and is already deployed or will be deployed in the coming months. The capabilities implemented were, in the same order as this report, chainable execution objects or executors, support for transfers between applications at the same level in a coupling scheme, support for boundary/subdomain restricted transfers, support for transfers between applications with different coordinate or unit systems, support for MOOSE applications in the NEAMS workbench, deployment of MOOSE application of the INL HPC OnDemand platform, addition of a triangular meshing library in libMesh and increased support of face variables.
97 MATHEMATICS AND COMPUTING↗
Pretraining Billion-Scale Geospatial Foundational Models on Frontier
As AI workloads increase in scope, generalization capability becomes challenging for small task-specific models and their demand for large amounts of labeled training samples increases. On the contrary, Foundation Models (FMs) are trained with internet-scale unlabeled data via self-supervised learning and have been shown to adapt to various tasks with minimal fine-tuning. Although large FMs have demonstrated significant impact in natural language processing and computer vision, efforts toward FMs for geospatial applications have been restricted to smaller size models, as pretraining larger models requires very large computing resources equipped with state-of-the-art hardware accelerators. Current satellite constellations collect 100+TBs of data a day, resulting in images that are billions of pixels and multimodal in nature. Such geospatial data poses unique challenges opening up new opportunities to develop FMs. We investigate billion scale FMs and HPC training profiles for geospatial applications by pretraining on publicly available data. We studied from end-to-end the performance and impact in the solution by scaling the model size. Our larger 3B parameter size model achieves up to 30% improvement in top1 scene classification accuracy when comparing a 100M parameter model. Moreover, we detail performance experiments on the Frontier supercomputer, America's first exascale system, where we study different model and data parallel approaches using PyTorch's Fully Sharded Data Parallel library. Specifically, we study variants of the Vision Transformer architecture (ViT), conducting performance analysis for ViT models with size up to 15B parameters. By discussing throughput and performance bottlenecks under different parallelism configurations, we offer insights on how to leverage such leadership-class HPC resources when developing large models for geospatial imagery applications.
Tsaris, Aristeidis (aris)↗
Defining quantum-ready primitives for hybrid HPC-QC supercomputing: a case study in Hamiltonian simulation
As computational demands in scientific applications continue to rise, hybrid high-performance computing (HPC) systems integrating classical and quantum computers (HPC-QC) are emerging as a promising approach to tackling complex computational challenges. One critical area of application is Hamiltonian simulation, a fundamental task in quantum physics and other large-scale scientific domains. This paper investigates strategies for quantum-classical integration to enhance Hamiltonian simulation within hybrid supercomputing environments. By analyzing computational primitives in HPC allocations dedicated to these tasks, we identify key components in Hamiltonian simulation workflows that stand to benefit from quantum acceleration. To this end, we systematically break down the Hamiltonian simulation process into discrete computational phases, highlighting specific primitives that could be effectively offloaded to quantum processors for improved efficiency. Our empirical findings provide insights into system integration, potential offloading techniques, and the challenges of achieving seamless quantum-classical interoperability. We assess the feasibility of quantum-ready primitives within HPC workflows and discuss key barriers such as synchronization, data transfer latency, and algorithmic adaptability. These results contribute to the ongoing development of optimized hybrid solutions, advancing the role of quantum-enhanced computing in scientific research.
97 MATHEMATICS AND COMPUTING↗
Enabling Low-Overhead HT-HPC Workflows at Extreme Scale using GNU Parallel
GNU Parallel is a versatile and powerful tool for process parallelization widely used in scientific computing. This paper demonstrates its effective application in high-performance computing (HPC) environments, particularly focusing on its scalability and efficiency in executing large-scale high-throughput high-performance computing (HT-HPC) workflows. Through real-world examples, we highlight GNU Parallel’s performance across various HPC workloads, including GPU computing, container-based workloads, and node-local NVMe storage. Our results on two leading supercomputers, OLCF’s Frontier and NERSC’s Perlmutter, showcase GNU Parallel’s rapid process dispatching ability and its capacity to maintain low overhead even at extreme scales. We explore GNU Parallel’s application in massive parallel file transfers using a scheduled Data Transfer Node (DTN) cluster, emphasizing its broad utility in diverse scientific workflows. Beyond its direct application as a viable workflow manager, GNU Parallel can be employed in conjunction with other workflow systems as a "last-mile" parallelizing driver and as a quick prototyping tool to design and extract parallel profiles from application executions. We then argue that the potential for GNU Parallel to transform workflow management at extreme scales is substantial, paving the way for more efficient and effective scientific discoveries.
Maheshwari, Ketan↗
High-Fidelity and High-Performance Computational Simulations for Rapid Design Optimization of Sulfur Thermal Energy Storage (CRADA Report)
NREL and Element 16 collaborated on sulfur thermal energy storage modeling using NREL’s high performance computing (HPC) resources to assist its application in industrial processes. Industrial process heat (IPH) accounts for ~70% of US manufacturing energy use and is primarily produced by fossil fuel combustion. Approximately, 1500 TWht (~60% Terawatt hour thermal) of IPH demand is in the temperature range of 100-300°C. Industrial applications in this temperature range include drying, hydrothermal processing, thermal enhanced oil recovery, food and beverage, bioethanol production, etc. Cost-effective thermal energy storage (TES) that increases the utilization of waste and renewable heat (solar, geothermal, etc.) could provide significant energy savings and reliable heat sources, decrease emissions, and increase US manufacturing competitiveness through reductions in fuel consumption. This HPC4EI project facilitated Element 16’s development of low-cost and high-impact molten sulfur TES for dispatchable IPH. The development of a high-fidelity model validated by experimental data and HPC simulations enabled the successful resolution of the complex interplay between fluid dynamics and heat transfer processes during transient operation of sulfur TES, overcoming the numerical challenges posed by the non-linear temperature-dependent physical properties of sulfur. The project helped accelerate Element 16’s molten sulfur TES product design and support its broad applications.
25 ENERGY STORAGE↗
The Benefits and Weaknesses of Containerizing Software for HPC
Containerization technology has emerged as a transformative tool for software engineers, offering consistent development and deployment environments, simplifying dependency management, and enhancing scalability and portability across diverse systems. However, its application in High-Performance Computing (HPC) presents unique challenges, including the management of virtualization overhead, the need for efficient resource allocation, and the maintenance of optimal performance for compute-intensiv
Ho, Eric Victor [Sandia National Laboratories (SNL↗
Optimization and Portability of a Fusion OpenACC-based FORTRAN HPC Code from NVIDIA to AMD GPUs
NVIDIA has been the main provider of GPU hardware in HPC systems for over a decade. Most applications that benefit from GPUs have thus been developed and optimized for the NVIDIA software stack. Recent exascale HPC systems are, however, introducing GPUs from other vendors, e.g. with the AMD GPU-based OLCF Frontier system just becoming available. AMD GPUs cannot be directly accessed using the NVIDIA software stack, and require a porting effort by the application developers. This paper provides an overview of our experience porting and optimizing the CGYRO code, a widely-used fusion simulation tool based on FORTRAN with OpenACC-based GPU acceleration. While the porting from the NVIDIA compilers was relatively straightforward using the CRAY compilers on the AMD systems, the performance optimization required more fine-tuning. In the optimization effort, we uncovered code sections that had performed well on NVIDIA GPUs, but were unexpectedly slow on AMD GPUs. After AMD-targeted code optimizations, performance on AMD GPUs has increased to meet our expectations. Modest speed improvements were also seen on NVIDIA GPUs, which was an unexpected benefit of this exercise.
Sfiligoi, Igor↗
Pre-exascale accelerated application development: The ORNL Summit experience
High-performance computing (HPC) increasingly relies on heterogeneous architectures to achieve higher performance. In the Oak Ridge Leadership Facility (OLCF), Oak Ridge, TN, USA, this trend continues as its latest supercomputer, Summit, entered production in early 2019. The combination of IBM POWER9 CPU and NVIDIA V100 GPU, along with a fast NVLink2 interconnect and other latest technologies, pushes system performance to a new height and breaks the exascale barrier by certain measures. Due to Summit's powerful GPUs and much higher GPU–CPU ratio, offloading to accelerators becomes a requirement for any application, which intends to effectively use the system. To facilitate navigating a complex landscape of competing heterogeneous architectures, a collection of applications from a wide spectrum of scientific domains is selected for early adoption on Summit. In this article, the experience and lessons learned are summarized, in the hope of providing useful guidance to address new programming challenges, such as scalability, performance portability, and software maintainability, for future application development efforts on heterogeneous HPC systems.
97 MATHEMATICS AND COMPUTING↗
Distributed-Memory Sparse Deep Neural Network Inference Using Global Arrays
Partitioned Global Address Space (PGAS) models exhibit tremendous promise in developing efficient and productive distributed-memory parallel applications. They have been used extensively in scientific computations due to conveniently offering a ``shared-memory''-like model and convenient interfaces that separate communication with synchronization. Traditionally, PGAS communication models have been applied to dense/contiguously distributed data, but most modern applications depict varied levels of sparsity. Existing PGAS models require certain adaptations to support distributed sparse computations, since associated computations often require matrix arithmetic, in addition to data movement. The Global Arrays toolkit from Pacific Northwest National Laboratory (PNNL) is one of the earliest PGAS models to combine one-sided data communication and distributed matrix operations and is still used in the popular NWChem quantum chemistry suite. Recently, we have expanded the Global Arrays toolkit to support common sparse operations, like sparse matrix-dense matrix multiplies (SpMM), sparse matrix-sparse matrix multiplication (SpGEMM) and Sampled Dense-Dense Matrix Multiplication (SDDMM). As it turns out, these operations are the bedrock of sparse Deep Learning (DL); sparse deep neural networks and Graph Neural Networks (GNNs) have gained increasing attention recently in achieving speedups on training and inference with reduced memory footprints. Unlike scientific applications in High Performance Computing (HPC), modern (distributed-memory capable) DL toolkits often rely on non-standardized and closed-source vendor software optimizations, creating challenges in software-hardware co-design at scale. Our goal is to support a variety of distributed-memory sparse matrix operations and helper functions in the newly created Sparse Global Arrays (SGA), such that it is possible to build portable and productive Machine Learning scenarios for algorithm/software and hardware codesign purposes. Contemporary data-parallel schemes for training/inference are undergoing a major overhaul since model replication limits scalability and causes resource inefficiencies. As such, we have adopted tensor parallelism in decomposing the model and inputs, to mitigate memory issues. Current implementation is built on top of MPI and uses CPUs to maximize the portability across the platforms.
Distributed computing, machine learning↗
Early Application Results on Pre-exascale Architecture with Analysis of Performance Challenges and Projections (Milestone PM-AD-1080 WBS 2.2)
This Exascale Computing Project (ECP) Milestone Report summarizes the status of all 30 ECP Applications Development (AD) sub-projects at the end of FY19. In August and September of 2019, a comprehensive assessment of AD projects was conducted jointly by the ECP leadership and a team of external subject matter experts. Reviews took place in person over five days-two at the National Renewable Energy Laboratory and three at Argonne National Laboratory and the University of Chicago. The review committees were tasked with evaluating each sub-project's progress in porting their code(s) to current multi-GPU architectures considered precursors to planned exascale machines. This includes characterizing which modules have been ported to multi-accelerator nodes, initial performance analyses, the status of software integration, and a current vision of successes, obstacles, and next steps. As such this report contains not only an accurate snapshot of each sub-project's current status, but also represents an unprecedentedly broad account of experiences porting large scientific applications to next-generation HPC architectures.
97 MATHEMATICS AND COMPUTING↗
Revolutionizing Neuromorphic Computing for Science (Brochure on the 2024 ASCR Workshop on Neuromorphic Computing for Science)
The ASCR basic research needs for Neuromorphic Computing for Science workshop was held in September 2024. The workshop brochure and report aim to inform and draft a set of grand challenges for advancing the field of neuromorphic computing and developing proof of principle neuromorphic circuits applicable for High Performance Computer (HPC) acceleration for scientific discovery, and brainstorm ideas needed for a successful, robust, and world leading basic research program. The resulting priority research directions are: (1) Neuromorphic computing circuit primitives; (2) Connectivity, communication, and hardware integration; (3) Neuroscience-derived dynamics and algorithms; and (4) Ecosystem for scalable neuromorphic co-design. Breakthroughs in understanding, designing, and prototyping the circuitry and simulation capabilities for a truly neuromorphic computer are essential to enable progress in the field.
97 MATHEMATICS AND COMPUTING↗
Revolutionizing Neuromorphic Computing for Science (Report for the 2024 ASCR Workshop on Neuromorphic Computing for Science)
The ASCR basic research needs for Neuromorphic Computing for Science workshop was held in September 2024. The workshop brochure and report aim to inform and draft a set of grand challenges for advancing the field of neuromorphic computing and developing proof of principle neuromorphic circuits applicable for High Performance Computer (HPC) acceleration for scientific discovery, and brainstorm ideas needed for a successful, robust, and world leading basic research program. The resulting priority research directions are: (1) Neuromorphic computing circuit primitives; (2) Connectivity, communication, and hardware integration; (3) Neuroscience-derived dynamics and algorithms; and (4) Ecosystem for scalable neuromorphic co-design. Breakthroughs in understanding, designing, and prototyping the circuitry and simulation capabilities for a truly neuromorphic computer are essential to enable progress in the field.
97 MATHEMATICS AND COMPUTING↗
HPC-FAIR: A Framework Managing Data and AI Models for Analyzing and Optimizing Scientific Applications
The increasing reliance on machine learning (ML) to analyze and optimize large-scale scientific applications on supercomputers faces a significant bottleneck: the lack of readily available, high-quality training datasets and the difficulty in reusing existing AI models. This project was motivated by the urgent need to address the “FAIR” principles (Findability, Accessibility, Interoperability, Reusability) for both training datasets and AI models in the high-performance computing (HPC) domain. The project developed HPC-FAIR, a high-performance computing data management framework designed to centralize HPC-related datasets and AI models within a unified hub. To ensure interoperability, the framework established a standardized representation and vocabulary (ontology) for both data and models. HPC-FAIR also implemented automated workflows to streamline data processing, model access, and benchmarking. Additionally, the project focused on optimizing data harnessing efficiency through advanced techniques like deep reuse and compression-based analytics.
97 MATHEMATICS AND COMPUTING↗
SYMBIOSYS: A Methodology for Performance Analysis of Composable HPC Data Services
Microservices are a powerful new way of building, customizing, and deploying distributed services owing to their flexibility and maintainability. Several large-scale distributed platforms have emerged to serve the growing needs of data-centric workloads and services in commercial computing. Concurrently, high-performance computing (HPC) systems and software are rapidly evolving to meet the demands of diversified applications and heterogeneity. The interplay of hardware factors, software configuration parameters, and the flexibility offered with a microservice architecture makes it nontrivial to estimate the optimal service instantiation for a given application workload. Further, this problem is exacerbated when considering that these services operate in a dynamic and heterogeneous HPC environment. An optimally integrated service can be vastly more performant than a haphazardly integrated one. Existing performance tools for HPC either fail to understand the request-response model of communication inherent to microservices or they operate within a narrow scope, limiting the insight that can be gleaned from employing them in isolation. We propose a methodology for integrated performance analysis of HPC microservices frameworks and applications called SYMBIOSYS. We describe its design and implementation within the context of the Mochi framework. This integration is achieved by combining distributed callpath profiling and tracing with a performance data exchange strategy that collects fine-grained, low-level metrics from the RPC communication library and network layers. The result is a portable, low-overhead performance analysis setup that provides a holistic profile of the dependencies among microservices and how they interact with the Mochi RPC software stack. Using HEPnOS, a production-quality Mochi data service, we demonstrate the low-overhead operation of SYMBIOSYS at scale and use it to identify the root causes of poorly performing service configurations.
microservices↗
Resilience Design Patterns: A Structured Approach to Resilience at Extreme Scale (V.2.0)
Reliability is a serious concern for future extreme-scale high-performance computing (HPC) systems. Projections based on the current generation of HPC systems and technology roadmaps suggest the prevalence of very high fault rates in future systems. The errors resulting from these faults will propagate and generate various kinds of failures, which may result in outcomes ranging from result corruptions to catastrophic application crashes. Therefore, the resilience challenge for extreme-scale HPC systems requires coordination between various hardware and software technologies that are capable of handling a broad set of fault models at accelerated fault rates. Also, due to practical limits on power consumption in future HPC systems, they are likely to embrace innovative architectures, increasing the levels of hardware and software complexities. Therefore, the techniques that seek to improve resilience must navigate the complex trade-off space between resilience and the overheads to power consumption and performance. While the HPC community has developed various resilience solutions, application-level techniques as well as system-based solutions, the solution space of HPC resilience techniques remains fragmented. There are no formal methods to integrate the various HPC resilience techniques into composite solutions, nor are there methods to holistically evaluate the adequacy and efficacy of such solutions in terms of their protection coverage, and their performance & power efficiency characteristics. Additionally, few implementations of current resilience solutions are portable to newer architectures and software environments that will be deployed on future systems. We developed a new structured approach to the management of HPC resilience using the concept of resilience-based design patterns. In general, a design pattern is a repeatable solution to a commonly occurring problem. We identified the well-known solutions that are commonly used to deal with faults, errors and failures in HPC systems. In the initial design patterns specification (version 1.0), we described the various solutions, which address specific problems in the design of resilient HPC environments, in the form of patterns. Each pattern describes a problem caused by a fault, error or failure event in an HPC environment, and then describes the core of the solution of the problem in such a way that this solution may be adapted to different systems and implemented at different layers of the system stack. The catalog of these resilience design patterns provides designers with a collection of design elements. To construct complete resilience solutions using combinations of various patterns, we defined a framework that enhances HPC designers' understanding of the important constraints and the opportunities for the design patterns to be implemented and deployed at various layers of the system stack. The design framework is also useful for establishing interfaces and mechanisms to coordinate flexible fault management across hardware and software components, as well as to consider the trade-off between performance, resilience, and power consumption when constructing a solution. The resilience design patterns specification version 1.1 included more detailed explanations of the pattern solutions, the context in which the patterns are applicable, and the implications for hardware or software design. It also provided several additional examples and detailed case studies to demonstrate the use of patterns to build realistic solutions. In version 1.2 of the specification document, we have improved the pattern descriptions, including graphical representations of the pattern components. These improvements are largely based on critical comments, feedback and suggestions received from pattern experts and readers of the previous versions of the specification. The pattern classification has been modified to further clarify the relationships between pattern categories. This version of the specification also introduces a pattern language for resilience design patterns. The pattern language presents the patterns in the catalog as a network, revealing the relations among the resilience patterns. The language provides designers with the means to explore alternative techniques for handling a specific fault model that may have different efficiency and complexity characteristics. Using the pattern language also enables the design and implementation of comprehensive resilience solutions as a set of interconnected resilience patterns that can be instantiated across layers of the system stack. The overall goal of this work is to provide hardware and software designers, as well as the users and operators of HPC systems, a systematic methodology for the design and evaluation of resilience technologies in HPC systems that keep scientific applications running to a correct solution in a timely and cost-efficient manner despite frequent faults, errors, and failures of various types. Version 2.0 expands the resilience design pattern classification and catalog to include self-stabilization patterns and reliability, availability and performance models for each structural pattern.
97 MATHEMATICS AND COMPUTING↗
High Performance Computing Application I/O Traces
The dataset comprises trace files from high performance computing (HPC) simulations. The trace files contain records of every I/O operation executed by a simulation application run, including I/O operations from HDF5, MPI-IO, and POSIX and all of the parameters supplied to those operations, e.g. file name, offset, and flags. The traces are generated by executing a simulation application that is linked with the Recorder tracing tool (https://github.com/uiuc-hpc/Recorder). The Recorder trace tool intercepts the I/O calls made by the application, records the I/O trace record, and then calls the intended I/O call so that the operation executes.
Wang, C.↗
Performance Analysis and Optimization for Scientific Data Workloads
Scientific data generated at experimental and observational facilities are increasingly being processed on large-scale compute systems. Most of the experimental data analysis workflows are not designed or implemented to run on large scale environments and take full advantage of HPC compute and storage resources. These applications are unlike the traditional tightly-coupled scientific applications and hence face significant performance and scalability challenges as the volume of data increases exponentially. In this paper, we conduct a performance and scalability analysis for experimental analysis applications and workflows operating on data from light sources. Our analysis detects and quantifies I/O performance, scalability and runtime bottlenecks for three data analysis applications that run on NERSC resources. Based on our analysis we propose and implement a set of optimizations that lead to reducing the amount of time spent on I/O operations by almost 90%.
97 MATHEMATICS AND COMPUTING↗