Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “DATA PROCESSOR”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 433 records · Page 24

ParaText : scalable text analysis and visualization.

Automated analysis of unstructured text documents (e.g., web pages, newswire articles, research publications, business reports) is a key capability for solving important problems in areas including decision making, risk assessment, social network analysis, intelligence analysis, scholarly research and others. However, as data sizes continue to grow in these areas, scalable processing, modeling, and semantic analysis of text collections becomes essential. In this paper, we present the ParaText text analysis engine, a distributed memory software framework for processing, modeling, and analyzing collections of unstructured text documents. Results on several document collections using hundreds of processors are presented to illustrate the exibility, extensibility, and scalability of the the entire process of text modeling from raw data ingestion to application analysis.

99 GENERAL AND MISCELLANEOUS↗

Solving a trillion unknowns per second with HPGMG on Sunway TaihuLight

Benchmarks for supercomputers are important tools, not only for evaluating and ranking modern supercomputers, but also for providing hints for future architecture design. As a new benchmark, HPGMG (high performance geometric multigrid) solves a linear equation set with a full geometric multi-grid algorithm. It involves computation on different scales, data movement with various volumes, global communication and neighbor communication with both large and small messages, etc., and is more correlated to real world applications than traditional benchmarks such as LINPACK. Therefore, it is desirable to examine how well HPGMG can perform on leadership supercomputers such as Sunway Taihulight. Sunway Taihulight, the No. 1 supercomputer in the Top 500 list from June 2016 to June 2018, which uses a specially designed many-core architecture SW26010, is of great interest to the community of high performance computing. With careful analysis and code design, we came up with an efficient implementation of HPGMG on SW26010 processors. We not only employed traditional optimization techniques such as 2.5D partitioning, double buffering, and collective data load, but also introduced a micro-benchmark to help with the choice of optimization direction and parameter tuning. Another contribution is that we proposed a new procedure for the major operations, by granulating and reordering the smooth function and the ghost exchange operation, leading to reduced memory copy and accelerated communication process. Our optimized implementation of HPGMG on Sunway TaihuLight achieved a ground-breaking performance of 1.036 × 10 12 Degrees of Freedom per second at the finest level, which is No. 1 on the HPGMG list of Nov 2017.

97 MATHEMATICS AND COMPUTING↗

Apache Calcite: A Foundational Framework for Optimized Query Processing Over Heterogeneous Data Sources

Apache Calcite is a foundational software framework that provides query processing, optimization, and query language support to many popular open-source data processing systems such as Apache Hive, Apache Storm, Apache Flink, Druid, and MapD. The goal of this paper is to formally introduce Calcite to the broader research community, brie y present its history, and describe its architecture, features, functionality, and patterns for adoption. Calcite's architecture consists of a modular and extensible query optimizer with hundreds of built-in optimization rules, a query processor capable of processing a variety of query languages, an adapter architecture designed for extensibility, and support for heterogeneous data models and stores (relational, semi-structured, streaming, and geospatial). This exible, embeddable, and extensible architecture is what makes Calcite an attractive choice for adoption in big-data frameworks. It is an active project that continues to introduce support for the new types of data sources, query languages, and approaches to query processing and optimization.

Begoli, Edmon↗

A Parallel Graph Environment for Real-World Data Analytics Workflows

Economic competitiveness and national security depend increasingly on the insightful analysis of large data sets. The diversity of real-world data sources and analytic workflows impose challenging hardware and software requirements for parallel graph platforms. The irregular nature of graph methods is not supported well by the deep memory hierarchies of conventional distributed systems, requiring new processor and runtime system designs to tolerate memory and synchronization latencies. Moreover, the efficiency of relational table operations and matrix computations are not attainable when data is stored in common graph data structures. In this paper, we present HAGGLE, a high-performance, scalable data analytics platform. The platform’s hybrid data model supports a variety of distributed, thread-safe data structures, parallel programming constructs, and persistent and streaming data. An abstract runtime layer enables us to map the stack to conventional, distributed computer systems with accelerators. The runtime uses multithreading, active messages, and data aggregation to hide memory and synchronization latencies on large-scale systems.

Graph Analytics↗

Special Analysis of the Area 3 Radioactive Waste Management Site at the Nevada National Security Site, Nye County, Nevada

This report describes the methods and results of a special analysis (SA) of the Area 3 Radioactive Waste Management Site (RWMS) on the Nevada National Security Site (NNSS). The purpose of the SA is to determine if the approved performance assessment (PA) and composite analysis (CA) (Shott et al., 2001) remain valid. The Area 3 RWMS PA and CA were prepared as a single document and received conditional approval on October 6, 1999. A conditional Disposal Authorization Statement (DAS) for the Area 3 RWMS was issued on October 20, 1999. Since preparation of the approved PA and CA, new information and additional environmental monitoring data have been used to update the PA and CA. At the same time, continual advancements in computer processors and software have allowed improvement to the PA and CA models. Annual reviews of the PA and CA required by U.S. Department of Energy (DOE) Order DOE O 435.1 have documented multiple changes occurring since preparation of the PA and CA. Potentially important changes include: Development of a new and improved baseline PA and CA model implemented in the probabilistic GoldSim simulation platform. A significant increase in the waste inventory disposed at the site. Revision and updating of model parameters based on additional years of site monitoring data and new research and development results. Although changes have occurred, many important PA/CA issues remain unchanged, including the site conceptual model, important features, events, and processes, and the points of compliance. The SA is performed to document the current status of the PA/CA model and to quantitatively assess the impact of cumulative changes on the PA and CA results. The results of the SA are used to assess the validity of the approved PA/CA and make a determination if revision of the PA or CA is necessary. The SA was performed using the Area 3 RWMS, version 2.102, GoldSim model, the current baseline PA/CA model. Comparison of the maximum SA results with the PA performance objectives indicates that there continues to be a reasonable expectation of compliance. The resident exposure scenario was evaluated for compliance with the air pathway and all-pathways annual total effective dose (TED) performance objectives. The maximum mean air pathway TED, 7E-6 millisievert (mSv) at 1,000 years (y) has decreased relative to the approved PA and is significantly less than the 0.1 mSv limit. The maximum mean all-pathways annual TED, 7E-5 mSv at 1,000 y has increased but remains a small fraction of the 0.25 mSv limit. The SA maximum mean radon-222 (222Rn) flux density, 0.03 becquerel per square meter per second (Bq m-2 s-1), has increased relative to the PA results but is significantly less than the 0.74 Bq m-2 s-1 limit. The SA results continue to support a conclusion that the disposed waste inventory is protective of intruders and groundwater resources. The maximum mean intruder TED, 0.01 mSv for an acute construction scenario at the U-3ah/at disposal unit, was less than the 5 mSv performance measure. Site monitoring data and research results continue to support a conclusion that a groundwater pathway will not exist within the 1,000 y compliance period. Projected releases to the environment are a small fraction of the performance objectives. Cost-effective options for reducing releases further are unlikely to exist. Therefore, releases from the Area 3 RWMS are judged to be as low as reasonably achievable. Comparison of the maximum CA result with the 0.3 mSv CA dose constraint indicates that no action is required to reduce the dose from the Area 3 RWMS and all interacting sources of residual radioactive contamination. The SA maximum mean CA annual TED, 0.02 mSv at 1,000 y, has increased from the approved CA result but remains less than 10% of the dose constraint. The CA TED continues to be due predominantly to inhalation of plutonium-239 resuspended from soils contaminated by nuclear weapons tests conducted near the Area 3 RWMS. The SA results estimated with the Area 3 RWMS version 2.102 model indicate that changes to the PA and CA do not significantly alter the PA/CA results or conclusions. Although increases occur for the all-pathways annual TED, the 222Rn flux density, and the CA annual TED, all PA and CA results are a small fraction of the performance objectives or dose constraints. The same conclusion was reached for the approved PA and CA. The SA results continue to support a conclusion that there is a reasonable expectation of meeting all performance objectives and the CA dose constraint. There is no need to revise the DAS at this time. The SA results support a conclusion that the Area 3 RWMS PA and CA remain valid, and revision is not necessary at this time.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

End-System Aware Peak Link Utilization Transport for Big Data Transfer over Dedicated Connections

High-performance networks provisioning 10/40-Gbps dedicated channels have been rapidly developed andincreasingly deployed to support big data transfers on the order of terabytes or petabytes in large-scale network-intensiveapplications. However, end users have not seen corresponding increase in application throughput mainly because traditional end-to-end transport methods are not optimized for such connections. New congestion or flow control mechanisms are desirable to meet the challenges brought to transport protocol design by dedicated connections. The advent and proliferation of multi-core processors make it now possible to improve application throughput by providing multiple processing and network resources to a single data transfer. Based on the existing PLUT method, we proposed a new dynamic buffer management scheme and an automatic parallelism tuning mechanism to achieve the maximal bottleneck rate. The performance superiority of PLUT is justified by the experimental results collected over a local 10Gbps dedicated connection.

Rao, Nageswara↗

Implementation of a level 1 trigger system using high speed serial (VXS) techniques for the 12GeV high luminosity experimental programs at Thomas Jefferson National Accelerator Facility

We will demonstrate a hardware and firmware solution for a complete fully pipelined multi-crate trigger system that takes advantage of the elegant high speed VXS serial extensions for VME. This trigger system includes three sections starting with the front end crate trigger processor (CTP), a global Sub-System Processor (SSP) and a Trigger Supervisor that manages the timing, synchronization and front end event readout. Within a front end crate, trigger information is gathered from each 16 Channel, 12 bit Flash ADC module at 4 nS intervals via the VXS backplane, to a Crate Trigger Processor (CTP). Each Crate Trigger Processor receives these 500 MB/S VXS links from the 16 FADC-250 modules, aligns skewed data inherent of Aurora protocol, and performs real time crate level trigger algorithms. The algorithm results are encoded using a Reed-Solomon technique and transmission of this Level 1 trigger data is sent to the SSP using a multi-fiber link. The multi-fiber link achieves an aggregate trigger data transfer rate to the global trigger at 8 Gb/s. The SSP receives and decodes Reed-Solomon error correcting transmission from each crate, aligns the data, and performs the global level trigger algorithms. The entire trigger system is synchronous and operates at 250 MHz with the Trigger Supervisor managing not only the front end event readout, but also the distribution of the critical timing clocks, synchronization signals, and the global trigger signals to each front end readout crate. These signals are distributed to the front end crates on a separate fiber link and each crate is synchronized using a unique encoding scheme to guarantee that each front end crate is synchronous with a fixed latency, independent of the distance between each crate. The overall trigger signal latency is <3 uS, and the proposed 12GeV experiments at Jefferson Lab require up to 200KHz Level 1 trigger rate.

43 PARTICLE ACCELERATORS↗

The Computational Capacity of Mem-LRC Reservoirs

Reservoir computing has a emerged as a powerful tool in data-driven time series analysis. The possibility of utilizing hardware reservoirs as specialized co-processors has generated interest in the properties of electronic reservoirs, especially those based on memristors as the nonlinearity of these devices should translate to an improved nonlinear computational capacity of the reservoir. However, designing these reservoirs requires a detailed understanding of how memristive networks process information which has thus far been lacking. In this work, we derive an equation for general memristor-inductor-resistor-capacitor (MEM-LRC) reservoirs that includes all network and dynamical constraints explicitly. Utilizing this we undertake a study of the computational capacity of these reservoirs. We demonstrate that hardware reservoirs may be constructed with extensive memory capacity and that the presence of memristors enacts a tradeoff between memory capacity and nonlinear computational capacity. Here, using these principles, we design reservoirs to tackle problems in signal processing, paving the way for applying hardware reservoirs to high-dimensional spatiotemporal systems.

Circuits with memory↗

Logic in Memory Emulator

Logic in Memory Emulator (LiME) is a hardware/software tool specially designed for memory system evaluation and experiment. Emerging memories display a wide range of bandwidths, latencies, and capacities, making it challenging for the computer architect to navigate the design space of potential memory configurations, and for the application developer to assess performance implications of using such memories. With the LiME framework, architectural ideas can be prototyped in great detail yet with sufficient performance to support realistic evaluation on long running applications. LiME consists of two fundamental components: 1) the hardware and OS infrastructure for the emulator, and 2) a suite of benchmark applications to assist in characterizing the performance of current and future computer architectures. Some of the applications have been collected from other open source projects. Uses: Logging, replay and analysis of an application's memory behavior Evaluate impact of emerging memory technology on application performance. Emulate complex memory interactions in whole applications orders of magnitude faster than software simulation. Emulate acceleration hardware co-located with the memory subsystem. Features: Capture and log external memory accesses to a separate off-chip memory device without affecting application execution. Memory traces include the address, length, timestamp, and optionally the data for each transaction. Captured trace data can be saved to an SD card for off-line analysis. Configure a wide range of memory latencies in sub-nanosecond increments that encompass highbandwidth and storage class memories. Specify regions of interest (ROI) in applications to reduce the amount of trace data captured for analysis. Currently supports execution on Xilinx Zynq SoC which integrates an ARM processor with FPGA logic on a single device. Applications can be run under Linux or in bare metal mode on the ARM cores.

Jain, AbhishekK↗

ParaDyn Implementation in the US Navy's DYSMAS Simulation System: FY08 Progress Report

The goal of this project is to increase the computational efficiency and capacity of the Navy's DYSMAS simulation system for full ship shock response to underwater explosion. Specifically, this project initiates migration to a parallel processing capability for the structural portion of the overall fluid-structure interaction model. The capstone objective for the first phase is to demonstrate operation of the DYSMAS simulation engine with a production model on a Naval Surface Warfare Center (IHD) parallel platform using the ParaDyn code for parallel processing of the structural dynamics. This year saw a successful launch to integrate ParaDyn, the high-parallel structural dynamics code from Lawrence Livermore National Laboratory (LLNL), into the DYSMAS system for simulating the response of ship structures to underwater explosion (UNDEX). The current LLNL version of DYNA3D, representing ten years of general development beyond the source branch used to initiate DYNA-N customization for DYSMAS, was first connected to the GEMINI flow code through DYSMAS Standard Coupler Interface (SCI). This permitted an early 'sanity check' by Naval Surface Warfare Center, Indian Head Division (NSWC-IHD) personnel that equivalent results were generated for their standard UNDEX test problems, thus ensuring the Verification & Validation pedigree they have developed remains intact. The ParaDyn code was then joined to the SCI in a manner requiring no changes to GEMINI. Three NSWC-IHD engineers were twice hosted at LLNL to become familiar with LLNL computer systems, the execution of the prototype software system, and to begin assessment of its accuracy and performance. Scaling data for the flow solver GEMINI was attained up to a one billion cell, 1000 processor run. The NSWC-IHD engineers were granted privileges to continue their evaluations through remote connections to LLNL's Open Computing Facility. Finally, the prototype changes were integrated into the mainline ParaDyn source repository and issued as part of its Version 8.1 beta release. This source was transmitted to NSWC-IHD and in collaboration with LLNL personnel the entire ParaDyn software suite successfully installed and demonstrated on its new SGI Altix machine. The ability of even minor numbers of processors for the structural dynamics to impact overall time-to-solution for DYSMAS has been demonstrated. Assessments of combined parallel efficiencies are beginning to highlight areas for further DYSMAS optimizations.

42 ENGINEERING↗

Bayesian Estimator for Angle Recovery: Event Classification and Reconstruction in Positron Emission Tomography

PET at the highest level is an inverse problem: reconstruct the location of the emission (which localize biological function) from detected photons. Ideally, one would like to directly measure an annihilation photon's incident direction on the detector. In the developed algorithm, Bayesian Estimation for Angle Recovery (BEAR), we utilized the increased information gathered from localizing photon interactions in the detector and developed a Bayesian estimator for a photon's incident direction. Probability distribution functions (PDFs) were filled using an interaction energy weighted mean or center of mass (COM) reference space, which had the following computational advantages: (1) a significant reduction in the size of the data in measurement space, making further manipulation and searches faster (2) the construction of COM space does not depend on measurement location, it takes advantage of measurement symmetries, and data can be added to the training set without knowledge and recalculation of prior training data, (3) calculation of posterior probability map is fully parallelizable, it can scale to any number of processors. These PDFs were used to estimate the point spread function (PSF) in incident angle space for (i) algorithm assessment and (ii) to provide probability selection criteria for classification. The algorithm calculates both the incident {theta} and {phi} angle, with {approx}16 degrees RMS in both angles, limiting the incoming direction to a narrow cone. Feature size did not improve using the BEAR algorithm as an angle filter, but the contrast ratio improved 40% on average.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Portable and Transparent Message Compression in MPI Libraries to Improve the Performance and Scalability of Parallel Applications

The goal of this project has been to develop a lossless compression algorithm for message-passing libraries that can accelerate HPC systems by reducing the communication time. Because both compression and decompression have to be performed in software in real time, the algorithm has to be extremely fast while still delivering a good compression ratio. During the first half of this project, they designed a new compression algorithm called FPC for scientific double-precision data, made the source code available on the web, and published two papers describing its operation, the first in the proceedings of the Data Compression Conference and the second in the IEEE Transactions on Computers. At comparable average compression ratios, this algorithm compresses and decompresses 10 to 100 times faster than BZIP2, DFCM, FSD, GZIP, and PLMI on the three architectures tested. With prediction tables that fit into the CPU's L1 data acache, FPC delivers a guaranteed throughput of six gigabits per second on a 1.6 GHz Itanium 2 system. The C source code and documentation of FPC are posted on-line and have already been downloaded hundreds of times. To evaluate FPC, they gathered 13 real-world scientific datasets from around the globe, including satellite data, crash-simulation data, and messages from HPC systems. Based on the large number of requests they received, they also made these datasets available to the community (with permission of the original sources). While FPC represents a great step forward, it soon became clear that its throughput was too slow for the emerging 10 gigabits per second networks. Hence, no speedup can be gained by including this algorithm in an MPI library. They therefore changed the aim of the second half of the project. Instead of implementing FPC in an MPI library, they refocused their efforts to develop a parallel compression algorithm to further boost the throughput. After all, all modern high-end microprocessors contain multiple CPUs on a single die and will undoubtedly contain even larger numbers of CPUs in the future. Hence, a compute node in a parallel system is likely to have multiple processors per interconnection link. Thus, they investigated how to best exploit multiple CPUs to compress and decompress data for a single link, i.e., they developed a parallel version of the FPC algorithm called pFPC. On a 4-core 3 GHz Xeon system, pFPC exceeds 10 gigabits per second both in compression and decompression throughput while still reaching a compression ratio of 18%, which is reasonable for their double-precision floating-point datasets. At half this throughput, it achieves a 41% compression ratio. Moreover, the algorithm scales well to four cores and promises even higher throughputs on larger machines. Additional results and a description of the pFPC algorithm have recently been published in the proceedings of the Data Compression Conference. The C source code of pFPC is also publicly available.

97 MATHEMATICS AND COMPUTING↗

Fine-grained parallelization of the Car-Parrinello ab initio molecular dynamics method on the IBM Blue Gene/L supercomputer

Important scientific problems can be treated via ab initio-based molecular modeling approaches, wherein atomic forces are derived from an energy Junction that explicitly considers the electrons. The Car-Parrinello ab initio molecular dynamics (CPAIMD) method is widely used to study small systems containing on the order of 10 to 103 atoms. However, the impact of CPAIMD has been limited until recently because of difficulties inherent to scaling the technique beyond processor numbers about equal to the number of electronic states. CPAIMD computations involve a large number of interdependent phases with high interprocessor communication overhead. These phases require the evaluation of various transforms and non-square matrix multiplications that require large interprocessor data movement when efficiently parallelized. Using the Charm++ parallel programming language and runtime system, the phases are discretized into a large number of virtual processors, which are, in turn, mapped flexibly onto physical processors, thereby allowing interleaving of work. Algorithmic and IBM Blue Gene/L(tm) system-specific optimizations are employed to scale the CPAIMD method to at least 30 times the number of electronic states in small systems consisting of 24 to 768 atoms (32 to 1,024 electronic states) in order to demonstrate fine-grained parallelism. The largest systems studied scaled well across the entire machine (20,480 nodes).

Bohm, E.↗

High-performance computing for airborne applications

Recently, there has been attempts to move common satellite tasks to unmanned aerial vehicles (UAVs). UAVs are significantly cheaper to buy than satellites and easier to deploy on an as-needed basis. The more benign radiation environment also allows for an aggressive adoption of state-of-the-art commercial computational devices, which increases the amount of data that can be collected. There are a number of commercial computing devices currently available that are well-suited to high-performance computing. These devices range from specialized computational devices, such as field-programmable gate arrays (FPGAs) and digital signal processors (DSPs), to traditional computing platforms, such as microprocessors. Even though the radiation environment is relatively benign, these devices could be susceptible to single-event effects. In this paper, we will present radiation data for high-performance computing devices in a accelerated neutron environment. These devices include a multi-core digital signal processor, two field-programmable gate arrays, and a microprocessor. From these results, we found that all of these devices are suitable for many airplane environments without reliability problems.

42 ENGINEERING↗

Unified Memory: GPGPU-Sim/UVM Smart Integration

CPU/GPU heterogeneous compute platforms are an ubiquitous element in computing and a programming model specified for this heterogeneous computing model is important for both performance and programmability. A programming model that exposes the shared, unified, address space between the heterogeneous units is a necessary step in this direction as it removes the burden of explicit data movement from the programmer while maintaining performance. GPU vendors, such as AMD and NVIDIA, have released software-managed runtimes that can provide programmers the illusion of unified CPU and GPU memory by automatically migrating data in and out of the GPU memory. However, this runtime support is not included in GPGPU-Sim, a commonly used framework that models the features of a modern graphics processor that are relevant to non-graphics applications. UVM Smart was developed, which extended GPGPU-Sim 3.x to in- corporate the modeling of on-demand pageing and data migration through the runtime. This report discusses the integration of UVM Smart and GPGPU-Sim 4.0 and the modifications to improve simulation performance and accuracy.

97 MATHEMATICS AND COMPUTING↗

A fast and memory-sparing probabilistic selection algorithm for the GPU

A fast and memory-sparing probabilistic top-N selection algorithm is implemented on the GPU. This probabilistic algorithm gives a deterministic result and always terminates. The use of randomization reduces the amount of data that needs heavy processing, and so reduces both the memory requirements and the average time required for the algorithm. This algorithm is well-suited to more general parallel processors with multiple layers of memory hierarchy. Probabilistic Las Vegas algorithms of this kind are a form of stochastic optimization and can be especially useful for processors having a limited amount of fast memory available.

99 GENERAL AND MISCELLANEOUS↗

Detection of Coherent Structures in Extreme-Scale Simulations

The analysis of coherent structures is a common problem in many scientific domains ranging from astrophysics to combustion, fusion, and materials science. The data from three-dimensional simulations are analyzed to detect the structures, extract statistics on them, and track them over time to gain insights into the phenomenon being modeled. This analysis is typically done off-line, using data that have been written out by the simulations. However, the move towards extreme scale architectures, with multi-core processors and graphical processing units, will affect how such analysis is done as it is unlikely that the systems will support the I/O bandwidth required for off-line analysis. Moving the analysis in-situ is a solution only if we know a priori what analysis will be done, as well as the algorithms used and their parameter settings. Even then, we need to ensure that this will not substantially increase the memory requirements or the data movement as the former will be limited and the latter will be expensive. In the Exa-DM project, a collaboration between Lawrence Livermore National Laboratory and University of Minnesota, we are exploring ways in which we can address the conflicting demands of coherent structure analysis of simulation data and the architecture of modern parallel systems, while enabling scientific discovery at the exascale. In this paper, we describe our work in two areas: the in situ implementation of an existing algorithm for coherent structure analysis and the use of graph-based techniques to efficiently compress the data.

79 ASTRONOMY AND ASTROPHYSICS↗

HPDR: High-Performance Portable Scientific Data Reduction Framework

The rapid growth in scientific data generation is outpacing advancements in computing systems necessary for efficient storage, transfer, and analysis, particularly in the context of exascale computing. With the deployment of first-generation exascale computing systems and next-generation experimental facilities, this gap is widening and necessitates effective data reduction techniques to manage enormous data volumes. Over the past decade, various data reduction methods, including lossless compression, error-controlled lossy compression, and data refactoring, have been developed to accelerate I/O in scientific workflows. Despite significant reductions in data volume, these methods introduce considerable computational overhead, which can become the new bottleneck in data processing. To mitigate this, GPU-accelerated data reduction algorithms have been introduced. However, challenges remain in their integration into exascale workflows, including limited portability across different GPU architectures, substantial memory transfer overhead, and reduced scalability on dense multi-GPU systems. To address these challenges, we propose HPDR, a high-performance and portable data reduction framework. HPDR is designed to enable the execution of state-of-the-art reduction algorithms across diverse processor architectures while reducing memory transfer overhead to 2.3 % of the original, resulting in up to 3.5× faster throughput compared to existing solutions. It also achieves up to 96% of the theoretical speedup in multi-GPU settings. In addition, evaluations on accelerating I/O operations at scale up to 1,024 nodes of the Frontier supercomputer demonstrate that HPDR can achieve up to 103 TB/s reduction throughput, providing up to 4× acceleration in parallel I/O performance compared to existing data reduction routines. This work highlights the potential of HPDR to significantly enhance data reduction efficiency in exascale computing environments.

Chen, Jieyang [University of Oregon]↗