Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “memory usage”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 253 records · Page 14

Trinity Benchmarks on the Intel Xeon Phi (Knights Corner)

This report documents the early experiences with porting and performance analysis of the Tri-Lab Trinity benchmark applications on Intel Xeon Phi (Knights Corner) (KNC) processor. KNC, the second generation of the Intel Many Integrated Core (MIC) architectures, uses a large number of small P54C-x86 cores with wide vector units and is deployed as PCI bus attached process accelerators. Sandia has experimental test beds of small InifiniBand clusters and workstations to investigate the performance of the MIC architecture. On these experimental test beds the programming models that may be investigated are "offload", "symmetric" and "native". Among these program usage models our primary interest is in the so called "native" mode, because the planned Trinity system to be deployed in 2016 using the next generation MIC processor architecture called Knights Landing would be self-hosted. Trinity / NERSC-8 benchmark programs cover a variety of scientific disciplines and they were used to guide the procurement of these systems. Architectures such as the Intel MIC are well suited to study evolving processor architectures and a usage model commonly referred to as MPI + X that facilitates migration of our applications to use both coarse grain and fine grain parallelism. Our focus with the applications selected is on the efficacy of algorithms in these applications to take advantage of features like: large number of cores, wide vector units, higher-bandwidth and deeper memory sub-system. This is a first step towards understanding applications, algorithms and programming environments for Trinity and future exascale computing systems.

97 MATHEMATICS AND COMPUTING↗

SeaStar Unchained: Multiplying the Performance of the Cray SeaStar Network

The Oak Ridge Leadership Computing Facility (OLCF) supports many different systems and many differ- ent interconnects. The only common programming interfaces across these systems are BSD Sockets and MPI. Due to the design assumptions such as implicit buffering leading to extra copies, Sockets performance is almost universally lower than the native interface. Even in the cases that Sockets provides similar bandwidth as the native interface, it suffers from excessive CPU usage. MPI is the de-facto interface for intra- job communication, but is difficult to use between jobs and provides no ability to communicate with service nodes or off- system nodes (e.g. for I/O forwarding). We have developed the Common Communication Interface (CCI), a programming interface that exposes the advances in interconnect hardware, notably Remote Direct Memory Access (RDMA) and operating system (OS) bypass, while imposing as little overhead as possible. This API directly supports inter-job as well as off- system communication. CCI is a lightweight abstraction layer that provides point-to-point messaging and remote memory access. The Cray SeaStar ASIC, with its programmable embedded processor, provides an excellent platform to investigate the properties of various network protocols and programming interfaces. This paper describes our native implementation CCI on the SeaStar platform, and details how we implemented full OS bypass for common operations. We demonstrate a 30% to 50% reduction in latency, more than a six-fold increase in message injection rate, and an almost 7x improvement in bandwidth for small message sizes when compared to the generic Cray Portals implementation.

Dillow, David A↗

Performance and Energy Usage of Workloads on KNL and Haswell Architectures. In: High Performance Computing Systems. Performance Modeling, Benchmarking, and Simulation

Manycore architectures are an energy-efficient step towards exascale computing within a constrained power budget. The Intel Knights Landing (KNL) manycore chip is a specific example of this and has seen early adoption by a number of HPC facilities. It is therefore important to understand the performance and energy usage characteristics of KNL. In this paper, we evaluate the performance and energy efficiency of KNL in contrast to the Xeon (Haswell) architecture for applications representative of the workload of users at NERSC. We consider the optimal MPI/OpenMP configuration of each application and use the results to characterize KNL in contrast to Haswell. As well as traditional DDR memory, KNL contains MCDRAM and we also evaluate its efficacy. Our results show that, averaged over our benchmarks, KNL is 1.84 × more energy efficient than Haswell and has 1.27 × greater performance.

Benchmarking↗

GPU Usage in ATLAS Reconstruction and Analysis

With Graphical Processing Units (GPUs) and other kinds of accelerators becoming ever more accessible, High Performance Computing Centres all around the world using them ever more, ATLAS has to find the best way of making use of such accelerators in much of its computing. Tests with GPUs - mainly with CUDA - have been performed in the past in the experiment. At that time the conclusion was that it was not advantageous for the ATLAS offline and trigger software to invest time and money into GPUs. However as the usage of accelerators has become cheaper and simpler in recent years, their re-evaluation in ATLAS's offline software is warranted. We show new results of using GPU accelerated calculations in ATLAS's offline software environment using the ATLAS offline/analysis (xAOD) Event Data Model. We compare the performance and flexibility of a couple of the available GPU programming methods, and show how different memory management setups affect our ability to offload different types of calculations to a GPU efficiently.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

GPU-friendly surface model for Monte-Carlo detector simulations

The demands for Monte-Carlo simulation are drastically increasing with the Large Hadron Collider’s high-luminosity upgrade, and are expected to exceed the currently available compute resources. At the same time, modern high-performance computing has adopted powerful hardware accelerators, particularly GPUs. The AdePT and Celeritas projects aim to address the demanding computational needs by leveraging these heterogeneous computing architectures. While both have successfully ported realistic detector simulations to GPUs using the VecGeom library, the complexity of geometry modeling emerged as a bottleneck. Thread divergence and high register usage were degrading the GPU performance. Therefore, a new, GPU-friendly surface-based model has been introduced in the VecGeom library that decomposes the divergent code of the 3D primitive solids into simpler and more balanced surface algorithms. In this work, we present the latest developments, focusing on the additions required to efficiently model complex setups like the CMS Phase-2 geometry. This includes memory reduction techniques, and adding accelerating structures for faster traversal.

Diederichs, Severin [CERN]↗

Workload Characterization of a Leadership Class Storage Cluster

Understanding workload characteristics is critical for optimizing and improving the performance of current systems and software, and architecting new storage systems based on observed workload patterns. In this paper, we characterize the scientific workloads of the world s fastest HPC (High Performance Computing) storage cluster, Spider, at the Oak Ridge Leadership Computing Facility (OLCF). Spider provides an aggregate bandwidth of over 240 GB/s with over 10 petabytes of RAID 6 formatted capacity. OLCFs flagship petascale simulation platform, Jaguar, and other large HPC clusters, in total over 250 thousands compute cores, depend on Spider for their I/O needs. We characterize the system utilization, the demands of reads and writes, idle time, and the distribution of read requests to write requests for the storage system observed over a period of 6 months. From this study we develop synthesized workloads and we show that the read and write I/O bandwidth usage as well as the inter-arrival time of requests can be modeled as a Pareto distribution.

97 MATHEMATICS AND COMPUTING↗

A multiphysics coupling framework for exascale simulation of fracture evolution in subsurface energy applications

Predicting the evolution of fractured media is challenging due to coupled thermal, hydrological, chemical and mechanical processes that occur over a broad range of spatial scales, from the microscopic pore scale to field scale. We present a software framework and scientific workflow that couples the pore scale flow and reactive transport simulator Chombo-Crunch with the field scale geomechanics solver in GEOS to simulate fracture evolution in subsurface fluid-rock systems. This new multiphysics coupling capability comprises several novel features. An HDF5 data schema for coupling fracture positions between the two codes is employed and leverages the coarse resolution of the GEOS mechanics solver which limits the size of data coupled, and is, thus, not taxed by data resulting from the high resolution pore scale Chombo-Crunch solver. The coupling framework requires tracking of both before and after coarse nodal positions in GEOS as well as the resolved embedded boundary in Chombo-Crunch. We accomplished this by developing an approach to geometry generation that tracks the fracture interface between the two different methodologies. The GEOS quadrilateral mesh is converted to triangles which are organized into bins and an accessible tree structure; the nodes are then mapped to the Chombo representation using a continuous signed distance function that determines locations inside, on and outside of the fracture boundary. The GEOS positions are retained in memory on the Chombo-Crunch side of the coupling. The time stepping cadence for coupled multiphysics processes of flow, transport, reactions and mechanics is stable and demonstrates temporal reach to experimental time scales. The approach is validated by demonstration of 9 days of simulated time of a core flood experiment with fracture aperture evolution due to invasion of carbonated brine in wellbore-cement and sandstone. We also demonstrate usage of exascale computing resources by simulating a high resolution version of the validation problem on OLCF Frontier.

97 MATHEMATICS AND COMPUTING↗

Towards reverse mode automatic differentiation of Kokkos-based codes

Derivative computation is a key component of optimization, sensitivity analysis, uncertainty quantification, and the solving of nonlinear problems. Automatic differentiation (AD) is a powerful technique for evaluating such derivatives, and in recent years, has been integrated into programming environments such as Jax, PyTorch, and TensorFlow to support derivative computations needed for training of machine learning models, facilitating wide-spread use of these technologies. The C++ language has become the de facto standard for scientific computing due to numerous factors, yet language complexity has made the wide-spread adoption of AD technologies for C++ difficult, hampering the incorporation of powerful differentiable programming approaches into C++ scientific simulations. This is exacerbated by the increasing emergence of architectures, such as GPUs, with limited memory capabilities and requiring massive thread-level concurrency. C++ AD tools must effectively use these environments to bring novel scientific simulations to next-generation DOE experimental and observational facilities. In this project, we investigated source transformation-based automatic differentiation using LLVM compiler infrastructure to automatically generate portable and efficient gradient computations of Kokkos-based code. We have demonstrated that our proposed strategy is feasible by investigating the usage of a prototype LLVM-based source transformation tool to generate gradients of simple functions made of sequences of simple Kokkos parallel regions. Speedups of up to 500x compared to Sacado were observed on NVIDIA V100 GPU.

97 MATHEMATICS AND COMPUTING↗

Intelligent Experiments through Real-Time AI: Fast Data Processing and Autonomous Detector Control for High-Energy Nuclear Experiments

The aim of this project is to develop software and hardware for fast real-time data processing and autonomous detector control and calibration for the sPHENIX and the future EIC experiments. Below summarizes Georgia Tech team efforts in the past year: 1. We developed a real-time clustering algorithm and FPGA-based pipeline architecture for processing fired pixel data from ALPIDE sensors in sPHENIX experiments. Our Columnar Clustering Co-Design introduces a hardware-aware, stream-friendly approach that segments pixel data by column pairs using a Column Pair Clustering (CPC) strategy, followed by Cluster Stitching to merge adjacent subclusters. Implemented in Vitis HLS, the pipeline comprises five stages—read-in, subclustering, stitching, analysis, and write-out—connected by tagged HLS streams with custom end-of-event signaling for robust synchronization. We designed a pipelined dataflow model optimized for throughput, low latency, and minimal buffering, enabling scalable clustering across events of arbitrary size. Our system maintains spatial precision via center-of-mass and shape key extraction and efficiently handles edge cases such as fragmented or nested clusters. Compared against DBSCAN in both software and hardware, our approach demonstrates competitive performance under FPGA constraints. 2. We also conducted a comprehensive algorithm-to-hardware co-design of connected component analysis tailored for sPHENIX experiments, focusing on real-time, low-latency processing using FPGAs and High-Level Synthesis (HLS). Starting from a Python-based particle tracking pipeline, the team translated the core logic—graph traversal via DFS and Union-Find—into an HLS-compatible C++ model, replacing dynamic memory and recursion with static arrays and pipelined control flow. The final design includes a fully streamed and dataflow-compatible Union-Find kernel optimized across five iterations, incorporating loop pipelining, array partitioning, AXI/FIFO interface tuning, and function flattening. Experimental results show up to 14.8× speedup over the CPU baseline, reducing per-graph latency to 1.58 μs and demonstrating strong resource efficiency with only ~7k LUTs and zero BRAM usage. The design maintains functional correctness against the Python reference using a Python-based C-simulation framework and Mean Squared Error metrics. This work validates the potential of HLS-driven FPGA designs for edge-level HEP data acquisition, laying a scalable foundation for future integration with real-time detector pipelines and multi-graph processing systems.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

On-the-fly Doppler Broadening with Probability Table Interpolation for Unresolved Resonance Region in RMC

With the rapid development of computational power and parallel algorithms, Monte Carlo method has been widely investigated and used in neutron transport simulations due to its advantages in geometry modelling and usage of continuous energy point-wise cross sections. Nowadays the road has been paved to couple Monte Carlo codes with thermal-hydraulics codes to make multi-physics calculations. One of the most important aspects of thermal-hydraulics feedback is the detailed temperature distribution, which demands temperature dependent cross sections in neutron transport simulations. The conventional pre-generated cross section libraries at each temperature point are no longer feasible for detailed temperature distributions because of tremendous memory footprint. Hence, different on-the-fly Doppler broadening methods have been proposed for resolved resonance energy region and thermal energy region, such as the interpolation, regress model fitting, multipole representation (MPR), target motion sampling (TMS) and different integration algorithms based on Doppler broadening equation. Compared to resolved resonance energy region and thermal energy region, the representation of cross sections in unresolved resonance region is slightly different. As a result of the limitation of experiment measurements, the overlapped resonances cannot be distinguished from each other and the statistic average resonance parameters are given in the evaluated nuclear data file. In early applications, infinite-dilute pointwise cross sections were obtained from average parameters and were directly used in neutron simulations. However, it was proven that these infinite-dilute pointwise cross sections cannot capture the energy self-shielding effect, thus leading to an overestimate of reaction rates and usually an underestimate of k{sub eff} owing to the increased absorption rates and increased scattering probability to energy below unresolved resonance region where the capture-to-fission ratio is typically higher. This phenomenon is of great importance especially for problems where there is a considerable flux in unresolved resonance region. Then the probability table method was put forward by Levitt to take energy self-shielding effect into account, which provides probability distribution function of cross sections in unresolved resonance region instead of specific cross section values. The probability distribution function is presented in the form of tables and the mean values are equal to the infinite dilution cross sections. To obtain the temperature dependent cross sections in unresolved energy region, there are two different methods. One is to calculate cross sections directly from average resonance parameters and their statistical distributions , which is effective and precise but time-consuming. The other is to interpolate between pre-generated probability table files, which is much more efficient. In this paper, on-the-fly Doppler broadening of cross sections in unresolved resonance regions based on probability tables interpolation was implemented in RMC codes as a supplement to on-the-fly Doppler broadening for resolved resonance energy region and thermal region, which is based on the integration of Doppler broadening equation. The results of criticality calculation and depletion calculation show that on-the-fly Doppler broadening for the whole energy region is accurate and efficient. (authors)

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

CrossSim

CrossSim is a simulator for modeling neural-inspired machine learning algorithms on analog hardware, such as resistive memory crossbars. It includes noise models for reading and updating the resistances, which can be based on idealized equations or experimental data. It can also introduce noise and finite precision effects when converting values from digital to analog and vice versa. All of these effects can be turned on or off as an algorithm processes a data set and attempts to learn its salient attributes so that it can be categorized in the machine learning training/classification context. CrossSim thus allows the robustness, accuracy, and energy usage of a machine learning algorithm to be tested on simulated hardware.

Plimpton, StevenJ.↗

TU-EF-304-07: Monte Carlo-Based Inverse Treatment Plan Optimization for Intensity Modulated Proton Therapy

Purpose: Intensity-modulated proton therapy (IMPT) is increasingly used in proton therapy. For IMPT optimization, Monte Carlo (MC) is desired for spots dose calculations because of its high accuracy, especially in cases with a high level of heterogeneity. It is also preferred in biological optimization problems due to the capability of computing quantities related to biological effects. However, MC simulation is typically too slow to be used for this purpose. Although GPU-based MC engines have become available, the achieved efficiency is still not ideal. The purpose of this work is to develop a new optimization scheme to include GPU-based MC into IMPT. Methods: A conventional approach using MC in IMPT simply calls the MC dose engine repeatedly for each spot dose calculations. However, this is not the optimal approach, because of the unnecessary computations on some spots that turned out to have very small weights after solving the optimization problem. GPU-memory writing conflict occurring at a small beam size also reduces computational efficiency. To solve these problems, we developed a new framework that iteratively performs MC dose calculations and plan optimizations. At each dose calculation step, the particles were sampled from different spots altogether with Metropolis algorithm, such that the particle number is proportional to the latest optimized spot intensity. Simultaneously transporting particles from multiple spots also mitigated the memory writing conflict problem. Results: We have validated the proposed MC-based optimization schemes in one prostate case. The total computation time of our method was ∼5–6 min on one NVIDIA GPU card, including both spot dose calculation and plan optimization, whereas a conventional method naively using the same GPU-based MC engine were ∼3 times slower. Conclusion: A fast GPU-based MC dose calculation method along with a novel optimization workflow is developed. The high efficiency makes it attractive for clinical usages.

60 APPLIED LIFE SCIENCES↗

A Tale of Two Systems: Flexibility of Usage of Kraken and Nautilus at the National Institute for Computational Sciences

The National Institute for Computational Sciences (NICS) at the University of Tennessee currently operates two computational resources for the eXtreme Science and Engineering Discovery Environment (XSEDE), Kraken, a 112,896-core Cray XT5 for general purpose computation, and Nautilus, a 1,024-core SGI Altix UV 1000 for data analysis and visualization. We analyze a year's worth of accounting logs for Kraken and Nautilus to understand how users take advantage of these two systems and how analysis jobs differ from general HPC computation We find that researchers take advantage of the flexibility offered by these systems, running a wide variety of jobs at many scales and using the full range of core counts and available memory for their jobs. The jobs on Nautilus tend to use less walltime and more memory per core than the jobs run on Kraken. Additionally, researchers are more likely to run interactive jobs on Nautilus than on Kraken. Small jobs experience a good quality of service on both systems. This information can be used for the management and allocation of time on existing HPC and analysis systems as well as for planning for deploying future HPC and analysis systems.

Szczepanski, Amy F↗

A Job Sizing Strategy for High-Throughput Scientific Workflows

The user of a computing facility must make a critical decision when submitting jobs for execution: how many resources (such as cores, memory, and disk) should be requested for each job? If the request is too small, the job may fail due to resource exhaustion; if the request is too big, the job may succeed, but resources will be wasted. This decision is especially important when running hundreds of thousands of jobs in a high throughput workflow, which may exhibit complex, long tailed distributions of resource consumption. In this paper, we present a strategy for solving the job sizing problem: (1) applications are monitored and measured in user-space as they run; (2) the resource usage is collected into an online archive; and (3) jobs are automatically sized according to historical data in order to maximize throughput or minimize waste. We evaluate the solution analytically, and present case studies of applying the technique to high throughput physics and bioinformatics workflows consisting of hundreds of thousands of jobs, demonstrating an increase in throughput of 10-400 percent compared to naive approaches.

Tovar, Benjamin (ORCID:0000000252942281)↗

Performance analysis of fully explicit and fully implicit solvers within a spectral element shallow-water atmosphere model

In this study, explicit Runge–Kutta methods and implicit multistep methods utilizing a Newton–Krylov nonlinear solver are evaluated for a range of configurations of the shallow-water dynamical core of the spectral element community atmosphere model to evaluate their computational performance. These configurations are designed to explore the attributes of each method under different but relevant model usage scenarios including varied spectral order within an element, static regional refinement, and scaling to the largest problem sizes. This analysis is performed within the shallow-water dynamical core option of a full climate model code base to enable a wealth of simulations for study, with the aim of informing solver development within the more complete hydrostatic dynamical core used for climate research. The limitations and benefits to using explicit versus implicit methods, with different parameters and settings, are discussed in light of the trade-offs with Message Passing Interface (MPI) communication and memory and their inherent efficiency bottlenecks. Given the performance behavior across the configurations analyzed here, the recommendation for future work using the implicit solvers is conditional based on scale separation and the stiffness of the problem. For the regionally refined configurations, the implicit method has about the same efficiency as the explicit method, without considering efficiency gains from a preconditioner. The potential for improvement using a preconditioner is greatest for higher spectral order configurations, where more work is shifted to the linear solver. Finally, initial simulations with OpenACC directives to utilize a Graphics Processing Unit (GPU) when performing function evaluations show improvements locally, and that overall gains are possible with adjustments to data exchanges.

58 GEOSCIENCES↗

Economizer Based Data Center Liquid Cooling with Advanced Metal Interfaces. Final Technical Report

A new chiller-less data center liquid cooling system utilizing the outside air environment has been shown to achieve up to 90% reduction in cooling energy compared to traditional chiller based data center cooling systems. The system removes heat from Volume servers inside a Sealed Rack and transports the heat using a liquid loop to an Outdoor Heat Exchanger which rejects the heat to the outdoor ambient environment. The servers in the rack are cooled using a hybrid cooling system by removing the majority of the heat generated by the processors and memory by direct thermal conduction using coldplates and the heat generated by the remaining components using forced air convection to an air- to- liquid heat exchanger inside the Sealed Rack. The anticipated benefits of such energy-centric configurations are significant energy savings at the data center level. When compared to a traditional 10 MW data center, which typically uses 25% of its total data center energy consumption for cooling this technology could potentially enable a cost savings of up to $800,000-$2,200,000/year (assuming electricity costs of 4 to 11 cents per kilowatt-hour) through the reduction in electrical energy usage.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Enhancements supporting IC usage of PEM libraries on next-gen platforms

This milestone reports on the culmination of several years of effort by multiple PEM support software development teams to provide capabilities for use in LLNL-developed integrated codes on next-gen ASC platforms, including GPU support. We will provide a survey of relevant Application Program Interfaces (API) that are required to support LLNL IC code capability on relevant architectures, with a focus on Sierra and El Capitan. We will identify and summarize all dependencies between PEM supported libraries and IC supported physics codes. We will provide an assessment of algorithmic improvements that have been deployed, as well as future developments that are required to complete the GPU porting efforts. This assessment will include a description of programming models adopted by each of the PEM projects, distinct algorithmic challenges for each of the capabilities, and information about sharing GPU memory between the APIs and host codes. We will develop targeted test problems to assess computational performance. Finally, this milestone will result in identification of gaps in our effort to assist the LLNL ASC program in prioritization of effort for porting software to El Capitan.

97 MATHEMATICS AND COMPUTING↗

A dynamic solvent chamber propagation estimation framework using RNN for warm solvent injection in heterogeneous reservoirs

Warm solvent injection (WSI), injecting low-temperature solvent into formations to reduce the viscosity of heavy oil, is a clean technology for heavy oil production through reducing greenhouse gas emissions and water usage. The success of WSI operation depends on the uniform development and propagation of solvent chambers in reservoirs. However, reservoir heterogeneity stemming from shale barriers plays a detrimental role in the conformance of solvent chamber development and oil production rate. In this work, we developed a novel recurrent neural network (RNN)-based framework with the capability of efficiently tracking and estimating the solvent chamber positions in heterogeneous reservoirs based on only production time-series data. The developed estimation model utilizes the “sequence-to-sequence" mapping methodology to correlate observed production time-series sequence and solvent chamber edge sequence via a long short-term memory (LSTM) algorithm. The trained RNN models exhibit high accuracy, evidenced by the predicted dynamic solvent chamber locations match the corresponding true locations from numerical simulation, with a high coefficient of determination (R 2 ) and a low mean squared error. Specifically, the achieved R 2 values exceed 0.98 on both the training and testing data. The developed RNN-based workflow was tested via several cases from both regularly- and irregularly-shaped shale barriers, and the results were promising. The predicted solvent chambers showed strong agreement with those obtained from numerical simulations. The major benefits of this workflow include reducing computational time and saving overall monitoring and tracking costs for conventional techniques. In conclusion, the present work would provide a good demonstration of the capability of practical integration of machine learning methods in solving engineering problems.

58 GEOSCIENCES↗