Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Heterogeneous computing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 307 records · Page 17
All sourcesNASA NTRSDOE OSTI
DOE OSTI2019

Performance portability study for massively parallel computational fluid dynamics application on scalable heterogeneous architectures

Patient-specific hemodynamic simulations have the possibility to greatly improve both the diagnosis and treatment of a variety of vascular diseases. Portability will enable wider adoption of computational fluid dynamics (CFD) applications in the biomedical research community and targeting to platforms ideally suited to different vascular regions. Here, we introduce a case study in performance portability that assesses (1) the ease of porting an MPI application optimized for one specific architecture to new platforms using variants of hybrid MPI+X programming models; (2) performance portability seen when simulating blood flow in three different vascular regions on diverse heterogeneous architectures; (3) model-based performance prediction for future architectures; and (4) performance scaling of the hybrid MPI+X programming on parallel heterogeneous systems. We report on the lessons learned in porting HARVEY, a massively parallel CFD application, from traditional multicore CPUs to diverse heterogeneous architectures ranging from NVIDIA/AMD GPUs to Intel MICs and Altera FPGAs.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

ExaWorks software development kit: a robust and scalable collection of interoperable workflows technologies

Scientific discovery increasingly requires executing heterogeneous scientific workflows on high-performance computing (HPC) platforms. Heterogeneous workflows contain different types of tasks (e.g., simulation, analysis, and learning) that need to be mapped, scheduled, and launched on different computing. That requires a software stack that enables users to code their workflows and automate resource management and workflow execution. Currently, there are many workflow technologies with diverse levels of robustness and capabilities, and users face difficult choices of software that can effectively and efficiently support their use cases on HPC machines, especially when considering the latest exascale platforms. We contributed to addressing this issue by developing the ExaWorks Software Development Kit (SDK). The SDK is a curated collection of workflow technologies engineered following current best practices and specifically designed to work on HPC platforms. We present our experience with (1) curating those technologies, (2) integrating them to provide users with new capabilities, (3) developing a continuous integration platform to test the SDK on DOE HPC platforms, (4) designing a dashboard to publish the results of those tests, and (5) devising an innovative documentation platform to help users to use those technologies. Our experience details the requirements and the best practices needed to curate workflow technologies, and it also serves as a blueprint for the capabilities and services that DOE will have to offer to support a variety of scientific heterogeneous workflows on the newly available exascale HPC platforms.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

CG-Kit: Code Generation Toolkit for performant and maintainable variants of source code applied to Flash-X hydrodynamics simulations

CG-Kit is a new Code Generation tool-Kit that we have developed as a part of the solution for portability and maintainability for multiphysics computing applications. The development of CG-Kit is rooted in the urgent need created by the shifting landscape of high-performance computing platforms and the algorithmic complexities of a particular large-scale multiphysics application: Flash-X. To efficiently use computing resources on a heterogeneous node, an application must have a map of computation to resources and a mechanism to move the data and computation to the resources according to the map. Most existing performance portability solutions are focussed on abstracting the expression of computations so that a unified source code can be specialized to run on different resources. However, such an approach is insufficient for a code like Flash-X, which has a multitude of code components that can be assembled in various permutations and combinations to form different instances of applications. Similar challenges apply to any code that has composability, where a single specified way of apportioning work among devices may not be optimal. Additionally, use cases arise where the optimal control flow of computation may differ for different devices while the underlying numerics remain identical. This combination leads to unique challenges including handling an existing large code base in Fortran and/or C/C++, subdivision of code into a great variety of units supporting a wide range of physics and numerical methods, different parallelization techniques for distributed and shared memory systems and accelerator devices, and heterogeneity of computing platforms requiring coexisting variants of parallel algorithms. All of these challenges demand that scientific software developers apply existing knowledge about domain applications, algorithms, and computing platforms to determine custom abstractions and granularity for code generation. There is a critical lack of tools to tackle those problems. CG-Kit is designed to fill this gap by providing a user with the ability to express their desired control flow and computation-to-resource map in the form a pseudocode-like recipe. It consists of standalone tools that can be combined into highly specific and, we argue, highly effective portability and maintainability toolchains. Here we present the design of our new tools: parametrized source trees, control flow graphs, and recipes. The tools are implemented in Python. They are agnostic to the programming language of the source code targeted for code generation. In conclusion, we demonstrate the capabilities of the toolkit with two examples, first, multithreaded variants of the basic AXPY operation, and second, variants of parallel algorithms within a hydrodynamics solver, called Spark, from Flash-X that operates on block-structured adaptive meshes.

Algorithmic portability↗
DOE OSTI2018

Cyber-Infrastructure for Data-Intensive Geospatial Computing

With the recent advent of heterogeneous High-performance Computing (HPC) to handle EO “Big Data” workloads, there is a need for a unified Cyber-infrastructure (CI) platform that can bridge the best of many HPC worlds. In this chapter, we discuss such a CI platform being developed at Geographic Information Science and Technology (GIST) group using novel and innovative techniques, and emerging technologies that are scalable to large-scale supercomputers. The CI platform utilizes a wide variety of computing such as GPGPU, distributed, real-time and cluster computing, which are being brought together architecturally to enable data-driven analysis, scientific understanding of earth system models, and research collaboration. This development addresses the need for close integration of EO and other geospatial information in the face of growing volumes of the data, and facilitates spatio-temporal analysis of disparate and dynamic data streams. Horizontal scalability and linear throughput are supported in the heart of the platform itself. It is being used to support very broad application areas, ranging from high-resolution settlement mapping, national bioenergy infrastructure to urban information and mobility systems. The platform provides spatio-temporal decision support capabilities in planning, policy and operational missions for US federal agencies. Also, the platform is designed to be functionally and technologically sustainable for continued support of the US energy and environment mission for the coming decades.

Karthik, Rajasekar↗
DOE OSTI2021

Computational Astrophysics in the Era of Technological Heterogeneity [Slides]

The HPC landscape is changing and we’re headed toward an era where compute specialization will be prevalent. There are opportunities for co-design that can influence this future. Technological heterogeneity will be a major challenge unless we shift our approach to developing the computational tools for astrophysics. Parthenon provides convenient functionality and a bright future for block structured AMR applications. Phoebus is a new (soon-to-be) open source code for relativistic astro that promises excellent performance, portability, and unique physics capabilities.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2025

IRIS-MEMFLOW: Data Flow-Enabled Portable Memory Orchestration in IRIS Runtime for Diverse Heterogeneity

Task-based programming models and execution paradigms provide a means to decompose a computation by expressing it as a graph in which each node represents a specific computation operating on memory objects and the edges define the dependencies in the execution flow. In this execution model, independent nodes in the graph can be executed concurrently in different computing devices, making it suitable for heterogeneous systems in which computing devices with different architectures coexist. However, careful memory orchestration across heterogeneous devices is needed because copies of the same memory object may reside in multiple devices during execution. Manually ensuring such an orchestration is quite challenging. Not only must an application developer guard against race conditions, but they must also optimize data movement between the host and devices because unnecessary data movement significantly impacts performance. To mitigate these challenges, we enhance the IRIS heterogeneous runtime and introduce IRIS-MEMFLOW–a data flow–enabled portable memory abstraction for seamlessly orchestrating memory in diverse heterogeneous computing environments. By using data-flow analysis, IRIS-MEMFLOW guards against race conditions while multiple heterogeneous devices access memory objects. IRIS-MEMFLOW also optimizes data movement between the host and devices without manual intervention. As a result, IRIS provides improved programming productivity, performance, and portability for multidevice heterogeneous executions in high-performance computing and cloud systems that run diverse architectures from different vendors. The efficacy of IRIS-MEMFLOW is evaluated through experiments that show its capability in terms of programming productivity, multidevice heterogeneity, portability, and low overhead versus the state of the art.

Monil, M. A. H. [ORNL] (ORCID:0000000334194037)↗
DOE OSTI2011

Dosimetric impact of image artifact from a wide-bore CT scanner in radiotherapy treatment planning

Purpose: Traditional computed tomography (CT) units provide a maximum scan field-of-view (sFOV) diameter of 50 cm and a limited bore size, which cannot accommodate a large patient habitus or an extended simulation setup in radiation therapy (RT). Wide-bore CT scanners with increased bore size were developed to address these needs. Some scanners have the capacity to reconstruct the CT images at an extended FOV (eFOV), through data interpolation or extrapolation, using projection data acquired with a conventional sFOV. Objects that extend past the sFOV for eFOV reconstruction may generate image artifacts resulting from truncated projection data; this may distort CT numbers and structure contours in the region beyond the sFOV. The purpose of this study was to evaluate the dosimetric impact of image artifacts from eFOV reconstruction with a wide-bore CT scanner in radiotherapy (RT) treatment planning. Methods: Testing phantoms (i.e., a mini CT phantom with equivalent tissue inserts, a set of CT normal phantoms and anthropomorphic phantoms of the thorax and the pelvis) were used to evaluate eFOV artifacts. Reference baseline images of these phantoms were acquired with the phantom centrally positioned within the sFOV. For comparison, the phantoms were then shifted laterally and scanned partially outside the sFOV, but still within the eFOV. Treatment plans were generated for the thoracic and pelvic anthropomorphic phantoms utilizing the Eclipse treatment planning system (TPS) to study the potential effects of eFOV artifacts on dose calculations. All dose calculations of baseline and test treatment plans were carried out using the same MU. Results: Results show that both body contour and CT numbers are altered by image artifacts in eFOV reconstruction. CT number distortions of up to -356 HU for bone tissue and up to 323 HU for lung tissue were observed in the mini CT phantom. Results from the large body normal phantom, which is close to a clinical patient size, show average CT number changes of up to -49 HU. Wider distribution (i.e., standard deviation) of the HU values was seen when the phantom was placed at more than 2.8 cm beyond the 50 cm sFOV. Anthropomorphic phantom studies with several standard beam configurations show that body contour distortion causes tumor dose calculation reduction of 3.0 and 1.9% for 6 and 23 MV x-rays, respectively, when not accounting for tissue heterogeneities during dose computation. When heterogeneity correction is used in planning, the competing effects of the body contour distortion and the CT number distortion cause a smaller error in tumor dose calculation. Less than 0.9% error in calculated dose was observed in volumetric modulated arc therapy (VMAT) treatment plans. Conclusions: The image artifacts from eFOV reconstruction alter the CT numbers and body contours of the imaged objects, which has the potential to produce inaccuracies in dose calculations during radiotherapy treatment planning. The radiation therapy team should be aware of these image artifacts and their effects on target dose calculations during CT simulation as well as treatment planning.

61 RADIATION PROTECTION AND DOSIMETRY↗
DOE OSTI2016

Panda: A Compiler Framework for Concurrent CPU $+$ GPU Execution of 3D Stencil Computations on GPU-accelerated Supercomputers

We present a new compiler framework for truly heterogeneous 3D stencil computation on GPU clusters. Our framework consists of a simple directive-based programming model and a tightly integrated source-to-source compiler. Annotated with a small number of directives, sequential stencil C codes can be automatically parallelized for large-scale GPU clusters. The most distinctive feature of the compiler is its capability to generate hybrid MPI$+$ CUDA$+$ OpenMP code that uses concurrent CPU$+$ GPU computing to unleash the full potential of powerful GPU clusters. The auto-generated hybrid codes hide the overhead of various data motion by overlapping them with computation. Test results on the Titan supercomputer and the Wilkes cluster show that auto-translated codes can achieve about 90 % of the performance of highly optimized handwritten codes, for both a simple stencil benchmark and a real-world application in cardiac modeling. The user-friendliness and performance of our domain-specific compiler framework allow harnessing the full power of GPU-accelerated supercomputing without painstaking coding effort.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Computational Investigation of the Role of Active Site Heterogeneity for aSupported Organovanadium(III) Hydrogenation Catalyst

A crucial consideration for supported heterogeneous catalysts is the nonuniformity of the active sites, particularly for supported organometallic catalysts. Standard spectroscopic techniques, such as X-ray absorption spectroscopy, reflect the nature of the most populated sites, which are often intrinsically structurally distinct from the most active catalytic sites. Additionally, with computational models, often, only a few representative structures are used to depict catalytic active sites on a surface, even though there are numerous observable factors of surface heterogeneity that contribute to the kinetically favorable active species. A previously reported study on the mechanism of a surface organovanadium-(III) catalyst [(SiO 2 )V III (Mes)(THF)] for styrene hydrogenation yielded two possible mechanisms: heterolytic cleavage and redox cycling. These two mechanistic scenarios are challenging to differentiate experimentally since the kinetic readouts of the catalyst are identical. To showcase the importance of modeling surface heterogeneity and its effect on catalytic activity, density functional theory (DFT) computational models of a series of potential active sites of [(SiO 2 )V III (Mes)(THF)] for the reaction pathways are applied in combination with kinetic Monte Carlo (kMC) simulations. Computed results were then compared to the previously reported experimental kinetic study: (1) DFT free-energy reaction pathways indicated the likely active site and pathway for styrene hydrogenation, a heterolytic cleavage pathway requiring a bare tripodal vanadium site. (2) From the kMC simulations, a mixture of different bond lengths from the support oxygen to the metal center was required to qualitatively describe the experimentally observed kinetic aspects of a supported organovanadium(III) catalyst for olefin hydrogenation. This work underscores the importance of modeling surface heterogeneity in computational catalysis.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2013

Accelerating Subsurface Transport Simulation on Heterogeneous Clusters

Reactive transport numerical models simulate chemical and microbiological reactions that occur along a flowpath. These models have to compute reactions for a large number of locations. They solve the set of ordinary differential equations (ODEs) that describes the reaction for each location through the Newton-Raphson technique. This technique involves computing a Jacobian matrix and a residual vector for each set of equation, and then solving iteratively the linearized system by performing Gaussian Elimination and LU decomposition until convergence. STOMP, a well known subsurface flow simulation tool, employs matrices with sizes in the order of 100x100 elements and, for numerical accuracy, LU factorization with full pivoting instead of the faster partial pivoting. Modern high performance computing systems are heterogeneous machines whose nodes integrate both CPUs and GPUs, exposing unprecedented amounts of parallelism. To exploit all their computational power, applications must use both the types of processing elements. For the case of subsurface flow simulation, this mainly requires implementing efficient batched LU-based solvers and identifying efficient solutions for enabling load balancing among the different processors of the system. In this paper we discuss two approaches that allows scaling STOMP's performance on heterogeneous clusters. We initially identify the challenges in implementing batched LU-based solvers for small matrices on GPUs, and propose an implementation that fulfills STOMP's requirements. We compare this implementation to other existing solutions. Then, we combine the batched GPU solver with an OpenMP-based CPU solver, and present an adaptive load balancer that dynamically distributes the linear systems to solve between the two components inside a node. We show how these approaches, integrated into the full application, provide speed ups from 6 to 7 times on large problems, executed on up to 16 nodes of a cluster with two AMD Opteron 6272 and a Tesla M2090 per node.

GPU↗
DOE OSTI2012

Long-term Differences in Tillage and Land Use Affect Intra-aggregate Pore Heterogeneity

Recent advances in computed tomography provide measurement tools to study internal structures of soil aggregates at micrometer resolutions and to improve our understanding of specific mechanisms of various soil processes. Fractal analysis is one of the data analysis tools that can be helpful in evaluating heterogeneity of the intra-aggregate internal structures. The goal of this study was to examine how long-term tillage and land use differences affect intra-aggregate pore heterogeneity. The specific objectives were: (i) to develop an approach to enhance utility of box-counting fractal dimension in characterizing intra-aggregate pore heterogeneity; (ii) to examine intra-aggregate pores in macro-aggregates (4-6 mm in size) using the computed tomography scanning and fractal analysis, and (iii) to compare heterogeneity of intra-aggregate pore space in aggregates from loamy Alfisol soil subjected to 20 yr of contrasting management practices, namely, conventional tillage (chisel plow) (CT), no-till (NT), and native succession vegetation (NS). Three-dimensional images of the intact aggregates were obtained with a resolution of 14.6 {micro}m at the Advanced Photon Source, Argonne National Laboratory, Argonne, IL. Proposed box-counting fractal dimension normalization was successfully implemented to estimate heterogeneity of pore voxel distributions without bias associated with different porosities in soil aggregates. The aggregates from all three studied treatments had higher porosity associated with large (>100 {micro}m) pores present in their centers than in their exteriors. Pores 15 to 60 {micro}m were equally abundant throughout entire aggregates but their distributions were more heterogeneous in aggregate interiors. The CT aggregates had greater numbers of pores 15 to 60 {micro}m than NT and NS. Distribution of pore voxels belonging to large pores was most heterogeneous in the aggregates from NS, followed by NT and by CT. This result was consistent with presence of very large pores associated with former root channels in NT and NS aggregates. Our findings indicate that mechanisms of macro-aggregate formation might differ in their importance in different land use and management practices.

43 PARTICLE ACCELERATORS↗
DOE OSTI2016

Computationally Probing the Performance of Hybrid, Heterogeneous, and Homogeneous Iridium-Based Catalysts for Water Oxidation

An attractive strategy to improve the performance of water oxidation catalysts would be to anchor a homogeneous molecular catalyst on a heterogeneous solid surface to create a hybrid catalyst. The idea of this combined system is to take advantage of the individual properties of each of the two catalyst components. We use Density Functional Theory to determine the stability and activity of a model hybrid water oxidation catalyst consisting of a dimeric Ir complex attached on the IrO 2 (110) surface through two oxygen atoms. We find that homogeneous catalysts can be bound to its matrix oxide without losing significant activity. Hence, designing hybrid systems that benefit from both the high tunability of activity of homogeneous catalysts and the stability of heterogeneous systems seems feasible.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2022

GPU-accelerated DNS of compressible turbulent flows

Here, this paper explores strategies to transform an existing CPU-based high-performance computational fluid dynamics solver, HyPar, for compressible flow simulations on emerging exascale heterogeneous (CPU+GPU) computing platforms. The scientific motivation for developing a GPU-enhanced version of HyPar is to simulate canonical turbulent flows at the highest resolution possible on such platforms. We show that optimizing memory operations and thread blocks results in 200x speedup of computationally intensive kernels compared with a CPU core. Using multiple GPUs and CUDA-aware MPI communication, we demonstrate both strong and weak scaling of our GPU-based HyPar implementation on the NVIDIA Volta V100 GPUs. We simulate the decay of homogeneous isotropic turbulence in a triply periodic box on grids with up to 1024 3 points (5.3 billion degrees of freedom) and on up to 1,024 GPUs. We compare the wall times for CPU-only and CPU+GPU simulations. The results presented in the paper are obtained on the Summit and Lassen supercomputers at Oak Ridge and Lawrence Livermore National Laboratories, respectively.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Secure Federated Learning Across Heterogeneous Cloud and High-Performance Computing Resources: A Case Study on Federated Fine-Tuning of LLaMA 2

Federated learning enables multiple data owners to collaboratively train robust machine learning models without transferring large or sensitive local datasets by only sharing the parameters of the locally trained models. Here, in this article, we elaborate on the design of our Advanced Privacy-Preserving Federated Learning (APPFL) framework, which streamlines end-to-end secure and reliable federated learning experiments across cloud computing facilities and high-performance computing resources by leveraging Globus Compute, a distributed function as a service platform, and Amazon Web Services. We further demonstrate the use case of APPFL in fine-tuning an LLaMA 2 7B model using several cloud resources and supercomputers.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2017

rvGAHP: push-based job submission using reverse SSH connections. In: WORKS '17 Proceedings of the 12th Workshop on Workflows in Support of Large-Scale Science, Article No. 3

Computational science researchers running large-scale scientific workflow applications often want to run their workflows on the largest available compute systems to improve time to solution. Workflow tools used in distributed, heterogeneous, high performance computing environments typically rely on either a push-based or a pull-based approach for resource provisioning from these compute systems. However, many large clusters have moved to two-factor authentication for job submission, making traditional automated push-based job submission impossible. On the other hand, pull-based approaches such as pilot jobs may lead to increased complexity and a reduction in node-hour efficiency. In this paper, we describe a new, efficient approach based on HTCondor-G called reverse GAHP (rvGAHP) that allows us to push jobs using reverse SSH submissions with better efficiency than pull-based methods. We successfully used this approach to perform a large probabilistic seismic hazard analysis study using SCEC's CyberShake workflow in March 2017 on the Titan Cray XK7 hybrid system at Oak Ridge National Laboratory.

Callaghan, Scott↗
DOE OSTI2021

Using computational singular perturbation as a diagnostic tool in ODE and DAE systems: a case study in heterogeneous catalysis

We have extended the computational singular perturbation (CSP) method to differential algebraic equation (DAE) systems and demonstrated its application in a heterogeneous-catalysis problem. The extended method obtains the CSP basis vectors for DAEs from a reduced Jacobian matrix that takes the algebraic constraints into account. Here we use a canonical problem in heterogeneous catalysis, the transient continuous stirred tank reactor (T-CSTR), for illustration. The T-CSTR problem is modelled fundamentally as an ordinary differential equation (ODE) system, but it can be transformed to a DAE system if one approximates typically fast surface processes using algebraic constraints for the surface species. We demonstrate the application of CSP analysis for both ODE and DAE constructions of a T-CSTR problem, illustrating the dynamical response of the system in each case. We also highlight the utility of the analysis in commenting on the quality of any particular DAE approximation built using the quasi-steady state approximation (QSSA), relative to the ODE reference case.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2018

Cabana

Physics simulations that use particles in their algorithms are ubiquitous on modern high performance computers. Cabana is a C++ library for developing particle-based simulations on heterogeneous high performance computing platforms. Cabana provides general data structures and algorithms for common particle operations in a way that is both performant and portable on HPC systems. This includes particle data structure allocatable and configurable on GPU and CPU architectures, and threaded and distributed algorithms for sorting particles, halo exchanges, neighbor list generation, and other basic particle operations.

Slattery, StuartR↗