Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “high performance parallel computing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,495 records · Page 83
All sourcesNASA NTRSDOE OSTI
DOE OSTI2017

Massively parallel first-principles simulation of electron dynamics in materials

Here we present a highly scalable, parallel implementation of first-principles electron dynamics coupled with molecular dynamics (MD). By using optimized kernels, network topology aware communication, and by fully distributing all terms in the time-dependent Kohn–Sham equation, we demonstrate unprecedented time to solution for disordered aluminum systems of 2000 atoms (22,000 electrons) and 5400 atoms (59,400 electrons), with wall clock time as low as 7.5 s per MD time step. Despite a significant amount of non-local communication required in every iteration, we achieved excellent strong scaling and sustained performance on the Sequoia Blue Gene/Q supercomputer at LLNL. We obtained up to 59% of the theoretical sustained peak performance on 16,384 nodes and performance of 8.75 Petaflop/s (43% of theoretical peak) on the full 98,304 node machine (1,572,864 cores). Lastly, scalable explicit electron dynamics allows for the study of phenomena beyond the reach of standard first-principles MD, in particular, materials subject to strong or rapid perturbations, such as pulsed electromagnetic radiation, particle irradiation, or strong electric currents.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Accelerating Thermochemical Equilibrium Calculations for Nuclear Reactor Applications

Thermochemical properties play a key role in modeling and simulation of several key phenomena in nuclear reactors. There has been an increasing interest in incorporating CALPHAD-based formulations in multiphysics simulations including for Molten Salt Reactors where knowledge of phase evolution of the salt and the chemical potentials of various elements are of utmost importance in source term analyses and redox control. However, the size of such simulations is often limited by the high computational cost of full thermodynamic equilibrium calculations. This work discusses the current efforts aimed at accelerating thermochemical equilibrium calculations for multiphysics simulations performed using the open-source finite element / finite volume code Multiphysics Object Oriented Simulation Environment (MOOSE) [1]. While several methods have been proposed for accelerating phase equilibrium calculations [2], most focus on relatively small systems and often rely on a- priori knowledge of the state-space of the system. Nuclear materials, however, are often multi-component systems owing to the evolution of composition under irradiation and an approach based on a-priori mapping of phase diagram is often not enough. This work is aimed at demonstrating an on-the-fly surrogate modeling framework that uses active learning to reduce the number of full equilibrium calculations that must be performed. By combining with efficient coupling approaches, the surrogate framework helps in reducing the computational cost of thermodynamic equilibrium informed multiphysics simulations of nuclear materials. The performance is benchmarked against full coupling with the thermochemistry library Thermochimica [3]. This work uses a machine learning based approach for constructing surrogate models to predict the stable phases in a multicomponent system. The surrogates were constructed using neural networks and Gaussian process classification. In this work, we compare the relative performance of the two methods. We also demonstrate the use of caching previous calculations by interpolating the values from nearest neighbors. References [1] Lindsay, A.D., et al. "2.0 – MOOSE: Enabling massively parallel multiphysics simulation", SoftwareX, 20 (2022): 101202. [2] Roos, W.A. and Zietsman J.H. "Accelerating complex chemical equilibrium calculations – A Review", Calphad, 77 (2022): 102380. [3] Piro, M.H.A., et al. "The thermochemistry library Thermochimica", Computational Materials Science, 67 (2013): 266-272.

36 MATERIALS SCIENCE↗
DOE OSTI2019

Striped Data Analysis Framework

Traditionally, High Energy data analysis is based on the model where data are stored in files and analyzed by running multiple analysis processes, each reading one or more of the data files. This process involves repeated data reduction step, which produces smaller files, which is time consuming and leads to data duplication. We propose an alternative approach to data storage and analysis, based on the Big Data technologies. The idea is to store each element of data once and only once in a distributed scalable database and analyze data by reading only "interesting" pieces of data from the database. To make this approach possible, we developed columnar Striped Data Representation Format as the basis of the framework. Traditional columnar approach allows for efficient analysis of complex data structures. While keeping all the benefits of columnar data representation, striped mechanism goes further by enabling efficient parallelization of computations and flexible dist ribution of data analysis. The framework includes scalable and elastic data storage, compute and user analysis backend components. The framework uses off-the shelf web services and data caching technologies as the compute/data co-location mechanism. Flexible architecture allows the framework run in the cloud using container technologies. The framework offers Python/Jupyter as the user analysis backend platform, but can also run in command line or batch mode. In the article, we will present the results of the FNAL-LDRD-2016-032 FNAL LDRD project, the design, implementation, features and performance characteristics of the Striped Data Analysis Framework.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2018

Coping with silent and fail-stop errors at scale by combining replication and checkpointing

This paper provides a model and an analytical study of replication as a technique to cope with silent errors, as well as a mixture of both silent and fail-stop errors on large-scale platforms. Compared with fail-stop errors that are immediately detected when they occur, silent errors require a detection mechanism. To detect silent errors, many application-specific techniques are available, either based on algorithms (e.g., ABFT), invariant preservation or data analytics, but replication remains the most transparent and least intrusive technique. We explore the right level (duplication, triplication or more) of replication for two frameworks: (i) when the platform is subject to only silent errors, and (ii) when the platform is subject to both silent and fail-stop errors. A higher level of replication is more expensive in terms of resource usage but enables to tolerate more errors and to even correct some errors, hence there is a trade-off to be found. Replication is combined with checkpointing and comes with two flavors: process replication and group replication. Process replication applies to message-passing applications with communicating processes. Each process is replicated, and the platform is composed of process pairs, or triplets. Group replication applies to black-box applications, whose parallel execution is replicated several times. The platform is partitioned into two halves (or three thirds). In both scenarios, results are compared before each checkpoint, which is taken only when both results (duplication) or two out of three results (triplication) coincide. Otherwise, one or more silent errors have been detected, and the application rolls back to the last checkpoint, as well as when fail-stop errors have struck. We provide a detailed analytical study for all of these scenarios, with formulas to decide, for each scenario, the optimal parameters as a function of the error rate, checkpoint cost, and platform size. We also report a set of extensive simulation results that nicely corroborates the analytical model.

Checkpointing↗
DOE OSTI2020

INTEGRATION OF FLEX EQUIPMENT AND OPERATOR ACTIONS IN PLANT FORCE-ON-FORCE MODELS WITH DYNAMIC RISK ASSESSMENT

The overall operation and maintenance cost to protect nuclear power plants accounts for approximately 7% of the total cost of power generation, with labor accounting for half of this cost. In the current research, from interaction with utilities and other stakeholders, it was determined that physical security forces account for nearly 20% of the entire workforce at several nuclear power plants. Labor costs continue to rise in the U.S., so any measures to reduce the cost of operating a nuclear power plant will need to include a reduction in labor. The physical security pathway within the DOE’s Light Water Reactor Sustainability program aims to lower the cost of physical security through directed research into modeling and simulation, application of advanced sensors or deployment of advanced weapons. This report presents a modeling and simulation framework for integrating Diverse and Flexible Mitigation Capability (FLEX) portable equipment performance with Force on Force models of a plant’s physical security posture. The generic framework is described in detail, followed by a case study of modeling an adversarial attack aimed at causing a radiological release by sabotaging the plant’s power supply and its ultimate heat sink capabilities at a hypothetical nuclear power plant. Two different FLEX deployment strategies, series and parallel, are modeled with distinct timelines. The results of the adversarial attack modeled in a commercial Force on Force tool are integrated with the FLEX deployment model in INL’s dynamic modeling tool EMRALD. Monte Carlo simulation is used to model the distribution of the timeline in FLEX deployment strategies. The results demonstrate that, even in the extreme case of a successful adversarial attack, deployment of FLEX equipment can result in a significantly high likelihood of preventing radiological release. The modeling and simulation framework integrating FLEX equipment with Force on Force models enables the nuclear power plants to credit FLEX portable equipment in the plant security posture, resulting in an efficient and optimized physical security.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2016

Optical CT imaging of solid radiochromic dosimeters in mismatched refractive index solutions using a scanning laser and large area detector

Purpose: The practical use of the PRESAGE® solid plastic dosimeter is limited by the inconvenience of immersing it in high-viscosity oils to achieve refractive index matching for optical computed tomography (CT) scanning. The oils are slow to mix and difficult to clean from surfaces, and the dosimeter rotation can generate dynamic Schlieren inhomogeneity patterns in the reference liquid, limiting the rotational and overall scan speed. Therefore, it would be beneficial if lower-viscosity, water-based solutions with slightly unmatched refractive index could be used instead. The purpose of this work is to demonstrate the feasibility of allowing mismatched conditions when using a scanning laser system with a large acceptance angle detector. A fiducial-based ray path measurement technique is combined with an iterative CT reconstruction algorithm to reconstruct images. Methods: A water based surrounding liquid with a low viscosity was selected for imaging PRESAGE® solid dosimeters. Liquid selection was optimized to achieve as high a refractive index as possible while avoiding rotation-induced Schlieren effects. This led to a refractive index mismatch of 6% between liquid and dosimeters. Optical CT scans were performed with a fan-beam scanning-laser optical CT system with a large area detector to capture most of the refracted rays. A fiducial marker placed on the wall of a cylindrical sample occludes a given light ray twice. With knowledge of the rotation angle and the radius of the cylindrical object, the actual internal path of each ray through the dosimeter can be calculated. Scans were performed with 1024 projections of 512 data samples each, and rays were rebinned to form 512 parallel-beam projections. Reconstructions were performed on a 512 × 512 grid using 100 iterations of the SIRT iterative CT algorithm. Proof of concept was demonstrated with a uniformly attenuating solution phantom. PRESAGE® dosimeters (11 cm diameter) were irradiated with Cobalt-60 irradiator to achieve either a uniform dose or a 2-level “step-dose” pattern. Results: With 6% refractive index mismatching, a circular field of view of 85% of the diameter of a cylindrical sample can be reconstructed accurately. Reconstructed images of the test solution phantom were uniform (within 3%) inside this radius. However, the dose responses of the PRESAGE® samples were not spatially uniform, with variations of at least 5% in sensitivity. The variation appears as a “cupping” artifact with less sensitivity in the middle than at the periphery of the PRESAGE® cylinder. Polarization effects were also detected for these samples. Conclusions: The fiducial-based ray path measurement scheme, coupled with an iterative reconstruction algorithm, enabled optical CT scanning of PRESAGE® dosimeters immersed in mismatched refractive index solutions. However, improvements to PRESAGE® dose response uniformity are required.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2018

Distributed Louvain Algorithm for Graph Community Detection

In most real-world networks, the nodes/vertices tend to be organized into tightly-knit modules known as communities or clusters, such that nodes within a community are more likely to be “related” to one another than they are to the rest of the network. The goodness of partitioning into communities is typically measured using a well known measure called modularity. However, modularity optimization is an NP-complete problem. In 2008, Blondel, et al. introduced a multi-phase, iterative heuristic for modularity optimization, called the Louvain method. Owing to its speed and ability to yield high quality communities, the Louvain method continues to be one of the most widely used tools for serial community detection. In this paper, we present the design of a distributed memory implementation of the Louvain algorithm for parallel community detection. Our approach begins with an arbitrarily partitioned distributed graph input, and employs several heuristics to speedup the computation of the different steps of the Louvain algorithm. We evaluate our implementation and its different variants using real-world networks from various application domains (including internet, biology, social networks). Our MPI+OpenMP implementation yields about 7x speedup (on 4K processes) for soc-friendster network (1.8B edges) over a state-of-the-art shared-memory multicore implementation (on 64 threads), without compromising output quality. Furthermore, our distributed implementation was able to process a larger graph (uk-2007; 3.3B edges) in 32 seconds on 1K cores (64 nodes) of NERSC Cori, when the state-of-the-art shared-memory implementation failed to run due to insufficient memory on a single Cori node containing 128 GB.

High performance computing, graph algorithms, Dist↗
DOE OSTI2017

Considerations on the Use of Custom Accelerators for Big Data Analytics

Accelerators, including Graphic Processing Units (GPUs) for gen- eral purpose computation, many-core designs with wide vector units (e.g., Intel Phi), have become a common component of many high performance clusters. The appearance of more stable and reliable tools tools that can automatically convert code written in high-level specifications with annotations (such as C or C++) to hardware de- scription languages (High-Level Synthesis - HLS), is also setting the stage for a broader use of reconfigurable devices (e.g., Field Pro- grammable Gate Arrays - FPGAs) in high performance system for the implementation of custom accelerators, helped by the fact that new processors include advanced cache-coherent interconnects for these components. In this chapter, we briefly survey the status of the use of accelerators in high performance systems targeted at big data analytics applications. We argue that, although the progress in the use of accelerators for this class of applications has been sig- nificant, differently from scientific simulations there still are gaps to close. This is particularly true for the ”irregular” behaviors exhibited by no-SQL, graph databases. We focus our attention on the limits of HLS tools for data analytics and graph methods, and discuss a new architectural template that better fits the requirement of this class of applications. We validate the new architectural templates by mod- ifying the Graph Engine for Multithreaded System (GEMS) frame- work to support accelerators generated with such a methodology, and testing with queries coming from the Lehigh University Benchmark (LUBM). The architectural template enables better supporting the task and memory level parallelism present in graph methods by sup- porting a new control model and a enhanced memory interface. We show that out solution allows generating parallel accelerators, pro- viding speed ups with respect to conventional HLS flows. We finally draw conclusions and present a perspective on the use of reconfig- urable devices and Design Automation tools for data analytics.

Data Analytics↗
DOE OSTI2020

kynema-sgf [SWR-20-85]

Kynema-SGF (formerly AMR-Wind), wherein SGF stands for structured-grid fluid dynamics, is a massively parallel, block-structured adaptive-mesh, incompressible flow solver. The codebase was initiated in 2019 from incflo. The solver is built on top of the AMReX library. AMReX library provides the mesh data structures, mesh adaptivity, as well as the linear solvers used for solving the governing equations. Kynema-SGF is actively developed and maintained by a dedicated multi-institutional team from Lawrence Berkeley National Laboratory, National Laboratory of the Rockies, and Sandia National Laboratories. The primary applications for Kynema-SGF are: performing large-eddy simulations (LES) of atmospheric boundary layer (ABL) flows, simulating wind farm turbine-wake interactions using actuator disk or actuator line models for turbines, and as a background solver when coupled with a near-body solver (e.g., Kynema-UGF) with overset methodology to perform blade-resolved simulations of multiple wind turbines within a wind farm. For offshore applications, the ability to model the air-sea interaction effects and its impact on the ABL characteristics is another focus for the code development effort. As with other codes in the Kynema ecosystem, Kynema-SGF shares the following objectives: *an open, well-documented implementation of the state-of-the-art computational models for modeling wind farm flow physics at various fidelities that are backed by a comprehensive verification and validation (V&V) process; *be capable of performing the highest-fidelity simulations of flow fields within wind farms; and *be able to leverage the high-performance leadership class computing facilities available at DOE national laboratories.

Ananthan, Shreyas↗
DOE OSTI2023

Scaling comb-driven resonator-based DWDM silicon photonic links to multi-Tb/s in the multi-FSR regime

The use of chip-based micro-resonator Kerr frequency combs in conjunction with dense wavelength-division multiplexing (DWDM) enables massively parallel intensity-modulated direct-detection data transmission with low energy consumption. Resonator-based modulators and filters used in such systems can limit the number of usable wavelength channels due to practical constraints on the maximum achievable free spectral range (FSR). In this work, we introduce the design of multi-Tb/s comb-driven resonator-based silicon photonic links by leveraging the multi-FSR regime. We demonstrate the viability of the link architecture with yield estimates that are supported by extensive wafer-scale measurements of 704 micro-resonators fabricated in a commercial complementary metal–oxide–semiconductor foundry. We show that a 2.80 Tb/s link is realizable with a ≥6 σ yield (∼99.999%), and that aggregate bandwidths of 3.76 Tb/s and 4.72 Tb/s are possible if yield targets are relaxed (3 σ and 1 σ , respectively). All designs represent a 1.94−3.28× boost to aggregate link bandwidth while maintaining BER≤10 −10 performance, with a theoretical bandwidth of 10.51 Tb/s being possible for sufficiently robust resonators. We use high-speed BER measurements to inform co-optimization of data rate and aggressor spacing ( λ ag ), limiting any additional loss-based power penalties to off-resonance insertion loss (IL) and routing loss. This work demonstrates that, through the multi-FSR regime, there is a clear path toward Kerr comb-driven ultra-broadband, high bandwidth silicon photonic links that can support next-generation data centers and high-performance computers.

James, Aneek (ORCID:0000000262527807)↗
DOE OSTI2023

A family of independent Variable Eddington Factor methods with efficient preconditioned iterative solvers

We present a family of discretizations for the Variable Eddington Factor (VEF) equations that have high-order accuracy on curved meshes and efficient preconditioned iterative solvers. The VEF discretizations are combined with the Discontinuous Galerkin transport discretization from to form effective high-order, linear transport methods. The VEF discretizations are derived by extending the unified analysis of Discontinuous Galerkin methods for elliptic problems presented by Arnold et al. to the VEF equations. This framework is used to define analogs of the interior penalty, second method of Bassi and Rebay, minimal dissipation local Discontinuous Galerkin, and continuous finite element methods. The analysis of subspace correction preconditioners, which use a continuous operator to iteratively precondition the discontinuous discretization, is extended to the case of the non-symmetric VEF system. Numerical results demonstrate that the VEF discretizations have arbitrary-order accuracy on curved meshes, preserve the thick diffusion limit, and are effective on a proxy problem from thermal radiative transfer in both outer transport iterations and inner preconditioned linear solver iterations. We demonstrate that the VEF solution converges to the S N transport solution as the mesh is refined on both problems with smooth and non-smooth behavior in angle. Parallel performance studies show that the interior penalty VEF discretization's linear solve weak scales out to 1024 processors and strong scales well on a single node. Particular attention is paid to the parallel performance of the VEF algorithm when used in combination with a parallel block Jacobi transport sweep.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Effect of cell compression on the performance and the structure of proton exchange membrane water electrolyzer (PEMWE) assembly

Here, in the field of water electrolysis, the proton exchange membrane water electrolyzer (PEMWE) is currently the most advanced technology for producing hydrogen without emitting CO 2 . Although PEMWE plants are already in operation, further research is needed to improve cell efficiency and reduce the use of rare materials, such as iridium oxide catalysts for the oxygen evolution reaction (OER). One of the main causes of performance loss in PEMWE is the relatively low electric conductivity of the porous transport layer (PTL) and of the anode catalyst layer, which results in ohmic losses and low catalyst utilization during high current density operation. The objective of this study is to investigate how optimization of the PTL and electrode interface can increase the cell performance. To this end, we tested different cell assemblies using fibrous and sintered PTLs, decreasing membrane thickness, reducing iridium loading, and inserting a microporous layer to increase contact surface area. Electrochemical characterization of each cell configuration was systematically performed at various compression levels as the pressure is a crucial parameter influencing the electrode/PTL contact area. In parallel, X-ray microcomputed tomography (micro-CT) was employed to investigate the effects of cell hydration and compression on the structure of PEMWE components. This study combining electrochemistry and micro-CT imaging presents how optimizing the electrode/PTL contact surface area, minimizes ohmic losses, and enables PEMWE operation with low iridium loading at high current densities.

Catalyst - PTL interface↗
DOE OSTI2024

Optimization and Experimental Validation of Annular Finned PCM-HX for a Domestic Hot Water Heater Application

The load profile for domestic water heating is time-dependent and can result in high energy demand during peak operating times. Shifting this peak load can have significant environmental and economic impacts. Phase change material (PCM)-based thermal energy storage (TES) is a potentially useful technology for peak load shifting in domestic hot water (DHW) applications thanks to its high latent heat and energy density. In this study, an annular finned-tube PCM-HX design concept was optimized for a load-shifting TES unit to meet the Department of Energy standard for a medium-usage DHW heater using a resistance-capacitance model (RCM) integrated with a Multi-Objective Genetic Algorithm. The optimized design comprised 70 identical annular finned-tube PCM-HX units connected in parallel and utilizing RT62HC as the PCM. A single PCM-HX unit was prototyped and tested in a vertically oriented setup with upward heat transfer fluid (HTF) flow. The hot water supply time was defined based on a cutoff temperature of 51.7°C. The as-designed mass flow rate (1.5 g/s) was tested to assess the performance of the prototyped PCM-HX unit for RCM validation. For the experimental investigation, RTD sensor bundles measured HTF temperature at the PCM-HX inlet and outlet, and a Coriolis flow meter accurately measured the HTF mass flow rate. The simulated discharging power underpredicted the experimental result by about 12%, and the simulated hot water supply time underpredicted the experimental result by approximately 13% for the as-designed mass flow rate (1.5 g/s). The average deviation of the hot water supply temperature between the experimental and RCM results during the complete PCM solidification process was 1.3 K for the as-designed mass flow rate. The overall good agreement between the experimental and RCM results provides confidence that computationally efficient models such as RCM can be utilized for design optimization of PCM-HXs.

42 ENGINEERING↗
DOE OSTI2015

Motivation and Design of the Sirocco Storage System Version 1.0.

Sirocco is a massively parallel, high performance storage system for the exascale era. It emphasizes client-to-client coordination, low server-side coupling, and free data movement to improve resilience and performance. Its architecture is inspired by peer-to-peer and victim- cache architectures. By leveraging these ideas, Sirocco natively supports several media types, including RAM, flash, disk, and archival storage, with automatic migration between levels. Sirocco also includes storage interfaces and support that are more advanced than typical block storage. Sirocco enables clients to efficiently use key-value storage or block-based storage with the same interface. It also provides several levels of transactional data updates within a single storage command, including full ACID-compliant updates. This transaction support extends to updating several objects within a single transaction. Further support is provided for con- currency control, enabling greater performance for workloads while providing safe concurrent modification. By pioneering these and other technologies and techniques in the storage system, Sirocco is poised to fulfill a need for a massively scalable, write-optimized storage system for exascale systems. This is version 1.0 of a document reflecting the current and planned state of Sirocco. Further versions of this document will be accessible at http://www.cs.sandia.gov/Scalable_IO/ sirocco .

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

PLEXUS: A Pattern-Oriented Runtime System Architecture for Resilient Extreme-Scale High-Performance Computing Systems

For high-performance computing (HPC) system designers and users, meeting the myriad challenges of next-generation exascale supercomputing systems requires rethinking their approach to application and system software design. Among these challenges, providing resiliency and stability to the scientific applications in the presence of high fault rates requires new approaches to software architecture and design. As HPC systems become increasingly complex, they require intricate solutions for detection and mitigation for various modes of faults and errors that occur in these large-scale systems, as well as solutions for failure recovery. These resiliency solutions often interact with and affect other system properties, including application scalability, power and energy efficiency. Therefore, resilience solutions for HPC systems must be thoughtfully engineered and deployed.In previous work, we developed the concept of resilience design patterns, which consist of templated solutions based on well-established techniques for detection, mitigation and recovery. In this paper, we use these patterns as the foundation to propose new approaches to designing runtime systems for HPC systems. The instantiation of these patterns within a runtime system enables flexible and adaptable end-to-end resiliency solutions for HPC environments. The paper describes the architecture of the runtime system, named Plexus, and the strategies for dynamically composing and adapting pattern instances under runtime control. This runtime-based approach enables actively balancing the cost-benefit trade-off between performance overhead and protection coverage of the resilience solutions. Based on a prototype implementation of PLEXUS, we demonstrate the resiliency and performance gains achieved by the pattern-based runtime system for a parallel linear solver application.

Hukerikar, Saurabh↗
DOE OSTI2024

Visualization at exascale: Making it all work with VTK-m

The VTK-m software library enables scientific visualization on exascale-class supercomputers. Exascale machines are particularly challenging for software development in part because they use GPU accelerators to provide the vast majority of their computational throughput. Algorithmic designs for GPUs and GPU-centric computing often deviate from those that worked well on previous generations of high-performance computers that relied on traditional CPUs. Fortunately, VTK-m provides scientific visualization algorithms for GPUs and other accelerators. VTK-m also provides a framework that simplifies the implementation of new algorithms and adds a porting layer to work across multiple processor types. This paper describes the main challenges encountered when making scientific visualization available at exascale. Here, we document the surprises and obstacles faced when moving from pre-exascale platforms to the final exascale designs and the performance on those systems including scaling studies on Frontier, an exascale machine with over 37,000 AMD GPUs. We also report on the integration of VTK-m with other exascale software technologies. Finally, we show how VTK-m helps scientific discovery for applications such as fusion and particle acceleration that leverage an exascale supercomputer.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2014

Thread-Level Parallelization and Optimization of NWChem for the Intel MIC Architecture

In the multicore era it was possible to exploit the increase in on-chip parallelism by simply running multiple MPI processes per chip. Unfortunately, manycore processors' greatly increased thread- and data-level parallelism coupled with a reduced memory capacity demand an altogether different approach. In this paper we explore augmenting two NWChem modules, triples correction of the CCSD(T) and Fock matrix construction, with OpenMP in order that they might run efficiently on future manycore architectures. As the next NERSC machine will be a self-hosted Intel MIC (Xeon Phi) based supercomputer, we leverage an existing MIC testbed at NERSC to evaluate our experiments. In order to proxy the fact that future MIC machines will not have a host processor, we run all of our experiments in tt native mode. We found that while straightforward application of OpenMP to the deep loop nests associated with the tensor contractions of CCSD(T) was sufficient in attaining high performance, significant effort was required to safely and efficiently thread the TEXAS integral package when constructing the Fock matrix. Ultimately, our new MPI OpenMP hybrid implementations attain up to 65x better performance for the triples part of the CCSD(T) due in large part to the fact that the limited on-card memory limits the existing MPI implementation to a single process per card. Additionally, we obtain up to 1.6x better performance on Fock matrix constructions when compared with the best MPI implementations running multiple processes per card.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2015

GPU/MIC Acceleration of the LHC High Level Trigger to Extend the Physics Reach at the LHC

The quest for rare new physics phenomena leads the PI to propose evaluation of coprocessors based on Graphics Processing Units (GPUs) and the Intel Many Integrated Core (MIC) architecture for integration into the trigger system at LHC. This will require development of a new massively parallel implementation of the well known Combinatorial Track Finder which uses the Kalman Filter to accelerate processing of data from the silicon pixel and microstrip detectors and reconstruct the trajectory of all charged particles down to momentums of 100 MeV. It is expected to run at least one order of magnitude faster than an equivalent algorithm on a quad core CPU for extreme pileup scenarios of 100 interactions per bunch crossing. The new tracking algorithms will be developed and optimized separately on the GPU and Intel MIC and then evaluated against each other for performance and power efficiency. The results will be used to project the cost of the proposed hardware architectures for the HLT server farm, taking into account the long term projections of the main vendors in the market (AMD, Intel, and NVIDIA) over the next 10 years. Extensive experience and familiarity of the PI with the LHC tracker and trigger requirements led to the development of a complementary tracking algorithm that is described in [arxiv: 1305.4855], [arxiv: 1309.6275] and preliminary results accepted to JINST.

97 MATHEMATICS AND COMPUTING↗