Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “NWChem”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6

An Atmospheric Aerosol Short Course for Early Career Scientists

The first Department of Energy (DOE) Environmental and Molecular Science Laboratory (EMSL)/Atmospheric Radiation Measurement (ARM) aerosol summer school was held July 15-19, 2019 at the EMSL user facility located at Pacific Northwest National Laboratory. The school set out to provide graduate students and early-career scientists an overview of atmospheric aerosol science and an introduction to the measurement techniques available through the EMSL and ARM user facilities to address current aerosol science issues. Instructors for the course were drawn from scientists from the EMSL (Laskin et al., 2016; China et al., 2018) and ARM facilities (McComiskey and Ferrare, 2016) as well as from the DOE Atmospheric System Research (ASR) program. Student support for the summer school was also jointly provided by EMSL, ARM, and ASR. The school spanned five days with each day covering a particular theme. 25 students from 21 institutions attended morning lectures on aerosol composition, properties, theory, measurements, and modeling. During afternoons, the students explored instruments that collect and analyze aerosol samples at EMSL and PNNL’s Atmospheric Measurements Laboratory. They also tested relevant computer modeling simulations and learned about NWChem, EMSL’s open-source computational chemistry software suite.

54 ENVIRONMENTAL SCIENCES↗

PAPI software-defined events for in-depth performance analysis

The methodology and standardization layer provided by the Performance Application Programming Interface (PAPI) has played a vital role in application profiling for almost two decades. It has enabled sophisticated performance analysis tool designers and performance-conscious scientists to gain insights into their applications by simply instrumenting their code using a handful of PAPI functions that “just work” across different hardware components. In the past, PAPI development had focused primarily on hardware-specific performance metrics. However, the rapidly increasing complexity of software infrastructure poses new measurement and analysis challenges for the developers of large-scale applications. In particular, acquiring information regarding the behavior of libraries and runtimes—used by scientific applications—requires low-level binary instrumentation, or APIs specific to each library and runtime. No uniform API for monitoring events that originate from inside the software stack has emerged. In this article, we present our efforts to extend PAPI’s role so that it becomes the de facto standard for exposing performance-critical events, which we refer to as software-defined events (SDEs), from different software layers. Upgrading PAPI with SDEs enables monitoring of both types of performance events—hardware- and software-related events—in a uniform way, through the same consistent PAPI. The goal of this article is threefold. First, we motivate the need for SDEs and describe our design decisions regarding the functionality we offer through PAPI’s new SDE interface. Second, we illustrate how SDEs can be utilized by different software packages, specifically, by showcasing their use in the numerical linear algebra library MAGMA-Sparse, the tensor algebra library TAMM that is part of the NWChem suite, and the compiler-based performance analysis tool Byfl. Third, we provide a performance analysis of the overhead that results from monitoring SDEs and discuss the trade-offs between overhead and functionality.

Jagode, Heike↗

An automated framework for NMR chemical shift calculations of small organic molecules

When using nuclear magnetic resonance (NMR) to assist in chemical identification in complex samples, researchers commonly rely on databases for chemical shift spectra. However, authentic standards are typically depended upon to build libraries experimentally. Considering complex biological samples, such as blood and soil, the entirety of NMR spectra required for all possible compounds would be infeasible to ascertain due to limitations of available standards and experimental processing time. As an alternative, we introduce the in silico Chemical Library Engine (ISiCLE) NMR chemical shift module to accurately and automatically calculate NMR chemical shifts of small organic molecules through use of quantum chemical calculations. ISiCLE performs density functional theory (DFT)-based calculations for predicting chemical properties - specifically NMR chemical shifts in this manuscript - via the open source, high-performance omputational chemistry software, NWChem. ISiCLE calculates the NMR chemical shifts of sets of molecules using any available combination of DFT method, solvent, and NMR-active nuclei, using both user-selected reference compounds and/or linear regression methods. Calculated NMR chemical shifts are provided to the user for each molecule, along with comparisons with respect to a number of metrics commonly used in the literature. Here, we demonstrate ISiCLE using a set of 312 molecules, ranging in size up to 90 carbon atoms. For each, calculation of NMR chemical shifts have been performed with 8 different levels of DFT theory, and with solvation effects using the implicit solvent Conductor-like Screening Model (COSMO). The DFT method dependence of the calculated chemical shifts have been systematically investigated through benchmarking and subsequently compared to experimental data available in the literature. Furthermore, ISiCLE has been applied to a set of 80 methylcyclohexane conformers, combined via Boltzmann weighting and compared to experimental values. We demonstrate that our protocol shows promise in the automation of chemical shift calculations and, ultimately, the expansion of chemical shift libraries.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Institute for Sustained Performance, Energy, and Resilience (SuPER)

The University of Tennessee (UTK) and University of Texas at El Paso (UTEP) partnership supported the three main thrusts of the SUPER project---performance, energy, and resilience. The UTK-UTEP effort thus helped advance the main goal of SUPER, which was to ensure that DOE's computational scientists can successfully exploit the emerging generation of high performance computing (HPC) systems. This goal is being met by providing application scientists with strategies and tools to productively maximize performance, conserve energy, and attain resilience. The primary vehicle through which UTK provided performance measurement support to SUPER and the larger HPC community is the Performance Application Programming Interface (PAPI). PAPI is an ongoing project that provides a consistent interface and methodology for collecting hardware performance information from various hardware and software components, including most major CPUs, GPUs and accelerators, interconnects, I/O systems, and power interfaces, as well as virtual cloud environments. The PAPI software is widely used for performance modeling of scientific and engineering applications---for example, the HOMME (High Order Methods Modeling Environment) climate code, and the GAMESS and NWChem computational chemistry codes---on DOE supercomputers. PAPI is widely deployed as middleware for use by higher-level profiling, tracing, and sampling tools (e.g., CrayPat, HPCToolkit, Scalasca, Score-P, TAU, Vampir, PerfExpert), making it the de facto standard for hardware counter analysis. PAPI has established itself as fundamental software infrastructure in every application domain (spanning academia, government, and industry), where improving performance can be mission critical. Ultimately, as more application scientists migrate their applications to HPC platforms, they will benefit from the extended capabilities this grant brought to PAPI to analyze and optimize performance in these environments, whether they use PAPI directly, or via third-party performance tools. Capabilities added to PAPI through this grant include support for new architectures such as the lastest GPU and Xeon Phi accelerators, and advanced power measurement and management features. Another important topic for the UTK team was providing support for a rich ecosystem of different fault management strategies in the context of parallel computing. Our long term efforts have been oriented toward proposing flexible strategies and providing building boxes that application developers can use to build the most efficient fault management technique for their application. These efforts span across the entire software spectrum, from theoretical models of existing strategies to easily assess their performance, to algorithmic modifications to take advantage of specific mathematical properties for data redundancy and to extensions to widely used programming paradigms to empower the application developers to deal with all types of faults. We have also continued our tight collaborations with users to help them adopt these technologies to ensure their application always deliver meaningful scientific data. Large supercomputer systems are becoming more and more power and energy constrained, and future systems and applications running on them will need to be optimized to run under power caps and/or minimize energy consumption. The UTEP team contributed to the SUPER energy thrust by developing power modeling methodologies and investigating power management strategies. Scalability modeling results showed that some applications can scale better with respect to an increasing power budget than with respect to only the number of processors. Power management, in particular shifting power to processors on the critical path of an application execution, can reduce perturbation due to system noise and other sources of runtime variability, which are growing problems on large-scale power-constrained computer systems.

42 ENGINEERING↗

Mechanisms of interfacial reactivity in near surface and extreme environments

The local water structure surrounding ions in aqueous solutions greatly affects their chemical properties such as reaction rates, ion association, and proton and electron transport. These properties result in the behavior of ions in natural aqueous environments. For example ore transport is facilitated by chloride ion pair formation and the reaction of ions in an interface is strongly dependent on the dehydration of the ion hydration shell. We are developing the use of high-­resolution XAFS observations and 1st principles based MD-­XAFS analysis (spectra simulated using 1st principle methods with no adjustable parameters, AIMD) to interpret the solution properties of strongly interacting aqueous solutes under arbitrary pressure and temperature conditions. In the 1st principle MD-­XAFS method density functional theory (DFT) based MD simulations(Car and Parrinello, 1985) are used to generate a large ensemble of structural snap shots of the hydration region. These are then used to generate scattering intensities. I emphasize three points about this novel approach to analyzing XAFS data. 1st: As illustrated in Figure 1, the level of agreement between the calculated and observed intensities is considerably higher than has been obtained by any XAFS analysis to date (note 2nd shell region, R> 2 Å). 2nd: This result was obtained from a parameter free simulation with no fitting of the interaction potentials to any data. This supports the use of these methods for more difficult environments and more complex solutes (polyions). 3rd: New information about the shell structure (Figure 1) is now available because of this more detailed agreement. We note also that both multiple scattering and second shell features are well represented in the analysis. As far as we know this is the 1st analysis of second shell structure and multiple scattering. Excellent agreement has been obtained for most of the third row metal ions: Ca 2+ , Zn 2+ , Cu 2+ , Ni 2+ , Co 2+ , Mn 2+ , Fe 3+ , Cr 3+ . Calculations on these systems are demanding because of their open electronic shells, and high ionic charge. Principal Investigator: Professor John Weare (University of California, San Diego) The prediction of the interactions of geochemical fluids with minerals, nanoparticles, and colloids under extreme near surface conditions of temperature (T) and pressure (P) is a grand challenge research need in geosciences (U.S. DOE 2007, Basic Research Needs for Geosciences: Facilitating the 21st Energy Systems.). To evaluate the impact of these processes on energy production and management strategies it is necessary to have a high level of understanding of the interaction between complex natural fluids and mineral formations. This program emphasizes 1st principle parameter free simulations of complex chemical processes in solutions, in the mineral phase, and in the interfaces between these phases The development of new computational tools (with emphasis on oxide materials and reaction dynamics) tailored to treat wide range of conditions and time scales experienced in such geochemical applications is has been developed. Because of the sensitivity of the interaction in these systems to electronic structure and local bonding environments, and of the need to describe bond breaking/formation, our simulations are based on interactions calculated at the electronic structure level (ab-initio molecular dynamics, AIMD). The progress in the computational aspects of program may be summarized in terms of the following themes (objectives); Development of efficient parameter free dynamical simulation technology based on 1st principles force and energy calculations especially adapted for geochemical applications (e.g., mineral, interfaces and aqueous solutions) (continuing program); Calculation of the dynamics of water structure of in the surface-water interface of transition metal oxides and oxihydroxides; and Development of improved (beyond DFT+GGA) electronic structure calculations for minerals and the interface region that more accurately calculate electron correlation, spin density, and localization. The focus of the program is also on the iron oxide and oxihydroxide minerals and Fe 2+ (aq)/Fe 3+ (cr) oxidation in the mineral solution interface region. These methods included the development of model Hamilitonian methods that can be solved to near convergence for single site models (DMFT) and many-body perturbation methods (MP2, GW); Development of time decomposition methods to extend time scales of molecular dynamics (MD) simulations and support the use of high complexity electronic structure calculations (MP2, CCSD(T)) of forces for use in dynamical simulations where very high chemical accuracy is required (microsolvated reactions in absorbed surface layers); and The development of a new linear scaling finite element solver for eigenvalue problem that supports solution of quantum problems with unusual potential and boundary values. Application progress of the above new simulation technology to problems of geochemical interests includes; The prediction of metal oxide surface structure and the reduction/oxidation of Fe 3+ (cr)/Fe 2+ (aq) in metal oxide (hematite, goethite)/solution interfaces. Result: water interacts strongly with the 001 Hematite surface; interaction of water with the 100 goethite is weak; The study of ion solvation and the composition of ion hydration shells under extreme conditions (focus on Fe 3+/2+ , Al 3+ and Mg 2+ and their hydroxide speciation). Result: Ion association in water solutions can be calculated from 1st principle methods. Efficient sampling of the free energy requires more development; The continued development of new high resolution analysis of XAFS scattering of disordered systems (particularly Al, Mg) and of XANES calculations for aqueous ions. Result: EXAFS spectra can be calculated to high accuracy with DFT level dynamic simulations; The exploration of electron localization and electron transport in metal oxides (highly correlated materials). Result: Proper description of electron localization requires levels of calculation beyond DFT; and Localization of electrons in DFT type Hamiltonians was studied. Result: For very Dirac high exchange new solutions (New unphysical bifrucations) to the eigenvalue problem are found. The program was highly collaborative involving faculty and students in mathematics, physics and computer science departments as well as coworkers at the Pacific Northwest National Laboratories (PNNL). The students in this program had the opportunity to develop skills in the development of methods, the implementation of method on high performance parallel computers and the application of these methods to problem in geochemical science. Much of the software that was developed was incorporated in the NWchem software package maintained by PNNL.

58 GEOSCIENCES↗

Support of A Summer School Workshop and Workshop Focused on Theory and Applications of Time-Dependent Density Functional Theory

The first US-based summer school and workshop on Time-Dependent Density Functional Theory (TDDFT) was held July 11-21, 2017 in Telluride, CO. This grant provided funding to enable 33 students to attend the school, specifically with lodging and registration fee reductions. TDDFT is increasingly used in computational molecular and materials science to calculate electronic-excitation spectra and dynamics in a wide variety of applications, including photocatalysis, photo-controlled bond dissociation, and light-induced charge transfer. Software development in this community targets multiple software packages, many of which are open source, such as octopus, NWchem and Qb@ll, which are the ones our school focused on. The goal of this first iteration was to create a home for a national community of scholars, including users and developers, with a deep understanding of TDDFT, its capabilities, limitations, and high-performance computing context. We used this opportunity to explore interest in such an event in the future and based on overwhelmingly positive feedback from students and teachers, we intend to hold a similar school+workshop every two years in the US, in order to maintain the high level of interest that we witnessed and the enthusiasm amongst participants.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Theory and Simulation of Ultrafast Multidimensional Nonlinear X-ray Spectroscopy of Molecules (Final Report)

Emerging X-ray free electron laser (XFEL) beam sources offer new types of probes of matter with unprecedented spatial and temporal resolutions. These experimental advances must be met by robust theoretical and computational tools that provide predictive modeling capacity of the underlining electronic and structural dynamics. The latter will be essential for the design of sophisticated multi-pulse experiments and for their interpretation. The proposed research effort will focus on developing cutting- edge simulation tools for nonlinear multidimensional X-ray/optical spectroscopies and aims to address key questions in Priority Research Opportunities 1 (Probing and controlling electron motion within a molecule) and 3 (Capturing rare events and intermediate states in the transformation of matter) as mentioned in the BES Roundtable Report “Opportunities for Basic Research at the Frontiers of XFEL Ultrafast Science”. XFEL multidimensional nonlinear techniques, which combine sequences of X-ray and possibly optical pulses, provide a unique experimental toolbox for probing the dynamics of core and valence electronic excitations, as well as material structure. Predictive modeling of these dynamical processes requires the combination of analytical theory for nonlinear interactions of light and matter, robust quantum-chemical methodologies for the accurate description of electronic structure of various materials, and multiscale ab initio electron and nuclear dynamics techniques operating beyond Born-Oppenheimer approximation. These challenges will be addressed with three research thrusts (i) Develop and implement theoretical apparatus for modeling a broad range of multidimensional spectroscopic techniques enabled by present and upcoming XFEL facilities. This thrust also includes the incorporation of a computational module in the DOE supported open-source NWChem computational chemistry package as well as the development of other open-source codes ready for dissemination across a broad user base; (ii) Propose and design new multi-pulse experiments that make use of the capabilities of the incoming LCLS-II facility; (iii) Perform selected applications to specific molecular systems that can be carried out at LCLS-II and demonstrate how these X-ray sources may be used to study nonadiabatic dynamics through conical interactions, electronic correlations in multi-core excitons, and charge transfer/energy transfer processes. The proposed research will be carried out by a multi-disciplinary four-institution research team which combines academia and national laboratories and spans the broad and necessary expertise in theoretical spectroscopy, nonlinear optics, quantum chemistry, molecular non-adiabatic dynamics and code development. The work will be performed in a highly interactive team environment with junior researchers shared between institutions thus cementing cross- disciplinary interactions. The developed simulation tools will be immediately deployed for XFEL facility users, both experimentalists and theorists, via freely distributed codes and databases. Altogether, this project will facilitate establishing XFEL-based multidimensional spectroscopies as a novel diagnostic tool for monitoring electronic and structural dynamics in molecular materials.

74 ATOMIC AND MOLECULAR PHYSICS↗

DOE STTR Phase I Final Report Report: Machine-learning Based Prediction of Thermal Limits for Conjugated Organic Materials

The SCANN-DT Phase I STTR project led by NLM Photonics and partnering with the National Renewable Energy Laboratory (NREL) sought to apply machine learning techniques based on graph neural networks (GNNs) towards the prediction of decomposition temperatures (Td) of organic semiconductors, based on prior work on bond dissociation energy (BDE) prediction as implemented in NREL’s ALFABET prediction tool. Using a curated set of experimental decomposition energies, the project examined GNN-based, classical quantitative structure-property relationship (QSPR) based on DFT calculations, and combinations of both methods to predict Td. While the best MAEs in Td achieved were near 40°C, below project targets, the project led to improvements in the ALFABET model, improvements in cloud-based implementations of NWChem software, and a substantial dataset of calculations on medium-sized conjugated organic molecules.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Cytochrome Nanowire XRD

Microbial metabolism plays an important role in the global cycling of carbon, nutrients, and metals. Recent discovery of microorganisms sharing energy using direct electrical connections might represent a prominent strategy by which anaerobic microorganisms interact with the diversity of environments. The purpose of the proposed research is to apply these newly developed principles of electron exchange in microbial communities to the study of biogeochemical cycling in climate and subsurface systems. Previous studies revealed that soil bacteria Geobacter sulfurreducens produce conductive pili nanofilaments that facilitate long-range electron transport to extracellular electron acceptors and other cells. This discovery has already transformed our understanding of the function of microbial communities in diverse environments, making basic contributions to microbial ecology and helping to improve practical applications such as increasing current output of microbial fuel cells and enhancing the conversion of organic waste to methane. Microbial electron exchange via pili may also contribute to methane production in terrestrial environments that are major sources of atmospheric methane. However, the mechanism of electron transport through pili is not well understood. Initial studies have suggested that pili exhibit metallic-like conductivity similar to conducting polymer polyaniline. However, these studies were limited to electrical measurements. X-ray diffraction studies have suggested that aromatic amino acids tyrosine and phenylalanine are closely packed in pili. But these studies used homology modeling to predict the pilus assembly and information about the number of monomers in each pilus assembly unit and their organization are still lacking. The central hypothesis of this proposal is aromatics in pili enable intermolecular electron delocalization due to pi-stacking that can give rise to metallic-like conductivity. By analyzing pili structure using infrared scattering-scanning near field optical microscope (IR s-SNOM) and combining with solution NMR, Focused Ion Beam (FIB and computational capabilities, this work will provide comprehensive understanding of the conduction mechanism from a structure-function perspective. The specific aims are: 1) With infrared scattering scanning near field optical microscope (IR s-SNOM), image the organization and density of pilin monomers and pH-induced conformational changes in a pilus filament. 2) With solution NMR, resolve pH-induced structural changes in the pilin monomer. 3) Using Focused Ion Beam (FIB), draw electrodes on single pili to measure their conductivity. 4) With computational modeling using the NWChem, simulate the effect of environment on the electronic properties of pili as a function of pH. The proposed work will gain the nanoscale insight into the structure of the pili, as the basis for understanding interspecies electrical communications. The improved understanding will be harnessed to accelerate bioenergy production and bioremediation using pili as well as for predictive modeling of carbon cycling in diverse environments to reach the central goals of DOE-BER to develop sustainable energy sources, regulate the contaminants in the subsurface and understand the effects of greenhouse gas emissions on biosphere. The proposed studies will provide design principles for engineering of electrical interactions among these communities by manipulating the amino acid composition of pili. Moreover, these studies will identify the forces that maintain the function of these communities in the face of environmental perturbation. Therefore, these studies will help to address the need "to incorporate process understanding of biogeochemical cycling into a scalable hierarchy of predictive capabilities" specified in EMSL's Science Themes.

59 BASIC BIOLOGICAL SCIENCES↗

On the Efficient Evaluation of the Exchange Correlation Potential on Graphics Processing Unit Clusters

The predominance of Kohn–Sham density functional theory (KS-DFT) for the theoretical treatment of large experimentally relevant systems in molecular chemistry and materials science relies primarily on the existence of efficient software implementations which are capable of leveraging the latest advances in modern high-performance computing (HPC). With recent trends in HPC leading toward increasing reliance on heterogeneous accelerator-based architectures such as graphics processing units (GPU), existing code bases must embrace these architectural advances to maintain the high levels of performance that have come to be expected for these methods. In this work, we purpose a three-level parallelism scheme for the distributed numerical integration of the exchange-correlation (XC) potential in the Gaussian basis set discretization of the Kohn–Sham equations on large computing clusters consisting of multiple GPUs per compute node. In addition, we purpose and demonstrate the efficacy of the use of batched kernels, including batched level-3 BLAS operations, in achieving high levels of performance on the GPU. We demonstrate the performance and scalability of the implementation of the purposed method in the NWChemEx software package by comparing to the existing scalable CPU XC integration in NWChem.

97 MATHEMATICS AND COMPUTING↗

Insulator Surface Charge as a Function of Pressure: Theory and Simulation

A two-phase equilibrium model was developed to explain the discontinuous surface charge decay versus atmospheric pressure of insulators that had been charged triboelectrically. The two-phase model is an electrostatic form of the Langmuir Isotherm for ions adsorbed on a surface in equilibrium with ions in the gas phase. In this paper, the model was extended to account for vibrational states of the adsorbed surface ions via the vibrational partition function. An analysis is performed that rules out Paschen discharge as the cause of the discharge observed. Also, a numerical simulation is performed using NWChem to calculate the adsorption energies of ions on insulator surfaces for comparison to curve fit adsorption energies developed from the model and experimental data.

Hogue, Michael D.↗

Network-friendly one-sided communication through multinode cooperation on petascale cray xt5 systems

ne-sided communication is important to enable asynchronous communication and data movement for Global Address Space (GAS) programming models. Such communication is typically realized through direct messages between initiator and target processes. For petascale systems with 10,000s of nodes and 100,000s of cores, these direct messages require dedicated communication buffers and/or channels, which can lead to significant scalability challenges for GAS programming models. In this paper, we describe a network-friendly communication model, multinode cooperation, to enable indirect one-sided communication. Compute nodes work together to handle one-sided requests through (1) request forwarding in which one node can intercept a request and forward it to a target node, and (2) request aggregation in which one node can aggregate many requests to a target node. We have implemented multinode cooperation for a popular GAS runtime library, Aggregate Remote Memory Copy Interface (ARMCI). Our experimental results on a large-scale Cray XT5 system demonstrate that, multinode cooperation is able to greatly increase the memory scalability by reducing the number of communication buffers. In addition, multinode cooperation improves the resiliency of GAS runtime system to network contention. Furthermore, multinode cooperation can benefit the performance of scientific applications. In one case, it reduces the total execution time of an NWChem application by 52%.

99 GENERAL AND MISCELLANEOUS↗

Network-friendly one-sided communication through multinode cooperation on petascale cray xt5 systems

ne-sided communication is important to enable asynchronous communication and data movement for Global Address Space (GAS) programming models. Such communication is typically realized through direct messages between initiator and target processes. For petascale systems with 10,000s of nodes and 100,000s of cores, these direct messages require dedicated communication buffers and/or channels, which can lead to significant scalability challenges for GAS programming models. In this paper, we describe a network-friendly communication model, multinode cooperation, to enable indirect one-sided communication. Compute nodes work together to handle one-sided requests through (1) request forwarding in which one node can intercept a request and forward it to a target node, and (2) request aggregation in which one node can aggregate many requests to a target node. We have implemented multinode cooperation for a popular GAS runtime library, Aggregate Remote Memory Copy Interface (ARMCI). Our experimental results on a large-scale Cray XT5 system demonstrate that, multinode cooperation is able to greatly increase the memory scalability by reducing the number of communication buffers. In addition, multinode cooperation improves the resiliency of GAS runtime system to network contention. Furthermore, multinode cooperation can benefit the performance of scientific applications. In one case, it reduces the total execution time of an NWChem application by 52%.

Tipparaju, Vinod↗

Dynamic Time-Variant Connection Management for PGAS Models on InfiniBand

InfiniBand (IB) has established itself as a promising network infrastructure for high-end cluster computing systems as evidenced by its usage in the Top500 supercomputers today. While the IB standard describes multiple communication models (including reliable-connection (RC), and unreliable datagram (UD)), most of its promising features such as remote direct memory access (RDMA), hardware atomics and network fault tolerance are only available for the RC model which requires connections between communicating process pairs. In the past, several researchers have proposed on-demand connection management techniques that establish connections when there is a need to communicate, and not before. While such techniques work well for algorithms and applications that only communicate with a small set of processes in their life-time, there exists a broad set of applications that do not follow this trend. For example, applications that perform dynamic load balancing and adaptive work stealing have a small set of communicating neighbors at any given time, but over time the total number of neighbors can be very high; in some cases, equal to the entire system size. In this paper, we present a dynamic time-variant connection management approach that establishes connections on-demand like previous approaches, but further intelligently tears down some of the unused connections as well. While connection tear-down itself is relevant for any programming model, different models have different complexities. In this paper, we study the Global Arrays (GA) PGAS model for two reasons: (1) the simple one-sided communication primitives provided by GA and other PGAS models ensure that connection requests are always initiated by the origin process without explicit synchronization with the target process---this makes connection tear-down simpler to handle; and (2) GA supports several applications that demonstrate this behavior making it an obvious first target for the proposed enhancements. We evaluate our proposed approach using several micro-benchmarks as well as the NWChem computational chemistry application on more than 6000 processes, and show that our approach can significantly reduce the memory requirements of the communication library while maintaining its performance. To the best of our knowledge, this is the first design, implementation and evaluation of connection tear-down protocols over InfiniBand.

99 GENERAL AND MISCELLANEOUS↗

Global Arrays

Global Arrays (GA) is a distributed-memory programming model that allows for shared-memory-style programming combined with one-sided communication, to create a set of tools that combine high performance with ease-of-use. GA exposes a relatively straightforward programming abstraction, while supporting fully-distributed data structures, locality of reference, and high-performance communication. GA was originally formulated in the early 1990’s to provide a communication layer for the Northwest Chemistry (NWChem) suite of chemistry modeling codes that was being developed concurrently.

global arrays↗

Scalable Heterogeneous Execution of a Coupled-Cluster Model with Perturbative Triples

The CCSD(T) coupled-cluster model with perturbative triples is considered a gold standard for computational modeling of the correlated behavior of electrons in molecular systems. A fundamental constraint is the relatively small global-memory capacity in GPUs compared to the main-memory capacity on host nodes, necessitating relatively smaller tile sizes for high-dimensional tensor contractions in NWChem's GPU-accelerated implementation of the CCSD(T) method. A coordinated redesign is described to address this limitation and associated data movement overheads, including a novel fused GPU kernel for a set of tensor contractions, along with inter-node communication optimization and data caching. The new implementation of GPU-accelerated CCSD(T) improves overall performance by 3.4x. Finally, we discuss the trade-offs in using this fused algorithm on current and future supercomputing platforms.

Kim, Jinsung↗

Computational Chemistry at the Petascale: Are We There Yet?

The field of electronic structure is struggling to get efficient parallel implementation on Petascaleclass hardware. One notable exception has been the achievement of Qbox, a planewavepseudopotential electronic structure code that obtained a performance of 207 TFlops on aBlueGene/L computer.Qbox makes use of the message-passing MPI library for parallelization. Instead, NWChemmakes use of the Global Arrays library; this allows the software developer to reach a high levelof abstraction and, at the same time, to use one-sided communication to efficiently exploitthe network hardware. In the remainder of the paper, we will discuss recent benchmarksand scientic results obtained with NWChem on a parallel computer whose theoretical peakperformance is in excess of 1 PFlops.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Enabling a Highly-Scalable Global Address Space Model for Petascale Computing

Over the past decade, the trajectory to the petascale has been built on increased complexity and scale of the underlying parallel architectures. Meanwhile, software de- velopers have struggled to provide tools that maintain the productivity of computational science teams using these new systems. In this regard, Global Address Space (GAS) programming models provide a straightforward and easy to use addressing model, which can lead to improved produc- tivity. However, the scalability of GAS depends directly on the design and implementation of the runtime system on the target petascale distributed-memory architecture. In this paper, we describe the design, implementation, and optimization of the Aggregate Remote Memory Copy Interface (ARMCI) runtime library on the Cray XT5 2.3 PetaFLOPs computer at Oak Ridge National Laboratory. We optimized our implementation with the flow intimation technique that we have introduced in this paper. Our optimized ARMCI implementation improves scalability of both the Global Arrays (GA) programming model and a real-world chemistry application NWChem from small jobs up through 180,000 cores.

97 MATHEMATICS AND COMPUTING↗