Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “parallelization”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 955 records · Page 53
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

An implicit barotropic mode solver for MPAS-ocean using a modern Fortran solver interface

Here, we demonstrate use of a modern Fortran solver interface to manage solver algorithms for an implicit barotropic mode solver in the Model for Predictions Across Scales-Ocean (MPAS-O). ForTrilinos, a Fortran interface to Trilinos that contains a large collection of solver capabilities written in C++, has been implemented in MPAS-O to provide access to a suite of linear solver options. By virtue of the simplified wrapper and interface generator (SWIG) automation tool that generates modern Fortran interfaces to C++ code, we were able to implement the Fortran solver interface in MPAS-O using a familiar Fortran coding style while minimizing performance degradation. The ForTrilinos solver interface is written within MPAS-O’s time stepping modules as a subroutine in conjunction with MPAS-O code. Applied to an idealized ocean and a high-resolution realistic ocean test case, parallel performance of ForTrilinos solvers is examined. It is found that parallel scalability of the ForTrilinos solvers is highly dependent on the number of global synchronization points per solver iteration in each iterative solver algorithm. ForTrilinos solvers perform best compared to the Fortran hand-crafted (FHC) solver when the amount of work per processor is large enough. However, parallel scalability is better with the FHC solver and so when the work per core is modest FHC outperforms ForTrilinos. The intercomparison between the ForTrilinos and FHC solvers reveals that this performance hit in the ForTrilinos solver mostly comes from the global synchronization process, while suggesting that the matrix-vector multiplication process in the FHC solver needs to be optimized for better performance.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Sequence length scaling in vision transformers for scientific images on frontier

Vision Transformers (ViTs) are pivotal for foundational models in scientific imagery, including Earth science applications, due to their capability to process large sequence lengths. While transformers for text have inspired scaling sequence lengths in ViTs, adapting these for ViTs introduces unique challenges. We develop distributed sequence parallelism for ViTs, enabling them to handle up to 1M tokens. Our approach, leveraging DeepSpeed-Ulysses and Long-Sequence-Segmentation with model sharding, is the first to apply sequence parallelism in ViT training, achieving a 94% batch scaling efficiency on 2,048 AMD-MI250X GPUs. Evaluating sequence parallelism in ViTs, particularly in models up to 10B parameters, highlighted substantial bottlenecks. We countered these with hybrid sequence, pipeline, and flash attention strategies, to scale beyond single GPU memory limits. Our method significantly enhances climate modeling accuracy by 20% in temperature predictions, marking the first training of a vision transformer model to convergence with a sequence length of 188K tokens, using full self-attention.

Tsaris, Aristeidis (aris) [ORNL] (ORCID:0000000277↗
DOE OSTI2021

Fracture Description of the HFTS-2 Slant Core, Delaware Basin, West Texas

Previous work in the Midland Basin Hydraulic Fracture Test Site project (HFTS1) has shown the value of core recovered from a stimulated volume in providing information on the geometry and extent of hydraulic fractures. This project (HFTS2) is in the Wolfcamp Formation of the Delaware Basin. A slant core through the stimulated volume was acquired above and below a stimulated well and the results of the fracture description are presented here. Here, nine, 3-inch-diameter whole cores from the slant well were examined for fractures prior to slabbing, together with a CT scan of the core. We used criteria developed during the HFTS1 project to identify hydraulic, natural, drilling-induced and core-handling fractures. Material collected from a few fracture faces was cleaned, sieved, and examined under a microscope to look for proppant. A total of 1261 fractures, including 500 hydraulic fractures, were described in 948 feet of core. Hydraulic fractures are not evenly distributed through the cores; they tend to occur in clusters. Many are remarkably planar, smooth, and featureless, while others have twist hackles, steps, plumose, and kinks. Natural, subvertical fractures, mostly less than 1 mm in width, sealed with blocky calcite, are oriented NE-SW (Set 1) and WNW-ESE (Set 2). Two Set 1 fractures have drusy cement on parted faces and were possibly open prior to drilling. A few subvertical sealed fractures with different orientations are also present. Bed-parallel natural fractures, most having fibrous cement (beef), and some showing top-to-north bed-parallel shear are present. Faults with normal/oblique movement were found in one part of the core. Drilling induced and core-handling breaks were identified in all cores, and were also noted to have reactivated some natural fractures. The results from this work will allow hydraulic fracture distribution relative to the stimulated well to be assessed. The number of fractures is generally greater than the number of perforations, but these can be compared for different stages and perforation clusters. Surface features provide information about the direction of hydraulic fracture propagation and about segmentation and bifurcation that might increase the number of fracture strands. Reactivation of natural fractures, including bedding-parallel fractures, occurs although the mechanism of parting is mostly unknown. The wider HFTS2 project will make use of this fracture description to improve hydraulic fracture modeling (geomechanics and engineering), to help explain pressure depletion and fiber optics observations (reservoir depletion and frac geometry), to help verify indirect fracture diagnostic techniques such as microseismic monitoring (geophysics), and to help calibrate horizontal well image logs.

58 GEOSCIENCES↗
DOE OSTI2023

Quantum-inspired tempering for ground state approximation using artificial neural networks

A large body of work has demonstrated that parameterized artificial neural networks (ANNs) can efficiently describe ground states of numerous interesting quantum many-body Hamiltonians. However, the standard variational algorithms used to update or train the ANN parameters can get trapped in local minima, especially for frustrated systems and even if the representation is sufficiently expressive. We propose a parallel tempering method that facilitates escape from such local minima. This methods involves training multiple ANNs independently, with each simulation governed by a Hamiltonian with a different "driver" strength, in analogy to quantum parallel tempering, and it incorporates an update step into the training that allows for the exchange of neighboring ANN configurations. We study instances from two classes of Hamiltonians to demonstrate the utility of our approach using Restricted Boltzmann Machines as our parameterized ANN. The first instance is based on a permutation-invariant Hamiltonian whose landscape stymies the standard training algorithm by drawing it increasingly to a false local minimum. The second instance is four hydrogen atoms arranged in a rectangle, which is an instance of the second quantized electronic structure Hamiltonian discretized using Gaussian basis functions. We study this problem in a minimal basis set, which exhibits false minima that can trap the standard variational algorithm despite the problem’s small size. We show that augmenting the training with quantum parallel tempering becomes useful to finding good approximations to the ground states of these problem instances.

Albash, Tameem↗
DOE OSTI2020

Development of MGMC: A proxy Multi-Group Monte Carlo Particle Transport Application

This document details the development and current state of MGMC: a proxy Multi-Group Monte Carlo transport application. The goal of this work was to develop a light-weight Monte Carlo transport solver that could be easily reconfigured to test parallelization strategies targeting advanced architectures and heterogeneous computing environments. Additionally, MGMC is able to act as a proxy-app for testing the development of algorithms and the integration of other libraries and into Monte Carlo transport codes. MGMC is currently able to run in parallel using OpenMP for CPU threads or CUDA for NVIDIA GPUs. MGMC was also used to investigate the using SYCL for portable parallelization targeting either CPU threads, vender-agnostic GPUs, and ARM chips, detailed in Section II. In the process of developing MGMC, several other C++ libraries have been created with the intent for re-use in future research projects and potential inclusion in production-level codes, detailed in Section III. The physics capabilities in MGMC are detailed in Section IV, code verification is detailed in Section V, and performance results are detailed in Section VI.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Multi-Node Program Fuzzing on High Performance Computing Resources

Significant effort is placed on tuning the internal parameters of fuzzers to explore the state space, measured as coverage, of binaries. In this work, we investigate the effects of the external environment on the resulting coverage after fuzzing two binaries with AFL for 24 hours. Parameters such as scaling to multiple nodes, node saturation, and parallel file system type on HPC resources are controlled in order to maximize coverage. It will be shown that employing a parallel file system such as IBM's General Parallel File System offers an advantage for fuzzing operations, since it contains enhancements for performance optimization. When combined with scaling to two and four nodes, while simultaneously restricting the number of coordinated AFL tasks per node on the low end (10-50% of available physical cores), coverage may be enhanced within a shorter period of time. Thus, controlling the external environment is a useful effort.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

GentenMPI: Distributed Memory Sparse Tensor Decomposition

GentenMPl is a toolkit of sparse canonical polyadic (CP) tensor decomposition algorithms that is designed to run effectively on distributed-memory high-performance computers. Its use of distributed-memory parallelism enables it to efficiently decompose tensors that are too large for a single compute node's memory. GentenMPl leverages Sandia's decades-long investment in the Trilinos solver framework for much of its parallel-computation capability. Trilinos contains numerical algorithms and linear algebra classes that have been optimized for parallel simulation of complex physical phenomena. This work applies these tools to the data science problem of sparse tensor decomposition. In this report, we describe the use of Trilinos in GentenMPl, extensions needed for sparse tensor decomposition, and implementations of the CP-ALS (CP via alternating least squares) and GCP-SGD (generalized CP via stochastic gradient descent) sparse tensor decomposition algorithms. We show that GentenMPl can decompose sparse tensors of extreme size, e.g., a 12.6-terabyte tensor on 8192 computer cores. We demonstrate that the Trilinos backbone provides good strong and weak scaling of the tensor decomposition algorithms.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

The Kokkos Ecosystem [Brief]

In 2016/2017, the field of High-Performance Computing (HPC) entered a new era driven by fundamental physics challenges to produce ever more energy and cost-efficient processors. Since the convergence on the Message-Passing Interface (MPI) standard in the mid-1990s, application developers enjoyed a seemingly static view of the underlying machine — that of a distributed collection of homogeneous nodes executing in collaboration. However, after almost two decades of dominance, the sole use of MPI to derive parallelism acted as a limiter to improved future performance. While MPI is widely expected to continue to function as the basic mechanism for communication between compute nodes for the immediate future, additional parallelism is required on the computing node itself if high performance and efficiency goals are to be realized. When reviewing the architectures of the top HPC systems today, the change in paradigm is clear: the compute nodes of the leading machines in the world are either powered by many-core chips with a few dozen cores each, or use heterogeneous designs, where traditional CPUs marshal work to massively parallel compute accelerators which has as many as 200,000 processing threads in flight simultaneously. Complicating matters further for application developers, each processor vendor has its own preferred way of writing code for their architecture.The Kokkos EcoSystem was released by Sandia in 2017 to address this new era in HPC system design by providing a vendor independent performance portable programming system for scientific, engineering, and mathematical software applications written in the C++ programming language. Using Kokkos, application developers can be more productive because they will not have to create and maintain separate versions of their software for each architecture, nor will they have to be experts in each architecture's peculiar requirements. Instead, they will have a single method of programming for the diverse set of modern HPC architectures. While Kokkos started in 2011 as a programming model only, it soon became clear that complex applications needed more. It is also critical to have a portable mathematical functions and developers need tools to debug their applications, gain insight into the performance characteristics of their codes and tune algorithm performance parameters through automated processes. The Kokkos EcoSystem addresses those needs through its three main components: the Kokkos Core programming model, the Kokkos Kernels math library, and the Kokkos Tools project.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

SST-GPU: A Scalable SST GPU Component for Performance Modeling and Profiling

Programmable accelerators have become commonplace in modern computing systems. Advances in programming models and the availability of unprecedented amounts of data have created a space for massively parallel accelerators capable of maintaining context for thousands of concurrent threads resident on-chip. These threads are grouped and interleaved on a cycle-by-cycle basis among several massively parallel computing cores. One path for the design of future supercomputers relies on an ability to model the performance of these massively parallel cores at scale. The SST framework has been proven to scale up to run simulations containing tens of thousands of nodes. A previous report described the initial integration of the open-source, execution-driven GPU simulator, GPGPU-Sim, into the SST framework. This report discusses the results of the integration and how to use the new GPU component in SST. It also provides examples of what it can be used to analyze and a correlation study showing how closely the execution matches that of a Nvidia V100 GPU when running kernels and mini-apps.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Charon User Manual (V.2.1) (Rev.01)

This manual gives usage information for the Charon semiconductor device simulator. Charon was developed to meet the modeling needs of Sandia National Laboratories and to improve on the capabilities of the commercial TCAD simulators; in particular, the additional capabilities are running very large simulations on parallel computers and modeling displacement damage and other radiation effects in significant detail. The parallel capabilities are based around the MPI interface which allows the code to be ported to a large number of parallel systems, including linux clusters and proprietary "big iron" systems found at the national laboratories and in large industrial settings.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2021

Assessment of Model Hub Height Wind Speed Performance Using DOE Lidar Buoy Data

As the offshore wind industry continues to gain momentum both in the U.S. and globally, reanalysis products remain essential tools for the lifecycle of a wind farm. From project siting and production estimates to determining construction and maintenance windows, reanalysis products play an important role. These regional and global long-term models also serve as inputs to higher resolution models. With such wide-ranging impacts on the offshore wind industry, the need for validation of reanalysis products is strong, particularly at typical turbine hub heights. Pacific Northwest National Laboratory (PNNL) operates two AXYS WindSentinelTM buoys for the U.S. Department of Energy (DOE) in order to collect meteorological and oceanographic data, including hub height wind speed, in areas of interest for offshore wind development. This work uses DOE buoy observations off the coasts of New Jersey and Virginia to evaluate the performance of Modern Era Retrospective Analysis for Research and Applications-2 (MERRA-2), the North American Regional Reanalysis (NARR), the European Center for Medium-Range Weather Forecasts (ECMWF) Reanalysis 5th generation (ERA5), and the National Oceanic and Atmospheric Administration (NOAA) assimilation system Rapid Refresh (RAP). Biases and degrees of correspondence are determined for each reanalysis product in order to provide insights on the performance and uncertainty for long-term wind resource characterization. With the baseline metrics determined, this work proceeds with an investigation into the sources of large deviations between the modeled and observed hub height wind speeds during the East Coast lidar buoy deployments. Consistent sources of reanalysis model error include stably stratified flow conditions, high wind shear, flow parallel to the coastline, and tropical and winter storms. For the near shore New Jersey location, offshore winds (i.e. blowing from land to water) tend to be associated with model overestimation of observed hub height wind speed, while winds parallel to the coastline are correlated with model underestimation of wind speed. Similarly, near the Virginia buoy, during southerly winds (parallel to the coastline) models underestimate the observed wind speed due to coastal upwelling. Reanalysis model wind speeds underestimate the hub height winds with increasing stable atmospheric stratification. Reanalysis model biases are seen when the observed wind speeds approach or exceed typical turbine cut-out speeds (around 20 m s-1). Large model errors during winter and tropical storm events, high wind speed events, and ramp events are often the result of wind speed timing and magnitude offsets. Overall, ERA5 provides the most successful representation of observed offshore hub height wind speeds at the U.S. East Coast buoy locations and is therefore best suited as an input boundary condition for model case studies. ERA5 performs well for a variety of atmospheric phenomena, including storms and sea breezes, however RAP, the next most successful reanalysis model, performs best in capturing the frequency of ramp events.

17 WIND ENERGY↗
DOE OSTI2022

Sierra/SolidMechanics 5.6 User's Manual

Sierra/SolidMechanics (Sierra/SM) is a three-dimensional solid mechanics code with a versatile element library, nonlinear material models, large deformation capabilities, and contact. It is built on the SIERRA Framework. SIERRA provides a data management framework in a parallel computing environment that allows the addition of capabilities in a modular fashion. Contact capabilities are parallel and scalable. This document provides information about the functionality in Sierra/SM and the command structure required to access this functionality in a user input file. This document is divided into chapters based primarily on functionality. For example, the command structure related to the use of various element types is grouped in one chapter; descriptions of material models are grouped in another chapter. Sierra/SM provides both explicit transient dynamics and implicit quasistatics and dynamics capabilities. Both the explicit and implicit modules are highly scalable in a parallel computing environment. In the past, the explicit and implicit capabilities were provided by two separate codes, known as Presto and Adagio, respectively. These capabilities have been consolidated into a single code. The executable is named Adagio, but it provides the full suite of solid mechanics capabilities, for both implicit and explicit. The Presto executable has been disabled as a consequence of this consolidation.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Charon User Manual: v. 2.2 (revision1)

This manual gives usage information for the Charon semiconductor device simulator. Charon was developed to meet the modeling needs of Sandia National Laboratories and to improve on the capabilities of the commercial TCAD simulators; in particular, the additional capabilities are running very large simulations on parallel computers and modeling displacement damage and other radiation effects in significant detail. The parallel capabilities are based around the MPI interface which allows the code to be ported to a large number of parallel systems, including linux clusters and proprietary “big iron” systems found at the national laboratories and in large industrial settings.

42 ENGINEERING↗
DOE OSTI2022

The Portals 4.3 Network Programming Interface

This report presents a specification for the Portals 4 network programming interface. Portals 4 is intended to allow scalable, high-performance network communication between nodes of a parallel computing system. Portals 4 is well suited to massively parallel processing and embedded systems. Portals 4 represents an adaption of the data movement layer developed for massively parallel processing platforms, such as the 4500-node Intel TeraFLOPS machine. Sandia's Cplant cluster project motivated the development of Version 3.0, which was later extended to Version 3.3 as part of the Cray Red Storm machine and XT line. Version 4 is targeted to the next generation of machines employing advanced network interface architectures that support enhanced offload capabilities.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

An Investigation of using FleCSI for Monte Carlo Radiation Transport

This document details the attempt to use FleCSI to provide MPI parallelization and domain decomposition for a Monte Carlo radiation transport application. FleCSI is a framework designed to support multi-physics application development with a focus on task-based parallelism and domain decomposition [1]. FleCSI also supports performance portability via a wrapper around a back-end performance portability layer. The goal of this work is to use FleCSI for domain decomposition and MPI parallelization inside of a Monte Carlo radiation transport application and document any pain points and shortcomings. The following section introduce the nomenclature used in FleCSI and its potential benefit for physics code developers, detail the code used to explore using FleCSI in a Monte Carlo radiation transport solver, and list the issues and concerns discovered during the work.

36 MATERIALS SCIENCE↗
DOE OSTI2024

SPARC-X: Quantum simulations at extreme scale - reactive dynamics from first principles

We have developed the massively parallel electronic structure code SPARC-X: a computational framework for performing Kohn-Sham Density Functional Theory (DFT) calculations that can scale linearly with the number of atoms in the system, while being able to leverage petascale and emerging exascale parallel computers to study chemical phenomena at unprecedented length and time scales. SPARC-X exploits a recent breakthrough in electronic structure methodologies: systematically improvable, strictly local, orthonormal, discontinuous real-space bases that efficiently and systematically capture the local chemistry of the system. With further adaptation using new machine-learning techniques and the use of the massively parallel Spectral Quadrature (SQ) electronic structure method, the algorithmic complexity and prefactor associated with DFT calculations involving semilocal as well as hybrid functionals are dramatically reduced. Using petascale computational resources, SPARC-X enables quantum mechanical simulations at length and time scales previously accessible only by empirical approaches, e.g., 1,000,000 atoms for a few picoseconds using semilocal functionals or 1,000 atoms for a few picoseconds using hybrid functionals. Using exascale resources, the sizes and times targeted are two orders of magnitude larger. Such a capability has applications in a wide variety of chemical sciences, including reactive interfaces where large length- and/or long time-scales are needed and traditional force fields fail. This is particularly important in dynamic catalysis, where bond breaking and formation must be understood in detail. We developed, tested, and employed the SPARC-X framework to understand the photocatalytic properties of TiO 2 nanoparticles, revealing finite size effects that cannot be captured with standard model systems or functionals. This integrated development and application strategy ensures that SPARC-X remains a robust, efficient, and scalable software package for quantum simulations on current petascale and emerging exascale computing resources.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Intelligently Partitioned Phasor-EMT Hybrid Simulations of Large-Scale, High-IBR Power Systems

As the penetration level of power electronics-interfaced renewables such as photovoltaics (PV) and wind has surged in modern electric grids, new operational risks caused by the dynamics of those inverter-based resources (IBRs) are emerging in parallel. Lessons learned from various grid events include that the impact of IBRs on system-level grid stability will become prominent along with the increase of renewables and that the short-timescale dynamic impacts of IBRs on grid stability are not fully captured by current commercial dynamic simulation tools [1] [2]. For example, IBRs can be controlled to mitigate those destabilizing interactions, but conventional phasor-domain tools (e.g. PSS/E, PSLF) often cannot capture that; likewise, the existing electromagnetic transient (EMT) simulation tools (e.g. PSCAD, EMTP) can simulate detailed IBR controls, but for large power systems with many IBRs, slow simulation speeds severely impede the ability to study dynamic events [3] [4]. Massively paralleling simulations using high-performance computing (HPC) can help address this, especially now that cloud-based HPC capability is widely available, but today s EMT tools are not HPC-compatible, and parallelization of dynamic simulation solvers is not trivial because each region can dynamically affect the others. Thus, dynamic simulation of grids with very large numbers of IBRs potentially poses a barrier to the ongoing energy transition.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2026

Calculations Document

Figure 1 shows a simplified model schematic of the Tetracage target design. In this figure the green lines represent the Tetracage yarns, which consist of two parallel yarns on the top and two parallel yarns on the bottom with the pairs in a cross hatched pattern. For simplicity, as we are only considering vertical forces, we will perform calculations without consideration for the two pairs of yarns being perpendicular to each other. Also worth noting is the model shows the yarns extending directly to the capsule then closely hugging it – in reality, the yarns behave as you would expect where they hug the capsule only as much as the geometry of their deflection allows, similarly to if you held a tennis ball with a pair of parallel strings anchored at opposing ends. The dashed black lines show the path the yarns take straight across from slot glue point to slot glue point before the target is closed and the capsule deforms them. The variable r corresponds to the capsule outer radius. The variable L corresponds to the hohlraum inner radius. The variable a corresponds to the yarn offset from the center of the diagnostic band. The variable d corresponds to the wire deflection vertically.

42 ENGINEERING↗