Workload-adaptive Scheduling for Efficient Use of Parallel File Systems in High-performance Computing Clusters
Explore the source record for details and available documents.
Goponenko, Alexander V.↗
SEARCH · Engineering Papers
Results for “parallelization”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Unorthodox Parallelization for Bayesian Quantum State Estimation
Bayesian inference enables informationally efficient quantum state tomography (QST) yet is challenging to scale computationally. We demonstrate a parallelizable Bayesian QST method that, although unorthodox, proves remarkably practical, attaining significant speedups in multiqubit state estimation.
Nguyen, Hanson H. [Arizona State University] (ORCI↗
Determining Levels of Detail for Simulators of Parallel and Distributed Computing Systems via Automated Calibration
Explore the source record for details and available documents.
Mathematics and Computing↗
PARALLEL FLEXURE-BASED RADSI INSTRUMENT FOR CURVED X-RAY MIRROR METROLOGY
Explore the source record for details and available documents.
36 MATERIALS SCIENCE↗
Scheduling and Performance of Asynchronous Tasks in Fortran 2018 with FEATS
Most parallel scientific programs contain compiler directives (pragmas) such as those from OpenMP (Hermanns in Parallel programming in Fortran 95 using openMP, 2002. School of Aeronautical Engineering, Universidad Politécnica de Madrid, España, 2011), explicit calls to runtime library procedures such as those implementing the Message Passing Interface (MPI) (in A message-passing interface standard version 4.0, 2021. https://www.mpi-forum.org/docs/mpi-4.0/mpi40-report.pdf), or compiler-specific language extensions such as those provided by CUDA (Ruetsch and Fatica in CUDA Fortran for scientists and engineers: best practices for efficient CUDA Fortran programming, Elsevier, 2013). By contrast, the recent Fortran standards empower developers to express parallel algorithms without directly referencing lower-level parallel programming models (Numrich in Parallel programming with co-arrays, CRC Press, 2018, and Curcic in Modern Fortran: building efficient parallel applications, Manning Publications, 2020). Fortran’s parallel features place the language within the Partitioned Global Address Space (PGAS) class of programming models. When writing programs that exploit data parallelism, application developers often find it straightforward to develop custom parallel algorithms. Problems involving complex, heterogeneous, staged calculations, however, pose much greater challenges. Such applications require careful coordination of tasks in a manner that respects dependencies prescribed by a directed acyclic graph. When rolling one’s own solution proves difficult, extending a customizable framework becomes attractive. Further, the paper presents the design, implementation, and use of the Framework for Extensible Asynchronous Task Scheduling (FEATS), which we believe to be the first task scheduling tool written in modern Fortran. We describe the benefits and compromises associated with choosing Fortran as the implementation language, and we propose ways in which future Fortran standards can best support the use case in this paper.
97 MATHEMATICS AND COMPUTING↗
PETSc/TAO Users Manual (Rev. 3.19)
This manual describes the use of the Portable, Extensible Toolkit for Scientific Computation (PETSc) and the Toolkit for Advanced Optimization (TAO) for the numerical solution of partial differential equations and related problems on high-performance computers. PETSc/TAO is a suite of data structures and routines that provide the building blocks for the implementation of large-scale application codes on parallel (and serial) computers. PETSc uses the MPI standard for all distributed memory communication. PETSc/TAO includes a large suite of parallel linear solvers, nonlinear solvers, time integrators, and opti mization that may be used in application codes written in Fortran, C, C++, and Python (via petsc4py; see Getting Started). PETSc provides many of the mechanisms needed within parallel application codes, such as parallel matrix and vector assembly routines. The library is organized hierarchically, enabling users to employ the level of abstraction that is most appropriate for a particular problem. By using techniques of object-oriented programming, PETSc provides enormous flexibility for users. PETSc is a sophisticated set of software tools; as such, for some users it initially has a much steeper learning curve than packages such as MATLAB or a simple subroutine library. In particular, for individuals without some computer science background, experience programming in C, C++, python, or Fortran and experience using a debugger such as gdb or lldb, it may require a significant amount of time to take full advantage of the features that enable efficient software use. However, the power of the PETSc design and the algorithms it incorporates may make the efficient implementation of many application codes simpler than “rolling them” yourself. For many tasks a package such as MATLAB is often the best tool; PETSc is not intended for the classes of problems for which effective MATLAB code can be written. There are several packages, built on PETSc, that may satisfy your needs without requiring directly using PETSc. We recommend reviewing these packages functionality before starting to code directly with PETSc. PETSc can be used to provide a “MPI parallel linear solver” in an otherwise sequential, or OpenMP parallel code. This approach cannot provide extremely large improvements in the application time by utilizing large numbers of MPI processes but can still improve the performance. Certainly all parts of a previously sequential code need not be parallelized but the matrix generation portion must be parallelized to expect true scalability to large numbers of MPI processes. See PCMPI for details on how to utilize the PETSc MPI linear solver server. Since PETSc is under continued development, small changes in usage and calling sequences of routines will occur. PETSc has been supported for twenty-five years; see mailing list information on our website for information on contacting support.
97 MATHEMATICS AND COMPUTING↗
Multigrid Reduction in Time for Chaotic and Hyperbolic Problems (Final Report)
The coming massive parallelism of exascale computing presents a pressing challenge for the many DOE simulations of time-dependent partial differential equations (PDEs), which typically use traditional sequential time stepping methods. Since this traditional approach is inherently serial, it presents a sequential bottleneck when moving to exascale computing, because future performance gains will come through greater concurrency, not faster clock speeds. Thus, the goal of this work is to research parallelism in time, i.e., methods that compute multiple time values simultaneously, not sequentially. The focus will be on hyperbolic and chaotic problems of interest to DOE, with the goal of enabling scalable simulations of time-dependent hyperbolic and chaotic problems on future architectures. The chosen methodology for solving these problems parallel-in-time is multigrid, because multigrid (when it works) is a powerful, optimal, and scalable solver for discretized PDEs. Multigrid is already commonly used in many DOE simulations for scalably and optimally solving space-only PDE problems. The areas of hyperbolic and chaotic problems are chosen because of their relevance to problems of programmatic interest to DOE. However, these problems are also well-known to be difficult for parallel-in-time methods, with the most common method, parareal, diverging in many cases. The current state of-the-art for parallel-in-time at LLNL is the multigrid reduction in time (MGRIT) XBraid package, which also struggles for such problems, while still showing some improvement over parareal. In summary, new methods are needed for an efficient parallel-in-time scheme for hyperbolic and chaotic problems, and this work shall research promising new multigrid methods in this area. In particular, this work shall continue researching the directions from the current collaboration with Dr. Falgout, which are laid out in the work Toward Parallel in Time for Chaotic Dynamical Systems and showed the first known results of a parallel-in-time speedup for a chaotic problem. This work outlines two key improvements to XBraid for chaotic problems, the so-called “theta” and “delta-correction” methods. Here, these two improvements will be further researched and improved (including with a new relaxation method inspired by on Least Squares Shadowing (LSS)) and explored for more complicated problems.
97 MATHEMATICS AND COMPUTING↗
T-FSM: A Scalable Distributed Task-Based System for Frequent Subgraph Pattern Mining from a Big Graph
Finding frequent subgraph patterns in a big graph is an important problem with many applications such as classifying chemical compounds and building indexes to speed up graph queries. Since this problem is NP-hard, some recent parallel and distributed systems have been developed to accelerate the mining. However, they often have a huge memory cost, very long running time, suboptimal load balancing, poor scale-out capability, and possibly inaccurate results. In this article, we propose an efficient system called T-FSM for parallel mining of frequent subgraph patterns in a big graph. T-FSM supports a new anti-monotonic frequentness measure called Fraction-Score, which is more accurate than the widely used MNI measure. The execution engine of T-FSM supports both intra-machine parallelism and inter-machine parallelism. For intra-machine parallelism, T-FSM adopts a novel task-based execution model to ensure high multithreading concurrency, bounded memory consumption, and effective load balancing. For inter-machine parallelism, T-FSM ensures good scale-out performance with a lightweight pattern rebalancing approach that reduces workload skewness of pattern evaluations among machines. To avoid recomputing the contexts for migrated patterns, we design a novel context cache table to support concurrent and asynchronous requesting and caching of remote context data, which can timely evict and garbage collect used pattern contexts that are no longer needed to keep memory consumption bounded. Extensive experiments show that T-FSM is orders of magnitude faster than existing state-of-the-art parallel systems (more than 10×, 51×, 131×, 55× speedup over ScaleMine, DistGraph, Pangolin and Peregrine, respectively) and distributed systems (more than 42× and 88× over ScaleMine and DistGraph, respectively) for frequent subgraph pattern mining, and it scales out satisfactorily to 512 CPU cores on the Polaris supercomputer at Argonne National Laboratory.
97 MATHEMATICS AND COMPUTING↗
Intrinsic Toroidal Rotation Driven by Turbulent and Neoclassical Processes in Tokamak Plasmas from Global Gyrokinetic Simulations
Gyrokinetic tokamak plasmas can exhibit intrinsic toroidal rotation driven by the residual stress. While most studies have attributed the residual stress to the parallel-momentum flux from the turbulent E × B motion, the parallel-momentum flux from the drift-orbit motion (denoted $Π^D_\parallel$) and the E × B-momentum flux from the E × B motion (denoted $Π_{E×B}$) are often neglected. Here, we use the global total-f gyrokinetic code XGC to study the residual stress in the core and the edge of a DIII-D H-mode plasma. Numerical results show that both $Π^D_\parallel$ and $Π_{E×B}$ make up a significant portion of the residual stress. In particular, $Π^D_\parallel$ in the core is higher than the collisional neoclassical level in the presence of turbulence, while in the edge it represents an outflux of countercurrent momentum even without turbulence. Using a recently developed “orbit-flux” formulation, we show that the higher-than-neoclassical-level $Π^D_\parallel$ in the core is driven by turbulence, while the outflux of countercurrent momentum from the edge is mainly due to collisional ion orbit loss. In conclusion, these results suggest that $Π^D_\parallel$ and $Π_{E×B}$ can be important for the study of intrinsic toroidal rotation.
70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
Multigrid Reduction in Time for Chaotic and Hyperbolic Problems (Final Report)
The coming massive parallelism of exascale computing presents a pressing challenge for the many DOE simulations of time-dependent partial differential equations (PDEs), which typically use traditional sequential time stepping methods. Since this traditional approach is inherently serial, it presents a sequential bottleneck when moving to exascale computing, because future performance gains will come through greater concurrency, not faster clock speeds. Thus, the goal of this work is to research parallelism in time, i.e., methods that compute multiple time values simultaneously, not sequentially. The focus will be on hyperbolic and chaotic problems of interest to DOE, with the goal of enabling scalable simulations of time-dependent hyperbolic and chaotic problems on future architectures. The chosen methodology for solving these problems parallel-in-time is multigrid, because multigrid (when it works) is a powerful, optimal, and scalable solver for discretized PDEs. Multigrid is already commonly used in many DOE simulations for scalably and optimally solving space-only PDE problems. The areas of hyperbolic and chaotic problems are chosen because of their relevance to problems of programmatic interest to DOE. However, these problems are also well-known to be difficult for parallelin-time methods, with the most common method, parareal, diverging in many cases. The current state-of-the-art for parallel-in-time at LLNL is the multigrid reduction in time (MGRIT) XBraid package, which also struggles for such problems, while still showing some improvement over parareal. In summary, new methods are needed for an efficient parallel-in-time scheme for hyperbolic and chaotic problems, and this work shall research promising new multigrid methods in this area. In particular, this work shall continue researching the directions from the current collaboration with Dr. Falgout, which are laid out in the work Toward Parallel in Time for Chaotic Dynamical Systems and showed the first known results of a parallel-in-time speedup for a chaotic problem. This work outlines two key improvements to XBraid for chaotic problems, the so-called “theta” and “delta-correction” methods. Here, these two improvements will be implemented in a high-performance but general way in XBraid and explored for more complicated problems. We will additionally research, as time allows, improvements to these techniques, as well as multigrid relaxation techniques based on Least Squares Shadowing (LSS by Wang) and a nonintrusive block tridiagonal solver based on MGRIT, called TriMGRIT.
97 MATHEMATICS AND COMPUTING↗
Accelerating the Lagrangian Particle Tracking in Hydrologic Modeling to Continental-Scale
Unprecedented climate change and anthropogenic activities have induced increasing ecohydrological problems, which have motivated the development of large-scale modeling for solutions. Water age/quality is as important as water quantity for understanding the water cycle. However, current scientific progress in tracking water parcels at large-scale with high spatiotemporal resolutions is far behind that in water balance/quantity owing to the lack of powerful tools. EcoSLIM is a particle tracking model that works with the hydrologic model ParFlow-CLM, which couples surface-subsurface hydrology with land surface processes. Here, we demonstrate a parallel framework to accelerate EcoSLIM to continental-scale on a distributed, multi-GPU platform with CUDA-Aware MPI. In tests from catchment-, to regional-, and then to continental-scale using 25-million to 1.6-billion particles, EcoSLIM shows significant speedup and excellent parallel performance. The parallel framework is portable to atmospheric and oceanic particle tracking models, where parallelization is inadequate and a standard parallel framework is absent. Parallelized EcoSLIM is a promising tool to accelerate our understanding of the terrestrial water cycle and the upscaling of subsurface hydrology to Earth system models.
54 ENVIRONMENTAL SCIENCES↗
Screening of resonant magnetic perturbation fields assuming various plasma flow models
A recently updated version of the MARS-F code [Y. Q. Liu et al., Phys. Plasmas 7, 3681 (2000); L. Li et al., Phys. Plasmas 25, 082512 (2018); and G. L. Xia et al., Nucl. Fusion 59, 126035 (2019)] is utilized to numerically investigate the plasma screening effect on the applied resonant magnetic perturbation (RMP) field, assuming various equilibrium flow models, including the toroidal flow, the parallel flow and their combinations, and poloidal and toroidal projections of the parallel flow. A parallel equilibrium flow with a uniform radial profile is found to have no effect on plasma screening of the RMP field. A sheared parallel flow, however, does change plasma screening. The poloidal projection of the parallel flow weakens plasma screening in the resistive-inertial regime. The effect on the favorable average curvature regime is found, however, to be non-monotonic. With the increasing flow speed, the poloidal projection first weakens Glasser-Green-Johnson (GGJ)-screening. Further increase in the flow speed results in enhanced GGJ-screening again. This non-monotonic behavior is related to the perturbed parallel shielding current, which appears also off the mode rational surface at fast flow due to additional resonances between the RMP perturbation and the sound wave continuum. These results indicate that flow induced plasma screening to the RMP field can have complicated characteristics, which, in turn, can have implications on the RMP field penetration into the plasma in experiments for controlling the edge localized modes.
70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
Reducing communication in algebraic multigrid with multi-step node aware communication
Algebraic multigrid (AMG) is often viewed as a scalable [Formula: see text] solver for sparse linear systems. Yet, AMG lacks parallel scalability due to increasingly large costs associated with communication, both in the initial construction of a multigrid hierarchy and in the iterative solve phase. This work introduces a parallel implementation of AMG that reduces the cost of communication, yielding improved parallel scalability. It is common in Message Passing Interface (MPI), particularly in the MPI-everywhere approach, to arrange inter-process communication, so that communication is transported regardless of the location of the send and receive processes. Performance tests show notable differences in the cost of intra- and internode communication, motivating a restructuring of communication. In this case, the communication schedule takes advantage of the less costly intra-node communication, reducing both the number and the size of internode messages. Node-centric communication extends to the range of components in both the setup and solve phase of AMG, yielding an increase in the weak and strong scaling of the entire method.
Computer Science↗
Development of Integrated Education, Training and Research Program with Interdisciplinary Applications
Plasma models have been formulated including realistic spatial profiles of both flow and radio frequency induced ponderomotive force. With these inclusions the picture of stability of various plasma and fluid instabilities, as expected, changed drastically with ground-breaking consequences. The inhomogeneous parallel flow and the radio frequency waves can actually shown to stabilize turbulence. This is different from the prevalent notion that both parallel flow shear and radio frequency waves are responsible for the excitation (destabilization) of plasma turbulence. This has several ground-breaking consequences:- (1) the stabilization by parallel flow clearly goes against the conventional notion of the origin of ionospheric oscillation which invokes parallel flow destabilization as the origin, (2) the stabilization by parallel flow opens us a new avenue for improved mode formation in fusion devices - which mostly rely on the perpendicular flow shear stabilization for improved mode formation but the perpendicular flow is damped in a tokamak - so the improved mode formed by the parallel flow can sustain longer and has more prospect for ignition, (3) the complete stabilization of the ITG mode (and consequent suppression of transport) not only explain many unknown phenomena in the space physics but it also raises a prospect for transport barrier formation by the RF waves but not by the RF induced flow (as most works suggest) which is never observed in a tokamak of that magnitude to create a barrier. These are indeed ground-breaking consequences.
70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
Development of Integrated Education, Training and Research Program with Interdisciplinary Applications II
Plasma models have been formulated including realistic spatial profiles of both flow and radio frequency induced ponderomotive force. With these inclusions the picture of stability of various plasma and fluid instabilities, as expected, changed drastically with ground-breaking consequences. The inhomogeneous parallel flow and the radio frequency waves can actually shown to stabilize turbulence. This is different from the prevalent notion that both parallel flow shear and radio frequency waves are responsible for the excitation (destabilization) of plasma turbulence. This has several ground-breaking consequences:- (1) the stabilization by parallel flow clearly goes against the conventional notion of the origin of ionospheric oscillation which invokes parallel flow destabilization as the origin, (2) the stabilization by parallel flow opens us a new avenue for improved mode formation in fusion devices - which mostly rely on the perpendicular flow shear stabilization for improved mode formation but the perpendicular flow is damped in a tokamak - so the improved mode formed by the parallel flow can sustain longer and has more prospect for ignition, (3) the complete stabilization of the ITG mode (and consequent suppression of transport) not only explain many unknown phenomena in the space physics but it also raises a prospect for transport barrier formation by the RF waves but not by the RF induced flow (as most works suggest) which is never observed in a tokamak of that magnitude to create a barrier. These are indeed ground-breaking consequences.
70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
Framework for Extensible, Asynchronous Task Scheduling (FEATS) in Fortran
Most parallel scientific programs contain compiler directives (pragmas) such as those from OpenMP, explicit calls to runtime library procedures such as those implementing the Message Passing Interface (MPI), or compiler-specific language extensions such as those provided by CUDA. By contrast, the recent Fortran standards empower developers to express parallel algorithms without directly referencing lower-level parallel programming models. Fortran’s parallel features place the language within the Partitioned Global Address Space (PGAS) class of programming models. When writing programs that exploit data-parallelism, application developers often find it straightforward to develop custom parallel algorithms. Problems involving complex, heterogeneous, staged calculations, however, pose much greater challenges. Such applications require careful coordination of tasks in a manner that respects dependencies prescribed by a directed acyclic graph. When rolling one’s own solution proves difficult, extending a customizable framework becomes attractive. The paper presents the design, implementation, and use of the Framework for Extensible Asynchronous Task Scheduling (FEATS), which we believe to be the first task-scheduling tool written in modern Fortran. We describe the benefits and compromises associated with choosing Fortran as the implementation language, and we propose ways in which future Fortran standards can best support the use case in this paper.
Richardson, Brad↗
System with a direct contact desublimating heat exchanger and temperature regulation
A process to prevent fouling using a desublimating heat exchanger is disclosed. An outlet stream from the desublimating heat exchanger may be split into a plurality of parallel streams. The parallel streams may be sent through a plurality of discrete unit operations, and the unit operations may change the temperature of at least one of the parallel streams. Parallel streams of differing temperature may emerge from the unit operations. The parallel streams which are of a similar temperature may be mixed to form a warm stream and a cool stream. The warm stream and the cool stream may be sent to a mixing chamber. A mixed stream of substantially uniform temperature may emerge from the mixing chamber, and the mixed stream may be recycled back to the desublimating heat exchanger. The mixing chamber may be separate from the desublimating heat exchanger, or the parallel streams of differing temperature may be mixed in the desublimating heat exchanger.
42 ENGINEERING↗
Linear delta systems, hexapod systems, and related methods
A linear delta system includes a support frame, rails mounted to the support frame, linear actuators, each linear actuator configured to translate along a longitudinal length of a respective rail, pairs of parallel rods each coupled to the linear actuators, a platform coupled to a longitudinal end of each of the pairs of parallel rods opposite the respective linear actuator, and an object coupled to the platform. Longitudinal axes of the rails are oriented parallel to each other and lie within a common plane or an uncommon plane. A method of forming a linear delta system includes mounting rails to a support frame, the rails having longitudinal axes that are parallel to each other and lying within a common plane, coupling a linear actuator to each of the rails, coupling a pair of parallel rods to each linear actuator, and coupling a platform to the pairs of parallel rods.
Crawford, Anthony L.↗