On Scalable Preconditioners for Implicit Continuum Multiphysics Plasma Systems.
Abstract not provided.
Shadid, John↗
SEARCH · Engineering Papers
Results for “scalable preconditioner”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Randomized Preconditioned Solvers for Strong Constraint 4D-Var Data Assimilation
The Strong Constraint 4D Variational (SC-4DVAR) data assimilation method is widely used in climate and weather applications. SC-4DVAR involves solving a minimization problem to compute the maximum a posteriori estimate, which we tackle using the Gauss-Newton method. The computation of the descent direction is expensive since it involves the solution of a large-scale and potentially ill-conditioned linear system, solved using the preconditioned conjugate gradient (PCG) method. Here, to address this cost, we efficiently construct scalable preconditioners using three different randomization techniques, which all rely on a certain low-rank structure involving the Gauss-Newton Hessian. The proposed techniques come with theoretical guarantees on the condition number, and at the same time, are amenable to parallelization. We also develop an adaptive approach to estimate the sketch size and choose between the reuse or recomputation of the preconditioner. We demonstrate the performance and effectiveness of our methodology on two representative model problems—the Burgers and barotropic vorticity equation—showing a drastic reduction in both the number of PCG iterations and the number of Gauss-Newton Hessian products after including the preconditioner construction cost.
Gauss-Newton↗
Finite Element Method for Electrochemical Transport
This code provides Finite Element solvers for Electrochemical Transport. A few examples from the literature are reproduced, with a focus on CO2 electrolysis. The Discontinuous Galerkin scheme for the electroneutral Nernst-Planck equations is from Roy, T., Andrej, J. and Beck, V.A., 2021. A scalable DG solver for the electroneutral Nernst-Planck equations. arXiv preprint arXiv:2112.09271. This work also includes a scalable preconditioner.
Beck, VictorA↗
NEAMS Technical Area Support in MOOSE
The MOOSE framework is a foundational capability used by the NEAMS program to create over 15 different simulation tools for advanced nuclear reactors. Due to this ubiquity, improvements to the framework in support of modeling and simulation goals are critical to the program. These improvements can take many forms including optimization, improved user experience, streamlined application programming interfaces (APIs), parallelism, and other new capabilities. The work transcribed in this report was conducted in direct support of the simulation tools and has already been deployed. The capabilities outlined in this report include enabling selective polynomial basis refinement, implementing a custom convergence system, building a scalable preconditioner for saddle-point problems, and much more.
97 MATHEMATICS AND COMPUTING↗
FROSch Preconditioners for Land Ice Simulations of Greenland and Antarctica
Numerical simulations of Greenland and Antarctic ice sheets involve the solution of large-scale highly nonlinear systems of equations on complex shallow geometries. This work is concerned with the construction of Schwarz preconditioners for the solution of the associated tangent problems, which are challenging for solvers mainly because of the strong anisotropy of the meshes and wildly changing boundary conditions that can lead to poorly constrained problems on large portions of the domain. Here, two-level GDSW (Generalized Dryja–Smith–Widlund) type Schwarz preconditioners are applied to different land ice problems, i.e., a velocity problem, a temperature problem, as well as the coupling of the former two problems. We employ the MPI-parallel implementation of multi-level Schwarz preconditioners provided by the package FROSch (Fast and Robust Schwarz)from the Trilinos library. The strength of the proposed preconditioner is that it yields out-of-the-box scalable and robust preconditioners for the single physics problems. To our knowledge, this is the first time two-level Schwarz preconditioners are applied to the ice sheet problem and a scalable preconditioner has been used for the coupled problem. The pre-conditioner for the coupled problem differs from previous monolithic GDSW preconditioners in the sense that decoupled extension operators are used to compute the values in the interior of the sub-domains. Several approaches for improving the performance, such as reuse strategies and shared memory OpenMP parallelization, are explored as well. In our numerical study we target both uniform meshes of varying resolution for the Antarctic ice sheet as well as non uniform meshes for the Greenland ice sheet are considered. We present several weak and strong scaling studies confirming the robustness of the approach and the parallel scalability of the FROSch implementation. Among the highlights of the numerical results are a weak scaling study for up to 32 K processor cores (8 K MPI-ranks and 4 OpenMP threads) and 566 M degrees of freedom for the velocity problem as well as a strong scaling study for up to 4 K processor cores (and MPI-ranks) and 68 M degrees of freedom for the coupled problem.
58 GEOSCIENCES↗
FROSch Preconditioners for Land Ice Simulations of Greenland and Antarctica
Numerical simulations of Greenland and Antarctic ice sheets involve the solution of large-scale highly nonlinear systems of equations on complex shallow geometries. This work is concerned with the construction of Schwarz preconditioners for the solution of the associated tangent problems, which are challenging for solvers mainly because of the strong anisotropy of the meshes and wildly changing boundary conditions that can lead to poorly constrained problems on large portions of the domain. In this study, two-level generalized Dryja--Smith--Widlund (GDSW)--type Schwarz preconditioners are applied to different land ice problems, i.e., a velocity problem, a temperature problem, as well as the coupling of the former two problems. We employ the message passing interface (MPI)--parallel implementation of multilevel Schwarz preconditioners provided by the package FROSch (fast and robust Schwarz) from the Trilinos library. The strength of the proposed preconditioner is that it yields out-of-the-box scalable and robust preconditioners for the single physics problems. To the best of our knowledge, this is the first time two-level Schwarz preconditioners have been applied to the ice sheet problem and a scalable preconditioner has been used for the coupled problem. The preconditioner for the coupled problem differs from previous monolithic GDSW preconditioners in the sense that decoupled extension operators are used to compute the values in the interior of the subdomains. Additionally, several approaches for improving the performance, such as reuse strategies and shared memory OpenMP parallelization, are explored. In our numerical study we target both uniform meshes of varying resolution for the Antarctic ice sheet as well as nonuniform meshes for the Greenland ice sheet. We present several weak and strong scaling studies confirming the robustness of the approach and the parallel scalability of the FROSch implementation. Among the highlights of the numerical results are a weak scaling study for up to 32,K processor cores (8,K MPI ranks and 4 OpenMP threads) and 566,M degrees of freedom for the velocity problem as well as a strong scaling study for up to 4,K processor cores (and MPI ranks) and 68,M degrees of freedom for the coupled problem.
97 MATHEMATICS AND COMPUTING↗
Multi-physics Preconditioning for Thermally Activated Batteries
Thermal batteries, also known as molten-salt batteries, are single-use reserve power systems activated by pyrotechnic heat generation, which transitions the solid electrolyte into a molten state. The simulation of these batteries relies on multiphysics modeling to evaluate performance and behavior under various conditions. This paper presents advancements in scalable preconditioning strategies for the Thermally Activated Battery Simulator (TABS) tool, enabling efficient solutions to the coupled electrochemical systems that dominate computational costs in thermal battery simulations. We propose a hierarchical block Gauss-Seidel preconditioner implemented through the Teko package in Trilinos, which effectively addresses the challenges posed by tightly coupled physics, including charge transport, porous flow, and species diffusion. The preconditioner leverages scalable subblock solvers, including smoothed aggregation algebraic multigrid (SA-AMG) methods and domain-decomposition techniques, to achieve robust convergence and parallel scalability. Strong and weak scaling studies demonstrate the solver’s ability to handle problem sizes up to 51.3 million degrees of freedom on 2048 processors, achieving near sub-second setup and solve times for the end-to-end electrochemical solve. These advancements significantly improve the computational efficiency and turnaround time of thermal battery simulations, paving the way for higher-resolution models and enabling the transition from 2D axisymmetric to full 3D simulations.
25 ENERGY STORAGE↗
Scalable DPG multigrid solver for Helmholtz problems: A study on convergence
This paper presents a scalable multigrid preconditioner targeting large-scale systems arising from discontinuous Petrov–Galerkin (DPG) discretizations of high-frequency wave operators. This work is built on previously developed multigrid preconditioning techniques of Petrides and Demkowicz (Comput. Math. Appl. 87 (2021) pp. 12–26) and extends the convergence results from $\mathscr{O}$(10 7 ) degrees of freedom (DOFs) to $\mathscr{O}$(10 9 ) DOFs using a new scalable parallel MPI/OpenMP implementation. Novel contributions of this paper include an alternative definition of coarse-grid systems based on restriction of fine-grid operators, yielding superior convergence results. In the uniform refinement setting, a detailed convergence study is provided, demonstrating h and p robust convergence and linear scaling with respect to the wave frequency. Finally, the paper concludes with numerical results on hp -adaptive simulations including a large-scale seismic modeling benchmark problem with high material contrast.
97 MATHEMATICS AND COMPUTING↗
Enhancing scalability of a matrix-free eigensolver for studying many-body localization
We propose several techniques to enhance the parallel scalability of a matrix-free eigensolver designed for studying many-body localization (MBL) of quantum spin chain models with nearest-neighbor interactions and on-site disorder. This type of problem is computationally challenging because the dimension of the associated Hamiltonian matrix grows exponentially with respect to the number of spins L, and we need to average over different realizations of the random disorder to obtain relevant statistical behavior. For each disorder realization, we need to compute eigenvalues from different regions of the spectrum and their corresponding eigenvectors. In previous work, the interior eigenstates for a single eigenvalue problem are computed via the shift-and-invert Lanczos algorithm. Due to the extremely high memory footprint of the LU factorizations, this technique is not well suited for large L’s. For example, we need thousands of compute nodes on modern high performance computing infrastructures to go beyond L = 24. The matrix-free approach does not suffer from this memory bottleneck, however, its scalability is limited by a computation and communication load imbalance. To reduce this imbalance and to significantly enhance the scalability of the matrix-free eigensolver, we reorder the matrix and leverage the consistent space runtime, CSPACER. We also show its efficiency in managing irregular communication patterns at scale compared to optimized MPI non-blocking two-sided and one-sided RMA implementation variants. This effort enables us to study MBL for spin chains with a larger number of spins. The efficiency and effectiveness of the proposed algorithm is demonstrated by computing eigenstates on a massively parallel many-core high performance computer.
METIS↗
Parallel performance of algebraic multigrid domain decomposition
Algebraic multigrid (AMG) is a widely used scalable solver and preconditioner for large-scale linear systems resulting from the discretization of a wide class of elliptic PDEs. While AMG has optimal computational complexity, the cost of communication has become a significant bottleneck that limits its scalability as processor counts continue to grow on modern machines. This article examines the design, implementation, and parallel performance of a novel algorithm, algebraic multigrid domain decomposition (AMG-DD), designed specifically to limit communication. The goal of AMG-DD is to provide a low-communication alternative to standard AMG V-cycles by trading some additional computational overhead for a significant reduction in communication cost. Numerical results show that AMG-DD achieves superior accuracy per communication cost compared with AMG, and speedup over AMG is demonstrated on a large GPU cluster.
97 MATHEMATICS AND COMPUTING↗
Algebraic Nonoverlapping Domain Decomposition Methods for Stabilized FEM and FV Discretizations
We consider preconditioning methods for convection dominated fluid flow problems based on a nonoverlapping Schur complement domain decomposition procedure for arbitrary triangulated domains. The triangulation is first partitioned into a number of subdomains and interfaces which induce a natural 2 x 2 partitioning of the p.d.e. discretization matrix. We view the Schur complement induced by this partitioning as an algebraically derived coarse space approximation. This avoids the known difficulties associated with the direct formation of an effective coarse discretization for advection dominated equations. By considering various approximations of the block factorization of the 2 x 2 system, we have developed a family of robust preconditioning techniques. A computer code based on these ideas has been developed and tested on the IBM SP2 using MPI message passing protocol. A number of 2-D CFD calculations will be presented for both scalar advection-diffusion equations and the Euler equations discretized using stabilized finite element and finite volume methods. These results show very good scalability of the preconditioner for various discretizations as the number of processors is increased while the number of degrees of freedom per processor is fixed.
Barth, Timothy J.↗
CSRI Summer Proceedings 2021
The Computer Science Research Institute (CSRI) brings university faculty and students to Sandia National Laboratories for focused collaborative research on Department of Energy (DOE) computer and computational science problems. The institute provides an opportunity for university researches to learn about problems in computer and computational science at DOE laboratories, and help transfer results of their research to programs at the labs. Some specific CSRI research interest areas are: scalable solvers, optimization, algebraic preconditioners, graph-based, discrete, and combinatorial algorithms, uncertainty estimation, validation and verification methods, mesh generation, dynamic load-balancing, virus and other malicious-code defense, visualization, scalable cluster computers, beyond Moore’s Law computing, exascale computing tools and application design, reduced order and multiscale modeling, parallel input/output, and theoretical computer science. The CSRI Summer Program is organized by CSRI and includes a weekly seminar series and the publication of a summer proceedings.
97 MATHEMATICS AND COMPUTING↗
CSRI Summer Proceedings 2021
The Computer Science Research Institute (CSRI) brings university faculty and students to Sandia National Laboratories for focused collaborative research on Department of Energy (DOE) computer and computational science problems. The institute provides an opportunity for university researches to learn about problems in computer and computational science at DOE laboratories, and help transfer results of their research to programs at the labs. Some specific CSRI research interest areas are: scalable solvers, optimization, algebraic preconditioners, graph-based, discrete, and combinatorial algorithms, uncertainty estimation, validation and verification methods, mesh generation, dynamic load-balancing, virus and other malicious-code defense, visualization, scalable cluster computers, beyond Moore’s Law computing, exascale computing tools and application design, reduced order and multiscale modeling, parallel input/output, and theoretical computer science. The CSRI Summer Program is organized by CSRI and includes a weekly seminar series and the publication of a summer proceedings.
97 MATHEMATICS AND COMPUTING↗
Low-Order Preconditioning for the High-Order Finite Element de Rham Complex
Here, we present a unified framework for constructing spectrally equivalent low-order-refined discretizations for the high-order finite element de Rham complex. This theory covers diffusion problems in H 1 , H(curl), and H(div) and is based on combining a low-order discretization posed on a refined mesh with a high-order basis for Nédélec and Raviart–Thomas elements that makes use of the concept of polynomial histopolation (polynomial fitting using prescribed mean values over certain regions). This spectral equivalence, coupled with algebraic multigrid methods constructed using the low-order discretization, results in highly scalable matrix-free preconditioners for high-order finite element problems in the full de Rham complex. Additionally, a new lowest-order (piecewise constant) preconditioner is developed for high-order interior penalty discontinuous Galerkin (DG) discretizations, for which spectral equivalence results and convergence proofs for algebraic multigrid methods are provided. In all cases, the spectral equivalence results are independent of polynomial degree and mesh size; for DG methods, they are also independent of the penalty parameter. These new solvers are flexible and easy to use; any “black-box” preconditioner for low-order problems can be used to create an effective and efficient preconditioner for the corresponding high-order problem. A number of numerical experiments are presented, based on an implementation in the finite element library MFEM. A range of challenging three-dimensional problems are used to corroborate the theoretical properties and demonstrate the flexibility and scalability of the method.
97 MATHEMATICS AND COMPUTING↗
Scalable Multiphysics Block Preconditioning for Low Mach Number Compressible Resistive MHD with Application to Magnetic Confinement Fusion
This study investigates multiphysics block preconditioners that are critical in devising scalable Newton–Krylov iterative solvers for longer time-scale fully implicit fluid plasma models. The specific model of interest is the visco-resistive, low Mach number, compressible magnetohydrodynamics (MHD) model. This model describes the dynamics of conducting fluids in the presence of electromagnetic fields and can be used to study aspects of astrophysical phenomena, important science and technology applications, and basic plasma physics. The specific application of interest that motivates this study is the macroscopic simulation of longer time-scale stability and disruptions of magnetic confinement fusion devices, specifically the ITER Tokamak. The computational solution of the governing balance equations for mass, momentum, heat transfer, and magnetic induction for resistive MHD systems can be extremely challenging. These difficulties arise from both the strong nonlinear, nonsymmetric coupling of fluid and electromagnetic phenomena as well as the significant range of time and length scales that the interactions of these physical mechanisms produce. To handle the range of time and spatial scales of interest, a fully implicit unstructured variational multiscale finite element formulation is employed. For the scalable solution of the Newton linearized systems, fully coupled block preconditioners are designed to leverage algebraic multigrid subsolves. In conclusion, results are presented for the strong and weak scaling of the method as well as the robustness of these techniques for a large range of Lundquist numbers.
97 MATHEMATICS AND COMPUTING↗
Diffusion Synthetic Acceleration for Heterogeneous Domains, Compatible with Voids
A standard approach to solving the S N transport equations is to use source iteration with diffusion synthetic acceleration (DSA). Although this approach is widely used and effective on many problems, there remain some practical issues with DSA preconditioning, particularly on highly heterogeneous domains. For large-scale parallel simulation, it is critical that both (a) preconditioned source iteration converges rapidly and (b) the action of the DSA preconditioner can be applied using fast, scalable solvers, such as algebraic multigrid (AMG). For heterogeneous domains, these two interests can be at odds. In particular, there exist DSA diffusion discretizations that can be solved rapidly using AMG, but they do not always yield robust/fast convergence of the larger source iteration. Conversely, there exist robust DSA discretizations where source iteration converges rapidly on difficult heterogeneous problems, but fast parallel solvers like AMG tend to struggle applying the action of such operators. Moreover, very few current methods for the solution of deterministic transport are compatible with voids. This paper develops a new heterogeneous DSA preconditioner based on only preconditioning the optically thick subdomains. The resulting method proves robust on a variety of heterogeneous transport problems, including a linearized hohlraum mesh related to inertial confinement fusion. Moreover, the action of the preconditioner is easily computed using O(1) AMG iterations, convergence of the transport iteration typically requires 2 to 5× fewer iterations than current state-of-the-art “full” DSA, and the proposed method is trivially compatible with voids. On the hohlraum problem, rapid convergence is obtained by preconditioning less than 3% of the mesh elements with five to ten AMG iterations.
73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
Scalable preconditioning for the stabilized contact mechanics problem
We present a family of preconditioning strategies for the contact problem in fractured and faulted porous media. We combine low-order continuous finite elements to simulate the bulk deformation with piecewise constant Lagrange multipliers to impose the frictional contact constraints. This formulation is not uniformly inf-sup stable and requires stabilization. We improve previous work by Franceschini et al. (2020) by introducing a novel jump stabilization technique that requires only local geometrical and mechanical properties. We then design scalable preconditioning strategies that take advantage of the block structure of the Jacobian matrix using a physics-based partitioning of the unknowns by field type, namely displacement and Lagrange multipliers. The key to the success of the proposed preconditioners is a pseudo-Schur complement obtained by eliminating the Lagrange multiplier degrees of freedom, which can then be efficiently solved using an optimal multigrid method. Numerical results, including complex real-world problems, are presented to illustrate theoretical properties, scalability and robustness of the preconditioner. A comparison with other approaches available in the literature is also provided.
58 GEOSCIENCES↗
A Scalable Interior‐Point Gauss–Newton Method for PDE‐Constrained Optimization With Bound Constraints
Here, we present a scalable approach to solve a class of partial differential equation (PDE)‐constrained optimization problems with bound constraints. This approach utilizes a robust full‐space interior‐point (IP)‐Gauss–Newton optimization method. To cope with the poorly‐conditioned IP‐Gauss–Newton saddle‐point linear systems that need to be solved approximately, once per optimization step, we propose two spectrally related preconditioners. These preconditioners leverage the limited informativeness of data in regularized PDE‐constrained optimization problems. A block Gauss–Seidel preconditioner is proposed for the GMRES‐based solution of the IP‐Gauss–Newton linear systems. It is shown, for a large‐class of PDE‐ and bound‐constrained optimization problems, that the spectrum of the block Gauss–Seidel preconditioned IP‐Gauss–Newton matrix is asymptotically independent of discretization and is not impacted by the ill‐conditioning that notoriously plagues interior‐point methods. We exploit symmetry of the IP‐Gauss–Newton linear systems and propose a regularization and log‐barrier Hessian preconditioner for the preconditioned conjugate gradient (PCG)‐based solution of the equivalent IP‐Gauss–Newton–Schur complement linear systems. The eigenvalues of the block Gauss–Seidel preconditioned IP‐Gauss–Newton matrix, that are not equal to one, are identical to the eigenvalues of the regularization and log‐barrier Hessian preconditioned Schur complement matrix. The scalability of the approach is demonstrated on two example problems. The numerical solution of these optimization problems is shown to require a discretization independent number of IP‐Gauss–Newton linear solves. Furthermore, the linear systems are solved in a discretization and IP ill‐conditioning independent number of preconditioned Krylov subspace iterations. The parallel scalability of the preconditioner, achieved via algebraic multigrid component solvers when applicable, and the aforementioned algorithmic scalability permits a parallel scalable means to compute solutions of a large class of PDE‐ and bound‐constrained problems.
PDE-constrained optimization↗