Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “krylov solvers”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

72 records · Page 4
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

A Comparison of Linear Solvers for Resolving Flow in Three-Dimensional Discrete Fracture Networks

We compare various methods for resolving steady flow within three-dimensional discrete fracture networks, including direct methods, Krylov subspace methods with and without preconditioning, and multi-grid methods. We compared the performance of the methods based on compute times and scaling of the solution as a function of the number of grid nodes and log-variance of the hydraulic aperture. The methods are applied to three test cases: (a) variable density of networks with a truncated power-law distribution of fracture lengths, (b) a fixed network composed of monodisperse fracture sizes but varied permeability/aperture heterogeneity, (c) and a network based on field site in Nevada, US. We chose these cases to allow us to study the impact of the mesh size and flow properties, as well as to demonstrate our conclusions on a large-scale, realistic problem (more than 40 million mesh nodes). A direct solution using Cholesky factorization outperformed other methods for every example but was closely followed in performance by some algebraic multigrid (AMG) preconditioned Krylov subspace methods. Among the Krylov methods, conjugate gradients (CG) with an AMG preconditioner performs the best. Generally, Cholesky factorization is recommended, but CG with an AMG preconditioner may be suitable for very large problems beyond 40 million nodes where the entire linear system cannot reside in memory.

58 GEOSCIENCES↗
DOE OSTI2020

Initial Exploration of the Use of Mixed Precision in Iterative Solvers (B639388 Subcontract Quarter I Report)

The first quarter of the project was primarily spent identifying potential projects at the intersection of finite precision analysis, mixed precision computation, and Krylov subspace methods. We summarize our findings in the remainder of the document. Other activities include attending biweekly xSDK meetings as well as contributing material to the technical report and journal versions of the multiprecision landscape paper. The subsequent quarter will be spent selecting a subset of the described projects to focus on, performing initial numerical experiments to evaluate the potential for the use of mixed precision, and developing initial theoretical analysis.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS2013

Design of a Variational Multiscale Method for Turbulent Compressible Flows

A spectral-element framework is presented for the simulation of subsonic compressible high-Reynolds-number flows. The focus of the work is maximizing the efficiency of the computational schemes to enable unsteady simulations with a large number of spatial and temporal degrees of freedom. A collocation scheme is combined with optimized computational kernels to provide a residual evaluation with computational cost independent of order of accuracy up to 16th order. The optimized residual routines are used to develop a low-memory implicit scheme based on a matrix-free Newton-Krylov method. A preconditioner based on the finite-difference diagonalized ADI scheme is developed which maintains the low memory of the matrix-free implicit solver, while providing improved convergence properties. Emphasis on low memory usage throughout the solver development is leveraged to implement a coupled space-time DG solver which may offer further efficiency gains through adaptivity in both space and time.

Design↗
DOE OSTI2023

On a fully-implicit VMS-stabilized FE formulation for low Mach number compressible resistive MHD with application to MCF

This study presents the development and evaluation of a fully-implicit variational multiscale (VMS) stabilized unstructured finite element (FE) formulation for compressible magnetohydrodynamics (MHD) model, at low Mach number regime. The model describes the dynamics of a compressible conducting fluid in the low Mach number limit in the presence of electromagnetic fields and can be used to study aspects of astrophysical phenomena, important science and technology applications, and basic plasma physics phenomena. The specific applications that motivate this study are macroscopic simulations of the longer time-scale stability and disruptions of magnetic confinement fusion (MCF) devices, specifically the ITER tokamak. The discussion considers the development of the VMS FE representation, the structure of the stabilizing terms that deal with significant convective flows, the stabilization of the nearly incompressible response of the fluid flow, and the stabilization of the constraint that enforces the solenoidal involution on the magnetic field. The nonlinear discretized system is solved with scalable preconditioned Newton–Krylov iterative methods, which employs a multiphysics block preconditioning method based on approximate block factorizations and Schur complements. The study presents an evaluation of the VMS method on a 2D cartesian tearing mode instability, and illustrates the scalability of the solvers on MCF relevant problems. A set of results are also presented for longer time-scale stability and disruptions for the ITER tokamak. These include a vertical displacement event (VDE), and a (1,1) internal kink mode. Here, the formulation is demonstrated to be scalable and also reasonably robust with respect to the Lundquist number scaling.

42 ENGINEERING↗
NASA NTRS1996

Practical Aspects of Krylov Subspace Iterative Methods in CFD

Implementation issues associated with the application of Krylov subspace iterative methods, such as Newton-GMRES, are presented within the framework of practical computational fluid dynamic (CFD) applications. This paper categorizes, evaluates, and contrasts the major ingredients (function evaluations, matrix-vector products, and preconditioners) of Newton-GMRES Krylov subspace methods in terms of their effect on the local linear and global nonlinear convergence, memory requirements, and accuracy. The discussion focuses on Newton-GMRES in both a structured multi-zone incompressible Navier-Stokes solver and an unstructured mesh finite-volume Navier-Stokes solver. Approximate versus exact matrix-vector products, effective preconditioners, and other pertinent issues are addressed.

Pulliam, Thomas H.↗
DOE OSTI2020

Compare linear-system solver and preconditioner stacks with emphasis on GPU performance and propose phase-2 NGP solver development pathway

The goal of the ExaWind project is to enable predictive simulations of wind farms comprised of many megawatt-scale turbines situated in complex terrain. Predictive simulations will require computational fluid dynamics (CFD) simulations for which the mesh resolves the geometry of the turbines and captures the rotation and large deflections of blades. Whereas such simulations for a single turbine are arguably petascale class, multi-turbine wind farm simulations will require exascale-class resources. The primary physics codes in the ExaWind project are Nalu-Wind, which is an unstructured-grid solver for the acoustically incompressible Navier-Stokes equations, and OpenFAST, which is a whole-turbine simulation code. The Nalu-Wind model consists of the mass-continuity Poisson-type equation for pressure and a momentum equation for the velocity. For such modeling approaches, simulation times are dominated by linear-system setup and solution for the continuity and momentum systems. For the ExaWind challenge problem, the moving meshes greatly affect overall solver costs as reinitialization of matrices and recomputation of preconditioners is required at every time step. In this report we evaluated GPU-performance baselines for the linear solvers in the Trilinos and hypre solver stacks using two representative Nalu-Wind simulations: an atmospheric boundary layer precursor simulation on a structured mesh, and a fixed-wing simulation using unstructured overset meshes. Both strong-scaling and weak-scaling experiments were conducted on the OLCF supercomputer Summit and similar proxy clusters. We focused on the performance of multi-threaded Gauss-Seidel and two-stage Gauss-Seidel that are extensions of classical Gauss-Seidel; of one-reduce GMRES, a communication-reducing variant of the Krylov GMRES; and algebraic multigrid methods that incorporate the afore-mentioned methods. The team has established that AMG methods are capable of solving linear systems arising from the fixed-wing overset meshes on CPU, a critical intermediate result for ExaWind FY20 Q3 and Q4 milestones. For the fixed-wing strong-scaling study (model with 3M grid-points), the team identified that Nalu-Wind simulations with the new Trilinos and hypre solvers scale to modest GPU counts, maintaining above 70% efficiency up to 6 GPUs. However, there still remain significant bottlenecks to performance: matrix assembly (hypre), AMG setup (hypre and Trilinos) In the weak-scaling experiments (going from 0.4M to 211M gridpoints), it's shown that the solver apply phases are faster on GPUs, but that Nalu-Wind simulation times grow, primarily due to the multigrid-setup process. Finally, based on the report outcomes, we propose a linear solver path-forward for the remainder of the ExaWind project. Near term, the NREL team will continue their work on GPU-based linear-system assembly. They will also investigate how the use of alternatives to the NVIDIA UVM (unified virtual memory) paradigm affects performance. Longer term, the NREL team will evaluate algorithmic performance on other types of accelerators and merge their improvements back to the main hypre repository branch. Near term, the Trilinos team will address performance bottlenecks identified in this milestone, such as implementing a GPU-based segregated momentum solve and reusing matrix graphs across linear-system assembly phases. Longer term, the Trilinos team will do detailed analysis and optimization of multigrid setup.

17 WIND ENERGY↗
DOE OSTI2024

Development of Segregated Thermal-Hydraulics Solvers in MOOSE

The simulation of fluid flows is an essential part of the design and analysis of nuclear systems. Algorithms able to simulate flows at different fidelity levels are available in the Multiphysics Object-Oriented Simulation Environment (MOOSE) and MOOSE-based applications such as Pronghorn \cite{novak2018pronghorn}, Pronghorn-Subchannel, RELAP-7, and SAM. Currently, significant effort is being invested in the development of coarse-mesh Computational Fluid Dynamics (CFD) capabilities within MOOSE and Pronghorn for the simulation of Generation IV nuclear reactors. Traditionally, the solution algorithms in MOOSE have relied on Newton or quasi-Newton methods (such as the preconditioned Jacobian-free Newton-Krylov method) where residuals and Jacobians (or approximations thereof) are constructed. Both Newton and quasi-Newton methods require the solution of a linear system at each nonlinear Newton iteration with the Jacobian as the system matrix. The Jacobian contains blocks originating from all variables in the problem (i.e., for thermal-hydraulics at least pressure, velocities, and temperature). Due to the formulation of the problem in a general multiphysics setting on unstructured mesh, creating a good preconditioner for the linear system can be challenging, thus many fluid applications have utilized direct solver-based methods such as LU factorization. However, with increasing system size and complexity in multi-dimensional problems, the direct solution of linear systems becomes computationally expensive both in execution time and and memory. For this reason, recent effort has focused on adapting segregated solution algorithms for CFD problems in MOOSE. These algorithms use fixed-point iteration between segregated systems whose assembly and preconditioning are easier those of the monolithic system. Initial results show that the segregated solution algorithm outperforms the monolithic approach in terms of memory usage and for large 3D problems in terms of CPU time as well.

42 ENGINEERING↗
DOE OSTI2024

Development of Segregated Thermal-Hydraulics Solvers in MOOSE

The simulation of fluid flows is an essential part of the design and analysis of nuclear systems. Algorithms able to simulate flows at different fidelity levels are available in the Multiphysics Object-Oriented Simulation Environment (MOOSE) and MOOSE-based applications such as Pronghorn \cite{novak2018pronghorn}, Pronghorn-Subchannel, RELAP-7, and SAM. Currently, significant effort is being invested in the development of coarse-mesh Computational Fluid Dynamics (CFD) capabilities within MOOSE and Pronghorn for the simulation of Generation IV nuclear reactors. Traditionally, the solution algorithms in MOOSE have relied on Newton or quasi-Newton methods (such as the preconditioned Jacobian-free Newton-Krylov method) where residuals and Jacobians (or approximations thereof) are constructed. Both Newton and quasi-Newton methods require the solution of a linear system at each nonlinear Newton iteration with the Jacobian as the system matrix. The Jacobian contains blocks originating from all variables in the problem (i.e., for thermal-hydraulics at least pressure, velocities, and temperature). Due to the formulation of the problem in a general multiphysics setting on unstructured mesh, creating a good preconditioner for the linear system can be challenging, thus many fluid applications have utilized direct solver-based methods such as LU factorization. However, with increasing system size and complexity in multi-dimensional problems, the direct solution of linear systems becomes computationally expensive both in execution time and and memory. For this reason, recent effort has focused on adapting segregated solution algorithms for CFD problems in MOOSE. These algorithms use fixed-point iteration between segregated systems whose assembly and preconditioning are easier those of the monolithic system. Initial results show that the segregated solution algorithm outperforms the monolithic approach in terms of memory usage and for large 3D problems in terms of CPU time as well.

42 ENGINEERING↗
NASA NTRS2003

Some Remarks on GMRES for Transport Theory

We review some work on the application of GMRES to the solution of the discrete ordinates transport equation in one-dimension. We note that GMRES can be applied directly to the angular flux vector, or it can be applied to only a vector of flux moments as needed to compute the scattering operator of the transport equation. In the former case we illustrate both the delights and defects of ILU right-preconditioners for problems with anisotropic scatter and for problems with upscatter. When working with flux moments we note that GMRES can be used as an accelerator for any existing transport code whose solver is based on a stationary fixed-point iteration, including transport sweeps and DSA transport sweeps. We also provide some numerical illustrations of this idea. We finally show how space can be traded for speed by taking multiple transport sweeps per GMRES iteration. Key Words: transport equation, GMRES, Krylov subspace

Patton, Bruce W.↗
NASA NTRSDate not supplied

Sparse Linear Algebra Toolkit for Computational Aerodynamics

Finding solutions to sparse linear systems of equations is an essential step in Computational Engineering applications of interest to NASA. Linear systems of equations are composed and solved in almost every computational engineering application. The characteristics of linear systems vary greatly from one application to another. Accordingly, there are a wide variety of methods for the solution of linear systems of equations. The operations and methods prepared by the authors are focused on linear systems of interest to NASA, primarily those associated with Computational Fluid Dynamics (CFD), Aeroelasticity, and Aeroacoustics. The Sparse Linear Algebra Toolkit (SLAT) is a coordinated collection of software featuring operations, methods, and data structures that are useful when solving sparse linear systems of equations on modern computer architectures. The implemented operations and methods are designed and tuned for parallelism in shared memory, in distributed memory, and across the hybrid combination of distributed-shared memory. The toolkit includes novel methods and implementations for modern architectures and facilitates development of new approaches for meeting NASA’s evolving computational engineering challenges using evolving computer architectures that are not available in vendor libraries. In this paper, significant features and interfaces within SLAT are presented and verified for simulations performed with NASA’s CFD solver, FUN3D. The runtime and scaling performance of the Generalized Minimum Residual (GMRES) method implemented in SLAT is analyzed for the linear subproblems within the solution of turbulent Navier-Stokes equations employed in the simulation of high-lift configurations. Prior to this work, the SPARSKIT GMRES implementation was the only Krylov subspace method available within FUN3D. A strong scaling study shows the SLAT GMRES implementation facilitates accurate Reynolds-averaged Navier-Stokes CFD solutions between 15% and 56% faster than the SPARSKIT GMRES implementation.

Stephen L Wood↗
DOE OSTI2023

Jacobian-free Newton–Krylov method for the simulation of non-thermal plasma discharges with high-order time integration and physics-based preconditioning

A preconditioning framework for the numerical simulation of non-thermal streamer discharges is developed using the Jacobian-free Newton-Krylov (JFNK) method. A reduced plasma fluid model is considered, consisting of electrons, one positive ion, one negative ion, and the electrostatic potential. Here, the plasma kinetics model includes ionization, electron-ion recombination, electron attachment, electron detachment, and ion-ion recombination. The governing equations are made dimensionless, discretized in space with finite differences, and integrated in time with a fully implicit method based on high-order backward differentiation formulas. The preconditioning framework is based on a linearized form of the governing equations and physics-based operator splitting. The efficiency of the preconditioning strategy is assessed through two test cases: streamer propagation between parallel plates and an axisymmetric pin-to-pin discharge. The fully implicit approach overcomes traditional restrictions in the time step size due to processes such as electron drift, electron diffusion, and dielectric relaxation. Excellent performance is observed through relevant statistics of the JFNK solver, although the number of linear iterations increases for the pin-to-pin discharge when nonlinear numerical boundary conditions are imposed at the electrodes. Performance studies show scalability with O(100-1000) processors for O(10M) unknowns with ample room for optimization.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2021

Replicated Computational Results (RCR) Report for “Adaptive Precision Block-Jacobi for High Performance Preconditioning in the Ginkgo Linear Algebra Software”

The article by Flegar et al. titled “Adaptive Precision Block-Jacobi for High Performance Preconditioning in the Ginkgo Linear Algebra Software” presents a novel, practical implementation of an adaptive precision block-Jacobi preconditioner. Performance results using state-of-the-art GPU architectures for the block-Jacobi preconditioner generation and application demonstrate the practical usability of the method, compared to a traditional full-precision block-Jacobi preconditioner. A production-ready implementation is provided in the Ginkgo numerical linear algebra library. In this report, the Ginkgo library is reinstalled and performance results are generated to perform a comparison to the original results when using Ginkgo’s Conjugate Gradient solver with either the full or the adaptive precision block-Jacobi preconditioner for a suite of test problems on an NVIDIA GPU accelerator. After completing this process, the published results are deemed reproducible.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

PyAMG: Algebraic Multigrid Solvers in Python

PyAMG is a Python package of algebraic multigrid (AMG) solvers and supporting tools for approximating the solution to large, sparse linear systems of algebraic equations, Ax = b, where A is an n × n sparse matrix. Sparse linear systems arise in a range of problems in science, from fluid flows to solid mechanics to data analysis. While the direct solvers available in SciPy’s sparse linear algebra package (scipy.sparse.linalg) are highly efficient, in many cases iterative methods are preferred due to overall complexity. However, the iterative methods in SciPy, such as CG and GMRES, often require an efficient preconditioner in order to achieve a lower complexity. Preconditioning is a powerful tool whereby the conditioning of the linear system and convergence rate of the iterative method are both dramatically improved. PyAMG constructs multigrid solvers for use as a preconditioner in this setting. A summary of multigrid and algebraic multigrid solvers can be found in Olson (2015a), in Olson (2015b), and in Falgout (2006); a detailed description can be found in Briggs et al. (2000) and Trottenberg et al. (2001).

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1994

Preconditioned conjugate gradient methods for the Navier-Stokes equations

A preconditioned Krylov subspace method (GMRES) is used to solve the linear systems of equations formed at each time-integration step of the unsteady, two-dimensional, compressible Navier-Stokes equations of fluid flow. The Navier-Stokes equations are cast in an implicit, upwind finite-volume, flux-split formulation. Several preconditioning techniques are investigated to enhance the efficiency and convergence rate of the implicit solver based on the GMRES algorithm. The superiority of the new solver is established by comparisons with a conventional implicit solver, namely line Gauss-Seidel relaxation (LGSR). Computational test results for low-speed (incompressible flow over a backward-facing step at Mach 0.1), transonic flow (trailing edge flow in a transonic turbine cascade), and hypersonic flow (shock-on-shock interactions on a cylindrical leading edge at Mach 6.0) are presented. For the Mach 0.1 case, overall speedup factors of up to 17 (in terms of time-steps) and 15 (in terms of CPU time on a CRAY-YMP/8) are found in favor of the preconditioned GMRES solver, when compared with the LGSR solver. The corresponding speedup factors for the transonic flow case are 17 and 23, respectively. The hypersonic flow case shows slightly lower speedup factors of 9 and 13, respectively. The study of preconditioners conducted in this research reveals that a new LUSGS-type preconditioner is much more efficient than a conventional incomplete LU-type preconditioner.

Ajmani, Kumud↗
NASA NTRS2019

Streamlined Convergence Acceleration for CFD Codes

Enigma, a simplified interface to the PETSc library, is shown to enable the rapid solution of discrete partial differential equations. Two CFD codes, LAURA and HyperSolve, use Enigma to compute steady solutions of the Navier-Stokes equations. Using PETSc, Enigma is shown to provide a Jacobian-Free Newton-Krylov method (JFNK), globalized with pseudotransient continuation, that improves efficiency over the point-implicit relaxation method traditionally used by LAURA. It is shown that iterative error has a large impact on surface heat transfer predicted by LAURA on an axisymmetric sphere-cone geometry. Also, the convergence rate of HyperSolve simulating subsonic flow over a delta wing geometry with the JFNK method is shown to be more efficient than employing a defect correction method as the nonlinear solver.

Thompson, Kyle B.↗
DOE OSTI2021

Quarter 4 Report: Report on Final Findings and Opportunities for Future Work in the Use of Mixed Precision in Iterative Solvers

The fourth quarter of the project was spent developing an error analysis of the s-step Lanczos and CG algorithms. Our theoretical bounds and numerical experiments show that the numerical behavior of the algorithm can be significantly improved by using extra precision in a small part of the computation related to the computation and application of the Gram matrix. We have published a technical report which includes all steps of the analysis [8]; a shortened version for journal submission is in preparation. We plan to submit this paper in the following weeks. Activities related to this also include a collaboration with Ichitaro Yamazaki on gathering performance results for these new mixed precision s-step Krylov subspace methods using single/double precision on GPUs. Namely, we would like to obtain performance results that show that the performance overhead of using double the working precision in these select computations is minimal. Other activities include attending biweekly xSDK meetings and presenting a pitch talk on this work to the group on February 25, 2021. In the remainder of the document, we summarize our findings on the potential for mixed precision in classical Krylov subspace methods and s-step Krylov subspace methods, as well as key opportunities for future work.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

An Efficient High-Order Solver for Diffusion Equations with Strong Anisotropy on Non-Anisotropy-Aligned Meshes

This paper concerns numerical solution of the diffusion equation with strong anisotropy on meshes not aligned with the anisotropic vector field. In order to resolve the numerical pollution for simulations on a non-anisotropy-aligned mesh and reduce the associated high computational cost we propose an effective preconditioner, extending our previous work. Similar to the anisotropy-aligned mesh case, we apply the auxiliary space preconditioning framework to design a preconditioner where a continuous finite element space is used as the auxiliary space for the discontinuous finite element space. The key component is an effective line smoother that can mitigate the high-frequency errors perpendicular to the magnetic field. We design a graph-based approach to find such a line smoother that is approximately perpendicular to the vector fields when the mesh does not align with the anisotropy. Finally, numerical experiments for several benchmark problems are presented, demonstrating the effectiveness and robustness of the proposed preconditioner when applied to Krylov iterative methods.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

GMRES with embedded ensemble propagation for the efficient solution of parametric linear systems in uncertainty quantification of computational models

In a previous work, embedded ensemble propagation was proposed to improve the efficiency of sampling-based uncertainty quantification methods of computational models on emerging computational architectures. It consists of simultaneously evaluating the model for a subset of samples together, instead of evaluating them individually. A first approach introduced to solve parametric linear systems with ensemble propagation is ensemble reduction. In Krylov methods for example, this reduction consists in coupling the samples together using an inner product that sums the sample contributions. Ensemble reduction has the advantages of being able to use optimized implementations of BLAS functions and having a stopping criterion which involves only one scalar. However, the reduction potentially decreases the rate of convergence due to the gathering of the spectra of the samples. In this paper, we investigate a second approach: ensemble propagation without ensemble reduction in the case of GMRES. This second approach solves each sample simultaneously but independently to improve the convergence compared to ensemble reduction. This raises two new issues which are solved in this paper: the fact that optimized implementations of BLAS functions cannot be used anymore and that ensemble divergence, whereby individual samples within an ensemble must follow different code execution paths, can occur. We tackle those issues by implementing a high-performing ensemble GEMV and by using masks. The proposed ensemble GEMV leads to a similar cost per GMRES iteration for both approaches, i.e. with and without reduction. For illustration, we study the performances of the new linear solver in the context of a mesh tying problem. Furthermore, this example demonstrates improved ensemble propagation speed-up without reduction.

BLAS↗