Engineering PapersSearch

SEARCH · Engineering Papers

Results for “Multigrid”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

38 records · Page 3

Optimal Transfer Operators in Algebraic Two-Level Methods for Nonsymmetric and Indefinite Problems

Consider an algebraic two-level method applied to the 𝑛-dimensional linear system 𝐴⁢𝒙 = 𝒃 using fine-space preconditioner (i.e., “relaxation” or “smoother”) 𝑀, with 𝑀 ≈ 𝐴, restriction and interpolation 𝑅 and 𝑃, and algebraic coarse-space operator 𝐴 𝑐 : = 𝑅 ∗ ⁢𝐴⁢𝑃. Then, what are the best possible transfer operators 𝑅 and 𝑃 of a given dimension 𝑛 𝑐 < 𝑛? Brannick et al. [12] showed that when 𝐴 and 𝑀 are Hermitian positive definite (HPD), the optimal interpolation is such that its range contains the 𝑛 𝑐 smallest generalized eigenvectors of the matrix pencil (𝐴, 𝑀). Recently, in Ali et al. [5] we generalized this framework to the non-HPD setting, by considering both right (interpolation) and left (restriction) generalized eigenvectors of (𝐴, 𝑀) and defining corresponding nonsymmetric transfer operators {𝑅#, 𝑃#}. Tight convergence bounds for {𝑅#, 𝑃#} are derived in spectral radius, as well as a proof of pseudo-optimality. Note, {𝑅#, 𝑃#} are typically complex valued, which is not practical for real-valued problems. Here, in this work, we build on [5], first characterizing all inner products in which the coarse-space correction defined by {𝑅#, 𝑃#} is orthogonal. We then develop tight two-level convergence bounds in these norms, and prove that the underlying transfer operators {𝑅#, 𝑃#} are genuinely optimal. As a special case, our theory both recovers and extends the HPD results from [12]. Finally, we show how to construct optimal, real-valued transfer operators in the case of that 𝐴 and 𝑀 are real valued, but are not HPD. Numerical examples arising from a discretized advection-reaction equation, wave-equation, and Stokes equations are used to verify and illustrate the theory.

97 MATHEMATICS AND COMPUTING

TorchBraid: High-Performance Layer-Parallel Training of Deep Neural Networks with MPI and GPU Acceleration

TorchBraid is a high-performance implementation of layer-parallel training for deep neural networks (DNNs) supporting MPI-based parallelism and GPU acceleration. Layer-parallel training has been developed to overcome the serialization inherent in forward and backward propagation of DNNs that limits utilization of computational resources in the strong scaling limit. To achieve this, TorchBraid integrates the PyTorch neural network framework with the state-of-the-art XBraid time-parallel library. Furthermore, this article presents the use and performance of TorchBraid, in addition to solutions for overcoming the algorithmic challenges inherent in combining automatic differentiation with layer-parallel. Results are presented with and without GPU acceleration for the Tiny ImageNet and MNIST image classification data sets, as well as recurrent neural networks. Overall, TorchBraid enables fast training of DNNs, both in a strong and weak scaling context. In addition to the TorchBraid software, several new advances in applying layer-parallel algorithms are detailed. Integration of layer-parallel with data-parallel algorithms is presented for the first time, showing the computational advantages of the combination. Standard deep learning techniques, like batch-normalization, are developed for layer-parallel training. Finally, a new approach combining layer-parallel with spatial coarsening in order to accelerate training for 3D image classification shows roughly a 10× speedup over serial execution.

Layer-parallel