Improving Scalability of Multilevel Markov Chain Monte Carlo Using Multiple Proposals
Explore the source record for details and available documents.
Mathematics and Computing↗
Engineering topics
Vassilevski, Panayot
Publications and source records attributed to Vassilevski, Panayot.
Multilevel-in-width training for deep neural network regression
A common challenge in regression is that for many problems, the degrees of freedom required for a high-quality solution also allows for overfitting. Regularization is a class of strategies that seek to restrict the range of possible solutions so as to discourage overfitting while still enabling good solutions, and different regularization strategies impose different types of restrictions. In this paper, we present a multilevel regularization strategy that constructs and trains a hierarchy of neural networks, each of which has layers that are wider versions of the previous network's layers. Here, we draw intuition and techniques from the field of Algebraic Multigrid (AMG), traditionally used for solving linear and nonlinear systems of equations, and specifically adapt the Full Approximation Scheme (FAS) for nonlinear systems of equations to the problem of deep learning. Training through V-cycles then encourage the neural networks to build a hierarchical understanding of the problem. We refer to this approach as multilevel-in-width to distinguish from prior multilevel works which hierarchically alter the depth of neural networks. The resulting approach is a highly flexible framework that can be applied to a variety of layer types, which we demonstrate with both fully connected and convolutional layers. We experimentally show with PDE regression problems that our multilevel training approach is an effective regularizer, improving the generalize performance of the neural networks studied.
97 MATHEMATICS AND COMPUTING↗
A Condensed Constrained Nonconforming Mortar-Based Approach for Preconditioning Finite Element Discretization Problems
This paper presents and studies an approach for constructing auxiliary space preconditioners for finite element problems using a constrained nonconforming reformulation that is based on a proposed modified version of the mortar method. The well-known mortar finite element discretization method is modified to admit a local structure, providing an element-by-element or subdomain-by-subdomain assembly property. This is achieved via the introduction of additional trace finite element spaces and degrees of freedom (unknowns) associated with the interfaces between adjacent elements or subdomains. The resulting nonconforming formulation and a reduced-via-static-condensation Schur complement form on the interfaces are used in the construction of auxiliary space preconditioners for a given conforming finite element discretization problem. Overall, the properties of these preconditioners are studied and their performance is illustrated on model second order scalar elliptic problems utilizing high order elements.
97 MATHEMATICS AND COMPUTING↗