Bayesian Federated Learning with Hamiltonian Monte Carlo: Algorithm and Theory
Not provided.
Mathematics↗
Engineering topics
Deng, Wei
Publications and source records attributed to Deng, Wei.
Frequency multiplication with toroidal mode number of kink/fishbone modes on a static HL-2A-like tokamak
In the presence of energetic particles (EPs), the long-lived mode (LLM) frequency multiplication with n = 1, 2, 3, or higher is often observed on HL-2A, where n is the toroidal mode number. Hybrid kinetic-MHD model simulations of the energetic particle (EP) driven kink/fishbone modes on a static HL-2A-like tokamak using NIMROD code find that when the background plasma pressure is relatively high, and the EP pressure and the beam energy are relatively low, the mode frequency increases almost linearly with EP pressure, and the frequency is proportional to n (‘frequency multiplication’), even in the absence of any equilibrium plasma rotation. In addition, the frequency multiplication persists as the safety factor at the magnetic axis q 0 varies. In the absence of EPs, the growth rate of the 1/1 mode is the largest; however, as the EP pressure increases, the growth rate of 2/2 modes or 3/3 modes becomes dominant, suggesting that higher-n modes are more vulnerable to EPs. These results may shed light on the understanding of the toroidal mode number dependence of kink/fishbone modes in the advanced scenarios of tokamaks with weak or reversed central magnetic shear.
70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
An adaptively weighted stochastic gradient MCMC algorithm for Monte Carlo simulation and global optimization
Not provided.
Computer Science↗
An adaptive Hessian approximated stochastic gradient MCMC method
Bayesian approaches have been successfully integrated into training deep neural networks. One popular family is stochastic gradient Markov chain Monte Carlo methods (SG-MCMC), which have gained increasing interest due to their ability to handle large datasets and the potential to avoid overfitting. Although standard SG-MCMC methods have shown great performance in a variety of problems, they may be inefficient when the random variables in the target posterior densities have scale differences or are highly correlated. Here, we present an adaptive Hessian approximated stochastic gradient MCMC method to incorporate local geometric information while sampling from the posterior. The idea is to apply stochastic approximation (SA) to sequentially update a preconditioning matrix at each iteration. The preconditioner possesses second-order information and can guide the random walk of a sampler efficiently. Instead of computing and saving the full Hessian of the log posterior, we use limited memory of the samples and their stochastic gradients to approximate the inverse Hessian-vector multiplication in the updating formula. Moreover, by smoothly optimizing the preconditioning matrix via SA, our proposed algorithm can asymptotically converge to the target distribution with a controllable bias under mild conditions. To reduce the training and testing computational burden, we adopt a magnitude-based weight pruning method to enforce the sparsity of the network. Our method is user-friendly and demonstrates better learning results compared to standard SG-MCMC updating rules. The approximation of inverse Hessian alleviates storage and computational complexities for large dimensional models. Numerical experiments are performed on several problems, including sampling from 2D correlated distribution, synthetic regression problems, and learning the numerical solutions of heterogeneous elliptic PDE. The numerical results demonstrate great improvement in both the convergence rate and accuracy.
97 MATHEMATICS AND COMPUTING↗
Bayesian sparse learning with preconditioned stochastic gradient MCMC and its applications
Deep neural networks have been successfully employed in an extensive variety of research areas, including solving partial differential equations. Despite its significant success, there are some challenges in effectively training DNN, such as avoiding overfitting in over-parameterized DNNs and accelerating the optimization in DNNs with pathological curvature. Here, we propose a Bayesian type sparse deep learning algorithm. The algorithm utilizes a set of spike-and-slab priors for the parameters in the deep neural network. The hierarchical Bayesian mixture will be trained using an adaptive empirical method. That is, one will alternatively sample from the posterior using preconditioned stochastic gradient Langevin Dynamics (PSGLD), and optimize the latent variables via stochastic approximation. The sparsity of the network is achieved while optimizing the hyperparameters with adaptive searching and penalizing. A popular SG-MCMC approach is Stochastic gradient Langevin dynamics (SGLD). However, considering the complex geometry in the model parameter space in nonconvex learning, updating parameters using a universal step size in each component as in SGLD may cause slow mixing. To address this issue, we apply a computationally manageable preconditioner in the updating rule, which provides a step-size parameter to adapt to local geometric properties. Moreover, by smoothly optimizing the hyperparameter in the preconditioning matrix, our proposed algorithm ensures a decreasing bias, which is introduced by ignoring the correction term in the preconditioned SGLD. According to the existing theoretical framework, we show that the proposed algorithm can asymptotically converge to the correct distribution with a controllable bias under mild conditions. Numerical tests are performed on both synthetic regression problems and learning solutions of elliptic PDE, which demonstrate the accuracy and efficiency of the present work.
71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
Water-Surface Drag Coating: A New Route Toward High-Quality Conjugated Small-Molecule Thin Films with Enhanced Charge Transport Properties
Electronic properties of organic semiconductor (OSC) thin films are largely determined by their morphologies and crystallinities. However, solution-processed conjugated small-molecule OSC thin films usually exhibit abundant grain boundaries and impure grain orientations because of complex fluid dynamics during solution coating. Here, a novel methodology, water-surface drag coating, is demonstrated to fabricate high-quality OSC thin films with greatly enhanced charge transport properties. This method utilizes the water surface to alter the evaporation dynamics of solution to enlarge the grain size, and a unique drag-coating process to achieve the unidirectional growth of organic crystals. Furthermore, using 2,8-difluoro-5,11-bis(triethylsilylethynyl)anthradithiophene (Dif-TES-ADT) as an example, thin films with millimeter-sized single-crystal domains and pure crystallographic orientations are achieved, revealing a significant enhancement (4.7 times) of carrier mobility. More importantly, the resulting film can be directly transferred onto any desired flexible substrates, and flexible transistors based on the Dif-TES-ADT thin films show a mobility as high as 16.1 cm 2 V -1 s -1 , which represents the highest mobility value for the flexible transistors reported thus far. The method is general for the growth of various high-quality OSC thin films, thus opening up opportunities for high-performance organic flexible electronics.
36 MATERIALS SCIENCE↗
One-dimensional hybrid satellite track model for the Dynamics Explorer 2 (DE 2) satellite
A one-dimensional hybrid satellite track model has been developed to calculate the high-latitude thermospheric/ionospheric structure below the satellite altitude using Dynamics Explorer 2 (DE 2) satellite measurements and theory. This model is based on Emery et al. (1985) satellite track code but also includes elements of Roble et al. (1987b) global mean thermosphere/ionosphere model. A number of parameterizations and data handling techniques are used to input satellite data from several DE 2 instruments into this model. Profiles of neutral atmospheric densities are determined from the Mass Spectrometer Incoherent Scatter 1990 (MSIS-90) model and measured neutral temperatures. Measured electron precipitation spectra are used in an auroral model to calculate particle impact ionization rates below the satellite. These rates are combined with a solar ionization rate profile and used to solve the O(+) diffusion equation, with the measured electron density as an upper boundary condition. The calculated O(+) density distribution, as well as the ionization profiles, are then used in a photochemical equilibrium model to calculate the electron and molecular ion densities. The electron temperature is also calculated by solving the electron energy equation with an upper boundary condition determined by the DE 2 measurement. The model enables calculations of altitude profiles of conductivity and Joule heating rate along and below the satellite track. In a first application of the new model, a study is made of thermospheric and ionospheric structure below the DE 2 satellite for a single orbit which occurred on October 25, 1981. The field-aligned Poynting flux, which is independently obtained for this orbit, is compared with the model predictions of the height-integrated energy conversion rate. Good quantitative agreement between these two estimates has been reached. In addition, measurements taken at the incoherent scatter radar site at Chatanika (65.1 deg N, 147.4 deg W) during a DE 2 overflight are compared with the model calculations. A good agreement was found in lower thermospheric conductivities and Joule heating rate.
Deng, Wei↗
One-Dimensional Hybrid Satellite Track Model for the Dynamics Explorer 2 (DE 2) Satellite
A one-dimensional hybrid satellite track model has been developed to calculate the high-latitude thermospheric/ionospheric structure below the satellite altitude using Dynamics Explorer 2 (DE 2) satellite measurements and theory. This model is based on Emery et al. satellite track code but also includes elements of Roble et al. global mean thermosphere/ionosphere model. A number of parameterizations and data handling techniques are used to input satellite data from several DE 2 instruments into this model. Profiles of neutral atmospheric densities are determined from the MSIS-90 model and measured neutral temperatures. Measured electron precipitation spectra are used in an auroral model to calculate particle impact ionization rates below the satellite. These rates are combined with a solar ionization rate profile and used to solve the O(+) diffusion equation, with the measured electron density as an upper boundary condition. The calculated O(+) density distribution, as well as the ionization profiles, are then used in a photochemical equilibrium model to calculate the electron and molecular ion densities. The electron temperature is also calculated by solving the electron energy equation with an upper boundary condition determined by the DE 2 measurement. The model enables calculations of altitude profiles of conductivity and Joule beating rate along and below the satellite track. In a first application of the new model, a study is made of thermospheric and ionospheric structure below the DE 2 satellite for a single orbit which occurred on October 25, 1981. The field-aligned Poynting flux, which is independently obtained for this orbit, is compared with the model predictions of the height-integrated energy conversion rate. Good quantitative agreement between these two estimates has been reached. In addition, measurements taken at the incoherent scatter radar site at Chatanika (65.1 deg N, 147.4 deg W) during a DE 2 overflight are compared with the model calculations. A good agreement was found in lower thermospheric conductivities and Joule heating rate.
Deng, Wei↗