Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Inference”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,477 records · Page 82

Prediction of Metabolite Concentrations, Rate Constants and Post-Translational Regulation Using Maximum Entropy-Based Simulations with Application to Central Metabolism of Neurospora crassa

We report the application of a recently proposed approach for modeling biological systems using a maximum entropy production rate principle in lieu of having in vivo rate constants. The method is applied in four steps: (1) a new ordinary differential equation (ODE) based optimization approach based on Marcelin’s 1910 mass action equation is used to obtain the maximum entropy distribution; (2) the predicted metabolite concentrations are compared to those generally expected from experiments using a loss function from which post-translational regulation of enzymes is inferred; (3) the system is re-optimized with the inferred regulation from which rate constants are determined from the metabolite concentrations and reaction fluxes; and finally (4) a full ODE-based, mass action simulation with rate parameters and allosteric regulation is obtained. From the last step, the power characteristics and resistance of each reaction can be determined. The method is applied to the central metabolism of Neurospora crassa and the flow of material through the three competing pathways of upper glycolysis, the non-oxidative pentose phosphate pathway, and the oxidative pentose phosphate pathway are evaluated as a function of the NADP/NADPH ratio. It is predicted that regulation of phosphofructokinase (PFK) and flow through the pentose phosphate pathway are essential for preventing an extreme level of fructose 1,6-bisphophate accumulation. Such an extreme level of fructose 1,6-bisphophate would otherwise result in a glassy cytoplasm with limited diffusion, dramatically decreasing the entropy and energy production rate and, consequently, biological competitiveness.

Cannon, William R.↗

Connecting CO intensity mapping to molecular gas and star formation in the epoch of galaxy assembly

Intensity mapping, which images a single spectral line from unresolved galaxies across cosmological volumes, is a promising technique for probing the early universe. Here we present predictions for the intensity map and power spectrum of the CO(1–0) line from galaxies at z∼2.4–2.8, based on a parameterized model for the galaxy–halo connection, and demonstrate the extent to which properties of high-redshift galaxies can be directly inferred from such observations. We find that our fiducial prediction should be detectable by a realistic experiment. Motivated by significant modeling uncertainties, we demonstrate the effect on the power spectrum of varying each parameter in our model. Using simulated observations, we infer constraints on our model parameter space with an MCMC procedure, and show corresponding constraints on the L{sub IR}–L{sub CO} relation and the CO luminosity function. These constraints would be complementary to current high-redshift galaxy observations, which can detect the brightest galaxies but not complete samples from the faint end of the luminosity function. By probing these populations in aggregate, CO intensity mapping could be a valuable tool for probing molecular gas and its relation to star formation in high-redshift galaxies.

79 ASTRONOMY AND ASTROPHYSICS↗

Discovery of a millisecond pulsar in the 5.4 day binary 3FGL J1417.5–4402: Observing the late phase of pulsar recycling

In a search of the unidentified Fermi gamma-ray source 3FGL J1417.5–4402 with the Parkes radio telescope, we discovered PSR J1417–4402, a 2.66 ms pulsar having the same 5.4 day orbital period as the optical and X-ray binary identified by Strader et al. The existence of radio pulsations implies that the neutron star is currently not accreting. Substantial outflows from the companion render the radio pulsar undetectable for more than half of the orbit, and may contribute to the observed Hα emission. Our initial pulsar observations, together with the optically inferred orbit and inclination, imply a mass ratio of 0.171 ± 0.002, a companion mass of M{sub 2}=0.33±0.03 M{sub ☉}, and a neutron star mass in the range 1.77⩽M{sub 1}⩽2.13 M{sub ☉}. However, there remains a discrepancy between the distance of 4.4 kpc inferred from the optical properties of the companion and the smaller radio dispersion measure distance of 1.6 kpc. The smaller distance would reduce the inferred Roche-lobe filling factor, increase the inferred inclination angle, and decrease the masses. As a wide binary, PSR J1417–4402 differs from the radio-eclipsing black widow and redback pulsars being discovered in large numbers by Fermi. It is probably a system that began mass transfer onto the neutron star after the companion star left the main sequence. The companion should end its evolution as a He white dwarf in a 6–20 day orbit, i.e., as a typical binary millisecond pulsar companion.

79 ASTRONOMY AND ASTROPHYSICS↗

Consequences of CCD imperfections for cosmology determined by weak lensing surveys: from laboratory measurements to cosmological parameter bias

Weak gravitational lensing causes subtle changes in the apparent shapes of galaxies due to the bending of light by the gravity of foreground masses. By measuring the shapes of large numbers of galaxies (millions in recent surveys, up to tens of billions in future surveys) we can infer the parameters that determine cosmology. Imperfections in the detectors used to record images of the sky can introduce changes in the apparent shapes of galaxies, which in turn can bias the inferred cosmological parameters. In this paper we consider the effect of two widely discussed sensor imperfections: tree rings, due to impurity gradients that cause transverse electric fields in the charge-coupled devices (CCDs), and pixel size variation, due to periodic CCD fabrication errors. These imperfections can be observed when the detectors are subject to uniform illumination (flat-field images). We develop methods to determine the spurious shear and convergence (due to the imperfections) from the flat-field images. We calculate how the spurious shear when added to the lensing shear will bias the determination of cosmological parameters. We apply our methods to candidate sensors of the Large Synoptic Survey Telescope (LSST) as a timely and important example, analyzing flat-field images recorded with LSST prototype CCDs in the laboratory. We find that tree rings and periodic pixel size variation present in the LSST CCDs will introduce negligible bias to cosmological parameters determined from the lensing power spectrum, specifically w, Ω{sub m}, and σ{sub 8}.

79 ASTRONOMY AND ASTROPHYSICS↗

Mapping and simulating systematics due to spatially-varying observing conditions in DES science verification data

Spatially-varying depth and characteristics of observing conditions, such as seeing, airmass, or sky background, are major sources of systematic uncertainties in modern galaxy survey analyses, in particular in deep multi-epoch surveys. We present a framework to extract and project these sources of systematics onto the sky, and apply it to the Dark Energy Survey (DES) to map the observing conditions of the Science Verification (SV) data. The resulting distributions and maps of sources of systematics are used in several analyses of DES SV to perform detailed null tests with the data, and also to incorporate systematics in survey simulations. We illustrate the complementarity of these two approaches by comparing the SV data with the BCC-UFig, a synthetic sky catalogue generated by forward-modelling of the DES SV images. We then analyse the BCC-UFig simulation to construct galaxy samples mimicking those used in SV galaxy clustering studies. We show that the spatially-varying survey depth imprinted in the observed galaxy densities and the redshift distributions of the SV data are successfully reproduced by the simulation and well-captured by the maps of observing conditions. The combined use of the maps, the SV data and the BCC-UFig simulation allows us to quantify the impact of spatial systematics on N(z), the redshift distributions inferred using photometric redshifts. We conclude that spatial systematics in the SV data are mainly due to seeing fluctuations and are under control in current clustering and weak lensing analyses. However, they will need to be carefully characterised in upcoming phases of DES in order to avoid biasing the inferred cosmological results. The framework presented is relevant to all multi-epoch surveys, and will be essential for exploiting future surveys such as the Large Synoptic Survey Telescope, which will require detailed null-tests and realistic end-to-end image simulations to correctly interpret the deep, high-cadence observations of the sky.

79 ASTRONOMY AND ASTROPHYSICS↗

Neural Posterior Estimation for Cataloging Astronomical Images with Spatially Varying Backgrounds and Point Spread Functions

Neural posterior estimation (NPE), a type of amortized variational inference, is a computationally efficient means of constructing probabilistic catalogs of light sources from astronomical images. To date, NPE has not been used to perform inference in models with spatially varying covariates. However, ground-based astronomical images exhibit spatially varying sky backgrounds and point spread functions (PSFs), and accounting for this variation is essential for constructing accurate catalogs of imaged light sources. In this work, we introduce a novel NPE-based cataloging method that trains an inference network with semisynthetic astronomical images generated using PSFs and backgrounds sampled from the Sloan Digital Sky Survey. In experiments with semisynthetic images, we evaluate the method on key cataloging tasks: light source detection, star/galaxy separation, and flux measurement. A “generalist” inference network—trained with diverse PSFs and backgrounds—performs as well as a “specialist” network even when both are evaluated on the specialist’s particular PSF/background combination. This result suggests that a single NPE network can generalize across spatial variations, eliminating the need for retraining on each observational condition.

astronomy image processing↗

Probabilistic Cosmological Mass Mapping from Weak Lensing Shear

We infer gravitational lensing shear and convergence fields from galaxy ellipticity catalogs under a spatial process prior for the lensing potential. We demonstrate the performance of our algorithm with simulated Gaussian-distributed cosmological lensing shear maps and a reconstruction of the mass distribution of the merging galaxy cluster Abell 781 using galaxy ellipticities measured with the Deep Lens Survey. Given interim posterior samples of lensing shear or convergence fields on the sky, we describe an algorithm to infer cosmological parameters via lens field marginalization. In the most general formulation of our algorithm we make no assumptions about weak shear or Gaussian-distributed shape noise or shears. Because we require solutions and matrix determinants of a linear system of dimension that scales with the number of galaxies, we expect our algorithm to require parallel high-performance computing resources for application to ongoing wide field lensing surveys.

79 ASTRONOMY AND ASTROPHYSICS↗

TARDIS. I. A Constrained Reconstruction Approach to Modeling the z ~ 2.5 Cosmic Web Probed by Lyα Forest Tomography

Recent Lyα forest tomography measurements of the intergalactic medium (IGM) have revealed a wealth of cosmic structures at high redshift (z ~ 2.5). In this work, we present the Tomographic Absorption Reconstruction and Density Inference Scheme (TARDIS), a new chronocosmographic analysis tool for understanding the formation and evolution of these observed structures. Here, we use maximum likelihood techniques with a fast nonlinear gravitational model to reconstruct the initial density field of the observed regions. We find that TARDIS allows accurate reconstruction of smaller-scale structures than standard Wiener-filtering techniques. Applying this technique to mock Lyα forest data sets that simulate ongoing and future surveys such as CLAMATO, Subaru PFS, or the ELTs, we are able to infer the underlying matter density field at observed redshift and classify the cosmic web structures. We find good agreement with the underlying truth in both the characteristic eigenvalues and eigenvectors of the pseudo-deformation tensor, with the eigenvalues inferred from 30 m class telescopes correlated at r = 0.95 relative to the truth. As an output of this method, we are able to further evolve the inferred structures to late time (z = 0) and also track the trajectories of coeval z = 2.5 galaxies to their z = 0 cosmic web environments.

79 ASTRONOMY AND ASTROPHYSICS↗

The Intrinsic Shapes of Low Surface Brightness Galaxies (LSBGs): A Discriminant of LSBG Galaxy Formation Mechanisms

We use the low surface brightness galaxy (LSBG) samples created from the Hyper Suprime-Cam Subaru Strategic Program (781 galaxies), the Dark Energy Survey (20977 galaxies), and the Legacy Survey (selected via H ι detection in the Arecibo Legacy Fast ALFA Survey, 188 galaxies) to infer the intrinsic shape distribution of the LSBG population. To take into account the effect of the surface brightness cuts employed when constructing LSBG samples, we simultaneously model both the projected ellipticity and the apparent surface brightness in our shape inference. We find that the LSBG samples are well characterized by oblate spheroids, with no significant difference between red and blue LSBGs. This inferred shape distribution is in good agreement with similar inferences made for ultra-diffuse cluster galaxy samples, indicating that environment does not play a key role in determining the intrinsic shape of LSBGs. In this study, we also find some evidence that LSBGs are more thickened than similarly massive high surface brightness dwarfs. We compare our results to intrinsic shape measures from contemporary cosmological simulations, and find that the observed LSBG intrinsic shapes place considerable constraints on the formation path of such galaxies. In particular, LSBG production via the migration of star formation to large radii produces intrinsic shapes in good agreement with our observational findings.

79 ASTRONOMY AND ASTROPHYSICS↗

The DESI PRObabilistic Value-added Bright Galaxy Survey (PROVABGS) Mock Challenge

The PRObabilistic Value-added Bright Galaxy Survey (PROVABGS) catalog will provide measurements of galaxy properties, such as stellar mass ($M*$), star formation rate (SFR), stellar metallicity (Z), and stellar age ($t_{age}$), for >10 million galaxies of the Dark Energy Spectroscopic Instrument (DESI) Bright Galaxy Survey. Full posterior distributions of the galaxy properties will be inferred using state-of-the-art Bayesian spectral energy distribution (SED) modeling of DESI spectroscopy and Legacy Surveys photometry. In this work, we present the SED model, the neural emulator for the model, and the Bayesian inference framework of PROVABGS. Furthermore, we apply the PROVABGS SED modeling on realistic synthetic DESI spectra and photometry, constructed using the L-Galaxies semi-analytic model. We compare the inferred galaxy properties to the true values of the simulation using a hierarchical Bayesian framework to quantify accuracy and precision. Overall, we accurately infer the true $M*$, SFR, Z, and $t_{age}$ of the simulated galaxies. However, the priors on galaxy properties induced by the SED model have a significant impact on the posteriors, which we characterize in detail. This work also demonstrates that a joint analysis of spectra and photometry significantly improves the constraints on galaxy properties over photometry alone and is necessary to mitigate the impact of the priors. With the methodology presented and validated in this work, PROVABGS will maximize information extracted from DESI observations and extend current galaxy studies to new regimes and unlock cutting-edge probabilistic analyses. https://github.com/changhoonhahn/provabgs/

79 ASTRONOMY AND ASTROPHYSICS↗

Applying the Velocity Gradient Technique in NGC 1333: Comparison with Dust Polarization Observations

Magnetic fields (B-fields) are ubiquitous in the interstellar medium (ISM), and they play an essential role in the formation of molecular clouds and subsequent star formation. However, B-fields in interstellar environments remain challenging to measure, and their properties typically need to be inferred from dust polarization observations over multiple physical scales. In this work, we seek to use a recently proposed approach called the velocity gradient technique (VGT) to study B-fields in star-forming regions and compare the results with dust polarization observations in different wavelengths. The VGT is based on the anisotropic properties of eddies in magnetized turbulence to derive B-field properties in the ISM. We investigate that this technique is synergistic with dust polarimetry when applied to a turbulent diffused medium for the purpose of measuring its magnetization. Specifically, we use the VGT on molecular line data toward the NGC 1333 star-forming region ( 12 CO, 13 CO, C 18 O, and N 2 H + ), and we compare the derived B-field properties with those inferred from 214 and 850 μm dust polarization observations of the region using Stratospheric Observatory for Infrared Astronomy/High-Resolution Airborne Wide-band Camera Plus and James Clerk Maxwell Telescope/POL-2, respectively. We estimate both the inclination angle and the 3D Alfvénic Mach number M A from the molecular line gradients. Crucially, testing this technique on gravitationally bound, dynamic, and turbulent regions, and comparing the results with those obtained from polarization observations at different wavelengths, such as the plane-of-sky field orientation, is an important test on the applicability of the VGT in various density regimes of the ISM. We in general do not find a close correlation between the velocity gradient inferred orientations and the dust inferred magnetic field orientations.

79 ASTRONOMY AND ASTROPHYSICS↗

Using a surrogate-assisted Bayesian framework to calibrate the runoff-generation scheme in the Energy Exascale Earth System Model (E3SM) v1

Abstract. Runoff is a critical component of the terrestrial water cycle, and Earth system models (ESMs) are essential tools to study its spatiotemporal variability. Runoff schemes in ESMs typically include many parameters so that model calibration is necessary to improve the accuracy of simulated runoff. However, runoff calibration at a global scale is challenging because of the high computational cost and the lack of reliable observational datasets. In this study, we calibrated 11 runoff relevant parameters in the Energy Exascale Earth System Model (E3SM) Land Model (ELM) using a surrogate-assisted Bayesian framework. First, the polynomial chaos expansion machinery with Bayesian compressed sensing is used to construct computationally inexpensive surrogate models for ELM-simulated runoff at 0.5∘ × 0.5∘ for 1991–2010. The error metric between the ELM simulations and the benchmark data is selected to construct the surrogates, which facilitates efficient calibration and avoids the more conventional, but challenging, construction of high-dimensional surrogates for the ELM simulated runoff. Second, the Sobol' index sensitivity analysis is performed using the surrogate models to identify the most sensitive parameters, and our results show that, in most regions, ELM-simulated runoff is strongly sensitive to 3 of the 11 uncertain parameters. Third, a Bayesian method is used to infer the optimal values of the most sensitive parameters using an observation-based global runoff dataset as the benchmark. Our results show that model performance is significantly improved with the inferred parameter values. Although the parametric uncertainty of simulated runoff is reduced after the parameter inference, it remains comparable to the multimodel ensemble uncertainty represented by the global hydrological models in ISMIP2a. Additionally, the annual global runoff trend during the simulation period is not well constrained by the inferred parameter values, suggesting the importance of including parametric uncertainty in future runoff projections.

58 GEOSCIENCES↗

A measurement of the mean central optical depth of galaxy clusters via the pairwise kinematic Sunyaev-Zel'dovich effect with SPT-3G and DES

We infer the mean optical depth of a sample of optically-selected galaxy clusters from the Dark Energy Survey (DES) via the pairwise kinematic Sunyaev-Zel'dovich (kSZ) effect. The pairwise kSZ signal between pairs of clusters drawn from the DES Year-3 cluster catalog is detected at $4.1 \sigma$ in cosmic microwave background (CMB) temperature maps from two years of observations with the SPT-3G camera on the South Pole Telescope. After cuts, there are 24,580 clusters in the $\sim 1,400$ deg$^2$ of the southern sky observed by both experiments. We infer the mean optical depth of the cluster sample with two techniques. The optical depth inferred from the pairwise kSZ signal is $\bar{\tau}_e = (2.97 \pm 0.73) \times 10^{-3}$, while that inferred from the thermal SZ signal is $\bar{\tau}_e = (2.51 \pm 0.55) \times 10^{-3}$. The two measures agree at $0.6 \sigma$. We perform a suite of systematic checks to test the robustness of the analysis.

79 ASTRONOMY AND ASTROPHYSICS↗

I Know I'm Right, But Does My Phone?

Transportation is the largest source of green-house gas emissions in the United States. Reducing transportation emissions depends on human travel behavior, which relies on local land use and planning. Travel diaries, consisting of sequences of trips between places for a particular individual, are typically used to instrument human travel behavior. However, these diaries are only as accurate as the underlying methods used to construct them. Travel diary algorithms have been a popular research topic since the advent of GPS tracking surveys. Mode inference algorithms in particular have been well represented in literature. However, these algorithms have typically been validated using prompted recall of pre-segmented trips, which doesn't account for segmentation error, thus disregarding the continuity of mode inference. Furthermore, phone operating systems and applications have adopted battery-conserving techniques, but we are not aware of prior work that has characterized the resulting data collection errors or evaluated procedures to mitigate them. We introduce a framework to evaluate accuracy of trip length computations and mode inference. We develop a temporal alignment procedure in analyzing continuous mode-segmented trajectories for groups of trips. We then apply our framework to evaluate an example set of travel diary algorithms from the open-source OpenPATH travel diary platform against MobilityNet, a public dataset containing information from three artificial timelines that cover 15 different travel modes. Our results show that inference based on an integration with map features results in weighted F_1 scores of 0.60 (iOS) and 0.74 (android). We also show that OpenPATH tends to under count trip length, with mean of signed relative error of -0.0438 on android and -0.0704 on iOS. We hope that other travel diary algorithms will be evaluated using this standardized process, and that the results used to understand and improve the state-of-the-art in this field.

ADVANCED PROPULSION SYSTEMS,ENERGY PLANNING, POLIC↗

Nuclear Forensics for Irradiated Materials using Kernel Regression Models

The ultimate purpose of nuclear forensics is to locate the origin of nuclear and other radioactive materials out of regulatory control used in nuclear security events such as classical forensics. For this purpose, some specific characteristics of interdicted materials (referred to as 'signatures') should be defined to compare the known sample database. In some cases, involved with irradiated material, the signatures could be specific amount of fission product inventory depending on reactor operational histories such as reactor type, initial enrichment, burn up rate of nuclear fuel and cooling period after irradiation in the reactor. In order to make inference model for irradiated material for nuclear forensics, we suggested 3 ways as follows; (i) analyzing ORIGEN produced nuclear substances for the fission product inventory, (ii) classifying the range of reactor type, (iii) predicting the operational history from the fission product inventory. In terms of nuclear substance analysis, the construction of spent fuel database has been internationally carried out since ITWG (Nuclear Smuggling International Technical Working Group) was established in 1995. However, enough sample data cannot be obtained in Korea to determine reactor type and operational histories because analyzing nuclear substances have been limited due to the international regulation. Thus, this study does not mention how to obtain mass or ratios of nuclides for the spent fuel. We just have generated simulation sample data as a fission product inventory using ORIGEN to construct spent fuel database. When unidentified radioactive materials are discovered, it is assumed that mass or ratios of the nuclides are analyzed by certain methods which are available in Korea as well as other countries. Then, a reactor type is determined, and finally, regression model can locate the origin or source of the radioactive materials by inferring operational histories using the fission product inventory. There are some studies on the classification of reactor type using statistical methods. As a part of predicting the operational history, this paper focuses on the development of spent fuel inference algorithm for nuclear forensics to estimate reactor operational histories that reflect characteristics of a reactor from the fission product inventory. The spent fuel inference algorithm for nuclear forensics has been developed on the basis of Kernel Regression (KR) which can conditionally generate the estimated output data by giving weight depending on geometric distance between sample data and input data. Also the KR which comprises Auto-Associative KR and Inferential KR have some benefits that can estimate only using data without assumption for specific physical model and re-learning process. Auto-associative KR (AAKR) was used to sort out outliers of input data and Inferential KR (IKR) was carried out to predict the operational histories for spent fuel. All of regression algorithms were performed in R programming tool. From the validation of both AAKR and IKR, it is expected that kernel regression has enough accuracy to be applied to nuclear forensics for spent fuel. Once the nuclear substance is measured to fill the fission product inventory with a certain reactor type, this kernel regression algorithm would predict the operational histories of spent fuel with high accuracy. Finally, this information is expected to be useful to find the origin of irradiated materials. (authors)

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

Predicting Missing Regions in Charged Particle Tracks Using a Sparse 3D Convolutional Neural Network

The 2x2 Demonstrator is a prototype of ND-LAr, the liquid argon time-projection chamber of the Deep Underground Neutrino Experiment’s Near Detector complex. Both the 2x2 Demonstrator and ND-LAr are modular detectors that will have pixelated charge readouts and inactive regions wherein there is no sensitivity to charge deposition and light signals that arise from charged particle interactions with liquid argon. In the 2x2, these inactive regions are located in between the active detector modules, which introduces the challenge of inferring what charge signals ought to look like in these regions. This study explores the use of a Sparse 3D Convolutional Neural Network (ConvNet) to infer missing regions in charged particle tracks. Hits corresponding to energy depositions are voxelized into a three-dimensional grid for each track. Voxels that fall into predefined inactive regions are removed to simulate the lack of detector output. The model is trained to infer the topology of the missing track voxels, with the ultimate goal of inferring the missing charge or energy values in these voxels as well. Results indicate that this approach shows promise in prediction of missing track regions with some accuracy.

Utaegbulam, Hilary↗

Non-intrusive data-driven model reduction for differential–algebraic equations derived from lifting transformations

In this paper we present a non-intrusive data-driven approach for model reduction of nonlinear systems. The approach considers the particular case of nonlinear partial differential equations (PDEs) that form systems of partial differential–algebraic equations (PDAEs) when lifted to polynomial form. Such systems arise, for example, when the governing equations include Arrhenius reaction terms (e.g., in reacting flow models) and thermodynamic terms (e.g., the Helmholtz free energy terms in a phase-field solidification model). Using the known structured form of the lifted algebraic equations, the approach computes the reduced operators for the algebraic equations explicitly, using straightforward linear algebra operations on the basis matrices. The reduced operators for the differential equations are inferred from lifted snapshot data using operator inference, which solves a linear least squares regression problem. The approach is illustrated for the nonlinear model of solidification of a pure material. The lifting transformations reformulate the solidification PDEs as a system of PDAEs that have cubic structure. The operators of the lifted system for this solidification example have affine dependence on key process parameters, permitting us to learn a parametric reduced model with operator inference. Numerical experiments show the effectiveness of the resulting reduced models in capturing key aspects of the solidification dynamics.

42 ENGINEERING↗

Spike-and-Slab Shrinkage Priors for Structurally Sparse Bayesian Neural Networks

Network complexity and computational efficiency have become increasingly significant aspects of deep learning. Sparse deep learning addresses these challenges by recovering a sparse representation of the underlying target function by reducing heavily overparameterized deep neural networks. Specifically, deep neural architectures compressed via structured sparsity (e.g., node sparsity) provide low-latency inference, higher data throughput, and reduced energy consumption. In this article, we explore two well-established shrinkage techniques, Lasso and Horseshoe, for model compression in Bayesian neural networks (BNNs). To this end, we propose structurally sparse BNNs, which systematically prune excessive nodes with the following: 1) spike-and-slab group Lasso (SS-GL) and 2) SS group Horseshoe (SS-GHS) priors, and develop computationally tractable variational inference, including continuous relaxation of Bernoulli variables. We establish the contraction rates of the variational posterior of our proposed models as a function of the network topology, layerwise node cardinalities, and bounds on the network weights. Furthermore, we empirically demonstrate the competitive performance of our models compared with the baseline models in prediction accuracy, model compression, and inference latency.

97 MATHEMATICS AND COMPUTING↗