Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Variational inference”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Projective Integral Updates for High-Dimensional Variational Inference

Variational inference is an approximation framework for Bayesian inference that seeks to improve quantified uncertainty in predictions by optimizing a simplified distribution over parameters to stand in for the full posterior. Capturing model variations that remain consistent with training data enables more robust predictions by reducing parameter sensitivity. This work introduces a fixed-point optimization for variational inference that is applicable when every feasible log density can be expressed as a linear combination of functions from a given basis. In such cases, the optimizer becomes a fixed-point of projective integral updates. When the basis spans univariate quadratics in each parameter, the feasible distributions are Gaussian mean-fields and the projective integral updates yield quasi-Newton variational Bayes (QNVB). Other bases and updates are also possible. Since these updates require high-dimensional integration, this work begins by proposing an efficient quasirandom sequence of quadratures for mean-field distributions. Each iterate of the sequence contains two evaluation points that combine to correctly integrate all univariate quadratic functions and, if the mean-field factors are symmetric, all univariate cubics. More importantly, averaging results over short subsequences achieves periodic exactness on a much larger space of multivariate polynomials of quadratic total degree. The corresponding variational updates require four loss evaluations with standard (not second-order) backpropagation to eliminate error terms from over half of all multivariate quadratic basis functions. Furthermore, this integration technique is motivated by first proposing stochastic blocked mean-field quadratures, which may be useful in other contexts. A PyTorch implementation of QNVB allows for better control over model uncertainty during training than competing methods. Experiments demonstrate superior generalizability for multiple learning problems and architectures.

Gaussian mean-field↗
DOE OSTI2023

Robust scalable initialization for Bayesian variational inference with multi-modal Laplace approximations

Predictive modeling typically relies on Bayesian model calibration to provide uncertainty quantification. Variational inference utilizing fully independent (“mean-field”) Gaussian distributions are often used as approximate probability density functions. This simplification is attractive since the number of variational parameters grows only linearly with the number of unknown model parameters. However, the resulting diagonal covariance structure and unimodal behavior can be too restrictive to provide useful approximations of intractable Bayesian posteriors that exhibit highly non-Gaussian behavior, including multimodality. High-fidelity surrogate posteriors for these problems can be obtained by considering the family of Gaussian mixtures. Gaussian mixtures are capable of capturing multiple modes and approximating any distribution to an arbitrary degree of accuracy, while maintaining some analytical tractability. Unfortunately, variational inference using Gaussian mixtures with full-covariance structures suffers from a quadratic growth in variational parameters with the number of model parameters. The existence of multiple local minima due to strong nonconvex trends in the loss functions often associated with variational inference present additional complications, These challenges motivate the need for robust initialization procedures to improve the performance and computational scalability of variational inference with mixture models. In this work, we propose a method for constructing an initial Gaussian mixture model approximation that can be used to warm-start the iterative solvers for variational inference. The procedure begins with a global optimization stage in model parameter space. In this step, local gradient-based optimization, globalized through multistart, is used to determine a set of local maxima, which we take to approximate the mixture component centers. Around each mode, a local Gaussian approximation is constructed via the Laplace approximation. Finally, the mixture weights are determined through constrained least squares regression. The robustness and scalability of the proposed methodology is demonstrated through application to an ensemble of synthetic tests using high-dimensional, multimodal probability density functions. Here, the practical aspects of the approach are demonstrated with inversion problems in structural dynamics.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Scaling and merging time-resolved pink-beam diffraction with variational inference

Time-resolved x-ray crystallography (TR-X) at synchrotrons and free electron lasers is a promising technique for recording dynamics of molecules at atomic resolution. While experimental methods for TR-X have proliferated and matured, data analysis is often difficult. Extracting small, time-dependent changes in signal is frequently a bottleneck for practitioners. Recent work demonstrated this challenge can be addressed when merging redundant observations by a statistical technique known as variational inference (VI). However, the variational approach to time-resolved data analysis requires identification of successful hyperparameters in order to optimally extract signal. In this case study, we present a successful application of VI to time-resolved changes in an enzyme, DJ-1, upon mixing with a substrate molecule, methylglyoxal. We present a strategy to extract high signal-to-noise changes in electron density from these data. Furthermore, we conduct an ablation study, in which we systematically remove one hyperparameter at a time to demonstrate the impact of each hyperparameter choice on the success of our model. We expect this case study will serve as a practical example for how others may deploy VI in order to analyze their time-resolved diffraction data.

47 OTHER INSTRUMENTATION↗
DOE OSTI2023

Industrial PLC Network Modeling and Parameter Identification Using Sensitivity Analysis and Mean Field Variational Inference

A multiple input multiple output (MIMO) power line communication (PLC) model for industrial facilities was developed that uses the physics of a bottom-up model but can be calibrated like top-down models. The PLC model considers 4-conductor cables (three-phase conductors and a ground conductor) and has several load types, including motor loads. The model is calibrated to data using mean field variational inference with a sensitivity analysis to reduce the parameter space. The results show that the inference method can accurately identify many of the model parameters, and the model is accurate even when the network is modified.

47 OTHER INSTRUMENTATION↗
DOE OSTI2022

A Variational Inference Approach to Inverse Problems with Gamma Hyperpriors

Hierarchical models with gamma hyperpriors provide a flexible, sparse-promoting framework to bridge L 1 and L 2 regularizations in Bayesian formulations to inverse problems. Despite the Bayesian motivation for these models, existing methodologies are limited to maximum a posteriori estimation. The potential to perform uncertainty quantification has not yet been realized. This paper introduces a variational iterative alternating scheme for hierarchical inverse problems with gamma hyperpriors. The proposed variational inference approach yields accurate reconstruction, provides meaningful uncertainty quantification, and is easy to implement. In addition, it lends itself naturally to conduct model selection for the choice of hyperparameters. Here, we illustrate the performance of our methodology in several computed examples, including a deconvolution problem and sparse identification of dynamical systems from time series data.

Bayesian shrinkage↗
DOE OSTI2024

Variational inference of effective range parameters for 3 He− 4 He scattering

We use two different methods, Monte Carlo sampling and variational inference (VI), to perform a Bayesian calibration of the effective-range parameters in 3 He– 4 He elastic scattering. The parameters are calibrated to data from a recent set of 3 He– 4 He elastic scattering differential cross section measurements. Analysis of these data for E lab ≤ 4.3 MeV yields a unimodal posterior for which both methods obtain the same structure. However, the effective-range expansion amplitude does not account for the 7/2 − state of 7 Be so, even after calibration, the description of data at the upper end of this energy range is poor. The data up to E lab = 2.6 MeV can be well described, but calibration to this lower-energy subset of the data yields a bimodal posterior. After adapting VI to treat such a multi-modal posterior we find good agreement between the VI results and those obtained with parallel-tempered Monte Carlo sampling.

effective field theory↗
NASA NTRS1994

Dynamical influences on the moment of inertia tensor from lateral viscosity variations inferred from seismic tomographic models

We have investigated the influences of lateral variations of viscosity on the moment of inertia tensor from viscous flows due to the density anomalies in the mantle inferred from seismic tomographic models. The scaling relations between the density and the seismic anomalies is taken as either a constant or a function increasing with depth in accord with the recent high-pressure experimental studies. The viscosity is taken as an exponential function of the 3D density anomaly. In models with an isoviscous background, the effects on the perturbed moment of inertia tensor from the lateral viscosity variations are smaller than those due to variations in the radial viscosity profiles. In mantle models with a background viscosity increasing with depth, the influences of the lateral viscosity variations are significant. The most striking feature in the latter case is that the two off-diagonal elements delta I(sub xz) and delta I(sub yz) in the inertia tensor exhibit greatest sensitivity to lateral variations of the viscosity. While the other elements of the inertia change by only about a few tens of percent in the range of lateral viscosity contrast considered (less than 300), delta I(sub xz) and delta I(sub yz) can vary up to 40 times even with a change in sign, depending on the radial viscosity stratification and the location of the strongest lateral variations. The increase in the velocity-density scaling relation with depth can reduce the influences of the lateral viscosity variations, but it does not change the overall sensitive nature of delta I(sub xz) and delta I(sub yz). This study demonstrates clearly that the lateral viscosity variations, especially in the upper mantle, must be considered in the determination of long-term polar wander, since the variations in the delta I(sub xz) and delta I(sub yz) terms are directly responsible for exciting rotational movements.

Zhang, Shuxia↗
NASA NTRS1985

Dynamics of two-year cosmic ray variations inferred from the data of spacecraft and stratospheric measurements and from the neutron monitor data in 1959-1981

The two year cosmic ray variations are studied using the spacecraft measurements of 1967 to 1976, the sonde measurements at high latitudes in the stratosphere (Murmansk, Mirny), and the neutron monitor data of 1959 to 1981. The two year variations are most pronounced from 1967 to 1975. An anticorrelation is observed between the two year variations in cosmic rays and in geomagnetic activity.

Charakhchyan, T. N.↗
NASA NTRS2005

Zonal-Mean Temperature Variations Inferred from SABER Measurements on TIMED Compared with UARS Observations

In the Numerical Spectral Model (NSM, Mayr et al., 2003), small-scale gravity waves propagating in the north/south direction can generate zonal mean (m = 0) meridional wind oscillations with periods between 2 and 4 months. These oscillations tend to be confined to low latitudes and have been interpreted to be the meridional counterpart of the wave-driven Quasi Biennial Oscillation in the zonal circulation. Wave driven meridional winds across the equator should generate, due to dynamical heating and cooling, temperature oscillations with opposite phase in the two hemispheres. We have analyzed SABER temperature measurements in the altitude range between 55 and 95 km to investigate the existence such variations. Because there are also strong tidal signatures (up to approximately 20 K) in the data, our algorithm estimates both mean values and tides together from the data. Based on SABER temperature data, the intra-annual variations with periods between 2 and 4 months can have amplitudes up to 5 K or more, depending on the altitude. Their amplitudes are in qualitative agreement with those inferred Erom UARS data (from different years). The SABER temperature variations also reveal pronounced hemispherical asymmetries, which are qualitatively consistent with wave driven meridional wind oscillations across the equator. Oscillations with similar periods have been seen in the meridional winds based on UARS data (Huang and Reber, 2003).

Huang, Frank T.↗
NASA NTRS2001

Horizontal Temperature Variability in the Stratosphere: Global Variations Inferred from CRISTA Data

In two separate orbital campaigns (November, 1994 and August, 1997), the Cryogenic Infrared Spectrometers and Telescopes for the Atmosphere (CRISTA) instrument acquired global stratospheric data of high accuracy and high spatial resolution. The standard limb-scanned CRISTA measurements resolved atmospheric spatial structures with vertical dimensions greater than or equal to 1.5 - 2 km and horizontal dimensions is greater than or equal to 100 - 200 km. A fluctuation analysis of horizontal temperature distributions derived from these data is presented. This method is somewhat complementary to conventional power-spectral analysis techniques.

FLUCTUATION ANALYSIS↗