Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “variational inference”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3
All sourcesNASA NTRSDOE OSTI
DOE OSTI2019

Approximate Bayesian Model Inversion for PDEs with Heterogeneous and State-Dependent Coefficients

We present two approximate Bayesian inference methods for parameter estimation in partial differential equation (PDE) models with space-dependent and state-dependent parameters. We demonstrate that these methods provide accurate and cost-effective alternatives to Markov Chain Monte Carlo simulation. We assume a parameterized Gaussian prior on the unknown functions, and approximate the posterior density by a parameterized multivariate Gaussian density. The parameters of the prior and posterior are estimated from sparse observations of the PDE model's states and the unknown functions themselves by maximizing the evidence lower bound (ELBO), a lower bound on the log marginal likelihood of the observations. The first method, Laplace-EM, employs the expectation maximization algorithm to maximize the ELBO, with a Laplace approximation of the posterior on the E-step, and minimization of a Kullback-Leibler divergence on the M-step. The second method, DSVI-EB, employs the doubly stochastic variational inference (DSVI) algorithm, in which the ELBO is maximized via gradient-based stochastic optimization, with nosiy gradients computed via simple Monte Carlo sampling and Gaussian backpropagation. We apply these methods to identifying diffusion coefficients in linear and nonlinear diffusion equations, and we find that both methods provide accurate estimates of posterior densities and the hyperparameters of Gaussian priors. While the Laplace-EM method is more accurate, it requires computing Hessians of the physics model. The DSVI-EB method is found to be less accurate but only requires gradients of the physics model.

approximate Bayesian inference, model inversion, v↗
DOE OSTI2024

A differentiable simulation package for performing inference of synchrotron-radiation-based diagnostics

The direction of particle accelerator development is ever-increasing beam quality, currents and repetition rates. This poses a challenge to traditional diagnostics that directly intercept the beam due to the mutual destruction of both the beam and the diagnostic. An alternative approach is to infer beam parameters non-invasively from the synchrotron radiation emitted in bending magnets. However, inferring the beam distribution from a measured radiation pattern is a complex and computationally expensive task. To address this challenge we present SYRIPY ( SYnchrotron Radiation In PYthon ), a software package intended as a tool for performing inference of synchrotron-radiation-based diagnostics. SYRIPY has been developed using PyTorch , which makes it both differentiable and able to leverage the high performance of GPUs, two vital characteristics for performing statistical inference. The package consists of three modules: a particle tracker, Lienard–Wiechert solver and Fourier optics propagator, allowing start-to-end simulation of synchrotron radiation detection to be carried out. SYRIPY has been benchmarked against SRW , the prevalent numerical package in the field, showing good agreement and up to a 50× speed improvement. Finally, we have demonstrated how SYRIPY can be used to perform Bayesian inference of beam parameters using stochastic variational inference.

43 PARTICLE ACCELERATORS↗
DOE OSTI2016

Joint Data Filtering and Labeling Using Gaussian Processes and Alternating Direction Method of Multipliers

Here, sequence labeling aims at assigning a label to every sample of a signal (or pixel of an image) while considering the sequentiality (or vicinity) of the samples. To perform this task, many works in the literature first filter and then label the data. Unfortunately, the filtering, which is performed independently from the labeling, is far from optimal and frequently makes the latter task harder. In this paper, a novel approach that trains a Gaussian process classifier and estimates the coefficients of an optimal filter jointly is presented. The new approach, based on Bayesian modeling and alternating direction method of multipliers (ADMMs) optimization, performs both tasks simultaneously. All unknowns are treated as stochastic variables, which are estimated using variational inference and filtering and labeling are linked with the use of ADMM. In the experimental section, synthetic and real experiments are presented to compare the proposed method with other existing approaches.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Simultaneous global and local clustering in multiplex networks with covariate information

Understanding both global and layer-specific group structures is useful for uncovering complex patterns in networks with multiple interaction types. In this work, we introduce a new model, the hierarchical multiplex stochastic blockmodel, which simultaneously detects communities within individual layers of a multiplex network while inferring a global node clustering across the layers. A stochastic blockmodel is assumed in each layer, with probabilities of layer-level group memberships determined by a node’s global group assignment. Our model uses a Bayesian framework, employing a probit stick-breaking process to construct node-specific mixing proportions over a set of shared Griffiths–Engen–McCloseky distributions. These proportions determine layer-level community assignment, allowing for an unknown and varying number of groups across layers, while incorporating nodal covariate information to inform the global clustering. We propose a scalable variational inference procedure with parallelisable updates for application to large networks. Extensive simulation studies demonstrate our model’s ability to accurately recover both global and layer-level clusters in complicated settings, and applications to real data showcase the model’s effectiveness in uncovering interesting latent network structure.

community detection↗
DOE OSTI2024

Unsupervised physics-informed disentanglement of multimodal data

Here, we introduce physics-informed multimodal autoencoders (PIMA) - a variational inference framework for discovering shared information in multimodal datasets. Individual modalities are embedded into a shared latent space and fused through a product-of-experts formulation, enabling a Gaussian mixture prior to identify shared features. Sampling from clusters allows cross-modal generative modeling, with a mixture-of-experts decoder that imposes inductive biases from prior scientific knowledge and thereby imparts structured disentanglement of the latent space. This approach enables cross-modal inference and the discovery of features in high-dimensional heterogeneous datasets. Consequently, this approach provides a means to discover fingerprints in multimodal scientific datasets and to avoid traditional bottlenecks related to high-fidelity measurement and characterization of scientific datasets.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Multi-Modal Bayesian Neural Network Surrogates with Conjugate Last-Layer Estimation

As data collection and simulation capabilities advance, multi-modal learning, the task of learning from multiple modalities and sources of data, is becoming an increasingly important area of research. Surrogate models that learn from data of multiple auxiliary modalities to support the modeling of a highly expensive quantity of interest have the potential to aid outer loop applications such as optimization, inverse problems, or sensitivity analyses when multi-modal data are available. We develop two multi-modal Bayesian neural network surrogate models and leverage conditionally conjugate distributions in the last layer to estimate model parameters using stochastic variational inference (SVI). We provide a method to perform this conjugate SVI estimation in the presence of partially missing observations. Here, we demonstrate improved prediction accuracy and uncertainty quantification compared to unimodal surrogate models for both scalar and time series data.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2019

Scalable and efficient learning from crowds with Gaussian processes

Over the last few years, multiply-annotated data has become a very popular source of information. Online platforms such as Amazon Mechanical Turk have revolutionized the labelling process needed for any classification task, sharing the effort between a number of annotators (instead of the classical single expert). This crowdsourcing approach has introduced new challenging problems, such as handling disagreements on the annotated samples or combining the unknown expertise of the annotators. Probabilistic methods, such as Gaussian Processes (GP), have proven successful to model this new crowdsourcing scenario. However, GPs do not scale up well with the training set size, which makes them prohibitive for medium-to-large datasets (beyond 10K training instances). This constitutes a serious limitation for current real-world applications. In this work, we introduce two scalable and efficient GP-based crowdsourcing methods that allow for processing previously-prohibitive datasets. The first one is an efficient and fast approximation to GP with squared exponential (SE) kernel. The second allows for learning a more flexible kernel at the expense of a heavier training (but still scalable to large datasets). Since the latter is not a GP-SE approximation, it can be also considered as a whole new scalable and efficient crowdsourcing method, useful for any dataset size. Both methods use Fourier features and variational inference, can predict the class of new samples, and estimate the expertise of the involved annotators. A complete experimentation compares them with state-of-the-art probabilistic approaches in synthetic and real crowdsourcing datasets of different sizes. Finally, they stand out as the best performing approach for large scale problems. Moreover, the second method is competitive with the current state-of-the-art for small datasets.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
DOE OSTI2019

Conditional deep surrogate models for stochastic, high-dimensional, and multi-fidelity systems

Here, we present a probabilistic deep learning methodology that enables the construction of predictive data-driven surrogates for stochastic systems. Leveraging recent advances in variational inference with implicit distributions, we put forth a statistical inference framework that enables the end-to-end training of surrogate models on paired input–output observations that may be stochastic in nature, originate from different information sources of variable fidelity, or be corrupted by complex noise processes. The resulting surrogates can accommodate high-dimensional inputs and outputs and are able to return predictions with quantified uncertainty. Furthermore, the effectiveness our approach is demonstrated through a series of canonical studies, including the regression of noisy data, multi-fidelity modeling of stochastic processes, and uncertainty propagation in high-dimensional dynamical systems.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Spike-and-Slab Shrinkage Priors for Structurally Sparse Bayesian Neural Networks

Network complexity and computational efficiency have become increasingly significant aspects of deep learning. Sparse deep learning addresses these challenges by recovering a sparse representation of the underlying target function by reducing heavily overparameterized deep neural networks. Specifically, deep neural architectures compressed via structured sparsity (e.g., node sparsity) provide low-latency inference, higher data throughput, and reduced energy consumption. In this article, we explore two well-established shrinkage techniques, Lasso and Horseshoe, for model compression in Bayesian neural networks (BNNs). To this end, we propose structurally sparse BNNs, which systematically prune excessive nodes with the following: 1) spike-and-slab group Lasso (SS-GL) and 2) SS group Horseshoe (SS-GHS) priors, and develop computationally tractable variational inference, including continuous relaxation of Bernoulli variables. We establish the contraction rates of the variational posterior of our proposed models as a function of the network topology, layerwise node cardinalities, and bounds on the network weights. Furthermore, we empirically demonstrate the competitive performance of our models compared with the baseline models in prediction accuracy, model compression, and inference latency.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1971

Geomagnetic storm fields near a synchronous satellite.

An apparent early recovery of the main phase of geomagnetic storms at the distance of the synchronous satellite is examined in terms of changing electric current distributions in the magnetosphere during magnetic storms. It is suggested that a rapid recession of the edge of the plasma sheet (after the advance toward the earth during an early epoch of the main phase) is partly responsible for the early recovery. Relevant plasma sheet variations during geomagnetic storms are found to be in agreement with the inferred variations.

Kawasaki, K.↗
NASA NTRS1974

Comparison of atomic oxygen measurements by incoherent scatter and satellite-borne mass spectrometer techniques

Atomic oxygen densities determined by the incoherent scatter technique are compared to densities deduced from satellite-borne mass spectrometer measurements and are found to agree within experimental error. The diurnal variations inferred from the incoherent scatter measurements do show, however, some departure from diurnal variations found by modeling the mass spectrometer results. Some implications of these departures are briefly discussed.

Hedin, A. E.↗
DOE OSTI2025

Neural Posterior Estimation for Cataloging Astronomical Images with Spatially Varying Backgrounds and Point Spread Functions

Neural posterior estimation (NPE), a type of amortized variational inference, is a computationally efficient means of constructing probabilistic catalogs of light sources from astronomical images. To date, NPE has not been used to perform inference in models with spatially varying covariates. However, ground-based astronomical images exhibit spatially varying sky backgrounds and point spread functions (PSFs), and accounting for this variation is essential for constructing accurate catalogs of imaged light sources. In this work, we introduce a novel NPE-based cataloging method that trains an inference network with semisynthetic astronomical images generated using PSFs and backgrounds sampled from the Sloan Digital Sky Survey. In experiments with semisynthetic images, we evaluate the method on key cataloging tasks: light source detection, star/galaxy separation, and flux measurement. A “generalist” inference network—trained with diverse PSFs and backgrounds—performs as well as a “specialist” network even when both are evaluated on the specialist’s particular PSF/background combination. This result suggests that a single NPE network can generalize across spatial variations, eliminating the need for retraining on each observational condition.

astronomy image processing↗
DOE OSTI2024

Uncertainty quantification of graph convolution neural network models of evolving processes

The application of neural network models to scientific machine learning tasks has proliferated in recent years. In particular, neural networks have proved to be adept at modeling processes with spatial–temporal complexity. Nevertheless, these highly parameterized models have garnered skepticism in their ability to produce outputs with quantified error bounds over the regimes of interest. Hence there is a need to find uncertainty quantification methods that are suitable for neural networks. In this work we present comparisons of the parametric uncertainty quantification of neural networks modeling complex spatial–temporal processes with Hamiltonian Monte Carlo and Stein variational gradient descent and its projected variant. Specifically we apply these methods to graph convolutional neural network models of evolving systems modeled with recurrent neural network and neural ordinary differential equations architectures. We show that Stein variational inference is a viable alternative to Monte Carlo methods with some clear advantages for complex neural network models. For our exemplars, Stein variational interference gave similar pushed forward uncertainty profiles through time compared to Hamiltonian Monte Carlo, albeit with generally more generous variance. As a result, projected Stein variational gradient descent also produced similar uncertainty profiles to the non-projected counterpart, but large reductions in the active weight space were confounded by the stability of the neural network predictions and the convoluted likelihood landscape.

36 MATERIALS SCIENCE↗
DOE OSTI2021

PNNL-CompBio/bayesian-inference

Bayesian Metabolic Inference is a platform to integrate genome-scale multi-omics data into an iterative method to obtain optimized recommendations for optimal target compound production. This method uses a variational inference approach to approximate new posterior data to analyze in order to obtain useful correlations and control coefficients.

Kumar, Neeraj↗
NASA NTRS1975

Winter bulge and diurnal variations in hydrogen inferred from AE-C composition measurements

In situ mass spectrometric measurements of ion and neutral particle thermospheric compositions have been used to infer the latitudinal and diurnal variations of thermospheric atomic hydrogen for solstice conditions. Local time-dependent and local time-independent components of the observed hydrogen distribution were separated on the basis of a model generated by expanding the log of neutral hydrogen concentration in terms of spherical harmonics. Results are compared with analogous data on N2 concentration; the comparison reveals an anticorrelation between gas temperature and hydrogen concentration. The slope of this anticorrelation line represents the 'zero flux condition' from the exosphere theory of Hodges (1973), thus further bearing out the conclusion that exospheric flow is the dominant process governing the global distribution of hydrogen.

Brinton, H. C.↗
DOE OSTI2022

GIGA-Lens: Fast Bayesian Inference for Strong Gravitational Lens Modeling

We present GIGA-Lens: a gradient-informed, GPU-accelerated Bayesian framework for modeling strong gravitational lensing systems, implemented in TensorFlow and JAX. The three components, optimization using multistart gradient descent, posterior covariance estimation with variational inference, and sampling via Hamiltonian Monte Carlo, all take advantage of gradient information through automatic differentiation and massive parallelization on graphics processing units (GPUs). We test our pipeline on a large set of simulated systems and demonstrate in detail its high level of performance. The average time to model a single system on four Nvidia A100 GPUs is 105 s. The robustness, speed, and scalability offered by this framework make it possible to model the large number of strong lenses found in current surveys and present a very promising prospect for the modeling of ${ \mathcal O }({10}^{5})$ lensing systems expected to be discovered in the era of the Vera C. Rubin Observatory, Euclid, and the Nancy Grace Roman Space Telescope.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2020

Single Episode Policy Transfer in Reinforcement Learning

Transfer and adaptation to new unknown environmental dynamics is a key challenge for reinforcement learning (RL). An even greater challenge is performing near-optimally in a single attempt at test time, possibly without access to dense rewards, which is not addressed by current methods that require multiple experience rollouts for adaptation. To achieve single episode transfer in a family of environments with related dynamics, we propose a general algorithm that optimizes a probe and an inference model to rapidly estimate underlying latent variables of test dynamics, which are then immediately used as input to a universal control policy. This modular approach enables integration of state-of-the-art algorithms for variational inference or RL. Moreover, our approach does not require access to rewards at test time, allowing it to perform in settings where existing adaptive approaches cannot. In diverse experimental domains with a single episode test constraint, we report our method significantly outperforms existing adaptive approaches and shows favorable performance against baselines for robust transfer.

42 ENGINEERING↗