Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Variational Bayesian”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4

Ensemble variational Fokker-Planck methods for data assimilation

Particle flow filters solve Bayesian inference problems by smoothly transforming a set of particles into samples from the posterior distribution. Particles move in state space under the flow of an McKean-Vlasov-Itˆo process. This work introduces the Variational Fokker-Planck (VFP) framework for data assimilation, a general approach that includes previously known particle flow filters as special cases. The McKean-Vlasov-Itˆo process that transforms particles is defined via an optimal drift that depends on the selected diffusion term. It is established that the underlying probability density - sampled by the ensemble of particles - converges to the Bayesian posterior probability density. For a finite number of particles the optimal drift contains a regularization term that nudges particles toward becoming independent random variables. Based on this analysis, we derive computationally-feasible approximate regularization approaches that penalize the mutual information between pairs of particles, and avoid particle collapse. Moreover, the diffusion plays a role akin to a particle rejuvenation approach that aims to alleviate particle collapse. The VFP framework is very flexible. Different assumptions on prior and intermediate probability distributions can be used to implement the optimal drift, and localization and covariance shrinkage can be applied to alleviate the curse of dimensionality. A robust implicit-explicit method is discussed for the efficient integration of stiff McKean- Vlasov-Itˆo processes. Here, the effectiveness of the VFP framework is demonstrated on three progressively more challenging test problems, namely the Lorenz ’63, Lorenz ’96 and the quasi-geostrophic equations.

97 MATHEMATICS AND COMPUTING↗

A differentiable simulation package for performing inference of synchrotron-radiation-based diagnostics

The direction of particle accelerator development is ever-increasing beam quality, currents and repetition rates. This poses a challenge to traditional diagnostics that directly intercept the beam due to the mutual destruction of both the beam and the diagnostic. An alternative approach is to infer beam parameters non-invasively from the synchrotron radiation emitted in bending magnets. However, inferring the beam distribution from a measured radiation pattern is a complex and computationally expensive task. To address this challenge we present SYRIPY ( SYnchrotron Radiation In PYthon ), a software package intended as a tool for performing inference of synchrotron-radiation-based diagnostics. SYRIPY has been developed using PyTorch , which makes it both differentiable and able to leverage the high performance of GPUs, two vital characteristics for performing statistical inference. The package consists of three modules: a particle tracker, Lienard–Wiechert solver and Fourier optics propagator, allowing start-to-end simulation of synchrotron radiation detection to be carried out. SYRIPY has been benchmarked against SRW , the prevalent numerical package in the field, showing good agreement and up to a 50× speed improvement. Finally, we have demonstrated how SYRIPY can be used to perform Bayesian inference of beam parameters using stochastic variational inference.

43 PARTICLE ACCELERATORS↗

Uncertainties in Parameters Estimated with Neural Networks: Application to Strong Gravitational Lensing

In Hezaveh et al. (2017) we showed that deep learning can be used for model parameter estimation and trained convolutional neural networks to determine the parameters of strong gravitational lensing systems. Here we demonstrate a method for obtaining the uncertainties of these parameters. We review the framework of variational inference to obtain approximate posteriors of Bayesian neural networks and apply it to a network trained to estimate the parameters of the Singular Isothermal Ellipsoid plus external shear and total flux magnification. We show that the method can capture the uncertainties due to different levels of noise in the input data, as well as training and architecture-related errors made by the network. To evaluate the accuracy of the resulting uncertainties, we calculate the coverage probabilities of marginalized distributions for each lensing parameter. By tuning a single hyperparameter, the dropout rate, we obtain coverage probabilities approximately equal to the confidence levels for which they were calculated, resulting in accurate and precise uncertainty estimates. Our results suggest that neural networks can be a fast alternative to Monte Carlo Markov Chains for parameter uncertainty estimation in many practical applications, allowing more than seven orders of magnitude improvement in speed.

Perreault Levasseur, Laurence↗

Bayesian modeling of inconsistent plastic response due to material variability

The advent of fabrication techniques such as additive manufacturing has focused attention on the considerable variability of material response due to defects and other microstructural aspects. This variability motivates the development of an enhanced design methodology that incorporates inherent material variability to provide robust predictions of performance. In this work, we develop plasticity models capable of representing the distribution of mechanical responses observed in experiments using traditional plasticity models of the mean response and recently developed uncertainty quantification (UQ) techniques. To account for material response variability through variations in physical parameters, we adapt a recent Bayesian embedded modeling error calibration technique. We use Bayesian model selection to determine the most plausible of a variety of plasticity models and the optimal embedding of parameter variability. To expedite model selection, we develop an adaptive importance-sampling-based numerical integration scheme to compute the Bayesian model evidence. In conclusion, we demonstrate that the new framework provides predictive realizations that are superior to more traditional ones, and how these UQ techniques can be used in model selection and assessing the quality of calibrated physical parameters.

36 MATERIALS SCIENCE↗

A rotorcraft in-flight ice detection framework using computational aeroacoustics and Bayesian neural networks

Abstract This work develops a novel ice detection framework specifically suitable for rotorcraft using computational aeroacoustics and Bayesian neural networks. In an offline phase of the work, the acoustic signature of glaze and rime ice shapes on an oscillating wing are computed. In addition, the aerodynamic performance indicators corresponding to the ice shapes are also monitored. These performance indicators include the lift, drag, and moment coefficients. A Bayesian neural network is subsequently trained using projected Stein variational gradient descent to create a mapping from the acoustic signature generated by the iced wings to predict their performance indicators along with quantified uncertainty that is highly important for time- and safety-critical decision-making scenarios. While the training is carried out fully offline, usage of the Bayesian neural network to make predictions can be conducted rapidly online allowing for an ice detection system that can be used in real time and in-flight.

42 ENGINEERING↗

CONCURRENT, CONDENSED STEIN VARIATIONAL GRADIENT DESCENT FOR UNCERTAINTY QUANTIFICATION OF NEURAL NETWORKS

In this work, we propose a Stein variational gradient descent (SVGD) method to concurrently sparsify, train, and provide uncertainty quantification (UQ) of a complexly parameterized model, such as a neural network (NN). It employs a graph reconciliation and condensation process to reduce complexity and increase similarity in the Stein ensemble of parameterizations. Therefore, the proposed concurrent, condensed SVGD (ccSVGD) method can provide UQ on parameters, not just outputs. Furthermore, the parameter reduction speeds up the convergence of the Stein gradient descent as it reduces the combinatorial complexity by aligning and differentiating the sensitivity to parameters. These properties are demonstrated with an illustrative example and an application to a mechanical response representation problem in solid mechanics.

42 ENGINEERING↗

Application of a Geochemically Informed Leak Detection (GILD) Model to CO 2 Injection Sites on the United States Gulf Coast

The Gulf Coast region possesses great potential for CO 2 enhanced oil recovery (EOR) and CO 2 storage. A geochemically informed leak detection (GILD) model has been applied to CO 2 injection sites on the Gulf Coast with considerations of measurement variability. The Jasper aquifer in Montgomery County, Texas, was chosen to demonstrate the method. Based on background data from wells in the area, combinations of mineral and fluid compositions were used to create 23 scenarios for the geochemical model. The output from the geochemical model was used to identify sensitive monitoring species, and response functions were generated for these as a function of the CO 2 leakage concentration. The sources of measurement variability for background conditions were characterized from the Jasper aquifer background data, and then normalized using the coefficient of variation of each species across the monitoring wells. Bayesian belief network (BBN) models were constructed, and measurement variability of different levels were added to compare leak detection probabilities. Increasing measurement variability decreased the power to detect a leak of a given size. For a moderately high CO 2 concentration of 0.2 mol/kg, the probability of detecting this leakage effect using pH as the monitoring variable in an aquifer with calcite decreases from 98% (no measurement variability) to 61% (medium variability) to 33% (high variability). The loss in power of the sampling protocol with increasing measurement variability is similar in magnitude when Ca 2+ or HCO 3 - is used as the monitoring parameter, but only for aquifers with calcite.

58 GEOSCIENCES↗

Hydro-scaling of direct-drive cylindrical implosions at the OMEGA and the National Ignition Facility

Deceleration-phase Rayleigh–Taylor instability (RTI) growth during inertial confinement fusion capsule implosions significantly affects the performance as it mixes cold ablator material into the fuel. Precise measurements of such instability growth are essential for both validating the existing simulation codes and improving our predictive capability. RTI measurements on the inner surface of a spherical shell are limited and are often inferred indirectly at limited convergence. In contrast, cylindrical implosions allow for direct diagnostic access to the converging interface by imaging down the cylinder axis while retaining the effects of convergence. We have performed direct-drive cylindrical implosion experiments at both the OMEGA and the NIF laser facilities using scaled targets. RTI growth is demonstrated to be scale-invariant between the cylindrical targets at OMEGA and similar targets at the NIF, which are scaled up by a factor of three in the radial dimension. Single-mode (m = 0) instability growth factors of ~14 are measured at a convergence ratio (CR) ~ 2.5 with nearly identical mode growth at both scales. The measurements are in agreement with xRAGE radiation-hydrodynamics simulations. In addition, we have developed the Bayesian-inference-engine method to account for the variations in the target alignment, magnification, and the parallax effect in the measurement, allowing a more precise comparison between the experimental data and the simulations.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Quantification of neural networks uncertainties with applications to SAFARI-1 axial neutron flux profiles

Deep Neural Networks (DNNs) have been widely used as a data-driven modelling tool in nuclear engineering. However, as a Machine Learning model, Artificial Neural Network (ANN) predictions are subjected to uncertainties originating from the noise in training data, incomplete coverage of the domain, and imperfect neural network architectures. In this work, we target at quantifying the prediction/approximation uncertainties of ANNs using Monte Carlo Dropout (MCD), as well as Bayesian Neural Networks (BNNs) which are solved by variational inference. With a demonstration problem in which neural networks are used to predict the assembly axial neutron flux profiles, the results have shown that the three different neural network models (regular DNNs, DNNs solved with MCD and BNNs) can produce results that agree very well among each other and with the measurement data, on cycles that are not used in the training process. Besides the excellent generalization capability, the uncertainty bands produced by MCD and BNN agree very well, and in general, they can fully envelop the noisy measurement data points. (authors)

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

Observation of CH4 and other Non-CO2 Green House Gas Emissions from California

In 2006, California passed the landmark assembly bill AB-32 to reduce California's emissions of greenhouse gases (GHGs) that contribute to global climate change. AB-32 commits California to reduce total GHG emissions to 1990 levels by 2020, a reduction of 25 percent from current levels. To verify that GHG emission reductions are actually taking place, it will be necessary to measure emissions. We describe atmospheric inverse model estimates of GHG emissions obtained from the California Greenhouse Gas Emissions Measurement (CALGEM) project. In collaboration with NOAA, we are measuring the dominant long-lived GHGs at two tall-towers in central California. Here, we present estimates of CH{sub 4} emissions obtained by statistical comparison of measured and predicted atmospheric mixing ratios. The predicted mixing ratios are calculated using spatially resolved a priori CH{sub 4} emissions and surface footprints, that provide a proportional relationship between the surface emissions and the mixing ratio signal at tower locations. The footprints are computed using the Weather Research and Forecast (WRF) coupled to the Stochastic Time-Inverted Lagrangian Transport (STILT) model. Integral to the inverse estimates, we perform a quantitative analysis of errors in atmospheric transport and other factors to provide quantitative uncertainties in estimated emissions. Regressions of modeled and measured mixing ratios suggest that total CH{sub 4} emissions are within 25% of the inventory estimates. A Bayesian source sector analysis obtains posterior scaling factors for CH{sub 4} emissions, indicating that emissions from several of the sources (e.g., landfills, natural gas use, petroleum production, crops, and wetlands) are roughly consistent with inventory estimates, but livestock emissions are significantly higher than the inventory. A Bayesian 'region' analysis is used to identify spatial variations in CH{sub 4} emissions from 13 sub-regions within California. Although, only regions near the tower are significantly constrained by the tower measurements, CH{sub 4} emissions from the south Central Valley appear to be underestimated in a manner consistent with the under-prediction of livestock emissions. Finally, we describe a pseudo-experiment using predicted CH{sub 4} signals to explore the uncertainty reductions that might be obtained if additional measurements were made by a future network of tall-tower stations spread over California. These results show that it should be possible to provide high-accuracy estimates of surface CH{sub 4} emissions for multiple regions as a means to verify future emissions reductions.

54 ENVIRONMENTAL SCIENCES↗

Learning from crowds with variational Gaussian processes

Solving a supervised learning problem requires to label a training set. This task is traditionally performed by an expert, who provides a label for each sample. The proliferation of social web services (e.g., Amazon Mechanical Turk) has introduced an alternative crowdsourcing approach. Anybody with a computer can register in one of these services and label, either partially or completely, a dataset. The effort of labeling is then shared between a great number of annotators. However, this approach introduces scientifically challenging problems such as combining the unknown expertise of the annotators, handling disagreements on the annotated samples, or detecting the existence of spammer and adversarial annotators. All these problems require probabilistic sound solutions which go beyond the naive use of majority voting plus classical classification methods. In this work we introduce a new crowdsourcing model and inference procedure which trains a Gaussian Process classifier using the noisy labels provided by the annotators. Variational Bayes inference is used to estimate all unknowns. The proposed model can predict the class of new samples and assess the expertise of the involved annotators. Moreover, the Bayesian treatment allows for a solid uncertainty quantification. Since when predicting the class of a new sample we might have access to some annotations for it, we also show how our method can naturally incorporate this additional information. Furthermore, a comprehensive experimental section evaluates the proposed method with synthetic and real experiments, showing that it consistently outperforms other state-of-the-art crowdsourcing approaches.

97 MATHEMATICS AND COMPUTING↗

Designing an Optimal Sensor Network via Minimizing Information Loss

Optimal experimental design is a classic topic in statistics, with many well-studied problems, applications, and solutions. The design problem we study is the placement of sensors to monitor spatiotemporal processes, explicitly accounting for the temporal dimension in our modeling and optimization. We observe that recent advancements in computational sciences often yield large datasets based on physics-based simulations, which are rarely leveraged in experimental design. We introduce a novel model-based sensor placement criterion, along with a highly-efficient optimization algorithm, which integrates physics-based simulations and Bayesian experimental design principles to identify sensor networks that “minimize information loss” from simulated data. Our technique relies on sparse variational inference and (separable) Gauss-Markov priors, and thus may adapt many techniques from Bayesian experimental design. We validate our method through a case study monitoring air temperature in Phoenix, Arizona, using state-of-the-art physics-based simulations. Our results show our framework to be superior to random or quasi-random sampling, particularly with a limited number of sensors. We conclude by discussing practical considerations and implications of our framework, including more complex modeling tools and real-world deployments.

54 ENVIRONMENTAL SCIENCES↗

Optimizing Optical Searches for Supermassive Black Hole Binaries in Active Galactic Nuclei Light Curves: Fourier versus Bayesian Periodicity Detection

Simulations predict that supermassive black hole binaries (SMBHBs) will exhibit periodic brightness variations that may exceed the stochastic variability intrinsic to active galactic nuclei (AGN). In this paper, we simulate SMBHBs with damped random walk (DRW) AGN variability and an added sinusoidal signal from the orbital motion, and test three methods—a generalized Lomb–Scargle periodogram (GLSP), a nested Bayesian sampler (NBS), and a weighted wavelet z-transform (or WWZ)—to determine which is best at recovering the periodicity. Our simulated light curves follow the properties of the Catalina Real-Time Transient Survey (or CRTS), Legacy Survey of Space and Time (LSST), and Zwicky Transient Facility (ZTF) to best inform current and future SMBHB searches. We map a broad range of parameter space and identify which DRW-only light curves best mimic periodicity and pass each method’s model selection. The NBS performs best at detecting periodicity and filtering out DRW-only light curves. Combined candidate selection with both the NBS and GLSP significantly reduces false-positive rates (FPRs) with marginal impact on true-positive rates (TPRs). With this joint model selection pipeline, we find the lowest FPRs in ZTF-like simulations and the highest detection rates in LSST-like simulations. Using a modified computation of the false-alarm probability with GLSP, we efficiently triage LSST AGN light curves (∼10 7 light curves in ∼10–30 hr) and achieve TPRs and FPRs of ∼40% and ∼0.5%, respectively.

Banaszak, Sebastian M. [Vanderbilt Univ., Nashvill↗

Image processing of multiphase images obtained via X-ray microtomography: A review

Easier access to X-ray microtomography (μCT) facilities has provided much new insight from high-resolution imaging for various problems in porous media research. Pore space analysis with respect to functional properties usually requires segmentation of the intensity data into different classes. Image segmentation is a nontrivial problem that may have a profound impact on all subsequent image analyses. This review deals with two issues that are neglected in most of the recent studies on image segmentation: (i) focus on multiclass segmentation and (ii) detailed descriptions as to why a specific method may fail together with strategies for preventing the failure by applying suitable image enhancement prior to segmentation. In this way, the presented algorithms become very robust and are less prone to operator bias. Three different test images are examined: a synthetic image with ground-truth information, a synchrotron image of precision beads with three different fluids residing in the pore space, and a μCT image of a soil sample containing macropores, rocks, organic matter, and the soil matrix. Image blur is identified as the major cause for poor segmentation results. Other impairments of the raw data like noise, ring artifacts, and intensity variation can be removed with current image enhancement methods. Bayesian Markov random field segmentation, watershed segmentation, and converging active contours are well suited for multiclass segmentation, yet with different success to correct for partial volume effects and conserve small image features simultaneously.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

A stochastic approach to quantifying the blur with uncertainty estimation for high-energy X-ray imaging systems

One of the primary causes of blur in a high-energy X-ray imaging system is the shape and extent of the radiation source, or ‘spot’. It is important to be able to quantify the size of the spot as it provides a lower bound on the recoverable resolution for a radiograph, and penumbral imaging methods – which involve the analysis of blur caused by a structured aperture – can be used to obtain the spot’s spatial profile. We present a Bayesian approach for estimating the spot shape that, unlike variational methods, is robust to the initial choice of parameters. The posterior is obtained from a normal likelihood, which was constructed from a weighted least squares approximation to a Poisson noise model, and prior assumptions that enforce both smoothness and non-negativity constraints. A Markov chain Monte Carlo algorithm is used to obtain samples from the target posterior, and the reconstruction and uncertainty estimates are the computed mean and variance of the samples, respectively. Lastly, synthetic data-sets are used to demonstrate accurate reconstruction, while real data taken with high-energy X-ray imaging systems are used to demonstrate applicability and feasibility.

97 MATHEMATICS AND COMPUTING↗

Adaptive Discovery and Mixed-Variable Optimization of Next Generation Synthesizable Microelectronic Materials

Design of new microelectronic materials is characterized by several challenges such as high-dimensionality of the atomic structure-composition variable space, formidable cost of directly using high-fidelity simulations for design optimization, dispersity in literature-reported similar materials and synthesis methods, complex physical mechanisms, and mixed qualitative and quantitative design variables that lead to a disjointed design space. Even though machine learning (ML) techniques have been employed to expedite materials innovation, existing methods treat ML and design optimization as two separate processes, failing to resolve the fundamental challenges associated with high dimensionality and mixed-variable complexity. We have developed a ML enhanced mixed-variable material design optimization framework to efficiently extract useful information from existing data in literature and physics-based simulations to guide the autonomous search for optimal materials. Our proposed framework is composed of four computational modules: (1) a natural language processing (NLP) based virtual screening module, (2) classification based concept exploration module, (3) a density functional theory (DFT)-based high-fidelity evaluation model, and (4) a novel latent-variable Gaussian process (LVGP) ML model for mixed-variable problems with uncertainty quantification, which seamlessly integrates with Bayesian Optimization (BO) and achieves superb efficiency through embedded physics-based dimension reduction. Our approach is demonstrated and validated using the testbed of functional materials exhibiting metal-insulation transitions (MITs), with the targeted reversible resistivity changes (∼10^5) near room temperature. At the end of the 30-month project, we have developed a series of new ML techniques using NLP, conditional variational autoencoders, active learning, latent-variable Gaussian processes, integrated with Bayesian optimization. Our project has resulted in new predicted MITs compounds and improved understanding of MITs microscopic mechanisms, which in turn will revolutionize microelectronics science to provide energy-saving solutions. Our research has improved both creativity and efficiency in transforming rare-event discoveries of new functional materials to persistent innovations. In addition to open-sourcing the online MIT database and the classification model, the LVGP open source code has been downloaded more than 15,000 times within two years. More than 40 MIT compounds have been identified and many have been pursued experimentally via collaborators. The research results are published in close to 20 collaborative papers in high-impact journals, such as Chem. Mater., Appl. Phys. Rev., Sci. Rep., among others of design space.

36 MATERIALS SCIENCE↗

A bayesian approach to estimate the size and structure of the broad-line region in active galactic nuclei using reverberation mapping data

This is the first paper in a series devoted to the systematic study of the size and structure of the broad-line region (BLR) in active galactic nuclei (AGNs) using reverberation mapping (RM) data. We employ a recently developed Bayesian approach that statistically describes the variability as a damped random walk process and delineates the BLR structure using a flexible disk geometry that can account for a variety of shapes, including disks, rings, shells, and spheres. We allow for the possibility that the line emission may respond non-linearly to the continuum, and we detrend the light curves when there is clear evidence for secular variation. We use a Markov Chain Monte Carlo implementation based on Bayesian statistics to recover the parameters and uncertainties for the BLR model. The corresponding transfer function is obtained self-consistently. We tentatively constrain the virial factor used to estimate black hole masses; more accurate determinations will have to await velocity-resolved RM data. Application of our method to RM data with Hβ monitoring for about 40 objects shows that the assumed BLR geometry can reproduce quite well the observed emission-line fluxes from the continuum light curves. We find that the Hβ BLR sizes obtained from our method are on average ∼20% larger than those derived from the traditional cross-correlation method. Nevertheless, we still find a tight BLR size-luminosity relation with a slope of α = 0.55 ± 0.03 and an intrinsic scatter of ∼0.18 dex. In particular, we demonstrate that our approach yields appropriate BLR sizes for some objects (such as Mrk 142 and PG 2130+099) where traditional methods previously encountered difficulties.

79 ASTRONOMY AND ASTROPHYSICS↗

A scalable variational method for estimating the latent infection-rate field of an outbreak

In this paper, we explore whether the infection-rate of a disease can serve as a robust monitoring variable in epidemiological surveillance algorithms. The infection-rate is dependent on population mixing patterns that do not vary erratically day-to-day; in contrast, daily case-counts used in contemporary surveillance algorithms are corrupted by reporting errors. The technical challenge lies in estimating the latent infection-rate from case-counts. Here we devise a Bayesian method to estimate the infection-rate across multiple adjoining areal units, and then use it, via an anomaly detector, to discern a change in epidemiological dynamics. We extend an existing model for estimating the infection-rate in an areal unit by incorporating a Markov random field model, so that we may estimate infection-rates across multiple areal units, while preserving spatial correlations observed in the epidemiological dynamics. To carry out the high-dimensional Bayesian inverse problem, we develop an implementation of mean-field variational inference specific to the infection model and integrate it with the random field model to incorporate correlations across counties. The method is tested on estimating the COVID-19 infection-rates across all 33 counties in New Mexico using data from the summer of 2020, and then employing them to detect the arrival of the Fall 2020 COVID-19 wave. We perform the detection using a temporal algorithm that is applied county-by-county. We also show how the infection-rate field can be used to cluster counties with similar epidemiological dynamics.

60 APPLIED LIFE SCIENCES↗