Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “bayesian inference”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 865 records · Page 48
All sourcesNASA NTRSDOE OSTI
DOE OSTI2019

Quantitative Uncertainty Analysis for Diffraction

Current software uses a least squares statistical method. For each model parameter, current software yields a point estimate and estimated standard error for each point estimate. In contrast, we have employed a Bayesian approach with sampling algorithm to provide a full posterior probability density function on each model parameter. This yields a different type of solution (probabilistic) with better fidelity. The Quantitative Uncertainty Analysis for Diffraction (QUAD) package returns a matrix of affect model parameters as a function of iteration through the MCMC analysis. QUAD is a Python code that allows for analysis of X-ray and neutron diffraction data to infer the structure of materials with quantifiable certainty. The code uses Bayesian statistics and Markov chain sampling algorithms to create posterior probability distributions on all material structure parameters that are modeled by a researcher. The statistical packages use the open source GSASII project as a library to calculate model diffraction patterns.

Fancher, ChristopherM↗
DOE OSTI2022

Towards Compact Neural Networks via End-to-End Training: A Bayesian Tensor Approach with Automatic Rank Determination

Post-training model compression can reduce the inference costs of deep neural networks, but uncompressed training still consumes enormous hardware resources and energy. To enable low-energy training on edge devices, it is highly desirable to directly train a compact neural network from scratch with a low memory cost. Low-rank tensor decomposition is an effective approach to reduce the memory and computing costs of large neural networks. However, directly training low-rank tensorized neural networks is a very challenging task because it is hard to determine a proper tensor rank a priori, and the tensor rank controls both model complexity and accuracy. Here, this paper presents a novel end-to-end framework for low-rank tensorized training. We first develop a Bayesian model that supports various low-rank tensor formats (e.g., CANDECOMP/PARAFAC, Tucker, tensor-train, and tensor-train matrix) and reduces neural network parameters with automatic rank determination during training. Then we develop a customized Bayesian solver to train large-scale tensorized neural networks. Our training methods shows orders-of-magnitude parameter reduction and little accuracy loss (or even better accuracy) in the experiments. On a very large deep learning recommendation system with over 4.2 ×10 9 model parameters, our method can reduce the parameter number to 1.6 ×10 5 automatically in the training process (i.e., by 2.6 ×10 4 times) while achieving almost the same accuracy. Code is available at https://github.com/colehawkins/bayesian-tensor-rank-determination.

compact neural networks↗
DOE OSTI2025

Bayesian model-data comparison incorporating theoretical uncertainties

Accurate comparisons between theoretical models and experimental data are critical for scientific progress. However, inferred physical model parameters can vary significantly with the chosen physics model, highlighting the importance of properly accounting for theoretical uncertainties. In this Letter, we present a Bayesian framework that explicitly quantifies these uncertainties by statistically modeling theory errors, guided by qualitative knowledge of a theory’s varying reliability across the input domain. We demonstrate the effectiveness of this approach using two systems: a simple ball drop experiment and multi-stage heavy-ion simulations. In both cases incorporating model discrepancy leads to improved parameter estimates, with systematic improvements observed as additional experimental observables are integrated.

Bayesian methods↗
DOE OSTI2024

Visible core spectroscopy at Wendelstein 7-X

This paper presents an overview of recent hardware extensions and data analysis developments to the Wendelstein 7-X visible core spectroscopy systems. These include upgrades to prepare the in-vessel components for long-pulse operation, nine additional spectrometers, a new line of sight array for passive spectroscopy, and a coherence imaging charge exchange spectroscopy diagnostic. Progress in data analysis includes ion temperatures and densities from multiple impurity species, a statistical comparison with x-ray crystal spectrometer measurements, neutral density measurements from thermal passive Balmer-alpha emission, and a Bayesian analysis of active hydrogen emission, which is able to infer electron density and main ion temperature profiles.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2021

Phylogenetic diversity, trichothecene potential, and pathogenicity within Fusarium sambucinum species complex

The Fusarium sambucinum species complex (FSAMSC) is one of the most taxonomically challenging groups of fusaria, comprising prominent mycotoxigenic plant pathogens and other species with various lifestyles. Among toxins produced by members of the FSAMSC, trichothecenes pose the most significant threat to public health. Herein a global collection of 171 strains, originating from diverse hosts or substrates, were selected to represent FSAMSC diversity. This strain collection was used to assess their species diversity, evaluate their potential to produce trichothecenes, and cause disease on wheat. Maximum likelihood and Bayesian analyses of a combined 3-gene dataset used to infer evolutionary relationships revealed that the 171 strains originally received as 48 species represent 74 genealogically exclusive phylogenetically distinct species distributed among six strongly supported clades: Brachygibbosum, Graminearum, Longipes, Novel, Sambucinum, and Sporotrichioides. Most of the strains produced trichothecenes in vitro but varied in type, indicating that the six clades correspond to type A, type B, or both types of trichothecene-producing lineages. Furthermore, five strains representing two putative novel species within the Sambucinum Clade produced two newly discovered type A trichothecenes, 15-keto NX-2 and 15-keto NX-3. Strains of the two putatively novel species together with members of the Graminearum Clade were aggressive toward wheat when tested for pathogenicity on heads of the susceptible cultivar Apogee. In planta, the Graminearum Clade strains produced nivalenol or deoxynivalenol and the aggressive Sambucinum Clade strains synthesized NX-3 and 15-keto NX-3. Other strains within the Brachygibbosum, Longipes, Novel, Sambucinum, and Sporotrichioides Clades were nonpathogenic or could infect the inoculated floret without spreading within the head. Moreover, most of these strains did not produce any toxin in the inoculated spikelets. These data highlight aggressiveness toward wheat appears to be influenced by the type of toxin produced and that it is not limited to members of the Graminearum Clade.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2025

CONCURRENT, CONDENSED STEIN VARIATIONAL GRADIENT DESCENT FOR UNCERTAINTY QUANTIFICATION OF NEURAL NETWORKS

In this work, we propose a Stein variational gradient descent (SVGD) method to concurrently sparsify, train, and provide uncertainty quantification (UQ) of a complexly parameterized model, such as a neural network (NN). It employs a graph reconciliation and condensation process to reduce complexity and increase similarity in the Stein ensemble of parameterizations. Therefore, the proposed concurrent, condensed SVGD (ccSVGD) method can provide UQ on parameters, not just outputs. Furthermore, the parameter reduction speeds up the convergence of the Stein gradient descent as it reduces the combinatorial complexity by aligning and differentiating the sensitivity to parameters. These properties are demonstrated with an illustrative example and an application to a mechanical response representation problem in solid mechanics.

42 ENGINEERING↗
DOE OSTI2016

AGNfitter: a Bayesian MCMC approach to fitting spectral energy distributions of AGNs

We present AGNfitter, a publicly available open-source algorithm implementing a fully Bayesian Markov Chain Monte Carlo method to fit the spectral energy distributions (SEDs) of active galactic nuclei (AGNs) from the sub-millimeter to the UV, allowing one to robustly disentangle the physical processes responsible for their emission. AGNfitter makes use of a large library of theoretical, empirical, and semi-empirical models to characterize both the nuclear and host galaxy emission simultaneously. The model consists of four physical emission components: an accretion disk, a torus of AGN heated dust, stellar populations, and cold dust in star-forming regions. AGNfitter determines the posterior distributions of numerous parameters that govern the physics of AGNs with a fully Bayesian treatment of errors and parameter degeneracies, allowing one to infer integrated luminosities, dust attenuation parameters, stellar masses, and star-formation rates. We tested AGNfitter’s performance on real data by fitting the SEDs of a sample of 714 X-ray selected AGNs from the XMM-COSMOS survey, spectroscopically classified as Type1 (unobscured) and Type2 (obscured) AGNs by their optical–UV emission lines. We find that two independent model parameters, namely the reddening of the accretion disk and the column density of the dusty torus, are good proxies for AGN obscuration, allowing us to develop a strategy for classifying AGNs as Type1 or Type2, based solely on an SED-fitting analysis. Our classification scheme is in excellent agreement with the spectroscopic classification, giving a completeness fraction of ∼86% and ∼70%, and an efficiency of ∼80% and ∼77%, for Type1 and Type2 AGNs, respectively.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2019

Bayesian Model Calibration for Extrapolative Prediction via Gibbs Posteriors

The current standard Bayesian approach to model calibration, which assigns a Gaussian process prior to the discrepancy term, often suffers from issues of unidentifiability and computational complexity and instability. When the goal is to quantify uncertainty in physical parameters for extrapolative prediction, then there is no need to perform inference on the discrepancy term. With this in mind, we introduce Gibbs posteriors as an alternative Bayesian method for model calibration, which updates the prior with a loss function connecting the data to the parameter. The target of inference is the physical parameter value which minimizes the expected loss. We propose to tune the loss scale of the Gibbs posterior to maintain nominal frequentist coverage under assumptions of the form of model discrepancy, and present a bootstrap implementation for approximating coverage rates. Our approach is highly modular, allowing an analyst to easily encode a wide variety of such assumptions. Furthermore, we provide a principled method of combining posteriors calculated from data subsets. We apply our methods to data from an experiment measuring the material properties of tantalum.

36 MATERIALS SCIENCE↗
DOE OSTI2021

B-PINNs: Bayesian Physics-informal Neural Networks for Forward and Inverse PDE Problems with Noisy Data

We propose a Bayesian physics-informed neural network (B-PINN) to solve both forward and inverse nonlinear problems described by partial differential equations (PDEs) and noisy data. In this Bayesian framework, the Bayesian neural network (BNN) combined with a PINN for PDEs serves as the prior while the Hamiltonian Monte Carlo (HMC) or the variational inference (VI) could serve as an estimator of the posterior. B-PINNs make use of both physical laws and scattered noisy measurements to provide predictions and quantify the aleatoric uncertainty arising from the noisy data in the Bayesian framework. Compared with PINNs, in addition to uncertainty quantification, B-PINNs obtain more accurate predictions in scenarios with large noise due to their capability of avoiding overfitting. We conduct a systematic comparison between the two different approaches for the B-PINNs posterior estimation (i.e., HMC or VI), along with dropout used for quantifying uncertainty in deep neural networks. Our experiments show that HMC is more suitable than VI with mean field Gaussian approximation for the B-PINNs posterior estimation, while dropout employed in PINNs can hardly provide accurate predictions with reasonable uncertainty. Finally, we replace the BNN in the prior with a truncated Karhunen-Loève (KL) expansion combined with HMC or a deep normalizing flow (DNF) model as posterior estimators. The KL is as accurate as BNN and much faster but this framework cannot be easily extended to high-dimensional problems unlike the BNN based framework.

Non linear PDEs, Noisy data, Bysian physics inform↗
DOE OSTI2022

Dynamic logistic regression and variable selection: Forecasting and contextualizing civil unrest

Civil unrest can range from peaceful protest to violent furor, and researchers are working to monitor, forecast, and assess such events to allocate resources better. Twitter has become a real-time data source for forecasting civil unrest because millions of people use the platform as a social outlet. Additionally, daily word counts are used as model features, and predictive terms contextualize the reasons for the protest. To forecast civil unrest and infer the reasons for the protest, we consider the problem of Bayesian variable selection for the dynamic logistic regression model and propose using penalized credible regions to select parameters of the updated state vector. This method avoids the need for shrinkage priors, is scalable to high-dimensional dynamic data, and allows the importance of variables to vary in time as new information becomes available. A substantial improvement in both precision and F1-score using this approach is demonstrated through simulation. Finally, we apply the proposed model fitting and variable selection methodology to the problem of forecasting civil unrest in Latin America. Our dynamic logistic regression approach shows improved accuracy compared to the static approach currently used in event prediction and feature selection.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

A unifying Bayesian framework for merging X-ray diffraction data

Novel X-ray methods are transforming the study of the functional dynamics of biomolecules. Key to this revolution is detection of often subtle conformational changes from diffraction data. Diffraction data contain patterns of bright spots known as reflections. To compute the electron density of a molecule, the intensity of each reflection must be estimated, and redundant observations reduced to consensus intensities. Systematic effects, however, lead to the measurement of equivalent reflections on different scales, corrupting observation of changes in electron density. Here, we present a modern Bayesian solution to this problem, which uses deep learning and variational inference to simultaneously rescale and merge reflection observations. We successfully apply this method to monochromatic and polychromatic single-crystal diffraction data, as well as serial femtosecond crystallography data. We find that this approach is applicable to the analysis of many types of diffraction experiments, while accurately and sensitively detecting subtle dynamics and anomalous scattering.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2019

Support for Reactor Operators in Case of Cyber-Security Threats (NEUP Final Report)

Increasing use of digital technology in nuclear power plants (NPPs) poses cyber-security as a crucial threat to public safety and to continuous energy production. Cyber-security risks are comprised of complex known and unknown interactions between various entities, system vulnerabilities, network protocols, human users and malicious attacks. There is little understanding or research geared towards plant operators’ response under cyber-security threats and operational procedures to cope with such threats. This is particularly critical when a cyber-security event masquerades as a safety incident or an evolving accident. Instead of leading operators to the remediation of the accident, the masked cyber event may lead them to circumvent the safety mechanisms of the plant. Given these challenges, the objective of this project is to develop a tool to characterize abnormal NPP events as “cyber” or “safety” incidents, and to further develop this tool as a real-time operator aid to assist in response to the incident. This NEUP Final Report summarizes the research efforts towards the above goals and the results in the following three aspects. First, a methodology for classifying abnormal NPP events as cyber-attacks or safety incidents was developed. This provides the basis for responding to potential cyber-attacks. In this methodology, the causal relationships between various variables (i.e., physical variables, network variables, and control variables) for a nuclear system under consideration are modeled using dynamic Bayesian networks. Based on observations of certain variables (i.e., the evidence), inference is performed to identify the state of hidden variables, which helps indicate the occurrence of cyber-attacks or safety incidents. The proposed methodology was applied to the full scope NPP simulator system at The Ohio State University as a case study. The result shows that the proposed methodology is capable of distinguishing between cyber-attacks and safety incidents. Second, a game-theoretic approach for responding to cyber-attacks on nuclear power plants was developed. This is the second major goal of the project, and the method is used to recommend optimal responses to reactor operators in the case of cyber-attacks. We consider two cases in this research. In the first case, the interaction between the defender (i.e., the reactor operator) and the attacker is modeled as a two-player, nonzero-sum, Markov game with an infinite time horizon. In the second case, we consider a semi-Markov game between the defender and the attacker within a finite time horizon. The model in the second case enables us to consider the time sensitive response of the defender. In each case, the method for deriving the Nash equilibrium is developed. The Nash Equilibrium of the game provides the valid prediction of both players’ actions because no single player can benefit from unilaterally deviating from the equilibrium policy if the other player adheres to his/hers, hence it provides the best response of the defender to cyber-attacks. The proposed method in each case is demonstrated through a simplified benchmark digital feedwater control system. Third, a prototype display is developed to integrate the above information. The prototype display depicts the communication information to the operator and incorporates the cyber-security tool developed by the team into the interface. It consists of the regular display in current control rooms, as well as a panel used to indicate the occurrence of cyber-attacks or safety events and a panel used to guide the operator’s response to detected cyber-attacks.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
NASA NTRS1995

Inductive Approaches to Improving Diagnosis and Design for Diagnosability

The first research area under this grant addresses the problem of classifying time series according to their morphological features in the time domain. A supervised learning system called CALCHAS, which induces a classification procedure for signatures from preclassified examples, was developed. For each of several signature classes, the system infers a model that captures the class's morphological features using Bayesian model induction and the minimum message length approach to assign priors. After induction, a time series (signature) is classified in one of the classes when there is enough evidence to support that decision. Time series with sufficiently novel features, belonging to classes not present in the training set, are recognized as such. A second area of research assumes two sources of information about a system: a model or domain theory that encodes aspects of the system under study and data from actual system operations over time. A model, when it exists, represents strong prior expectations about how a system will perform. Our work with a diagnostic model of the RCS (Reaction Control System) of the Space Shuttle motivated the development of SIG, a system which combines information from a model (or domain theory) and data. As it tracks RCS behavior, the model computes quantitative and qualitative values. Induction is then performed over the data represented by both the 'raw' features and the model-computed high-level features. Finally, work on clustering for operating mode discovery motivated some important extensions to the clustering strategy we had used. One modification appends an iterative optimization technique onto the clustering system; this optimization strategy appears to be novel in the clustering literature. A second modification improves the noise tolerance of the clustering system. In particular, we adapt resampling-based pruning strategies used by supervised learning systems to the task of simplifying hierarchical clusterings, thus making post-clustering analysis easier.

Fisher, Douglas H.↗
DOE OSTI2021

Bayesian estimations of orientation distribution functions from small-angle scattering enable direct prediction of mechanical stress in anisotropic materials

Properties of soft materials are influenced by their anisotropic structuring under nonequilibrium fields. Although anisotropic structure-property relationships have been extensively explored theoretically, comparison to experiments requires determination of the microstructural orientation probability distribution function (OPDF) of microstructural elements. Small angle scattering (SAS) measurements encode information about the OPDF, but tools to navigate this connection are incomplete. Here, we develop and validate an explicit framework to link arbitrary OPDFs to SAS measurements. Specifically, we propose, validate, and apply a method, maximum a posteriori scattering inference (MAPSI), whereby the OPDF may be obtained from SAS measurements using a Bayesian estimation method. Using this method, we obtain estimates of the full 3D OPDF for two model semidilute fd-virus (rodlike) dispersions at concentrations that are approximately equal to and twice the overlap concentration. From the OPDF, we calculate its second and fourth moments and compare these to predictions for a dilute suspension of rigid rods and to a recent theory for semidilute suspensions. Finally, we use both the theoretical and measured moments to calculate the stress, both for dilute and semidilute suspensions. These predictions are not only compared to each other, but also to measured values of the shear stress, and point to new insights into the behavior of suspensions of highly elongated particles in the transition between dilute and semidilute behavior. We also use this new framework to provide perspective on the connection between scalar parameterizations of scattering and the OPDF that have frequently been used in the past. Here, the new tools developed in this work provide an unprecedented path toward experimental validation of dynamical theories of rodlike colloids and polymers, and for measurement of nonequilibrium structures and stresses of other complex fluids and soft materials with SAS.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Calibrating Bayesian generative machine learning for Bayesiamplification

Recently, combinations of generative and Bayesian deep learning have been introduced in particle physics for both fast detector simulation and inference tasks. These neural networks aim to quantify the uncertainty on the generated distribution originating from limited training statistics. The interpretation of a distribution-wide uncertainty however remains ill-defined. We show a clear scheme for quantifying the calibration of Bayesian generative machine learning models. For a Continuous Normalizing Flow applied to a low-dimensional toy example, we evaluate the calibration of Bayesian uncertainties from either a mean-field Gaussian weight posterior, or Monte Carlo sampling network weights, to gauge their behaviour on unsteady distribution edges. Well calibrated uncertainties can then be used to roughly estimate the number of uncorrelated truth samples that are equivalent to the generated sample and clearly indicate data amplification for smooth features of the distribution.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2017

Bayesian Redshift Classification of Emission-line Galaxies with Photometric Equivalent Widths

We present a Bayesian approach to the redshift classification of emission-line galaxies when only a single emission line is detected spectroscopically. We consider the case of surveys for high-redshift Lyα-emitting galaxies (LAEs), which have traditionally been classified via an inferred rest-frame equivalent width (EW; W{sub Lyα}) greater than 20 Å. Our Bayesian method relies on known prior probabilities in measured emission-line luminosity functions and EW distributions for the galaxy populations, and returns the probability that an object in question is an LAE given the characteristics observed. This approach will be directly relevant for the Hobby–Eberly Telescope Dark Energy Experiment (HETDEX), which seeks to classify ∼10{sup 6} emission-line galaxies into LAEs and low-redshift [O II] emitters. For a simulated HETDEX catalog with realistic measurement noise, our Bayesian method recovers 86% of LAEs missed by the traditional W{sub Lyα} > 20 Å cutoff over 2 < z < 3, outperforming the EW cut in both contamination and incompleteness. This is due to the method’s ability to trade off between the two types of binary classification error by adjusting the stringency of the probability requirement for classifying an observed object as an LAE. In our simulations of HETDEX, this method reduces the uncertainty in cosmological distance measurements by 14% with respect to the EW cut, equivalent to recovering 29% more cosmological information. Rather than using binary object labels, this method enables the use of classification probabilities in large-scale structure analyses. It can be applied to narrowband emission-line surveys as well as upcoming large spectroscopic surveys including Euclid and WFIRST.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2020

Application of the Approximate Bayesian Computation Algorithm to Gamma-Ray Spectroscopy

Radioisotope identification (RIID) algorithms for gamma-ray spectroscopy aim to infer what isotopes are present and in what amounts in test items. RIID algorithms either use all energy channels in the analysis region or only energy channels in and near identified peaks. Because many RIID algorithms rely on locating peaks and estimating each peak’s net area, peak location and peak area estimation algorithms continue to be developed for gamma-ray spectroscopy. This paper shows that approximate Bayesian computation (ABC) can be effective for peak location and area estimation. Algorithms to locate peaks can be applied to raw or smoothed data, and among several smoothing options, the iterative bias reduction algorithm (IBR) is recommended; the use of IBR with ABC is shown to potentially reduce uncertainty in peak location estimation. Extracted peak locations and areas can then be used as summary statistics in a new ABC-based RIID. ABC allows for easy experimentation with candidate summary statistics such as goodness-of-fit scores and peak areas that are extracted from relatively high dimensional gamma spectra with photopeaks (1024 or more energy channels) consisting of count rates versus energy for a large number of gamma energies.

Burr, Tom↗
NASA NTRS1995

Inferring the spatial and energy distribution of gamma-ray burst sources. 1: Methodology

We describe Bayesian methods for analyzing the distribution of gamma-ray burst peak photon fluxes and directions. These methods fit the differential distribution, and have the following advantages over rival methods: (1) they do not destroy information by binning or averaging the data (as do, say, chi squared, the averaged value of V/V(sub max), and angular moment analyses); (2) they straightforwardly handle uncertainties in the measured quantities; (3) they analyze the strength and direction information jointly; (4) they use information available about nondetections; and (5) they automatically identify and account for biases and selection effects given a precise description of the experiment. In these methods, the most important information needed about the instrument threshold is not its value at the times of burst triggers, as is used in the average value of V/V(sub max) analyses, but rather the value of the threshold at times when no trigger occurred. We show that this information can be summarized as an average detection efficiency that is similar to the product of the exposure and efficiency reported in the First Burst and Transient Source Experiment (BATSE) Burst (1B) Catalog, but significantly different from it at low fluxes. We also quantify an important bias that results from estimating the peak flux by scanning the burst to find the peak number of counts in a window of specified duration, as was done for the 1B Catalog. When the duration of the peak of the light curve is longer than the window duration, a simple flux estimate based on the peak counts significantly overestimates the peak flux in a nonlinear fashion that distorts the shape of the log(N)-log(P) distribution. This distortion also corrupts analyses of the V/V(sub max) distribution that use ratios of counts above background to estimate V/V(sub max). The Bayesian calculation specifies how to account for this bias. Implementation of the Bayesian approach requires some changes in the way burst data are reported that we describe in detail. Subsequent papers will report analyses of the 1B Catalog data using the methods described here.

Loredo, Thomas J.↗