Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Stochastic inference”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Latent Stochastic Differential Equations for Modeling Quasar Variability and Inferring Black Hole Properties

Quasars are bright and unobscured active galactic nuclei (AGN) thought to be powered by the accretion of matter around supermassive black holes at the centers of galaxies. The temporal variability of a quasar’s brightness contains valuable information about its physical properties. The UV/optical variability is thought to be a stochastic process, often represented as a damped random walk described by a stochastic differential equation (SDE). Upcoming wide-field telescopes such as the Rubin Observatory Legacy Survey of Space and Time (LSST) are expected to observe tens of millions of AGN in multiple filters over a ten year period, so there is a need for efficient and automated modeling techniques that can handle the large volume of data. Latent SDEs are machine learning models well suited for modeling quasar variability, as they can explicitly capture the underlying stochastic dynamics. In this work, we adapt latent SDEs to jointly reconstruct multivariate quasar light curves and infer their physical properties such as the black hole mass, inclination angle, and temperature slope. Our model is trained on realistic simulations of LSST ten year quasar light curves, and we demonstrate its ability to reconstruct quasar light curves even in the presence of long seasonal gaps and irregular sampling across different bands, outperforming a multioutput Gaussian process regression baseline. Our method has the potential to provide a deeper understanding of the physical properties of quasars and is applicable to a wide range of other multivariate time series with missing data and irregular sampling.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2025

Validating sequential Monte Carlo for gravitational-wave inference

Nested sampling (NS) is the preferred stochastic sampling algorithm for gravitational-wave inference for compact binary coalescences. It can handle the complex nature of the gravitational-wave likelihood surface and provides an estimate of the Bayesian model evidence. However, there is another class of algorithms that meets the same requirements, but has not been used for gravitational-wave analyses: sequential Monte Carlo (SMC), an extension of importance sampling that maps samples from an initial density to a target density via a series of intermediate densities. In this work, we validate a type of SMC algorithm, called persistent sampling (PS), for gravitational-wave inference. We consider a range of different scenarios including binary black holes and binary neutron stars and real and simulated data and show that PS produces results that are consistent with NS whilst being, on average, 2 times more efficient and 2.74 times faster. This demonstrates that PS is a viable alternative to NS that should be considered for future gravitational-wave analyses.

black hole mergers↗
NASA NTRSDate not supplied

Quantum-Assisted Variational Segmentation for Image-to-Image Wildfire Detection Using Satellite Data

The quantum computing community has been searching for suitable applications to demonstrate the potential of near-term quantum devices. Quantum machine learning is a potential candidate, particularly using models that cannot be efficiently simulated with classical computers [1, 2]. This work focuses on a transition phase of quantum computers where the quantum machine learning model is still simulable classically but projected not to be simulable as the size of the model grows. Ultimately quantum computers may have advantages for high-dimensional real-world problems. Due to the limited number of qubits in current noisy intermediate-scale quantum (NISQ) devices, the direct application of quantum computers in high dimensional data is not feasible. To remedy this problem, an encoder-decoder architecture can be utilized. The encoder model would transform the high-dimensional data into a compact representation, to a level that small quantum computers can be used today (or in the near future), and the decoder would take the quantum processed outputs back to the high-dimensional space. Addressing the two challenges of quantum machine learning, this work investigates a hybrid supervised generative model with a quantum Ising Born machine embedded as the latent distribution. The model contains four main parts (Figure 1.a.): (1) a U-NET architecture responsible for learning segmentation flow, (2) a Prior network responsible for learning an encoded latent distribution of the input data, (3) a Born machine which represents the latent distribution, and (4) a Posterior network in charge of learning the joint encoded latent distribution of inputs and target data. The initial model, proposed by [3], is optimized by (1) maximizing the overlap of the prior and posterior latent distributions, and (2) minimizing the segmentation loss. The proposed model is designed to be investigated in a simulation environment applied to the real-world application of wildfire segmentation. Specifically, the model is designed to solve the patchy wildfire segmentations of Moderate Resolution Imaging Spectroradiometer (MODIS) by taking the MODIS observations and using Visible Infrared Imaging Radiometer Suite’s (VIIRS) consistent wildfire product as the target. The model solves patchy wildfire segmentations and provides insight into the epistemic errors sourced from model variation. The model utilizes the Born machine as a QUBO solver to represent the latent space as a Bernoulli distribution. The proposed configuration allows the variational segmentation model to leverage the true quantum probabilistic nature and derive a more expressive latent configuration, increasing the model performance in describing wildfire segmentations. The quantum probabilistic information of the Born machine is directly incorporated in the Kullback-Leibler divergence loss in the prior and posterior distributions, forcing the Bernoulli latent distribution to maximize the overlap of input and joint input-target distributions. The proposed model is then trained and compared with a baseline only consisting of direct Bernoulli latent distribution with no Born machine representing the latent space. The models are evaluated based on the segmentation metrics, such as precision, recall, intersect of union, with uncertainty boundaries accounting for the stochastic nature of the model. Our findings show that even in low latent-dimensional space (due to the limit in computational power of the classical quantum simulator), we are able to effectively capture the latent representation and hence the model performs better than the baseline. The findings are a projection for scaling the model into higher dimensional latent space with the Born machine surpassing the baseline performance. Figure 1. Sub-figure (a) demonstrates the architecture for the training phase. The model consists of a Prior and Posterior network that encode inputs and joint input-target data into compact representations, respectively. The Born machine represents the latent distribution, and the U-NET branch learns the segmentation patterns of the data. The stochasticity is introduced to the U-NET through its last layer to create meaningful but stochastic segmentations. Sub-figure (b) represents the inference phase where the model takes the stochastic behavior from the prior network and injects that into the U-NET. Each attempt of inference will generate different but similar segmentations from the same distribution of the wildfire event. REFERENCES [1] Coyle, B., Mills, D., Danos, V., & Kashefi, E. (2020). The Born supremacy: quantum advantage and training of an Ising Born machine. npj Quantum Information, 6(1), 1-11. [2] Liu, J. G., & Wang, L. (2018). Differentiable learning of quantum circuit born machines. Physical Review A, 98(6), 062324. [3] Kohl, S., Romera-Paredes, B., Meyer, C., De Fauw, J., Ledsam, J. R., Maier-Hein, K., ... & Ronneberger, O. (2018). A probabilistic u-net for segmentation of ambiguous images. Advances in neural information processing systems, 31.

quantum machine learning↗
DOE OSTI2021

On the Stochastic Stability of Deep Markov Models

Deep Markov models (DMM) are generative models which are scalable and expressive generalization of Markov models for representation, learning, and inference problems. However, the fundamental stochastic stability guarantees of such models have not been thoroughly investigated. In this paper, we present a novel stability analysis method and provide sufficient conditions of DMM's stochastic stability. The proposed stability analysis is based on the contraction of probabilistic maps modeled by deep neural networks. We make connections between the spectral properties of neural network's weights and different types of used activation function on the stability and overall dynamic behavior of DMMs with Gaussian distributions. Based on the theory, we propose a few practical methods for designing constrained DMMs with guaranteed stability. We empirically substantiate our theoretical results via intuitive numerical experiments using the proposed stability constraints.

Drgona, Jan↗
DOE OSTI2024

Bayesian inference of structured latent spaces from neural population activity with the orthogonal stochastic linear mixing model

The brain produces diverse functions, from perceiving sounds to producing arm reaches, through the collective activity of populations of many neurons. Determining if and how the features of these exogenous variables (e.g., sound frequency, reach angle) are reflected in population neural activity is important for understanding how the brain operates. Often, high-dimensional neural population activity is confined to low-dimensional latent spaces. However, many current methods fail to extract latent spaces that are clearly structured by exogenous variables. This has contributed to a debate about whether or not brains should be thought of as dynamical systems or representational systems. Here, we developed a new latent process Bayesian regression framework, the orthogonal stochastic linear mixing model (OSLMM) which introduces an orthogonality constraint amongst time-varying mixture coefficients, and provide Markov chain Monte Carlo inference procedures. We demonstrate superior performance of OSLMM on latent trajectory recovery in synthetic experiments and show superior computational efficiency and prediction performance on several real-world benchmark data sets. We primarily focus on demonstrating the utility of OSLMM in two neural data sets: μ ECoG recordings from rat auditory cortex during presentation of pure tones and multi-single unit recordings form monkey motor cortex during complex arm reaching. We show that OSLMM achieves superior or comparable predictive accuracy of neural data and decoding of external variables (e.g., reach velocity). Most importantly, in both experimental contexts, we demonstrate that OSLMM latent trajectories directly reflect features of the sounds and reaches, demonstrating that neural dynamics are structured by neural representations. Together, these results demonstrate that OSLMM will be useful for the analysis of diverse, large-scale biological time-series datasets.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2020

UQpy: A general purpose Python package and development environment for uncertainty quantification

In this paper, we present the UQpy software toolbox, an open-source Python package for general uncertainty quantification (UQ) in mathematical and physical systems. The software serves as both a user-ready toolbox that includes many of the latest methods for UQ in computational modeling and a convenient development environment for Python programmers advancing the field of UQ. The paper presents an introduction to the software's architecture and existing capabilities, divided in the code in a set of modules centered around different UQ tasks such as sampling methods, generation of random processes and random fields, probabilistic inverse modeling, reliability analysis, surrogate modeling, and active learning. The paper also highlights the importance of the RunModel module, which is used to drive simulations in the uncertainty analyses performed in UQpy. This module conveniently allows the user to define computational models directly in Python, or to run simulations from a third-party software in serial or in parallel. To illustrate the various capabilities, two examples are tracked throughout the paper and analyzed repeatedly for various UQ tasks. The first is a Python model solving a nonlinear structural dynamics problem, used to illustrate UQpy's capabilities in sampling and forward propagation of high dimensional random vectors (stochastic processes), and probabilistic inference. The second model is a third-party Abaqus finite element model solving the thermomechanical response of a beam structure. This example is used to illustrate UQpy's capabilities in variance reduction sampling techniques, reliability analysis, surrogate modeling and active learning techniques.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1987

Isotropic probability measures in infinite dimensional spaces: Inverse problems/prior information/stochastic inversion

Let R be the real numbers, R(n) the linear space of all real n-tuples, and R(infinity) the linear space of all infinite real sequences x = (x sub 1, x sub 2,...). Let P sub n :R(infinity) approaches R(n) be the projection operator with P sub n (x) = (x sub 1,...,x sub n). Let p(infinity) be a probability measure on the smallest sigma-ring of subsets of R(infinity) which includes all of the cylinder sets P sub n(-1) (B sub n), where B sub n is an arbitrary Borel subset of R(n). Let p sub n be the marginal distribution of p(infinity) on R(n), so p sub n(B sub n) = p(infinity)(P sub n to the -1(B sub n)) for each B sub n. A measure on R(n) is isotropic if it is invariant under all orthogonal transformations of R(n). All members of the set of all isotropic probability distributions on R(n) are described. The result calls into question both stochastic inversion and Bayesian inference, as currently used in many geophysical inverse problems.

Backus, George↗
NASA NTRS1987

Isotropic probability measures in infinite-dimensional spaces

Let R be the real numbers, R(n) the linear space of all real n-tuples, and R(infinity) the linear space of all infinite real sequences x = (x sub 1, x sub 2,...). Let P sub in :R(infinity) approaches R(n) be the projection operator with P sub n (x) = (x sub 1,...,x sub n). Let p(infinity) be a probability measure on the smallest sigma-ring of subsets of R(infinity) which includes all of the cylinder sets P sub n(-1) (B sub n), where B sub n is an arbitrary Borel subset of R(n). Let p sub n be the marginal distribution of p(infinity) on R(n), so p sub n(B sub n) = p(infinity) (P sub n to the -1 (B sub n)) for each B sub n. A measure on R(n) is isotropic if it is invariant under all orthogonal transformations of R(n). All members of the set of all isotropic probability distributions on R(n) are described. The result calls into question both stochastic inversion and Bayesian inference, as currently used in many geophysical inverse problems.

Backus, George↗
DOE OSTI2021

Magnetostriction of α-RuCl 3 Flakes in the Zigzag Phase

Motivated by the possibility of enhanced magnetic fluctuations in exfoliated α-RuCl 3 flakes, we study magneto-Raman spectra of exfoliated multilayer α-RuCl 3 in out-of-plane magnetic fields of -6 to 6 T at temperatures of 670 mK to 4 K. While the literature currently suggests that bulk α-RuCl 3 is in an antiferromagnetic zigzag phase with R$\bar3$ symmetry at low temperatures, we do not observe R$\bar3$ symmetry in exfoliated α-RuCl 3 at low temperatures. While we saw no magnetic field-driven transitions, the Raman modes exhibit unexpected stochastic shifts in response to the applied magnetic field that are above the uncertainties inferred from Bayesian analysis. Finally, these stochastic shifts are consistent with the emergence of magnetostrictive interactions in exfoliated α-RuCl 3 .

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2026

Simultaneous global and local clustering in multiplex networks with covariate information

Understanding both global and layer-specific group structures is useful for uncovering complex patterns in networks with multiple interaction types. In this work, we introduce a new model, the hierarchical multiplex stochastic blockmodel, which simultaneously detects communities within individual layers of a multiplex network while inferring a global node clustering across the layers. A stochastic blockmodel is assumed in each layer, with probabilities of layer-level group memberships determined by a node’s global group assignment. Our model uses a Bayesian framework, employing a probit stick-breaking process to construct node-specific mixing proportions over a set of shared Griffiths–Engen–McCloseky distributions. These proportions determine layer-level community assignment, allowing for an unknown and varying number of groups across layers, while incorporating nodal covariate information to inform the global clustering. We propose a scalable variational inference procedure with parallelisable updates for application to large networks. Extensive simulation studies demonstrate our model’s ability to accurately recover both global and layer-level clusters in complicated settings, and applications to real data showcase the model’s effectiveness in uncovering interesting latent network structure.

community detection↗
DOE OSTI2021

Development of a framework for sequential Bayesian design of experiments: Application to a pilot-scale solvent-based CO 2 capture process

In this paper, a methodology is developed for sequential design of experiments (SDoE) for process systems and applied to a solvent-based CO 2 capture system. In this approach, the prior knowledge of the system is used to prioritize process data collection at specific operating conditions. These data are then incorporated into a Bayesian inference methodology for updating a stochastic model by refining estimations of its underlying parameters, and the updated model is then used to generate the next set of test runs. Thus, the new knowledge obtained from the data is used to guide subsequent iterations of the experimental runs, ensuring that the overall data collection is maximally informative given that most experimental campaigns, especially at pilot or higher-scale plants, are costly, time-consuming, and resource-limited. The test run objective for this work was to minimize the maximum model prediction uncertainty for key output variables, but the methodology is generic and can be readily applied to other test run objectives. This methodology is applied to an aqueous monoethanolamine (MEA) pilot plant campaign at the National Carbon Capture Center (NCCC) in Wilsonville, Alabama, USA. The SDoE framework was utilized for two iterations, while collecting 18 sets of data representing different process conditions, and this resulted in an overall average reduction in uncertainty of approximately 50% in the prediction of CO 2 capture percentage. Moreover, 11 additional data sets were obtained with variation of absorber packing height for further model validation. This work shows the capability of the SDoE framework to maximize learning given limited resources, allowing for the reduction of model uncertainty, which is of great importance for many applications including reduction of technical risk associated with scale-up and economic analysis.

20 FOSSIL-FUELED POWER PLANTS↗
NASA NTRS1987

Comparing hard and soft prior bounds in geophysical inverse problems

In linear inversion of a finite-dimensional data vector y to estimate a finite-dimensional prediction vector z, prior information about X sub E is essential if y is to supply useful limits for z. The one exception occurs when all the prediction functionals are linear combinations of the data functionals. Two forms of prior information are compared: a soft bound on X sub E is a probability distribution p sub x on X which describeds the observer's opinion about where X sub E is likely to be in X; a hard bound on X sub E is an inequality Q sub x(X sub E, X sub E) is equal to or less than 1, where Q sub x is a positive definite quadratic form on X. A hard bound Q sub x can be softened to many different probability distributions p sub x, but all these p sub x's carry much new information about X sub E which is absent from Q sub x, and some information which contradicts Q sub x. Both stochastic inversion (SI) and Bayesian inference (BI) estimate z from y and a soft prior bound p sub x. If that probability distribution was obtained by softening a hard prior bound Q sub x, rather than by objective statistical inference independent of y, then p sub x contains so much unsupported new information absent from Q sub x that conclusions about z obtained with SI or BI would seen to be suspect.

Backus, George E.↗
NASA NTRS1988

Comparing hard and soft prior bounds in geophysical inverse problems

In linear inversion of a finite-dimensional data vector y to estimate a finite-dimensional prediction vector z, prior information about X sub E is essential if y is to supply useful limits for z. The one exception occurs when all the prediction functionals are linear combinations of the data functionals. Two forms of prior information are compared: a soft bound on X sub E is a probability distribution p sub x on X which describes the observer's opinion about where X sub E is likely to be in X; a hard bound on X sub E is an inequality Q sub x(X sub E, X sub E) is equal to or less than 1, where Q sub x is a positive definite quadratic form on X. A hard bound Q sub x can be softened to many different probability distributions p sub x, but all these p sub x's carry much new information about X sub E which is absent from Q sub x, and some information which contradicts Q sub x. Both stochastic inversion (SI) and Bayesian inference (BI) estimate z from y and a soft prior bound p sub x. If that probability distribution was obtained by softening a hard prior bound Q sub x, rather than by objective statistical inference independent of y, then p sub x contains so much unsupported new information absent from Q sub x that conclusions about z obtained with SI or BI would seen to be suspect.

Backus, George E.↗
DOE OSTI2024

Community detection in hypergraphs via mutual information maximization

Abstract The hypergraph community detection problem seeks to identify groups of related vertices in hypergraph data. We propose an information-theoretic hypergraph community detection algorithm which compresses the observed data in terms of community labels and community-edge intersections. This algorithm can also be viewed as maximum-likelihood inference in a degree-corrected microcanonical stochastic blockmodel. We perform the compression/inference step via simulated annealing. Unlike several recent algorithms based on canonical models, our microcanonical algorithm does not require inference of statistical parameters such as vertex degrees or pairwise group connection rates. Through synthetic experiments, we find that our algorithm succeeds down to recently-conjectured thresholds for sparse random hypergraphs. We also find competitive performance in cluster recovery tasks on several hypergraph data sets.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Cosmic Coincidences of Primordial-Black-Hole Dark Matter

If primordial black holes (PBHs) contribute more than 10% of the dark matter (DM) density, their energy density today is of the same order as that of the baryons. Such a cosmic coincidence might hint at a mutual origin for the formation scenario of PBHs and the baryon asymmetry of the Universe. Baryogenesis can be triggered by a sharp transition of the rolling rate of inflaton from slow-roll to (nearly) ultraslow-roll phases that produce large curvature perturbations for PBH formation in single-field inflationary models. We show that the baryogenesis requirement drives the PBH contribution to DM, along with the inferred PBH mass range, the resulting stochastic gravitational wave background frequency window, and the associated cosmic microwave background tensor-to-scalar ratio amplitude, into potentially observable regimes.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2023

Nonparametric Inference for the Reproductive Rate in Generalized Compartmental Models

We develop a tractable nonparametric model for the time-varying reproductive rate of infectious diseases that combines the structure of a deterministic compartmental model and a stochastic model for incidence data. We use Bayesian inference to estimate, with uncertainty, the reproductive rate of the Coronavirus 2019 outbreak in the U.S. states of California, Florida, Michigan, New Mexico, New York, and Texas from January 2020 to March 2022. Employing the inferred reproductive rates, we estimate the posterior distribution of the time-varying reproduction numbers for each state. Compering the time-varying reproduction numbers across the states, we identify some epidemic waves, potentially driven from changes in human behavior and virus mutations.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Application of Sequential Design of Experiments (SDoE) to Large Pilot-Scale Solvent-Based CO2 Capture Process at Technology Centre Mongstad (TCM)

The United States Department of Energy’s Carbon Capture Simulation for Industry Impact (CCSI2) program has developed a framework for sequential design of experiments (SDoE) that aims to maximize knowledge gained from budget- and schedule-limited pilot scale testing. SDoE was applied to the planning and execution of campaigns for testing CO2 capture systems at pilot-scale in order to optimally allocate resources available for the testing. In this methodology, a stochastic process model is developed by quantifying the parametric uncertainty in submodels of interest; for a solvent-based CO2 capture system, these may include physical properties and equipment performance submodels (e.g., mass transfer, interfacial area). This uncertainty is propagated through the full process model, over variable operating conditions, for estimating the resulting uncertainty in key model outputs (e.g., percentage of CO2 capture, solvent regeneration energy requirement). In developing a data collection plan, the predicted output uncertainty is incorporated into an algorithm that seeks simultaneously to select process operating conditions for which the predicted uncertainty is relatively high and to ensure that the entire space of operation is well represented. This test plan is then used to guide operation of the pilot plant at varying steady-state conditions, with resulting process data incorporated into the existing model using Bayesian inference to refine parameter distributions. The updated stochastic model, with reduced parametric uncertainty from data collected, is then used to guide additional data collection, thus the sequential nature of the experimental design. The SDoE process was implemented at the pilot test unit (12 MWe in scale) at Norway’s Technology Centre Mongstad (TCM) in a summer 2018 test campaign with aqueous monoethanolamine (MEA). During the test campaign, the varied operating conditions included the flowrates of circulated solvent, flue gas, and reboiler steam and the CO2 concentration in the flue gas. The process data were used to update probability distributions of mass transfer and interfacial area parameters of a stochastic process model developed by the CCSI2 team. Two iterations of the SDoE process were executed, resulting in the uncertainty in model predicted CO2 capture percentage decreasing by an average of 58.0 ± 4.7% over the full input space of interest. This work demonstrates the potential of the SDoE process for model refinement through reduction in process model parametric uncertainty, and ultimately risk in scale-up, in CO2 capture technology performance.

carbon capture↗