Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Stochastic partial differential equation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Probabilistic learning on manifolds constrained by nonlinear partial differential equations for small datasets

A novel extension of the Probabilistic Learning on Manifolds (PLoM) is presented. It makes it possible to synthesize solutions to a wide range of nonlinear stochastic boundary value problems described by partial differential equations (PDEs) for which a stochastic computational model (SCM) is available and which depend on a vector-valued random control parameter. The cost of a single numerical evaluation of this SCM is assumed to be such that only a limited number of points can be computed for constructing the training dataset (small data). Each point of the training dataset is made up of realizations from a vector-valued stochastic process (the stochastic solution) and the associated random control parameter on which it depends. The presented PLoM constrained by PDE allows for generating a large number of learned realizations of the stochastic process and its corresponding random control parameter. These learned realizations are generated so as to minimize the vector-valued random residual of the PDE in the mean-square sense. Appropriate novel methods are developed to solve this challenging problem. Three applications are presented. The first one is a simple uncertain nonlinear dynamical system with a nonstationary stochastic excitation. The second one concerns the 2D nonlinear unsteady Navier–Stokes equations for incompressible flows in which the Reynolds number is the random control parameter. Here, the last one deals with the nonlinear dynamics of a 3D elastic structure with uncertainties. The results obtained make it possible to validate the PLoM constrained by stochastic PDE but also provide further validation of the PLoM without constraint.

Machine learning↗
DOE OSTI2022

Discovering causal structure with reproducing-kernel Hilbert space ε -machines

We merge computational mechanics’ definition of causal states (predictively equivalent histories) with reproducing-kernel Hilbert space (RKHS) representation inference. The result is a widely applicable method that infers causal structure directly from observations of a system’s behaviors whether they are over discrete or continuous events or time. A structural representation—a finite- or infinite-state kernel ϵ-machine—is extracted by a reduced-dimension transform that gives an efficient representation of causal states and their topology. In this way, the system dynamics are represented by a stochastic (ordinary or partial) differential equation that acts on causal states. We introduce an algorithm to estimate the associated evolution operator. Paralleling the Fokker–Planck equation, it efficiently evolves causal-state distributions and makes predictions in the original data space via an RKHS functional mapping. We demonstrate these techniques, together with their predictive abilities, on discrete-time, discrete-value infinite Markov-order processes generated by finite-state hidden Markov models with (i) finite or (ii) uncountably infinite causal states and (iii) continuous-time, continuous-value processes generated by thermally driven chaotic flows. The method robustly estimates causal structure in the presence of varying external and measurement noise levels and for very high-dimensional data.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2025

Score-Based Physics-Informed Neural Networks for High-Dimensional Fokker–Planck Equations

The Fokker-Planck (FP) equation is a foundational partial differential equation (PDE) in stochastic processes involving Brownian motions. However, the curse of dimensionality (CoD) poses a formidable challenge when dealing with high-dimensional FP equations. Although Monte Carlo simulation and (vanilla) Physics-Informed Neural Networks (PINNs) have shown the potential to tackle CoD, both methods exhibit significant numerical errors in high dimensions when dealing with the probability density function (PDF) associated with Brownian motion. The point-wise PDF values tend to decrease exponentially as dimensionality increases, surpassing the precision of numerical simulations and resulting in substantial errors. In addition, due to its massive sampling, Monte Carlo fails to offer fast sampling. Modeling the logarithm likelihood (LL) via vanilla PINNs transforms the FP equation into a notoriously difficult Hamilton-Jacobi-Bellman (HJB) equation, which is impractical for PINN learning, whose error grows rapidly with dimension. To this end, we propose a novel approach utilizing a score-based solver to fit the score function in stochastic differential equations (SDEs). The score function, defined as the gradient of the LL, plays a fundamental role in inferring LL and PDF and enables fast SDE sampling, offering an effective means to overcome the CoD. Three fitting methods, Score Matching (SM), Sliced Score Matching (SSM), and Score-PINN, are introduced, each contributing unique advantages in computational complexity, accuracy, and generality. The proposed score-based SDE solver operates in two stages: first, employing score matching or Score-PINN to acquire the score function; and second, solving the LL via an ordinary differential equation (ODE) using the obtained score function. Comparative evaluations across these methods showcase varying trade-offs. The proposed methodology is evaluated across diverse SDEs, including anisotropic Ornstein-Uhlenbeck processes, geometric Brownian motion, and Brownian motion with varying eigenspace. We also test various distributions, including Gaussian, Log-normal, Laplace, and Cauchy distributions. The numerical results demonstrate the score-based SDE solver’s stability, speed, and performance across different experimental settings, solidifying its potential as a solution to CoD for high-dimensional FP equations.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Effect of diffusion constant on the morphology of dendrite growth in lithium metal batteries

Lithium dendrites can lead to a short circuit and battery failure, and developing strategies for their suppression is of considerable importance. In this work, we study the growth of dendrites in a simple model system where the solvent is a continuum and the lithium ions are hard spheres that can deposit by sticking to existing spheres or the electrode surface. Using stochastic dynamics simulations, we investigate the effect of applied voltage and diffusion constant on the growth of dendrites. We find that the diffusion constant is the most significant factor, and the inhomogeneity of the electric field does not play a significant role. Additionally, the growth is most pronounced when the applied voltage and diffusion constant are both low. We observe a structural change from broccoli to cauliflower shape as the diffusion constant is increased. The simulations suggest that a control of electrolyte parameters that impact lithium diffusion might be an attractive route to controlling dendrite growth.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2022

Exponential Decay of Sensitivity in Graph-Structured Nonlinear Programs

We study solution sensitivity for nonlinear programs (NLPs) whose structures are induced by graphs. These NLPs arise in many applications such as dynamic optimization, stochastic optimization, optimization with partial differential equations, and network optimization. We illustrate that for a given pair of nodes, the sensitivity of the primal-dual solution at one node against a data perturbation at the other node decays exponentially with respect to the distance between these two nodes on the graph. In other words, the solution sensitivity decays as one moves away from the perturbation point. This result, which we call exponential decay of sensitivity, holds under the strong second-order sufficiency condition and the linear independence constraint qualification. We also present conditions under which the decay rate remains uniformly bounded; this allows us to characterize the sensitivity behavior of NLPs defined over subgraphs of infinite graphs. The theoretical developments are illustrated with numerical examples.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Adaptive Uncertainty Quantification for Stochastic Hyperbolic Conservation Laws

Here, we propose a predictor-corrector adaptive method for the study of hyperbolic partial differential equations (PDEs) under uncertainty. Constructed around the framework of stochastic finite volume (SFV) methods, our approach circumvents sampling schemes or simulation ensembles while also preserving fundamental properties, in particular hyperbolicity of the resulting systems and conservation of the discrete solutions. Furthermore, we augment the existing SFV theory with a priori convergence results for statistical quantities, in particular push-forward densities, which we demonstrate through numerical experiments. By linking refinement indicators to regions of the physical and stochastic spaces, we drive anisotropic refinements of the discretizations, introducing new degrees of freedom where deemed profitable. To illustrate our proposed method, we consider a series of numerical examples for nonlinear hyperbolic PDEs based on Burgers’ and Euler’s equations.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

A Feynman-Kac based numerical method for the exit time probability of a class of transport problems

The exit time probability, which gives the likelihood that an initial condition leaves a prescribed region of the phase space of a dynamical system at, or before, a given time, is arguably one of the most natural and important transport problems. In this work, we present an accurate and efficient numerical method for computing this probability for systems described by non-autonomous (time-dependent) stochastic differential equations (SDEs) or their equivalent Fokker-Planck partial differential equations. The method is based on the direct approximation of the Feynman-Kac formula that establishes a link between the adjoint Fokker-Planck equation and the forward SDE. The Feynman-Kac formula is approximated using the Gauss-Hermite quadrature rules and piecewise cubic Hermite interpolating polynomials, and a GPU accelerated matrix representation is used to compute the entire time evolution of the exit time probability using a single pass of the algorithm. The method is unconditionally stable, exhibits second order convergence in space, first order convergence in time, and it is straightforward to parallelize. Applications are presented to the advection diffusion of a passive tracer in a fluid flow exhibiting chaotic advection, and to the runaway acceleration of electrons in a plasma in the presence of an electric field, collisions, and radiation damping. Benchmarks against analytical solutions as well as comparisons with explicit and implicit finite difference standard methods for the adjoint Fokker-Planck equation are presented.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2022

Kernel learning backward SDE filter for data assimilation

In this paper, we develop a kernel learning backward SDE filter method to estimate the state of a stochastic dynamical system based on its partial noisy observations. A system of forward backward stochastic differential equations is used to propagate the state of the target dynamical model, and Bayesian inference is applied to incorporate the observational information. Further, to characterize the dynamical model in the entire state space, we introduce a kernel learning method to learn a continuous global approximation for the conditional probability density function of the target state by using discrete approximated density values as training data. Numerical experiments demonstrate that the kernel learning backward SDE is highly effective.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Stochastic Learning Approach for Binary Optimization: Application to Bayesian Optimal Design of Experiments

Here, we present a novel stochastic approach to binary optimization suited for optimal experimental design (OED) for Bayesian inverse problems governed by mathematical models such as partial differential equations. The OED utility function, namely, the regularized optimality criterion, is cast into a stochastic objective function in the form of an expectation over a multivariate Bernoulli distribution. The probabilistic objective is then solved by using a stochastic optimization routine to find an optimal observational policy. This formulation (a) is generally applicable to binary optimization problems with soft constraints and is ideal for OED and sensor placement problems; (b) does not require differentiability of the original objective function (e.g., a utility function in OED applications) with respect to the design variable, and thus it enables direct employment of sparsity-enforcing penalty functions such as $\ell_0$, without needing to utilize a continuation procedure or apply a rounding technique; (c) exhibits much lower computational cost than traditional gradient-based relaxation approaches; and (d) can be applied to both linear and nonlinear OED problems with proper choice of the utility function. The proposed approach is analyzed from an optimization perspective with detailed convergence analysis of the optimization approach and is also analyzed from a machine learning perspective with correspondence to policy gradient reinforcement learning. The approach is demonstrated numerically by using an idealized two-dimensional Bayesian linear inverse problem and validated by extensive numerical experiments carried out for sensor placement in a parameter identification setup.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Block triangular preconditioning for stochastic Galerkin method

Here, in this paper, we develop a new block triangular preconditioner for solving partial differential equations with random coefficients. We prove spectral bounds for the preconditioned system. Several numerical examples are provided to demonstrate the efficiency of this preconditioner, especially for stochastic problems with large variance.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Stochastic finite volume method for uncertainty quantification of transient flow in gas pipeline networks

We develop a weakly intrusive framework to simulate the propagation of uncertainty in solutions of generic hyperbolic partial differential equation systems on graph-connected domains with nodal coupling and boundary conditions. The method is based on the Stochastic Finite Volume (SFV) approach and can be applied for uncertainty quantification (UQ) of the dynamical state of fluid flow over actuated transport networks. The numerical scheme has specific advantages for modeling intertemporal uncertainty in time-varying boundary parameters, which cannot be characterized by strict upper and lower (interval) bounds. We describe the scheme for a single pipe, and then formulate the controlled junction Riemann problem (JRP) that enables the extension to general network structures. In conclusion, we demonstrate the method's capabilities and performance characteristics using a standard benchmark test network.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

A survey of unsupervised learning methods for high-dimensional uncertainty quantification in black-box-type problems

Constructing surrogate models for uncertainty quantification (UQ) on complex partial differential equations (PDEs) having inherently high-dimensional O(10 n ), n ≥ 2, stochastic inputs (e.g., forcing terms, boundary conditions, initial conditions) poses tremendous challenges. The “curse of dimensionality” can be addressed with suitable unsupervised learning techniques used as a pre-processing tool to encode inputs onto lower-dimensional subspaces while retaining its structural information and meaningful properties. In this work, we review and investigate thirteen dimension reduction methods including linear and nonlinear, spectral, blind source separation, convex and non-convex methods and utilize the resulting embeddings to construct a mapping to quantities of interest via polynomial chaos expansions (PCE). Here, we refer to the general proposed approach as manifold PCE (m-PCE), where manifold corresponds to the latent space resulting from any of the studied dimension reduction methods. To investigate the capabilities and limitations of these methods we conduct numerical tests for three physics-based systems (treated as black-boxes) having high-dimensional stochastic inputs of varying complexity modeled as both Gaussian and non-Gaussian random fields to investigate the effect of the intrinsic dimensionality of input data. We demonstrate both the advantages and limitations of the unsupervised learning methods and we conclude that a suitable m-PCE model provides a cost-effective approach compared to alternative algorithms proposed in the literature, including recently proposed expensive deep neural network-based surrogates and can be readily applied for high-dimensional UQ in stochastic PDEs.

42 ENGINEERING↗
DOE OSTI2020

An Efficient Numerical Algorithm for Solving Data Driven Feedback Control Problems

The goal of this paper is to solve a class of stochastic optimal control problems numerically, in which the state process is governed by an Itô type stochastic differential equation with control process entering both in the drift and the diffusion, and is observed partially. The optimal control of feedback form is determined based on the available observational data. In this work, we call this type of control problems the data driven feedback control. The computational framework that we introduce to solve such type of problems aims to find the best estimate for the optimal control as a conditional expectation given the observational information. To make our method feasible in providing timely feedback to the controlled system from data, we develop an efficient stochastic optimization algorithm to implement our computational framework.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Learning and meta-learning of stochastic advection–diffusion–reaction systems from sparse measurements

Physics-informed neural networks (PINNs) were recently proposed in [18] as an alternative way to solve partial differential equations (PDEs). A neural network (NN) represents the solution, while a PDE-induced NN is coupled to the solution NN, and all differential operators are treated using automatic differentiation. Here, we first employ the standard PINN and a stochastic version, sPINN, to solve forward and inverse problems governed by a non-linear advection–diffusion–reaction (ADR) equation, assuming we have some sparse measurements of the concentration field at random or pre-selected locations. Subsequently, we attempt to optimise the hyper-parameters of sPINN by using the Bayesian optimisation method (meta-learning) and compare the results with the empirically selected hyper-parameters of sPINN. In particular, for the first part in solving the inverse deterministic ADR, we assume that we only have a few high-fidelity measurements, whereas the rest of the data is of lower fidelity. Hence, the PINN is trained using a composite multi-fidelity network, first introduced in [12], that learns the correlations between the multi-fidelity data and predicts the unknown values of diffusivity, transport velocity and two reaction constants as well as the concentration field. For the stochastic ADR, we employ a Karhunen–Loève (KL) expansion to represent the stochastic diffusivity, and arbitrary polynomial chaos (aPC) to represent the stochastic solution. Correspondingly, we design multiple NNs to represent the mean of the solution and learn each aPC mode separately, whereas we employ a separate NN to represent the mean of diffusivity and another NN to learn all modes of the KL expansion. For the inverse problem, in addition to stochastic diffusivity and concentration fields, we also aim to obtain the (unknown) deterministic values of transport velocity and reaction constants. The available data correspond to 7spatial points for the diffusivity and 20 space–time points for the solution, both sampled 2000 times. We obtain good accuracy for the deterministic parameters of the order of 1–2% and excellent accuracy for the mean and variance of the stochastic fields, better than three digits of accuracy. In the second part, we consider the previous stochastic inverse problem, and we use Bayesian optimisation to find five hyper-parameters of sPINN, namely the width, depth and learning rate of two NNs for learning the modes. Here, we obtain much deeper and wider optimal NNs compared to the manual tuning, leading to even better accuracy, i.e., errors less than 1% for the deterministic values, and about an order of magnitude less for the stochastic fields.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2019

Stochastic self-tuning hybrid algorithm for reaction-diffusion systems

Many biochemical phenomena involve reactants with vastly different concentrations, some of which are amenable to continuum-level descriptions, while the others are not. We present a hybrid self-tuning algorithm to model such systems. The method combines microscopic (Brownian) dynamics for diffusion with mesoscopic (Gillespie-type) methods for reactions and remains efficient in a wide range of regimes and scenarios with large variations of concentrations. Furthermore, its accuracy, robustness, and versatility are balanced by redefining propensities and optimizing the mesh size and time step. We use a bimolecular reaction to demonstrate the potential of our method in a broad spectrum of scenarios: from almost completely reaction-dominated systems to cases where reactions rarely occur or take place very slowly. The simulation results show that the number of particles present in the system does not degrade the performance of our method. This makes it an accurate and computationally efficient tool to model complex multireaction systems.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2021

Bayesian sparse learning with preconditioned stochastic gradient MCMC and its applications

Deep neural networks have been successfully employed in an extensive variety of research areas, including solving partial differential equations. Despite its significant success, there are some challenges in effectively training DNN, such as avoiding overfitting in over-parameterized DNNs and accelerating the optimization in DNNs with pathological curvature. Here, we propose a Bayesian type sparse deep learning algorithm. The algorithm utilizes a set of spike-and-slab priors for the parameters in the deep neural network. The hierarchical Bayesian mixture will be trained using an adaptive empirical method. That is, one will alternatively sample from the posterior using preconditioned stochastic gradient Langevin Dynamics (PSGLD), and optimize the latent variables via stochastic approximation. The sparsity of the network is achieved while optimizing the hyperparameters with adaptive searching and penalizing. A popular SG-MCMC approach is Stochastic gradient Langevin dynamics (SGLD). However, considering the complex geometry in the model parameter space in nonconvex learning, updating parameters using a universal step size in each component as in SGLD may cause slow mixing. To address this issue, we apply a computationally manageable preconditioner in the updating rule, which provides a step-size parameter to adapt to local geometric properties. Moreover, by smoothly optimizing the hyperparameter in the preconditioning matrix, our proposed algorithm ensures a decreasing bias, which is introduced by ignoring the correction term in the preconditioned SGLD. According to the existing theoretical framework, we show that the proposed algorithm can asymptotically converge to the correct distribution with a controllable bias under mild conditions. Numerical tests are performed on both synthetic regression problems and learning solutions of elliptic PDE, which demonstrate the accuracy and efficiency of the present work.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2025

Accelerating multilevel Markov Chain Monte Carlo using machine learning models

Here, this work presents an efficient approach for accelerating multilevel Markov Chain Monte Carlo (MCMC) sampling for large-scale problems using low-fidelity machine learning models. While conventional techniques for large-scale Bayesian inference often substitute computationally expensive high-fidelity models with machine learning models, thereby introducing approximation errors, our approach offers a computationally efficient alternative by augmenting high-fidelity models with low-fidelity ones within a hierarchical framework. The multilevel approach utilizes the low-fidelity machine learning model (MLM) for inexpensive evaluation of proposed samples thereby improving the acceptance of samples by the high-fidelity model. The hierarchy in our multilevel algorithm is derived from geometric multigrid hierarchy. We utilize an MLM to accelerate the coarse level sampling. Training machine learning model for the coarsest level significantly reduces the computational cost associated with generating training data and training the model. We present an MCMC algorithm to accelerate the coarsest level sampling using MLM and account for the approximation error introduced. We provide theoretical proofs of detailed balance and demonstrate that our multilevel approach constitutes a consistent MCMC algorithm. Additionally, we derive the expression for cost reduction due to machine learning model to facilitate cost analysis of the hierarchical sampling algorithm. Our technique is demonstrated on a standard benchmark inference problem in groundwater flow, where we estimate the probability density of a quantity of interest using a four-level MCMC algorithm. Our proposed algorithm accelerates multilevel sampling by a factor of two while achieving similar accuracy compared to sampling using the standard multilevel algorithm.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Accurate data-driven surrogates of dynamical systems for forward propagation of uncertainty

Stochastic collocation (SC) is a well-known non-intrusive method of constructing surrogate models for uncertainty quantification. In dynamical systems, SC is especially suited for full-field uncertainty propagation that characterizes the distributions of the high-dimensional solution fields of a model with stochastic input parameters. However, due to the highly nonlinear nature of the parameter-to-solution map in even the simplest dynamical systems, the constructed SC surrogates are often inaccurate. Here, this work presents an alternative approach, where we apply the SC approximation over the dynamics of the model, rather than the solution. By combining the data-driven sparse identification of nonlinear dynamics framework with SC, we construct dynamics surrogates and integrate them through time to construct the surrogate solutions. We demonstrate that the SC-over-dynamics framework leads to smaller errors, both in terms of the approximated system trajectories as well as the model state distributions, when compared against full-field SC applied to the solutions directly. We present numerical evidence of this improvement using three test problems: a chaotic ordinary differential equation, and two partial differential equations from solid mechanics.

42 ENGINEERING↗