Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Stochastic partial differential equation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

Path Properties of Atmospheric Transitions: Illustration with a Low-Order Sudden Stratospheric Warming Model

Many rare weather events, including hurricanes, droughts, and floods, dramatically impact human life. To accurately forecast these events and characterize their climatology requires specialized mathematical techniques to fully leverage the limited data that are available. Here we describe transition path theory (TPT), a framework originally developed for molecular simulation, and argue that it is a useful paradigm for developing mechanistic understanding of rare climate events. TPT provides a method to calculate statistical properties of the paths into the event. As an initial demonstration of the utility of TPT, we analyze a low-order model of sudden stratospheric warming (SSW), a dramatic disturbance to the polar vortex that can induce extreme cold spells at the surface in the midlatitudes. SSW events pose a major challenge for seasonal weather prediction because of their rapid, complex onset and development. Climate models struggle to capture the long-term statistics of SSW, owing to their diversity and intermittent nature. We use a stochastically forced Holton–Mass-type model with two stable states, corresponding to radiative equilibrium and a vacillating SSW-like regime. In this stochastic bistable setting, from certain probabilistic forecasts TPT facilitates estimation of dominant transition pathways and return times of transitions. These “dynamical statistics” are obtained by solving partial differential equations in the model’s phase space. With future application to more complex models, TPT and its constituent quantities promise to improve the predictability of extreme weather events through both generation and principled evaluation of forecasts.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

Surrogate approximation of the Grad–Shafranov free boundary problem via stochastic collocation on sparse grids

In magnetic confinement fusion devices, the equilibrium configuration of a plasma is determined by the balance between the hydrostatic pressure in the fluid and the magnetic forces generated by an array of external coils and the plasma itself. The location of the plasma is not known a priori and must be obtained as the solution to a free boundary problem. The partial differential equation that determines the behavior of the combined magnetic field depends on a set of physical parameters (location of the coils, intensity of the electric currents going through them, magnetic permeability, etc.) that are subject to uncertainty and variability. The confinement region is in turn a function of these stochastic parameters as well. In this work, we consider variations on the current intensities running through the external coils as the dominant source of uncertainty. This leads to a parameter space of dimension equal to the number of coils in the reactor. With the aid of a surrogate function built on a sparse grid in parameter space, a Monte Carlo strategy is used to explore the effect that stochasticity in the parameters has on important features of the plasma boundary such as the location of the x-point, the strike points, and shaping attributes such as triangularity and elongation. Furthermore, the use of the surrogate function reduces the time required for the Monte Carlo simulations by factors that range between 7 and over 30.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2026

Large-scale white noise and cosmology

The generation of white noise on large scales is a generic property of the dynamics of physical systems described by local nonlinear partial differential equations. Nonlinearities prevent the small-scale dynamics from being erased by smoothing. Unresolved small-scale dynamics act as an uncorrelated (white or Poissonian) noise (seemingly stochastic but actually deterministic) contribution to large-scale dynamics. This white noise exists even when the dynamics is very nearly linear. In cases where the power spectrum is sub-Poissonian on large scales, this noise will dominate on the largest scale power no matter the amplitude of the inhomogeneities. Such is the case in the standard model of cosmology, where the primordial density power spectrum is expected to have an almost Harrison-Zel’dovich, P [ k ] ∼ k , spectrum on a much broader range of scales than can be observed. Even though linear gravitational evolution dominates nonlinear corrections by a factor of ∼ 10 5 , the nonobservation of white noise on the Hubble scale precludes the extrapolation of this power law below the comoving 1 pc scale. More generally, observation or nonobservation of large-scale white noise provides a powerful probe of the Universe on very small scales in the early Universe. Gravitational radiation, phase transitions, vorticity, and running of the spectral index are all phenomena that can be probed with large-scale white noise. Large-scale white noise is a nonoptional feature of all cosmological models but one which has not heretofore been appreciated.

Barenboim, Gabriela [Valencia U., IFIC; Valencia U↗
DOE OSTI2024

Inferring Stochastic Rates from Heterogeneous Snapshots of Particle Positions

Many imaging techniques for biological systems—like fixation of cells coupled with fluorescence microscopy—provide sharp spatial resolution in reporting locations of individuals at a single moment in time but also destroy the dynamics they intend to capture. In this study, these snapshot observations contain no information about individual trajectories, but still encode information about movement and demographic dynamics, especially when combined with a well-motivated biophysical model. The relationship between spatially evolving populations and single-moment representations of their collective locations is well-established with partial differential equations (PDEs) and their inverse problems. However, experimental data is commonly a set of locations whose number is insufficient to approximate a continuous-in-space PDE solution. Here, motivated by popular subcellular imaging data of gene expression, we embrace the stochastic nature of the data and investigate the mathematical foundations of parametrically inferring demographic rates from snapshots of particles undergoing birth, diffusion, and death in a nuclear or cellular domain. Toward inference, we rigorously derive a connection between individual particle paths and their presentation as a Poisson spatial process. Using this framework, we investigate the properties of the resulting inverse problem and study factors that affect quality of inference. One pervasive feature of this experimental regime is the presence of cell-to-cell heterogeneity. Rather than being a hindrance, we show that cell-to-cell geometric heterogeneity can increase the quality of inference on dynamics for certain parameter regimes. Altogether, the results serve as a basis for more detailed investigations of subcellular spatial patterns of RNA molecules and other stochastically evolving populations that can only be observed for single instants in their time evolution.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2024

A meshless stochastic method for Poisson–Nernst–Planck equations

A plethora of biological, physical, and chemical phenomena involve transport of charged particles (ions). Its continuum-scale description relies on the Poisson–Nernst–Planck (PNP) system, which encapsulates the conservation of mass and charge. The numerical solution of these coupled partial differential equations is challenging and suffers from both the curse of dimensionality and difficulty in efficiently parallelizing. We present a novel particle-based framework to solve the full PNP system by simulating a drift–diffusion process with time- and space-varying drift. We leverage Green’s functions, kernel-independent fast multipole methods, and kernel density estimation to solve the PNP system in a meshless manner, capable of handling discontinuous initial states. The method is embarrassingly parallel, and the computational cost scales linearly with the number of particles and dimension. We use a series of numerical experiments to demonstrate both the method’s convergence with respect to the number of particles and computational cost vis-à-vis a traditional partial differential equation solver.

Chemistry↗
DOE OSTI2022

Time-series forecasting using manifold learning, radial basis function interpolation, and geometric harmonics

We address a three-tier numerical framework based on nonlinear manifold learning for the forecasting of high-dimensional time series, relaxing the “curse of dimensionality” related to the training phase of surrogate/machine learning models. At the first step, we embed the high-dimensional time series into a reduced low-dimensional space using nonlinear manifold learning (local linear embedding and parsimonious diffusion maps). Then, we construct reduced-order surrogate models on the manifold (here, for our illustrations, we used multivariate autoregressive and Gaussian process regression models) to forecast the embedded dynamics. Finally, we solve the pre-image problem, thus lifting the embedded time series back to the original high-dimensional space using radial basis function interpolation and geometric harmonics. The proposed numerical data-driven scheme can also be applied as a reduced-order model procedure for the numerical solution/propagation of the (transient) dynamics of partial differential equations (PDEs). In conclusion, we assess the performance of the proposed scheme via three different families of problems: (a) the forecasting of synthetic time series generated by three simplistic linear and weakly nonlinear stochastic models resembling electroencephalography signals, (b) the prediction/propagation of the solution profiles of a linear parabolic PDE and the Brusselator model (a set of two nonlinear parabolic PDEs), and (c) the forecasting of a real-world data set containing daily time series of ten key foreign exchange rates spanning the time period 3 September 2001–29 October 2020.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2022

Learning generative neural networks with physics knowledge

Deep generative neural networks have enabled modeling complex distributions, but incorporating physics knowledge into the neural networks is still challenging and is at the core of current physics-based machine learning research. To this end, we propose a physics generative neural network (PhysGNN), a new class of generative neural networks for learning unknown distributions in a physical system described by partial differential equations (PDE). PhysGNN couples PDE systems with generative neural networks. It is a fully differentiable model that allows back-propagation of gradients through both numerical PDE solvers and generative neural networks, and is trained by minimizing the discrete Wasserstein distance between generated and observed probability distributions of the PDE outputs using the stochastic gradient descent method. Moreover, PhysGNN does not require adversarial training like standard generative neural networks, which offers better stability than adversarial training. We show that PhysGNN can learn complex distributions in stochastic inverse problems, where conventional methods such as maximum likelihood estimation and momentum matching methods may be inapplicable when little knowledge is known about the form of unknown distributions or the physical model is too complex. Furthermore, our method allows physics-based generative neural network training for learning complex distributions in the context of differential equations.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Bias-Variance Trade-Off in Physics-Informed Neural Networks with Randomized Smoothing for High-Dimensional PDEs

Physics-Informed Neural Networks (PINNs) have triggered a paradigm shift in scientific computing, leveraging mesh-free properties and robust approximation capabilities. While proving effective for low-dimensional partial differential equations (PDEs), the computational cost of PINNs remains a hurdle in high-dimensional scenarios. This is particularly pronounced when computing high-order and high-dimensional derivatives in the physics-informed loss. Randomized Smoothing PINN (RS-PINN) introduces Gaussian noise for stochastic smoothing of the original neural net model, enabling the use of Monte Carlo methods for derivative approximation, which eliminates the need for costly automatic differentiation. Despite its computational efficiency, especially in the approximation of high-dimensional derivatives, RS-PINN introduces biases in both loss and gradients, negatively impacting convergence, especially when coupled with stochastic gradient descent (SGD) algorithms. We present a comprehensive analysis of biases in RS-PINN, attributing them to the nonlinearity of the Mean Squared Error (MSE) loss as well as the intrinsic nonlinearity of the PDE itself. We propose tailored bias correction techniques, delineating their application based on the order of PDE nonlinearity. The derivation of an unbiased RS-PINN allows for a detailed examination of its advantages and disadvantages compared to the biased version. Specifically, the biased version has a lower variance and runs faster than the unbiased version, but it is less accurate due to the bias. To optimize the bias-variance trade-off, we combine the two approaches in a hybrid method that balances the rapid convergence of the biased version with the high accuracy of the unbiased version. In addition to methodological contributions, we present an enhanced implementation of RS-PINN. Extensive experiments on diverse high-dimensional PDEs, including Fokker-Planck, Hamilton-Jacobi-Bellman (HJB), viscous Burgers’, Allen-Cahn, and Sine-Gordon equations, illustrate the bias-variance trade-off and highlight the effectiveness of the hybrid RS-PINN. Empirical guidelines are provided for selecting biased, unbiased, or hybrid versions, depending on the dimensionality and nonlinearity of the specific PDE problem.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Assimilating partial observation to enhance feedback control of stochastic dynamical systems

Here, in this paper, we present a novel methodology to tackle feedback optimal control problems in scenarios where the exact state of the controlled process is unknown. It integrates data assimilation techniques and optimal control solvers to manage partial observation of the state process, a common occurrence in practical scenarios. Traditional stochastic optimal control methods assume full state observation, which is often not feasible in real-world fluid dynamics control problems. Our approach underscores the significance of utilizing observational data to inform control policy design. Specifically, we introduce a kernel learning backward stochastic differential equation (SDE) filter to enhance data assimilation efficiency and propose a sample-wise stochastic optimization method within the stochastic maximum principle framework. We demonstrate the efficacy and accuracy of our method in the control of advection-diffusion-reaction flow problem and the Dubins airplane maneuvering problem with model uncertainty.

data driven↗
DOE OSTI2026

Structural Aspects of Neutron Survival Probabilities

The neutron survival probability (and related quantities including probabilities of extinction and initiation) is a central element of the broader stochastic theory of neutron populations and finds application in fields including reactor start-up, analysis of reactor power bursts and criticality accidents, and safeguards. In a full neutron transport formulation, the equation governing the single-neutron survival probability is a backward or adjoint-like integro-partial differential equation with the added complexity of being highly nonlinear. Analogous formulations of this equation exist in the context of many approximate theories of neutron transport, with the point kinetics formulation having received significant theoretical attention since the 1940s. This work continues this tradition by providing a novel analysis of the single-neutron survival probability equation using the tools of boundary layer theory. The analysis reveals that the “fully dynamic” solution of the single-neutron survival probability equation—and some key probability distributions derived from it—may be cast as a singular perturbation around the underlying quasi-static single-neutron probability of initiation. In this perturbation solution, the expansion parameter is the ratio of the neutron generation time to a macroscopic time scale characterizing the overall system evolution; this interpretation illuminates some of the fundamental structural aspects of neutron survival phenomena.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Pyomo.DOE: An open-source package for model-based design of experiments in Python

Predictive mathematical models are a cornerstone of science and engineering. Yet selecting, calibrating, and validating said science-based models often remains an art in practice. Model-based design of experiments (MBDoE) provides a systematic framework to maximize information gain from experiments while minimizing time and resource costs. But MBDoE remains limited to niche application areas, in part because practitioners must integrate expertise in statistics, computational optimization, and modeling. To help reduce this barrier, we introduce Pyomo.DOE, an open-source package for MBDoE. Pyomo.DOE uses a nonlinear sensitivity analysis code k_aug to quickly approximate the Fisher information matrix and leverages a new stochastic programming abstraction. We demonstrate Pyomo.DOE with the first application of MBDoE to fixed-bed breakthrough experiments, which highlights the power of Pyomo.DOE to quantify the value of experimental modifications a priori for large-scale partial differential-algebraic equation (PDAE) models. Here we also provide a mathematical primer on MBDoE targeted at general chemical engineers.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2021

Fractional Modeling in Action: A Survey of Nonlocal Models for Subsurface Transport, Turbulent Flows, and Anomalous Materials

Modeling of phenomena such as anomalous transport via fractional-order differential equations has been established as an effective alternative to partial differential equations, due to the inherent ability to describe large-scale behavior with greater efficiency than fully-resolved classical models. In this review article, we first provide a broad overview of fractional-order derivatives with a clear emphasis on the stochastic processes that underlie their use. We then survey three exemplary application areas – subsurface transport, turbulence, and anomalous materials – in which fractional-order differential equations provide accurate and predictive models. For each area, we report on the evidence of anomalous behavior that justifies the use of fractional-order models, and survey both foundational models as well as more expressive state-of-the-art models. We also propose avenues for future research, including more advanced and physically sound models, as well as tools for calibration and discovery of fractional-order models.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

A Probabilistic Scheme for Semilinear Nonlocal Diffusion Equations with Volume Constraints

This work presents a probabilistic scheme for solving semilinear nonlocal diffusion equations with volume constraints and integrable kernels. The nonlocal model of interest is defined by a time-dependent semilinear partial integro-differential equation (PIDE), in which the integro-differential operator consists of both local convection-diffusion and nonlocal diffusion operators. Here, our numerical scheme is based on the direct approximation of the nonlinear Feynman–Kac formula that establishes a link between nonlinear PIDEs and stochastic differential equations. The exploitation of the Feynman–Kac representation avoids solving dense linear systems arising from nonlocal operators. Compared with existing stochastic approaches, our method can achieve first-order convergence after balancing the temporal and spatial discretization errors, which is a significant improvement of existing probabilistic/stochastic methods for nonlocal diffusion problems. Error analysis of our numerical scheme is established. The effectiveness of our approach is shown in two numerical examples. The first example considers a three-dimensional nonlocal diffusion equation to numerically verify the error analysis results. The second example presents a physics problem motivated by the study of heat transport in magnetically confined fusion plasmas.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Predicting rare events using neural networks and short-trajectory data

Estimating the likelihood, timing, and nature of events is a major goal of modeling stochastic dynamical systems. When the event is rare in comparison with the timescales of simulation and/or measurement needed to resolve the elemental dynamics, accurate prediction from direct observations becomes challenging. In such cases a more effective approach is to cast statistics of interest as solutions to Feynman-Kac equations (partial differential equations). Here, we develop an approach to solve Feynman-Kac equations by training neural networks on short-trajectory data. Our approach is based on a Markov approximation but otherwise avoids assumptions about the underlying model and dynamics. This makes it applicable to treating complex computational models and observational data. Additionally, we illustrate the advantages of our method using a low-dimensional model that facilitates visualization, and this analysis motivates an adaptive sampling strategy that allows on-the-fly identification of and addition of data to regions important for predicting the statistics of interest. Finally, we demonstrate that we can compute accurate statistics for a 75-dimensional model of sudden stratospheric warming. This system provides a stringent test bed for our method.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

NSGA-PINN: A Multi-Objective Optimization Method for Physics-Informed Neural Network Training

This paper presents NSGA-PINN, a multi-objective optimization framework for the effective training of physics-informed neural networks (PINNs). The proposed framework uses the non-dominated sorting genetic algorithm (NSGA-II) to enable traditional stochastic gradient optimization algorithms (e.g., ADAM) to escape local minima effectively. Additionally, the NSGA-II algorithm enables satisfying the initial and boundary conditions encoded into the loss function during physics-informed training precisely. We demonstrate the effectiveness of our framework by applying NSGA-PINN to several ordinary and partial differential equation problems. In particular, we show that the proposed framework can handle challenging inverse problems with noisy data.

Lu, Binghang (ORCID:0009000160016632)↗
DOE OSTI2022

Learning functional priors and posteriors from data and physics

In this work, we develop a new Bayesian framework based on deep neural networks to be able to extrapolate in space-time using historical data and to quantify uncertainties arising from both noisy and gappy data in physical problems. Specifically, the proposed approach has two stages: (1) prior learning and (2) posterior estimation. At the first stage, we employ the physics-informed Generative Adversarial Networks (PI-GAN) to learn a functional prior either from a prescribed function distribution, e.g., Gaussian process, or from historical data and physics. At the second stage, we employ the Hamiltonian Monte Carlo (HMC) method to estimate the posterior in the latent space of PI-GANs. In addition, we use two different approaches to encode the physics: (1) automatic differentiation, used in the physicsinformed neural networks (PINNs) for scenarios with explicitly known partial differential equations (PDEs), and (2) operator regression using the deep operator network (DeepONet) for PDE-agnostic scenarios. We then test the proposed method for (1) meta-learning for one-dimensional regression, and forward/inverse PDE problems (combined with PINNs); (2) PDE-agnostic physical problems (combined with DeepONet), e.g., fractional diffusion as well as saturated stochastic (100-dimensional) flows in heterogeneous porous media; and (3) spatial-temporal regression problems, i.e., inference of a marine riser displacement field using experimental data from the Norwegian Deepwater Programme (NDP). The results demonstrate that the proposed approach can provide accurate predictions as well as uncertainty quantification given very limited scattered and noisy data, since historical data could be available to provide informative priors. In summary, the proposed method is capable of learning flexible functional priors, e.g., both Gaussian and non-Gaussian process, and can be readily extended to big data problems by enabling mini-batch training using stochastic HMC or normalizing flows since the latent space is generally characterized as low dimensional.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2024

Hutchinson Trace Estimation for high-dimensional and high-order Physics-Informed Neural Networks

Physics-Informed Neural Networks (PINNs) have proven effective in solving partial differential equations (PDEs), especially when some data are available by seamlessly blending data and physics. However, extending PINNs to high-dimensional and even high-order PDEs encounters significant challenges due to the computational cost associated with automatic differentiation in the residual loss function calculation. Herein, we address the limitations of PINNs in handling high-dimensional and high-order PDEs by introducing the Hutchinson Trace Estimation (HTE) method. Starting with the second-order high-dimensional PDEs, which are ubiquitous in scientific computing, HTE is applied to transform the calculation of the entire Hessian matrix into a Hessian vector product (HVP). This approach not only alleviates the computational bottleneck via Taylor-mode automatic differentiation but also significantly reduces memory consumption from the Hessian matrix to an HVP’s scalar output. We further showcase HTE’s convergence to the original PINN loss and its unbiased behavior under specific conditions. Comparisons with the Stochastic Dimension Gradient Descent (SDGD) highlight the distinct advantages of HTE, particularly in scenarios with significant variability and variance among dimensions. We further extend the application of HTE to higher-order and higher-dimensional PDEs, specifically addressing the biharmonic equation. By employing tensor-vector products (TVP), HTE efficiently computes the colossal tensor associated with the fourth-order high-dimensional biharmonic equation, saving memory and enabling rapid computation. The effectiveness of HTE is illustrated through experimental setups, demonstrating comparable convergence rates with SDGD under memory and speed constraints. Additionally, HTE proves valuable in accelerating the Gradient-Enhanced PINN (gPINN) version as well as the Biharmonic equation. Overall, HTE opens up a new capability in scientific machine learning for tackling high-order and high-dimensional PDEs.

Curse of dimensionality↗
DOE OSTI2025

Scalable multilevel Monte Carlo methods exploiting parallel redistribution on coarse levels

Here, we study an element agglomeration coarsening strategy that requires data redistribution at coarse levels when the number of coarse elements becomes smaller than the number of MPI processes used on the finest level. The overall procedure generates coarse elements (general unstructured unions of fine grid elements) within the framework of element-based algebraic multigrid methods (or AMGe) studied previously. The AMGe-generated coarse spaces have the ability to exhibit approximation properties of the same order as the fine-level spaces since by construction they contain the piecewise polynomials of the same order as on the fine level. These approximation properties are key for the successful use of AMGe in multilevel solvers for nonlinear partial differential equations as well as for multilevel Monte Carlo (MLMC) simulations. The ability to coarsen without being constrained by the number of MPI processes, as described in the present paper, allows to improve the scalability of these solvers as well as the overall MLMC method. The paper illustrates this latter fact with detailed scalability study of MLMC simulations applied to model Darcy equations with a stochastic log-normal permeability field.

AMGe↗