Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “differential equations”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 613 records · Page 34
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Learning functional priors and posteriors from data and physics

In this work, we develop a new Bayesian framework based on deep neural networks to be able to extrapolate in space-time using historical data and to quantify uncertainties arising from both noisy and gappy data in physical problems. Specifically, the proposed approach has two stages: (1) prior learning and (2) posterior estimation. At the first stage, we employ the physics-informed Generative Adversarial Networks (PI-GAN) to learn a functional prior either from a prescribed function distribution, e.g., Gaussian process, or from historical data and physics. At the second stage, we employ the Hamiltonian Monte Carlo (HMC) method to estimate the posterior in the latent space of PI-GANs. In addition, we use two different approaches to encode the physics: (1) automatic differentiation, used in the physicsinformed neural networks (PINNs) for scenarios with explicitly known partial differential equations (PDEs), and (2) operator regression using the deep operator network (DeepONet) for PDE-agnostic scenarios. We then test the proposed method for (1) meta-learning for one-dimensional regression, and forward/inverse PDE problems (combined with PINNs); (2) PDE-agnostic physical problems (combined with DeepONet), e.g., fractional diffusion as well as saturated stochastic (100-dimensional) flows in heterogeneous porous media; and (3) spatial-temporal regression problems, i.e., inference of a marine riser displacement field using experimental data from the Norwegian Deepwater Programme (NDP). The results demonstrate that the proposed approach can provide accurate predictions as well as uncertainty quantification given very limited scattered and noisy data, since historical data could be available to provide informative priors. In summary, the proposed method is capable of learning flexible functional priors, e.g., both Gaussian and non-Gaussian process, and can be readily extended to big data problems by enabling mini-batch training using stochastic HMC or normalizing flows since the latent space is generally characterized as low dimensional.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2022

Single-stage gradient-based stellarator coil design: Optimization for near-axis quasi-symmetry

Here we present a new coil design paradigm for magnetic confinement in stellarators. Our approach directly optimizes coil shapes and coil currents to produce a vacuum quasi-symmetric magnetic field with a target rotational transform on the magnetic axis. This approach differs from the traditional two-stage approach in which first a magnetic configuration with desirable physics properties is found, and then coils to approximately realize this magnetic configuration are designed. The proposed single-stage approach allows us to find a compromise between confinement and engineering requirements, i.e., find easy-to-build coils with good confinement properties. Using forward and adjoint sensitivities, we derive derivatives of the physical quantities in the objective, which is constrained by a nonlinear periodic differential equation. In two numerical examples, we compare different gradient-based descent algorithms and find that incorporating approximate second-order derivative information through a quasi-Newton method is crucial for convergence. We also explore the optimization landscape in the neighborhood of a minimizer and find many directions in which the objective is mostly flat, indicating ample freedom to find simple and thus easy-to-build coils.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

High-order accurate finite difference discretisations on fully unstructured dual quadrilateral meshes

Here, we present a novel approach for high-order accurate numerical differentiation on unstructured meshes of quadrilateral elements. To differentiate a given function, an auxiliary function with greater smoothness properties is defined which when differentiated provides the derivatives of the original function. The method generalises traditional finite difference methods to meshes of arbitrary topology in any number of dimensions for any order of derivative and accuracy. We demonstrate the accuracy of the numerical scheme using dual quadrilateral meshes and a refinement method based on subdivision surfaces. The scheme is applied to the solution of a range of partial differential equations, including both linear and nonlinear, second and fourth order equations, and a time-dependent first order equation.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

A survey of unsupervised learning methods for high-dimensional uncertainty quantification in black-box-type problems

Constructing surrogate models for uncertainty quantification (UQ) on complex partial differential equations (PDEs) having inherently high-dimensional O(10 n ), n ≥ 2, stochastic inputs (e.g., forcing terms, boundary conditions, initial conditions) poses tremendous challenges. The “curse of dimensionality” can be addressed with suitable unsupervised learning techniques used as a pre-processing tool to encode inputs onto lower-dimensional subspaces while retaining its structural information and meaningful properties. In this work, we review and investigate thirteen dimension reduction methods including linear and nonlinear, spectral, blind source separation, convex and non-convex methods and utilize the resulting embeddings to construct a mapping to quantities of interest via polynomial chaos expansions (PCE). Here, we refer to the general proposed approach as manifold PCE (m-PCE), where manifold corresponds to the latent space resulting from any of the studied dimension reduction methods. To investigate the capabilities and limitations of these methods we conduct numerical tests for three physics-based systems (treated as black-boxes) having high-dimensional stochastic inputs of varying complexity modeled as both Gaussian and non-Gaussian random fields to investigate the effect of the intrinsic dimensionality of input data. We demonstrate both the advantages and limitations of the unsupervised learning methods and we conclude that a suitable m-PCE model provides a cost-effective approach compared to alternative algorithms proposed in the literature, including recently proposed expensive deep neural network-based surrogates and can be readily applied for high-dimensional UQ in stochastic PDEs.

42 ENGINEERING↗
DOE OSTI2022

Towards sharp error analysis of extended Lagrangian molecular dynamics

The extended Lagrangian molecular dynamics (XLMD) method provides a useful framework for reducing the computational cost of a class of molecular dynamics simulations with constrained latent variables. The XLMD method relaxes the constraints by introducing a fictitious mass ε for the latent variables and solving a set of singularly perturbed ordinary differential equations. While favorable numerical performance of XLMD has been demonstrated in several different contexts in the past decade, mathematical analysis of the method remains scarce. Here, we propose the first error analysis of the XLMD method in the context of a classical polarizable force field model. While the dynamics with respect to the atomic degrees of freedom are general and nonlinear, the key mathematical simplification of the polarizable force field model is that the constraints on the latent variables are given by a linear system of equations. We prove that when the initial value of the latent variables is compatible in a sense that we define, XLMD converges as the fictitious mass ε is made small with $\mathscr{O}$(ε) error for the atomic degrees of freedom and with $\mathscr{O}$($\sqrt{ε}$) error for the latent variables, when the dimension of the latent variable d' is 1. Furthermore, when the initial value of the latent variables is improved to be optimally compatible in a certain sense, we prove that the convergence rate can be improved to $\mathscr{O}$(ε) for the latent variables as well. Numerical results verify that both estimates are sharp not only for d'=1, but also for arbitrary d'. In the setting of general d', we do obtain convergence, but with the non-sharp rate of $\mathscr{O}$($\sqrt{ε}$) for both the atomic and latent variables.

74 ATOMIC AND MOLECULAR PHYSICS↗
DOE OSTI2023

Optimal design of chemoepitaxial guideposts for the directed self-assembly of block copolymer systems using an inexact Newton algorithm

Directed self-assembly (DSA) of block copolymers (BCPs) is one of the most promising developments in the cost-effective production of nanoscale devices. The process makes use of the natural tendency for BCP melts to form nanoscale structures upon phase separation. The phase separation can be directed through the use of chemically patterned substrates to promote the formation of morphologies that are essential to the production of semiconductor devices. Moreover, the design of substrate pattern can be formulated as an optimization problem for which we seek optimal substrate designs that effectively produce given target morphologies. In this paper, we adopt a phase field model given by a nonlocal Cahn–Hilliard partial differential equation (PDE) based on the minimization of the Ohta–Kawasaki free energy, and present an efficient PDE-constrained optimization framework for the optimal design problem. The design variables are the locations of circular- or strip-shaped guiding posts that are used to model the substrate chemical pattern. To solve the ensuing optimization problem, we propose a variant of an inexact Newton conjugate gradient algorithm tailored to this problem. Additionally, we demonstrate the effectiveness of our computational strategy on numerical examples that span a range of target morphologies. Owing to our second-order optimizer and fast state solver, the numerical results demonstrate five orders of magnitude reduction in computational cost over previous work. The efficiency of our framework and the fast convergence of our optimization algorithm enable us to rapidly solve the optimal design problem in not only two, but also three spatial dimensions.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

QuadConv: Quadrature-based convolutions with applications to non-uniform PDE data compression

We present a new convolution layer for deep learning architectures which we call QuadConv — an approximation to continuous convolution via quadrature. Our operator is developed explicitly for use on non-uniform, mesh-based data, and accomplishes this by learning a continuous kernel that can be sampled at arbitrary locations. Moreover, the construction of our operator admits an efficient implementation which we detail and construct. As an experimental validation of our operator, we consider the task of compressing partial differential equation (PDE) simulation data from fixed meshes. Here, we show that QuadConv can match the performance of standard discrete convolutions on uniform grid data by comparing a QuadConv autoencoder (QCAE) to a standard convolutional autoencoder (CAE). Further, we show that the QCAE can maintain this accuracy even on non-uniform data. In both cases, QuadConv also outperforms alternative unstructured convolution methods such as graph convolution.

Compression↗
DOE OSTI2024

Dyn$\mathrm{AMO}$: Multi-agent reinforcement learning for dynamic anticipatory mesh optimization with applications to hyperbolic conservation laws

Here we introduce DynAMO, a reinforcement learning paradigm for Dynamic Anticipatory Mesh Optimization. Adaptive mesh refinement is an effective tool for optimizing computational cost and solution accuracy in numerical methods for partial differential equations. However, traditional adaptive mesh refinement approaches for time-dependent problems typically rely only on instantaneous error indicators to guide adaptivity. As a result, standard strategies often require frequent remeshing to maintain accuracy. In the DynAMO approach, multi-agent reinforcement learning is used to discover new local refinement policies that can anticipate and respond to future solution states by producing meshes that deliver more accurate solutions for longer time intervals. By applying DynAMO to discontinuous Galerkin methods for the linear advection and compressible Euler equations in two dimensions, we demonstrate that this new mesh refinement paradigm can outperform conventional threshold-based strategies while also generalizing to different mesh sizes, remeshing and simulation times, and initial conditions.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

GrainGNN: A dynamic graph neural network for predicting 3D grain microstructure

We propose GrainGNN, a surrogate model for the evolution of polycrystalline grain structure under rapid solidification conditions in metal additive manufacturing. High fidelity simulations of solidification microstructures are typically performed using multicomponent partial differential equations (PDEs) with moving interfaces. The inherent randomness of the PDE initial conditions (grain seeds) necessitates ensemble simulations to predict microstructure statistics, e.g., grain size, aspect ratio, and crystallographic orientation. Here, currently such ensemble simulations are prohibitively expensive and surrogates are necessary.In GrainGNN, we use a dynamic graph to represent interface motion and topological changes due to grain coarsening. We use a reduced representation of the microstructure using hand-crafted features; we combine pattern finding and altering graph algorithms with two neural networks, a classifier (for topological changes) and a regressor (for interface motion). Both networks have an encoder-decoder architecture; the encoder has a multi-layer transformer long-short-term-memory architecture; the decoder is a single layer perceptron.We evaluate GrainGNN by comparing it to high-fidelity phase field simulations for in-distribution and out-of-distribution grain configurations for solidification under laser power bed fusion conditions. GrainGNN results in 80%–90% pointwise accuracy; and nearly identical distributions of scalar quantities of interest (QoI) between phase field and GrainGNN simulations compared using Kolmogorov-Smirnov test. GrainGNN's inference speedup (PyTorch on single x86 CPU) over a high-fidelity phase field simulation (CUDA on a single NVIDIA A100 GPU) is 150×–2000× for 100-initial grain problem. Further, using GrainGNN, we model the formation of 11,600 grains in 220 seconds on a single CPU core.

36 MATERIALS SCIENCE↗
DOE OSTI2024

One-sweep moment-based semi-implicit-explicit integration for gray thermal radiation transport

Thermal radiation transport (TRT) is a time dependent, high dimensional partial integro-differential equation. In practical applications such as inertial confinement fusion, TRT is coupled to other physics such as hydrodynamics, plasmas, etc., and the timescales one is interested in capturing are often much slower than the radiation timescale. As a result, TRT is treated implicitly, and due to its stiffness and high dimensionality, is often a dominant computational cost in multiphysics simulations. Here we develop a new approach for implicit-explicit (IMEX) integration of gray TRT in the deterministic SN setting, which requires only one sweep per stage, with the simplest first-order method requiring only one sweep per time step. The partitioning of equations is done via a moment-based high-order low-order formulation of TRT, where the streaming operator and first two moments are used to capture the asymptotic stiff regimes of the streaming limit and diffusion limit. Absorption-reemission is treated explicitly, and although stiff, is sufficiently damped by the implicit solve that we achieve stable accurate time integration without incorporating the coupling of the high order and low order equations implicitly. Due to nonlinear coupling of the high-order and low-order equations through temperature-dependent opacities, to facilitate IMEX partitioning and higher-order methods, we use a semi-implicit integration approach amenable to nonlinear partitions. In conclusion, results are demonstrated on thick Marshak and crooked pipe benchmark problems, demonstrating orders of magnitude improvement in accuracy and wallclock compared with the standard first-order implicit integration typically used.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Multi-head physics-informed neural networks for learning functional priors and uncertainty quantification

In numerous applications, the integration of prior knowledge and historical information is essential, particularly for tasks requiring the solution of ordinary or partial differential equations (ODEs/PDEs) in data-sparse or noisy environments. For instance, achieving accurate solutions to time-dependent PDEs with limited initial condition measurements necessitates an effective strategy for embedding prior knowledge. Hard-parameter sharing architectures in neural networks (NNs) have demonstrated success in both traditional and scientific machine learning domains, facilitating the learning of informative representations. Here, in this study, we introduce a novel, yet efficient, method to enhance physics-informed neural networks (PINNs) by incorporating a multi-head structure that enables the learning of functional priors from both empirical data and governing physical laws. This prior information can then be used to address data sparsity and high-level noise in solving ODE/PDE problems with uncertainty quantification (UQ). The approach, termed Multi-Head PINN (MH-PINN), consists of a shared body NN and multiple head NNs, each corresponding to an individual PINN instance. Our framework for functional prior learning is carried out in two stages: (1) training the MH-PINNs to develop a shared body NN alongside multiple head NNs, and (2) employing these trained head NNs to estimate a prior distribution through a normalizing flow-based density estimator. The learned functional prior can then be applied as a regularization mechanism in deterministic contexts or as an informative prior within a Bayesian inference framework, aiding in the resolution of subsequent ODE/PDE tasks. We evaluate the efficacy of MH-PINNs across five benchmark problems, including a high-dimensional parametric PDE, all characterized by data sparsity or substantial noise levels. Our findings reveal that MH-PINNs deliver accurate solutions and robust UQ, demonstrating adaptability across a range of complex and challenging scenarios.

Bayesian inference↗
DOE OSTI2025

Self-adaptive weights based on balanced residual decay rate for physics-informed neural networks and deep operator networks

Physics-informed deep learning has emerged as a promising alternative for solving partial differential equations. However, for complex problems, training these networks can still be challenging, often resulting in unsatisfactory accuracy and efficiency. In this work, we demonstrate that the failure of plain physics-informed neural networks arises from the significant discrepancy in the convergence rate of residuals at different training points, where the slowest convergence rate dominates the overall solution convergence. Based on these observations, we propose a pointwise adaptive weighting method that balances the residual decay rate across different training points. The performance of our proposed adaptive weighting method is compared with current state-of-the-art adaptive weighting methods on benchmark problems for both physics-informed neural networks and physics-informed deep operator networks. In conclusion, through extensive numerical results we demonstrate that our proposed approach of balanced residual decay rates offers several advantages, including bounded weights, high prediction accuracy, fast convergence rate, low training uncertainty, low computational cost, and ease of hyperparameter tuning.

Balanced convergence rate↗
DOE OSTI2026

ECLEIRS: Exact conservation law embedded identification of reduced states for parameterized nonlinear conservation laws from sparse and noisy data

Multi-query applications such as parameter estimation, uncertainty quantification and design optimization for parameterized partial differential equation (PDE) systems are expensive. While reduced/latent state dynamics approaches for parameterized PDEs offer a viable alternative, these approaches rely on high-quality data and struggle with highly sparse spatiotemporal noisy measurements typically obtained from experiments. Furthermore, there is no guarantee that these models satisfy governing physical conservation laws. In this article, we propose a reduced state dynamics approach, referred to as ECLEIRS, that embeds exact conservation in the solution and flux representation by utilizing a space-time divergence-free neural network formulation. We compare ECLEIRS with other reduced state dynamics approaches, those that do not enforce any physical constraints and those with physics-informed loss functions, for three shock-propagation problems: 1-D advection, 1-D Burgers and 2-D Euler equations. In conclusion, the numerical experiments conducted in this study demonstrate that ECLEIRS provides the most accurate prediction of dynamics for unseen parameters even in the presence of highly sparse and noisy data.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Enabling probabilistic learning on manifolds through double diffusion maps

Here, we present a generative learning framework for probabilistic sampling that extends Probabilistic Learning on Manifolds (PLoM), which is designed to generate statistically consistent realizations of a random vector in a finite-dimensional Euclidean space, informed by a (representative) set of observations. In its original form, PLoM constructs a reduced-order probabilistic model by combining three main components: (a) kernel density estimation to approximate the underlying probability measure, (b) Diffusion Maps to characterize the manifold of the data, and (c) a reduced-order Itô Stochastic Differential Equation (ISDE) to sample from the learned distribution. However, its sampling dynamics are posed in the ambient space and the retained number of reduced coordinates is chosen by projection-reconstruction error. In practice, this often (i) requires more coordinates than the data’s intrinsic dimension to achieve stable sampling and (ii) lacks a smooth, basis-independent lifting back to the data domain; moreover, standard Diffusion Maps emphasize harmonic eigenfunctions and can miss non-harmonic latent structure. We address these limitations by decoupling geometry learning from sampling: a first Diffusion Maps pass identifies non-harmonic coordinates on which we formulate a full-order ISDE directly in the latent space, while Double Diffusion Maps captures multiscale geometric features and Geometric Harmonics (GH) learns a smooth lifting map to the ambient variables that is independent of the particular diffusion basis. This hybrid design preserves the system’s dynamical richness with a compact geometric representation and enables principled out-of-sample inference. The effectiveness and robustness of the proposed method are illustrated through two numerical studies: one based on data generated from two-dimensional Hermite polynomial functions and another based on high-fidelity simulations of a detonation wave in a reactive flow.

Double diffusion maps↗
DOE OSTI2026

Goal-oriented real-time Bayesian inference for linear autonomous dynamical systems with application to digital twins for tsunami early warning

We present a goal-oriented framework for constructing digital twins with the following properties: (1) they employ discretizations of high-fidelity partial differential equation (PDE) models governed by autonomous dynamical systems, leading to large-scale forward problems; (2) they solve a linear inverse problem to assimilate observational data to infer uncertain model components followed by a forward prediction of the evolving dynamics; and (3) the entire end-to-end, data-to-inference-to-prediction computation is carried out without approximation and in real time through a Bayesian framework that rigorously accounts for uncertainties. Several challenges must be overcome to realize this framework, including the large scale of the forward problem, the high dimensionality of the parameter space, and for a class of problems including those we target, the slow decay of the singular values of the parameter-to-observable map. Here we introduce a methodology to overcome these challenges by exploiting the autonomous structure of the forward model to decompose the solution of the inverse problem into a one-time-only offline phase in which the PDE model is solved a limited number of times (equal to the number of sensors), and an online phase that maps well onto GPUs and computes the parameter inference and prediction of quantities of interest in real time, given observational data. Our ultimate goal is to apply this framework to construct digital twins for subduction zones, including Cascadia, to provide early warning for tsunamis generated by megathrust earthquakes. To this end, we demonstrate how our methodology can be used to employ seafloor pressure observations, along with the coupled acoustic–gravity wave equations, to infer the earthquake-induced spatiotemporal seafloor motion (discretized with $\mathscr{O}$ (10 9 ) parameters) and forward predict the tsunami propagation. We present results of an end-to-end inference, prediction, and uncertainty quantification for a representative test problem with $\mathscr{O}$ (10 8 ) inversion parameters for which goal-oriented Bayesian inference is accomplished exactly and in real time, that is, in a matter of seconds.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

What do physics-informed DeepONets learn? Understanding and improving training for scientific computing applications

Physics-informed deep operator networks (DeepONets) have emerged as a promising approach toward numerically approximating the solution of partial differential equations (PDEs). In this work, we aim to develop further understanding of what is being learned by physics-informed DeepONets by assessing the universality of the extracted basis functions and demonstrating their potential toward model reduction with spectral methods. Results provide clarity about measuring the performance of a physics-informed DeepONet through the decays of singular values and expansion coefficients. In addition, we propose a transfer learning approach for improving training for physics-informed DeepONets between parameters of the same PDE as well as across different, but related, PDEs where these models struggle to train well. This approach results in significant error reduction and learned basis functions that are more effective in representing the solution of a PDE.

Deep operator networks↗
DOE OSTI2021

Liouville operators over the Hardy space

The role of Liouville operators in the study of dynamical systems through the use of occupation measures has been an active area of research in control theory over the past decade. This article investigates Liouville operators over the Hardy space, which encode complex ordinary differential equations in an operator over a reproducing kernel Hilbert space.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

A general solution for accelerating screw dislocations in arbitrary slip systems with reflection symmetry

Solutions to the differential equations of linear elasticity in the continuum limit in arbitrary crystal symmetry are known only for steady-state dislocations of arbitrary character, i.e. line defects moving at constant velocity. Troubled by singularities at certain ‘critical’ velocities (typically close to certain sound speeds), these dislocation fields are thought to be too idealized, and divergences are usually attributed to neglecting the finite size of the core and to the restriction to constant velocity. In the isotropic limit, accelerating pure screw and edge dislocations were studied some time ago. A generalization to anisotropic crystals has been attempted for pure screw and edge dislocations only for some special cases. This work aims to fill the gap of deriving a general anisotropic solution for pure screw dislocations applicable to slip systems featuring a reflection symmetry, a prerequisite to studying pure screw dislocations without mixing with edge dislocations. Finally, further generalizations to arbitrary mixed dislocations as well as regularizations of the dislocation core are beyond the scope of this paper and are left for future work.

42 ENGINEERING↗