Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Equation learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Correlative image learning of chemo-mechanics in phase-transforming solids

We report constitutive laws underlie most physical processes in nature. However, learning such equations in heterogeneous solids (for example, due to phase separation) is challenging. One such relationship is between composition and eigenstrain, which governs the chemo-mechanical expansion in solids. Here we developed a generalizable, physically constrained image-learning framework to algorithmically learn the chemo-mechanical constitutive law at the nanoscale from correlative four-dimensional scanning transmission electron microscopy and X-ray spectro-ptychography images. We demonstrated this approach on Li X FePO 4 , a technologically relevant battery positive electrode material. We uncovered the functional form of the composition–eigenstrain relation in this two-phase binary solid across the entire composition range (0 ≤ X ≤ 1), including inside the thermodynamically unstable miscibility gap. The learned relation directly validates Vegard’s law of linear response at the nanoscale. Our physics-constrained data-driven approach directly visualizes the residual strain field (by removing the compositional and coherency strain), which is otherwise impossible to quantify. Heterogeneities in the residual strain arise from misfit dislocations and were independently verified by X-ray diffraction line profile analysis. Our work provides the means to simultaneously quantify chemical expansion, coherency strain and dislocations in battery electrodes, which has implications on rate capabilities and lifetime. Broadly, this work also highlights the potential of integrating correlative microscopy and image learning for extracting material properties and physics.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Symbolic pregression: Discovering physical laws from distorted video

In this work, we present a method for unsupervised learning of equations of motion for objects in raw and optionally distorted unlabeled synthetic video (or, more generally, for discovering and modeling predictable features in time-series data). We first train an autoencoder that maps each video frame into a low-dimensional latent space where the laws of motion are as simple as possible, by minimizing a combination of nonlinearity, acceleration, and prediction error. Differential equations describing the motion are then discovered using Pareto-optimal symbolic regression. We find that our pre-regression (“pregression”) step is able to rediscover Cartesian coordinates of unlabeled moving objects even when the video is distorted by a generalized lens. Using intuition from multidimensional knot theory, we find that the pregression step is facilitated by first adding extra latent space dimensions to avoid topological problems during training and then removing these extra dimensions via principal component analysis. An inertial frame is autodiscovered by minimizing the combined equation complexity for multiple experiments.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2021

Uncovering turbulent plasma dynamics via deep learning from partial observations

One of the most intensely studied aspects of magnetic confinement fusion is edge plasma turbulence which is critical to reactor performance and operation. Drift-reduced Braginskii two-fluid theory has for decades been widely applied to model boundary plasmas with varying success. Towards better understanding edge turbulence in both theory and experiment, we demonstrate that a novel multi-network physics-informed deep learning framework constrained by partial differential equations can accurately learn turbulent fields consistent with the two-fluid theory from partial observations of electron pressure which is not otherwise possible using conventional equilibrium models. Furthermore, this technique presents a novel paradigm for the advanced design of plasma diagnostics and validation of magnetized plasma turbulence theories in challenging thermonuclear environments.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2022

Model-Form Epistemic Uncertainty Quantification for Modeling with Differential Equations: Application to Epidemiology

Modeling real-world phenomena to any degree of accuracy is a challenge that the scientific research community has navigated since its foundation. Lack of information and limited computational and observational resources necessitate modeling assumptions which, when invalid, lead to model-form error (MFE). The work reported herein explored a novel method to represent model-form uncertainty (MFU) that combines Bayesian statistics with the emerging field of universal differential equations (UDEs). The fundamental principle behind UDEs is simple: use known equational forms that govern a dynamical system when you have them; then incorporate data-driven approaches – in this case neural networks (NNs) – embedded within the governing equations to learn the interacting terms that were underrepresented. Utilizing epidemiology as our motivating exemplar, this report will highlight the challenges of modeling novel infectious diseases while introducing ways to incorporate NN approximations to MFE. Prior to embarking on a Bayesian calibration, we first explored methods to augment the standard (non-Bayesian) UDE training procedure to account for uncertainty and increase robustness of training. In addition, it is often the case that uncertainty in observations is significant; this may be due to randomness or lack of precision in the measurement process. This uncertainty typically manifests as “noisy” observations which deviate from a true underlying signal. To account for such variability, the NN approximation to MFE is endowed with a probabilistic representation and is updated using available observational data in a Bayesian framework. By representing the MFU explicitly and deploying an embedded, data-driven model, this approach enables an agile, expressive, and interpretable method for representing MFU. In this report we will provide evidence that Bayesian UDEs show promise as a novel framework for any science-based, data-driven MFU representation; while emphasizing that significant advances must be made in the calibration of Bayesian NNs to ensure a robust calibration procedure.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

What Can We Learn about the Unstable Equation-of-state Branch from Neutron Star Mergers?

Abstract The equation of state (EOS) of dense strongly interacting matter can be probed by astrophysical observations of neutron stars (NS), such as X-ray detections of pulsars or the measurement of the tidal deformability of NSs during the inspiral stage of NS mergers. These observations constrain the EOS at most up to the density of the maximum-mass configuration, n TOV , which is the highest density that can be explored by stable NSs for a given EOS. However, under the right circumstances, binary neutron star (BNS) mergers can create a postmerger remnant that explores densities above n TOV . In this work, we explore whether the EOS above n TOV can be measured from gravitational-wave or electromagnetic observations of the postmerger remnant. We perform a total of 25 numerical-relativity simulations of BNS mergers for a range of EOSs and find no case in which different descriptions of the matter above n TOV have a detectable impact on postmerger observables. Hence, we conclude that the EOS above n TOV can likely not be probed through BNS merger observations for the current and next generation of detectors.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2021

Uncovering turbulent plasma dynamics via deep learning from partial observations

One of the most intensely studied aspects of magnetic confinement fusion is edge plasma turbulence which is critical to reactor performance and operation. Drift-reduced Braginskii two-fluid theory has for decades been widely applied to model boundary plasmas with varying success. Towards better understanding edge turbulence in both theory and experiment, we demonstrate that a novel multi-network physics-informed deep learning framework constrained by partial differential equations can accurately learn turbulent fields consistent with the two-fluid theory from partial observations of electron pressure which is not otherwise possible using conventional equilibrium models. This technique presents a novel paradigm for the advanced design of plasma diagnostics and validation of magnetized plasma turbulence theories in challenging thermonuclear environments.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2021

Probabilistic learning on manifolds constrained by nonlinear partial differential equations for small datasets

A novel extension of the Probabilistic Learning on Manifolds (PLoM) is presented. It makes it possible to synthesize solutions to a wide range of nonlinear stochastic boundary value problems described by partial differential equations (PDEs) for which a stochastic computational model (SCM) is available and which depend on a vector-valued random control parameter. The cost of a single numerical evaluation of this SCM is assumed to be such that only a limited number of points can be computed for constructing the training dataset (small data). Each point of the training dataset is made up of realizations from a vector-valued stochastic process (the stochastic solution) and the associated random control parameter on which it depends. The presented PLoM constrained by PDE allows for generating a large number of learned realizations of the stochastic process and its corresponding random control parameter. These learned realizations are generated so as to minimize the vector-valued random residual of the PDE in the mean-square sense. Appropriate novel methods are developed to solve this challenging problem. Three applications are presented. The first one is a simple uncertain nonlinear dynamical system with a nonstationary stochastic excitation. The second one concerns the 2D nonlinear unsteady Navier–Stokes equations for incompressible flows in which the Reynolds number is the random control parameter. Here, the last one deals with the nonlinear dynamics of a 3D elastic structure with uncertainties. The results obtained make it possible to validate the PLoM constrained by stochastic PDE but also provide further validation of the PLoM without constraint.

Machine learning↗
DOE OSTI2024

Machine-learning based model reduction for partial differential equations

We develop a novel synergistic approach between model reduction and machine learning. The specific goal of this project is to aid in the construction of reduced order models for basis functions that are custom-made to represent the solution of partial differential equations. Partial differential equations (PDEs) are one of the main mathematical tools for describing physical phenomena. However, due to either efficiency or necessity, for many real-world problems, we are interested in constructing reduced order models (ROMs) which focus only on the explicit computation of subsets of the active spatio-temporal scales in the problem, while treating the interaction with the rest of the scales approximately. The task of accurate representation of such interactions (usually called memory terms) constitutes a vast area of research known as model reduction. PI Stinis has significant expertise in the construction of ROMs for complex systems. In addition, in recent work with the project key participant Qadeer, they have utilized machine learning to acquire custom-made basis functions (CBFs) to expand the solutions of PDEs. In the proposed work, we will merge the two concepts by constructing ROMs for subsets of the CBFs needed to represent the solution of a PDE. Specifically, we will use the Mori-Zwanzig model reduction formalism to construct ROMs for subsets of CBFs for nonlinear PDEs of various complexity, as well as investigate the usage of CBFs in the spectral vanishing viscosity method for problems that can form shocks in finite time. The outcome of the research is aimed to be proof-of-concept about a novel synergistic approach between model reduction and machine learning, thus advancing the field of scientific machine learning. Such a capability will benefit the efficient modeling of physical systems appearing in various areas of interest to the DOE.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Discrete generative diffusion models without stochastic differential equations: A tensor network approach

Diffusion models (DMs) are a class of generative machine learning methods that sample a target distribution by transforming samples of a trivial (often Gaussian) distribution using a learned stochastic differential equation. In standard DMs, this is done by learning a “score function” that reverses the effect of adding diffusive noise to the distribution of interest. Here we consider the generalisation of DMs to lattice systems with discrete degrees of freedom, and where noise is added via Markov chain jump dynamics. We show how to use tensor networks (TNs) to efficiently define and sample such “discrete diffusion models” (DDMs) without explicitly having to solve a stochastic differential equation. We show the following: (i) by parametrising the data and evolution operators as TNs, the denoising dynamics can be represented exactly; (ii) the auto-regressive nature of TNs allows to generate samples efficiently and without bias; (iii) for sampling Boltzmann-like distributions, TNs allow to construct an efficient learning scheme that integrates well with Monte Carlo. We illustrate this approach to study the equilibrium of two models with non-trivial thermodynamics, the d = 1 constrained Fredkin chain and the d = 2 Ising model. Published by the American Physical Society 2025

Causer, Luke (ORCID:0000000194243473)↗
DOE OSTI2021

A novel implicit hybrid machine learning model and its application for reinforcement learning

A novel methodology to develop implicit hybrid models is presented. PyTorch is used to integrate physics-based equations with machine learning models. Automatic differentiation of the hybrid model is leveraged to solve the implicit equations. Iterative solving enables gradient based updates to the machine learning model. The novel methodology is compared to an explicit hybrid approach on a continuously stirred tank reactor (CSTR). The novel method results in a lower modelling error. Both hybrid models effectively train with noisy data. To test the implicit hybrid model, it is employed as a reinforcement learning (RL) training model. The RL algorithm trained on the hybrid model outperforms real time optimization of the CSTR and performs nearly as well as RL trained directly on the CSTR and a traditional gradient based approach. Training RL directly on the CSTR requires over 60,000 system interactions compared to 6000 historical data points for hybrid model development.

42 ENGINEERING↗
DOE OSTI2024

Ab initio generalized Langevin equation

We introduce a machine learning–based approach called ab initio generalized Langevin equation (AIGLE) to model the dynamics of slow collective variables (CVs) in materials and molecules. In this scheme, the parameters are learned from atomistic simulations based on ab initio quantum mechanical models. Force field, memory kernel, and noise generator are constructed in the context of the Mori–Zwanzig formalism, under the constraint of the fluctuation–dissipation theorem. Combined with deep potential molecular dynamics and electronic density functional theory, this approach opens the way to multiscale modeling in a variety of situations. Here, we demonstrate this capability with a study of two mesoscale processes in crystalline lead titanate, namely the field-driven dynamics of a planar ferroelectric domain wall, and the dynamics of an extensive lattice of coarse-grained electric dipoles. In the first case, AIGLE extends the reach of ab initio simulations to a regime of noise-driven motions not accessible to molecular dynamics. In the second case, AIGLE deals with an extensive set of CVs by adopting a local approximation for the memory kernel and retaining only short-range noise correlations. The scheme is computationally more efficient than molecular dynamics by several orders of magnitude and mimics the microscopic dynamics at low frequencies where it reproduces accurately the dominant far-infrared absorption frequency.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Mapping out the thermodynamic stability of a QCD equation of state with a critical point using active learning

The Beam Energy Scan Theory (BEST) collaboration's equation of state (EoS) incorporates a three-dimensional Ising model critical point into the quantum chromodynamics (QCD) equation of state from lattice simulations. Furthermore, it contains four free parameters related to the size and location of the critical region in the QCD phase diagram. Certain combinations of the free parameters lead to acausal or unstable realizations of the EoS that should not be considered. In this work, we use an active learning framework to rule out pathological EoS efficiently. We find that checking stability and causality for a small portion of the parameters' range is sufficient to construct algorithms that perform with > 96 % accuracy across the entire parameter space. Though in this work we focus on a specific case, our approach can be generalized to any EoS containing a parameter space-class correspondence.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2022

A backward SDE method for uncertainty quantification in deep learning

Here, we develop a backward stochastic differential equation based probabilistic machine learning method, which formulates a class of stochastic neural networks as a stochastic optimal control problem. An efficient stochastic gradient descent algorithm is introduced with the gradient computed through a backward stochastic differential equation. Convergence analysis for stochastic gradient descent optimization and numerical experiments for applications of stochastic neural networks are carried out to validate our methodology in both theory and performance.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Machine Learning Moment Closure Models for the Radiative Transfer Equation III: Enforcing Hyperbolicity and Physical Characteristic Speeds

This is the third paper in a series in which we develop machine learning (ML) moment closure models for the radiative transfer equation. In our previous work (Huang et al. in J Comput Phys 453:110941, 2022), we proposed an approach to learn the gradient of the unclosed high order moment, which performs much better than learning the moment itself and the conventional $P_N$ closure. However, while the ML moment closure has better accuracy, it is not able to guarantee hyperbolicity and has issues with long time stability. In our second paper (Huang et al., in: Machine learning moment closure models for the radiative transfer equation II: enforcing global hyperbolicity in gradient based closures, 2021. arXiv:2105.14410), we identified a symmetrizer which leads to conditions that enforce that the gradient based ML closure is symmetrizable hyperbolic and stable over long time. The limitation of this approach is that in practice the highest moment can only be related to four, or fewer, lower moments. In this paper, we propose a new method to enforce the hyperbolicity of the ML closure model. Motivated by the observation that the coefficient matrix of the closure system is a lower Hessenberg matrix, we relate its eigenvalues to the roots of an associated polynomial. Here, we design two new neural network architectures based on this relation. The ML closure model resulting from the first neural network is weakly hyperbolic and guarantees the physical characteristic speeds, i.e., the eigenvalues are bounded by the speed of light. The second model is strictly hyperbolic and does not guarantee the boundedness of the eigenvalues. Several benchmark tests including the Gaussian source problem and the two-material problem show the good accuracy, stability and generalizability of our hyperbolic ML closure model.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Bayesian reduced-order deep learning surrogate model for dynamic systems described by partial differential equations

We propose a reduced-order deep-learning surrogate model for dynamic systems described by time-dependent partial differential equations. This method employs space–time Karhunen–Loève expansions (KLEs) of the state variables and space-dependent KLEs of space-varying parameters to identify the reduced (latent) dimensions. Subsequently, a deep neural network (DNN) is used to map the parameter latent space to the state variable latent space. An approximate Bayesian method is developed for uncertainty quantification (UQ) in the proposed KL-DNN surrogate model. The KL-DNN method is tested for the linear advection–diffusion and nonlinear diffusion equations, and the Bayesian approach for UQ is compared with the deep ensembling (DE) approach, commonly used for quantifying uncertainty in DNN models. It was found that the approximate Bayesian method provides a more informative distribution of the PDE solutions in terms of the coverage of the reference PDE solutions (the percentage of nodes where the reference solution is within the confidence interval predicted by the UQ methods) and log predictive probability. The DE method is found to underestimate uncertainty and introduce bias. For the nonlinear diffusion equation, we compare the KL-DNN method with the Fourier Neural Operator (FNO) method and find that KL-DNN is 10% more accurate and needs less training time than the FNO method.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Learning Only on Boundaries: A Physics-Informed Neural Operator for Solving Parametric Partial Differential Equations in Complex Geometries

Recently, deep learning surrogates and neural operators have shown promise in solving partial differential equations (PDEs). However, they often require a large amount of training data and are limited to bounded domains. In this work, we present a novel physics-informed neural operator method to solve parameterized boundary value problems without labeled data. By reformulating the PDEs into boundary integral equations (BIEs), we can train the operator network solely on the boundary of the domain. This approach reduces the number of required sample points from $O(N^d)$ to $O(N^{d-1}$), where $d$ is the domain’s dimension, leading to a significant acceleration of the training process. Additionally, our method can handle unbounded problems, which are unattainable for existing physics-informed neural networks (PINNs) and neural operators. Finally, our numerical experiments show the effectiveness of parameterized complex geometries and unbounded problems.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Machine-learning-based spectral methods for partial differential equations

Spectral methods are an important part of scientific computing’s arsenal for solving partial differential equations (PDEs). However, their applicability and effectiveness depend crucially on the choice of basis functions used to expand the solution of a PDE. The last decade has seen the emergence of deep learning as a strong contender in providing efficient representations of complex functions. In the current work, we present an approach for combining deep neural networks with spectral methods to solve PDEs. In particular, we use a deep learning technique known as the Deep Operator Network (DeepONet) to identify candidate functions on which to expand the solution of PDEs. We have devised an approach that uses the candidate functions provided by the DeepONet as a starting point to construct a set of functions that have the following properties: (1) they constitute a basis, (2) they are orthonormal, and (3) they are hierarchical, i.e., akin to Fourier series or orthogonal polynomials. We have exploited the favorable properties of our custom-made basis functions to both study their approximation capability and use them to expand the solution of linear and nonlinear time-dependent PDEs. The proposed approach advances the state of the art and versatility of spectral methods and, more generally, promotes the synergy between traditional scientific computing and machine learning.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Nonlocal Kernel Network (NKN): a Stable and Resolution-Independent Deep Neural Network.

Neural operators have recently become popular tools for designing solution maps between function spaces in the form of neural networks. Differently from classical scientific machine learning approaches that learn parameters of a known partial differential equation (PDE) for a single instance of the input parameters at a fixed resolution, neural operators approximate the solution map of a family of PDEs [6, 7]. Despite their success, the uses of neural operators are so far restricted to relatively shallow neural networks and confined to learning hidden governing laws. In this work, we propose a novel nonlocal neural operator, which we refer to as nonlocal kernel network (NKN), that is resolution independent, characterized by deep neural networks, and capable of handling a variety of tasks such as learning governing equations and classifying images. Our NKN stems from the interpretation of the neural network as a discrete nonlocal diffusion reaction equation that, in the limit of infinite layers, is equivalent to a parabolic nonlocal equation, whose stability is analyzed via nonlocal vector calculus. The resemblance with integral forms of neural operators allows NKNs to capture long-range dependencies in the feature space, while the continuous treatment of node-to-node interactions makes NKNs resolution independent. The resemblance with neural ODEs, reinterpreted in a nonlocal sense, and the stable network dynamics between layers allow for generalization of NKN’s optimal parameters from shallow to deep networks. This fact enables the use of shallow-to-deep initialization techniques [8]. Our tests show that NKNs outperform baseline methods in both learning governing equations and image classification tasks and generalize well to different resolutions and depths.

97 MATHEMATICS AND COMPUTING↗