Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “physical learning, networks”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Artificial to Spiking Neural Networks Conversion with Calibration in Scientific Machine Learning

Here, we introduce a method to convert physics-informed neural networks (PINNs), commonly used in scientific machine learning, to spiking neural networks (SNNs), which are expected to have higher energy efficiency compared to traditional artificial neural networks (ANNs). We first extend the calibration technique of SNNs to arbitrary activation functions beyond ReLU, making it more versatile, and we prove a theorem that ensures the effectiveness of the calibration. We successfully convert PINNs to SNNs, enabling computational efficiency for diverse regression tasks in solving multiple differential equations, including the unsteady Navier–Stokes equations. We demonstrate great gains in terms of overall efficiency, including separable PINNs (SPINNs), which accelerate the training process. Overall, this is the first work of this kind and the proposed method achieves relatively good accuracy with low spike rates.

PINN↗
DOE OSTI2022

Multi-variance replica exchange SGMCMC for inverse and forward problems via Bayesian PINN

Physics-informed neural network (PINN) has been successfully applied in solving a variety of nonlinear non-convex forward and inverse problems. However, the training is challenging because of the non-convex loss functions and the multiple optima in the Bayesian inverse problem. In this work, we propose a multi-variance replica exchange stochastic gradient Langevin dynamics method to tackle the challenge of the multiple local optima in the optimization and the challenge of the multiple modal posterior distribution in the inverse problem. Replica exchange methods are capable of escaping from the local traps and accelerating the convergence; two chains with different temperatures are designed where the low temperature chain aims for the local convergence, and the target of the high temperature chain is to travel globally and explore the whole loss function entropy landscape. However, it may not be efficient to solve mathematical inversion problems by using the vanilla replica method directly since the method doubles the computational cost in evaluating the forward solvers (likelihood functions) in the two chains. To address this issue, we propose to make different assumptions on the energy function estimation and this facilities one to use solvers of different fidelities in the likelihood function evaluation. More precisely, one can use a solver with low fidelity in the high temperature chain while using a solver with high fidelity in the low temperature chain. Our proposed method significantly lowers the computational cost in the high temperature chain, meanwhile preserving the accuracy and converging very fast. Here we give an unbiased estimate of the swapping rate and give an estimation of the discretization error of the scheme. To verify our idea, we design and solve four inverse problems which have multiple modes. The proposed method is also employed to train the Bayesian PINN to solve the forward and inverse problems; faster and more accurate convergence has been observed when compared to the stochastic gradient Langevin dynamics (SGLD) method and vanilla replica exchange methods.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Physics-informed machine learning of the correlation functions in bulk fluids

The Ornstein–Zernike (OZ) equation is the fundamental equation for pair correlation function computations in the modern integral equation theory for liquids. In this work, machine learning models, notably physics-informed neural networks and physics-informed neural operator networks, are explored to solve the OZ equation. The physics-informed machine learning models demonstrate great accuracy and high efficiency in solving the forward and inverse OZ problems of various bulk fluids. The results highlight the significant potential of physics-informed machine learning for applications in thermodynamic state theory.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Separable physics-informed DeepONet: Breaking the curse of dimensionality in physics-informed machine learning

The deep operator network (DeepONet) has shown remarkable potential in solving partial differential equations (PDEs) by mapping between infinite-dimensional function spaces using labeled datasets. However, in scenarios lacking labeled data, the physics-informed DeepONet (PI-DeepONet) approach, which utilizes the residual loss of the governing PDE to optimize the network parameters, faces significant computational challenges, particularly due to the curse of dimensionality. This limitation has hindered its application to high-dimensional problems, making even standard 3D spatial with 1D temporal problems computationally prohibitive. Additionally, the computational requirement increases exponentially with the discretization density of the domain. Here, to address these challenges and enhance scalability for high-dimensional PDEs, we introduce the Separable physics-informed DeepONet (Sep-PI-DeepONet). This framework employs a factorization technique, utilizing sub-networks for individual one-dimensional coordinates, thereby reducing the number of forward passes and the size of the Jacobian matrix required for gradient computations. By incorporating forward-mode automatic differentiation (AD), we further optimize computational efficiency, achieving linear scaling of computational cost with discretization density and dimensionality, making our approach highly suitable for high-dimensional PDEs. We demonstrate the effectiveness of Sep-PI-DeepONet through three benchmark PDE models: the viscous Burgers’ equation, Biot’s consolidation theory, and a parameterized heat equation. Our framework maintains accuracy comparable to the conventional PI-DeepONet while reducing training time by two orders of magnitude. Notably, for the heat equation solved as a 4D problem, the conventional PI-DeepONet was computationally infeasible (estimated 289.35 h), while the Sep-PI-DeepONet completed training in just 2.5 h. These results underscore the potential of Sep-PI-DeepONet in efficiently solving complex, high-dimensional PDEs, marking a significant advancement in physics-informed machine learning.

Neural operator↗
DOE OSTI2021

Learning unknown physics of non-Newtonian fluids

Here, we use physics-informed neural networks (PINNs) to learn viscosity models of two non-Newtonian systems (polymer melts and suspensions of particles) using only velocity measurements. For synthetic velocity data generated with the power-law viscosity model, the PINN-inferred viscosity model agrees with the analytical model for shear rates with large absolute values but deviates for shear rates near zero where the analytical model has an unphysical singularity. Once the viscosity model is learned the PINN method can solve the momentum conservation equation using only the boundary conditions.

42 ENGINEERING↗
DOE OSTI2020

Extended Physics-Informed Neural Networks (XPINNs): A Generalized Space-Time Domain Decomposition Based Deep Learning Framework for Nonlinear Partial Differential Equations

Here we propose a generalized space-time domain decomposition approach for the physics-informed neural networks (PINNs) to solve nonlinear partial differential equations (PDEs) on arbitrary complex-geometry domains. The proposed framework, named eXtended PINNs ( X P I N N s ), further pushes the boundaries of both PINNs as well as conservative PINNs (cPINNs), which is a recently proposed domain decomposition approach in the PINN framework tailored to conservation laws. Compared to PINN, the XPINN method has large representation and parallelization capacity due to the inherent property of deployment of multiple neural networks in the smaller subdomains. Unlike cPINN, XPINN can be extended to any type of PDEs. Moreover, the domain can be decomposed in any arbitrary way (in space and time), which is not possible in cPINN. Thus, XPINN offers both space and time parallelization, thereby reducing the training cost more effectively. In each subdomain, a separate neural network is employed with optimally selected hyperparameters, e.g., depth/width of the network, number and location of residual points, activation function, optimization method, etc. A deep network can be employed in a subdomain with complex solution, whereas a shallow neural network can be used in a subdomain with relatively simple and smooth solutions. We demonstrate the versatility of XPINN by solving both forward and inverse PDE problems, ranging from one-dimensional to three-dimensional problems, from time-dependent to time-independent problems, and from continuous to discontinuous problems, which clearly shows that the XPINN method is promising in many practical problems. The proposed XPINN method is the generalization of PINN and cPINN methods, both in terms of applicability as well as domain decomposition approach, which efficiently lends itself to parallelized computation. The XPINN code is available on h t t p s : / / g i t h u b . c o m / A m e y a J a g t a p / X P I N N s .

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Construction of generalized quasilinear diffusion coefficient using neural networks with physical restrictions

The quasilinear diffusion coefficient (D QL ) derived from our machine learning framework shows comparable trends with the ground truth D QL obtained from GENRAY-CQL3D simulations. Additionally, for the strong absorption cases, the radial current drive profiles generated using the D QL from our model exhibit consistent behavior with those obtained from the original simulation. These findings indicate the potential of our surrogate modeling approach with physical restrictions to replicate key wave–plasma interaction characteristics while reducing computational costs. Traditionally, calculating D QL for wave–particle interactions relies on computationally intensive wave simulations coupled with Fokker–Planck solvers. To address this challenge, we developed a machine learning-based surrogate model with physical restrictions derived from cold plasma theory and bounce-averaged damping effects. First, we establish the propagation domain of Lower Hybrid Waves in the (N∥, ρ) space by identifying the accessibility limit and determining the upper and lower bounds of N∥ using the Potential Power Deposition (PPD) method. Subsequently, leveraging a database constructed using Latin hypercube sampling alongside the underlying physical restrictions (e.g. PPD), machine learning methods including U-Net and Recurrent Neural Networks are employed to design a physics-restricted machine learning framework capable of reconstructing D QL .

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2025

Learning and discovering multiple solutions using physics-informed neural networks with random initialization and deep ensemble

In this work we explore the capability of physics-informed neural networks (PINNs) to discover multiple solutions. Many real-world phenomena governed by nonlinear differential equations (DEs), such as fluid flow, exhibit multiple solutions under the same conditions, yet capturing this solution multiplicity remains a significant challenge. A key difficulty lies in providing appropriate initial conditions or guesses, as widely used time-marching schemes and Newton’s method are highly sensitive to these choices when solving complex computational problems. While machine learning models, particularly PINNs, have shown promise in solving DEs, their ability to capture multiple solutions remains underexplored. In this work, we propose a simple and practical approach using PINNs to learn and discover multiple solutions. We first demonstrate that PINNs, when combined with random initialization and deep ensemble method—originally developed for uncertainty quantification—can effectively uncover multiple solutions to nonlinear ordinary and partial DEs. Although training large ensembles of PINNs may appear computationally demanding, this can be done efficiently using vectorization techniques supported by modern deep learning frameworks, allowing many networks to be trained simultaneously. Our approach highlights the critical role of initialization in shaping solution diversity, addressing an often-overlooked aspect of machine learning for scientific computing. Furthermore, we propose utilizing PINN-generated solutions as initial conditions or initial guesses for conventional numerical solvers to enhance accuracy and efficiency in capturing multiple solutions. Extensive numerical experiments, including the Allen–Cahn equation and cavity flow, where our approach successfully identifies both stable and unstable solutions, validate the effectiveness of our method. These findings establish a general and efficient framework for addressing solution multiplicity in nonlinear DEs.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Multi-Area Distribution System State Estimation Using Decentralized Physics-Aware Neural Networks

The development of active distribution grids requires more accurate and lower computational cost state estimation. In this paper, the authors investigate a decentralized learning-based distribution system state estimation (DSSE) approach for large distribution grids. The proposed approach decomposes the feeder-level DSSE into subarea-level estimation problems that can be solved independently. The proposed method is decentralized pruned physics-aware neural network (D-P2N2). The physical grid topology is used to parsimoniously design the connections between different hidden layers of the D-P2N2. Monte Carlo simulations based on one-year of load consumption data collected from smart meters for a three-phase distribution system power flow are developed to generate the measurement and voltage state data. The IEEE 123-node system is selected as the test network to benchmark the proposed algorithm against the classic weighted least squares and state-of-the-art learning-based DSSE approaches. Numerical results show that the D-P2N2 outperforms the state-of-the-art methods in terms of estimation accuracy and computational efficiency.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2021

Encoder–decoder neural network for solving the nonlinear Fokker–Planck–Landau collision operator in XGC

An encoder–decoder neural network has been used to examine the possibility for acceleration of a partial integro-differential equation, the Fokker–Planck–Landau collision operator. This is part of the governing equation in the massively parallel particle-in-cell code XGC, which is used to study turbulence in fusion energy devices. The neural network emphasizes physics-inspired learning, where it is taught to respect physical conservation constraints of the collision operator by including them in the training loss, along with the ℓ 2 loss. In particular, network architectures used for the computer vision task of semantic segmentation have been used for training. A penalization method is used to enforce the ‘soft’ constraints of the system and integrate error in the conservation properties into the loss function. During training, quantities representing the particle density, momentum and energy for all species of the system are calculated at each configuration vertex, mirroring the procedure in XGC. This simple training has produced a median relative loss, across configuration space, of the order of 10 –4 , which is low enough if the error is of random nature, but not if it is of drift nature in time steps. The run time for the current Picard iterative solver of the operator is O(n 2 ), where n is the number of plasma species. As the XGC1 code begins to attack problems including a larger number of species, the collision operator will become expensive computationally, making the neural network solver even more important, especially since its training only scales as O(n). Here, a wide enough range of collisionality has been considered in the training data to ensure the full domain of collision physics is captured. An advanced technique to decrease the losses further will be subject of a subsequent report. Eventual work will include expansion of the network to include multiple plasma species.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2021

Tractable Learning in Underexcited Power Grids

Estimating the structure of physical flow networks, such as power grids, is critical to secure delivery of energy. This article discusses statistical structure estimation in power grids in the “underexcited” regime, where a subset of internal nodes has zero injection fluctuations. Prior estimation algorithms based on nodal voltages fail for such grids as the voltage covariance matrix is not invertible. Here, we propose a novel topology learning algorithm for learning underexcited general networks. Our algorithm uses physics-informed conservation laws to first identify the zero-injection buses and their neighbors, and then estimates the remaining edges in the grid. We prove the asymptotic correctness of our algorithm for grids with nonadjacent internal zero-injection nodes. More important, we theoretically analyze our algorithm’s efficacy under noisy measurements, and determine bounds on maximum noise under which asymptotically correct recovery is guaranteed. Our approach is validated through simulations with voltage samples generated on test distribution grids with real injection data and nonlinear power flow models.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2024

Training toward significance with the decorrelated event classifier transformer neural network

Experimental particle physics uses machine learning for many tasks, where one application is to classify signal and background events. This classification can be used to bin an analysis region to enhance the expected significance for a mass resonance search. In natural language processing, one of the leading neural network architectures is the transformer. In this work, an event classifier transformer is proposed to bin an analysis region, in which the network is trained with special techniques. The techniques developed here can enhance the significance and reduce the correlation between the network’s output and the reconstructed mass. It is found that this trained network can perform better than boosted decision trees and feed-forward networks. Published by the American Physical Society 2024

Astronomy & Astrophysics↗
DOE OSTI2021

Uncovering turbulent plasma dynamics via deep learning from partial observations

One of the most intensely studied aspects of magnetic confinement fusion is edge plasma turbulence which is critical to reactor performance and operation. Drift-reduced Braginskii two-fluid theory has for decades been widely applied to model boundary plasmas with varying success. Towards better understanding edge turbulence in both theory and experiment, we demonstrate that a novel multi-network physics-informed deep learning framework constrained by partial differential equations can accurately learn turbulent fields consistent with the two-fluid theory from partial observations of electron pressure which is not otherwise possible using conventional equilibrium models. Furthermore, this technique presents a novel paradigm for the advanced design of plasma diagnostics and validation of magnetized plasma turbulence theories in challenging thermonuclear environments.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2021

Uncovering turbulent plasma dynamics via deep learning from partial observations

One of the most intensely studied aspects of magnetic confinement fusion is edge plasma turbulence which is critical to reactor performance and operation. Drift-reduced Braginskii two-fluid theory has for decades been widely applied to model boundary plasmas with varying success. Towards better understanding edge turbulence in both theory and experiment, we demonstrate that a novel multi-network physics-informed deep learning framework constrained by partial differential equations can accurately learn turbulent fields consistent with the two-fluid theory from partial observations of electron pressure which is not otherwise possible using conventional equilibrium models. This technique presents a novel paradigm for the advanced design of plasma diagnostics and validation of magnetized plasma turbulence theories in challenging thermonuclear environments.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2021

Spatial Mode Correction of Single Photons Using Machine Learning

Abstract Spatial modes of light constitute valuable resources for a variety of quantum technologies ranging from quantum communication and quantum imaging to remote sensing. Nevertheless, their vulnerabilities to phase distortions, induced by random media, impose significant limitations on the realistic implementation of numerous quantum‐photonic technologies. Unfortunately, this problem is exacerbated at the single‐photon level. Over the last two decades, this challenging problem has been tackled through conventional schemes that utilize optical nonlinearities, quantum correlations, and adaptive optics. In this article, the self‐learning and self‐evolving features of artificial neural networks are exploited to correct the complex spatial profile of distorted Laguerre–Gaussian modes at the single‐photon level. Furthermore, the potential of this technique is used to improve the channel capacity of an optical communication protocol that relies on structured single photons. The results have important implications for real‐time turbulence correction of structured photons and single‐photon images.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2026

RadAI

A physics-informed neural network for learning the steady-state, one-dimensional radiative transfer equation in plane-parallel geometries for stellar atmosphere applications.

Ristić, Marko [LANL]↗