Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Neural equations”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3
All sourcesNASA NTRSDOE OSTI
DOE OSTI2026

Active operator learning with predictive uncertainty quantification for partial differential equations

With the increased prevalence of neural operators being used to provide rapid solutions to partial differential equations (PDEs), understanding the accuracy of model predictions and the associated error levels is necessary for deploying reliable surrogate models in scientific applications. Existing uncertainty quantification (UQ) frameworks employ ensembles or Bayesian methods, which can incur substantial computational costs during both training and inference. Here, we propose a lightweight predictive UQ method tailored for Deep operator networks (DeepONets) that also generalizes to other operator networks. Numerical experiments on linear and nonlinear PDEs demonstrate that the framework’s uncertainty estimates are unbiased and provide accurate out-of-distribution uncertainty predictions with a sufficiently large training dataset. Our framework provides fast inference and uncertainty estimates that can efficiently drive outer-loop analyses that would be prohibitively expensive with conventional solvers. We demonstrate how predictive uncertainties can be used in the context of Bayesian optimization and active learning problems to yield improvements in accuracy and data-efficiency for outer-loop optimization procedures. In the active learning setup, we extend the framework to Fourier Neural Operators (FNO) and describe a generalized method for other operator networks. To enable real-time deployment, we introduce an inference strategy based on precomputed trunk outputs and a sparse placement matrix, reducing evaluation time by more than a factor of five. Our method provides a practical route to uncertainty-aware operator learning in time-sensitive settings.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Equation-Free Coarse Control of Distributed Parameter Systems via Local Neural Operators

The control of high-dimensional distributed parameter systems (DPS) remains a challenge when explicit coarse-grained equations are unavailable. Classical equation-free (EF) approaches rely on fine-scale simulators treated as black-box timesteppers. However, repeated simulations for steady-state computation, linearization, and control design are often computationally prohibitive, or the microscopic timestepper may not even be available, leaving us with data as the only resource. We propose a data-driven alternative that uses local neural operators, trained on spatiotemporal microscopic/mesoscopic data, to obtain efficient short-time solution operators. These surrogates are employed within Krylov subspace methods to compute coarse steady and unsteady-states, while also providing Jacobian information in a matrix-free manner. Krylov-Arnoldi iterations then approximate the dominant eigenspectrum, yielding reduced models that capture the open-loop slow dynamics without explicit Jacobian assembly. Both discrete-time Linear Quadratic Regulator (dLQR) and pole-placement (PP) controllers are based on this reduced system and lifted back to the full nonlinear dynamics, thereby closing the feedback loop.

93B52, 93C20, 47N70, 65J15, 65M32, 68T07, 68T20, 6↗
DOE OSTI2025

Optimizing the optimizer for physics-informed neural networks and Kolmogorov-Arnold networks

Physics-Informed Neural Networks (PINNs) have revolutionized the computation of PDE solutions by integrating partial differential equations (PDEs) into the neural network’s training process as soft constraints, becoming an important component of the scientific machine learning (SciML) ecosystem. More recently, physics-informed Kolmogorv-Arnold networks (PIKANs) have also shown to be effective and comparable in accuracy with PINNs. In their current implementation, both PINNs and PIKANs are mainly optimized using first-order methods like Adam, as well as quasi-Newton methods such as BFGS and its low-memory variant, L-BFGS. However, these optimizers often struggle with highly nonlinear and non-convex loss landscapes, leading to challenges such as slow convergence, local minima entrapment, and (non)degenerate saddle points. In this study, we investigate the performance of Self- Scaled BFGS (SSBFGS), Self-Scaled Broyden (SSBroyden) methods and other advanced quasi-Newton schemes, including BFGS and L-BFGS with different line search strategies. These methods dynamically rescale updates based on historical gradient information, thus enhancing training efficiency and accuracy. We systematically compare these optimizers – using both PINNs and PIKANs – on key challenging PDEs, including the Burgers, Allen-Cahn, Kuramoto-Sivashinsky, Ginzburg-Landau, and Stokes equations. Additionally, we evaluate the performance of SSBFGS and SSBroyden for Deep Operator Network (DeepONet) architectures, demonstrating their effectiveness for data-driven operator learning. Our findings provide state-of-the-art results with orders-of-magnitude accuracy improvements without the use of adaptive weights or any other enhancements typically employed in PINNs. More broadly, our work reveal insights into the effectiveness of quasi-Newton optimization strategies in significantly improving the convergence and accurate generalization of PINNs and PIKANs.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Reducing Frequency Bias of Fourier Neural Operators in 3D Seismic Wavefield Simulations Through Multistage Training

The recent development of neural operator (NeurOp) learning for solutions to the elastic wave equation shows promising results and provides the basis for fast large-scale simulations for different seismological applications. In this article, we use the Fourier neural operator (FNO) model to directly solve the 3D Helmholtz wave equation for fast seismic ground-motion simulations on different frequencies and show the frequency bias of the FNO model, that is, it learns the lower frequencies better comparing to the higher frequencies. To reduce the frequency bias, we adopt the multistage FNO training, that is, after training a stage 1 FNO model for estimating the ground motion, we use a second FNO model as the stage 2 to learn from the residual, which greatly reduced the errors on the higher frequencies. By adopting this multistage training, the FNO models show reduced biases on higher frequencies, which enhanced the overall results of the ground-motion simulations. Thus the multistage training FNO improves the accuracy and realism of the ground-motion simulations.

earthquakes↗
DOE OSTI2026

Neural operator transformers capture bifurcating drift-wave turbulence in fusion plasma simulations

Self-consistent modeling of turbulence-driven transport is critical for optimizing confinement in magnetically confined fusion plasmas, such as tokamaks and stellarators. In particular, capturing the long-term co-evolution of turbulence, flow, and background plasma profiles remains computationally challenging. Direct numerical simulation of these multiscale, highly nonlinear processes is often demanding and impractical for real-time control or design optimization. To address this bottleneck, we investigate transformer-based neural operator partial differential equation surrogates for emulating the dynamics of drift-wave turbulence bifurcation mediated by zonal flows, using the modified Hasegawa–Wakatani (MHW) model as a prototypical system. We find that the finetuned neural operator model has excellent performance in capturing the multi-spatiotemporal-scales of MHW turbulence bifurcation and is robust to testing on rare and out-of-distribution dynamics. Specifically, we demonstrate that a single unified model accurately predicts both quasi-steady-state turbulence and a wide range of dynamical transition processes, such as nonlinear saturation, spontaneous suppression of turbulence, and the emergence of macroscopic zonal flows, over time horizons vastly exceeding the local turbulence correlation time. This computationally efficient approach establishes a strong foundation for fast, AI-based modeling of complex, multiscale phenomena in magnetized fusion plasmas.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2025

Structure-preserving neural networks for the regularized entropy-based closure of a linear, kinetic, radiative transport equation

The main challenge of large-scale numerical simulation of radiation transport is the high memory and computation time requirements of discretization methods for kinetic equations. In this work, we derive and investigate a neural network-based approximation to the entropy-based closure method to accurately compute the solution of the multi-dimensional moment system with a low memory footprint and competitive computational time. We extend methods developed for the standard entropy-based closure to the regularized entropy-based closures. The main idea is to interpret structure-preserving neural network approximations of the regularized entropy-based closure as a two-stage approximation to the original entropy-based closure. We conduct a numerical analysis of this approximation and investigate optimal parameter choices. Our numerical experiments demonstrate that the method has a much lower memory footprint than traditional methods with competitive computation times and simulation accuracy. The code and all trained networks are provided on GitHub.

entropy closure↗
DOE OSTI2024

CryoTRANS: predicting high-resolution maps of rare conformations from self-supervised trajectories in cryo-EM

Cryogenic electron microscopy (cryo-EM) has revolutionized structural biology, enabling efficient determination of structures at near-atomic resolutions. However, a common challenge arises from the severe imbalance among various conformations of vitrified particles, leading to low-resolution reconstructions in rare conformations due to a lack of particle images in these quasi-stable states. We introduce CryoTRANS, a method that predicts high-resolution maps of rare conformations by constructing a self-supervised pseudo-trajectory between density maps of varying resolutions. This trajectory is represented by an ordinary differential equation parameterized by a deep neural network, ensuring retention of detailed structures from high-resolution density maps. By leveraging a single high-resolution density map, CryoTRANS significantly improves the reconstruction of rare conformations and has been validated on four real-world datasets: alpha-2-macroglobulin, actin-binding protein complexes, SARS-CoV-2 spike glycoprotein, and the 70S ribosome. CryoTRANS can also predict high-resolution structures in cryogenic electron tomography maps using a high-resolution cryo-EM map.Cryogenic electron microscopy (cryo-EM) has revolutionized structural biology, enabling efficient determination of structures at near-atomic resolutions. However, a common challenge arises from the severe imbalance among various conformations of vitrified particles, leading to low-resolution reconstructions in rare conformations due to a lack of particle images in these quasi-stable states. We introduce CryoTRANS, a method that predicts high-resolution maps of rare conformations by constructing a self-supervised pseudo-trajectory between density maps of varying resolutions. This trajectory is represented by an ordinary differential equation parameterized by a deep neural network, ensuring retention of detailed structures from high-resolution density maps. By leveraging a single high-resolution density map, CryoTRANS significantly improves the reconstruction of rare conformations and has been validated on four real-world datasets: alpha-2-macroglobulin, actin-binding protein complexes, SARS-CoV-2 spike glycoprotein, and the 70S ribosome. CryoTRANS can also predict high-resolution structures in cryogenic electron tomography maps using a high-resolution cryo-EM map.

47 OTHER INSTRUMENTATION↗
DOE OSTI2025

Three and Two Phase Rotating Field Inductive Couplers for Wireless Power Transfer with One Phase per Layer Windings

Multiphase inductive wireless charging coils have been proposed recently to improve coupler surface power density, reduce component stress and size, and provide near-constant power delivery to charge mobile electric systems. Several aspects for the fundamental characterization of multiphase coils are explored up to six phases including approximate mutual inductance with size and turn variation, induced voltage, and output power estimation. The relative component stress and size of passive components for resonant operation are compared between the multiphase variants. A combination of an experimentally validated 3D electromagnetic finite element analysis (FEA) and power electronic co-simulations are used to validate the estimated quantities approximated with a mixture of analytical equations and an artificial neural network model for mutual inductance. A novel three-phase transmitter, two-phase receiver coil pair is also proposed for electric vehicle charging to reduce the number of connections and compensation complexity on the vehicle-side with improved power output compared to a two-phase configuration.

Lewis, Donovin D. [University of Kentucky]↗
DOE OSTI2026

Direct inference of nuclear equation-of-state parameters from gravitational-wave observations

The observation of neutron star mergers with gravitational waves (GWs) has provided a new method to constrain the dense-matter equation of state (EOS) and to better understand its nuclear physics. However, inferring nuclear microphysics from GW observations necessitates the sampling of EOS model parameters that serve as input for each EOS used during the GW data analysis. The sampling of the EOS parameters requires solving the Tolman–Oppenheimer–Volkoff (TOV) equations a large number of times—a process that slows down each likelihood evaluation in the analysis on the order of a few seconds. Here, we employ emulators for the TOV equations built using multilayer perceptron neural networks to enable direct inference of nuclear EOS parameters from GW strain data. Our emulators allow us to rapidly solve the TOV equations, taking in EOS parameters and outputting the associated tidal deformability of a neutron star in only a few tens of milliseconds. We implement these emulators in PyCBC to directly infer the EOS parameters using the event GW170817, providing posteriors on these parameters informed solely by GWs. We benchmark these runs against analyses performed using the full TOV solver and find that the emulators achieve speed ups of nearly two orders of magnitude, with negligible differences in the recovered posteriors. Additionally, we constrain the slope and curvature of the symmetry energy at the 90% upper credible interval to be $L$ sym ≲ 106 MeV and $K$ sym ≲ 26 MeV.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2025

Application of physics-informed neural networks (PINNs) solution to coupled thermal and hydraulic processes in silty sands

Abstract The accurate modeling of water and heat transport in soils is crucial for both geo-environmental and geothermal engineering. Traditional modeling methods are problematic because they require well-defined boundaries and initial conditions. Recently, physics-informed neural networks (PINNs), which incorporate partial differential equations (PDEs) to solve forward and inverse problems, have attracted increasing attention in machine learning research. In this study, we applied PINNs to tackle hydraulic and thermal transport coupling forward problems in silty sands. A fully connected deep neural network was utilized for training. This neural network model leverages automatic differentiation to apply the governing equations as constraints, based on the mathematical approximations established by the neural network itself. We conducted forward problems and compared the solutions derived from PINNs with those from Finite Element Method (FEM) simulations. The forward problem results demonstrate the PINNs model’s capability in predicting hydraulic transport, heat transport, and thermal–hydraulic coupling in silty sands under various boundary conditions. The PINNs exhibited great performance in simulating the thermal–hydraulic coupling problem. The accuracy of the PINNs solutions shows its potential for simulation in geotechnical engineering.

Feng, Yuan↗
DOE OSTI2024

Quantifying local and global mass balance errors in physics-informed neural networks

Physics-informed neural networks (PINN) have recently become attractive for solving partial differential equations (PDEs) that describe physics laws. By including PDE-based loss functions, physics laws such as mass balance are enforced softly in PINN. This paper investigates how mass balance constraints are satisfied when PINN is used to solve the resulting PDEs. We investigate PINN’s ability to solve the 1D saturated groundwater flow equations (diffusion equations) for homogeneous and heterogeneous media and evaluate the local and global mass balance errors. We compare the obtained PINN’s solution and associated mass balance errors against a two-point finite volume numerical method and the corresponding analytical solution. We also evaluate the accuracy of PINN in solving the 1D saturated groundwater flow equation with and without incorporating hydraulic heads as training data. We demonstrate that PINN’s local and global mass balance errors are significant compared to the finite volume approach. Tuning the PINN’s hyperparameters, such as the number of collocation points, training data, hidden layers, nodes, epochs, and learning rate, did not improve the solution accuracy or the mass balance errors compared to the finite volume solution. Mass balance errors could considerably challenge the utility of PINN in applications where ensuring compliance with physical and mathematical properties is crucial.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

Scaling Field-Theoretic Simulation for Multicomponent Mixtures with Neural Operators

Multicomponent polymer mixtures are ubiquitous in biological self-organization but are notoriously difficult to study computationally. Plagued by both slow single molecule relaxation times and slow equilibration within dense mixtures, molecular dynamics simulations are typically infeasible at the spatial scales required to study the stability of mesophase structure. Polymer field theories offer an attractive alternative, but analytical calculations are only tractable for mean-field theories and nearby perturbations, constraints that become especially problematic for fluctuation-induced effects such as coacervation. Here, we show that a recently developed technique for obtaining numerical solutions to partial differential equations based on operator learning, neural operators, lends itself to a highly scalable training strategy by parallelizing per-species operator maps. We illustrate the efficacy of our approach on six-component mixtures with randomly selected compositions and that it significantly outperforms the state-of-the-art pseudospectral integrators for field-theoretic simulations, especially as polymer lengths become long.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2024

Tackling the curse of dimensionality in fractional and tempered fractional PDEs with physics-informed neural networks

Fractional and tempered fractional partial differential equations (PDEs) are effective models of long-range interactions, anomalous diffusion, and non-local effects. Traditional numerical methods for these problems are mesh-based, thus struggling with the curse of dimensionality (CoD). Physics-informed neural networks (PINNs) offer a promising solution due to their universal approximation, generalization ability, and mesh-free training. In principle, Monte Carlo fractional PINN (MC-fPINN) estimates fractional derivatives using Monte Carlo methods and thus could lift CoD. However, this may cause significant variance and errors, hence affecting convergence; in addition, MC-fPINN is sensitive to hyperparameters. In general, numerical methods and specifically PINNs for tempered fractional PDEs are under-developed. Herein, we extend MC-fPINN to tempered fractional PDEs to address these issues, resulting in the Monte Carlo tempered fractional PINN (MC-tfPINN). To reduce possible high variance and errors from Monte Carlo sampling, we replace the one-dimensional (1D) Monte Carlo with 1D Gaussian quadrature, applicable to both MC-fPINN and MC-tfPINN. We validate our methods on various forward and inverse problems of fractional and tempered fractional PDEs, scaling up to 100,000 dimensions. Our improved MC-fPINN/MC-tfPINN using quadrature consistently outperforms the original versions in accuracy and convergence speed in very high dimensions.

42 ENGINEERING↗
DOE OSTI2026

Constraining Hamiltonians from chiral effective field theory with neutron-star data

Multi-messenger observations of neutron stars (NSs) and their mergers have placed strong constraints on the dense-matter equation of state (EOS). The EOS, in turn, depends on microscopic nuclear interactions that are described by nuclear Hamiltonians. These Hamiltonians are commonly derived within chiral effective field theory (EFT). Ideally, multi-messenger observations of NSs could be used to directly inform our understanding of EFT interactions, but such a direct inference necessitates millions of model evaluations. This is computationally prohibitive because each evaluation requires us to calculate the EOS from a Hamiltonian by solving the quantum many-body problem with methods such as auxiliary-field diffusion Monte Carlo (AFDMC), which provides very accurate and precise solutions but at a significant computational cost. Additionally, we need to solve the stellar structure equations for each EOS which further slows down each model evaluation by a few seconds. In this work, we combine emulators for AFDMC calculations of neutron matter, built using parametric matrix models, and for the stellar structure equations, built using multilayer perceptron neural networks, with the PyCBC data-analysis framework to enable a direct inference of coupling constants in an EFT Hamiltonian using multi-messenger observations of NSs. We find that astrophysical data can provide informative constraints on two-nucleon couplings despite the high densities probed in NS interiors.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2024

Physics-constrained machine learning for electrodynamics without gauge ambiguity based on Fourier transformed Maxwell’s equations

We utilize a Fourier transformation-based representation of Maxwell’s equations to develop physics-constrained neural networks for electrodynamics without gauge ambiguity, which we label the Fourier–Helmholtz–Maxwell neural operator method. In this approach, both of Gauss’s laws and Faraday’s law are built in as hard constraints, as well as the longitudinal component of Ampère–Maxwell in Fourier space, assuming the continuity equation. An encoder–decoder network acts as a solution operator for the transverse components of the Fourier transformed vector potential, $\hat{A}_⟂(k,t)$, whose two degrees of freedom are used to predict the electromagnetic fields. This method was tested on two electron beam simulations. Among the models investigated, it was found that a U-Net architecture exhibited the best performance as it trained quicker, was more accurate and generalized better than the other architectures examined. We demonstrate that our approach is useful for solving Maxwell’s equations for the electromagnetic fields generated by intense relativistic charged particle beams and that it generalizes well to unseen test data, while being orders of magnitude quicker than conventional simulations. We show that the model can be re-trained to make highly accurate predictions in as few as 20 epochs on a previously unseen data set.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

CLPNets: Coupled Lie–Poisson neural networks for multi-part Hamiltonian systems with symmetries

To accurately compute data-based prediction of Hamiltonian systems, it is essential to utilize methods that preserve the structure of the equations over time. We consider a particularly challenging case of systems with interacting parts that do not reduce to pure momentum evolution. Such systems are essential in scientific computations, such as discretization of a continuum elastic rod, which can be viewed as the group of rotations and translations $SE(3)$. The evolution involves not only the momenta but also the relative positions and orientations of the particles. The presence of Lie group-valued elements, such as relative positions and orientations, poses a problem for applying previously derived methods for data-based computing. We develop a novel method of data-based computation and complete phase space learning of such systems. We follow the original framework of SympNets (Jin et al., 2020) and LPNets (Eldred et al., 2024), building the neural network from phase space mappings that preserve the Lie–Poisson structure. We derive a novel system of mappings that are built into neural networks describing the evolution of such systems. We call such networks Coupled Lie–Poisson Neural Networks, or CLPNets. We consider increasingly complex examples for the applications of CLPNets, starting with the rotation of two rigid bodies about a common axis, progressing to the free rotation of two rigid bodies, and finally to the evolution of two connected and interacting $SE(3)$ components, describing the discretization of an elastic rod into two elements. Our method preserves all Casimir invariants to machine precision, preserves energy to high accuracy, and shows good resistance to the curse of dimensionality, requiring only a few thousand data points for all cases studied (three to eighteen dimensions). Additionally, the method is highly economical in memory requirements, requiring only about 200 parameters for the most complex case considered.

Data-based modeling↗
DOE OSTI2024

Learning Constitutive Relations From Soil Moisture Data via Physically Constrained Neural Networks

Abstract The constitutive relations of the Richardson‐Richards equation encode the macroscopic properties of soil water retention and conductivity. These soil hydraulic functions are commonly represented by models with a handful of parameters. The limited degrees of freedom of such soil hydraulic models constrain our ability to extract soil hydraulic properties from soil moisture data via inverse modeling. We present a new free‐form approach to learning the constitutive relations using physically constrained neural networks. We implemented the inverse modeling framework in a differentiable modeling framework, JAX, to ensure scalability and extensibility. For efficient gradient computations, we implemented implicit differentiation through a nonlinear solver for the Richardson‐Richards equation. We tested the framework against synthetic noisy data and demonstrated its robustness against varying magnitudes of noise and degrees of freedom of the neural networks. We applied the framework to soil moisture data from an upward infiltration experiment and demonstrated that the neural network‐based approach was better fitted to the experimental data than a parametric model and that the framework can learn the constitutive relations.

54 ENVIRONMENTAL SCIENCES↗