Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Neural equations”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
DOE OSTI2026

A Data-Driven Method for Modeling Creep-Fatigue Stress- Strain Behavior Using Neural ODEs

In this paper, we introduce a data-driven machine learning approach for modeling one-dimensional stress–strain behavior under cyclic loading, utilizing experimental data from the nickel-based Alloy 617. The study employs uniaxial creep–fatigue test data acquired under various loading histories and compares two distinct neural network-based ODE models. The first model, known as the black-box model, comprehensively describes the strain–stress relationship using a Neural ODE equation. To interpret this black-box model, we apply the Sparse Identification of Nonlinear Dynamical Systems (SINDy) technique, transforming the black-box model into an equation-based model using symbolic regression. The second model, the Neural flow rule model, incorporates Hooke’s Law for the linear elastic component, with the nonlinear part characterized by a Neural ODE. Both models are trained with experimental data to accurately reflect the observed stress–strain behavior. We conduct a detailed comparison with the standard Chaboche model, which includes three back stresses. Our results demonstrate that the neural network-based ODE models precisely capture the experimental creep–fatigue mechanical behavior, exceeding the standard Chaboche model’s accuracy. Furthermore, an interpretable model derived from the black-box neural ODE model through symbolic regression achieves accuracy comparable to the Chaboche model, enhancing its interpretability. The results highlight the potential of neural network-based ODE models to depict complex creep–fatigue behavior, eliminating the necessity for experts to define a specific, material-focused model form.

creep-fatigue↗
DOE OSTI2026

Physics vs structure: A systematic benchmark of learning strategies for multi-zone building thermal dynamics

Recent advances in physics-informed and data-driven machine learning promise improved thermal models for advanced building control, yet there is limited quantitative evidence on when added physics structure and architectural complexity are beneficial. Here, this work presents a systematic benchmark of five representative system identification methods for modeling multi-zone building thermal dynamics: linear state-space models, multi-layer perceptrons, neural state-space models, neural ordinary differential equations, and physically-consistent neural networks. The methods are evaluated across multiple data regimes and zone coupling strategies. Using a high-fidelity multi-zone commercial building emulator, we examine short-term and long-term prediction accuracy, computational efficiency, and ease of development. Our results reveal critical trade-offs between prediction performance, model complexity, and physical consistency. We demonstrate that decoupled, nonlinear black-box models consistently outperform coupled physics-constrained architectures in both predictive accuracy and out-of-distribution robustness in majority of the test cases for the building type considered in the study. Our findings quantify the cost of complexity in building thermal modeling and provide concrete, actionable, scenario-based guidelines for selecting model classes for control-oriented applications.

Building thermal modeling↗
DOE OSTI2024

Neural network representations of multiphase Equations of State

Abstract Equations of State model relations between thermodynamic variables and are ubiquitous in scientific modelling, appearing in modern day applications ranging from Astrophysics to Climate Science. The three desired properties of a general Equation of State model are adherence to the Laws of Thermodynamics, incorporation of phase transitions, and multiscale accuracy. Analytic models that adhere to all three are hard to develop and cumbersome to work with, often resulting in sacrificing one of these elements for the sake of efficiency. In this work, two deep-learning methods are proposed that provably satisfy the first and second conditions on a large-enough region of thermodynamic variable space. The first is based on learning the generating function (thermodynamic potential) while the second is based on structure-preserving, symplectic neural networks, respectively allowing modifications near or on phase transition regions. They can be used either “from scratch” to learn a full Equation of State, or in conjunction with a pre-existing consistent model, functioning as a modification that better adheres to experimental data. We formulate the theory and provide several computational examples to justify both approaches, highlighting their advantages and shortcomings.

Science & Technology - Other Topics↗
DOE OSTI2024

Data-driven discovery of dynamics from time-resolved coherent scattering

Coherent X-ray scattering (CXS) techniques are capable of interrogating dynamics of nano- to mesoscale materials systems at time scales spanning several orders of magnitude. However, obtaining accurate theoretical descriptions of complex dynamics is often limited by one or more factors—the ability to visualize dynamics in real space, computational cost of high-fidelity simulations, and effectiveness of approximate or phenomenological models. In this work, we develop a data-driven framework to uncover mechanistic models of dynamics directly from time-resolved CXS measurements without solving the phase reconstruction problem for the entire time series of diffraction patterns. Our approach uses neural differential equations to parameterize unknown real-space dynamics and implements a computational scattering forward model to relate real-space predictions to reciprocal-space observations. This method is shown to recover the dynamics of several computational model systems under various simulated conditions of measurement resolution and noise. Moreover, the trained model enables estimation of long-term dynamics well beyond the maximum observation time, which can be used to inform and refine experimental parameters in practice. Finally, we demonstrate an experimental proof-of-concept by applying our framework to recover the probe trajectory from a ptychographic scan. Our proposed framework bridges the wide existing gap between approximate models and complex data.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Scientific machine learning for closure models in multiscale problems: A review

Here, closure problems are omnipresent when simulating multiscale systems, where some quantities and processes cannot be fully prescribed despite their effects on the simulation's accuracy. Recently, scientific machine learning approaches have been proposed as a way to tackle the closure problem, combining traditional (physics-based) modeling with data-driven (machine-learned) techniques, typically through enriching differential equations with neural networks. This paper reviews the different reduced model forms, distinguished by the degree to which they include known physics, and the different objectives of a priori and a posteriori learning. The importance of adhering to physical laws (such as symmetries and conservation laws) in choosing the reduced model form and choosing the learning method is discussed. The effect of spatial and temporal discretization and recent trends toward discretization-invariant models are reviewed. In addition, we make the connections between closure problems and several other research disciplines: inverse problems, Mori-Zwanzig theory, and multi-fidelity methods. In conclusion, much progress has been made with scientific machine learning approaches for solving closure problems, but many challenges remain. In particular, the generalizability and interpretability of learned models is a major issue that needs to be addressed further.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Sparsified time-dependent Fourier neural operators for fusion simulations

This paper presents a sparsified Fourier neural operator for coupled time-dependent partial differential equations (ST-FNO) as an efficient machine learning surrogate for fluid and particle-based fusion codes such as NIMROD (Non-Ideal Magnetohydrodynamics with Rotation - Open Discussion) and GTC (Gyrokinetic Toroidal Code). ST-FNO leverages the structures in the governing equations and utilizes neural operators to represent Green's function-like numerical operators in the corresponding numerical solvers. Once trained, ST-FNO can rapidly and accurately predict dynamics in fusion devices compared with first-principle numerical algorithms. In general, ST-FNO represents an efficient and accurate machine learning surrogate for numerical simulators for multi-variable nonlinear time-dependent partial differential equations, with the proposed architectures and loss functions. The efficacy of ST-FNO has been demonstrated using quiescent H-mode simulation data from NIMROD and kink-mode simulation data from GTC. The ST-FNO H-mode results show orders of magnitude reduction in memory and central processing unit usage in comparison with the numerical solvers in NIMROD when computing fields over a selected poloidal plane. The ST-FNO kink-mode results achieve a factor of 2 reduction in the number of parameters compared to baseline FNO models without accuracy loss.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2025

Score-Based Physics-Informed Neural Networks for High-Dimensional Fokker–Planck Equations

The Fokker-Planck (FP) equation is a foundational partial differential equation (PDE) in stochastic processes involving Brownian motions. However, the curse of dimensionality (CoD) poses a formidable challenge when dealing with high-dimensional FP equations. Although Monte Carlo simulation and (vanilla) Physics-Informed Neural Networks (PINNs) have shown the potential to tackle CoD, both methods exhibit significant numerical errors in high dimensions when dealing with the probability density function (PDF) associated with Brownian motion. The point-wise PDF values tend to decrease exponentially as dimensionality increases, surpassing the precision of numerical simulations and resulting in substantial errors. In addition, due to its massive sampling, Monte Carlo fails to offer fast sampling. Modeling the logarithm likelihood (LL) via vanilla PINNs transforms the FP equation into a notoriously difficult Hamilton-Jacobi-Bellman (HJB) equation, which is impractical for PINN learning, whose error grows rapidly with dimension. To this end, we propose a novel approach utilizing a score-based solver to fit the score function in stochastic differential equations (SDEs). The score function, defined as the gradient of the LL, plays a fundamental role in inferring LL and PDF and enables fast SDE sampling, offering an effective means to overcome the CoD. Three fitting methods, Score Matching (SM), Sliced Score Matching (SSM), and Score-PINN, are introduced, each contributing unique advantages in computational complexity, accuracy, and generality. The proposed score-based SDE solver operates in two stages: first, employing score matching or Score-PINN to acquire the score function; and second, solving the LL via an ordinary differential equation (ODE) using the obtained score function. Comparative evaluations across these methods showcase varying trade-offs. The proposed methodology is evaluated across diverse SDEs, including anisotropic Ornstein-Uhlenbeck processes, geometric Brownian motion, and Brownian motion with varying eigenspace. We also test various distributions, including Gaussian, Log-normal, Laplace, and Cauchy distributions. The numerical results demonstrate the score-based SDE solver’s stability, speed, and performance across different experimental settings, solidifying its potential as a solution to CoD for high-dimensional FP equations.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Uncertainty quantification of graph convolution neural network models of evolving processes

The application of neural network models to scientific machine learning tasks has proliferated in recent years. In particular, neural networks have proved to be adept at modeling processes with spatial–temporal complexity. Nevertheless, these highly parameterized models have garnered skepticism in their ability to produce outputs with quantified error bounds over the regimes of interest. Hence there is a need to find uncertainty quantification methods that are suitable for neural networks. In this work we present comparisons of the parametric uncertainty quantification of neural networks modeling complex spatial–temporal processes with Hamiltonian Monte Carlo and Stein variational gradient descent and its projected variant. Specifically we apply these methods to graph convolutional neural network models of evolving systems modeled with recurrent neural network and neural ordinary differential equations architectures. We show that Stein variational inference is a viable alternative to Monte Carlo methods with some clear advantages for complex neural network models. For our exemplars, Stein variational interference gave similar pushed forward uncertainty profiles through time compared to Hamiltonian Monte Carlo, albeit with generally more generous variance. As a result, projected Stein variational gradient descent also produced similar uncertainty profiles to the non-projected counterpart, but large reductions in the active weight space were confounded by the stability of the neural network predictions and the convoluted likelihood landscape.

36 MATERIALS SCIENCE↗
DOE OSTI2025

Gradient flow based phase-field modeling using separable neural networks

Allen–Cahn equation is a reaction–diffusion equation and is widely used for modeling phase separation. Machine learning methods for solving the Allen–Cahn equation in its strong form suffer from inaccuracies in collocation techniques, errors in computing higher-order spatial derivatives, and the large system size required by the space–time approach. To overcome these challenges, we propose solving the gradient flow of the Ginzburg–Landau free energy functional, which is equivalent to the Allen–Cahn equation, thereby avoiding the second-order spatial derivatives associated with the Allen–Cahn equation. A minimizing movement scheme is employed to solve the gradient flow problem, eliminating the complexities of a space–time approach. We utilize a separable neural network that efficiently represents the phase field through low-rank tensor decomposition. As we use the minimizing movement scheme to numerically solve the gradient flow problem, we thus, refer to the proposed method as the Separable Deep Minimizing Movement (SDMM) method. The evaluation of the functional in the minimizing movement scheme using the Gauss quadrature technique bypasses the inaccuracies associated with collocation techniques traditionally used to solve partial differential equations. A hyperbolic tangent transformation is introduced on the phase field prior to the evaluation of the functional to ensure that it remains strictly bounded within the values of the two phases. For this transformation, theoretical guarantee for energy stability of the minimizing movement scheme is established. Our results suggest that this transformation helps to improve the accuracy and efficiency significantly. The proposed method resolves the challenges faced by state-of-the-art machine learning techniques, outperforming them in both accuracy and efficiency. It is also the first machine learning method to achieve an order of magnitude speed improvement over the finite element method. In addition to its formulation and computational implementation, several case studies illustrate the applicability of the proposed method.

42 ENGINEERING↗
DOE OSTI2025

Optical neural engine for solving scientific partial differential equations

Abstract Solving partial differential equations (PDEs) is the cornerstone of scientific research and development. Data-driven machine learning (ML) approaches are emerging to accelerate time-consuming and computation-intensive numerical simulations of PDEs. Although optical systems offer high-throughput and energy-efficient ML hardware, their demonstration for solving PDEs is limited. Here, we present an optical neural engine (ONE) architecture combining diffractive optical neural networks for Fourier space processing and optical crossbar structures for real space processing to solve time-dependent and time-independent PDEs in diverse disciplines, including Darcy flow equation, the magnetostatic Poisson’s equation in demagnetization, the Navier-Stokes equation in incompressible fluid, Maxwell’s equations in nanophotonic metasurfaces, and coupled PDEs in a multiphysics system. We numerically and experimentally demonstrate the capability of the ONE architecture, which not only leverages the advantages of high-performance dual-space processing for outperforming traditional PDE solvers and being comparable with state-of-the-art ML models but also can be implemented using optical computing hardware with unique features of low-energy and highly parallel constant-time processing irrespective of model scales and real-time reconfigurability for tackling multiple tasks with the same architecture. The demonstrated architecture offers a versatile and powerful platform for large-scale scientific and engineering computations.

Tang, Yingheng (ORCID:0009000153622546)↗
DOE OSTI2024

Machine-learned closure of URANS for stably stratified turbulence: connecting physical timescales & data hyperparameters of deep time-series models

Stably stratified turbulence (SST), a model that is representative of the turbulence found in the oceans and atmosphere, is strongly affected by fine balances between forces and becomes more anisotropic in time for decaying scenarios. Moreover, there is a limited understanding of the physical phenomena described by some of the terms in the Unsteady Reynolds-Averaged Navier–Stokes (URANS) equations—used to numerically simulate approximate solutions for such turbulent flows. Rather than attempting to model each term in URANS separately, it is attractive to explore the capability of machine learning (ML) to model groups of terms, i.e. to directly model the force balances. We develop deep time-series ML for closure modeling of the URANS equations applied to SST. We consider decaying SST which are homogeneous and stably stratified by a uniform density gradient, enabling dimensionality reduction. We consider two time-series ML models: long short-term memory and neural ordinary differential equation. Both models perform accurately and are numerically stable in a posteriori (online) tests. Furthermore, we explore the data requirements of the time-series ML models by extracting physically relevant timescales of the complex system. We find that the ratio of the timescales of the minimum information required by the ML models to accurately capture the dynamics of the SST corresponds to the Reynolds number of the flow. The current framework provides the backbone to explore the capability of such models to capture the dynamics of high-dimensional complex dynamical system like SST flows.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Computing Nonlinear Power Spectra Across Dynamical Dark Energy Model Space with Neural ODEs

I show how to compute the nonlinear power spectrum across the entire $w(z)$ dynamical dark energy model space. Using synthetic ΛCDM data, I train a neural ordinary differential equation (ODE) to infer the evolution of the nonlinear matter power spectrum as a function of the background expansion and mean matter density across ∼9 Gyr of cosmic evolution. After training, the model generalises to any dynamical dark energy model parameterised by $w(z)$. With little optimisation, the neural ODE is accurate to within 4% up to $k = 5\, h\, {\mathrm Mpc}^{−1}$. Unlike simulation rescaling methods, neural ODEs naturally extend to summary statistics beyond the power spectrum that are sensitive to the growth history.

cosmology↗
DOE OSTI2024

Neural simulation-based inference of the neutron star equation of state directly from telescope spectra

Neutron stars provide a unique opportunity to study strongly interacting matter under extreme density conditions. The intricacies of matter inside neutron stars and their equation of state are not directly visible, but determine bulk properties, such as mass and radius, which affect the star's thermal X-ray emissions. However, the telescope spectra of these emissions are also affected by the stellar distance, hydrogen column, and effective surface temperature, which are not always well-constrained. Uncertainties on these nuisance parameters must be accounted for when making a robust estimation of the equation of state. In this study, we develop a novel methodology that, for the first time, can infer the full posterior distribution of both the equation of state and nuisance parameters directly from telescope observations. This method relies on the use of neural likelihood estimation, in which normalizing flows use samples of simulated telescope data to learn the likelihood of the neutron star spectra as a function of these parameters, coupled with Hamiltonian Monte Carlo methods to efficiently sample from the corresponding posterior distribution. Our approach surpasses the accuracy of previous methods, improves the interpretability of the results by providing access to the full posterior distribution, and naturally scales to a growing number of neutron star observations expected in the coming years.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2024

Forecasting high-dimensional spatio-temporal systems from sparse measurements

This paper introduces a new neural network architecture designed to forecast high-dimensional spatio-temporal data using only sparse measurements. The architecture uses a two-stage end-to-end framework that combines neural ordinary differential equations (NODEs) with vision transformers. Initially, our approach models the underlying dynamics of complex systems within a low-dimensional space; and then it reconstructs the corresponding high-dimensional spatial fields. Many traditional methods involve decoding high-dimensional spatial fields before modeling the dynamics, while some other methods use an encoder to transition from high-dimensional observations to a latent space for dynamic modeling. In contrast, our approach directly uses sparse measurements to model the dynamics, bypassing the need for an encoder. This direct approach simplifies the modeling process, reduces computational complexity, and enhances the efficiency and scalability of the method for large datasets. We demonstrate the effectiveness of our framework through applications to various spatio-temporal systems, including fluid flows and global weather patterns. Although sparse measurements have limitations, our experiments reveal that they are sufficient to forecast system dynamics accurately over long time horizons. Our results also indicate that the performance of our proposed method remains robust across different sensor placement strategies, with further improvements as the number of sensors increases. This robustness underscores the flexibility of our architecture, particularly in real-world scenarios where sensor data is often sparse and unevenly distributed.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Adaptive PID Gain Scheduling Control for Hydropower Turbine Using Neural CDE and Stochastic Distribution Shaping

This paper introduces a gain-scheduling PID controller design strategy for hydroturbine frequency control mode. This scheme first uses real data to learn the nonlinear dynamics of the hydroturbine using neural controlled differential equations and then perturbs the obtained nonlinear system at different equilibrium points, based on which a static output feedback adaptive dynamic programming algorithm is then used to optimize the PID gains for each equilibrium point. Moreover, a continuous-time version of stochastic distribution control is proposed to further fine-tune the optimized PID gains. Finally, the controller is obtained by implementing linear interpolation between the optimized PID control gains. The simulation results show that the proposed gain-scheduling PID controller can control a larger range of operation points compared with the given fixed PID controller and the baseline method. Compared with the given fixed PID controller, the proposed gain-scheduling PID controller can regulate hydroturbine frequency against disturbances induced by power-load variation with over 50% less overshoot for some operation points.

13 HYDRO ENERGY↗
DOE OSTI2026

PDEHats

This is code used to train and evaluate neural partial differential equation solvers on an open source fluid flow data. We evaluate two standard deep learning algorithms for their ability to generalize, a desirable capability for trusthworthy and performant models.

Amarel, James↗