Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Neural equations”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Neural Active Manifolds: Nonlinear Dimensionality Reduction for Uncertainty Quantification

We present a new approach for nonlinear dimensionality reduction, specifically designed for computationally expensive mathematical models. We leverage autoencoders to discover a one-dimensional neural active manifold (NeurAM) capturing the model output variability, through the aid of a simultaneously learnt surrogate model with inputs on this manifold. Our method only relies on model evaluations and does not require the knowledge of gradients. The proposed dimensionality reduction framework can then be applied to assist outer loop many-query tasks in scientific computing, like sensitivity analysis and multifidelity uncertainty propagation. In particular, we prove, both theoretically under idealized conditions, and numerically in challenging test cases, how NeurAM can be used to obtain multifidelity sampling estimators with reduced variance by sampling the models on the discovered low-dimensional and shared manifold among models. Several numerical examples illustrate the main features of the proposed dimensionality reduction strategy and highlight its advantages with respect to existing approaches in the literature.

Autoencoders↗
DOE OSTI2025

Real-time inference and extrapolation with Time-Conditioned UNet: Applications in hypersonic flows, incompressible flows, and global temperature forecasting

Neural Operators are fast and accurate surrogates for nonlinear mappings between functional spaces within training domains. Extrapolation beyond the training domain remains a grand challenge across all application areas. We present Time-Conditioned UNet (TC-UNet) as an operator learning method to solve time-dependent PDEs continuously in time without any temporal discretization, including in extrapolation scenarios. TC-UNet incorporates the temporal evolution of the PDE into its architecture by combining a parameter conditioning approach with the attention mechanism from the Transformer architecture. After training, TC-UNet makes real-time inferences on an arbitrary temporal grid. We demonstrate its extrapolation capability on a climate problem by estimating the global temperature for several years and also for inviscid hypersonic flow around a double cone. We propose different training strategies involving temporal bundling and sub-sampling. We demonstrate performance improvements for several benchmarks, performing extrapolation for long time intervals and zero-shot super-resolution time.

Deep learning↗
DOE OSTI2025

Data-driven analysis of dipole strength functions using artificial neural networks

Here, we present a data-driven analysis of dipole strength functions across the nuclear chart, employing an artificial neural network to model nuclear dipole responses. We train the network on a dataset of experimentally measured dipole strength functions for 216 different nuclei. To assess its predictive capability, we test the trained model on an additional set of 10 new nuclei, where experimental data exist. We demonstrate that the artificial neural network not only accurately reproduces known data but also identifies potential inconsistencies in experimental datasets, indicating which results may warrant further review or possible rejection. For nuclei where experimental data are sparse or unavailable, the network confirms theoretical calculations, reinforcing its utility as a predictive tool in nuclear physics. Finally, utilizing the predicted electric dipole polarizability, we extract the value of the symmetry energy at saturation density and find it consistent with results from the literature.

artificial neural networks↗
DOE OSTI2025

Extended Galerkin Neural Network Approximation of Singular Variational Problems with Error Control

We present extended Galerkin neural networks, a variational framework for approximating general boundary value problems (BVPs) with error control. The main contributions of this work are (1) a rigorous theory guiding the construction of new weighted least squares variational formulations suitable for use in neural network approximation of general BVPs, and (2) an “extended” feedforward network architecture which incorporates and is even capable of learning singular solution structures, thus greatly improving approximability of singular solutions. Furthermore, numerical results are presented for several problems, including steady Stokes flow around reentrant corners and in convex corners with Moffatt eddies in order to demonstrate efficacy of the method.

a posteriori error estimate↗
DOE OSTI2024

When ancient numerical demons meet physics-informed machine learning: adjoint-based gradients for implicit differentiable modeling

Recent advances in differentiable modeling, a genre of physics-informed machine learning that trains neural networks (NNs) together with process-based equations, have shown promise in enhancing hydrological models' accuracy, interpretability, and knowledge-discovery potential. Current differentiable models are efficient for NN-based parameter regionalization, but the simple explicit numerical schemes paired with sequential calculations (operator splitting) can incur numerical errors whose impacts on models' representation power and learned parameters are not clear. Implicit schemes, however, cannot rely on automatic differentiation to calculate gradients due to potential issues of gradient vanishing and memory demand. Here we propose a “discretize-then-optimize” adjoint method to enable differentiable implicit numerical schemes for the first time for large-scale hydrological modeling. The adjoint model demonstrates comprehensively improved performance, with Kling–Gupta efficiency coefficients, peak-flow and low-flow metrics, and evapotranspiration that moderately surpass the already-competitive explicit model. Therefore, the previous sequential-calculation approach had a detrimental impact on the model's ability to represent hydrological dynamics. Furthermore, with a structural update that describes capillary rise, the adjoint model can better describe baseflow in arid regions and also produce low flows that outperform even pure machine learning methods such as long short-term memory networks. The adjoint model rectified some parameter distortions but did not alter spatial parameter distributions, demonstrating the robustness of regionalized parameterization. Despite higher computational expenses and modest improvements, the adjoint model's success removes the barrier for complex implicit schemes to enrich differentiable modeling in hydrology.

58 GEOSCIENCES↗
DOE OSTI2025

SPIKANs: separable physics-informed Kolmogorov–Arnold networks

Physics-Informed Neural Networks (PINNs) have emerged as a promising method for solving partial differential equations (PDEs) in scientific computing. While PINNs typically use multilayer perceptrons (MLPs) as their underlying architecture, recent advancements have explored alternative neural network structures. One such innovation is the Kolmogorov–Arnold Network (KAN), which has demonstrated benefits over traditional MLPs, including faster neural scaling and better interpretability. The application of KANs to physics-informed learning has led to the development of Physics-Informed KANs (PIKANs), enabling the use of KANs to solve PDEs. However, despite their advantages, KANs often suffer from slower training speeds, particularly in higher-dimensional problems where the number of collocation points grows exponentially with the dimensionality of the system. To address this challenge, we introduce Separable Physics-Informed Kolmogorov–Arnold Networks (SPIKANs). This novel architecture applies the principle of separation of variables to PIKANs, decomposing the problem such that each dimension is handled by an individual KAN. This approach drastically reduces the computational complexity of training without sacrificing accuracy, facilitating their application to higher-dimensional PDEs. Through a series of benchmark problems, we demonstrate the effectiveness of SPIKANs, showcasing their superior scalability and performance compared to PIKANs and highlighting their potential for solving complex, high-dimensional PDEs in scientific computing.

Kolmogorov-Arnold networks↗
DOE OSTI2024

Speeding up and reducing memory usage for scientific machine learning via mixed precision

Scientific machine learning (SciML) has emerged as a versatile approach to address complex computational science and engineering problems. Within this field, physics-informed neural networks (PINNs) and deep operator networks (DeepONets) stand out as the leading techniques for solving partial differential equations by incorporating both physical equations and experimental data. However, training PINNs and DeepONets require significant computational resources, including long computational times and large amounts of memory. In search of computational efficiency, training neural networks using half precision (float16) rather than the conventional single (float32) or double (float64) precision has gained substantial interest, given the inherent benefits of reduced computational time and memory consumed. However, we find that float16 cannot be applied to SciML methods, because of gradient divergence at the start of training, weight updates going to zero, and the inability to converge to a local minima. To overcome these limitations, we explore mixed precision, which is an approach that combines the float16 and float32 numerical formats to reduce memory usage and increase computational speed. Our experiments showcase that mixed precision training not only substantially decreases training times and memory demands but also maintains model accuracy. Here, we also reinforce our empirical observations with a theoretical analysis. The research has broad implications for SciML in various computational applications.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

From disorganized data to emergent dynamic models: Questionnaires to partial differential equations

Starting with sets of disorganized observations of spatially varying and temporally evolving systems, obtained at different (also disorganized) sets of parameters, we demonstrate the data-driven derivation of parameter dependent, evolutionary partial differential equation (PDE) models capable of generating the data. This tensor type of data is reminiscent of shuffled (multidimensional) puzzle tiles. The independent variables for the evolution equations (their “space” and “time”) as well as their effective parameters are all emergent , i.e. determined in a data-driven way from our disorganized observations of behavior in them. We use a diffusion map based questionnaire approach to build a smooth parametrization of our emergent space/time/parameter space for the data. This approach iteratively processes the data by successively observing them on the “space,” the “time” and the “parameter” axes of a tensor. Once the data become organized, we use machine learning (here, neural networks) to approximate the operators governing the evolution equations in this emergent space. Our illustrative examples are based (i) on a simple advection–diffusion model; (ii) on a previously developed vertex-plus-signaling model of Drosophila embryonic development; and (iii) on two complex dynamic network models (one neuronal and one coupled oscillator model) for which no obvious smooth embedding geometry is known a priori. This allows us to discuss features of the process like symmetry breaking, translational invariance, and autonomousness of the emergent PDE model, as well as its interpretability.

generative models↗
NASA NTRS1991

A simple method to derive bounds on the size and to train multilayer neural networks

A new derivation is presented for the bounds on the size of a multilayer neural network to exactly implement an arbitrary training set; namely, the training set can be implemented with zero error with two layers and with the number of the hidden-layer neurons equal to no.1 is greater than p - 1. The derivation does not require the separation of the input space by particular hyperplanes, as in previous derivations. The weights for the hidden layer can be chosen almost arbitrarily, and the weights for the output layer can be found by solving no.1 + 1 linear equations. The method presented exactly solves (M), the multilayer neural network training problem, for any arbitrary training set.

Sartori, Michael A.↗
DOE OSTI2026

Causality-respecting adaptive refinement for PINNs: enabling precise interface evolution in phase field modeling

Physics-informed neural networks (PINNs) have emerged as a powerful tool for solving physical systems described by partial differential equations (PDEs). However, their accuracy in dynamical systems, particularly those involving sharp moving boundaries with complex initial morphologies, remains a challenge. Here, this study introduces an approach combining residual-based adaptive refinement (RBAR) with causality-informed training to enhance the performance of PINNs in solving spatio-temporal PDEs. Our method employs a three-step iterative process: initial causality-based training, RBAR-guided domain refinement, and subsequent causality training on the refined mesh. Applied to the Allen-Cahn equation, a widely-used model in phase field simulations, our approach demonstrates significant improvements in solution accuracy and computational efficiency over traditional PINNs. Notably, we observe an ‘overshoot and relocate’ phenomenon in dynamic cases with complex morphologies, showcasing the method’s adaptive error correction capabilities. This synergistic interaction between RBAR and causality training enables accurate capture of interface evolution, even in challenging scenarios where traditional PINNs fail. Our framework not only resolves the limitations of uniform refinement strategies but also provides a generalizable methodology for solving a broad range of spatio-temporal PDEs. The enhanced performance of the RBAR–causality combined framework demonstrates its strong potential for advancing PINN-based modeling of physical systems characterized by complex, evolving interfaces.

Allen-Cahn equations↗
DOE OSTI2025

Enhancing Drinking Water Quality Modeling: Leveraging Physics Informed Neural Networks for Learning with Imperfect Reaction Models and Partial Data

Chemical kinetics models, typically formulated as systems of ordinary or partial differential equations, are valuable tools for simulating drinking water quality. However, these models often face inaccuracies due to discrepancies between the laboratory and the real-world conditions, as well as limitations in experimental analytical methods, hindering the accurate representation of the true underlying chemical mechanisms. In this study, we propose a Physics Informed Neural Network (PINN), using the eXtreme Theory of Functional Connections, to improve the prediction of chemical concentrations over time. The PINN method accounts for imperfect chemical models and incorporates partial data to improve predictions. Focusing on reactions describing water disinfection residual and disinfectant byproduct formation, which are crucial for public health and regulatory compliance, we demonstrate that the PINN model is able to accurately predict the concentrations of chemical species across various pH values. Notably, the model extends its accuracy to predict concentrations of chemical species not originally included in its training data. The developed method can be extended to a variety of chemical systems, offering a wide array of potential applications.

13 HYDRO ENERGY↗
DOE OSTI2024

NeuroFEM

SAND2025-00525O NeuroFEM is a software tool that demonstrates a neuromorphic algorithm for solving finite element problems. It sets up a 2D finite element problem for the Poisson equation on a disk, constructs synaptic matrices, and simulates neural dynamics to solve the resulting sparse linear system. The software illustrates how the algorithm converges to the solution and plots the results, showcasing a neuromorphic counterpart to traditional methods like Conjugate Gradient or GMRES. This tool is designed to highlight the potential of neuromorphic algorithms for solving sparse linear systems, which are prevalent in various computational applications. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525.

SciDAC↗
DOE OSTI2024

EFIT‐AI: Machine Learning and Artificial Intelligence Assisted Equilibrium Reconstruction for Tokamak Experiments and Burning Plasmas (Final Report)

The EFIT-AI project is creating a modern advanced equilibrium reconstruction code suitable for tokamak experiments of burning plasmas. EFIT [1,2] was the first and is the most extensively used equilibrium reconstruction code in the world. This project builds on the production-level experience and adds key elements as follows. 1. A Model Order Reduction (MOR) version of the two-dimensional (2D) Grad-Shafranov equation solver (EFIT-MORNN) using physics-informed neural networks. 2. Improved optimization and data analysis capabilities using a Bayesian framework enhanced with machine learning. 3. A MOR version of the three-dimensional (3D) perturbed equilibrium reconstruction tool.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2025

Model Development and Analysis of a High-Fidelity Neutron Transport Sensor: The Quadrupole Detector Concept for Measurement of the Neutron Flux Gradient

Accurate reconstruction of the neutron flux distribution within a reactor core is essential for safe and efficient reactor operation. Traditional power shape synthesis in Light Water Reactors relies on hundreds of in-core detectors. However, this approach becomes impractical for Advanced Reactors and Microreactors due to limited space and harsh environments. To address this challenge, we propose a data-driven methodology that combines high-fidelity modeling with real-time ex-core sensor measurements, enabling the reconstruction of core power distribution while minimizing the reliance on intrusive in-core instrumentation. This project began in FY24 and achieved two initial milestones: (1) the definition of a three-year development plan for a Digital Twin framework and (2) the development of high-fidelity neutronics models of the Purdue University Reactor One (PUR-1) using both MCNP6 and OpenMC. The PUR-1 reactor, a zero-power facility, was selected due to its suitability for neutronics-focused modeling and the availability of experimental data for validation. Both models were benchmarked using neutron flux measurements obtained from irradiated gold foils, which were strategically placed within the core during a dedicated campaign in July 2024. This report marks the continuation and completion of those foundational tasks. The OpenMC model has been refined (improved geometric accuracy, expanded cross-section libraries, and refined sampling) and validated using additional experimental data. An updated sensor design—based on quadrupole configuration—was designed to measure both ex-core flux and its spatial gradient. These measurements will serve as inputs to a neural network-based reconstruction algorithm. Finally, the methodology was demonstrated on a two-dimensional test case representative of the heterogeneous material composition of the PUR-1 reactor core. A neural network implementation of the Kirchhoff-Helmholtz integral equation was employed to solve the boundary value problem using peripheral sensor measurements. The preliminary results confirm the strong potential of the proposed approach for accurate and minimally invasive neutron flux reconstruction.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
NASA NTRS1981

Benefits of detailed models of muscle activation and mechanics

Recent biophysical and physiological studies identified some of the detailed mechanisms involved in excitation-contraction coupling, muscle contraction, and deactivation. Mathematical models incorporating these mechanisms allow independent estimates of key parameters, direct interplay between basic muscle research and the study of motor control, and realistic model behaviors, some of which are not accessible to previous, simpler, models. The existence of previously unmodeled behaviors has important implications for strategies of motor control and identification of neural signals. New developments in the analysis of differential equations make the more detailed models feasible for simulation in realistic experimental situations.

Lehman, S. L.↗
NASA NTRS1997

High precision computing with charge domain devices and a pseudo-spectral method therefor

The present invention enhances the bit resolution of a CCD/CID MVM processor by storing each bit of each matrix element as a separate CCD charge packet. The bits of each input vector are separately multiplied by each bit of each matrix element in massive parallelism and the resulting products are combined appropriately to synthesize the correct product. In another aspect of the invention, such arrays are employed in a pseudo-spectral method of the invention, in which partial differential equations are solved by expressing each derivative analytically as matrices, and the state function is updated at each computation cycle by multiplying it by the matrices. The matrices are treated as synaptic arrays of a neural network and the state function vector elements are treated as neurons. In a further aspect of the invention, moving target detection is performed by driving the soliton equation with a vector of detector outputs. The neural architecture consists of two synaptic arrays corresponding to the two differential terms of the soliton-equation and an adder connected to the output thereof and to the output of the detector array to drive the soliton equation.

Barhen, Jacob↗
DOE OSTI2025

High‐Resolution National‐Scale Water Modeling Is Enhanced by Multiscale Differentiable Physics‐Informed Machine Learning

Abstract The National Water Model (NWM) is a key tool for flood forecasting, planning, and water management. Key challenges facing the NWM include calibration and parameter regionalization when confronted with big data. We present two novel versions of high‐resolution (∼37 km 2 ) differentiable models (a type of hybrid model): one with implicit, unit‐hydrograph‐style routing and another with explicit Muskingum‐Cunge routing in the river network. The former predicts streamflow at basin outlets whereas the latter presents a discretized product that seamlessly covers rivers in the conterminous United States (CONUS). Both versions use neural networks to provide a multiscale parameterization and process‐based equations to provide a structural backbone, which were trained simultaneously (“end‐to‐end”) on 2,807 basins across the CONUS and evaluated on 4,997 basins. Both versions show great potential to elevate future NWM performance for extensively calibrated as well as ungauged sites: the median daily Nash‐Sutcliffe efficiency of all 4,997 basins is improved to around 0.68 from 0.48 of NWM3.0. As they resolve spatial heterogeneity, both versions greatly improved simulations in the western CONUS and also in the Prairie Pothole Region, a long‐standing modeling challenge. The Muskingum‐Cunge version further improved performance for basins >10,000 km 2 . Overall, our results show how neural‐network‐based parameterizations can improve NWM performance for providing operational flood predictions while maintaining interpretability and multivariate outputs. The modeling system supports the Basic Model Interface (BMI), which allows seamless integration with the next‐generation NWM. We also provide a CONUS‐scale hydrologic data set for further evaluation and use.

Song, Yalan [Civil and Environmental Engineering T↗
NASA NTRS1992

Learning a trajectory using adjoint functions and teacher forcing

A new methodology for faster supervised temporal learning in nonlinear neural networks is presented which builds upon the concept of adjoint operators to allow fast computation of the gradients of an error functional with respect to all parameters of the neural architecture, and exploits the concept of teacher forcing to incorporate information on the desired output into the activation dynamics. The importance of the initial or final time conditions for the adjoint equations is discussed. A new algorithm is presented in which the adjoint equations are solved simultaneously (i.e., forward in time) with the activation dynamics of the neural network. We also indicate how teacher forcing can be modulated in time as learning proceeds. The results obtained show that the learning time is reduced by one to two orders of magnitude with respect to previously published results, while trajectory tracking is significantly improved. The proposed methodology makes hardware implementation of temporal learning attractive for real-time applications.

Toomarian, Nikzad B.↗