Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “neural”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2026

Quantum-inspired weight-constrained neural network: Reducing variable numbers by 100× compared to standard neural networks

Although quantum machine learning has shown great promise, the practical application of quantum computers remains constrained in the noisy intermediate-scale quantum era. To take advantage of quantum machine learning, we investigate the underlying mathematical principles of these quantum models and find that the quantum neural network with amplitude encoding is equivalent to a weight-constrained neural network. Motivated by this discovery, we develop a classical weight-constrained neural network. We find that this approach can reduce the number of variables in a classical neural network by a factor of 135 while preserving its accuracy. In addition, we develop a dropout method to enhance the robustness of quantum machine learning models, which are highly susceptible to adversarial attacks. This technique can also be applied to improve the adversarial robustness of the classical weight-constrained neural network, which is essential for industry applications, such as self-driving vehicles. Our work offers an approach to reduce the complexity of large classical neural networks, addressing a critical challenge in machine learning.

quantum algorithms & computation↗
DOE OSTI2025

Accelerating Hamiltonian Monte Carlo for Bayesian inference in neural networks and neural operators

Hamiltonian Monte Carlo (HMC) is a powerful and accurate method to sample from the posterior distribution in Bayesian inference. However, HMC techniques are computationally demanding for Bayesian neural networks due to the high dimensionality of the network’s parameter space and the non-convexity of their posterior distributions. Therefore, various approximation techniques, such as variational inference (VI) or stochastic gradient MCMC, are often employed to infer the posterior distribution of the network parameters. Such approximations introduce inaccuracies in the inferred distributions, resulting in unreliable uncertainty estimates. In this work, we propose a hybrid approach that combines inexpensive VI and accurate HMC methods to efficiently and accurately quantify uncertainties in neural networks and neural operators. The proposed approach leverages an initial VI training on the full network. We examine the influence of individual parameters on the prediction uncertainty, which shows that a large proportion of the parameters do not contribute substantially to uncertainty in the network predictions. This information is then used to significantly reduce the dimension of the parameter space, and HMC is performed only for the subset of network parameters that strongly influence prediction uncertainties. This yields a framework for accelerating the full batch HMC for posterior inference in neural networks. We demonstrate the efficiency and accuracy of the proposed framework on deep neural networks and operator networks, showing that inference can be performed for large networks with tens to hundreds of thousands of parameters. Finally, we show that this method can effectively learn surrogates for complex physical systems by modeling the operator that maps from upstream conditions to wall-pressure data on a cone in hypersonic flow.

Bayesian inference↗
DOE OSTI2026

Neural entropy-stable conservative flux form neural networks for learning hyperbolic conservation laws

We propose a neural entropy-stable conservative flux form neural network (NESCFN) for learning hyperbolic conservation laws and their associated entropy functions directly from solution trajectories, without requiring any predefined numerical discretization. While recent neural network architectures have successfully integrated classical numerical principles into learned models, most rely on prior knowledge of the governing equations or assume a fixed discretization. Our approach removes this dependency by embedding entropy-stable design principles into the learning process itself, enabling the discovery of physically consistent dynamics in a fully data-driven setting. By jointly learning both the flux function and a corresponding entropy, NESCFN promotes conservation and entropy dissipation, which is critical for long-term stability and fidelity in the system of hyperbolic conservation laws. Furthermore, numerical results demonstrate that the method achieves stability and conservation over extended time horizons and accurately captures shock propagation speeds, even without oracle access to future-time solution profiles in the training data.

Conservative flux form↗
DOE OSTI2025

Co-training of multiple neural networks for simultaneous optimization and training of physics-informed neural networks for composite curing

This paper introduces a Physics-Informed Neural Network (PINN) technique that co-trains neural networks (NNs) that represent each function in a system of equations to simultaneously solve equations representing an out-of-autoclave (OOA) cure process while conducting optimization in adherence to process requirements. Specifically, this co-training approach benefits from using NNs to represent OOA inputs (air temperature profile) and outputs (part and tool temperature profiles and degree of cure). Production requirements can then be levied on the inputs, such as maximum air temperature and minimum cure cycle, and simultaneously on the outputs, such as degree of cure, maximum part temperature, and part temperature rate limits. The technique is validated with finite element (FE) simulations and physical experiments for curing a Toray T830H-6 K/3900-2D composite panel. Furthermore, this novel approach efficiently models and optimizes the OOA cure process.

Composite curing↗
DOE OSTI2025

Neural chaos: A spectral stochastic neural operator

Building surrogate models for operators with uncertainty quantification capabilities is essential for many engineering applications where randomness–such as variability in material properties, boundary conditions, and initial conditions–is unavoidable. Polynomial Chaos Expansion (PCE) is widely recognized as a go-to method for constructing stochastic surrogates in both intrusive and non-intrusive ways, and it has recently been used in the context of operator learning. However, its application becomes challenging for complex or high-dimensional processes, as achieving accuracy requires higher-order polynomials, which can increase computational demand and/or the risk of overfitting. Furthermore, PCE requires specialized treatments to manage random variables that are not independent, and these treatments may be problem-dependent or may fail with increasing complexity. Here, in this work, we adopt the same formalism as the spectral expansion used in PCE; however, we replace the classical polynomial basis functions with neural network (NN) basis functions to leverage their expressivity. To achieve this, we propose an algorithm that identifies NN-parameterized basis functions in a purely data-driven manner, without any prior assumptions about the joint distribution of the random variables involved, whether independent or dependent, or about their marginal distributions. The proposed algorithm identifies each NN-parameterized basis function sequentially, ensuring they are orthogonal with respect to the data distribution. The basis functions are constructed directly on the joint stochastic variables without requiring a tensor product structure or assuming independence of the random variables. This approach may offer greater flexibility for complex stochastic models, while simplifying implementation compared to the tensor product structures typically used in PCE to handle random vectors. This is particularly advantageous given the current state of open-source packages, where building and training neural networks can be done with just a few lines of code and extensive community support. We demonstrate the effectiveness of the proposed scheme through several numerical examples of varying complexity and provide comparisons with classical PCE.

Polynomial chaos expansion↗
DOE OSTI2025

Large-scale experimental validation of thermochemical water-splitting oxides discovered by defect graph neural networks

Thermochemical water-splitting (TCH) based on 2-step thermal redox cycles in metal oxides is a promising approach to generating H 2 , but state-of-the-art (SOTA) CeO 2 has several practical limitations, which has motivated continued materials discovery efforts in this field. Here, in this study, we improve upon a SOTA defect graph neural network (dGNN) surrogate model's oxygen vacancy predictions and combine them with materials project phase diagrams to down-select and discover structurally diverse, experimentally known metal oxides whose TCH performance was previously unknown. Amongst twelve candidates selected based on our high-throughput screening and down-selection criteria, we achieved ∼80% accuracy in identifying materials with stable redox cycling and hydrogen production in stagnation flow reactor water-splitting experiments. Closer to 100% accuracy can be achieved if higher-accuracy, hybrid DFT-predicted vacancy formation energies were computed and used in lieu of the most uncertain dGNN-based screening predictions, as they correct false positives to true negatives. Notably, two discovered candidates, Sr 3 PrMn 2 O 8 and Ba 2 Fe 2 O 5 , display hydrogen yields greater than CeO 2 under specific redox conditions. In conclusion, these results demonstrate our ability to computationally predict and experimentally validate promising candidate TCH materials that have the potential to compete with CeO 2 .

08 HYDROGEN↗
DOE OSTI2026

ProtNHF: Neural Hamiltonian Flows for Controllable Protein Sequence Generation

This dataset accompanies the publication "ProtNHF: Neural Hamiltonian Flows for Controllable Protein Sequence Generation". This paper introduces a new AI model for protein sequence generation. This dataset contains data related to experiments discussed in the publication. This includes generated sequences and evaluation metrics supporting all unconditional and bias-controlled experiments in the ProtNHF paper.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2025

Understanding latent timescales in neural ordinary differential equation models of advection-dominated dynamical systems

The neural ordinary differential equation (ODE) framework has shown considerable promise in recent years in developing highly accelerated surrogate models for complex physical systems characterized by partial differential equations (PDEs). For PDE-based systems, state-of-the-art neural ODE strategies leverage a two-step procedure to achieve this acceleration: a nonlinear dimensionality reduction step provided by an autoencoder, and a time integration step provided by a neural-network based model for the resultant latent space dynamics (the neural ODE). This work explores the applicability of such autoencoder-based neural ODE strategies for PDEs in which advection terms play a critical role. More specifically, alongside predictive demonstrations, physical insight into the sources of model acceleration (i.e., how the neural ODE achieves its acceleration) is the scope of the current study. Such investigations are performed by quantifying the effects of both autoencoder and neural ODE components on latent system time-scales using eigenvalue analysis of dynamical system Jacobians. To this end, the sensitivity of various critical training parameters – de-coupled versus end-to-end training, latent space dimensionality, and the role of training trajectory length, for example – to both model accuracy and the discovered latent system timescales is quantified. Furthermore, this work specifically uncovers the key role played by the training trajectory length (the number of rollout steps in the loss function during training) on the latent system timescales: larger trajectory lengths correlate with an increase in limiting neural ODE time-scales, and optimal neural ODEs are found to recover the largest time-scales of the full-order (ground-truth) system. Demonstrations are performed across fundamentally different unsteady fluid dynamics configurations influenced by advection: (1) the Kuramoto–Sivashinsky equations (2) Hydrogen-Air channel detonations (the compressible reacting Navier–Stokes equations with detailed chemistry), and (3) 2D Atmospheric flow.

Advection-dominated dynamical systems↗
DOE OSTI2026

A Data-Driven Method for Modeling Creep-Fatigue Stress- Strain Behavior Using Neural ODEs

In this paper, we introduce a data-driven machine learning approach for modeling one-dimensional stress–strain behavior under cyclic loading, utilizing experimental data from the nickel-based Alloy 617. The study employs uniaxial creep–fatigue test data acquired under various loading histories and compares two distinct neural network-based ODE models. The first model, known as the black-box model, comprehensively describes the strain–stress relationship using a Neural ODE equation. To interpret this black-box model, we apply the Sparse Identification of Nonlinear Dynamical Systems (SINDy) technique, transforming the black-box model into an equation-based model using symbolic regression. The second model, the Neural flow rule model, incorporates Hooke’s Law for the linear elastic component, with the nonlinear part characterized by a Neural ODE. Both models are trained with experimental data to accurately reflect the observed stress–strain behavior. We conduct a detailed comparison with the standard Chaboche model, which includes three back stresses. Our results demonstrate that the neural network-based ODE models precisely capture the experimental creep–fatigue mechanical behavior, exceeding the standard Chaboche model’s accuracy. Furthermore, an interpretable model derived from the black-box neural ODE model through symbolic regression achieves accuracy comparable to the Chaboche model, enhancing its interpretability. The results highlight the potential of neural network-based ODE models to depict complex creep–fatigue behavior, eliminating the necessity for experts to define a specific, material-focused model form.

creep-fatigue↗
DOE OSTI2025

On the Effectiveness of Neural Operators at Zero-Shot Weather Downscaling [SWR-25-20]

Code repository for the experiments performed in the paper: On the Effectiveness of Neural Operators at Zero-Shot Weather Downscaling (https://doi.org/10.1017/eds.2025.11) Overall, our work investigates the zero-shot downscaling potential of neural operators. To summarize, our contributions are: 1. We provide a comparative analysis based on two challenging weather downscaling problems, between various neural operator and non-neural-operator methods with large upsampling factors (e.g., 8x and 15x) and fine grid resolutions (e.g., 2 km × 2 km wind speed). 2. We examine whether neural operator layers provide unique advantages when testing downscaling models on upsampling factors higher than those seen during training, i.e., zero-shot downscaling. Our results instead show the surprising success of an approach that combines a powerful transformer-based model with a parameter-free interpolation step at zero-shot weather downscaling. 3. We find that this Swin-Transformer-based approach mostly outperforms all neural operator models in terms of average error metrics, whereas an enhanced super-resolution generative adversarial network (ESRGAN)-based approach is better than most models in capturing the physics of the system, and suggests their use in future work as strong baselines. However, these approaches still do not capture variations at smaller spatial scales well, including the physical characteristics of turbulence in the HR data. This suggests a potential for improvement in transformer or GAN-based methods and neural-operator-based methods for zero-shot weather downscaling.

Sinha, Saumya [National Renewable Energy Laborator↗
DOE OSTI2024

Tracking the topology of neural manifolds across populations

Neural manifolds summarize the intrinsic structure of the information encoded by a population of neurons. Advances in experimental techniques have made simultaneous recordings from multiple brain regions increasingly commonplace, raising the possibility of studying how these manifolds relate across populations. However, when the manifolds are nonlinear and possibly code for multiple unknown variables, it is challenging to extract robust and falsifiable information about their relationships. We introduce a framework, called the method of analogous cycles, for matching topological features of neural manifolds using only observed dissimilarity matrices within and between neural populations. We demonstrate via analysis of simulations and in vivo experimental data that this method can be used to correctly identify multiple shared circular coordinate systems across both stimuli and inferred neural manifolds. Conversely, the method rejects matching features that are not intrinsic to one of the systems. Further, as this method is deterministic and does not rely on dimensionality reduction or optimization methods, it is amenable to direct mathematical investigation and interpretation in terms of the underlying neural activity. We thus propose the method of analogous cycles as a suitable foundation for a theory of cross-population analysis via neural manifolds.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Active Learning of Microgrid Frequency Dynamics Using Neural Ordinary Differential Equations

Accurate frequency modelling of inverter‐based resource (IBR)‐dominated power systems is crucial for ensuring stable, reliable and resilient operations, particularly given their inherent low‐inertia characteristics and fast dynamics that traditional swing equation‐based models inadequately capture. This paper explores neural ordinary differential equations (Neural ODEs) as a computationally efficient, data‐driven framework for modelling power system frequency dynamics, specifically within microgrids integrating high penetrations of distributed energy resources (DERs). The developed neural ODEs framework incorporates a neural network architecture designed to capture input dynamics. By actively perturbing the system with a known signal, the Python‐based neural ODEs framework was trained using measured system states and inputs, without the need for detailed system information. The framework, tested on a model of the Cordova, AK, microgrid, achieved a goodness of fit ranging from 60% to 99% across different state variables and maintained a mean square error in the 10 -6 p.u. range under square and step excitation signals. The proposed approach demonstrated robustness to measurement noise and initial condition variations while maintaining low computational complexity suitable for real‐time power system control applications. Furthermore, transfer learning enabled the neural ODEs model to adapt to the following changes in system topology or generator dispatch, highlighting its effectiveness for dynamic microgrids with frequently evolving configurations and diverse DERs.

Aryal, Tara [South Dakota State Univ., Brookings, ↗
DOE OSTI2025

Direct effects of prolonged TNF-α and IL-6 exposure on neural activity in human iPSC-derived neuron-astrocyte co-cultures

Cognitive impairment is one of the many symptoms reported by individuals suffering from long-COVID and other post-viral infection disorders such as myalgic encephalomyelitis/chronic fatigue syndrome (ME/CFS). A common factor among these conditions is a sustained immune response and increased levels of inflammatory cytokines. Tumor necrosis factor alpha (TNF-α) and interleukin-6 (IL-6) are two such cytokines that are elevated in patients diagnosed with long-COVID and ME/CFS. In this study, we characterized the changes in neural functionality, secreted cytokine profiles, and gene expression in co-cultures of human iPSC-derived neurons and primary astrocytes in response to prolonged exposure to TNF-α and IL-6. We found that exposure to TNF-α produced both a concentration-independent and concentration-dependent response in neural activity. Burst duration was significantly reduced within a few days of exposure regardless of concentration (1 pg/mL – 100 ng/mL) but returned to baseline after 7 days. Treatment with low concentrations of TNF-α (e.g., 1 and 25 pg/mL) did not lead to changes in the secreted cytokine profile or gene expression but still resulted in significant changes to electrophysiological features such as interspike interval and burst duration. Conversely, treatment with high concentrations of TNF-α (e.g., 10 and 100 ng/mL) led to reduced spiking activity, which may be correlated to changes in neural health, gene expression, and increases in inflammatory cytokine secretion (e.g., IL-1β, IL-4, and CXCL-10) that were observed at higher TNF-α concentrations. Prolonged exposure to IL-6 led to changes in bursting features, with significant reduction in the number of spikes in bursts across a wide range of treatment concentrations (i.e., 1 pg/mL–10 ng/mL). In combination, the addition of IL-6 appears to counteract the changes to neural function induced by low concentrations of TNF-α, while at high concentrations of TNF-α the addition of IL-6 had little to no effect. Conversely, the changes to electrophysiological features induced by IL-6 were lost when the cultures were co-stimulated with TNF-α regardless of the concentration, suggesting that TNF-α may play a more pronounced role in altering neural function. These results indicate that increased concentrations of key inflammatory cytokines associated with long-COVID can directly impact neural function and may be a component of the cognitive impairment associated with long-COVID and other post-viral infection disorders.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2024

Overview of RFID Applications Utilizing Neural Networks

As Radio Frequency Identification (RFID) methods continue to evolve to higher levels of complexity, one form of machine learning is making its appearance. The use of Neural Networks (NN) in the RFID field is steadily increasing, and in the fields of localization and activity recognition, promising results are being shown from a variety of research. RFID applications fall primarily under two types of problems including regression and classification. We analyze RIFD localization techniques which fall under regression, and activity recognition which falls under classification. Many works don’t classify themselves as activity recognition methods, but because they fall under the classification category, we still consider them as activity recognition techniques. This research overviews the Neural Network models in the localization field based on whether they can perform independently of the environment in which they were tested. For activity recognition and accessory fields, the major methods involve tag-based and tag-free approaches. In conclusion, after the models are surveyed, a comparison study is given to examine what may be the cause for increased accuracy between different Neural Network models.

42 ENGINEERING↗
DOE OSTI2025

Mitigating spectral bias in neural operators via high-frequency scaling for physical systems

Neural operators have emerged as powerful surrogates for modeling complex physical problems. However, they suffer from spectral bias making them oblivious to high-frequency modes, which are present in multiscale physical systems. Therefore, they tend to produce over-smoothed solutions, which is particularly problematic in modeling turbulence and for systems with intricate patterns and sharp gradients such as multi-phase flow systems. In this work, we introduce a new approach named high-frequency scaling (HFS) to mitigate spectral bias in convolutional-based neural operators. By integrating HFS with proper variants of UNet, we demonstrate a higher prediction accuracy by mitigating spectral bias in single and two-phase flow problems. Unlike Fourierbased techniques, HFS is directly applied to the latent space, thus eliminating the computational cost associated with the Fourier transform. Additionally, we investigate alternative spectral bias mitigation through a diffusion model conditioned on neural operators. While the diffusion model integrated with the standard neural operator may still suffer from significant errors, these errors are substantially reduced when the diffusion model is integrated with a HFS-enhanced neural operator.

97 MATHEMATICS AND COMPUTING↗