Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “NEURAL NETWORKS”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 523 records · Page 29
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

Modeling a Thermionic Electron Source Using a Physics-Informed Neural Network

We explore the application of Physics-Informed Neural Networks (PINNs) for simulation of thermionic electron sources. This is motivated by the need for quick surrogate models used to simulate such sources within a digital twin of a complete particle accelerator. Here, a PINN was developed on a simplified model of the thermionic source: the planar diode. This model very accurately simulated the system, and performed significantly better than a traditional neural network while also using less training data. We hope to apply this proof-of-concept in motivating the development of a PINN model for a full thermionic electron source at the University of Chicago.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2025

TorchBraid: High-Performance Layer-Parallel Training of Deep Neural Networks with MPI and GPU Acceleration

TorchBraid is a high-performance implementation of layer-parallel training for deep neural networks (DNNs) supporting MPI-based parallelism and GPU acceleration. Layer-parallel training has been developed to overcome the serialization inherent in forward and backward propagation of DNNs that limits utilization of computational resources in the strong scaling limit. To achieve this, TorchBraid integrates the PyTorch neural network framework with the state-of-the-art XBraid time-parallel library. Furthermore, this article presents the use and performance of TorchBraid, in addition to solutions for overcoming the algorithmic challenges inherent in combining automatic differentiation with layer-parallel. Results are presented with and without GPU acceleration for the Tiny ImageNet and MNIST image classification data sets, as well as recurrent neural networks. Overall, TorchBraid enables fast training of DNNs, both in a strong and weak scaling context. In addition to the TorchBraid software, several new advances in applying layer-parallel algorithms are detailed. Integration of layer-parallel with data-parallel algorithms is presented for the first time, showing the computational advantages of the combination. Standard deep learning techniques, like batch-normalization, are developed for layer-parallel training. Finally, a new approach combining layer-parallel with spatial coarsening in order to accelerate training for 3D image classification shows roughly a 10× speedup over serial execution.

Layer-parallel↗
NASA NTRS1993

Coarse-coded higher-order neural networks for PSRI object recognition

A higher-order neural network (HONN) can be designed to be invariant to changes in scale, translation, and inplane rotation. Invariances are built directly into the architecture of a HONN and do not need to be learned. Consequently, fewer training passes and a smaller training set are required to learn to distinguish between objects. The size of the input field is limited, however, because of the memory required for the large number of interconnections in a fully connected HONN. By coarse coding the input image, the input field size can be increased to allow the larger input scenes required for practical object recognition problems. We describe a coarse coding technique and present simulation results illustrating its usefulness and its limitations. Our simulations show that a third-order neural network can be trained to distinguish between two objects in a 4096 x 4096 pixel input field independent of transformations in translation, in-plane rotation, and scale in less than ten passes through the training set. Furthermore, we empirically determine the limits of the coarse coding technique in the object recognition domain.

Spirkovska, Lilly↗
DOE OSTI2024

PyTorch Implementation of Log-Additive Convolutional Neural Networks

This code is a collection of python code that defines, trains, and tests Log-Additive Convolutional Neural Networks. The model components and training routine are based on the PyTorch python library. The code implements the Log-Additive Convolutional Neural Networks as described in Pagendam et al. 2023. In addition to the Log-Additive Convolutional Neural Networks, this library also defines the Log-Normal Density loss function as described in Pagendam et al. 2023. Code from this paper is not publicly available, so the Pytorch implementation of this type of model is unique to this library.

Callis, Skylar↗
DOE OSTI2021

Physics-Informed Neural Networks for Heat Transfer Problems

Abstract Physics-informed neural networks (PINNs) have gained popularity across different engineering fields due to their effectiveness in solving realistic problems with noisy data and often partially missing physics. In PINNs, automatic differentiation is leveraged to evaluate differential operators without discretization errors, and a multitask learning problem is defined in order to simultaneously fit observed data while respecting the underlying governing laws of physics. Here, we present applications of PINNs to various prototype heat transfer problems, targeting in particular realistic conditions not readily tackled with traditional computational methods. To this end, we first consider forced and mixed convection with unknown thermal boundary conditions on the heated surfaces and aim to obtain the temperature and velocity fields everywhere in the domain, including the boundaries, given some sparse temperature measurements. We also consider the prototype Stefan problem for two-phase flow, aiming to infer the moving interface, the velocity and temperature fields everywhere as well as the different conductivities of a solid and a liquid phase, given a few temperature measurements inside the domain. Finally, we present some realistic industrial applications related to power electronics to highlight the practicality of PINNs as well as the effective use of neural networks in solving general heat transfer problems of industrial complexity. Taken together, the results presented herein demonstrate that PINNs not only can solve ill-posed problems, which are beyond the reach of traditional computational methods, but they can also bridge the gap between computational and experimental heat transfer.

Engineering↗
NASA NTRS1994

Neural network classification of clinical neurophysiological data for acute care monitoring

The purpose of neurophysiological monitoring of the 'acute care' patient is to allow the accurate recognition of changing or deteriorating neurological function as close to the moment of occurrence as possible, thus permitting immediate intervention. Results confirm that: (1) neural networks are able to accurately identify electroencephalogram (EEG) patterns and evoked potential (EP) wave components, and measuring EP waveform latencies and amplitudes; (2) neural networks are able to accurately detect EP and EEG recordings that have been contaminated by noise; (3) the best performance was obtained consistently with the back propagation network for EP and the HONN for EEG's; (4) neural network performed consistently better than other methods evaluated; and (5) neural network EEG and EP analyses are readily performed on multichannel data.

Sgro, Joseph↗
NASA NTRS2001

Engine With Regression and Neural Network Approximators Designed

At the NASA Glenn Research Center, the NASA engine performance program (NEPP, ref. 1) and the design optimization testbed COMETBOARDS (ref. 2) with regression and neural network analysis-approximators have been coupled to obtain a preliminary engine design methodology. The solution to a high-bypass-ratio subsonic waverotor-topped turbofan engine, which is shown in the preceding figure, was obtained by the simulation depicted in the following figure. This engine is made of 16 components mounted on two shafts with 21 flow stations. The engine is designed for a flight envelope with 47 operating points. The design optimization utilized both neural network and regression approximations, along with the cascade strategy (ref. 3). The cascade used three algorithms in sequence: the method of feasible directions, the sequence of unconstrained minimizations technique, and sequential quadratic programming. The normalized optimum thrusts obtained by the three methods are shown in the following figure: the cascade algorithm with regression approximation is represented by a triangle, a circle is shown for the neural network solution, and a solid line indicates original NEPP results. The solutions obtained from both approximate methods lie within one standard deviation of the benchmark solution for each operating point. The simulation improved the maximum thrust by 5 percent. The performance of the linear regression and neural network methods as alternate engine analyzers was found to be satisfactory for the analysis and operation optimization of air-breathing propulsion engines (ref. 4).

Patnaik, Surya N.↗
NASA NTRS1997

Design of Neural Networks for Fast Convergence and Accuracy: Dynamics and Control

A procedure for the design and training of artificial neural networks, used for rapid and efficient controls and dynamics design and analysis for flexible space systems, has been developed. Artificial neural networks are employed, such that once properly trained, they provide a means of evaluating the impact of design changes rapidly. Specifically, two-layer feedforward neural networks are designed to approximate the functional relationship between the component/spacecraft design changes and measures of its performance or nonlinear dynamics of the system/components. A training algorithm, based on statistical sampling theory, is presented, which guarantees that the trained networks provide a designer-specified degree of accuracy in mapping the functional relationship. Within each iteration of this statistical-based algorithm, a sequential design algorithm is used for the design and training of the feedforward network to provide rapid convergence to the network goals. Here, at each sequence a new network is trained to minimize the error of previous network. The proposed method should work for applications wherein an arbitrary large source of training data can be generated. Two numerical examples are performed on a spacecraft application in order to demonstrate the feasibility of the proposed approach.

Maghami, Peiman G.↗
DOE OSTI2023

Applying Physics-Informed Neural Networks to Solve Navier–Stokes Equations for Laminar Flow around a Particle

In recent years, Physics-Informed Neural Networks (PINNs) have drawn great interest among researchers as a tool to solve computational physics problems. Unlike conventional neural networks, which are black-box models that “blindly” establish a correlation between input and output variables using a large quantity of labeled data, PINNs directly embed physical laws (primarily partial differential equations) within the loss function of neural networks. By minimizing the loss function, this approach allows the output variables to automatically satisfy physical equations without the need for labeled data. The Navier–Stokes equation is one of the most classic governing equations in thermal fluid engineering. This study constructs a PINN to solve the Navier–Stokes equations for a 2D incompressible laminar flow problem. Flows passing around a 2D circular particle are chosen as the benchmark case, and an elliptical particle is also examined to enrich the research. The velocity and pressure fields are predicted by the PINNs, and the results are compared with those derived from Computational Fluid Dynamics (CFD). Additionally, the particle drag force coefficient is calculated to quantify the discrepancy in the results of the PINNs as compared to CFD outcomes. The drag coefficient maintained an error within 10% across all test scenarios.

Hu, Beichao (ORCID:0009000163215151)↗
NASA NTRS1991

Higher-order neural network software for distortion invariant object recognition

The state-of-the-art in pattern recognition for such applications as automatic target recognition and industrial robotic vision relies on digital image processing. We present a higher-order neural network model and software which performs the complete feature extraction-pattern classification paradigm required for automatic pattern recognition. Using a third-order neural network, we demonstrate complete, 100 percent accurate invariance to distortions of scale, position, and in-plate rotation. In a higher-order neural network, feature extraction is built into the network, and does not have to be learned. Only the relatively simple classification step must be learned. This is key to achieving very rapid training. The training set is much smaller than with standard neural network software because the higher-order network only has to be shown one view of each object to be learned, not every possible view. The software and graphical user interface run on any Sun workstation. Results of the use of the neural software in autonomous robotic vision systems are presented. Such a system could have extensive application in robotic manufacturing.

Reid, Max B.↗
DOE OSTI2024

Deep Koopman Neural Network for Analyzing High-Energy-Density Simulations of Electrical Wire Explosions

Megaampere-scale electrical wire experiments (EWEs) provide a platform for studying magnetohydrodynamic (MHD) instability growth in magneto-inertial fusion (MIF) devices. Even when nonlinear simulations of these experiments can digitally reproduce much of the experimentally observed instability growth, interpreting the results and understanding mode growth and evolution can be non-trivial. As a first step toward providing better interpretation of these simulation features, this work investigates the use of a deep neural network that uses Koopman operator theory to analyze the dynamics of pulsed-power-driven explosions of EWEs. This deep neural network is trained on 1-D resistive MHD simulations of EWEs. This neural network learns to transform the nonlinear data into a lower-dimensional representation where the time dynamics are linear. Layers of this neural network are shown to learn features of the simulations, including the locations of shock waves and different physical regimes of the simulation. Using the learned features, the network can compress a time state of the simulation consisting of 5120 data point into a 36-parameter lower-dimensional latent space embedding. Furthermore, these embeddings are shown to be clustered in the latent space by initial radius and time state.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
NASA NTRS2004

Neural network tracking and extension of positive tracking periods

Feature detectors have been considered for the role of supplying additional information to a neural network tracker. The feature detector focuses on areas of the image with significant information. Basically, if a picture says a thousand words, the feature detectors are looking for the key phrases (keypoints). These keypoints are rotationally invariant and may be matched across frames. Application of these advanced feature detectors to the neural network tracking system at JPL has promising potential. As part of an ongoing program, an advanced feature detector was tested for augmentation of a neural network based tracker. The advance feature detector extended tracking periods in test sequences including aircraft tracking, rover tracking, and simulated Martian landing. Future directions of research are also discussed.

feature extraction↗
NASA NTRS1990

Neural Networks in Nonlinear Aircraft Control

Recent research indicates that artificial neural networks offer interesting learning or adaptive capabilities. The current research focuses on the potential for application of neural networks in a nonlinear aircraft control law. The current work has been to determine which networks are suitable for such an application and how they will fit into a nonlinear control law.

Linse, Dennis J.↗
DOE OSTI2022

Gradient-enhanced physics-informed neural networks for forward and inverse PDE problems

Deep learning has been shown to be an effective tool in solving partial differential equations (PDEs) through physics-informed neural networks (PINNs). PINNs embed the PDE residual into the loss function of the neural network, and have been successfully employed to solve diverse forward and inverse PDE problems. However, one disadvantage of the first generation of PINNs is that they usually have limited accuracy even with many training points. Here, we propose a new method, gradient-enhanced physics-informed neural networks (gPINNs), for improving the accuracy of PINNs. gPINNs leverage gradient information of the PDE residual and embed the gradient into the loss function. We tested gPINNs extensively and demonstrated the effectiveness of gPINNs in both forward and inverse PDE problems. Our numerical results show that gPINN performs better than PINN with fewer training points. Additionally, we combined gPINN with the method of residual-based adaptive refinement (RAR), a method for improving the distribution of training points adaptively during training, to further improve the performance of gPINN, especially in PDEs with solutions that have steep gradients.

42 ENGINEERING↗
DOE OSTI2025

Application of physics-informed neural networks (PINNs) solution to coupled thermal and hydraulic processes in silty sands

Abstract The accurate modeling of water and heat transport in soils is crucial for both geo-environmental and geothermal engineering. Traditional modeling methods are problematic because they require well-defined boundaries and initial conditions. Recently, physics-informed neural networks (PINNs), which incorporate partial differential equations (PDEs) to solve forward and inverse problems, have attracted increasing attention in machine learning research. In this study, we applied PINNs to tackle hydraulic and thermal transport coupling forward problems in silty sands. A fully connected deep neural network was utilized for training. This neural network model leverages automatic differentiation to apply the governing equations as constraints, based on the mathematical approximations established by the neural network itself. We conducted forward problems and compared the solutions derived from PINNs with those from Finite Element Method (FEM) simulations. The forward problem results demonstrate the PINNs model’s capability in predicting hydraulic transport, heat transport, and thermal–hydraulic coupling in silty sands under various boundary conditions. The PINNs exhibited great performance in simulating the thermal–hydraulic coupling problem. The accuracy of the PINNs solutions shows its potential for simulation in geotechnical engineering.

Feng, Yuan↗
DOE OSTI2022

Learning stochastic dynamics with statistics-informed neural network

We introduce a machine-learning framework named statistics-informed neural network (SINN) for learning stochastic dynamics from data. This new architecture was theoretically inspired by a universal approximation theorem for stochastic systems, which we introduce in this paper, and the projection-operator formalism for stochastic modeling. Here, we devise mechanisms for training the neural network model to reproduce the correct statistical behavior of a target stochastic process. Numerical simulation results demonstrate that a well-trained SINN can reliably approximate both Markovian and non-Markovian stochastic dynamics. We demonstrate the applicability of SINN to coarse-graining problems and the modeling of transition dynamics. Furthermore, we show that the obtained reduced-order model can be trained on temporally coarse-grained data and hence is well suited for rare-event simulations.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

NeuroCoreX: An Open-Source FPGA-Based Spiking Neural Network Emulator with On-Chip Learning

Spiking Neural Networks (SNNs) are computational models inspired by the event-driven communication and connectivity patterns of biological neural circuits. They enable high energy efficiency and natural support for diverse architectures ranging from layered networks to small-world and graphstructured topologies. In this work, we introduce NeuroCoreX, an open-source, FPGA-based spiking neural network emulator that provides real-time, on-chip learning and flexible network organization. NeuroCoreX supports both feedforward sensory inputs streamed directly from sensors or PCs via UART and recurrent on-chip connectivity, enabling simultaneous processing and learning from external stimuli and internal network dynamics-capabilities rarely available in existing FPGA SNN platforms. The system implements a Leaky Integrate-and-Fire (LIF) neuron model with current-based synapses and supports pair-based STDP learning on both feedforward and recurrent synapses. A lightweight Python interface enables interactive configuration, live monitoring, weight read-back, and experiment control. Importantly, NeuroCoreX is tightly integrated with the SuperNeuroMAT simulator, allowing SNN models to be transferred seamlessly from software to hardware for hardware-in-the-loop development. By combining real-time plasticity, flexible connectivity, and an open-source VHDL implementation, NeuroCoreX provides an extensible and accessible platform for neuromorphic research, algorithm-hardware co-design, and energy-efficient edge intelligence.

Gautam, Ashish [ORNL]↗
NASA NTRSDate not supplied

Li-ion Battery Aging with Hybrid Physics-Informed Neural Networks and Fleet-wide Data

In this work, we propose a hybrid model for Li-ion battery discharge and aging prediction that leverages fleet-wide data to predict future capacity drops.The model is built upon an hybrid approach merging physics-based and empirical equations, as well as neural network models in a recurrent neural network cell. The hybrid physics-informed neural network can predict voltage discharge cycles given the loading profile, and estimate the used capacity of the battery under random-loading conditions by tracking aging parameters connected to the residual capacity of the battery. By merging information on the battery aging parameters with existing fleet-wide aging data, the model can predict the future residual capacity of the battery that is being monitored, and therefore enable predictions of voltage discharge curves far ahead in the battery life cycle. We validated the approach using the NASA Prognostics Data Repository Battery data-set, which contains experimental data on Li-ion batteries discharged at random loading conditions in a controlled environment. The approach also allows the identification of discrepancies between the battery aging trend and the trend observed at the fleet level, so that batteries behaving differently from the rest of the fleet can be subject to closer monitoring and further testing to refine predictions.

PINN↗