Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “recurrent neural networks”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Woven ceramic matrix composite surrogate model based on physics-informed recurrent neural network

A recurrent neural network (RNN) based surrogate model is developed to emulate the nonlinear constitutive behavior of woven ceramic matrix composites (CMCs) driven by matrix damage at multiple length scales. Physics-informed constraints are introduced into the surrogate model through regularization to ground the prediction in physics and improve its predictive capabilities. Training data is generated using the multiscale generalized method of cells (MSGMC) approach coupled with a matrix damage model. This coupling permits simulating the nonlinear behavior of woven CMCs based on constituent response at the micro-, meso-, and macroscales. The multiscale repeating unit cell is loaded under non-monotonic conditions including multiple load / unload cycles and tension / compression. The fiber volume fraction as well as the intra- and intertow void volume fractions are also varied in the generation of training data. Therefore, the RNN-based surrogate model is tasked with predicting, as a function of variable input strain sequence and fiber and void volume fractions, the resulting stress versus strain response while satisfying physical constraints such as positive semi-definiteness of the tangent stiffness matrix and linear elastic unloading. Further, the trained surrogate model effectively matches the stress versus strain response and successfully predicts the tangent modulus throughout the loading regime. Neural network based surrogate models can offer efficient alternatives to running computationally intensive multiscale material models to simulate the nonlinear response of large structural models. Therefore the presented work provides evidence towards the feasibility of developing, training, and running such models for CMCs with complex architectures, nonlinear multiaxial material response, and under non-monotonic loading conditions.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2022

Hybrid Recurrent Neural Network Modeling for Traffic Delay Prediction at Signalized Intersections Along an Urban Arterial

This paper studies the traffic delay prediction modeling for multiple signalized intersections along the Ala Moana Boulevard and Nimitz Highway in Hawaii. Several machine learning (ML) based approaches have been studied in the literature, and most of them focused on prediction accuracy rather than the end use of real-time control and implementation. These ML models tend to be very complex and non-linear in nature, making it challenging to achieve fast inferences and are computationally heavy for real-time signal control implementation. As such, in this paper, a simple yet accurate hybrid modeling method is proposed to predict traffic delay one-step ahead with the model made suitable for real-time implementation to control traffic flow. Since real-time road-side measurements are recorded in unstructured form, the paper also discusses other issues related to data extraction and the pre-processing process. Finally, a simple signal control loop is developed to demonstrate the proposed modeling approach, which has shown advantages in model accuracy and computation efficiency compared against several existing modeling methods.

42 ENGINEERING↗
DOE OSTI2022

Path sampling of recurrent neural networks by incorporating known physics

Recurrent neural networks have seen widespread use in modeling dynamical systems in varied domains such as weather prediction, text prediction and several others. Often one wishes to supplement the experimentally observed dynamics with prior knowledge or intuition about the system. While the recurrent nature of these networks allows them to model arbitrarily long memories in the time series used in training, it makes it harder to impose prior knowledge or intuition through generic constraints. In this work, we present a path sampling approach based on principle of Maximum Caliber that allows us to include generic thermodynamic or kinetic constraints into recurrent neural networks. We show the method here for a widely used type of recurrent neural network known as long short-term memory network in the context of supplementing time series collected from different application domains. These include classical Molecular Dynamics of a protein and Monte Carlo simulations of an open quantum system continuously losing photons to the environment and displaying Rabi oscillations. Our method can be easily generalized to other generative artificial intelligence models and to generic time series in different areas of physical and social sciences, where one wishes to supplement limited data with intuition or theory based corrections.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2020

Time-Sequenced Flow Field Prediction in an Optima Spark-Ignition Direct-Injection Engine Using Bidirectional Recurrent Neural Network (bi-RNN) with Long Short-Term Memory

To further improve the energy conversion efficiency of internal combustion engine, the transient and complex air flow movement inside the cylinder needs to be better understood and controlled. Although the in-cylinder flow fields are highly stochastic with strong cycle-to-cycle fluctuations, machine learning can still provide an efficient way to learn and regress the complex flow movement process inside the cylinder. In this work, a bidirectional recurrent neural network (bi-RNN) model with long short-term memory was applied to predict the in-cylinder flow fields at different time steps using training data from mull-cycle particle image velocimetry (PIV) measurements. To evaluate the agreement between the true and predicted flow fields, structure and magnitude comparison indices are calculated both globally and locally. The comparison results show that the bi-RNN model can accurately predict the bulk flow and vortex motions from early intake stroke to compression stroke. This work demonstrates that the machine learning model has the potential to predict the underlying dynamics of the interaction between in-cylinder flows and provides a reliable way to improve temporal resolution in PIV flow data to better reveal transient in-cylinder flow features.

Bi-RNN model↗
DOE OSTI2024

Secure authentication using recurrent neural networks

A computer-implemented method of user authentication is provided. The method comprises combining, by a computer system, a user recurrent neural network with a system recurrent neural network to form a unique combined recurrent neural network. The user recurrent neural network is configured to generate a unique user key, and the system recurrent neural network is configured to generate a system key. The computer system inputs a predetermined input into the combined recurrent neural network, and the combined recurrent neural network generates a unique combined key from the input, wherein the combined key differs from both the user key and system key. The computer system then associates the combined key with a unique access authorization to authenticate a user.

Aimone, James Bradley↗
DOE OSTI2023

Ultra-low latency recurrent neural network inference on FPGAs for physics applications with hls4ml

Abstract Recurrent neural networks have been shown to be effective architectures for many tasks in high energy physics, and thus have been widely adopted. Their use in low-latency environments has, however, been limited as a result of the difficulties of implementing recurrent architectures on field-programmable gate arrays (FPGAs). In this paper we present an implementation of two types of recurrent neural network layers—long short-term memory and gated recurrent unit—within the hls4ml framework. We demonstrate that our implementation is capable of producing effective designs for both small and large models, and can be customized to meet specific design requirements for inference latencies and FPGA resources. We show the performance and synthesized designs for multiple neural networks, many of which are trained specifically for jet identification tasks at the CERN Large Hadron Collider.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Short-Term Forecasting of Thermostatic and Residential Loads Using Long Short-Term Memory Recurrent Neural Networks

Internet of Things (IoT) devices in smart grids enable intelligent energy management for grid managers and personalized energy services for consumers. Investigating a smart grid with IoT devices requires a simulation framework with IoT devices modeling. However, there lack comprehensive study on the modeling of IoT devices in smart grids. This paper investigates the IoT device modeling of a thermostatic load and implements the recurrent neural networks model for short-term load forecasting in this IoT-based thermostatic load. The recurrent neural network structure is leveraged to build a load forecasting model on temporal correlation. The temporal recurrent neural network layers including long short-term memory cells are employed to learn the data from both the simulation platform and New South Wales residential datasets. The simulation results are provided for demonstration.

electric load forecasting↗
DOE OSTI2025

River Dissolved Oxygen Prediction Using Machine Learning Models and Wireless Sensor Measurements

Simultaneous flooding&heat and droughts&heat events can potentially destabilize hydro-meteorological conditions to deteriorate the water quality of Neches River. Machine learning (ML) models utilizing wireless sensor measurements have been applied to predict water quality and optimize various water management strategies. This study aims to develop ML models to predict dissolved oxygen (DO) prediction under various hydro-meteorological conditions and enhance water management decision-making. Wireless sensor measurements of DO, water temperature, sample depth, conductivity, turbidity, and pH, along with discharge from the United States Geological Survey stations, are collected for model inputs at the Pine Island Bayou C749 station (PIB-C749) and Neches River Saltwater Barrier (SWB). Multilayer perceptron neural networks, recurrent neural networks, long short-term memory (LSTM), and bidirectional LSTM (BiLSTM) with and without attention mechanism (AT) are tested to determine the best model, which is applied the rolling forecast method to predict 14-day DO. Traditional and recurrent transfer learning (TL and RTL) methods are adopted to overcome insufficient data at the SWB. The input feature importance analysis using the integrated gradients (IG) algorithm is applied to determine dominant inputs. The results show LSTM-based models are capable handling long sequential data. AT-BiLSTM and RTL-LSTM demonstrate the best performance at the PIB-C749 (RMSE=0.054) and the SWB (RMSE=0.028), respectively. TL and RTL methods significantly improve model performance at the SWB. DO, temperature, and pH show higher importance, consistent with hydrodynamics and water chemistry. Both best models are applied to predict 14-day DO and demonstrate reasonable performance for decision-making. Hydro-meteorological conditions of 2017 flood and 2012 drought events are simulated and reveal that possible hypoxia occurs after flooding due to increasing temperature and turbidity, and DO concentration decreases significantly under heat and drought conditions. In conclusion, LSTM-based models utilizing wireless sensor data can be a timely and effective approach to make appropriate decisions on water resource management.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

Deep Kronecker neural networks: A general framework for neural networks with adaptive activation functions

Here we propose a new type of neural networks, Kronecker neural networks (KNNs), that form a general framework for neural networks with adaptive activation functions. KNNs employ the Kronecker product, which provides an efficient way of constructing a very wide network while keeping the number of parameters low. Our theoretical analysis reveals that under suitable conditions, KNNs induce a faster decay of the loss than that by the feed-forward networks. This is also empirically verified through a set of computational examples. Furthermore, under certain technical assumptions, we establish global convergence of gradient descent for KNNs. As a specific case, we propose the Rowdy activation function that is designed to get rid of any saturation region by injecting sinusoidal fluctuations, which include trainable parameters. The proposed Rowdy activation function can be employed in any neural network architecture like feed-forward neural networks, Recurrent neural networks, Convolutional neural networks etc. The effectiveness of KNNs with Rowdy activation is demonstrated through various computational experiments including function approximation using feed-forward neural networks, solution inference of partial differential equations using the physics-informed neural networks, and standard deep learning benchmark problems using convolutional and fully-connected neural networks.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Deep convolutional neural networks for multi-scale time-series classification and application to tokamak disruption prediction using raw, high temporal resolution diagnostic data

In this paper we discuss recent advances in deep convolutional neural networks (CNN) for sequence learning, which allow identifying long-range, multi-scale phenomena in long sequences, such as those found in fusion plasmas. We point out several benefits of these deep CNN architectures, such as not requiring experts such as physicists to hand-craft input data features, the ability to capture longer range dependencies compared to the more common sequence neural networks (recurrent neural networks like long short-term memory (LSTM) networks), and the comparative computational efficiency. We apply this neural network architecture to the popular problem of disruption prediction in fusion energy tokamaks, utilizing raw data from a single diagnostic, the Electron Cyclotron Emission imaging (ECEi) diagnostic from the DIII-D tokamak. Initial results trained on a large ECEi dataset show promise, achieving an F 1 -score of ~91% on individual time-slices using only the ECEi data. This indicates the ECEi diagnostic by itself can be sensitive to a number of pre-disruption markers useful for predicting disruptions on timescales not only for mitigation but also avoidance. Future opportunities for utilizing these deep CNN architectures with fusion data are outlined, including impact of recent upgrades to the ECEi diagnostic.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2023

AI-Based EMT Dynamic Model of PV Systems

Several electromagnetic transient (EMT) dynamic modeling methods are available to model systems like photovoltaic (PV) plants, wind power plants, variable-speed drives, among others. The methods include: (a) physics-based models and (b) data-driven models. The physics-based dynamic models may include high-fidelity switched system model and average-value model that both require the control algorithms included in the models. However, manufacturers typically prefer to provide black-box models to avoid disclosing proprietary. One of the solutions to prevent disclosing control algorithms is the use of data-driven dynamic EMT models of PV systems. In this paper, data-driven dynamic EMT model based on artificial intelligence (AI) algorithms are presented. The AI algorithms evaluated include convolutional neural networks, recurrent neural networks, and nonlinear auto-regressive exogenous model. Automation in generating data and training these models is also discussed in this paper. The results generated by the best AI algorithms have been observed to be greater than 95 % accurate.

Debnath, Suman↗
DOE OSTI2025

Physics-Informed Recurrent Neural Networks to Predict Reactor Operations of the AGN-201 Nuclear Reactor

4 page paper submitted to ANS Student conference. Summary of paper similar to the following abstract: The ability to predict how a reactor will operate, understand when anomalous conditions arise, and ensure a reactor is being operated as expected is crucial for deploying new nuclear facilities. Digital twins serve as a unique solution to recognizing reactor behavior; however, they require data to be useful. For next-generation reactors, this data may not currently be available. To explore how synthetic physics-informed reactor data can be used to predict reactor operations, a recurrent neural network was implemented for the Idaho State University AGN-201 digital twin. The goal of this work is to determine how synthetic data can be used to train a recurrent neural network model for predicting the reactor power of the AGN-201. The recurrent neural network was validated using both synthetic and real operational data. We envision this approach will help bridge the gap between the virtual and physical sides of a digital twin, where reactor physics models based on as-built data can be corrected for actual operating parameters to ensure the virtual model mirrors reality.

98 NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL P↗
DOE OSTI2023

Video frame prediction of microbial growth with a recurrent neural network

The recent explosion of interest and advances in machine learning technologies has opened the door to new analytical capabilities in microbiology. Using experimental data such as images or videos, machine learning, in particular deep learning with neural networks, can be harnessed to provide insights and predictions for microbial populations. This paper presents such an application in which a Recurrent Neural Network (RNN) was used to perform prediction of microbial growth for a population of two Pseudomonas aeruginosa mutants. The RNN was trained on videos that were acquired previously using fluorescence microscopy and microfluidics. Of the 20 frames that make up each video, 10 were used as inputs to the network which outputs a prediction for the next 10 frames of the video. The accuracy of the network was evaluated by comparing the predicted frames to the original frames, as well as population curves and the number and size of individual colonies extracted from these frames. Overall, the growth predictions are found to be accurate in metrics such as image comparison, colony size, and total population. Yet, limitations exist due to the scarcity of available and comparable data in the literature, indicating a need for more studies. Both the successes and challenges of our approach are discussed.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2023

Time-warping invariant quantum recurrent neural networks via quantum-classical adaptive gating

Adaptive gating plays a key role in temporal data processing via classical recurrent neural networks (RNNs), as it facilitates retention of past information necessary to predict the future, providing a mechanism that preserves invariance to time warping transformations. This paper builds on quantum RNNs (QRNNs), a dynamic model with quantum memory, to introduce a novel class of temporal data processing quantum models that preserve invariance to time-warping transformations of the (classical) input-output sequences. The model, referred to as time warping-invariant QRNN (TWI-QRNN), augments a QRNN with a quantum–classical adaptive gating mechanism that chooses whether to apply a parameterized unitary transformation at each time step as a function of the past samples of the input sequence via a classical recurrent model. The TWI-QRNN model class is derived from first principles, and its capacity to successfully implement time-warping transformations is experimentally demonstrated on examples with classical or quantum dynamics.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Reduced-order modeling of advection-dominated systems with recurrent neural networks and convolutional autoencoders

A common strategy for the dimensionality reduction of nonlinear partial differential equations (PDEs) relies on the use of the proper orthogonal decomposition (POD) to identify a reduced subspace and the Galerkin projection for evolving dynamics in this reduced space. However, advection-dominated PDEs are represented poorly by this methodology since the process of truncation discards important interactions between higher-order modes during time evolution. In this study, we demonstrate that encoding using convolutional autoencoders (CAEs) followed by a reduced-space time evolution by recurrent neural networks overcomes this limitation effectively. We demonstrate that a truncated system of only two latent space dimensions can reproduce a sharp advecting shock profile for the viscous Burgers equation with very low viscosities, and a six-dimensional latent space can recreate the evolution of the inviscid shallow water equations. Additionally, the proposed framework is extended to a parametric reduced-order model by directly embedding parametric information into the latent space to detect trends in system evolution. Furthermore, our results show that these advection-dominated systems are more amenable to low-dimensional encoding and time evolution by a CAE and recurrent neural network combination than the POD-Galerkin technique.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Utah FORGE 6-3712: Probabilistic Estimation of Seismic Response Using Physics-Informed Recurrent Neural Networks - 2024 Annual Workshop Presentation

This is a presentation on the Probabilistic Estimation of Seismic Response Using Physics-Informed Recurrent Neural Networks by GTC Analytics, presented by Jesse Williams. This video slide presentation discusses the development of machine learning-based predictive tools to estimate the magnitude-frequency response of stimulation-induced seismicity. This presentation was featured in the Utah FORGE R&D Annual Workshop on August 15, 2024.

15 GEOTHERMAL ENERGY↗