SEARCH · Engineering Papers
Results for “Recurrent neural networks”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Wake State Estimation of Downwind Turbines using Recurrent Neural Networks.
Abstract not provided.
Enhancing the Operational Resilience of Advanced Reactors with Digital Twins by Recurrent Neural Networks
Because of a lack of operation data during abnormal and accident scenarios, along with the existence of uncertainty in the evaluation model for transient and accident analysis, the established abnormal and emergency operating procedures can be biased in characterizing the reactor states and ensuring operational resilience. To improve state awareness and ensure operational flexibility for minimizing effects on the system due to anomaly, digital twin (DT) technology is suggested to support operator's decision-making by effectively extracting and using knowledge of the current and future plant states from the knowledge base. To demonstrate DT's capability for recovering the complete states of reactors and for predicting the future reactor behaviors, this paper develops and assesses both the diagnosis and prognosis DTs in a nearly autonomous management and control system for an Experimental Breeder Reactor-II simulator during different loss-of-flow scenarios.
Data-Driven Compact Modeling of Bipolar Junction Transistors with Recurrent Neural Networks.
Abstract not provided.
Identification and localization of rotational spectra using recurrent neural networks
A method of identifying molecular parameters in a complex mixture may include receiving a set of combined transition frequencies and analyzing the set of combined transition frequencies using a first trained artificial neural network to generate a plurality of separated transition frequency sets. Each of the plurality of separated frequency sets may be analyzed using a second trained artificial neural network to generate a respective set of estimated spectral parameters. The method may include identifying a set of molecular parameters corresponding to the set of separated transition frequencies.
Skip-Connected Self-Recurrent Spiking Neural Networks with Joint Intrinsic Parameter and Synaptic Weight Training
As an important class of spiking neural networks (SNNs), recurrent spiking neural networks (RSNNs) possess great computational power and have been widely used for processing sequential data like audio and text. However, most RSNNs suffer from two problems. 1. Due to the lack of architectural guidance, random recurrent connectivity is often adopted, which does not guarantee good performance. 2. Training of RSNNs is in general challenging, bottlenecking achievable model accuracy. To address these problems, we propose a new type of RSNNs called Skip-Connected Self-Recurrent SNNs (ScSr-SNNs). Recurrence in ScSr-SNNs is introduced in a stereotyped manner by adding self-recurrent connections to spiking neurons. The SNNs with self-recurrent connections can realize recurrent behaviors similar to those of more complex RSNNs while the error gradients can be more straightforwardly calculated due to the mostly feedforward nature of the network. The network dynamics is enriched by skip connections between nonadjacent layers. Moreover, we propose a new backpropagation (BP) method called backpropagated intrinsic plasticity (BIP) to further boost the performance of ScSr-SNNs by training intrinsic model parameters. Unlike standard intrinsic plasticity rules that adjust the neuron's intrinsic parameters according to neuronal activity, the proposed BIP method optimizes intrinsic parameters based on the backpropagated error gradient of a well-defined global loss function in addition to synaptic weight training. Here, based on challenging speech, neuromorphic speech, and neuromorphic image datasets, the proposed ScSr-SNNs can boost performance by up to 2.85% compared with other types of RSNNs trained by state-of-the-art BP methods.
Modeling net ecosystem exchange of CO2 with gated recurrent unit neural networks
Not provided.
Training neural networks using Metropolis Monte Carlo and an adaptive variant
Abstract We examine the zero-temperature Metropolis Monte Carlo (MC) algorithm as a tool for training a neural network by minimizing a loss function. We find that, as expected on theoretical grounds and shown empirically by other authors, Metropolis MC can train a neural net with an accuracy comparable to that of gradient descent (GD), if not necessarily as quickly. The Metropolis algorithm does not fail automatically when the number of parameters of a neural network is large. It can fail when a neural network’s structure or neuron activations are strongly heterogenous, and we introduce an adaptive Monte Carlo algorithm (aMC) to overcome these limitations. The intrinsic stochasticity and numerical stability of the MC method allow aMC to train deep neural networks and recurrent neural networks in which the gradient is too small or too large to allow training by GD. MC methods offer a complement to gradient-based methods for training neural networks, allowing access to a distinct set of network architectures and principles.
Data-Driven Model Predictive Control for Temperature Management of Heat Pipe Microreactor
A data-driven model predictive control (MPC) was developed to enable the self-regulating capability of heat pipe (HP) nuclear microreactors. The MPC can proactively respond to potential disturbances of HP microreactors using three approaches for system identifications: linear state-space model, feedforward neural network, and recurrent neural networks with long short-term memory units. We present numerical results of data-driven MPCs to control the temperatures of selected HPs in a 37-HP test article. Our results show qualitatively that all data-driven MPCs produced similar control actions, while quantitatively, with artificial neural nets (especially feedforward neural nets), MPC can better follow drastic changes in setpoints with small errors.
Data-Driven Model Predictive Control for Temperature Management of Heat Pipe Microreactor
To enable the self-regulating capability of heat pipe (HP) microreactors, an anticipatory control strategy through model predictive control (MPC) could proactively respond to potential disturbances and deviations in operating setpoints. However, a key factor prohibiting the widespread adoption of MPCs in nuclear applications is the effort and computational costs associated with learning and calibrating first-principles-based process models when the target system is complex and when there are gaps between modeled and target reactor systems. In this paper, we demonstrate data-driven MPC using three approaches for modeling the system dynamics, including a linear state-space model, feedforward neural network, and recurrent neural networks long short-term memory. We present the development and validation process of each model and compare the performance of data-driven MPCs in controlling the temperatures of selected HPs at the evaporator and condenser regions in a 37-HP-monolith system. Our results show that, qualitatively, all data-driven MPCs are producing similar control actions, while quantitatively, with artificial neural nets (especially feedforward neural nets), MPC can better follow drastic changes in setpoints with smallest errors.
Recurrent convolutional neural networks for modeling nonadiabatic dynamics of quantum-classical systems
Recurrent neural networks (RNNs) have recently been extensively applied to model the time evolution in fluid dynamics, weather predictions, and even chaotic systems due to their ability to capture temporal dependencies and sequential patterns in data. Here we present an RNN model based on convolutional neural networks for modeling the nonlinear nonadiabatic dynamics of hybrid quantum-classical systems. The dynamical evolution of the hybrid systems is governed by equations of motion for classical degrees of freedom and von Neumann equation for electrons. The Physics-Aware Recurrent Convolution (PARC) neural network structure incorporates a differentiator-integrator architecture that inductively models the spatiotemporal dynamics of generic physical systems. Here, we apply our RNN approach to learn the space-time evolution of a one-dimensional semiclassical Holstein model after an interaction quench. For shallow quenches (small changes in electron-lattice coupling), the deterministic dynamics can be accurately captured using a single-CNN-based recurrent network. In contrast, deep quenches induce chaotic evolution, making long-term trajectory prediction significantly more challenging. Nonetheless, we demonstrate that the PARC-CNN architecture can effectively learn the statistical climate of the Holstein model under deep-quench conditions.
Graph-Based Prediction of Spatio-Temporal Vaccine Hesitancy From Insurance Claims Data
Growing vaccine hesitancy is contributing to the decline in immunization rates for highly contagious, vaccine-preventable childhood diseases. Therefore, there has been a significant interest in understanding how hesitancy is spreading at higher spatio-temporal resolutions, enabling more targeted interventions. Motivated by this, we study the problem of prediction of vaccine hesitancy at the ZIP Code level, referred to as the VaxHesitancy problem. A significant challenge for this problem is the lack of high-resolution data that indicates hesitancy. Here, we develop a hybrid VaxHesSTL framework that combines a Graph Neural Network (GNN) and a Recurrent Neural Network (RNN) to address the VaxHesitancy problem. The GNN uses a ZIP Code-level network to capture spatial signals from neighboring areas, while the RNN models the temporal dynamics present in the data. We train and evaluate VaxHesSTL using a large dataset, namely the All-Payer Claims Databases (APCD), for Virginia, consisting of insurance claims from over five million individuals for six years. We find that an aggregated contact network or graph, developed from a detailed activity-based population network, plays an important role in the performance of VaxHesSTL, compared to graph models based solely on spatial proximity. Experiments demonstrate that VaxHesSTL outperforms a range of state-of-the-art baselines, which rely solely on historical time series data without accounting for spatial relationships. Since hesitancy data at higher spatial resolution is often unavailable or hard to get, we incorporate an active learning approach with our VaxHesSTL framework to optimize the training set without compromising the prediction performance. We find that hesitancy data for only 18% of ZIP Codes selected by active learning allows us to forecast hesitancy for all the ZIP Codes in the Virginia.
Recurrent neural network-based multiaxial plasticity model with regularization for physics-informed constraints
We report a recurrent neural network (RNN) based model is developed as a surrogate to predict nonlinear plastic response under multiaxial loading. The RNN-based model is trained and tested on stress versus strain curves generated using a numerical solution based on the classical radial return method. Besides simply learning the basic constitutive relationship, a novel approach is taken to enforce certain physical conditions. Specifically, regularization is employed to maintain non-negative plastic power density throughout the loading history thereby ensuring monotonically increasing plastic work and thermodynamic consistency. Enforcing physics in this manner permits coupling of the data-driven RNN approach with physics-based knowledge and laws. This has the effect of reducing the necessary amount of data and ensuring known physical laws are not violated. Since, once trained, the model need not perform the expensive task of solving nonlinear equations, its efficiency is orders of magnitude greater than its numerical counterpart. The RNN-based model has been trained on varied sets of data and the accuracy on test datasets validated. The developed model is general and robust and has widespread application such as in the simulation of metal forming, large scale plasticity, and part life prediction.
Real-Time event reconstruction for Nuclear Physics Experiments using Artificial Intelligence
Charged track reconstruction is a critical task in nuclear physics experiments, enabling the identification and analysis of particles produced in high-energy collisions. Machine learning (ML) has emerged as a powerful tool for this purpose, addressing the challenges posed by complex detector geometries, high event multiplicities, and noisy data. Traditional methods rely on pattern recognition algorithms like the Kalman filter, but ML techniques, such as neural networks, graph neural networks (GNNs), and recurrent neural networks (RNNs), offer improved accuracy and scalability. By learning from simulated and real detector data, ML models can identify and classify tracks, predict trajectories, and handle ambiguities caused by overlapping or missing hits. Moreover, ML-based approaches can process data in near-real-time, enhancing the efficiency of experiments at large-scale facilities like the Large Hadron Collider (LHC) and Jefferson Lab (JLAB). As detector technologies and computational resources evolve, ML-driven charged track reconstruction continues to push the boundaries of precision and discovery in nuclear physics. In these proceedings, we highlight advancements in charged track identification leveraging Artificial Intelligence within the CLAS12 detector, achieving a notable enhancement in experimental statistics compared to traditional methods. Additionally, we showcase real-time event reconstruction capabilities, including the inference of charged particle properties, such as momentum, direction, and species identification, at speeds matching data acquisition rates. These innovations enable the extraction of physics observables directly from the experiment in real-time.
Integrating static PRA information with risk informed safety margin characterization (RISMC) simulation methods
The overall objective of the project was to develop a computationally feasible and user-friendly mechanized process to integrate traditional probabilistic risk assessment (PRA) and dynamic PRA (DPRA) results. Starting with the systematic identification of items in an existing PRA that need dynamic augmentation, the project used a generic 4-loop pressurized reactor (PWR) and 3-loop PWR as example plants. Station blackout (SBO) and large break loss of coolant accident SBLOCA) were selected as the example initiating events. Using the traditional event-tree (ET)/fault-tree (FT) methodology augmented by dynamic evet tree approach, the potential consequences of the initiating events were simulated with RELAP-3D and MELCOR/RASCAL codes to cover Level 1 through Level 3 of PRA. RAVEN and ADAPT software were used to generate Level 1 simulations with RELAP-3D and Level 2/3 simulations with MELCOR (Level 2)/RASCAL (Level 3), respectively. Example branching conditions (BCs) for SBO included AC power recovery time, valve repair failure time, reactor coolant pump leak time/break size and emergency power supply duration to a total of 9. Example BCs for LOCA included off-site power recovery time, diesel generator power recovery time, auxiliary feed water system operation time, safety relief valve failure to open upon demand, reactor coolant pump seal break time and size to a total of 21. Each RELAP-3D simulation (9,587 scenarios) was labelled OK or Core Damage based on the maximum allowed peak clad temperature (2,100oF). Each MELCOR simulation (4610 scenarios) was labeled as Bin over 10rem or Bin 0-10rem based on the dose at the site boundary. The scenarios were clustered based on the criteria above using the mean shift methodology. Classical PRA (CPRA) and DPRA results were compared to identify the ET sequences that need DPRA augmentation. Several approaches were proposed for the incorporation of these sequences into CPRA using clustering with the mean shift methodology, restructuring the CPRA ETs by adding new BCs/sequences, and using the concept of a limit surface. Procedures for decision making regarding the possible consequences of an initiating event (e.g. core damage or not, site evacuation or not) were developed using a convolutional neural network (CNN), a recurrent neural network (RNN) and a transformer neural network (TNN). The project has led to two PhD degrees, three archival journal papers and five refereed conference proceedings.
STFM: Accurate Spatio-Temporal Fusion Model for Weather Forecasting
Meteorological prediction is crucial for various sectors, including agriculture, navigation, daily life, disaster prevention, and scientific research. However, traditional numerical weather prediction (NWP) models are constrained by their high computational resource requirements, while the accuracy of deep learning models remains suboptimal. In response to these challenges, we propose a novel deep learning-based model, the Spatiotemporal Fusion Model (STFM), designed to enhance the accuracy of meteorological predictions. Our model leverages Fifth-Generation ECMWF Reanalysis (ERA5) data and introduces two key components: a spatiotemporal encoder module and a spatiotemporal fusion module. The spatiotemporal encoder integrates the strengths of convolutional neural networks (CNNs) and recurrent neural networks (RNNs), effectively capturing both spatial and temporal dependencies. Meanwhile, the spatiotemporal fusion module employs a dual attention mechanism, decomposing spatial attention into global static attention and channel dynamic attention. This approach ensures comprehensive extraction of spatial features from meteorological data. The combination of these modules significantly improves prediction performance. Experimental results demonstrate that STFM excels in extracting spatiotemporal features from reanalysis data, yielding predictions that closely align with observed values. In comparative studies, STFM outperformed other models, achieving a 7% improvement in ground and high-altitude temperature predictions, a 5% enhancement in the prediction of the u/v components of 10 m wind speed, and an increase in the accuracy of potential height and relative humidity predictions by 3% and 1%, respectively. This enhanced performance highlights STFM’s potential to advance the accuracy and reliability of meteorological forecasting.
Predicting lake surface water phosphorus dynamics using process-guided machine learning
Phosphorus (P) loading to lakes is degrading the quality and usability of water globally. Accurate predictions of lake P dynamics are needed to understand whole-ecosystem P budgets, as well as the consequences of changing lake P concentrations for water quality. However, complex biophysical processes within lakes, along with limited observational data, challenge our capacity to reproduce short-term lake dynamics needed for water quality predictions, as well as long-term dynamics needed to understand broad scale controls over lake P. Here we use an emerging paradigm in modeling, process-guided machine learning (PGML), to produce a phosphorus budget for Lake Mendota (Wisconsin, USA) and to accurately predict epilimnetic phosphorus over a time range of days to decades. In our implementation of PGML, which we term a Process-Guided Recurrent Neural Network (PGRNN), we combine a process-based model for lake P with a recurrent neural network, and then constrain the predictions with ecological principles. We test independently the process-based model, the recurrent neural network, and the PGRNN to evaluate the overall approach. The process-based model accounted for most of the observed pattern in lake P; however it missed the long-term trend in lake P and had the worst performance in predicting winter and summer P in surface waters. The root mean square error (RMSE) for the process-based model, the recurrent neural network, and the PGRNN was 33.0 μg P L -1 , 22.7 μg P L -1 , and 20.7 μg P L -1 , respectively. All models performed better during summer, with RMSE values for the three models (same order) equal to 14.3 μg P L -1 , 10.9 μg P L -1 , and 10.7 μg P L -1 . Although the PGRNN had only marginally better RMSE during summer, it had lower bias and reproduced long-term decreases in lake P missed by the other two models. For all seasons and all years, the recurrent neural network had better predictions than process alone, with root mean square error (RMSE) of 23.8 μg P L -1 and 28.0 μg P L -1 , respectively. The output of PGRNN indicated that new processes related to water temperature, thermal stratification, and long term changes in external loads are needed to improve the process model. By using ecological knowledge, as well as the information content of complex data, PGML shows promise as a technique for accurate prediction in messy, real-world ecological dynamics, while providing valuable information that can improve our understanding of process.
Spatial-Temporal Recurrent Graph Neural Networks for Fault Diagnostics in Power Distribution Systems
Fault diagnostics are extremely important to decide proper actions toward fault isolation and system restoration. The growing integration of inverter-based distributed energy resources imposes strong influences on fault detection using traditional overcurrent relays. This paper utilizes emerging graph learning techniques to build new temporal recurrent graph neural network models for fault diagnostics. The temporal recurrent graph neural network structures can extract the spatial-temporal features from data of voltage measurement units installed at the critical buses. From these features, fault event detection, fault type/phase classification, and fault location are performed. Compared with previous works, the proposed temporal recurrent graph neural networks provide a better generalization for fault diagnostics. Moreover, the proposed scheme retrieves the voltage signals instead of current signals so that there is no need to install relays at all lines of the distribution system. Therefore, the proposed scheme is generalizable and not limited by the number of relays installed. The effectiveness of the proposed method is comprehensively evaluated on the Potsdam microgrid and IEEE 123-node system in comparison with other neural network structures.