Modeling net ecosystem exchange of CO2 with gated recurrent unit neural networks
Not provided.
Agriculture↗
SEARCH · Engineering Papers
Results for “recurrence”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Secure authentication using recurrent neural networks
A computer-implemented method of user authentication is provided. The method comprises combining, by a computer system, a user recurrent neural network with a system recurrent neural network to form a unique combined recurrent neural network. The user recurrent neural network is configured to generate a unique user key, and the system recurrent neural network is configured to generate a system key. The computer system inputs a predetermined input into the combined recurrent neural network, and the combined recurrent neural network generates a unique combined key from the input, wherein the combined key differs from both the user key and system key. The computer system then associates the combined key with a unique access authorization to authenticate a user.
Aimone, James Bradley↗
Towards a more general understanding of the algorithmic utility of recurrent connections
Lateral and recurrent connections are ubiquitous in biological neural circuits. Yet while the strong computational abilities of feedforward networks have been extensively studied, our understanding of the role and advantages of recurrent computations that might explain their prevalence remains an important open challenge. Foundational studies by Minsky and Roelfsema argued that computations that require propagation of global information for local computation to take place would particularly benefit from the sequential, parallel nature of processing in recurrent networks. Such “tag propagation” algorithms perform repeated, local propagation of information and were originally introduced in the context of detecting connectedness, a task that is challenging for feedforward networks. Here, we advance the understanding of the utility of lateral and recurrent computation by first performing a large-scale empirical study of neural architectures for the computation of connectedness to explore feedforward solutions more fully and establish robustly the importance of recurrent architectures. In addition, we highlight a tradeoff between computation time and performance and construct hybrid feedforward/recurrent models that perform well even in the presence of varying computational time limitations. We then generalize tag propagation architectures to propagating multiple interacting tags and demonstrate that these are efficient computational substrates for more general computations of connectedness by introducing and solving an abstracted biologically inspired decision-making task. Our work thus clarifies and expands the set of computational tasks that can be solved efficiently by recurrent computation, yielding hypotheses for structure in population activity that may be present in such tasks.
59 BASIC BIOLOGICAL SCIENCES↗
Ultra-low latency recurrent neural network inference on FPGAs for physics applications with hls4ml
Abstract Recurrent neural networks have been shown to be effective architectures for many tasks in high energy physics, and thus have been widely adopted. Their use in low-latency environments has, however, been limited as a result of the difficulties of implementing recurrent architectures on field-programmable gate arrays (FPGAs). In this paper we present an implementation of two types of recurrent neural network layers—long short-term memory and gated recurrent unit—within the hls4ml framework. We demonstrate that our implementation is capable of producing effective designs for both small and large models, and can be customized to meet specific design requirements for inference latencies and FPGA resources. We show the performance and synthesized designs for multiple neural networks, many of which are trained specifically for jet identification tasks at the CERN Large Hadron Collider.
97 MATHEMATICS AND COMPUTING↗
Path sampling of recurrent neural networks by incorporating known physics
Recurrent neural networks have seen widespread use in modeling dynamical systems in varied domains such as weather prediction, text prediction and several others. Often one wishes to supplement the experimentally observed dynamics with prior knowledge or intuition about the system. While the recurrent nature of these networks allows them to model arbitrarily long memories in the time series used in training, it makes it harder to impose prior knowledge or intuition through generic constraints. In this work, we present a path sampling approach based on principle of Maximum Caliber that allows us to include generic thermodynamic or kinetic constraints into recurrent neural networks. We show the method here for a widely used type of recurrent neural network known as long short-term memory network in the context of supplementing time series collected from different application domains. These include classical Molecular Dynamics of a protein and Monte Carlo simulations of an open quantum system continuously losing photons to the environment and displaying Rabi oscillations. Our method can be easily generalized to other generative artificial intelligence models and to generic time series in different areas of physical and social sciences, where one wishes to supplement limited data with intuition or theory based corrections.
59 BASIC BIOLOGICAL SCIENCES↗
Short-Term Forecasting of Thermostatic and Residential Loads Using Long Short-Term Memory Recurrent Neural Networks
Internet of Things (IoT) devices in smart grids enable intelligent energy management for grid managers and personalized energy services for consumers. Investigating a smart grid with IoT devices requires a simulation framework with IoT devices modeling. However, there lack comprehensive study on the modeling of IoT devices in smart grids. This paper investigates the IoT device modeling of a thermostatic load and implements the recurrent neural networks model for short-term load forecasting in this IoT-based thermostatic load. The recurrent neural network structure is leveraged to build a load forecasting model on temporal correlation. The temporal recurrent neural network layers including long short-term memory cells are employed to learn the data from both the simulation platform and New South Wales residential datasets. The simulation results are provided for demonstration.
electric load forecasting↗
Process prediction and detection of faults using probabilistic bidirectional recurrent neural networks on real plant data
Attaining Industry 4.0 for manufacturing operations requires advanced monitoring systems and real-time data analytics of plant data, among other topics. We propose a Probabilistic Bidirectional Recurrent Network (PBRN) for industrial process monitoring for the early detection of faults. The model is based on a Gated Recurrent Unit (GRU) neural network that allows the model to retain long-term dependencies between sensor data along a time horizon, hence learning the dynamic behavior of the process. To reduce the false-positive detection rate of the model, we compel the model to learn from a highly noisy sensor reading while outputting noise-free sensor outputs. The performance of the proposed model is compared to other data-driven statistical process monitoring schemes using real plant data from an industrial Air Separations Unit (ASU) containing noisy sensor readings. We show that the model can learn from noisy data without reducing its performance. Using two different fault cases, we demonstrate the model’s ability to carry out early fault detection with average false-positive rates of 2.9% and 4.9% for both fault cases. The missed detection rates are 0.1% and 0.2%, respectively.
32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
Response of soil nutrient pools and microbiomes to recurrent wildfire disturbance and varying burn severities in a mixed conifer forest
Wildfire is a pervasive disturbance in mixed-conifer forests, yet the relative influence of fire recurrence versus burn severity on soil biogeochemistry and microbial communities remains poorly quantified. We examined a natural gradient of fire history (0–3 prior fires) and burn severity (low–high) spanning 50 yr in a mixed-conifer ecosystem to assess how repeated fire shapes soil carbon (C) and nitrogen (N) pools, their isotopic signatures, mineral and particulate fractions, microbial community composition, carbon-use, CO₂ fluxes, and vegetation cover. Successive fires produced progressively higher bare-ground percentages and lower tree cover, which were tightly linked to declines in microbial diversity and reductions bulk %C, and %N. δ 13 C increased with fire frequency, indicating preferential loss of labile C through combustion or enhanced microbial oxidation, thereby explaining the observed net soil-C decline. Conversely, δ 15 N decreased and pH increased as tree density declined, reflecting altered N cycling and reduced acidification in post-fire soils. Fire recurrence, more than severity, corresponded with a marked shift in the bacterial community: for example, Xanthobacteraceae—key N-fixers and C-cyclers—diminished, while N-fixing Bacillaceae increased, underscoring the tightly coupled nature of soil nutrient dynamics and microbiome composition after repeated burns. Our results demonstrate that fire recurrence appears to be a stronger driver of post-fire soil ecosystem responses in this mixed-conifer forest, influencing both abiotic nutrient pools and the functional potential of the soil microbiome. These findings provide a more enhanced assessment and understanding to date of the biogeochemical consequences of repeated wildfire disturbance that can be used to inform management strategies aimed at preserving soil health in fire-prone landscapes.
54 ENVIRONMENTAL SCIENCES↗
Source-agnostic gravitational-wave detection with recurrent autoencoders
Abstract We present an application of anomaly detection techniques based on deep recurrent autoencoders (AEs) to the problem of detecting gravitational wave (GW) signals in laser interferometers. Trained on noise data, this class of algorithms could detect signals using an unsupervised strategy, i.e. without targeting a specific kind of source. We develop a custom architecture to analyze the data from two interferometers. We compare the obtained performance to that obtained with other AE architectures and with a convolutional classifier. The unsupervised nature of the proposed strategy comes with a cost in terms of accuracy, when compared to more traditional supervised techniques. On the other hand, there is a qualitative gain in generalizing the experimental sensitivity beyond the ensemble of pre-computed signal templates. The recurrent AE outperforms other AEs based on different architectures. The class of recurrent AEs presented in this paper could complement the search strategy employed for GW detection and extend the discovery reach of the ongoing detection campaigns.
47 OTHER INSTRUMENTATION↗
Recurrent convolutional neural networks for modeling nonadiabatic dynamics of quantum-classical systems
Recurrent neural networks (RNNs) have recently been extensively applied to model the time evolution in fluid dynamics, weather predictions, and even chaotic systems due to their ability to capture temporal dependencies and sequential patterns in data. Here we present an RNN model based on convolutional neural networks for modeling the nonlinear nonadiabatic dynamics of hybrid quantum-classical systems. The dynamical evolution of the hybrid systems is governed by equations of motion for classical degrees of freedom and von Neumann equation for electrons. The Physics-Aware Recurrent Convolution (PARC) neural network structure incorporates a differentiator-integrator architecture that inductively models the spatiotemporal dynamics of generic physical systems. Here, we apply our RNN approach to learn the space-time evolution of a one-dimensional semiclassical Holstein model after an interaction quench. For shallow quenches (small changes in electron-lattice coupling), the deterministic dynamics can be accurately captured using a single-CNN-based recurrent network. In contrast, deep quenches induce chaotic evolution, making long-term trajectory prediction significantly more challenging. Nonetheless, we demonstrate that the PARC-CNN architecture can effectively learn the statistical climate of the Holstein model under deep-quench conditions.
Holstein model↗
Spatial-Temporal Recurrent Graph Neural Networks for Fault Diagnostics in Power Distribution Systems
Fault diagnostics are extremely important to decide proper actions toward fault isolation and system restoration. The growing integration of inverter-based distributed energy resources imposes strong influences on fault detection using traditional overcurrent relays. This paper utilizes emerging graph learning techniques to build new temporal recurrent graph neural network models for fault diagnostics. The temporal recurrent graph neural network structures can extract the spatial-temporal features from data of voltage measurement units installed at the critical buses. From these features, fault event detection, fault type/phase classification, and fault location are performed. Compared with previous works, the proposed temporal recurrent graph neural networks provide a better generalization for fault diagnostics. Moreover, the proposed scheme retrieves the voltage signals instead of current signals so that there is no need to install relays at all lines of the distribution system. Therefore, the proposed scheme is generalizable and not limited by the number of relays installed. The effectiveness of the proposed method is comprehensively evaluated on the Potsdam microgrid and IEEE 123-node system in comparison with other neural network structures.
24 POWER TRANSMISSION AND DISTRIBUTION↗
Exceedance Probabilities and Recurrence Intervals for Extended Power Outages in the United States
Power outages cause significant economic and societal impacts. An increasing likelihood of extreme weather coupled with aging grid infrastructure is leading to a higher prevalence of extended power outages, which can leave customers without power for multiple days or even weeks. Planners at the facility, local, state, and federal levels are interested in resilience solutions to reduce the impacts of extended power outages. Resilience solutions—such as installing backup systems, integrating microgrid solutions, weatherizing buildings, and hardening distribution and transmission components—can reduce the consequences of extended power outages, but these solutions come with increased capital costs. Conducting cost-benefit analyses is important for determining which steps to take to mitigate the impact of power outages without investing in ineffective and cost-prohibitive resilience solutions. The expected benefits of resilience investments depend on the frequency of power outages of various durations, particularly extended outages lasting several hours, days, or weeks. A significant barrier to resilience planning is the lack of publicly available data on the frequency and duration of extended power outages. The absence of outage duration information severely limits the ability to conduct quantitative cost-benefit analyses of resilience investments. This report provides estimates of recurrence intervals and conditional exceedance probabilities for major power outages by U.S. region between 2015 and 2021. Additionally, we provide estimates for grid management, particularly outages caused by California’s public safety power shutoffs (PSPS), and for natural outages caused by major hurricanes. Outage recurrence intervals are the average number of years between outage events, and conditional exceedance probabilities are the likelihoods that a customer who experiences a major power outage will experience an outage exceeding a given duration. Major outage events are those that affect 10,000 or more customers, as defined by the U.S. Department of Energy’s (DOE’s) Electric Emergency Incident and Disturbance Report, called OE-417 (DOE 2020). These results can be applied to determine the likelihood of experiencing long-duration outages, which can be integrated into cost-benefit analyses of resilience solutions and broader energy resilience studies. We developed a methodology to estimate customer outage durations for extended outage events using publicly available data on customer outages. Shorter-duration power outages are relatively common, with customers experiencing on average more than one power outage each year lasting fewer than 12 hours. An outage event lasting between 1 day and 1 week is expected to occur between once every 16 years and once every 42 years, depending on the region, with an average recurrence rate of once every 32 years across the contiguous United States.
24 POWER TRANSMISSION AND DISTRIBUTION↗
Physics-Informed Recurrent Neural Networks to Predict Reactor Operations of the AGN-201 Nuclear Reactor
4 page paper submitted to ANS Student conference. Summary of paper similar to the following abstract: The ability to predict how a reactor will operate, understand when anomalous conditions arise, and ensure a reactor is being operated as expected is crucial for deploying new nuclear facilities. Digital twins serve as a unique solution to recognizing reactor behavior; however, they require data to be useful. For next-generation reactors, this data may not currently be available. To explore how synthetic physics-informed reactor data can be used to predict reactor operations, a recurrent neural network was implemented for the Idaho State University AGN-201 digital twin. The goal of this work is to determine how synthetic data can be used to train a recurrent neural network model for predicting the reactor power of the AGN-201. The recurrent neural network was validated using both synthetic and real operational data. We envision this approach will help bridge the gap between the virtual and physical sides of a digital twin, where reactor physics models based on as-built data can be corrected for actual operating parameters to ensure the virtual model mirrors reality.
98 NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL P↗
Monitoring of Liquid Metal Reactor Heater Zones with Recurrent Neural Network Learning of Temperature Time Series
Advanced high-temperature fluid reactors (ARs), such as sodium fast reactors (SFRs) and molten salt cooled reactors (MSCRs) utilize high-temperature fluids at ambient pressure. To melt the fluid during reactor startup and prevent fluid freezing during cooldown, the thermal–hydraulic systems of such ARs include heater zones consisting of specific heaters with controllers, temperature sensors, and thermal insulation. The failure of heater zones due to insulation material degradation or improper installation, resulting in parasitic heat losses, can lead to fluid freezing. The detection of faults using a heat-transfer model is difficult because of a lack of knowledge of the experimental details. Data-driven machine learning of heater zone temperature time series offers a viable alternative. In this study, we benchmarked the performance of recurrent neural networks (RNNs) in an analysis of heat-up transient temperature time series of heater zones installed on a liquid sodium vessel. The RNN models include long short-term memory (LSTM) and gated recurrent unit (GRU) networks, as well as their bi-directional variants, BiLSTM and BiGRU. Anomalous temperature points were designated using a percentile-based threshold applied to residual fluctuations in the detrended temperature time series. Additionally, the impact of the exponentially weighted moving average (EWMA) method on detection accuracy was examined. The RNN models’ performance was assessed using precision, recall, and F 1 score metrics. Results demonstrated that RNN models effectively detect anomalies in temperature time series with the best models for each heater zone achieving F 1 scores of over 93%. To explain the variations in RNN model performance across different heater zones, we used Kullback–Leibler (KL) divergence to quantify the relative entropy between training and testing data, and the Detrended Fluctuation Analysis (DFA) to assess long-range temporal correlations. For datasets with strong long-range correlations and minimal relative entropy between training and testing data, GRU is the best-performing model. When the data exhibits weaker long-term correlations and a significant relative entropy between training and testing distributions, BiGRU shows the best performance. For the data sets with intermediate values of both KL divergence and DFA, the best performance is obtained with LSTM and BiLSTM, respectively.
gated recurrent unit↗
Time-warping invariant quantum recurrent neural networks via quantum-classical adaptive gating
Adaptive gating plays a key role in temporal data processing via classical recurrent neural networks (RNNs), as it facilitates retention of past information necessary to predict the future, providing a mechanism that preserves invariance to time warping transformations. This paper builds on quantum RNNs (QRNNs), a dynamic model with quantum memory, to introduce a novel class of temporal data processing quantum models that preserve invariance to time-warping transformations of the (classical) input-output sequences. The model, referred to as time warping-invariant QRNN (TWI-QRNN), augments a QRNN with a quantum–classical adaptive gating mechanism that chooses whether to apply a parameterized unitary transformation at each time step as a function of the past samples of the input sequence via a classical recurrent model. The TWI-QRNN model class is derived from first principles, and its capacity to successfully implement time-warping transformations is experimentally demonstrated on examples with classical or quantum dynamics.
97 MATHEMATICS AND COMPUTING↗
NCAPH drives breast cancer progression and identifies a gene signature that predicts luminal a tumour recurrence
Luminal A tumours generally have a favourable prognosis but possess the highest 10-year recurrence risk among breast cancers. Additionally, a quarter of the recurrence cases occur within 5 years post-diagnosis. Identifying such patients is crucial as long-term relapsers could benefit from extended hormone therapy, while early relapsers might require more aggressive treatment. We conducted a study to explore non-structural chromosome maintenance condensin I complex subunit H’s (NCAPH) role in luminal A breast cancer pathogenesis, both in vitro and in vivo, aiming to identify an intratumoural gene expression signature, with a focus on elevated NCAPH levels, as a potential marker for unfavourable progression. Our analysis included transgenic mouse models overexpressing NCAPH and a genetically diverse mouse cohort generated by backcrossing. A least absolute shrinkage and selection operator (LASSO) multivariate regression analysis was performed on transcripts associated with elevated intratumoural NCAPH levels. We found that NCAPH contributes to adverse luminal A breast cancer progression. The intratumoural gene expression signature associated with elevated NCAPH levels emerged as a potential risk identifier. Transgenic mice overexpressing NCAPH developed breast tumours with extended latency, and in Mouse Mammary Tumor Virus (MMTV)-NCAPH ErbB2 double-transgenic mice, luminal tumours showed increased aggressiveness. High intratumoural Ncaph levels correlated with worse breast cancer outcome and subpar chemotherapy response. A 10-gene risk score, termed Gene Signature for Luminal A 10 (GSLA10), was derived from the LASSO analysis, correlating with adverse luminal A breast cancer progression. The GSLA10 signature outperformed the Oncotype DX signature in discerning tumours with unfavourable outcomes, previously categorised as luminal A by Prediction Analysis of Microarray 50 (PAM50) across three independent human cohorts. This new signature holds promise for identifying luminal A tumour patients with adverse prognosis, aiding in the development of personalised treatment strategies to significantly improve patient outcomes.
60 APPLIED LIFE SCIENCES↗
Time series anomaly detection in power electronics signals with recurrent and ConvLSTM autoencoders
The anomalies in the high voltage converter modulator (HVCM) remain a major down time for the spallation neutron source facility, that delivers the most intense neutron beam in the world for scientific materials research. In this work, we propose neural network architectures based on Recurrent AutoEncoders (RAE) to detect anomalies ahead of time in the power signals coming from the HVCM. Bi-directional gated recurrent unit, bi-directional long-short term memory (LSTM), and convolutional LSTM (ConvLSTM) are developed, trained, and tested using real experimental signals from the HVCM module. The results show a good performance of the proposed RAE models, achieving precision up to 91%, recall up to 88%, false omission rate as low as 20% (i.e. 80% of the anomalies were detected), and area under the ROC curve up to 0.9. The three RAE models provide very comparable performance, with LSTM showing slightly better performance than GRU and ConvLSTM. The RAE models are benchmarked against other anomaly detection methods, including isolation forest, support vector machine, local outlier factor, feedforward and convolutional autoencoders, and others; showing a better performance. Here, the results of this study demonstrate the promising potential of RAE in anomaly detection for real-world power systems, and for increasing the reliability of the HVCM modules in the spallation neutron source.
42 ENGINEERING↗