Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Long Short-term Memory”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

266 records · Page 15
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Interlayer Exciton Polarons in Mesoscopic V 2 O 5 for Broadband Optoelectronic Synapses

Persistent photoconductivity and optoelectronic synaptic behavior are demonstrated in solution-processed mesoscopic α-phase vanadium pentoxide (V 2 O 5 ) thin films. First-principles simulations coupled with the two-site Holstein polaron hopping model show that vacancies at the terminal oxygen position lead to long recombination times because photoexcited electrons and holes reside on different layers separated by the van der Waals gap, forming a weakly coupled interlayer exciton polaron. Mid-gap polaronic states also significantly broaden the photoresponse of the films to span across visible and infrared wavelengths. By controlling the amplitude/intensity, duration, and/or number of optical pulses, tunable optoelectronic memory functions, such as short-term and long-term plasticity, are experimentally established in V 2 O 5 -based optoelectronic synapses. Furthermore, device fabrication was extended to mechanically flexible ultrathin glass substrates. Flexible optoelectronic synapses maintained high performance after 150 bending cycles.

Phan, Thanh Luan [National Renewable Energy Labora↗
DOE OSTI2026

Protonic nickelate device networks for spatiotemporal neuromorphic computing

Computation in biological neural circuits arises from the interplay of nonlinear temporal responses and spatially distributed dynamic network interactions. Replicating this richness in hardware has remained challenging, as most neuromorphic devices emulate only isolated neuron- or synapse-like functions. Here we introduce an integrated neuromorphic computing platform in which both nonlinear spatiotemporal processing and programmable memory are realized within a single perovskite nickelate material system. By engineering symmetric and asymmetric hydrogenated NdNiO 3 junction devices on the same wafer, we combine ultrafast, proton-mediated transient dynamics with stable multilevel resistance states. Networks of symmetric NdNiO 3 junctions exhibit emergent spatial interactions mediated by proton redistribution, while each node simultaneously provides short-term temporal memory, enabling nanosecond-scale operation with an energy cost of ~0.2 nJ per input. When interfaced with asymmetric output units serving as reconfigurable long-term weights, these networks allow both feature transformation and linear classification in the same material system. Leveraging these emergent interactions, the platform enables real-time pattern recognition and achieves high accuracy in spoken digit classification and early seizure detection, outperforming temporal-only or uncoupled architectures. These results position protonic nickelates as a compact, energy-efficient, CMOS-compatible platform that integrates processing and memory for scalable intelligent hardware.

Electrical and electronic engineering↗
DOE OSTI2024

Selective consolidation of learning and memory via recall-gated plasticity

In a variety of species and behavioral contexts, learning and memory formation recruits two neural systems, with initial plasticity in one system being consolidated into the other over time. Moreover, consolidation is known to be selective; that is, some experiences are more likely to be consolidated into long-term memory than others. Here, we propose and analyze a model that captures common computational principles underlying such phenomena. The key component of this model is a mechanism by which a long-term learning and memory system prioritizes the storage of synaptic changes that are consistent with prior updates to the short-term system. This mechanism, which we refer to as recall-gated consolidation, has the effect of shielding long-term memory from spurious synaptic changes, enabling it to focus on reliable signals in the environment. We describe neural circuit implementations of this model for different types of learning problems, including supervised learning, reinforcement learning, and autoassociative memory storage. These implementations involve synaptic plasticity rules modulated by factors such as prediction accuracy, decision confidence, or familiarity. We then develop an analytical theory of the learning and memory performance of the model, in comparison to alternatives relying only on synapse-local consolidation mechanisms. We find that recall-gated consolidation provides significant advantages, substantially amplifying the signal-to-noise ratio with which memories can be stored in noisy environments. We show that recall-gated consolidation gives rise to a number of phenomena that are present in behavioral learning paradigms, including spaced learning effects, task-dependent rates of consolidation, and differing neural representations in short- and long-term pathways.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2024

Selective consolidation of learning and memory via recall-gated plasticity

In a variety of species and behavioral contexts, learning and memory formation recruits two neural systems, with initial plasticity in one system being consolidated into the other over time. Moreover, consolidation is known to be selective; that is, some experiences are more likely to be consolidated into long-term memory than others. Here, we propose and analyze a model that captures common computational principles underlying such phenomena. The key component of this model is a mechanism by which a long-term learning and memory system prioritizes the storage of synaptic changes that are consistent with prior updates to the short-term system. This mechanism, which we refer to as recall-gated consolidation, has the effect of shielding long-term memory from spurious synaptic changes, enabling it to focus on reliable signals in the environment. We describe neural circuit implementations of this model for different types of learning problems, including supervised learning, reinforcement learning, and autoassociative memory storage. These implementations involve synaptic plasticity rules modulated by factors such as prediction accuracy, decision confidence, or familiarity. We then develop an analytical theory of the learning and memory performance of the model, in comparison to alternatives relying only on synapse-local consolidation mechanisms. We find that recall-gated consolidation provides significant advantages, substantially amplifying the signal-to-noise ratio with which memories can be stored in noisy environments. We show that recall-gated consolidation gives rise to a number of phenomena that are present in behavioral learning paradigms, including spaced learning effects, task-dependent rates of consolidation, and differing neural representations in short- and long-term pathways.

Lindsey, Jack W. (ORCID:0000000309307327)↗
DOE OSTI2020

Building thermal load prediction through shallow machine learning and deep learning

Building thermal load prediction informs the optimization of cooling plant and thermal energy storage. Physics-based prediction models of building thermal load are constrained by the model and input complexity. In this study, we developed 12 data-driven models (7 shallow learning, 2 deep learning, and 3 heuristic methods) to predict building thermal load and compared shallow machine learning and deep learning. The 12 prediction models were compared with the measured cooling demand. It was found XGBoost (Extreme Gradient Boost) and LSTM (Long Short Term Memory) provided the most accurate load prediction in the shallow and deep learning category, and both outperformed the best baseline model, which uses the previous day's data for prediction. Then, we discussed how the prediction horizon and input uncertainty would influence the load prediction accuracy. Major conclusions are twofold: first, LSTM performs well in short-term prediction (1 h ahead) but not in long term prediction (24 h ahead), because the sequential information becomes less relevant and accordingly not so useful when the prediction horizon is long. Second, the presence of weather forecast uncertainty deteriorates XGBoost's accuracy and favors LSTM, because the sequential information makes the model more robust to input uncertainty. Training the model with the uncertain rather than accurate weather data could enhance the model's robustness. Finally, our findings have two implications for practice. First, LSTM is recommended for short-term load prediction given that weather forecast uncertainty is unavoidable. Second, XGBoost is recommended for long term prediction, and the model should be trained with the presence of input uncertainty.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2025

Short-Term Electric Load Forecasting for a Residential Household in Alaska

Accurate short-term load forecasting at a fine scale is essential for demand response programs, peak shaving, and load-shedding strategies [1]. While traditionally, only aggregate short-term consumption data was available, advanced metering infrastructure (AMI) now provides data at the individual consumer level [1]. There is increasing interest in utilizing this data for short-term load forecasting (from an hour to a few days) to optimize grid operations. Electricity consumption in individual households is highly influenced by residents’ personal behaviors [2]. As a result, unlike aggregate loads, electrical power usage in single households often shows significant volatility, making meter-level load forecasting for individual users particularly challenging [3], [4]. Deep learning methods, with their strong ability to model nonlinear data, have become popular for improving the accuracy of household electricity consumption forecasting [4]. Notably, the Long ShortTerm Memory (LSTM) has attracted significant attention [5], [6].

42 ENGINEERING↗
DOE OSTI2022

Evidence for long-term potentiation in phospholipid membranes

Biological supramolecular assemblies, such as phospholipid bilayer membranes, have been used to demonstrate signal processing via short-term synaptic plasticity (STP) in the form of paired pulse facilitation and depression, emulating the brain’s efficiency and flexible cognitive capabilities. However, STP memory in lipid bilayers is volatile and cannot be stored or accessed over relevant periods of time, a key requirement for learning. Using droplet interface bilayers (DIBs) composed of lipids, water and hexadecane, and an electrical stimulation training protocol featuring repetitive sinusoidal voltage cycling, we show that DIBs displaying memcapacitive properties can also exhibit persistent synaptic plasticity in the form of long-term potentiation (LTP) associated with capacitive energy storage in the phospholipid bilayer. The time scales for the physical changes associated with the LTP range between minutes and hours, and are substantially longer than previous STP studies, where stored energy dissipated after only a few seconds. STP behavior is the result of reversible changes in bilayer area and thickness. On the other hand, LTP is the result of additional molecular and structural changes to the zwitterionic lipid headgroups and the dielectric properties of the lipid bilayer that result from the buildup of an increasingly asymmetric charge distribution at the bilayer interfaces.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2024

Mixed Delay/Nondelay Embeddings Based Neuromorphic Computing with Patterned Nanomagnet Arrays

Patterned nanomagnet arrays (PNAs) have been shown to exhibit a strong geometrically frustrated dipole interaction. Some PNAs have also shown emergent domain wall dynamics. Previous works have demonstrated methods to physically probe these magnetization dynamics of PNAs to realize neuromorphic reservoir systems that exhibit chaotic dynamical behavior and high-dimensional nonlinearity. These PNA reservoir systems from prior works leverage echo state properties and linear/nonlinear short-term memory of component reservoir nodes to map and preserve the dynamical information of the input time-series data into nondelay spatial embeddings. Such mappings enable these PNA reservoir systems to imitate and predict/forecast the input time series data. However, these prior PNA reservoir systems are based solely on the nondelay spatial embeddings obtained at component reservoir nodes. As a result, they require a massive number of component reservoir nodes, or a very large spatial embedding (i.e., high-dimensional spatial embedding) per reservoir node, or both, to achieve acceptable imitation and prediction accuracy. These requirements reduce the practical feasibility of such PNA reservoir systems. To address this shortcoming, we present a mixed delay/nondelay embeddings-based PNA reservoir system. Our system uses a single PNA reservoir node with the ability to obtain a mixture of delay/nondelay embeddings of the dynamical information of the time-series data applied at the input of a single PNA reservoir node. Our analysis shows that when these mixed delay/nondelay embeddings are used to train a perceptron at the output layer, our reservoir system outperforms existing PNA-based reservoir systems for the imitation of NARMA 2, NARMA 5, NARMA 7, and NARMA 10 time series data, and for the short-term and long-term prediction of the Mackey Glass time series data.

Ti, Changpeng↗
DOE OSTI2021

Boost of the Bio-memristor Performance for Artificial Electronic Synapses by Surface Reconstruction

Biomaterial-based memristors (bio-memristors) are often adopted to emulate biological synapse functions and applied to construct neural computing networks in brain-inspired chip systems. However, the randomness of conductive filament formation in bio-memristors inhibits their switching performance by causing the dispersion of the device-switching parameters. In this case, a facile porous silk fibroin (p-SF) memristor was obtained through a protein surface reconstruction strategy, in which the size of the hole can be adjusted by the density of hybrid nanoseeds. Here, the porous SF memristors exhibit greatly enhanced electrical characteristics, including uniform I-V cycles, centralized distribution of the switching voltages, and both high and low resistances, compared to devices without pores. The results of three-dimensional (3D) simulations based on classical density functional theory (cDFT) suggest that the reconstructed pores in the SF layers guide the formation and fracture of Ag filaments under an electric field and enhance the overall conductivity by separating Ag + ion and electron diffusion pathways. Ag + ions are predicted to preferentially diffuse through pores, whereas electrons diffuse through the SF network. Interestingly, the device conductance can be bidirectionally modulated gradually by positive and negative voltages, can faithfully simulate short-term and long-term plasticity, and can even realize the triplet-spike-timing-dependent plasticity (triplet-STDP) rule, which can be used for pattern recognition in biological systems. The simulation results reveal that a memristor network of this type has an accuracy of ~95.78% in memory learning and the capability of pattern learning. This work provides a facile technology route to improve the performance of bionic-material memristors.

36 MATERIALS SCIENCE↗
DOE OSTI2023

Hybrid stochastic synapses enabled by scaled ferroelectric field-effect transistors

Achieving brain-like density and performance in neuromorphic computers necessitates scaling down the size of nanodevices emulating neuro-synaptic functionalities. However, scaling nanodevices results in reduction of programming resolution and emergence of stochastic non-idealities. While prior work has mainly focused on binary transitions, in this work, we leverage the stochastic switching of a three-state ferroelectric field-effect transistor to implement a long-term and short-term two-tier stochastic synaptic memory with a single device. Experimental measurements are performed on a scaled 28 nm high-k metal gate technology-based device to develop a probabilistic model of the hybrid stochastic synapse. In addition to the advantage of ultra-low programming energies afforded by scaling, our hardware–algorithm co-design analysis reveals the efficacy of the two-tier memory in comparison to binary stochastic synapses in on-chip learning tasks—paving the way for algorithms exploiting multi-state devices with probabilistic transitions beyond deterministic ones.

77 NANOSCIENCE AND NANOTECHNOLOGY↗
DOE OSTI2020

Wattile: Probabilistic Deep Learning-based Forecasting of Building Energy Consumption [SWR-20-94]

Accurate energy forecasting is becoming critical due to many reasons: i ) optimal distributed energy resources operations and dispatch, ii) fault detection and diagnostics, and iii) meeting operational energy efficiency targets. Wattile uses deep learning (DL) for the building's short-term load forecasting application. Two specific types of neural networks called, Long Short Term Memory (LSTM) and Sequence-to-Sequence (S2S) models are used to make predictions. Forecasting models are trained using online historical weather and occupancy indicator data streams from the Intelligent Campus Program's data acquisition systems at the National Renewable Energy Laboratory (NREL) for main meters and sub-meters of multiple building types. These models use probabilistic methods to provide quantile-based forecasts in addition to nominal conditional median predictions of electricity consumption.

Frank, Stephen↗
DOE OSTI2020

Multi-Frequency Analysis of Simulated versus Observed Variability in Tropospheric Temperature

Studies seeking to identify a human-caused global warming signal generally rely on climate model estimates of the “noise” of intrinsic natural variability. Assessing the reliability of these noise estimates is of critical importance. We evaluate here the statistical significance of differences between climate model and observational natural variability spectra for global-mean mid- to upper-tropospheric temperature (TMT). We use TMT information from satellites and large multimodel ensembles of forced and unforced simulations. Our main goal is to explore the sensitivity of model-versus-data spectral comparisons to a wide range of subjective decisions. These include the choice of satellite and climate model TMT datasets, the method for separating signal and noise, the frequency range considered, and the statistical model used to represent observed natural variability. Of particular interest is the amplitude of the interdecadal noise against which an anthropogenic tropospheric warming signal must be detected. We find that on time scales of 5–20 years, observed TMT variability is (on average) overestimated by the last two generations of climate models participating in the Coupled Model Intercomparison Project. This result is relatively insensitive to different plausible analyst choices, enhancing confidence in previous claims of detectable anthropogenic warming of the troposphere and indicating that these claims may be conservative. A further key finding is that two commonly used statistical models of short-term and long-term memory have deficiencies in their ability to capture the complex shape of observed TMT spectra.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Heterosynaptic plasticity in biomembrane memristors controlled by pH

Abstract In biology, heterosynaptic plasticity maintains homeostasis in synaptic inputs during associative learning and memory, and initiates long-term changes in synaptic strengths that nonspecifically modulate different synapse types. In bioinspired neuromorphic circuits, heterosynaptic plasticity may be used to extend the functionality of two-terminal, biomimetic memristors. In this article, we explore how changes in the pH of droplet interface bilayer aqueous solutions modulate the memristive responses of a lipid bilayer membrane in the pH range 4.97–7.40. Surprisingly, we did not find conclusive evidence for pH-dependent shifts in the voltage thresholds ( V* ) needed for alamethicin ion channel formation in the membrane. However, we did observe a clear modulation in the dynamics of pore formation with pH in time-dependent, pulsed voltage experiments. Moreover, at the same voltage, lowering the pH resulted in higher steady-state currents because of increased numbers of conductive peptide ion channels in the membrane. This was due to increased partitioning of alamethicin monomers into the membrane at pH 4.97, which is below the pKa (~5.3–5.7) of carboxylate groups on the glutamate residues of the peptide, making the monomers more hydrophobic. Neutralization of the negative charges on these residues, under acidic conditions, increased the concentration of peptide monomers in the membrane, shifting the equilibrium concentrations of peptide aggregate assemblies in the membrane to favor greater numbers of larger, increasingly more conductive pores. It also increased the relaxation time constants for pore formation and decay, and enhanced short-term facilitation and depression of the switching characteristics of the device. Modulating these thresholds globally and independently of alamethicin concentration and applied voltage will enable the assembly of neuromorphic computational circuitry with enhanced functionality. Impact statement We describe how to use pH as a modulatory “interneuron” that changes the voltage-dependent memristance of alamethicin ion channels in lipid bilayers by changing the structure and dynamical properties of the bilayer. Having the ability to independently control the threshold levels for pore conduction from voltage or ion channel concentration enables additional levels of programmability in a neuromorphic system. In this article, we note that barriers to conduction from membrane-bound ion channels can be lowered by reducing solution pH, resulting in higher currents, and enhanced short-term learning behavior in the form of paired-pulse facilitation. Tuning threshold values with environmental variables, such as pH, provide additional training and learning algorithms that can be used to elicit complex functionality within spiking neural networks. Graphical abstract

36 MATERIALS SCIENCE↗
DOE OSTI2022

Short-Term Rainfall Prediction Based on Radar Echo Using an Improved Self-Attention PredRNN Deep Learning Model

Accurate short-term precipitation forecast is extremely important for urban flood warning and natural disaster prevention. In this paper, we present an innovative deep learning model named ISA-PredRNN (improved self-attention PredRNN) for precipitation nowcasting based on radar echoes on the basis of the advanced PredRNN-V2. We introduce the self-attention mechanism and the long-term memory state into the model and design a new set of gating mechanisms. To better capture different intensities of precipitation, the loss function with weights was designed. We further train the model using a combination of reverse scheduled sampling and scheduled sampling to learn the long-term dynamics from the radar echo sequences. Experimental results show that the new model (ISA-PredRNN) can effectively extract the spatiotemporal features of radar echo maps and obtain radar echo prediction results with a small gap from the ground truths. From the comparison with the other six models, the new ISA-PredRNN model has the most accurate prediction results with a critical success index (CSI) of 0.7001, 0.5812 and 0.3052 under the radar echo thresholds of 10 dBZ, 20 dBZ and 30 dBZ, respectively.

Wu, Dali (ORCID:0000000231177074)↗