Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “long memory”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 325 records · Page 18
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Comprehension of Spatial Constraints by Neural Logic Learning from a Single RGB-D Scan

Autonomous industrial assembly relies on the precise measurement of spatial constraints as designed by computer-aided design (CAD) software such as SolidWorks. This paper proposes a framework for an intelligent industrial robot to understand the spatial constraints for model assembly. An extended generative adversary network (GAN) with a 3D long short-term memory (LSTM) network was designed to composite 3D point clouds from a single RGB-D scan. The spatial constraints of the segmented point clouds are identified by a neural-logic network that incorporates general knowledge of spatial constraints in terms of first-order logic. The model was designed to comprehend a complete set of spatial constraints that are consistent with industrial CAD software, including left, right, above, below, front, behind, parallel, perpendicular, concentric, and coincident relations. The accuracy of 3D model composition and spatial constraint identification was evaluated by the RGB-D scans and 3D models in the ABC dataset. The proposed model achieved 57.23% intersection over union (IoU) in 3D model composition, and over 99% in comprehending all spatial constraints.

Wang, Dali↗
DOE OSTI2022

An interpretable machine learning model for advancing terrestrial ecosystem predictions

We apply an interpretable Long Short-Term Memory (iLSTM) network for land-atmosphere carbon flux predictions based on time series observations of seven environmental variables. iLSTM enables interpretability of variable importance and variable-wise temporal importance to the prediction of targets by exploring internal network structures. The application results indicate that iLSTM not only improves prediction performance by capturing different dynamics of individual variables, but also reasonably interprets the different contribution of each variable to the target and its different temporal relevance to the target. This variable and temporal importance interpretation of iLSTM advances terrestrial ecosystem model development as well as our predictive understanding of the system.

Lu, Dan↗
DOE OSTI2022

Application of Dimensionality Reduction in Machine Learning Modeling of CO2 Storage

In this study, we developed deep learning models that are capable of predicting spatio-temporal outputs of CO2 saturation, pressure, and brine production in a 3D saline storage reservoir over 30 years of continuous CO2 injection and a 50-year post-injection timeframe. To improve computational efficiency and maintain performance accuracy, the model framework involves ensembling multi-layer autoencoder networks that provide dimensionality reduction of geologic inputs with fully connected long short-term memory (LSTM) neural networks that generate time-series prediction. This study was presented as poster at the 2022 Carbon Management Project Review Meeting held in Pittsburgh, PA (August 15 – 19, 2022).

Bello, Kolawole↗
DOE OSTI2021

Machine Learning for Improved Availability of the SNS Klystron High Voltage Converter Modulators

Beam availability has increased at the SNS, however, the targeted availability is greater than 95 %, while the SNS has failed to meet lower targets in the past. The HVCM used to power the linac klystrons have been one source of lost beam time and was chosen to explore using AI/ML techniques to improve reliability. Among the possibilities being explored are automating the tuning of HVCMs and predicting component failures such as capacitor aging, rectifier assemblies containing hundreds of diodes, and insulating oil degradation. The methodology pursued includes data cleaning, de-noising, post-analysis data labeling, and machine learning model development. We explore using Long Short-Term Memory and autoencoders for anomaly detection and prognostication used to schedule maintenance. We evaluate the use of model regularizers and constraints to improve the performance of the model and investigate methods to estimate the uncertainty of the models to provide a robust prediction with statistical interoperability. This paper describes the operational experience and known failures of the HVCMs and the proposed ML methodology and the preliminary results of training the AI/ML algorithms.

Pappas, Chris↗
DOE OSTI2023

Data-Driven Model Predictive Control for Temperature Management of Heat Pipe Microreactor

To enable the self-regulating capability of heat pipe (HP) microreactors, an anticipatory control strategy through model predictive control (MPC) could proactively respond to potential disturbances and deviations in operating setpoints. However, a key factor prohibiting the widespread adoption of MPCs in nuclear applications is the effort and computational costs associated with learning and calibrating first-principles-based process models when the target system is complex and when there are gaps between modeled and target reactor systems. In this paper, we demonstrate data-driven MPC using three approaches for modeling the system dynamics, including a linear state-space model, feedforward neural network, and recurrent neural networks long short-term memory. We present the development and validation process of each model and compare the performance of data-driven MPCs in controlling the temperatures of selected HPs at the evaporator and condenser regions in a 37-HP-monolith system. Our results show that, qualitatively, all data-driven MPCs are producing similar control actions, while quantitatively, with artificial neural nets (especially feedforward neural nets), MPC can better follow drastic changes in setpoints with smallest errors.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2024

Automating Rabi & Ramsey Measurements via ML

As quantum computers scale up, the manual process of qubit tune-up becomes increasingly impractical due to its time-consuming and repetitive nature. While existing research has explored some automation techniques, many models remain underutilized for this purpose. This research aims to answer the question: can qubit tune-up be automated using the Long Short-Term Memory (LSTM) model? For the purposes of this project, only the rabi and ramsey measurement cycle was automated. These measurements are used to fine-tune a rough qubit frequency by repeating them until the optimal qubit frequency is obtained. The LSTM model uses the qubit frequency at one time step to forecast the qubit frequency at the next time step. A rabi-ramsey simulation was made to fabricate a dataset to train and test the LSTM model. As the model was trained, the error of the model decreased. Although there wasn t enough training data to generate perfect predictions, this shows it is possible to utilize forecasting models in automating the tune-up process.

Roberts, Rachel↗
DOE OSTI2024

Automating Rabi & Ramsey Measurements via Machine Learning

As quantum computers scale up, the manual process of qubit tune-up becomes increasingly impractical due to its time-consuming and repetitive nature. While existing research has explored some automation techniques, many models remain underutilized for this purpose. This research aims to answer the question: can qubit tune-up be automated using the Long Short-Term Memory (LSTM) model? For the purposes of this project, only the rabi and ramsey measurement cycle was automated. These measurements are used to fine-tune a rough qubit frequency by repeating them until the optimal qubit frequency is obtained. The LSTM model uses the qubit frequency at one time step to forecast the qubit frequency at the next time step. A rabi-ramsey simulation was made to fabricate a dataset to train and test the LSTM model. As the model was trained, the error of the model decreased. Although there wasn't enough training data to generate perfect predictions, this shows it is possible to utilize forecasting models in automating the tune-up process.

Roberts, Rachel↗
DOE OSTI2024

Temporal sequence transformer to advance long-term streamflow prediction

Accurate streamflow prediction is crucial for understanding climate change impacts on water resources and for effective management of extreme hydrological events. While Long Short-Term Memory (LSTM) networks have been the dominant data-driven approach for streamflow forecasting, recent advancements in transformer architectures for time series tasks have shown promise in outperforming traditional LSTM models. This study introduces a transformer-based model that integrates historical streamflow data with climatic variables to enhance streamflow prediction accuracy. We evaluated our transformer model against a benchmark LSTM across five diverse basins in the United States. Results demonstrate that the transformer architecture consistently outperforms the LSTM model across all evaluation metrics, highlighting its potential as a more effective tool for hydrological forecasting. This research contributes to the ongoing development of advanced AI techniques for improved water resource management and climate change adaptation strategies.

Singh, Ruhaan [Farragut High School]↗
DOE OSTI2026

Estimating Flexibility Envelopes for Residential Customers From Utility Smart Meter Data: Preprint

Demand response from residential customers has significant potential to support power system operations, but accurate flexibility estimation is challenging due to the limited resolution of advanced metering infrastructure (AMI) data. Most utility AMI measurements are recorded at hourly intervals, with only a small portion at higher resolutions, and even fewer households have appliance-level energy usage data. To address this issue, this paper proposes a two-stage long short-term memory (LSTM) framework for estimating household flexibility envelopes from low-resolution AMI data. In the first stage, the heating, ventilating, and air-conditioning (HVAC) load and non-HVAC loads are estimated by using a model trained on a small set of households with appliance-level profiles. These estimated data are then used to compute the upper- and lower-flexibility bounds, which are subsequently down-sampled to lower-resolution data. In the second stage, these flexibility bounds serve as training inputs for another LSTM model, enabling direct prediction of flexibility envelopes for households with only hourly AMI data. This method is validated using Pecan Street data from two different areas, and the results demonstrate its applicability and effectiveness.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2022

Data-driven modeling of power generation for a coal power plant under cycling

Increased penetration of renewables for power generation has negatively impacted the dynamics of conventional fossil fuel-based power plants. The power plants operating on the base load are forced to cycle, to adjust to the fluctuating power demands. This results in an inefficient operation of the coal power plants, which leads up to higher operating losses. To overcome such operational challenge associated with cycling and to develop an optimal process control, this work analyzes a set of models for predicting power generation. Moreover, the power generation is intrinsically affected by the state of the power plant components, and therefore our model development also incorporates additional power plant process variables while forecasting the power generation. We present and compare multiple state-of-the-art forecasting data-driven methods for power generation to determine the most adequate and accurate model. We also develop an interpretable attention-based transformer model to explain the importance of process variables during training and forecasting. The trained deep neural network (DNN) LSTM model has good accuracy in predicting gross power generation under various prediction horizons with/without cycling events and outperforms the other models for long-term forecasting. The DNN memory-based models show significant superiority over other state-of-the-art machine learning models for short, medium and long range predictions. The transformer-based model with attention enhances the selection of historical data for multi-horizon forecasting, and also allows to interpret the significance of internal power plant components on the power generation. This newly gained insights can be used by operation engineers to anticipate and monitor the health of power plant equipment during high cycling periods.

01 COAL, LIGNITE, AND PEAT↗
DOE OSTI2023

Adenosine A2A Receptor Up-Regulation Pre-Dates Deficits of Synaptic Plasticity and of Memory in Mice Exposed to Aβ1–42 to Model Early Alzheimer’s Disease

The intracerebroventricular (icv) injection of amyloid peptides (Aβ) models Alzheimer’s disease (AD) in mice, as typified by the onset within 15 days of deficits of memory and of hippocampal long-term potentiation (LTP) that are prevented by the blockade of adenosine A2A receptors (A2AR). Since A2AR overfunction is sufficient to trigger memory deficits, we tested if A2AR were upregulated in hippocampal synapses before the onset of memory deficits to support the hypothesis that A2AR overfunction could be a trigger of AD. Six to eight days after Aβ-icv injection, mice displayed no alterations of hippocampal dependent memory; however, they presented an increased excitability of hippocampal synapses, a slight increase in LTP magnitude in Schaffer fiber-CA1 pyramid synapses and an increased density of A2AR in hippocampal synapses. A2AR blockade with SCH58261 (50 nM) normalized excitability and LTP in hippocampal slices from mice sacrificed 7–8 days after Aβ-icv injection. Fifteen days after Aβ-icv injection, mice displayed evident deficits of hippocampal-dependent memory deterioration, with reduced hippocampal CA1 LTP but no hyperexcitability and a sustained increase in synaptic A2AR, which blockade restored LTP magnitude. This shows that the upregulation of synaptic A2AR precedes the onset of deterioration of memory and of hippocampal synaptic plasticity, supporting the hypothesis that the overfunction of synaptic A2AR could be a trigger of memory deterioration in AD.

Lopes, Cátia R.↗
DOE OSTI2020

An encoder–decoder LSTM-based EMPC framework applied to a building HVAC system

Numerous studies have demonstrated the benefit of economic model predictive control (EMPC) applied to building heating, ventilation, and air conditioning (HVAC) systems. However, the construction and training of predictive models for building HVAC systems are widely recognized as a key technological barrier preventing large-scale adoption of EMPC for buildings. In this work, an encoder–decoder long short-term memory-based EMPC framework is developed. The key advantage of the approach is that a model may be automatically generated from a list of inputs and outputs. From the definition of inputs and outputs, the constructed model may be trained and automatically embedded into the EMPC framework for real-time estimation and control. The overall end-to-end EMPC framework from model training to on-line estimation and control are described. To this end, the encoder–decoder model provides a natural framework for state estimation (encoder), which is required to provide an initial condition for the predictive model of EMPC (decoder). Closed-loop simulations using EnergyPlus are performed to demonstrate the approach. The simulated closed-loop system consists of a building zone from a multi-zone building, which is served by an air handling unit-variable air volume HVAC system. For the HVAC example considered, the trained encoder–decoder model can predict the indoor air temperature and HVAC sensible cooling rate of a building zone over a two-day horizon with high accuracy. Overall, we find that considering a time-of-use electric rate structure, the EMPC, which manipulates the zone temperature setpoint, can reduce the HVAC power consumption cost relative to keeping the zone temperature setpoint at its maximum value (i.e., minimum energy approach).

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2023

Cations Control Lipid Bilayer Memcapacitance Associated with Long-Term Potentiation

Phospholipid bilayers can be described as capacitors whose capacitance per unit area (specific capacitance, C m ) is determined by their thickness and dielectric constant–independent of applied voltage. It is also widely assumed that the C m of membranes can be treated as a “biological constant”. Recently, using droplet interface bilayers (DIBs), it was shown that zwitterionic phosphatidylcholine (PC) lipid bilayers can act as voltage-dependent, nonlinear memory capacitors, or memcapacitors. When exposed to an electrical “training” stimulation protocol, capacitive energy storage in lipid membranes was enhanced in the form of long-term potentiation (LTP), which enables biological learning and long-term memory. LTP was the result of membrane restructuring and the progressive asymmetric distribution of ions across the lipid bilayer during training, which is analogous, for example, to exponential capacitive energy harvesting from self-powered nanogenerators. Here, we describe how LTP could be produced from a membrane that is continuously pumped into a nonequilibrium steady state, altering its dielectric properties. During this time, the membrane undergoes static and dynamic changes that are fed back to the system’s potential energy, ultimately resulting in a membrane whose modified molecular structure supports long-term memory storage and LTP. Here, we also show that LTP is very sensitive to different salts (KCl, NaCl, LiCl, and TmCl 3 ), with LiCl and TmCl 3 having the most profound effect in depressing LTP, relative to KCl. This effect is related to how the different cations interact with the bilayer zwitterionic PC lipid headgroups primarily through electric-field-induced changes to the statistically averaged orientations of water dipoles at the bilayer headgroup interface.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2025

SpecSims: A Scalable Speculative Tree-based Simulation Cloning Framework for Finite Memory Machines

Simulation cloning is a technique in which cloned simulations whose state spaces differ partially from their parent simulation due to intervening events are spawned at runtime and concurrently advanced. It is a powerful method to carry out what-if analysis by speculatively exploring and evaluating the impact of various permutations of intervening cascade of events. Due to the exponential growth in the number of possible clones even for a small number of distinct intervening events, the practical efficacy of the approach is often severely limited by the maximum available memory of the computing host. In this paper, we introduce a novel speculative simulation cloning framework that executes a simulation cloning campaign capable of efficiently exploring an exponentially large space of clone simulations created by permutation of intervening events under a finite memory constraint. We provide a theoretical analysis of the runtime characteristics of our proposed approach and highlight its novel advantages such as memory-aware and as-long-as-needed execution. Furthermore, in support of our analytical findings and to demonstrate its practical feasibility, we implement a prototype of the cloning framework on a shared memory system and report its performance characteristics in the context of a heat diffusion simulation, and a power grid simulation subject to cascading disruptions from geomagnetic disturbances.

Simulation framework↗
DOE OSTI2024

Verbal Learning and Memory Deficits across Neurological and Neuropsychiatric Disorders: Insights from an ENIGMA Mega Analysis

Deficits in memory performance have been linked to a wide range of neurological and neuropsychiatric conditions. While many studies have assessed the memory impacts of individual conditions, this study considers a broader perspective by evaluating how memory recall is differentially associated with nine common neuropsychiatric conditions using data drawn from 55 international studies, aggregating 15,883 unique participants aged 15–90. The effects of dementia, mild cognitive impairment, Parkinson’s disease, traumatic brain injury, stroke, depression, attention-deficit/hyperactivity disorder (ADHD), schizophrenia, and bipolar disorder on immediate, short-, and long-delay verbal learning and memory (VLM) scores were estimated relative to matched healthy individuals. Random forest models identified age, years of education, and site as important VLM covariates. A Bayesian harmonization approach was used to isolate and remove site effects. Regression estimated the adjusted association of each clinical group with VLM scores. Memory deficits were strongly associated with dementia and schizophrenia (p < 0.001), while neither depression nor ADHD showed consistent associations with VLM scores (p > 0.05). Differences associated with clinical conditions were larger for longer delayed recall duration items. By comparing VLM across clinical conditions, this study provides a foundation for enhanced diagnostic precision and offers new insights into disease management of comorbid disorders.

Neurosciences & Neurology↗
DOE OSTI2020

Large-scale integration of artificial atoms in hybrid photonic circuits

A central challenge in developing quantum computers and long-range quantum networks is the distribution of entanglement across many individually controllable qubits. Colour centres in diamond have emerged as leading solid-state ‘artificial atom’ qubits because they enable on-demand remote entanglement, coherent control of over ten ancillae qubits with minute-long coherence times and memory-enhanced quantum communication. A critical next step is to integrate large numbers of artificial atoms with photonic architectures to enable large-scale quantum information processing systems. So far, these efforts have been stymied by qubit inhomogeneities, low device yield and complex device requirements. Here we introduce a process for the high-yield heterogeneous integration of ‘quantum microchiplets’—diamond waveguide arrays containing highly coherent colour centres—on a photonic integrated circuit (PIC). We use this process to realize a 128-channel, defect-free array of germanium-vacancy and silicon-vacancy colour centres in an aluminium nitride PIC. Photoluminescence spectroscopy reveals long-term, stable and narrow average optical linewidths of 54 megahertz (146 megahertz) for germanium-vacancy (silicon-vacancy) emitters, close to the lifetime-limited linewidth of 32 megahertz (93 megahertz). We show that inhomogeneities of individual colour centre optical transitions can be compensated in situ by integrated tuning over 50 gigahertz without linewidth degradation. The ability to assemble large numbers of nearly indistinguishable and tunable artificial atoms into phase-stable PICs marks a key step towards multiplexed quantum repeaters and general-purpose quantum processors

36 MATERIALS SCIENCE↗
DOE OSTI2021

Short-term apartment-level load forecasting using a modified neural network with selected auto-regressive features

Residential electricity load profiles and their diversity have become increasingly important to realize the benefits of Smart or Transactive Energy Networks (TENs). An important element of TENs will be practical, accurate, and implementable residential load forecasting techniques. While there have been many approaches to short-term load forecasting, few have included forecasting for individual households, partly because the high volatility and idiosyncrasies present in individual household load data can pose significant challenges. In this study, we develop a Convolutional Long Short-Term Memory-based neural network with Selected Autoregressive Features (termed a CLSAF model) to improve short-term household electricity load forecasting accuracy by employing three strategies: autoregressive features selection, exogenous features selection, and a “default” state to avoid overfitting at times of high load volatility. We include aggregations of apartments to floor and building level, because utilities may favor transactive approaches that rely on aggregator models, e.g., a cluster of consumers as opposed to an individual. We demonstrate that the CLSAF model, by virtue of its enhanced feature representation and modest computational resources, can accomplish load forecasting in a multi-family residential building across three spatial granularities (individual apartment/household, floor, and building levels), with an accuracy improvement of up to 25% compared to a persistence model. We propose a data screening technique to characterize time-series electricity-load data. This technique is suitable for integration into a TEN ecosystem and allows one to estimate confidence levels of the load forecasts to optimize computational resources and the risks associated with uncertain forecasts.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2021

Model fusion with physics-guided machine learning: Projection-based reduced-order modeling

The unprecedented amount of data generated from experiments, field observations, and large-scale numerical simulations at a wide range of spatiotemporal scales has enabled the rapid advancement of data-driven and especially deep learning models in the field of fluid mechanics. Although these methods are proven successful for many applications, there is a grand challenge of improving their generalizability. This is particularly essential when data-driven models are employed within outer-loop applications like optimization. In this work, we put forth a physics-guided machine learning (PGML) framework that leverages the interpretable physics-based model with a deep learning model. Leveraging a concatenated neural network design from multi-modal data sources, the PGML framework is capable of enhancing the generalizability of data-driven models and effectively protects against or inform about the inaccurate predictions resulting from extrapolation. We apply the PGML framework as a novel model fusion approach combining the physics-based Galerkin projection model and long- to short-term memory (LSTM) network for parametric model order reduction of fluid flows. We demonstrate the improved generalizability of the PGML framework against a purely data-driven approach through the injection of physics features into intermediate LSTM layers. Our quantitative analysis shows that the overall model uncertainty can be reduced through the PGML approach, especially for test data coming from a distribution different than the training data. Moreover, we demonstrate that our approach can be used as an inverse diagnostic tool providing a confidence score associated with models and observations. The proposed framework also allows for multi-fidelity computing by making use of low-fidelity models in the online deployment of quantified data-driven models.

42 ENGINEERING↗