Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “long short term memory model”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
NASA NTRSDate not supplied

Using Long-Short Term Memory Models to Predict Solar Active Regions Emergence

We train Long Short-Term Memory (LSTM) models that predict the formation of active regions (ARs), the main source of eruptive solar activity. Using the Doppler shift velocity, the continuum intensity and the magnetic field full-disk maps from SDO/HMI we have created time-series datasets of acoustic power and magnetic flux which are used to train Long Short-Term Memory (LSTM) models on predicting decreases in continuum intensity 12 hours in advance. Testing of the models' performance was done on data from 5 ARs, unseen from the model during training. The model predicted the emergence of AR11726, AR13165 and AR13179, 10, 29 and 5 hours in advance, and variations of this model achieved average RMSE values of 0.11 for both active and quiet parts of the solar disc, showing the ability of the model to capture acoustic power anomalies and predict continuum intensity variations. This work sets the foundations for the very first ML-aided prediction of solar ARs.

SMD↗
DOE OSTI2020

Evaluating the Potential and Challenges of an Uncertainty Quantification Method for Long Short–Term Memory Models for Soil Moisture Predictions

Recently, recurrent deep networks have shown promise to harness newly available satellite–sensed data for long–term soil moisture projections. However, to be useful in forecasting, deep networks must also provide uncertainty estimates. Here we evaluated Monte Carlo dropout with an input–dependent data noise term (MCD+N), an efficient uncertainty estimation framework originally developed in computer vision, for hydrologic time series predictions. MCD+N simultaneously estimates a heteroscedastic input–dependent data noise term (a trained error model attributable to observational noise) and a network weight uncertainty term (attributable to insufficiently constrained model parameters). Although MCD+N has appealing features, many heuristic approximations were employed during its derivation, and rigorous evaluations and evidence of its asserted capability to detect dissimilarity were lacking. To address this, we provided an in–depth evaluation of the scheme's potential and limitations. We showed that for reproducing soil moisture dynamics recorded by the Soil Moisture Active Passive (SMAP) mission, MCD+N indeed gave a good estimate of predictive error, provided that we tuned a hyperparameter and used a representative training data set. The input–dependent term responded strongly to observational noise, while the model term clearly acted as a detector for physiographic dissimilarity from the training data, behaving as intended. However, when the training and test data were characteristically different, the input–dependent term could be misled, undermining its reliability. Additionally, due to the data–driven nature of the model, data noise also influences network weight uncertainty, and therefore the two uncertainty terms are correlated. Altogether, this approach has promise, but care is needed to interpret the results.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Surrogate modelling for urban building energy simulation based on the bidirectional long short-term memory model

Here, the urban microclimate is essential for accurate simulation-based urban building energy modelling (UBEM). However, a high spatial-resolution microclimate can increase the computational resources demands of UBEM. Surrogate modelling is one of the promising approaches for fast UBEM. This study proposes a bidirectional Long Short-Term Memory (LSTM)-based approach for simulation-based UBEM surrogate modelling. The estimations are aggregated into census tracts using total building floor area. A case study using UBEM to estimate annual hourly building energy use and anthropogenic heat from all existing buildings in Los Angeles County found that most of the surrogate models can complete the annual hourly simulation within 90 minutes with a normalized mean absolute error lower than 10%, and that the bidirectional LSTM outperforms the standard LSTM in accuracy. This study demonstrates the advantages of bidirectional RNN architecture in building energy surrogate modelling and is expected to promote long-term and high-resolution UBEM with detailed microclimates.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2023

Comparing quantile regression forest and mixture density long short-term memory models for probabilistic post-processing of satellite precipitation-driven streamflow simulations

Abstract. Deep learning (DL) and machine learning (ML) are widely used in hydrological modelling, which plays a critical role in improving the accuracy of hydrological predictions. However, the trade-off between model performance and computational cost has always been a challenge for hydrologists when selecting a suitable model, particularly for probabilistic post-processing with large ensemble members. This study aims to systematically compare the quantile regression forest (QRF) model and countable mixtures of asymmetric Laplacians long short-term memory (CMAL-LSTM) model as hydrological probabilistic post-processors. Specifically, we evaluate their ability in dealing with biased streamflow simulations driven by three satellite precipitation products across 522 nested sub-basins of the Yalong River basin in China. Model performance is comprehensively assessed using a series of scoring metrics from both probabilistic and deterministic perspectives. Our results show that the QRF model and the CMAL-LSTM model are comparable in terms of probabilistic prediction, and their performances are closely related to the flow accumulation area (FAA) of the sub-basin. The QRF model outperforms the CMAL-LSTM model in most sub-basins with smaller FAA, while the CMAL-LSTM model has an undebatable advantage in sub-basins with FAA larger than 60 000 km2 in the Yalong River basin. In terms of deterministic predictions, the CMAL-LSTM model is preferred, especially when the raw streamflow is poorly simulated and used as input. However, setting aside the differences in model performance, the QRF model with 100-member quantiles demonstrates a noteworthy advantage by exhibiting a 50 % reduction in computation time compared to the CMAL-LSTM model with the same ensemble members in all experiments. As a result, this study provides insights into model selection in hydrological post-processing and the trade-offs between model performance and computational efficiency. The findings highlight the importance of considering the specific application scenario, such as the catchment size and the required accuracy level, when selecting a suitable model for hydrological post-processing.

Geology↗
DOE OSTI2022

Technical note: Using long short-term memory models to fill data gaps in hydrological monitoring networks

Abstract. Quantifying the spatiotemporal dynamics in subsurface hydrological flows over a long time window usually employs a network of monitoring wells. However, such observations are often spatially sparse with potential temporal gaps due to poor quality or instrument failure. In this study, we explore the ability of recurrent neural networks to fill gaps in a spatially distributed time-series dataset. We use a well network that monitors the dynamic and heterogeneous hydrologic exchanges between the Columbia River and its adjacent groundwater aquifer at the U.S. Department of Energy's Hanford site. This 10-year-long dataset contains hourly temperature, specific conductance, and groundwater table elevation measurements from 42 wells with gaps of various lengths. We employ a long short-term memory (LSTM) model to capture the temporal variations in the observed system behaviors needed for gap filling. The performance of the LSTM-based gap-filling method was evaluated against a traditional autoregressive integrated moving average (ARIMA) method in terms of error statistics and accuracy in capturing the temporal patterns of river corridor wells with various dynamics signatures. Our study demonstrates that the ARIMA models yield better average error statistics, although they tend to have larger errors during time windows with abrupt changes or high-frequency (daily and subdaily) variations. The LSTM-based models excel in capturing both high-frequency and low-frequency (monthly and seasonal) dynamics. However, the inclusion of high-frequency fluctuations may also lead to overly dynamic predictions in time windows that lack such fluctuations. The LSTM can take advantage of the spatial information from neighboring wells to improve the gap-filling accuracy, especially for long gaps in system states that vary at subdaily scales. While LSTM models require substantial training data and have limited extrapolation power beyond the conditions represented in the training data, they afford great flexibility to account for the spatial correlations, temporal correlations, and nonlinearity in data without a priori assumptions. Thus, LSTMs provide effective alternatives to fill in data gaps in spatially distributed time-series observations characterized by multiple dominant frequencies of variability, which are essential for advancing our understanding of dynamic complex systems.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

HydroEcoLSTM: A Python package with graphical user interface for hydro-ecological modeling with long short-term memory neural network

Machine learning (ML) is emerging as a promising tool for modeling hydro-ecological processes due to the increasing availability of large environmental data. However, the use of ML requires sufficient programming knowledge due to a lack of a graphical user interface (GUI). In this study, we introduced a GUI package, named HydroEcoLSTM, with the long short-term memory network (LSTM) as the core model, that allows non-ML experts to utilize their domain knowledge to construct complex ML models. We demonstrated the functionalities of HydroEcoLSTM with two practical examples, including (1) predictions of streamflow in both gauged and ungauged catchments and (2) predictions of multiple outputs (i.e., streamflow and isotope transport from two catchments). The simulation results obtained in both case experiments are satisfactory. In the first example, the average Nash–Sutcliffe Efficiency (NSE) for streamflow simulation during the testing period is 0.79 while the application of the trained model in two assumed ungauged catchments also achieves the average NSE of 0.68. In the second example, the average NSE for streamflow and instream isotope simulation during the testing period is 0.71. Ultimately, applications of HydroEcoLSTM with real-world examples demonstrate its potential use for practical applications and research without requiring extensive coding skills.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Digital Twin Empowered PV Power Prediction

The accurate prediction of photovoltaic (PV) power generation is significant to ensure the economic and safe operation of power systems. To this end, the paper proposes a new digital twin (DT) empowered PV power prediction framework that is capable of ensuring reliable data transmission and employing the DT to achieve high accuracy of power prediction. With this framework, considering potential data contamination in the collected PV data, a generative adversarial network is employed to restore the historical data set, which offers a prerequisite to ensure accurate mapping from the physical space to the digital space. Further, a new DT empowered PV power prediction method is proposed. Therein, we model a DT that encompasses a digital physical model for reflecting the physical operation mechanism and a neural network model (i.e., a parallel network of convolution and bidirectional long-short-term memory model) for capturing the hidden spatial-temporal features. The proposed method enables the use of the DT to take advantages of the digital physical model and the neural network model, resulting in enhanced prediction accuracy. Finally, a real data set is conducted to access the effectiveness of the proposed method.

14 SOLAR ENERGY↗
DOE OSTI2021

A large-scale comparison of Artificial Intelligence and Data Mining (AI&DM) techniques in simulating reservoir releases over the Upper Colorado Region

In recent years, the Artificial Intelligence and Data Mining (AI&DM) models have become popular tools in assisting various aspects of reservoir operation. However, the practical uses are still rarely reported. Comparison experiment of many AI&DM models over a large number of reservoir cases is particularly valuable to help reservoir operators first examine the usefulness and transferability of different AI&DM models, and then identify the most stable and reliable AI&DM model in assist of various decision-making processes. In this study, a total of 12 AI&DM models with different parameterizations and simulation scenarios are comprehensively tested out and compared in simulating the controlled reservoir outflows of 33 reservoir cases over the Upper Colorado Region, United States. Results show that the Random Forecast and the Long-Short-Term-Memory model could consistently derive the best statistical performance than other models under the baseline simulation scenario. The employed AI&DM models could obtain satisfactory statistical interquartile ranges (25–75%) between [0.6–0.9], [0.3–0.8], and [0.2–0.8], for CORR, NSE, and KGE measurements, respectively, and [1.5–6.5], [–15 to 20], and [0.5–8.5] for the normalized RMSE, PBIAS and RSR measurements, respectively. Results also show Multi-Layer Perceptron model and Extreme Gradient Boosting Tree Algorithm produced more stable and superior performance than other models under more complex input scenarios. We also found that the performance of different AI&DM models are closely relevant to the reservoir elevations, sizes, and functionalities. Discussions were made about the sensitivity of AI&DM models’ parameterizations and the key advantages of AI&DM models over the rule-based reservoir models. We further identify that the main advantage of AI&DM models is the flexibility in designing input structures, whereas the rule-based simulation model is rather limited. Future studies were suggested regarding the best way reservoir operators and researchers could use, select, and apply different AI&DM models in simulating reservoir releases under different natural and modeling environments. Finally, this comparison study also serves as a reference and a piece of groundwork for further promoting the practical uses of AI&DM models in assisting reservoir operation.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

Can artificial intelligence and data-driven machine learning models match or even replace process-driven hydrologic models for streamflow simulation?: A case study of four watersheds with different hydro-climatic regions across the CONUS

With recent developments in computational techniques, Data-driven Machine Learning Models (DMLs) have shown great potential in simulating streamflow and capturing the rainfall-runoff relationship in given watersheds, which are traditionally fulfilled by Process-based Hydrologic Models (PHMs). There are debates on whether the DMLs can outperform and possibly replace the classical PHMs for streamflow simulation and river forecasting, but no clear conclusions have been made. This study aims to investigate whether the newer DMLs have any potential in further improving the simulation accuracy of classical PHMs, and vice versa. To do this, we compared a few popular PHMs and DMLs over four watersheds across the Continental US (CONUS) that are associated with different input, climate, and regional conditions. A total of five hydrologic models were chosen, including (1) two classical lumped models, i.e., the Sacramento Soil Moisture Accounting (SAC-SMA) and Xinanjiang (XAJ); (2) one modern distributed model, termed Coupled Routing and Excess Storage (CREST); (3) and two DMLs including an Artificial Neural Networks (ANN) and a deep learning model, termed Long Short Term Memory (LSTM). Our results demonstrated that the DMLs still significantly biased when using the baseline input scenario with the PHMs. However, the DMLs fed with delayed input scenarios had great potential and can reach high simulation accuracy. The DMLs, especially the ANN, outperformed other employed models under the rainfall-runoff relationship in which rainfall dominantly drives. Furthermore, the DMLs also showed better performance in the high-flow regime, while the PHMs had a better performance for the low-flow regime, implying both PHMs and DMLs have their own merits and are worthy of joint development. In general, our study indicated a great potential of using DMLs to simulate streamflow, but further studies are still needed to verify the transferability and scalability of DMLs in large-scale experiments, such as the Distributed Model Intercomparison Projects 1&2 conducted by National Weather Services but to compare modern DMLs and PHMs.

58 GEOSCIENCES↗
DOE OSTI2020

A long short-term memory embedding for hybrid uplifted reduced order models

In this paper, we introduce an uplifted reduced order modeling (UROM) approach through the integration of standard projection based methods with long short-term memory (LSTM) embedding. Our approach has three modeling layers or components. In the first layer, we utilize an intrusive projection approach to model dynamics represented by the largest modes. The second layer consists of an LSTM model to account for residuals beyond this truncation. This closure layer refers to the process of including the residual effect of the discarded modes into the dynamics of the largest scales. However, the feasibility of generating a low rank approximation tails off for higher Kolmogorov n -width systems due to the underlying nonlinear processes. The third uplifting layer, called super-resolution, addresses this limited representation issue by expanding the span into a larger number of modes utilizing the versatility of LSTM. Therefore, our model integrates a physics-based projection model with a memory embedded LSTM closure and an LSTM based super-resolution model. In several applications, we exploit the use of Grassmann manifold to construct UROM for unseen conditions. We performed numerical experiments by using the Burgers and Navier-Stokes equations with quadratic nonlinearity. Finally, our results show robustness of the proposed approach in building reduced order models for parameterized systems and confirm the improved trade-off between accuracy and efficiency.

42 ENGINEERING↗
DOE OSTI2025

sPHENIX heavy flavor jet tagging studies in p+p at $\sqrt{s_{NN}}=200~GeV$

Heavy-flavor jets, which are initiated from heavy quarks, are ideal probes for studying flavor dependent parton energy loss. We report on the performance of jet flavor tagging using two Neural Network Machine Learning (ML) models: the Long Short-Term Memory (LSTM) model and an Attention-based Neural Network, in simulations of 200 GeV p + p collisions. The tagging performance of bottom quark initiated jets with both ML models surpasses that of the traditional cut-based method. Technical details, including sample and kinematic variable selections, the machine learning training and testing setup with parameter tuning, and outcome comparisons, will be discussed.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2024

Machine-learned closure of URANS for stably stratified turbulence: connecting physical timescales & data hyperparameters of deep time-series models

Stably stratified turbulence (SST), a model that is representative of the turbulence found in the oceans and atmosphere, is strongly affected by fine balances between forces and becomes more anisotropic in time for decaying scenarios. Moreover, there is a limited understanding of the physical phenomena described by some of the terms in the Unsteady Reynolds-Averaged Navier–Stokes (URANS) equations—used to numerically simulate approximate solutions for such turbulent flows. Rather than attempting to model each term in URANS separately, it is attractive to explore the capability of machine learning (ML) to model groups of terms, i.e. to directly model the force balances. We develop deep time-series ML for closure modeling of the URANS equations applied to SST. We consider decaying SST which are homogeneous and stably stratified by a uniform density gradient, enabling dimensionality reduction. We consider two time-series ML models: long short-term memory and neural ordinary differential equation. Both models perform accurately and are numerically stable in a posteriori (online) tests. Furthermore, we explore the data requirements of the time-series ML models by extracting physically relevant timescales of the complex system. We find that the ratio of the timescales of the minimum information required by the ML models to accurately capture the dynamics of the SST corresponds to the Reynolds number of the flow. The current framework provides the backbone to explore the capability of such models to capture the dynamics of high-dimensional complex dynamical system like SST flows.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Long Short-Term Memory Networks for Pattern Recognition of Synthetical Complete Earthquake Catalog

Exploring the spatiotemporal distribution of earthquake activity, especially earthquake migration of fault systems, can greatly to understand the basic mechanics of earthquakes and the assessment of earthquake risk. By establishing a three-dimensional strike-slip fault model, to derive the stress response and fault slip along the fault under regional stress conditions. Our study helps to create a long-term, complete earthquake catalog. We modelled Long-Short Term Memory (LSTM) networks for pattern recognition of the synthetical earthquake catalog. The performance of the models was compared using the mean-square error (MSE). Our results showed clearly the application of LSTM showed a meaningful result of 0.08% in the MSE values. Our best model can predict the time and magnitude of the earthquakes with a magnitude greater than Mw = 6.5 with a similar clustering period. These results showed conclusively that applying LSTM in a spatiotemporal series prediction provides a potential application in the study of earthquake mechanics and forecasting of major earthquake events.

58 GEOSCIENCES↗
DOE OSTI2022

Forecasting of in situ electron energy loss spectroscopy

Abstract Forecasting models are a central part of many control systems, where high-consequence decisions must be made on long latency control variables. These models are particularly relevant for emerging artificial intelligence (AI)-guided instrumentation, in which prescriptive knowledge is needed to guide autonomous decision-making. Here we describe the implementation of a long short-term memory model (LSTM) for forecasting in situ electron energy loss spectroscopy (EELS) data, one of the richest analytical probes of materials and chemical systems. We describe key considerations for data collection, preprocessing, training, validation, and benchmarking, showing how this approach can yield powerful predictive insight into order-disorder phase transitions. Finally, we comment on how such a model may integrate with emerging AI-guided instrumentation for powerful high-speed experimentation.

36 MATERIALS SCIENCE↗
DOE OSTI2023

Predicting Kyasanur forest disease in resource-limited settings using event-based surveillance and transfer learning

In recent years, the reports of Kyasanur forest disease (KFD) breaking endemic barriers by spreading to new regions and crossing state boundaries is alarming. Effective disease surveillance and reporting systems are lacking for this emerging zoonosis, hence hindering control and prevention efforts. We compared time-series models using weather data with and without Event-Based Surveillance (EBS) information, i.e., news media reports and internet search trends, to predict monthly KFD cases in humans. We fitted Extreme Gradient Boosting (XGB) and Long Short-Term Memory models at the national and regional levels. We utilized the rich epidemiological data from endemic regions by applying Transfer Learning (TL) techniques to predict KFD cases in new outbreak regions where disease surveillance information was scarce. Overall, the inclusion of EBS data, in addition to the weather data, substantially increased the prediction performance across all models. The XGB method produced the best predictions at the national and regional levels. The TL techniques outperformed baseline models in predicting KFD in new outbreak regions. Novel sources of data and advanced machine-learning approaches, e.g., EBS and TL, show great potential towards increasing disease prediction capabilities in data-scarce scenarios and/or resource-limited settings, for better-informed decisions in the face of emerging zoonotic threats.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2023

Temperature outweighs light and flow as the predominant driver of dissolved oxygen in US rivers

The concentration of dissolved oxygen (DO), an important measure of water quality and river metabolism, varies tremendously in time and space. Riverine DO is commonly perceived as regulated by interacting and competing drivers (light, temperature and flow) that define rivers’ climate. Its continental-scale drivers, however, have remained elusive, partly due to the scarcity and spatio-temporal inconsistency of water quality data. Here we show, via a deep learning model (long short-term memory) trained using data from 580 rivers, that temperature predominantly drives daily DO dynamics in the contiguous United States. Light comes a close second, whereas flow imparts minimal influence. This work showcases the promise of using deep learning models for data filling that enables large-scale systematic analysis of patterns and drivers. Results show fairly accurate prediction of DO by temperature alone, and declining DO in warming rivers, which has important implications for water security and ecosystem health in the future climate.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Data and scripts associated with the manuscript "Encoding Diel Hysteresis and the Birch Effect in Dryland Soil Respiration Models through Knowledge-Guided Deep Learning"

This package contains the data and scripts used in "Encoding Diel Hysteresis and the Birch Effect in Dryland Soil Respiration Models through Knowledge-Guided Deep Learning" (Jiang et al., 2022). The data.zip file contains the flux tower and automated chamber observations used for developing the deep learning model for modeling soil respiration. The scripts.zip file contains the Jupyter notebooks and python scripts for preprocessing the data, training the deep learning models, and postprocessing the results. The src.zip contains the source code for training the deep learning model, performing mutual information analysis, and plotting functions. The trained_models.zip contains multiple folders used for hosting the trained deep-learning models and the associated soil respiration predictions. The whole process is performed using python. We include the REAMD.md to document the python package requirements.Soil respiration in dryland ecosystems is challenging to model due to its complex interactions with environmental drivers. Knowledge-guided deep learning provides a much more effective means of accurately representing these complex interactions than traditional Q10-based models. Mutual information analysis revealed that future soil temperature shares more information with soil respiration than past soil temperature, consistent with their clockwise diel hysteresis. We explicitly encoded diel hysteresis, soil drying, and soil rewetting effects on soil respiration dynamics in a newly designed Long Short Term Memory (LSTM) model. The model takes both past and future environmental drivers as inputs to predict soil respiration. The new LSTM model substantially outperformed three Q10-based models and the Community Land Model when reproducing the observed soil respiration dynamics in a semi-arid ecosystem. The new LSTM model clearly demonstrated its superiority for temporally extrapolating soil respiration dynamics, such that the resulting correlation with observational data is up to 0.7 while the correlations of both Q10-based models and the Community Land Model (CLM) are less than 0.4. Our results underscore the high potential for knowledge-guided deep learning to replace Q10-based soil respiration modules in Earth system models.

54 ENVIRONMENTAL SCIENCES↗