SEARCH · Engineering Papers

Results for “Recurrent neural network”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4

All sources NASA NTRS DOE OSTI

DOE OSTI2024

Improved Subseasonal Forecasting of Extreme Polar Vortices Using Machine Learning

Our research was focused on forecasting the position and shape of the winter stratospheric polar vortex at a subseasonal timescale of 15 days in advance. To achieve this, we employed both statistical and neural network machine learning techniques. The analysis was performed on 42 winter seasons of reanalysis data provided by NASA giving us a total of 6,342 days of data. The state of the polar vortex for determined by using geometric moments to calculate the centroid latitude and the aspect ratio of an ellipse fit onto the vortex. Timeseries for thirty additional precursors were calculated to help improve the predictive capabilities of the algorithm. Feature importance of these precursors was performed using random forest to measure the predictive importance and the ideal number of precursors. Then, using the precursors identified as important, various statistical methods were tested for predictive accuracy with random forest and nearest neighbor performing the best. An echo state network, a type of recurrent neural network that features sparsely connected hidden layer and a reduced number of trainable parameters that allows for rapid training and testing, was also implemented for the forecasting problem. Hyperparameter tuning was performed for each methods using a subset of the training data. The algorithms were trained and tuned on the first 41 years of data, then tested for accuracy on the final year. In general, the centroid latitude of the polar vortex proved easier to predict than the aspect ratio across all algorithms. Random forest outperformed other statistical forecasting algorithms overall but struggled to predict extreme values. Forecasting from echo state network suggested a strong predictive capability past 15 days, but further work is required to fully realize the potential of recurrent neural network approaches.

54 ENVIRONMENTAL SCIENCES↗

NASA NTRSDate not supplied

Li-ion Battery Aging with Hybrid Physics-Informed Neural Networks and Fleet-wide Data

In this work, we propose a hybrid model for Li-ion battery discharge and aging prediction that leverages fleet-wide data to predict future capacity drops.The model is built upon an hybrid approach merging physics-based and empirical equations, as well as neural network models in a recurrent neural network cell. The hybrid physics-informed neural network can predict voltage discharge cycles given the loading profile, and estimate the used capacity of the battery under random-loading conditions by tracking aging parameters connected to the residual capacity of the battery. By merging information on the battery aging parameters with existing fleet-wide aging data, the model can predict the future residual capacity of the battery that is being monitored, and therefore enable predictions of voltage discharge curves far ahead in the battery life cycle. We validated the approach using the NASA Prognostics Data Repository Battery data-set, which contains experimental data on Li-ion batteries discharged at random loading conditions in a controlled environment. The approach also allows the identification of discrepancies between the battery aging trend and the trend observed at the fleet level, so that batteries behaving differently from the rest of the fleet can be subject to closer monitoring and further testing to refine predictions.

PINN↗

DOE OSTI2025

Echo state network for coarsening dynamics of charge density waves

An echo state network (ESN) is a type of reservoir computer that uses a recurrent neural network with a sparsely connected hidden layer. Compared with other recurrent neural networks, one great advantage of ESN is the simplicity of its training process. Yet, despite the seemingly restricted learnable parameters, ESN has been shown to successfully capture the spatial-temporal dynamics of complex patterns. Here we build an ESN to model the coarsening dynamics of charge-density waves (CDWs) in a semiclassical Holstein model, which exhibits a checkerboard electron density modulation at half-filling stabilized by a commensurate lattice distortion. The inputs to the ESN are local CDW order parameters in a finite neighborhood centered around a given site, while the output is the predicted CDW order of the center site at the next time step. Special care is taken in the design of couplings between hidden layer and input nodes to ensure lattice symmetries are properly incorporated into the ESN model. Since the model predictions depend only on CDW configurations of a finite domain, the ESN is scalable and transferrable in the sense that a model trained on dataset from a small system can be directly applied to dynamical simulations on larger lattices. Furthermore, our work opens avenues for efficient dynamical modeling of pattern formations in functional electron materials.

2-dimensional systems↗

DOE OSTI2026

Reduced‐Order Modeling of Energetic Materials Using Physics‐Aware Recurrent Convolutional Neural Networks in a Latent Space (LatentPARC)

Physics-aware deep learning (PADL) has gained popularity for use in spatiotemporal dynamics simulations, such as those in computational modeling of energetic materials (EM). We show that the challenge PADL methods face while learning complex field evolution problems can be simplified and accelerated by decoupling it into two tasks: learning complex geometric features in evolving fields and modeling dynamics over these features in a lower-dimensional feature space. We build upon our previous work on physics-aware recurrent convolutional neural networks (PARC). PARC embeds knowledge of underlying physics into its neural network architecture for more robust and accurate prediction of evolving physical fields. PARC was shown to effectively learn complex nonlinear features such as the formation of hotspots and coupled shock fronts in various initiation scenarios of EMs, as a function of microstructures, serving effectively as a microstructure-aware burn model. Here, we further accelerate PARC and reduce its computational cost by projecting the original dynamics onto a lower-dimensional invariant manifold, or “latent space.” The projected latent representation encodes the complex geometry of evolving fields (e.g., temperature and pressure) in a set of data-driven features. The reduced dimension of this latent space allows us to learn the dynamics during the initiation of EM with a lighter and more efficient model. We observe a significant decrease in training and inference time while maintaining results comparable to PARC at inference. This work takes steps towards enabling rapid prediction of EM thermomechanics at larger scales and characterization of EM structure–property–performance linkages at a full application scale.

Mathematics and Computing↗

DOE OSTI2022

Data coverage assessment on neural network based digital twins for autonomous control system

We report in a recently developed Nearly Autonomous Management and Control (NAMAC) system, neural networks (NNs) are used to develop digital twins for diagnosis (DT-Ds). However, NNs are not usually considered extrapolation models and may result in large errors if they are applied to unseen data outside the training data (uncovered). In this study, we propose a data coverage assessment (DCA) to determine if the NN-based DT-Ds are extrapolated based on their epistemic uncertainty. The uncertainty quantification algorithms and uncertainty thresholds are selected based on the confusion matrix of classifying evaluation data into covered or uncovered data. To demonstrate the adaptability of the proposed framework, we applied it to a basic feedforward neural network and a more advanced recurrent neural network based on a more nonlinear database. Case studies show that the proposed framework can distinguish unseen data for both basic and advanced applications with proper uncertainty quantification algorithms and thresholds.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

DOE OSTI2022

Dynamic machine learning-based optimization algorithm to improve boiler efficiency

With decreasing computational costs, improvement in algorithms, and the aggregation of large industrial and commercial datasets, machine learning is becoming a ubiquitous tool for process and business innovations. Machine learning is still lacking applications in the field of dynamic optimization for real-time control. This work presents a novel framework for performing constrained dynamic optimization using a recurrent neural network model combined with a metaheuristic optimizer. The framework is designed to augment an existing control system and is purely data-driven, like most industrial Model Predictive Control applications. Several recurrent neural network models are compared as well as several metaheuristic optimizers. Hyperparameters and optimizer parameters are tuned with parameter sweeps, and the resulting values are reported. Further, the best parameters for each optimizer and model combination are demonstrated in closed-loop control of a dynamic simulation, and several recommendations are made for generalizing this framework to other systems. Up to 0.953% improvement is realized over the non-optimized case for a simulated coal-fired boiler. While this is not a large improvement in percentage, the total economic impact is $991,000 per year, and this study builds a foundation for future machine learning with dynamic optimization.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

DOE OSTI2023

Machine Learning-Based Model Predictive Control of Two-Time-Scale Systems

In this study, we present a general form of nonlinear two-time-scale systems, where singular perturbation analysis is used to separate the dynamics of the slow and fast subsystems. Machine learning techniques are utilized to approximate the dynamics of both subsystems. Specifically, a recurrent neural network (RNN) and a feedforward neural network (FNN) are used to predict the slow and fast state vectors, respectively. Moreover, we investigate the generalization error bounds for these machine learning models approximating the dynamics of two-time-scale systems. Next, under the assumption that the fast states are asymptotically stable, our focus shifts toward designing a Lyapunov-based model predictive control (LMPC) scheme that exclusively employs the RNN to predict the dynamics of the slow states. Additionally, we derive sufficient conditions to guarantee the closed-loop stability of the system under the sample-and-hold implementation of the controller. A nonlinear chemical process example is used to demonstrate the theory. In particular, two RNN models are constructed: one to model the full two-time-scale system and the other to predict solely the slow state vector. Both models are integrated within the LMPC scheme, and we compare their closed-loop performance while assessing the computational time required to execute the LMPC optimization problem.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2023

Performance analysis and comparison of data-driven models for predicting indoor temperature in multi-zone commercial buildings

Building thermal models, which characterize the properties of a building’s envelope and thermal mass, are essential for accurate indoor temperature and cooling/heating demand prediction. Because of their flexibility and ease of use, data-driven models are increasingly used. Here, this study compared and analyzed the performance of gray-box (resistance-capacitance) and black-box (recurrent neural network) models for predicting indoor air temperature in a real multi-zone commercial building. The developed resistance-capacitance model served as a benchmark model for which full sets of temporal data and building information were used as inputs. The recurrent neural network models were trained and tested assuming various available types and amounts of temporal data and known building physical information to investigate the effects of data and information availability. Feature importance analysis was conducted to select the key variables for different prediction targets under different scenarios. This research provides guidance in selecting an appropriate building thermal response modeling method based on the measured data availability, building physical information, and application.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

DOE OSTI2025

Link Scheduling in Satellite Networks via Machine Learning Over Riemannian Manifolds

Low Earth Orbit (LEO) satellites play a crucial role in enhancing global connectivity, serving a complementary solution to existing terrestrial systems. In wireless networks, scheduling is a vital process that allocates time-frequency resources to users for interference management. However, LEO satellite networks face significant challenges in scheduling their links towards ground users due to the satellites’ mobility and overlapping coverage. This paper addresses the dynamic link scheduling problem in LEO satellite networks by considering spatio-temporal correlations introduced by the satellites’ movements. The first step in the proposed solution involves modeling the network over Riemannian manifolds, thanks to their representation as symmetric positive definite matrices. We introduce two machine learning (ML)-based link scheduling techniques that model the dynamic evolution of satellite positions and link conditions over time and space. To accurately predict satellite link states, we present a recurrent neural network (RNN) over Riemannian manifolds, which captures spatio-temporal characteristics over time. Furthermore, we introduce a separate model, the convolutional neural network (CNN) over Riemannian manifolds, which captures geometric relationships between satellites and users by extracting spatial features from the network topology across all links. Simulation results demonstrate that both RNN and CNN over Riemannian manifolds deliver comparable performance to the fractional programming-based link scheduling (FPLinQ) benchmark. Remarkably, unlike other ML-based models that require extensive training data, both models only need 30 training samples to achieve over 99% of the sum rate while maintaining similar computational complexity relative to the benchmark.

42 ENGINEERING↗

DOE OSTI2025

Applying Machine Learning and Bayesian Inference to Identify and Locate Moving Anthropogenic Sources Using Distributed Acoustic Sensing Data

Distributed acoustic sensing (DAS) systems, which use existing telecommunication fibers, offer high‐resolution capabilities ideal for recording anthropogenic sources. However, the complexity of urban environments and the large amount of data recorded by DAS require automated methods to efficiently detect and categorize anthropogenic sources. Here, we evaluate how well three machine learning models (k‐nearest neighbor [k‐NN], convolutional neural networks, and recurrent‐convolutional neural networks) can identify various anthropogenic sources recorded by DAS. Our findings reveal that both k‐NN and neural network methods perform well in high signal‐to‐noise ratio (SNR) settings. However, their accuracy decreases at SNRs <4. We also use Kalman filtering, a form of Bayesian inference, on backprojected locations of these sources to recover locations that generally fall within standard smartphone Global Positioning System errors. By combining machine learning and Kalman filter results, we calculate a multidimensional model of moving anthropogenic sources. These results demonstrate the potential of DAS data in urban seismology for accurately identifying and locating such sources. Depending on the research objectives, these sources can be further studied or filtered out to improve the quality of seismic data for earthquake studies. Such methods provide a valuable tool for urban seismology and seismic hazard analysis.

Luckie, Thomas William [Sandia National Laboratori↗

NASA NTRSDate not supplied

Classification of Notices to Airmen using Natural Language Processing

This paper establishes the feasibility of using Natural Language Processing (NLP) to classify NOTAMs or Notices to Airmen – a pilot messaging framework to gather real-time situational awareness. Present day air mobility operations heavily rely on NOTAMs. However, pilots often have difficulty interpreting NOTAMs due to the sheer volume of inapplicable messages and unclear abbreviations. Using NLP, the presented study analyzes the accuracy of classifying NOTAMs and, thereby, the efficiency of generating actionable interpretations in real time. To this effect, efficacies of four NLP neural network architectures were analyzed, including three Recurrent Neural Networks (RNNs) with GloVe, Word2Vec, and FastText word embeddings, and one trained Bi-Directional Encoder Representations from Transformers (BERT) model. The four neural networks were trained and evaluated on three open-source datasets of varying text lengths, vocabularies, and grammars, taken from e-commerce product descriptions, social media tweets, and unstructured descriptions for data and analytics services on open data marketplaces such as NASA’s Data and Reasoning Fabric (DRF) platform. This provided cross-analysis of each neural network architecture’s performance per text type. The best performing architecture, BERT, was then fine-tuned on a collection of open-source NOTAM data. Post-training, a real-time NOTAM classification service was implemented to draw inference on new NOTAMs using the trained model, which demonstrated close to 99% accuracy in classification. This modular classification service is envisioned to be integrated with a data and analytics delivery platform, such as the DRF, thus availing real-time contextualization of NOTAMs to air mobility clients, humans, and machines for enhanced decision making.

Aiden C. Szeto↗

NASA NTRSDate not supplied

Classification of Notices to Airmen using Natural Language Processing

Aiden Szeto↗

DOE OSTI2024

Third-integer Resonant Extraction Regulation System for Mu2e

A third-integer resonant slow extraction system is being developed for Fermilab's Delivery Ring to deliver protons to the upcoming Mu2e experiment. The timescale of the extraction (or spill) duration is 43 milliseconds, which is extremely short and unprecedented. Additionally, the experiment's strict and challenging requirements on the quality of the spill at this time scale has led to the development of a new Spill Regulation System (SRS) design. The SRS primarily consists of three components - slow regulation, fast regulation, and harmonic content suppressor. Contributions to the first two components of the SRS, i.e., Slow Regulation and Fast Regulation subsystems, will be presented in which new adaptive learning algorithm schemes for the slow regulation of the spill -- validated using particle tracking simulations -- shall be described. In addition to these novel methods for the enhancement of the spill regulation system, results of employing Machine Learning in enhancing the performance of the resonant extraction are also presented. At the forefront of applying ML techniques to solve non-linear accelerator control problems, this work includes optimizing the PID gains as well as the replacement of the traditional PID controller using Recurrent Neural Networks and Gated Recurrent Unit (GRU) ML models to achieve efficiencies greater than a PID controller. Cutting-edge on-going Reinforcement Learning efforts, including an actor-critic family of learning algorithms, to regulate the spill rate will be reviewed, as well as present analytical calculations pertaining the transit time of particles in a third-integer resonant extraction. Detailed numerical investigations and validations of such calculations, the model of which could be exported and reliably used in future analytical modeling of any resonant extraction, are discussed.

43 PARTICLE ACCELERATORS↗

DOE OSTI2024

Performance Comparison of Machine Learning Models for Ultrasonic Nondestructive Evaluation of Alkali-Silica Reaction in Concrete

Alkali-silica reaction (ASR) causes concrete degradation, leading to cracking, rebar corrosion, and reduced structural integrity, which raises safety concerns. Ultrasonic nondestructive evaluation (NDE) effectively assesses concrete properties and monitors ASR progression. However, its deployment and analysis require specialized expertise and subjective interpretation. As computational power increases, artificial intelligence (AI) and machine learning (ML) algorithms are increasingly being used to automate NDE data analysis across various industries for AI-assisted automation. Regulatory agencies are adapting to this technological shift, prompting a need to evaluate current ML technologies’ capabilities and limitations in assessing concrete material properties and damage. This report presents a comparative analysis of four ML regression models for predicting concrete material damage induced by ASR expansion using long-term ultrasonic data monitoring. The models investigated include linear regression (LR), support vector regression (SVR), shallow neural networks (NN), and deep neural networks (DNN). LR, SVR, and shallow NN models use features extracted from ultrasonic signals, whereas the DNN model processes time-domain ultrasonic signals and frequency spectra directly. The study systematically compared the models’ performance from various perspectives, including model input, prediction performance, and generalization ability. The findings indicate significant variability in model performance, with some ML algorithms achieving very high or very low prediction accuracy depending on the preprocessing and feature engineering (extraction and selection) applied. Key insights include the observation that shallow ML models (LR, SVR, and shallow NNs) require meticulous preprocessing and feature extraction to achieve high accuracy. In contrast, the DNN model, although it bypasses the need for feature engineering, necessitates extensive preprocessing to mitigate noise and computational demands. The SVR model emerged as the top performer among the shallow models, and the DNN model exhibited superior performance on specific datasets but struggled with generalization across specimens from different batches. Additionally, the SVR model is sensitive to temperature variations, whereas the DNN model is robust in this regard. Using recurrent neural networks is recommended for future ASR expansion prediction studies. Recurrent neural networks’ inherent ability to capture temporal dependencies and long-term patterns makes them well suited for analyzing sequential ultrasonic monitoring data. Overall, the results and conclusions of this study could provide insights into the capabilities and effectiveness of ML when applied to ultrasonic NDE data and help identify best practices for using ML for ultrasonic NDE of concrete material properties.

36 MATERIALS SCIENCE↗

NASA NTRSDate not supplied

Flight Trajectory Prediction Based on Hybrid-Recurrent Networks

The development of future technologies for the National Airspace System (NAS) will be reliant on a new communications infrastructure capable of managing the limited available spectrum for communications among aircraft and ground systems. Emerging approaches to autonomous allocation of aviation spectrum mostlyrely on machine learning techniques, where 4D (longitude, latitude, altitude, time) trajectory prediction is an important data input to enable real-time resource allocation. This study explores and evaluates effective data sources and deep recurrent neural network techniques when determining flight trajectories. Specifically, data are collected and evaluated in a 100-day and 14-day period. Sources of data include NASA Sherlock Data Warehouse, MIT Lincoln Labs Corridor Integrated Weather Service (CIWS), and assorted NOAA weather datasets. Deep learning models for 4D predictions all utilize a hybrid-recurrent technique. A baseline model is considered via the convolutional-LSTM design from the existing literature. The modified design considers Gated Recurrent Units (GRU), Independently Recurrent Neural Networks (IndRNN), and stand-alone self-attention layers. Results indicatethe effectiveness of LSTM and GRUcells for state-of-the-art data processing (interpolation). Additionally, GRUs may be quickly trained with limited data, allowing for exacting improvements with optimizer selection. Attention mechanisms provide notable performance improvements to convolutional layers and may extend dimensional capabilities of a learning model. Finally, NOAA measurements provide only a supplemental value, requiring support from tailored measurements for Air Traffic Management.

Nathan Schimpf↗

NASA NTRS2023

Prediction of Stiffness and Fatigue Lives of Polymer Matrix Composite Laminates Using Artificial Neural Networks

Machine learning (ML) models are increasingly being used in many engineering fields due to the advancements in ML algorithms and availability of high-speed computing power. One of the most popular ML class of models is artificial neural networks (ANN). ML is increasingly being used in the design and analysis of composite materials and structures, specifically in the constitutive modeling of composite materials with the focus on greatly accelerating multiscale analyses of composite materials and structures through development of surrogate models. Towards that end, both Python and MATLAB-based neural nets have been developed to predict initial stiffness and fatigue life of an eight-ply symmetric polymer matrix composite laminate. Two types of neural networks, a Multilayer Perceptron (MLP) and a Recurrent Neural Network (RNN), have been developed for both platforms. Results show that the both neural net types can provide an excellent estimate of initial stiffness as well as fatigue life of eight-ply symmetric polymer matrix composite laminate. RNNs are better able to capture the shape of the fatigue curve of a laminate. This tool can be very useful for system level studies to obtain an estimate of desired properties and life of PMC composite laminates. The associated surrogate models could also be used in composite multiscale analyses to replace the actual physics-based calculations at lower scales and thereby significantly increase the computational efficiency of such analyses and thus make multiscale analyses a viable industrial tool for large scale structural problems.

Composite↗

NASA NTRSDate not supplied

Stiffness and Fatigue Life Estimator for Polymer Composite Laminates Using Machine Learning

Machine learning (ML) models are increasingly being used in many engineering fields due to the advancements in ML algorithms and availability of high-speed computing power. One of the most popular ML class of models is artificial neural networks (ANN). ML is increasingly being used in the design and analysis of composite materials and structures, specifically in the constitutive modeling of composite materials with the focus on greatly accelerating multiscale analyses of composite materials and structures through development of surrogate models. Towards that end, Python-based neural nets have been developed to predict initial stiffness and fatigue life of an eight-ply symmetric polymer matrix composite laminate. Two types of neural networks, a Multilayer Perceptron (MLP) and a Recurrent Neural Network (RNN), have been established. Results show that both neural net type algorithms can provide an excellent estimate of initial laminate stiffness as well as fatigue life of eight-ply symmetric polymer matrix composite laminates (PMCs). RNNs are better able to capture the shape of the fatigue curve of a laminate. The resulting tool and GUI can be very useful for system level studies to obtain an estimate of desired properties and life of PMC composite laminates. Further, the associated surrogate models can also be used in composite multiscale analyses to replace the actual physics-based calculations at lower scales and thereby significantly increase the computational efficiency of such analyses and thus make micromechanics-based multiscale analyses a viable industrial tool for large scale structural problems.

multiscale analysis↗

NASA NTRS2023

Stiffness and Fatigue Life Estimator for Polymer Composite Laminates Using Machine Learning

Machine learning (ML) models are increasingly being used in many engineering fields due to the advancements in ML algorithms and availability of high-speed computing power. One of the most popular ML class of models is artificial neural networks (ANN). ML is increasingly being used in the design and analysis of composite materials and structures, specifically in the constitutive modeling of composite materials with the focus on greatly accelerating multiscale analyses of composite materials and structures through development of surrogate models. Towards that end, Python-based neural nets have been developed to predict initial stiffness and fatigue life of an eight-ply symmetric polymer matrix composite laminate. Two types of neural networks, a Multilayer Perceptron (MLP) and a Recurrent Neural Network (RNN), have been established. Results show that both neural net type algorithms can provide an excellent estimate of initial laminate stiffness as well as fatigue life of eight-ply symmetric polymer matrix composite laminates (PMCs). RNNs are better able to capture the shape of the fatigue curve of a laminate. The resulting tool and GUI can be very useful for system level studies to obtain an estimate of desired properties and life of PMC composite laminates. Further, the associated surrogate models can also be used in composite multiscale analyses to replace the actual physics-based calculations at lower scales and thereby significantly increase the computational efficiency of such analyses and thus make micromechanics-based multiscale analyses a viable industrial tool for large scale structural problems.

multiscale analysis↗