Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “REINFORCEMENT”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,045 records · Page 58
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Optimal Coordination of Electric Vehicles for Grid Services using Deep Reinforcement Learning

Recent research has shown the effectiveness of reinforcement learning (RL) in coordinating electric vehicles (EVs) with vehicle-to-grid capabilities for grid services. However, many of these studies rely on lookup table and deep Q-network techniques, which can be impractical when dealing with continuous states and actions. In addition, existing RL designs inadequately account for battery aging effects, EV user satisfaction, uncertain departure and arrival time, and trip distance, which may compromise effective coordination. This paper aims to bridge these gaps by developing an innovative deep deterministic policy gradient-based RL framework for optimal coordination of EVs. Case studies were carried out using a test system with 100 EVs, and numerical analysis results showed that the proposed RL framework can effectively coordinate EVs to maximize economic benefits and user satisfaction while ensuring the expected battery lifespan.

Das, Avijit↗
DOE OSTI2024

Reinforcement Learning-Based Approach for EMT Automation of Large-Scale PV Plants

In the pursuit of efficient and precise modeling of large-scale power systems, particularly utility-scale photovoltaic (PV) plants, Electromagnetic Transient (EMT) simulations play a crucial role. As utility-scale PV plants increase in size and complexity, traditional computational methods become inadequate, necessitating more advanced techniques. This paper highlights the progressive efforts made to accelerate EMT simulations. A novel continuous reinforcement learning (RL) strategy is explored to automate the differentiation and categorization of stiff and non-stiff differential algebraic equations (DAEs). The use of stiff and non-stiff integration methods applied to relevant parts of the DAEs assists with the speed-up of the simulations. The paper details the data acquisition, development and offline training of the RL model, leading to its validation that demonstrates a high precision in optimizing simulation methods. The proposed RL promises to significantly enhance the efficacy of EMT simulations, offering a robust framework for the future of power system analysis.

Xia, Qianxue↗
DOE OSTI2025

A Sequential Model Predictive and Deep Reinforcement Learning-Based Controller for Distribution System Outage Mitigation under Hurricane Events

This paper proposes a proactive outage mitigation framework for power distribution networks to withstand hurricane-induced disruptions. It leverages Model Predictive Control (MPC) to identify safe lines for proactive switching during hurricanes, minimizing the risk of cascading failures and voltage violations. The switching strategies optimized by MPC are sequentially integrated with a Deep Reinforcement Learning agent using the Advantage Actor-Critic algorithm, enabling dynamic line switching to maximize connected buses and minimize voltage violations in real time. Using a probabilistic hurricane model, the framework predicts line failures and adapts to varying conditions to enhance grid resilience. Simulations on the IEEE 123-bus system demonstrate its effectiveness in maintaining high connectivity and minimizing disruptions. Real-time testing with an RTDS confirms the practicality and reliability of the proposed approach.

Selim, Alaa [University of Connecticut]↗
DOE OSTI2025

Reinforcement Learning-Based Secondary Control Strategy for Voltage and Frequency Regulation in Islanded Inverter-Based Microgrids

This paper presents a reinforcement learning (RL) approach for secondary voltage and frequency control in islanded inverter-based microgrids. The proposed control strategy aims to restore voltage and frequency deviations caused by the primary droop control while ensuring proper power sharing between distributed generators. The RL agent is designed to provide correction signals to the primary control, considering communication delays and system constraints. The effectiveness of the proposed control strategy is validated through simulation results in MATLAB/Simulink environment, demonstrating superior performance in maintaining voltage and frequency within the nominal values.

Rodriguez Martinez, Omar Felipe [University of Pue↗
DOE OSTI2020

RLScheduler: An Automated HPC Batch Job Scheduler Using Reinforcement Learning

Today’s high-performance computing (HPC) platforms are still dominated by batch jobs. Accordingly, effective batch job scheduling is crucial to obtain high system efficiency. Existing HPC batch job schedulers typically leverage heuristic priority functions to prioritize and schedule jobs. But, once configured and deployed by the experts, such priority functions can hardly adapt to the changes of job loads, optimization goals, or system settings, potentially leading to degraded system efficiency when changes occur. To address this fundamental issue, we present RLScheduler, an automated HPC batch job scheduler built on reinforcement learning. RLScheduler relies on minimal manual interventions or expert knowledge, but can learn high-quality scheduling policies via its own continuous ‘trial and error’. We introduce a new kernel-based neural network structure and trajectory filtering mechanism in RLScheduler to improve and stabilize the learning process. Through extensive evaluations, we confirm that RLScheduler can learn high-quality scheduling policies towards various workloads and various optimization goals with relatively low computation cost. Moreover, we show that the learned models perform stably even when applied to unseen workloads, making them practical for production use.

Zhang, Di↗
DOE OSTI2025

Topology-Aware Reinforcement Learning for Voltage Control: Centralized and Decentralized Strategies

Volt-VAR control (VVC) methods based on deep reinforcement learning (DRL) can effectively control distribution grid voltage and minimize power loss by implementing corrective and preventive control measures on the reactive power output of inverter-based distributed energy resources (DERs). However, model-free DRL-based VVC approaches usually cannot capture the important topological feature of the power system since they use a fully-connected network (FCN) to deliver the action. Therefore, this paper proposes a graph convolutional network (GCN)-based DRL approach that can employ the topological information of the network to take better control action for regulating the voltage. Our implementation allows for both centralized and decentralized configurations, utilizing a single agent and multiple agents respectively. Although the centralized GCN-based DRL approach has its advantages of minimizing voltage fluctuation and power loss, it is not suitable for large scale power systems due to its challenges in terms of scalability, computation speed and potential single points of failure. Therefore, these problems can be resolved using the decentralized GCN-based DRL approach. Moreover, to ensure the safe operation of the model, our proposed approach incorporates an exponential barrier function while formulating the reward function for each agent. To validate performance of the proposed approaches, the proposed model is tested on modified IEEE test systems and the performances are measured in terms on voltage fluctuation reduction, minimization of power loss and computational speed. Finally, the results show that the proposed topology-aware approach outperforms the FCN-based DRL approach in terms of reducing voltage fluctuation and minimizing power loss of the network. Moreover, it is shown that the decentralized GCN-based DRL has faster computational speed than other approaches.

42 ENGINEERING↗
DOE OSTI2026

Safe Deep Reinforcement Learning for Robust Frequency and Voltage-Constrained Networked Microgrid Restoration

Here, this paper proposes a safe soft actor-critic reinforcement learning (RL) algorithm–based controller for networked microgrid restoration. It formulates the post black-start start as a finite-horizon constrained Markov decision process. The RL agent co-optimizes real and reactive power set-points for both grid-forming and grid-following inverters under explicit voltage and frequency constraints, while enforcing proper power sharing via the Mean Active Power Sharing Index (MPSI) and Mean Reactive Power Sharing Index (MQSI). Numerical results obtained on the IEEE 123-bus distribution system show that the proposed method achieves a mean voltage build-up time of 0.01 s without breaching the 5% sharing-violation budget under various load scenarios, considering MPSI and MQSI indices. These findings demonstrate that the proposed method yields fast and safe black-start schedules without resorting to heuristic penalties.

Selim, Alaa [Dartmouth College, Hanover, NH (Unite↗
DOE OSTI2025

Deep Reinforcement Learning-Based Control of Energy Storage for Interarea Oscillation Damping

With the increasing electricity consumption and lack of transmission investment, today's power systems are operated much closer to their limits, raising concerns of inter-area oscillations that deteriorate the system stability. Here, this article presents a novel energy storage placement and control approach for enhanced damping of interarea oscillations. Combining the residual analysis and dominant mode analysis, we are able to identify the advantageous locations for placing energy storage that achieve improved damping performance. To overcome the challenges, such as fixed control parameters and insufficient damping, we propose to use a deep reinforcement learning-based approach for energy storage control. A state-of-the-art guided surrogate-gradient-based evolutionary strategy is used to train a learning agent in a robust, efficient, and reproducible manner. Parallel computing is also adopted to speed up the training process. The proposed strategy has been tested on both medium and large-scale systems. The proposed methods have demonstrated their effectiveness in mitigating various interarea oscillations within a timeframe of 20 s, thereby averting system collapse and enhancing power grid stability effectively.

25 ENERGY STORAGE↗
DOE OSTI2023

Challenges and Opportunities in Deep Reinforcement Learning With Graph Neural Networks: A Comprehensive Review of Algorithms and Applications

Deep reinforcement learning (DRL) has empowered a variety of artificial intelligence fields, including pattern recognition, robotics, recommendation-systems, and gaming. Similarly, graph neural networks (GNN) have also demonstrated their superior performance in supervised learning for graph-structured data. In recent times, the fusion of GNN with DRL for graph-structured environments has attracted a lot of attention. Here, this paper provides a comprehensive review of these hybrid works. These works can be classified into two categories: (1) algorithmic enhancement, where DRL and GNN complement each other for better utility; (2) application-specific enhancement, where DRL and GNN support each other. This fusion effectively addresses various complex problems in engineering and life sciences. Based on the review, we further analyze the applicability and benefits of fusing these two domains, especially in terms of increasing generalizability and reducing computational complexity. Finally, the key challenges in integrating DRL and GNN, and potential future research directions are highlighted, which will be of interest to the broader machine learning community.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Multi-Agent Graph-Attention Deep Reinforcement Learning for Post-Contingency Grid Emergency Voltage Control

Grid emergency voltage control (GEVC) is paramount in electric power systems to improve voltage stability and prevent cascading outages and blackouts in case of contingencies. While most deep reinforcement learning (DRL)-based paradigms perform single agents in a static environment, real-world agents for GEVC are expected to cooperate in a dynamically shifting grid. Moreover, due to high uncertainties from combinatory natures of various contingencies and load consumption, along with the complexity of dynamic grid operation, the data efficiency and control performance of the existing DRL-based methods are challenged. To address these limitations, we propose a multi-agent graph-attention (GATT)-based DRL algorithm for GEVC in multi-area power systems. Here, we develop graph convolutional network (GCN)-based agents for feature representation of the graph-structured voltages to improve the decision accuracy in a data-efficient manner. Furthermore, a cutting-edge attention mechanism concentrates on effective information sharing among multiple agents, synergizing different-sized subnetworks in the grid for cooperative learning. We address several key challenges in the existing DRL-based GEVC approaches, including low scalability and poor stability against high uncertainties. Test results in the IEEE benchmark system verify the advantages of the proposed method over several recent multi-agent DRL-based algorithms.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2021

Graph Convolutional Network-Based Topology Embedded Deep Reinforcement Learning for Voltage Stability Control

Topological variations in power system is a common phenomenon and can impose significant challenges to traditional controllers of power system. Recent study revealed the strength of deep reinforcement learning (DRL) based approaches in power system preventive and corrective control. But topological variations are difficult to capture using classical fully connected neural network (FCN) model and has not been explicitly modeled in previous work. Hence, we develop a Graph Convolutional Network (GCN) based DRL framework to tackle topology changes in control design of power system. The GCN model exploits the graph structure of the power network and helps the DRL agent to embed the topology information during learning process. Our GCN based approach is evaluated using the IEEE-39 bus system and it outperforms the FCN-based DRL scheme in terms of training convergence and control performance considering grid topology changes.

Hossain, Ramij Raja↗
DOE OSTI2021

Deep Reinforcement Scheduling of Energy Storage Systems for Real-time Voltage Regulation in Unbalanced LV Networks with High PV Penetration

The ever-growing higher penetration of distributed energy resources (DERs) in low-voltage (LV) distribution systems brings both opportunities and challenges to voltage support and regulation. This paper proposes a deep reinforcement learning (DRL)-based scheduling scheme of energy storage systems (ESSs) to mitigate system voltage deviations in unbalanced LV distribution networks. The ESS-based voltage regulation problem is formulated as a multi-stage quadratic stochastic program, with the objective of minimizing the expected total daily voltage regulation cost while satisfying operational constraints. While existing voltage regulation methods are mostly focused on onetime- step control, this paper explores a day-horizon systemwide voltage regulation problem. In other words, the size of action and state spaces are extremely high-dimensional and need to be delicately handled. Furthermore, in order to overcome the difficulty of modeling uncertainties and develop a realtime solution, a learn-to-schedule feedback control framework is proposed by adapting the problem to a model-free DRL setting. The proposed algorithm is tested on a customized 6-bus system and a modified IEEE 34-bus system. Simulation results validate the effectiveness and near-optimality of voltage regulation by ESS in comparison with a deterministic quadratic program solution.

Wang, Shengyi↗
DOE OSTI2013

Dispersion and thermal analysis of carbon nanotube reinforced AA 4032 alloy produced by high energy ball milling

In this work, multi-walled carbon nanotubes (MWNTs) were successfully reinforced in AA 4032 nanocrystalline matrix. The elemental powders of AA 4032 composites were mechanically alloyed in the high energy ball mill for 30 h to get nanocrystalline structure. The carbon nanotubes (CNTs) were added at the 29th hour of ball milling process. The morphological changes during milling were studied, using scanning electron microscope analysis and the distribution of the CNTs within the aluminum matrix was successfully revealed, using transmission electron microscope analysis. The results show that the CNTs are effectively distributed within the aluminum matrix without any structural damage. The variation in the melting point was studied using simultaneous thermal analyzer.

77 NANOSCIENCE AND NANOTECHNOLOGY↗
DOE OSTI2013

Measuring delamination severity of glass fiber-reinforced epoxy composites during drilling process

Glass fiber-reinforced epoxy composites are one of the potential lightweight structural materials used in various engineering applications due to its excellent properties. Drilling is most widely applied for fastening the composite structures; nevertheless, the damage induced by this operation may reduce the component performance drastically. To establish the damage level, delamination is measured quantitatively using digital imaging techniques. In this study, to quantify the delamination severity effectively, a new refined delamination factor (F{sub DR}) is proposed and validated using experimental results observed from three-point bend tests (3PT) and modified short beam shear tests. The value of determined refined delamination factor (F{sub DR}) is more accurate compared to the calculated conventional (F{sub D}) and adjusted (F{sub DA}) delamination factors.

36 MATERIALS SCIENCE↗
DOE OSTI2020

Mechanical and Corrosion Assessment of Friction Self-Piercing Rivet Joint of Carbon Fiber-Reinforced Polymer and Magnesium Alloy AZ31B

In the present work, thermoset carbon fiber–reinforced polymer (CFRP) was spot joined to magnesium alloy AZ31B by a friction self-piercing riveting (F-SPR) process. Lap shear tensile and cross-tension testing were used to evaluate the mechanical joint performance. An average lap shear tensile load of 5.18 kN was achieved, while an average of 2.81 kN was found from cross-tension testing. All F-SPR samples showed a pullout of AZ31B after mechanical testing, indicating good mechanical interlocking between the steel rivet and AZ31B. Corrosion potential was measured for each material to establish the galvanic corrosion characteristics. As expected, AZ31B was found to be the most active, while thermoset CFRP was the most noble. The steel rivet fell between the AZ31B (active) and the thermoset CFRP (noble). Salt fog corrosion testing (ASTM B-117) was performed to evaluate the corrosion performance of the uncoated F-SPR joint. With up to 200 h of exposure, the post-corroded F-SPR joint integrity retained 81.2% of the pre-exposure F-SPR joint strength with AZ31B pullout failure mode. From cross-sectional analysis of the F-SPR joint, extensive corrosion of AZ31B was observed at the joint and other exposure areas. However, steel rivet was not significantly corroded due to sacrificial anode effect by which AZ31B corroded first in the galvanic couple.

36 MATERIALS SCIENCE↗
DOE OSTI2020

MECHANICAL AND CORROSION ASSESSMENT OF FRICTION SELF-PIERCING RIVET JOINT OF CARBON FIBER-REINFORCED POLYMER AND MAGNESIUM ALLOY AZ31B

In the present work, thermoset carbon fiber–reinforced polymer (CFRP) was spot joined to magnesium alloy AZ31B by a friction self-piercing riveting (F-SPR) process. Lap shear tensile and cross-tension testing were used to evaluate the mechanical joint performance. An average lap shear tensile load of 5.18 kN was achieved, while an average of 2.81 kN was found from cross-tension testing. All F-SPR samples showed pull-out of AZ31B after mechanical testing, indicating good mechanical interlocking between the steel rivet and AZ31B. Corrosion potential was measured for each material to establish the galvanic corrosion characteristics. As expected, AZ31B was found to be the most active, while thermoset CFRP was the most noble. The steel rivet fell between the AZ31B (active) and the thermoset CFRP (noble). Salt fog corrosion testing (ASTM B-117) was performed to evaluate the corrosion performance of the uncoated F-SPR joint. With up to 200 h of exposure, the post-corroded F-SPR joint integrity retained 81.2% of the pre-exposure F-SPR joint strength with AZ31B pull-out failure mode. From cross-sectional analysis of the F-SPR joint, extensive corrosion of AZ31B was observed at the joint and other exposure areas. However, steel rivet was not significantly corroded potentially due to sacrificial anode effect by which AZ31B corroded first in the galvanic couple.

Lim, Yong Chae↗
DOE OSTI2019

Enhanced piezoresistive sensing of fiber-reinforced composites via embedded nanoparticles

Carbon fiber composites offer outstanding structural performance with high specific strength and are experiencing significant commercial adoption as the fiber price continues to decrease. Composite research efforts now need to focus on creating multifunctional composites, which can offer sensing capabilities in addition to structural attributes. This work focuses on creating multifunctional carbon fiber composites with structural health monitoring capabilities through the integration of piezoresistive nanoparticles on the surface of carbon fiber. Prior research introduced the development of coating silicon carbide nanoparticles on the surface of carbon fiber in a continuous feed-through process to achieve increased SHM sensitivity with enhanced interlaminar strength and tunable mechanical damping properties. One benefit of that coating process is the compatibility with various nanomaterials. This research capitalizes on that benefit by coating different nanoparticles, such as titanium dioxide, on carbon fiber to further enhance the sensing capabilities. A modification to the prior coating process is made in this research to enable significantly higher nanoparticle loading to be achieved. The resulting composites more accurately measure an applied force by responding with a more profound electrical resistance change. This research lays the foundation for efficiently integrating nanoparticles onto fibers leading to homogenously dispersed nanoparticles throughout a fiber reinforced composite for multifunctional performance.

Bowland, Christopher↗
DOE OSTI2022

Multifunctional fiber-reinforced composites for passive sensing and energy harvesting with enhanced mechanical performance

Fiber-reinforced polymer composites (FRPCs) are valid alternatives to metals, especially in the aerospace industry, for their superior strength-to-weight ratio. FRPCs are generally manufactured by stacking layers of fibers and infusing them with epoxy-based adhesives to result in laminates. Such layer-by-layer structure often ensues a poor interlaminar property of the composites. Additionally, FRPCs can sustain complex damage modes because of their inherent anisotropic characteristics. For example, delamination can occur due to low-velocity impact at a subsurface level, which can result in premature catastrophic failures if undetected. Thus, structural monitoring is crucial to identify such damage. Point-based sensors (e.g., strain gauges, accelerometers, among many) can be embedded in the FRPCs for structural monitoring. However, their electrical power demand often embroils their usage. Therefore, the main objective of this research is to design and implement multifunctional composites that can simultaneously perform passive sensing and energy harvesting while exhibiting better mechanical performance. Previous research efforts have demonstrated that a continuous feed-through deposition of functional materials on fiber surfaces simultaneously enhanced the mechanical and sensing properties of the FRPCs. This study explores a similar approach to encode passive sensing and energy harvesting properties in the FRPCs by integrating ferroelectric microparticles on the fiber surfaces. Upon ensuring their superior interlaminar shear strength, sensing and energy harvesting properties were characterized through experimental studies. The outcome is a multifunctional composite fabricated by coating the fibers with functional microparticles through a high throughput, scalable, and low-cost approach that enables passive sensing, energy harvesting, and improved mechanical performance.

Gupta, Sumit↗