Engineering topics

Gao, Tianlu

Publications and source records attributed to Gao, Tianlu.

5 records

DOE OSTI2024

A Generation-Storage Coordination Dispatch Strategy for Power System Based on Causal Reinforcement Learning

In the backdrop of global energy transformation, power systems integrating high proportions of renewable energy sources are facing unprecedented challenges in operational stability and dispatch efficiency. To address these challenges, this study introduces a generation-storage coordination real-time dispatch strategy based on Causal Power System Dynamic Reinforcement Learning (CPSDRL). Diverging from traditional reinforcement learning approaches, CPSDRL innovatively incorporates causal inference within the state prediction model - the crux of model-based reinforcement learning - thereby establishing the Power Causal Dynamic Model (PCDM). Assisted by the prior knowledge of power systems, the model significantly enhances prediction accuracy and reliability through a two-stage training process. Utilizing PCDM, this study further applies a direct policy search algorithm to optimize the real-time dispatch strategy. Experimental results indicate that the proposed method improves the stability of generation-storage coordination real-time dispatch and exhibits competitive advantages in sample efficiency and computational speed, compared to traditional model-based and model-free reinforcement learning algorithms. This method is expected to enhance the practicality and adaptability of causal reinforcement learning techniques in power system scheduling and control.

causal reinforcement learning↗

DOE OSTI2022

Enhanced Oblique Decision Tree Enabled Policy Extraction for Deep Reinforcement Learning in Power System Emergency Control

Deep reinforcement learning (DRL) algorithms have successfully solved many challenging problems in various power system control scenarios. However, their decision-making process is usually regarded as black-boxes. Furthermore, how DRL models interact with human intelligence remains an open problem. Thus, this paper proposes a policy extraction framework to extract a complex DRL model into an explainable policy. This framework includes three parts: 1) DRL training and data generation. We train an agent for a specific control task and generate data, which contains the control policy of the agent. 2) Policy extraction. We propose an information gain rate based weighted oblique decision tree (IGR-WODT) for DRL policy extraction. 3) Policy evaluation. We define three metrics to evaluate the performance of the proposed approach. A case study for the under-voltage load shedding problem shows that the IGR-WODT presents a performance enhancement compared with DRL, weighted oblique decision tree, and univariate decision tree. The proposed policy extraction method could provide an intuitive explanation of the neural network decision-making process to the dispatchers when making final decisions on power grid operation. Also, the resulted rule-based controller could replace the deep neural network-based controller in many field edge devices with limited computing resources, providing comparable performance.

deep reinforcement learning↗

DOE OSTI2022

Black Swan Event Small-Sample Transfer Learning (BEST-L) and its Case Study on Electrical Power Prediction in COVID-19

The black swan event will usually cause a great impact on the normal operation of society. The scarcity of such events leads to a lack of relevant data and challenges in dealing with related problems. Different situations also make the traditional methods invalid. In this paper, a transfer learning framework and a convolutional neuron network are proposed to deal with the black swan small-sample events (BEST-L). Taking the COVID-19 as a typical black swan event, the BEST-L is utilized to achieve accurate mid-term load forecasting using the relationship between economy and electricity consumption. The experiment results show that the transfer learning model can effectively learn the basic knowledge about the relationship between the adopted input and output data and use a relatively small amount of data during the black swan event to improve the target areas' generalization. The approach and results can provide an effective approach to respond and react to sudden changes quickly and effectively in similar open problems.

black swan event↗

DOE OSTI2021

Blockchain Enabled Intelligence of Federated Systems (BELIEFS): An attack-tolerant trustable distributed intelligence paradigm

In this article, a Blockchain Enabled Intelligence of Federated Systems (BELIEFS) is proposed to conduct cooperative control for the multi-regional large-scale power system with a multi-agents system (MAS). By establishing a two levels blockchain, each regional AI agent can simultaneously manage intra-regional controllers and cooperate with other AI agents. Under the consensus mechanism, the agents, which respectively conducted distributed deep reinforcement learning (DDRL) algorithm in multi-regions, can have the tolerant capability of malicious attacks in their training process. The demonstration of the proposed approach is within a multi-regional large-scale interconnected power system. Under the mode of "centralized dispatching and hierarchical management", this article aims to definite a mathematical model to deal with the control problem of the power systems. With the comparison experiments, the effectiveness and efficiency of our proposed method in the training process are verified. In addition, malicious attacks are set on the main chain and shard chains to verify the attack-tolerant capability. We expect that such approach and results can suggest a new paradigm of attack-tolerant trustable distributed AI deployment.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2021

Applications of Digital Twin System in a Smart City System with Multi-Energy

With the increasing number of electric vehicles (EV), the smart transportation system is becoming more closely related to the smart energy system in a smart city. However, because of the high complexity, dynamics, and large-scale of these two systems, it is challenging to study their operation problems systematically, especially in severe conditions, e.g., the 2021 winter Texas power traffic crisis. With artificial intelligence, federated learning, edge computing, and automatic control, a digital twin based smart city is proposed in this paper and it focuses on two applications: smart transportation and smart energy grid. Both systems are presented in a structured design manner with several components in the digital twin system, which also contains many real-implementations of different scenarios. The proposed digital twin of a smart city provides a potential clue for the problems mentioned above.

artificial intelligence↗