Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Game theory”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Dynamic probabilistic risk assessment and game theory for cyber security risk analysis in nuclear power plants

Nuclear Power Plants and energy systems have become more prone to cyber-attacks with their digitalization and the increased use of smart equipment. Hence, it is important to quantify the risk associated with cyber-attacks in such systems. Dynamic Probabilistic Risk Assessment which involves studying the evolution of a system due to random events and operator and attacker actions during a cyber-attack by employing a physics-based model of the system is a suitable framework to quantify cybersecurity risk in nuclear power plants. In addition to the plant dynamics, it is also important to model the strategies of the attackers and plant operators for an effective cybersecurity risk assessment. Game theory provides a set of necessary tools to model such strategic interactions. In this research, a framework that integrates dynamic probabilistic risk assessment with game theory for cybersecurity risk analysis in nuclear power plants is presented. The mathematical formulation is derived based on the theory of continuous event trees. We propose a game theory based action model, that utilizes physics-based rewards to define the strategies of attackers and operators at every decision epoch. As a case study, the risk associated with cyber-attacks on the digital components in the secondary side of a pressurized water reactor is studied using a reduced order model. A set of attacker actions and a set of operator actions are defined for the system. The operator and attacker interactions were modelled using simultaneous game, their action policies were computed using the concept of mixed strategy Nash equilibrium and the evolution of the system was studied.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Game Theory in Cybersecurity: A Review of Applications, Models, and Challenges

There are no aspects of society left unchanged by the Internet. From commerce to communication, from entertainment to engineering, every industry and sphere of influence exists in a new state thanks to the advent of the information age. This convenience, rarely controverted, comes with capricious actors, who wish to gain access to these systems for myriad reasons, be it financial, political, or mere intellectual gain. This paper presents information that can help readers to better understand the fundamentals of cybersecurity from a game theory perspective, such as coalitional games, Colonel Blotto systems, or how to find Nash Equilibrium. This paper also reviews how game theory can enhance understanding of cybersecurity through cooperative and non-cooperative methods, such as information sharing, defense modeling, and attacker analysis. The authors use the analysis to serve as a bridge for those in either field who need a way to best comprehend the methodologies of their contrasting fields.

Egan, Matthew↗
DOE OSTI2022

Cyber risk assessment and investment optimization using game theory and ML-based anomaly detection and mitigation for wide-area control in smart grids

The electric power grid is increasingly becoming susceptible to cyber attacks that exploit vulnerabilities in the smart grid control, information, and physical layers. Successful cyber attacks can have catastrophic impacts on the social and economic well-being of any nation all over the globe. It has, thus, become imperative to secure the smart grid against such adversarial actions to ensure stable, secure, and reliable operation of the grid. The existing research and industry practices prove to be inadequate in terms of providing pragmatic and effective defense methodologies and measures for long-term cybersecurity planning and real-time cybersecurity for grid operation. For example, existing works lack models that incorporate uncertain behavior of cyber-attackers and pragmatic defense measures for cyber risk assessment and cybersecurity investment optimization which often provide unreliable and strictly qualitative solutions to these problems. At the same time, with the growing number of cyber incidents in the grid, there still exists a need to develop attack-resilient algorithms for wide-area monitoring, protection, and control (WAMPAC) applications like the wide-area voltage control systems (WAVCS) for Flexible AC Transmissions Systems (FACTS) that lack in scalable and feasible solutions from the cybersecurity perspective. This dissertation proposes novel models and methodologies for: (1) Cybersecurity planning, and (2) Cybersecurity for system operation. The cybersecurity planning is achieved through cyber risk assessment and cybersecurity resource investment optimization for long-term cybersecurity of the grid using game theory and attack-defense trees. Cybersecurity for system operation consists of development of cyber anomaly detection and mitigation algorithms for flexible AC transmission system (FACTS) controller-based wide-area voltage control systems (WAVCS) using machine learning (ML), and software defined networking-based moving target defense network routing for achieving real-time cyber-physical security for grid operations. This is followed by hardware-in-the-loop (HIL) implementation and evaluation of these attack prevention, detection, and mitigation algorithms and methodologies showcasing their feasibility in a close to real-world environment. For cybersecurity planning, a novel approach involving a combination of game theory and attack defense trees (ADT) for optimal cybersecurity resource allocation in the smart grid is proposed. This methodology involves modeling of the cyber-physical smart grid substations as ADTs, defining attacker costs, defense costs, and attack probabilities for attack access points. Using game theoretical formulation, optimal defense strategies for the defender of the system to invest cybersecurity resources in the grid are obtained. Additionally, a game-theoretic framework is developed for quantitative cyber-physical risk assessment of the grid under a dynamically changing cyber threat space and uncertain behavior of cyber attackers which is further used to optimize investments in the smart grid's cybersecurity resources. The attacker, defender, and the smart grid system are modeled while incorporating attacker-stochasticity and federal guidelines for smart grid cybersecurity. This allows quantification of threat, vulnerabilities, and attack impact of the grid for quantitative risk assessment. The defender's budget to invest in the security resources in the grid is optimized based on the strategies leading to minimum system risk. The evaluation of the proposed solutions highlight the feasibility for practical implementation of these methodologies and algorithms in the smart grid, while taking the federal requirements and guidelines for smart grid security into consideration. For achieving cybersecurity for system operation, attack prevention, detection, and mitigation algorithms and methodologies are developed specifically for FACTS-based WAVCS. Anomaly detection and mitigation in the WAVCS are achieved using algorithms based on machine learning which involves offline training and testing of ML models with CPS datasets incorporating physics-based features that allow accurate distinction between system faults and cyber attacks. For attack prevention, a methodology based on software defined network (SDN)-based moving target defense (MTD) network routing is proposed that enables prevention of Denial of Service (DoS) type attacks on the smart grid communication system. Subsequently, these methodologies and algorithms are implemented and evaluated on an HIL testbed that allows for real-time attack prevention, detection, and mitigation of emulated cyber attacks on the WAVCS in a close to real-world environment. The results show highly accurate and efficient performance of the implemented algorithms and methodologies with the smart grid system operating within the NERC's system operation limits even in the presence of DoS and data integrity cyber attacks. This work opens up future research opportunities in other directions such as (1) Expanding cybersecurity planning methodologies to real-time cyber contingency analysis with different game formulations; and (2) Applying the cybersecurity for system operation algorithms to broader categories of wide-area control applications.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2025

Game Theory Approaches for System-level Incentive Design

This report presents a generalized Stackelberg game framework for designing and evaluating financial incentives that enhance power system resilience through strategic deployment of distributed energy resources(DERs) under various contingencies. The proposed approach addresses the challenge of coordinating individual community investment decisions to meet system-wide resilience objectives. The framework is demonstrated in a three-community test system subjected to two transmission contingency scenarios: inter-community line failure (Case 1) and complete main grid disconnection (Case 2). In both cases, three incentive levels are compared: a Base case with no financial incentives, and low and high incentive cases. In Case 1, the Base case (no incentives) results in a total installed DER capacity of 217.2 MW, with no load shedding due to alternative routing, but community costs remain high. Increasing incentives raises DER deployment to 286.9 MW, lowers aggregate community costs by $22M annually, and completely avoids the need for costly new transmission line construction. In Case 2, the Base case results in 24.3 MWh of unserved load; introducing incentives eliminates all load shedding and ensures up to 89 MWh of battery storage is available for emergency reserve. These results demonstrate that targeted incentives can dramatically improve grid resilience and cost-effectiveness. The framework thus offers policymakers and system planners a robust tool to quantify and compare the effectiveness of incentive programs for multi-community transmission networks behavior, system resilience, and economic efficiency.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2020

Solving the duck curve in a smart grid environment using a non-cooperative game theory and dynamic pricing profiles

With the intermittency that comes with electricity generation from renewables, utilizing dynamic pricing will encourage the demand-side to respond in a smart way that would minimize the electricity costs and flatten the net electricity demand curve. Determining the optimal dynamic pricing profile that would leverage distributed storage to flatten the curve is a novel idea that needs to be studied. Moreover, the economic feasibility of utilizing distributed electrical energy storage is still not given in the literature. Therefore, in this paper, a novel way of solving a citywide dynamic model using a bilevel programming algorithm is introduced. The problem is developed as a novel non-cooperative Stackelberg game that utilizes air-conditioning systems and electrical storage through the end-users to determine the optimal dynamic pricing profile. The results show that the combined effect of utilizing demand-side air-conditioning systems and distributed storage together can flatten the curve while employing the optimal dynamic pricing profile. An economic study is performed to determine the economic feasibility of 20 different cases with different battery designs and the level of solar penetration. Three metrics were used to evaluate the economic performance of each case: the levelized cost of storage, the levelized cost of energy, and the simple payback period. Most cases had levelized cost of storage values lower than 0.457 $/kWh, which is the lower bound available in the literature. Seven out of 16 cases have a simple payback period shorter than the lifetime of the system (25 years). The case with a 100 MW PV power plant and a battery storage of size 597 MWh, was found to be the most promising case with a simple payback period of 12.71 years for the photovoltaic plant and 19.86 years for the demand-side investments.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2021

Identifying Key Drivers of Wildfires in the Contiguous US Using Machine Learning and Game Theory Interpretation

Abstract Understanding the complex interrelationships between wildfire and its environmental and anthropogenic controls is crucial for wildfire modeling and management. Although machine learning (ML) models have yielded significant improvements in wildfire predictions, their limited interpretability has been an obstacle for their use in advancing understanding of wildfires. This study builds an ML model incorporating predictors of local meteorology, land‐surface characteristics, and socioeconomic variables to predict monthly burned area at grid cells of 0.25° × 0.25° resolution over the contiguous United States. Besides these predictors, we construct and include predictors representing the large‐scale circulation patterns conducive to wildfires, which largely improves the temporal correlations in several regions by 14%–44%. The Shapley additive explanation is introduced to quantify the contributions of the predictors to burned area. Results show a key role of longitude and latitude in delineating fire regimes with different temporal patterns of burned area. The model captures the physical relationship between burned area and vapor pressure deficit, relative humidity (RH), and energy release component (ERC), in agreement with the prior findings. Aggregating the contribution of predictor variables of all the grids by region, analyses show that ERC is the major contributor accounting for 14%–27% to large burned areas in the western US. In contrast, there is no leading factor contributing to large burned areas in the eastern US, although large‐scale circulation patterns featuring less active upper‐level ridge‐trough and low RH two months earlier in winter contribute relatively more to large burned areas in spring in the southeastern US.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

A Cooperative Game Theory-based Approach to Under-frequency Load Shedding Control

This paper proposes a cooperative game theory-based under-frequency load shedding (UFLS) approach for frequency stability and control in power systems. UFLS is a crucial factor for frequency stability and control especially in power grids with high penetration of renewable energy sources and restructured power systems. Conventional UFLS methods, most of which are off-line, usually shed fixed amounts of predetermined loads based on a predetermined schedule which can lead to over or under curtailment of load. This paper presents a co-operative game theory-based two-stage strategy to effectively and precisely determine locations and amounts of loads to be shed for UFLS control. In the first stage, the total amount of loads to be shed, also referred to as deficit in generation or the disturbance power, is computed using the initial rate of change of frequency (ROCOF) referred to the equivalent inertial center. In the second stage, the Shapley value, one of the solution concepts of cooperative game theory, is used to determine load shedding amounts and locations. The proposed method is implemented on the reduced 9-bus 3-machine Western Electricity Coordinating Council (WECC) system and simulated on Real-time Digital Simulators (RTDS). The results show that the proposed UFLS approach can effectively return the system to normal state after disturbances.

42 ENGINEERING↗
DOE OSTI2021

A Cooperative Game Theory-based Secondary Frequency Regulation in Distribution Systems

Participation of distribution systems in frequency regulation has become an important factor for grid operation after the integration of distributed energy resources (DERs). While available reserves from a single distribution system may not be sufficient for frequency regulation, aggregated reserves from several distribution systems can provide frequency regulation at grid-scale. This paper proposes a cooperative game theory-based approach for secondary frequency regulation in distribution systems consisting of distributed energy resources (DERs). A two-stage strategy is proposed to effectively and precisely determine a change in active power set points of DERs following a command from system operators. In the first stage, the value or worth of each DER and their coalitions are determined using initial rates of change of frequency (ROCOF) for each DER/coalition. In the second stage, the Shapley value, one of the solution concepts of cooperative game theory, is used to determine changes in active power set points of DERs. The proposed method is implemented on several distribution systems including a modified IEEE 13-node system and modified 33-node distribution system.

Gautam, Mukesh↗
DOE OSTI2022

Hierarchical Model-Free Transactive Control of Building Loads to Support Grid Services

Residential buildings consume 4.4 quads of electricity annually, approximately 37% of the total electricity consumption in the United States. This represents a vast resource that can be used for demand management and other ancillary services. This project aims to develop a robust, scalable hierarchical transactional control mechanism incorporating elements of model-free control (MFC) and game theory to harness buildings to provide ancillary services to the grid. This approach is being taken to address the challenges of incorporating traditional transactional control schemes into existing buildings. The challenges include small individual building sizes requiring aggregation of many buildings, unpredictable energy usage that makes model identification difficult, and satisfying the sensitive occupant comfort constraints. In the proposed approach, by separating the control mechanism into two layers above and below the load aggregator, MFC can be used below the aggregator to modulate flexible building loads in response to pricing signals with guaranteed performance. This allows the burden of identifying an accurate model of the system to be shifted to the above-aggregator layer, where fluctuations in individual building usage have less impact on predicted building system behavior. Game theory concepts can then be used to determine pricing curves and control signals among regional aggregators. Managing this control in a game-theoretic approach will allow us to build in financial incentives that increase customer engagement. Additionally, the usage of MFC necessitates less burdensome computational and communication requirements, thus, it is easily deployable on small, embedded devices. In a broader sense, developing a strategy capable of effectively incorporating residential and small commercial buildings will allow greater throughput of existing and emerging grid services in addition to future transactive energy grid management methods. Using MFC within a hierarchical control architecture will allow the shifting of existing forecasting challenges to an aggregate level, where dynamics are slower and more predictable. This will enable a smooth interface between the grid services requests of utilities and the reliable control required by participating buildings. MFC, which supports distributed control architecture, permits a scalable solution that can be deployed to neighborhood-size systems as well as individual buildings. This project focuses on three objectives: (1) developing the mathematical framework, algorithm toolkit, and software toolset of the two-layer transactive control testbed; (2) developing a scalable solution for application over many residential and small-size commercial buildings with sparse distributed communication; and (3) field testing and implementation on hardware of the control strategies developed in the previous two objectives. The research and development activities are focused and designed to be impactful within the relevant 2025 targets timeframe. An open-source control framework for exploiting variability and dispatchability of building loads will be delivered as the outcome of the project. This capability enables greater participation of loads in electricity markets and ancillary services that are both useful for the utility and financially beneficial for building owners.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2026

From Modular ADMS to Plug-and-Play Ops: Distribution Grid Operations with Platform-Level Orchestration to Enable Ambitious App Hosting

The core function of the distribution grid is to provide electricity to consumers affordably, reliably, and securely. In pursuing these core objectives, distribution utilities are accountable to customers, regulators, and in some cases, shareholders. Other third parties such as aggregators and microgrids can also have a stake in the smooth operation of the grid. Each of these stakeholders has economic, business, and/or governance objectives that inform their expectations of the distribution grid. This multi-objective, multi-stakeholder environment creates tension that must be reconciled to successfully design and operate the distribution grid. Innovative companies are competing to bring high-tech solutions to electric utilities and their customers that address each of these objectives. Many developers of advanced distribution management systems (ADMS) and distributed energy resource management systems (DERMS) have adopted a modular architecture that allows grid operators to select functions and features according to their individual system needs. A modular platform also allows the solution provider to develop and integrate specific new product modules; however, the need to pursue multiple objectives with a fixed set of controllable devices makes integration expensive whether it is done at the product development stage or the deployment stage. This cost creates a significant barrier to adoption and can lengthen the product to market time of new solutions. To fundamentally address the complexity of system integration for distribution grid operations, the U.S. Department of Energy Office of Electricity has funded the GridAPPS-D project at PNNL, which streamlines integration by contributing to standards development, defining system architecture, applying advanced mathematics, and developing open-source software to demonstrate the concept of an open data-integration platform for distribution operations. The open data-integration platform concept enables system operators and solution providers to deploy ambitious, best-of-breed applications (or apps) without continually reengineering for integration. Ambitious apps developed by different solution providers will inevitably attempt to achieve different control objectives with the same set of controllable devices. If the open platform itself can resolve these conflicts in a way that achieves the best available outcomes for all apps, doesn’t restrict the ambitious design of apps, and ensures safe and secure operations, apps will be able to plug-and-play with the platform at the same time as other ambitious apps. In this paper, we describe a framework called App Deconfliction that empowers a platform to assign setpoints to controllable devices based on the values preferred by different apps (and even external stakeholder entities like customers or aggregators). The App Deconfliction framework is compatible with several methods for determining setpoint values. We present two methods based on game theory that provide a subtle built-in incentive structure for developers to adapt their apps to the fact that they will be operating in a moderated multi-app environment and to favor device setpoints that have the most effect on their objectives over those that have the least effect. Our simulation-based demonstrations have shown that game-theory-based deconfliction can lead to a 7% improvement in control space utilization compared to design-based methods.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2021

Mutual Detectability: A Targeted SETI Strategy That Avoids the SETI Paradox

As our ability to undertake searches for extraterrestrial intelligence (SETI) grows, so does interest in the controversial endeavor of messaging extraterrestrial intelligence (METI). METI proponents point to the SETI Paradox—if all civilizations refrain from METI then SETI is futile. I introduce mutual detectability as a game-theoretic strategy to increase the success potential of targeted SETI. Mutual detectability comprises four laws that establish how SETI participants can engage each other based on mutual evidence of mutual existence. I argue that the party whom both SETI participants can judge to have better quality evidence, or common denominator information (CDI), has an onus to transmit to avoid the SETI Paradox. Transiting exoplanets within the Earth Transit Zone form a target subset that satisfies mutual detectability requirements. I identify the intrinsic time-integrated transit signal strength, which for Earth is 10{sup 3} L{sub ⊙} ppm hours yr{sup −1}, as suitable CDI. Civilizations on habitable-zone planets of radius R{sub p}/R{sub ⊕}≲(L{sub ∗}/L{sub ⊙}){sup −1/7} have superior CDI on us, and so under the mutual detectability framework have game-theory incentive (onus) to transmit. While the onus to transmit falls on us for habitable planets around L{sub *} > L{sub ⊙} stars, considerations of relative stellar frequency, main-sequence lifetime and planet occurrence rates mean that such systems are likely to be in a small minority. Surveys of the Earth Transit Zone for Earth-analog transiting planets around subsolar luminosity hosts would facilitate targeted SETI programs for civilizations who have game-theory incentive to transmit signals to us. A choice to remain silent, by not engaging in METI toward such systems, does not in this case fuel concerns of a SETI Paradox.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2020

Hypergames and Cyber-Physical Security for Control Systems

The identification of the Stuxnet worm in 2010 provided a highly publicized example of a cyber attack that physically damaged an industrial control system. This raised public awareness about the possibility of similar attacks against other industrial targets—including critical infrastructure. Here, we use hypergames to analyze how strategic perturbations of sensor readings and calibrated parameters can be used to manipulate a system that employs optimal control. Hypergames form an extension of game theory that enables us to model strategic interactions where the players may have significantly different perceptions of the game(s) they are playing. Past work with hypergames has focused on relatively simple interactions consisting of a small set of discrete choices for each player. Here, we apply single-stage hypergames to larger systems with continuous variables. We find that manipulating constraints can be a more effective attacker strategy than manipulating objective function parameters. Moreover, the attacker need not change the underlying system to carry out a successful attack—it may be sufficient to deceive the defender controlling the system. It is possible to scale our approach up to even larger systems, but this will depend on the characteristics of the system in question, and we identify several characteristics that will make those systems amenable to hypergame analysis.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

The Kalai-Smorodinsky solution for many-objective Bayesian optimization

An ongoing aim of research in multiobjective Bayesian optimization is to extend its applicability to a large number of objectives. While coping with a limited budget of evaluations, recovering the set of optimal compromise solutions generally requires numerous observations and is less interpretable since this set tends to grow larger with the number of objectives. We thus propose to focus on a specific solution originating from game theory, the Kalai-Smorodinsky solution, which possesses attractive properties. In particular, it ensures equal marginal gains over all objectives. We further make it insensitive to a monotonic transformation of the objectives by considering the objectives in the copula space. A novel tailored algorithm is proposed to search for the solution, in the form of a Bayesian optimization algorithm: sequential sampling decisions are made based on acquisition functions that derive from an instrumental Gaussian process prior. Our approach is tested on four problems with respectively four, six, eight, and nine objectives. The method is available in the R package GPGame available on CRAN at https://cran.r-project.org/package=GPGame.

Game theory↗
DOE OSTI2020

A non-cooperative meta-modeling game for automated third-party calibrating, validating and falsifying constitutive laws with parallelized adversarial attacks

The evaluation of constitutive models, especially for high-risk and high-regret engineering applications, requires efficient and rigorous third-party calibration, validation and falsification. While there are numerous efforts to develop paradigms and standard procedures to validate models, difficulties may arise due to the sequential, manual, and often biased nature of the commonly adopted calibration and validation processes, thus slowing down data collections, hampering the progress towards discovering new physics, increasing expenses and possibly leading to misinterpretations of the credibility and application ranges of proposed models. This work attempts to introduce concepts from game theory and machine learning techniques to overcome many of these existing difficulties. Here, we introduce an automated meta-modeling game where two competing AI agents systematically generate experimental data to calibrate a given constitutive model and to explore its weakness such that the experiment design and model robustness can be improved through competitions. The two agents automatically search for the Nash equilibrium of the meta-modeling game in an adversarial reinforcement learning framework without human intervention. In particular, a protagonist agent seeks to find the more effective ways to generate data for model calibrations, while an adversary agent tries to find the most devastating test scenarios that expose the weaknesses of the constitutive model calibrated by the protagonist. By capturing all possible design options of the laboratory experiments into a single decision tree, we recast the design of experiments as a game of combinatorial moves that can be resolved through deep reinforcement learning by the two competing players. Our adversarial framework emulates idealized scientific collaborations and competitions among researchers to achieve a better understanding of the application range of the learned material laws and prevent misinterpretations caused by conventional AI-based third-party validation. Numerical examples are given to demonstrate the wide applicability of the proposed meta-modeling game with adversarial attacks on both human-crafted constitutive models and machine learning models.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Genetic algorithm for demand response: a stackelberg game approach

Demand response (DR) has gained a significant recent interest due to its potential for mitigating many power system problems. Game theory is a very effective tool to be utilized in DR management. In this paper, the DR between a distribution system operator (DSO) and load aggregators (LAs) is designed as a Stackelberg game, where the DSO acts as the leader and LAs are regarded as the followers. Due to the limitations of the centralized solution approaches, a genetic algorithm-based decentralized approach is proposed. To demonstrate the proposed approach, a case study concerning a day-ahead optimization for a real-time pricing market with a single DSO and three LAs is designed and optimized. The proposed approach is able to shift the demand peaks and prove that it has a great potential to be used for the Stackelberg game between a DSO and multiple LAs to fully exploit the potential of DR.

Amasyali, Kadir↗
DOE OSTI2020

An Analysis of Post Attack Impacts and Effects of Learning Parameters on Vulnerability Assessment of Power Grid

Due to the increasing number of heterogeneous devices connected to electric power grid, the attack surface increases the threat actors. Game theory and machine learning are being used to study the power system failures caused by external manipulation. Most of existing works in the literature focus on one-shot process of attacks and fail to show the dynamic evolution of the defense strategy. In this paper, we focus on an adversarial multistage sequential game between the adversaries of the smart electric power transmission and distribution system. We study the impact of exploration rate and convergence of the attack strategies (sequences of action that creates large scale blackout based on the system capacity) based on the reinforcement learning approach. We also illustrate how the learned attack actions disrupt the normal operation of the grid by creating transmission line outages, bus voltage violations, and generation loss. This simulation studies are conducted on IEEE 9 and 39 bus systems. The results show the improvement of the defense strategy through the learning process. The results also prove the feasibility of the learned attack actions by replicating the disturbances created in simulated power system.

attack impacts↗