Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “numerical cyber modeling”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Cyber‐Resilient Distributed Energy Resource Control Algorithms for Smart Distribution Grids

ABSTRACT This paper focuses on the development of cyber‐resilient gradient‐based optimisation algorithms and theoretical proof for grid‐interactive distributed energy resource (DER) control to enable two grid services of virtual power plants (VPPs) dispatch and grid voltage regulation, considering the communication and security impacts. Firstly, the combined DER dispatch and voltage regulation as a real‐time gradient‐based optimisation problem is recapped. Thereafter, we consider a probabilistic traffic model to characterise packet delays and loss in a communication network, and study how the delays enter the process of information exchange among the grid measurement units, local DER controllers and the grid control centre that execute this control algorithm in a coordinated manner. Then, a strategy combining delay thresholds and message update rules is proposed to immunity the asynchrony resulting from the communications traffic and it avoids possible numerical instabilities and sensitivities of the power tracking and voltage regulation capabilities, resulting as cyber‐resilient DER control algorithms. Additionally, their convergence is theoretically proved. Effectiveness of proposed cyber‐resilient algorithms has been validated on the IEEE 37‐bus system in terms of convergence, VPP tracking and voltage regulation performance for smart distribution systems with high penetration of DERs.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2021

Cyber Physical Grid-Interactive Distributed Energy Resources Control for VPP Dispatch and Regulation

This paper presents a cyber-physical algorithm for grid-interactive Distributed Energy Resource (DER) control to enable two features of Virtual Power Plants (VPPs) dispatch and grid voltage regulation, considering the communication and security impacts. We first formulate the DER dispatch problem as a real-time, iterative, and grid-interactive DER control problem. Thereafter, we consider a probabilistic traffic model to characterize packet delays and loss in a communication network, and study how the delays enter the process of information exchange among the grid measurement units, local DER controllers and the grid control center that coordinately execute this dispatch algorithm. Finally, a strategy combining delay threshold and modified message update rules is proposed to immune the asynchrony resulting from the communications network traffic and it avoids possible numerical instabilities and sensitivities of the tracking and regulation capabilities of this DER control algorithm. By implementing the proposed cyber-physical algorithm on the modified IEEE 37-node system, our preliminary results exhibit that the uncertainties of the underlying communications infrastructure must be considered for the VPP tracking and regulation capabilities of any DER in a generic Cyber-Physical System (CPS), because the delayed voltage measurements in the uplink/bi-link cases result in the off-track in VPP dispatch and jittery in voltage regulation.

cyber-physical algorithm↗
DOE OSTI2020

Automated Adversary Emulation for Cyber-Physical Systems via Reinforcement Learning

Adversary emulation is an offensive exercise that provides a comprehensive assessment of a system’s resilience against cyber attacks. However, adversary emulation is typically a manual process, making it costly and hard to deploy in cyber-physical systems (CPS) with complex dynamics, vulnerabilities, and operational uncertainties. In this paper, we develop an automated, domain-aware approach to adversary emulation for CPS. We formulate a Markov Decision Process (MDP) model to determine an optimal attack sequence over a hybrid attack graph with cyber (discrete) and physical (continuous) components and related physical dynamics. We apply model-based and model-free reinforcement learning (RL) methods to solve the discrete-continuous MDP in a tractable fashion. As a baseline, we also develop a greedy attack algorithm and compare it with the RL procedures. We summarize our findings through a numerical study on sensor deception attacks in buildings to compare the performance and solution quality of the proposed algorithms.

Bhattacharya, Arnab↗
DOE OSTI2020

An Interior-Point Solver for Optimal Power Flow Problem Considering Distributed FACTS Devices

In this paper, we propose an AC optimal power flow (ACOPF) model considering distributed flexible AC transmission system (D-FACTS) devices, in which the reactance of D-FACTS equipped lines are introduced as decision variables. This is motivated by increasing interests in using D-FACTS devices to address system operational and cyber-security concerns. First, D-FACTS devices can be incorporated in real-time operations for economic benefits such as managing power congestions and reducing system losses. Second, D-FACTS devices can be utilized by moving target defense (MTD), an emerging concept against cyber-attacks, to prevent attackers from knowing true system configurations. Therefore, system operators can use the proposed ACOPF model to achieve economic benefits and provide the setpoints of D-FACTS devices for MTD at the same time. In addition, we rigorously derive the gradient and Hessian matrices of the objective function and constraints, which are further used to build an interior-point solver of the proposed ACOPF. Numerical results on the IEEE 118-bus transmission system show the validity of the proposed ACOPF model as well as the efficacy of the interior-point solver in minimizing system losses and generation costs.

Liu, Bo↗
DOE OSTI2021

Risk-Averse Bi-Level Stochastic Network Interdiction Model for Cyber-Security Risk Management

Security of cyber networks is crucial; recent severe cyber-attacks have had a devastating effect on many large organizations. The attack graph, which maps the potential attack paths of a cyber network, is a popular tool for analyzing cyber system vulnerability. In this study, we propose a bi-level stochastic network interdiction model on an attack graph to enable a risk-averse, resource constrained cyber network defender to optimally deploy security countermeasures that protect against attackers with an uncertain budget. This risk- averse conditional-value-at-risk (CVaR) model minimizes a weighted sum of the expected maximum loss over all scenarios and the expected maximum loss from the most damaging attack scenarios. We develop a customized constraint and column generation algorithm to solve our model as well as several acceleration techniques to improve the computational efficiency. Numerical experiments demonstrate that the acceleration techniques enable the solution of relatively large problems within a reasonable amount of time: applying all the acceleration techniques also reduces the average computation time of the basic algorithm by 71% for 100-node graphs. Using metrics called mean-risk value of stochastic solution and value of risk-aversion, computational results suggest that our stochastic risk-averse model significantly outperforms deterministic and risk-neutral models when 1) the distribution of attacker budget is heavy-right-tailed and 2) the defender is highly risk-averse.

Bhuiyan, Tanveer H.↗
DOE OSTI2022

Optimizing power system restoration with damaged communications

Utility procedures for power system blackstart and restoration typically assume that energization decisions can be reliably communicated across the grid. In reality, the communications and control network would likely also be affected in power outages, such as those caused by extreme weather events or cyber-attacks. This paper studies the effect of damage to the power system communications and control infrastructure on restoration operations following a blackout. We model the communications infrastructure as a graph, overlaying the power grid, and imposing the requirement that every energized element in the power grid be observable from a control center. We expand on a specialized branch-and-bound algorithm from the literature to optimize the restoration process and devise an initialization heuristic and a rounding heuristic to improve solution speed. We perform numerical experiments on synthetic systems for Illinois and Texas with outages based on a solar flare or hurricane. We compare the results of our specialized branch-and-bound algorithm to the results from (i) the initialization heuristic alone, (ii) a variation of this heuristic that we use as a baseline, and (iii) the restoration optimization for the power system without communications constraints. Here, we find that damage to the communications infrastructure significantly increases the time required to re-energize the grid. Moreover, by simultaneously optimizing communications repairs and grid energization decisions, we are able to re-energize the grid significantly faster than if communications repairs and energization decisions were made independently or with partial coordination, motivating improvements to current industry practice.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Cyber Physical Grid-Interactive Distributed Energy Resources Control for VPP Dispatch and Regulation: Preprint

This paper presents a cyber-physical algorithm for grid interactive DER control to enable two features of Virtual Power Plants (VPPs) dispatch and grid voltage regulation, considering the communication and security impacts. We first formulate the DER dispatch problem as a real-time, iterative, and grid-interactive DER control problem. Thereafter, we consider a probabilistic traffic model to characterize packet delays and loss in a communication network, and study how the delays enter the process of information exchange among the grid measurement units, local DER controllers and the grid control center that coordinately execute this dispatch algorithm. Finally, we propose a cyber-physical grid-interactive DER control algorithm using the previous message strategy. The tracking and regulation capabilities of this proposed algorithm can be made immune to the asynchrony resulting from the communications network traffic. We carry out simulations to show possible numerical instabilities and sensitivities of the tracking and regulation capabilities on the proposed strategy. Our results exhibit that the uncertainties of the underlying communications infrastructure must be considered in the control algorithm for the VPP tracking and regulation capabilities of any DER in a generic Cyber-Physical System (CPS).

co-simulation↗
DOE OSTI2023

Reinforcement Learning of Structured Stabilizing Control for Linear Systems With Unknown State Matrix

This paper delves into designing feedback control gains for a continuous-time linear quadratic regulator (LQR) problem that is constrained to certain predefined structure with unknown state matrix. We bring forth the ideas from reinforcement learning (RL) in conjunction with sufficient stability and performance guarantees in order to design these structured gains using the trajectory measurements of states and controls. Here we first formulate a model-based framework using dynamic programming (DP) to embed the structural constraint to the LQR gain computation in the continuous-time setting, and then subsequently, formulate a policy iteration RL algorithm that can alleviate the requirement of known state matrix in conjunction with maintaining the feedback gain structure. The design enables a distributed learning control design which is necessary for many large-scale cyber-physical systems. Theoretical guarantees are provided for stability and convergence of the structured reinforcement learning (SRL) algorithm. We validate our theoretical results with numerical simulations on a multi-agent networked linear time-invariant (LTI) dynamic system.

42 ENGINEERING↗
DOE OSTI2020

Securing Vehicle Charging Infrastructure

As the US electrifies the transportation sector, cyber attacks targeting vehicle charging could bring consequences to electrical system infrastructure. This is a growing area of concern as charging stations increase power delivery and must communicate to a range of entities to authorize charging, sequence the charging process, and manage load (grid operators, vehicles, OEM vendors, charging network operators, etc.). The research challenges are numerous and are complicated because there are many end users, stakeholders, and software and equipment vendors interests involved. Poorly implemented electric vehicle supply equipment (EVSE), electric vehicle (EV), or grid communication system cybersecurity could be a significant risk to EV adoption because the political, social, and financial impact of cyberattacks - or public perception of such - ripples across the industry and has lasting and devastating effects. Unfortunately, there is no comprehensive EVSE cybersecurity approach and limited best practices have been adopted by the EV/EVSE industry. There is an incomplete industry understanding of the attack surface, interconnected assets, and unsecured interfaces. Thus, comprehensive cybersecurity recommendations founded on sound research are necessary to secure EV charging infrastructure. This project is providing the power, security, and automotive industry with a strong technical basis for securing this infrastructure by developing threat models, determining technology gaps, and identifying or developing effective countermeasures. Specifically, the team is creating a cybersecurity threat model and performing a technical risk assessment of EVSE assets, so that automotive, charging, and utility stakeholders can better protect customers, vehicles, and power systems in the face of new cyber threats.

33 ADVANCED PROPULSION SYSTEMS↗
DOE OSTI2023

Role of reinforcement learning for risk-based robust control of cyber-physical energy systems

Critical infrastructures such as cyber-physical energy systems (CPS-E) integrate information flow and physical operations that are vulnerable to natural and targeted failures. Safe, secure, and reliable operation and control of CPS-E is critical to ensure societal well-being and economic prosperity. Automated control is key for real-time operations and may be mathematically cast as a sequential decision-making problem under uncertainty. Emergence of data-driven techniques for decision making under uncertainty, such as reinforcement learning (RL), have led to promising advances for addressing sequential decision-making problems for risk-based robust CPS-E control. However, existing research challenges include understanding the applicability of RL methods across diverse CPS-E applications, addressing the effect of risk preferences across multiple RL methods, and development of open-source domain-aware simulation environments for RL experimentation within a CPS-E context. This article systematically analyzes the applicability of four types of RL methods (model-free, model-based, hybrid model-free and model-based, and hierarchical) for risk-based robust CPS-E control. Problem features and solution stability for the RL methods are also discussed. We demonstrate and compare the performance of multiple RL methods under different risk specifications (risk-averse, risk-neutral, and risk-seeking) through the development and application of an open-source simulation environment. Motivating numerical simulation examples include representative single-zone and multizone building control use cases. Finally, six key insights for future research and broader adoption of RL methods are identified, with specific emphasis on problem features, algorithmic explainability, and solution stability.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2025

Resilient State Recovery Using Prior Measurement Support Information

Resilient state recovery of cyber-physical systems has attracted much research attention due to the unique challenges posed by the tight coupling between communication, computation, and the underlying physics of such systems. By modeling attacks as additive adversary signals to a sparse subset of measurements, this resilient recovery problem can be formulated as an error correction problem. To achieve exact state recovery, most existing results require less than 50% of the measurement nodes to be compromised, which limits the resiliency of the estimators. In this paper, we show that observer resiliency can be further improved by incorporating data-driven prior information. Here, we provide an analytical bridge between the precision of prior information and the resiliency of the estimator. By quantifying the relationship between the estimation error of the weighted ℓ 1 observer and the precision of the support prior, this quantified relationship provides guidance for the estimator’s weight design to achieve optimal resiliency. Several numerical simulations and an application case study are presented to validate the theoretical claims.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2024

A graph embedding‐based approach for automatic cyber‐physical power system risk assessment to prevent and mitigate threats at scale

Abstract Power systems are facing an increasing number of cyber incidents, potentially leading to damaging consequences to both physical and cyber aspects. However, the development of analytical methods for the study of large‐scale power infrastructures as cyber‐physical systems is still in its early stages. Drawing inspiration from machine‐learning techniques, the authors introduce a method inspired by the principles of graph embedding that is tailored for quantitative risk assessment and the exploration of possible mitigation strategies of large‐scale cyber‐physical power systems. The primary advantage of the graph embedding approach lies in its ability to generate numerous random walks on a graph, simulating potential access paths. Meanwhile, it enables capturing high‐dimensional structures in low‐dimensional spaces, facilitating advanced machine‐learning applications, and ensuring scalability and adaptability for comprehensive network analysis. By employing this graph embedding‐based approach, the authors present a structured and methodical framework for risk assessment in cyber‐physical systems. The proposed graph embedding‐based risk analysis framework aims to provide a more insightful perspective on cyber‐physical risk assessment and situation awareness for power systems. To validate and demonstrate its applicability, the method has been tested on two cyber‐physical power system models: the Western System Coordinating Council (WSCC) 9‐Bus System and the Illinois 200‐Bus System , thereby showing its advantages in enhancing the accuracy of risk analysis and comprehensiveness of situational awareness.

Sun, Shining↗
DOE OSTI2023

Automated Adversary-in-the-Loop Cyber-Physical Defense Planning

Security of cyber-physical systems (CPS) continues to pose new challenges due to the tight integration and operational complexity of the cyber and physical components. To address these challenges, this article presents a domain-aware, optimization-based approach to determine an effective defense strategy for CPS in an automated fashion—by emulating a strategic adversary in the loop that exploits system vulnerabilities, interconnection of the CPS, and the dynamics of the physical components. Our approach builds on an adversarial decision-making model based on a Markov Decision Process (MDP) that determines the optimal cyber (discrete) and physical (continuous) attack actions over a CPS attack graph. The defense planning problem is modeled as a non-zero-sum game between the adversary and defender. We use a model-free reinforcement learning method to solve the adversary’s problem as a function of the defense strategy. We then employ Bayesian optimization (BO) to find an approximate best-response for the defender to harden the network against the resulting adversary policy. This process is iterated multiple times to improve the strategy for both players. We demonstrate the effectiveness of our approach on a ransomware-inspired graph with a smart building system as the physical process. Numerical studies show that our method converges to a Nash equilibrium for various defender-specific costs of network hardening.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Man‐in‐the‐middle attacks and defence in a power system cyber‐physical testbed

Abstract Man‐in‐The‐Middle (MiTM) attacks present numerous threats to a smart grid. In a MiTM attack, an intruder embeds itself within a conversation between two devices to either eavesdrop or impersonate one of the devices, making it appear to be a normal exchange of information. Thus, the intruder can perform false data injection (FDI) and false command injection (FCI) attacks that can compromise power system operations, such as state estimation, economic dispatch, and automatic generation control (AGC). Very few researchers have focused on MiTM methods that are difficult to detect within a smart grid. To address this, we are designing and implementing multi‐stage MiTM intrusions in an emulation‐based cyber‐physical power system testbed against a large‐scale synthetic grid model to demonstrate how such attacks can cause physical contingencies such as misguided operation and false measurements. MiTM intrusions create FCI, FDI, and replay attacks in this synthetic power grid. This work enables stakeholders to defend against these stealthy attacks, and we present detection mechanisms that are developed using multiple alerts from intrusion detection systems and network monitoring tools. Our contribution will enable other smart grid security researchers and industry to develop further detection mechanisms for inconspicuous MiTM attacks.

Wlazlo, Patrick↗
DOE OSTI2025

Self-Supervised and Interpretable Anomaly Detection Using Network Transformers

Machine learning and deep neural networks (DNNs) have been proposed as a tool to identify anomalies in computer network communications. However, due the obfuscated nature of off-the-shelf machine learning models, their output often does not provide enough information to isolate the source of the anomaly to take corrective measures. In this article, we introduce the network transformer (NeT), a DNN model for anomaly detection that incorporates the graph structure of the communication network in order to improve interpretability. Further, the presented approach has the following advantages: first, enhanced interpretability by incorporating the graph structure of computer networks; second, provides a hierarchical set of features that enables analysis at different levels of granularity; second, self-supervised training that does not require labeled data. The NeT model was evaluated on a set of anomalous scenarios executed in a real industrial control system. The presented approach successfully identified the anomalies, the devices affected, and the specific connections causing the anomalies, providing a data-driven hierarchical approach to analyze the behavior of a cyber network.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

A Secure Learning Control Strategy via Dynamic Camouflaging for Unknown Dynamical Systems under Attacks

This paper presents a secure reinforcement learning (RL) based control method for unknown linear time-invariant cyber-physical systems (CPSs) that are subjected to compositional attacks such as eavesdropping and covert attack. We consider the attack scenario where the attacker learns about the dynamic model during the exploration phase of the learning conducted by the designer to learn a linear quadratic regulator (LQR), and thereafter, use such information to conduct a covert attack on the dynamic system, which we refer to as doubly learning-based control and attack (DLCA) framework. We propose a dynamic camouflaging based attack-resilient reinforcement learning (ARRL) algorithm which can learn the desired optimal controller for the dynamic system, and at the same time, can inject sufficient misinformation in the estimation of system dynamics by the attacker. The algorithm is accompanied by theoretical guarantees and extensive numerical experiments on a consensus multi-agent system and on a benchmark power grid model.

Mukherjee, Sayak↗