Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “cyber attack localization”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3
All sourcesNASA NTRSDOE OSTI
NASA NTRS2017

Detection, Localization, and Tracking of Unauthorized UAS and Jammers

Small unmanned aircraft systems (UASs) are expected to take major roles in future smart cities, for example, by delivering goods and merchandise, potentially serving as mobile hot spots for broadband wireless access, and maintaining surveillance and security. Although they can be used for the betterment of the society, they can also be used by malicious entities to conduct physical and cyber attacks to infrastructure, private/public property, and people. Even for legitimate use-cases of small UASs, air traffic management (ATM) for UASs becomes of critical importance for maintaining safe and collusion-free operation. Therefore, various ways to detect, track, and interdict potentially unauthorized drones carries critical importance for surveillance and ATM applications. In this paper, we will review techniques that rely on ambient radio frequency signals (emitted from UASs), radars, acoustic sensors, and computer vision techniques for detection of malicious UASs. We will present some early experimental and simulation results on radar-based range estimation of UASs, and receding horizon tracking of UASs. Subsequently, we will overview common techniques that are considered for interdiction of UASs.

surveillance↗
DOE OSTI2023

Precursor Analysis Report: SQL Slammer Worm Infection of Davis-Besse Nuclear Power Plant 2003

The SQL Slammer Worm Infection of Davis-Besse Nuclear Power Plant 2003 Precursor Analysis Report leverages publicly available information about Davis-Besse’s 2003 cyber attack and catalogs anomalous observables for each technique employed in the attack. This analysis is based upon the methodology of the Cybersecurity for the Operational Technology Environment (CyOTE) program. On 25 January 2003, the SQL Slammer worm infected more than 90% of vulnerable hosts and crashed the internet in 10 to 15 minutes, making it one of the fastest spreading worms in history. SQL Slammer is a fileless, memory-resident worm that remotely exploits a stack-based buffer overflow vulnerability on local hosts to intensively scan and rapidly self-propagate across the internet. The worm infected approximately 300,000 unpatched hosts running Microsoft Structured Query Language (SQL) Server 2000 or Microsoft Desktop Engine (MSDE) 2000 with SQL Server Resolution Service. The SQL Slammer worm indirectly infected FirstEnergy’s Davis-Besse nuclear power plant by first infecting a consultant’s company network server and then propagating through an external misconfigured connection into Davis-Besse’s site network. The infection caused major network congestion, slow performance, data overloads, and the inability of local hosts to communicate with each other, which eventually caused a loss of availability and a loss of view when the Safety Parameter Display System (SPDS) and Plant Process Computer (PPC) crashed. At the time of the infection, the plant was already offline, the digital monitoring systems had redundant analog backups, and the plant control and safety functions were not affected, so there were no concerns of a safety breach. However, this incident resulted in many lessons learned and spawned important discussions about cybersecurity’s role in nuclear safety and electric power reliability regulation, policy, and guidance. Researchers and analysts identified 10 unique techniques utilized during the attack with a total of 640 observables using MITRE ATT&CK® for Industrial Control Systems. The CyOTE program assesses observables accompanying techniques used prior to the triggering event to identify opportunities to detect malicious activity. If observables accompanying the attack techniques are perceived and investigated prior to the triggering event, earlier comprehension of malicious activity can take place. Eight of the identified techniques used during Davis-Besse cyber attack were precursors to the triggering event. Analysis identified 596 observables associated with these precursor techniques, 428 of which were assessed to have an increased likelihood of being perceived in the 331 days preceding the triggering event. The response and comprehension time could have been reduced if the observables had been identified earlier. The information gathered in this report contributes to a library of observables tied to a repository of artifacts, data sources, and technique detection references for practitioners and developers to support the comprehension of indicators of attack. Asset owners and operators can use these products if they experience similar observables or to prepare for comparable scenarios.

45 MILITARY TECHNOLOGY, WEAPONRY, AND NATIONAL DEF↗
DOE OSTI2023

Maximum-impact Adversary Design for Network-based Control System: A Case Study on Grid-interactive Efficient Buildings

The Internet of Things (IoT) technology has dramatically improved the efficiency of today's building operation and management. By connecting controllable devices into a communication network, control signals can be easily passed to the devices, and operating status can be acquired from measurable ends with minimal effort. However, this all-connected configuration could also expose the network-based control system (NBCS) to malicious actions, such as cyberattacks. One of the common NBCSs is the building automation system. With the promotion of grid-interactive efficient buildings (GEBs), there has been increasing attention on securing the buildings from the network perspective. This research proposes a maximum-impact adversary design framework so that the adversary can provide the most adversarial impact on the controlled system while remaining stealthy. The proposed framework is numerically demonstrated on a network-based building energy and control system. The building energy system is built in a Modelica-based simulation environment and controlled by the state-of-the-art ASHRAE Guideline 36 control sequences. The control commands at the supervisory level, generated from the Guideline 36 controller, are assumed to be sent to local devices through communication networks using the BACnet protocol. Simulation results show that the proposed maximum-impact adversary on such a system can stealthily affect the building system's performance to its maximum extent. It is anticipated that results can be used by researchers and practitioners in the building automation industry to design efficient and robust cyber-attack detection algorithms, especially for stealthy attacks.

Chu, Mengyuan↗
DOE OSTI2021

Peer-to-Peer Energy Management System for Distributed Microgrid Coordination [SWR-21-92]

Resiliency is one of the key challenges in today's power system. Natural disasters and cyber-attacks both can limit communications between microgrids and the central management system. Thus, having a Distributed Microgrid Coordination (DMC) algorithm can improve the system resiliency, which enables the microgrids to operate without communication with the central management system. Peer-to-Peer Energy Management System for Distributed Microgrid Coordination adopts a primal-dual approach, where each microgrid controller keeps a local estimate of the dual variables. The estimate is updated with local measurements and peer-to-peer communication, leading to a fully distributed algorithm. While the DMC is developed for microgrid coordination, it can be used for general distributed control purpose.

Li, Yashen↗
DOE OSTI2023

Robust Decentralized Secondary Control Scheme for Inverter-based Power Networks

Inverter-dominated microgrids are quickly becoming a key building block of future power systems. They rely on centralized controllers that can provide reliability and resiliency in extreme events. Nonetheless, communication failures due to cyber-physical attacks or natural disasters can make autonomous operation of islanded microgrids challenging. This paper examines a unified decentralized secondary control scheme that is robust to inverter clock synchronization errors and can be seamlessly applied to grid-following or grid-forming control architectures. The proposed scheme overcomes the well-known stability problem that arises from parallel operation of local integral controllers. Theoretical guarantees for stability are provided along with criteria to appropriately tune the secondary control gains to achieve good frequency regulation performance while ensuring fair power sharing. The efficacy of our approach is demonstrated through simulations on a 5-bus microgrid with four grid-forming inverters.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2023

Spatio-Temporal Deep Graph Network for Event Detection, Localization, and Classification in Cyber-Physical Electric Distribution System

This work proposes a deep graph learning framework to identify, locate, and classify power, cyber, and cyber power events at the distribution system level. The proposed algorithm jointly exploits spatial, temporal, and node-level cyber and physical data features. The developed graph neural network, together with a deep autoencoder, utilizes physical measurements from distribution level phasor measurement units and cyber data from communication network logs. The spatial structure of the synchrophasor measurements and network is incorporated through a weighted adjacency matrix. The temporal structure is incorporated by defining a spatial operation in the gated recurrent unit. This spatio-temporal learning element resides inside a power event detection, localization, and classification module that provides the degree of confidence for an event label. To accurately pinpoint the location of an event to the nearest bus equipped with a measurement unit, a combination of squared error and proximity score is utilized. Also included is a cyber event detection module that employs heteroskedasticity to analyze the significance of various cyber features during different types of attacks. Finally, a dual-bit cyber-power decision table determines the nature of the event. The proposed method is validated on two distribution systems modeled in OPAL-RT/Hypersim with limited phasor measurement units for different possible physical and cyber events. Further analyses include comparison with other state-of-the-art methods and validation in the presence of measurement noise. As a result, our method outperforms existing approaches and achieves an average detection accuracy of 97.97%, F1-score of 96.88%, precision of 96.53%, and recall of 98.57%.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2024

Data Security Defense: Modeling and Detection of Synchrophasor Data Spoofing Attack for Grid Edge

Data security and cyberattack have become critical issues in the distributed power system where adversaries can swap the source information of sensors or even spoof and alter measurements. However, the cyber security of the power system is challenged by the unpredictability and stealth of the spoofing attacks. Here, to protect the data security at the grid edge, this paper developed a synchrophasor data spoofing attack detection framework based on the time-frequency feature extraction techniques including the short-time Fourier transform (STFT) and object detection network for real-time synchrophasor data categorization and spoofing attack localization. The proposed approach outperforms earlier work in terms of spoofing attack detection and offers a vital localization function employing distributed synchrophasor sensors.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2021

Self-Security for Grid-Interactive Smart Inverters Using Steady-State Reference Model

Smart inverters exchange information with other devices through a shared communication link, making the inverters more prone to receive harmful commands from external parties. This erroneous data can be received due to an anomaly in the system, such as a device fault, unintentional utility operator action, or a cyber-attack. In this paper, a device-level self-security strategy is implemented using reference models for a grid-interactive inverter to examine the incoming power setpoints, detect the anomalies, and protect the system accordingly. The PQ setpoints received from the utility supervisory controller are autonomously examined using the inverter’s normal and stable operating regions before engaging the setpoints to the inverter’s local controller. Grid parameters are estimated in real-time during the examination process. The efficacy of the self-security algorithm is tested using a three-phase 3-kVA SiC-MOSFET inverter and a 12-kW NHR 9410 regenerative grid emulator. The results verify that the proposed method can detect harmful PQ setpoints that can cause abnormal or unstable inverter operation.

Gursoy, Mehmetcan↗
DOE OSTI2023

Robust Resilient Signal Reconstruction under Adversarial Attacks

We consider the problem of signal reconstruction for a system under sparse signal corruption by a malicious agent. The reconstruction problem follows the standard error coding problem that has been studied extensively in the literature. We include a new challenge of robust estimation of the attack support. The problem is then cast as a constrained optimization problem merging promising techniques in the area of deep learning and estimation theory. A pruning algorithm is developed to reduce the "false positive" uncertainty of data-driven attack localization results, thereby improving the probability of correct signal reconstruction. Sufficient conditions for the correct reconstruction and the associated reconstruction error bounds are obtained for both exact and inexact attack support estimation. Moreover, a simulation of a water distribution system is presented to validate the proposed techniques.

Robust, Signal reconstruction, Resilient estimator↗
DOE OSTI2021

A Cyber-Physical Anomaly Detection for Wide-Area Protection Using Machine Learning

Wide-area protection scheme (WAPS) provides system-wide protection by detecting and mitigating small and large-scale disturbances that are difficult to resolve using local protection schemes. As this protection scheme is evolving from a substation-based distributed remedial action scheme (DRAS) to the control center-based centralized RAS (CRAS), it presents severe challenges to their cybersecurity because of its heavy reliance on an insecure grid communication, and its compromise would lead to system failure. This article presents an architecture and methodology for developing a cyber-physical anomaly detection system (CPADS) that utilizes synchrophasor measurements and properties of network packets to detect data integrity and communication failure attacks on measurement and control signals in CRAS. The proposed machine leaning-based methodology applies a rules-based approach to select relevant input features, utilizes variational mode decomposition (VMD) and decision tree (DT) algorithms to develop multiple classification models, and performs final event identification using a rules-based decision logic. Here, we have evaluated the proposed methodology of CPADS using the IEEE 39 bus system for several performance measures (accuracy, recall, precision, and F-measure) in a cyber-physical testbed environment. Furthermore, our experimental results reveal that the proposed algorithm (VMD-DT) of CPADS outperforms the existing machine learning classifiers during noisy and noise-free measurements while incurring an acceptable processing overhead.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2023

Development of A Hardware-In-the-Loop (HIL) Testbed for Cyber-Physical Security in Smart Buildings

As smart buildings move towards open communication technologies, providing access to the Building Automation System (BAS) through the building's intranet, or even remotely through the Internet, has become a common practice. However, BAS was historically developed as a closed environment and designed with limited cyber-security considerations. Thus, smart buildings are vulnerable to cyber-attacks with the increased accessibility. This study introduces the development and capability of a Hardware-in-the-Loop (HIT) testbed for testing and evaluating the cyber-physical security of typical BASs in smart buildings. The testbed consists of three subsystems: (1) a real-time HIL emulator simulating the behavior of a virtual building as well as the Heating, Ventilation, and Air Conditioning (HVAC) equipment via a dynamic simulation in Modelica; (2) a set of real HVAC controllers monitoring the virtual building operation and providing local control signals to control HVAC equipment in the HIL emulator; and (3) a BAS server along with a web-based service for users to fully access the schedule, setpoints, trends, alarms, and other control functions of the HVAC controllers remotely through the BACnet network. The server generates rule-based setpoints to local HVAC controllers. Based on these three subsystems, the HIL testbed supports attack/fault-free and attack/fault-injection experiments at various levels of the building system. The resulting test data can be used to inform the building community and support the cyber-physical security technology transfer to the building industry.

Li, Guowen↗
DOE OSTI2025

Application of Cyber-Informed Engineering for Protecting BESS

This white paper synthesizes an array of crucial grid services provided by BESS technology, assesses its architecture and communications, and presents a case study for analysis against the principles introduced by Cyber-Informed Engineering (CIE). Furthermore, in walking through the analysis, this paper presents a framework to evaluate risks and solutions when considering BESS components. Asset owners and buyers could perform this analysis to assess their BESS product implementations, alternative inverter-based resources (IBR), and energy management systems (EMS). Battery systems fulfill various roles contingent on the unique market demands and the specific challenges presented by regional grid infrastructures. These roles also vary due to the differing utility models for ownership and operation, which are adapted to meet regional and local capabilities and requirements. Concerns have been raised regarding the potential for adversaries to exploit knowledge of battery operational patterns to orchestrate decisive attacks. However, the security of operational data for these systems may not be the primary vulnerability, as much of this information is already well-understood within the community. Applying a modest degree of subject matter expertise can often yield valuable predictions regarding how a battery will respond under certain conditions, such as grid emergencies, high or low-temperature days, Public Safety Power Shutoff (PSPS) events, and outages. The operational characteristics of batteries are well-documented, and their capabilities, including the risks associated with misoperation and the resulting consequences, are published and understood within the industry. CIE practices represent the next step in gaining functional assurance and providing an acceptable level of risk, regardless of whether a battery vendor can support a trusted and validated supply chain. While this issue has exacerbated supply chain challenges, it is not an isolated condition. This foreign supply route is the primary source of BESS for the U.S. market. Significant efforts are underway through the Bipartisan Infrastructure Law (BIL) to change that. Still, strategic short-term operational mitigations are needed to ensure the security of our operational technology (OT) systems, which are enhanced by instilling trust and are separate from vendors implementing CIE principles.

25 ENERGY STORAGE↗
DOE OSTI2025

Cybersecurity Considerations for Hydrogen Infrastructure in Airport Environments

This report explores key cybersecurity concerns and best practices within environments that serve as reference points for the development of hydrogen fueling infrastructure for aviation. This cybersecurity analysis leverages prior NREL studies: 1) hydrogen fueling station component validation to identify vulnerabilities and failure events documented in physical equipment, and 2) electric aircraft charging infrastructure analysis to explore primary cybersecurity vulnerabilities. It reviews the criticality of digitized technologies in sustaining hydrogen fuel production, storage, and fueling systems, noting cybersecurity concerns that are universal to power systems and industrial control systems in general. In considering cybersecurity vulnerabilities within a future landscape of hydrogen energy for aviation applications, a reference architecture was intended to reveal the points of connection between assets and the potential sensors that are vulnerable to manipulation in the event of compromised access or communication within a SCADA system. A generalized reference architecture can help stakeholders, engineers, or strategists understand connections, criticalities, and standard practices when it comes to designing and planning for new systems. There are several gaps to account for in assessing the future of hydrogen production, storage, and fueling for aviation. Engaging stakeholders, including aircraft manufacturers, electric utilities, site property owners, and local communities, will inform decision-making around site structure, operations, and resources for future hydrogen fueling infrastructure to understand operational needs and cybersecurity awareness. Cybersecurity mitigation strategy must consider physical attack vectors that emerge with the integration of hydrogen systems into existing airport security requirements. The cybersecurity risk assessment contained in this report is an entry point into potential future granular-level analyses to be conducted as part of hazard and risk assessments for safe aviation hydrogen infrastructure, determining how the scale of hydrogen fuel infrastructure for aviation impacts the volume of cyber attack vectors, and what, if any, are the vulnerabilities associated with different types of on-board hydrogen systems. In this nascent development phase, assessing how best to integrate cybersecurity practices into an evolving U.S. aviation landscape provides critical insights into building increased awareness and stakeholder engagement to support a cyber-resilient infrastructure.

08 HYDROGEN↗
DOE OSTI2020

Anomaly Detection and Mitigation for Wide-Area Damping Control using Machine Learning

In an interconnected multi-area power system, wide-area measurement based damping controllers are used to damp out inter-area oscillations, which jeopardize grid stability and constrain the power flows below to their transmission capacity. The effect of wide-area damping control (WADC) significantly depends on both power and cyber systems. At the cyber system layer, an adversary can inflict the WADC process by compromising either measurement signals, control signals or both. Stealthy and coordinated cyber-attacks may bypass the conventional cybersecurity measures to disrupt the seamless operation of WADC. This paper proposes an anomaly detection (AD) algorithm using supervised Machine Learning and a model-based logic for mitigation. The proposed AD algorithm considers measurement signals (input of WADC) and control signals (output of WADC) as input to evaluate the type of activity such as normal, perturbation (small or large signal faults), attack and perturbation-and-attack. Upon anomaly detection, the mitigation module tunes the WADC signal and sets the control status mode as either wide-area mode or local mode. The proposed anomaly detection and mitigation (ADM) module works inline with the WADC at the control center for attack detection on both measurement and control signals and eliminates the need for ADMs at the geographically distributed actuators. Here, we consider coordinated and primitive data-integrity attack vectors such as pulse, ramp, relay-trip and replay attacks. The performance of the proposed ADM algorithms was evaluated under these attack vector scenarios on a testbed environment for 2-area 4-machine power system. The ADM module shows effective performance with 96:5% accuracy to detect anomalies.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2023

On the Feasibility of Market Manipulation and Energy Storage Arbitrage via Load-Altering Attacks

Around the globe, electric power networks are transforming into complex cyber–physical energy systems (CPES) due to the accelerating integration of both information and communication technologies (ICT) and distributed energy resources. While this integration improves power grid operations, the growing number of Internet-of-Things (IoT) controllers and high-wattage appliances being connected to the electric grid is creating new attack vectors, largely inherited from the IoT ecosystem, that could lead to disruptions and potentially energy market manipulation via coordinated load-altering attacks (LAAs). In this article, we explore the feasibility and effects of a realistic LAA targeted at IoT high-wattage loads connected at the distribution system level, designed to manipulate local energy markets and perform energy storage (ES) arbitrage. Realistic integrated transmission and distribution (T&D) systems are used to demonstrate the effects that LAAs have on locational marginal prices at the transmission level and in distribution systems adjacent to the targeted network.

25 ENERGY STORAGE↗
DOE OSTI2021

Cybersecurity for Grid Connected eXtreme Fast Charging (XFC) Station (CyberX) (Final Scientific/Technical Report)

This report summarizes the activities conducted under the DOE VTO funded project DE- EE0008451, where ABB Inc. (ABB), in collaboration with Idaho National Laboratory (INL), APS Global (APS), and XOS Trucks (XOS) pursued the development of a cyber-resilient extreme fast charging (XFC) management system. This project entitled Cybersecurity for Grid Connected eXtreme Fast Charging (XFC) Station (CyberX) focuses on a resilient architecture for smart charging EV Supply Equipment (EVSE) device control and Coordinated Anomaly Detection System (CADS) features that can be added at the charging site depot level to increase cybersecurity. The project was split into two budget periods focused first on developing the threat model and resilient control concepts and second on testing, improving, and validating those developed resilient control algorithms and features with a focus on key vulnerabilities identified during the threat assessment portion of the project. During the first budget period of the CyberX project, the ABB led team focused on activities to identify, model, and quantitatively prioritize high-impact attack scenarios with potential cyber-physical effects while also modeling and developing concepts for a resilient control system that could securely address integration of DERs and other resources with EV charging. Development of the security focused XFC management system (XMS) was accomplished first by offline simulation using a developed XFC station or depot with 480V input level and simulating measurement inputs to monitoring and control systems in concept development. A representative distribution grid model was developed supporting an EV charging site model with BESS and 6 general EV charging models. These EV charging models allowed multiple configurations of charging level, multiple connected protection and measurement devices, and simulation function to show general compromise of EV, BESS, and protection features based on parallel threat analysis. During the second budget period, the EV site and supporting systems model was developed in more detail and converted from offline model to real-time to real-time with EV charging hardware in the loop (HIL). The resilient control architecture developed as concept in the first part of the project was further tested and validated for integration of local energy resources and XFC charging station site equipment while maintaining cybersecure operating principles. The proposed resilient architecture for smart charging and cybersecurity features consists of two main concepts developed and tested within the project. The first concept is an XFC management system (XMS) consisting of a hardware gateway, software platform, and Supervisory Control and Data Acquisition (SCADA) or Distribution Management System integration components. The second concept is a Coordinated Anomaly Detection System (CADS) which forms a primarily software-related subsystem of the total CyberX solution focused on monitoring system measurements, estimation of measurement states, and predicting current at the utility point of interaction based on machine learning for anomaly detection.

33 ADVANCED PROPULSION SYSTEMS↗
DOE OSTI2020

Innovation in Radiological Security, Part 1 of 2 – Using a Cloud Solution to Compress the Timely Detection Model

The design and implementation of new security technologies must account for numerous and complex operational challenges. For example, traditionally isolated systems must now survive amid the proliferation of network connectivity and endure the dynamic environments created by organizations tolerating bring-your-own-device policies. When evaluating the cyber, physical, or cyber-physical security of an asset, a common practice in the security industry is to apply the Timely Detection Model (TDM)—a versatile concept that relates the security functions of detection, delay, and response to the progression of an oppositional force’s attack timeline. If implemented correctly, security enhancements can compress the TDM to offer efficiencies to the stakeholders responsible for adjudicating threat scenarios. The efficiencies gained by the response force allow for more effective protection strategies to be realized. One such enhancement, deployed within the radiological security domain—the Sentry-Remote Monitoring System (Sentry-RMS)—is a stand-alone security system that detects, assesses, and communicates priority alarms as a means of thwarting internal and external threats. The SEntry-RMS CommUnications and REsponse (Sentry-SECURE) platform—an optional feature of the Sentry-RMS—is being developed to facilitate more efficient alarm adjudication by site stakeholders and, if necessary, a faster response by law enforcement. This Cloud-hosted platform receives protected information from deployed Sentry-RMS units and relays it to stakeholders that have vested interests in maintaining an elevated level of situational awareness. Operationally, this enables real-time delivery of high-priority alarm and video imagery directly to an identified response stakeholder, such as local law enforcement or site management, via natively developed mobile applications or full platform integration. The Sentry-SECURE platform, as implemented by the U.S. Department of Energy’s Office of Radiological Security, is an example of an innovative security technology that compresses the TDM by (1) enabling a more efficient time to target and (2) better informing a response force’s predetermined tactics, techniques, and procedures. This paper explores the platform’s operational roles, highlights its principal functions, and presents a use case that demonstrates how the platform provides enhanced situational awareness when adjudicating priority alarms.

Sentry-RMS, Timely Detection Model, Sentry-SECURE↗
DOE OSTI2023

Reinforcement Learning Approach to Cybersecurity in Space (RELACSS)

Securing satellite groundstations against cyber-attacks is vital to national security missions. However, these cyber threats are constantly evolving. As vulnerabilities are discovered and patched, new vulnerabilities are discovered and exploited. In order to automate the process of discovering existing vulnerabilities and the means to exploit them, a reinforcement learning framework is presented in this report. We demonstrate that this framework can learn to successfully navigate an unknown network and detect nodes of interest despite the presence of a moving target defense. The agent then exfiltrates a file of interest from the node as quickly as possible. This framework also incorporates a defensive software agent that learns to impede the attacking agents progress. This setup allows for the agents to work against each other and improve their abilities. We anticipate that this capability will help uncover unforeseen vulnerabilities and the means to mitigate them. The modular nature of the framework enables users to swap out learning algorithms and modify the reward functions in order to adapt the learning tasks to various use cases and environments. Several algorithms, viz., tabular Q learning, deep Q networks, proximal policy optimization, advantage actor-critic, generative adversarial imitation learning, are explored for the agents and the results highlighted. The agent learns to solve the tasks in a light-weight abstract environment. Once the agent learns to perform sufficiently well, it can be deployed in a minimega virtual machine environment (or a real network) with wrappers that map abstract actions to software commands. The agent also uses a local representation of the actions called a ‘slot-mechanism’. This allows the agent to learn in a certain network and generalize it to different networks. The defensive agent learns to predict the actions taken by an offensive agent and uses that information to anticipate the threat. This information can then either be used to raise an alarm or to take actions to thwart the attack. We believe that with the appropriate reward design, a representative environment, and action set, this framework can be generalized to tackle other cybersecurity tasks. By sufficiently training these agents, we can anticipate vulnerabilities leading to robust future designs. We can also deploy automated defensive agents that can help secure satellite groundstation and their vital national security missions.

45 MILITARY TECHNOLOGY, WEAPONRY, AND NATIONAL DEF↗