DOE OSTI · 3367033
Augmenting LLM-Based Agents for Improved Performance in Pentesting and Commissioning Operational Technology in Critical Infrastructure
Abstract
Artificial intelligence (AI), and more specifically large language models (LLMs) have the potential for use in penetration testing (“pentesting”) against devices, networks, and computer systems in information technology (IT). We explore the possibility of extending pentesting from IT systems to operational technology (OT) systems, which are more obscure than IT systems in their protocols and design. A challenge therefore exists when applying pretrained LLMs to OT systems as corpora are likely to underrepresent OT systems in comparison to other more prevalent systems. We evaluate augmentations of LLMs with various methods, especially retrieval augmented generation (RAG), to improve performance of the LLMs in the OT domain. In addition to pentesting, some of the testing of these OT devices may include commissioning to ensure that the newly installed devices work correctly. Our framework may also be applied in such cases.
Explore related subjects
Keep this discovery
Explore connections, maps & timelines
Werth, Aaron [Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States)] (ORCID:0000000319365627), Hahn, Gary [Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States)] (ORCID:0000000158529765), Borges Hink, Raymond [Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States)] (ORCID:0000000224169594), Piesciorovsky, Emilio [Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States)], Prowell, Stacy J. [Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States)] (ORCID:0000000221045641), Torkelson, Lauren [Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States)] (ORCID:0009000909989082), Oesch, T. Sean [Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States)] (ORCID:0000000269091022), Searle, Justin [ControlThings.io, Hurricane, UT (United States)], Roberts, Jesse [Tennessee Technological Univ., Cookeville, TN (United States)]. 2026-06-01. Augmenting LLM-Based Agents for Improved Performance in Pentesting and Commissioning Operational Technology in Critical Infrastructure. https://doi.org/10.2172/3367033
Cite the original work for its findings. Save a collection to share your selection of sources.