Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Adversarial attacks”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Robustness of Deep Learning Classification to Adversarial Input on GPUs: Asynchronous Parallel Accumulation Is a Source of Vulnerability

The ability of machine learning (ML) classification models to resist small, targeted input perturbations—known as adversarial attacks—is a key measure of their safety and reliability. We show that floating-point non associativity (FPNA) coupled with asynchronous parallel programming on GPUs is sufficient to result in misclassification, without any perturbation to the input. Additionally, we show that this misclassification is particularly significant for inputs close to the decision boundary and that standard adversarial robustness results may be overestimated up to 4.6 when not considering machine-level details. We first study a linear classifier, before focusing on standard Graph Neural Network (GNN) architectures and datasets used in robustness assessments. We develop a novel black-box attack using Bayesian optimization to discover external workloads that can change the instruction scheduling which bias the output of reductions on GPUs and reliably lead to misclassification. Motivated by these results, we present a new learnable permutation (LP) gradient-based approach to learning floating-point operation orderings that lead to misclassifications. The LP approach provides a worst-case estimate in a computationally efficient manner, avoiding the need to run identical experiments tens of thousands of times over a potentially large set of possible GPU states or architectures. Finally, using instrumentation-based testing, we investigate parallel reduction ordering across different GPU architectures under external background workloads, when utilizing multi-GPU virtualization, and when applying power capping. Our results demonstrate that parallel reduction ordering varies significantly across architectures under the first two conditions, substantially increasing the search space required to fully test the effects of this parallel scheduler-based vulnerability. These results and the methods developed here can help to include machine-level considerations into adversarial robustness assessments, which can make a difference in safety and mission critical applications.

Shanmugavelu, Sanjif [Maxeler Technologies, a Groq↗
NASA NTRS2018

Engineering Trade-off Considerations Regarding Design-for-Security, Design-for-Verification, and Design-for-Test

The United States government has identified that application specific integrated circuit (ASIC) and field programmable gate array (FPGA) hardware are at risk from a variety of adversary attacks. This finding affects system security and trust. Consequently, processes are being developed for system mitigation and countermeasure application. The scope of this tutorial pertains to potential vulnerabilities and countermeasures within the ASIC/FPGA design cycle. The presentation demonstrates how design practices can affect the risk for the adversary to: change circuitry, steal intellectual property, and listen to data operations. An important portion of the design cycle is assuring the design is working as specified or as expected. This is accomplished by exhaustive testing of the target design. Alternatively, it has been shown that well established schemes for test coverage enhancement (design-for-verification (DFV) and design-for-test (DFT)) can create conduits for adversary accessibility. As a result, it is essential to perform a trade between robust test coverage versus reliable design implementation. The goal of this tutorial is to explain the evolution of design practices; review adversary accessibility points due to DFV and DFT circuitry insertion (back door circuitry); and to describe common engineering trade-off considerations for test versus adversary threats.

Design for reliability (DFR)↗
DOE OSTI2022

Modeling design and control problems involving neural network surrogates

Here, we consider nonlinear optimization problems that involve surrogate models represented by neural networks. We demonstrate first how to directly embed neural network evaluation into optimization models, highlight a difficulty with this approach that can prevent convergence, and then characterize stationarity of such models. We then present two alternative formulations of these problems in the specific case of feedforward neural networks with ReLU activation: as a mixed-integer optimization problem and as a mathematical program with complementarity constraints. For the latter formulation we prove that stationarity at a point for this problem corresponds to stationarity of the embedded formulation. Each of these formulations may be solved with state-of-the-art optimization methods, and we show how to obtain good initial feasible solutions for these methods. We compare our formulations on three practical applications arising in the design and control of combustion engines, in the generation of adversarial attacks on classifier networks, and in the determination of optimal flows in an oil well network.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Quantum adversarial learning for kernel methods

We show that hybrid quantum classifiers based on quantum kernel methods and support vector machines are vulnerable against adversarial attacks, namely small engineered perturbations of the input data can deceive the classifier into predicting the wrong result. Nonetheless, we also show that simple defense strategies based on data augmentation with a few crafted perturbations can make the classifier robust against new attacks. Our results find applications in security-critical learning problems and in mitigating the effect of some forms of quantum noise, since the attacker can also be understood as part of the surrounding environment.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2022

Prediction of Power Measurements Using Adaptive Filters

With the advent of smart grid concept, Internet of Things (IoT) and the deployment of smart meters, the cyberattack threats on power networks have increased due to the use of communication systems that can be accessed by adversaries. Attackers will have the ability to manipulate the outcomes of smart meters which in turn influence the core application of Energy Management System 9EMS): State Estimation (SE). Bad data analytic tools may fail to detect some attacks into measurements. Meanwhile, Machine Learning (ML) solutions have been proposed for detecting False Data Injection (FDI) attacks. However, there is a lack of ML time-series solutions presented in the state-of-the-art that is yet to be complex. In signal processing, time-series solutions do not only consider the signal, but also the statistics of the signal over time. Therefore, in this paper, a machine learning for time-series solutions is presented as an application to model the measurements of the power grid that are used in SE. The presented model takes into account adaptive linear and non-linear filters: Finite Impulse Response (FIR), and Infinite Impulse Response (IIR). The presented models are implemented and performed on the IEEE-118 bus system. The results indicate the advantage of applying those filters over the state-of-the-art machine learning solutions.

Hamad, Khaled↗
DOE OSTI2025

Safe and Robust Binary Classification and Fault Detection Using Reinforcement Learning

In this paper, we propose a learning-based method utilizing the Soft Actor-Critic (SAC) algorithm to train a binary Support Vector Machine (SVM) classifier. This classifier is designed to identify valid input spaces in high-dimensional, highly constrained systems while minimizing the total runtime of offline simulations. The simulations adapt their runtime based on the likelihood that a given training input will be informative to the classifier. Furthermore, we introduce a method for using the trained SAC model to predict whether a desired system input is likely to violate constraints, along with a technique to adjust the input as necessary. Additionally, we explore the potential of this model to detect faults or adversarial attacks within the system. The effectiveness of our approach is demonstrated through various simulations of challenging classification problems and a constrained quadrotor model.

Netter, Josh [Georgia Institute of Technology, Atl↗
DOE OSTI2024

Double Visual Defense

This is the official code for the paper "Double Visual Defense: Adversarial Pre-training and Instruction Tuning for Improving Vision-Language Model Robustness". This code can be used to produce vision language models (VLMs), like LLaVA, with enhanced robustness to adversarial attacks (e.g. jailbreaks).

Bartoldson, Brian [Lawrence Livermore National Lab↗
NASA NTRS2022

Neurosymbolic Hybrid Approach to Driver Collision Warning

There are two main algorithmic approaches to autonomous driving systems: (1) An end-to-end system in which a single deep neural network learns to map sensory input directly into appropriate warning and driving responses. (2) A mediated hybrid recognition system in which a system is created by combining independent modules that detect each semantic feature. While some researchers believe that deep learning can solve any problem, others believe that a more engineered and symbolic approach is needed to cope with complex environments with less data. Deep learning alone has achieved state-of-the-art results in many areas, from complex gameplay to predicting protein structures. In particular, in image classification and recognition, deep learning models have achieved accuracies as high as humans. But sometimes it can be very difficult to debug if the deep learning model doesn't work. Deep learning models can be vulnerable and are very sensitive to changes in data distribution. Generalization can be problematic. It's usually hard to prove why it works or doesn't. Deep learning models can also be vulnerable to adversarial attacks. Here, we combine deep learning-based object recognition and tracking with an adaptive neurosymbolic network agent, called the Non-Axiomatic Reasoning System (NARS), that can adapt to its environment by building concepts based on perceptual sequences. We achieved an improved intersection-over-union (IOU) object recognition performance of 0.65 in the adaptive retraining model compared to IOU 0.31 in the COCO data pre-trained model. We improved the object detection limits using RADAR sensors in a simulated environment, and demonstrated the weaving car detection capability by combining deep learning-based object detection and tracking with a neurosymbolic model.

Wang, Pei↗
DOE OSTI2026

Evaluating lightweight unsupervised online IDS for masquerade attacks in CAN

Vehicular controller area networks (CANs) are susceptible to masquerade attacks by malicious adversaries. In masquerade attacks, adversaries silence a targeted ID and then send malicious frames with forged content at the expected timing of benign frames. As masquerade attacks could seriously harm vehicle functionality and are the stealthiest attacks to detect in CAN, recent work has devoted attention to compare frameworks for detecting masquerade attacks in CAN. However, most existing works report offline evaluations using CAN logs already collected using simulations that do not comply with the domain’s real-time constraints. Here we contribute to advance the state of the art by presenting a comparative evaluation of four different non-deep learning (DL)-based unsupervised online intrusion detection systems (IDS) for masquerade attacks in CAN. Our approach differs from existing comparative evaluations in that we analyze the effect of controlling streaming data conditions in a sliding window setting. In doing so, we use realistic masquerade attacks being replayed from the ROAD dataset. We show that although evaluated IDS are not effective at detecting every attack type, the method that relies on detecting changes in the hierarchical structure of clusters of time series produces the best results at the expense of higher computational overhead. We discuss limitations, open challenges, and how the evaluated methods can be used for practical unsupervised online CAN IDS for masquerade attacks.

Anomaly detection↗
DOE OSTI2025

The Effects of Compounded Model Size Reductions on Adversarial Robustness

Recent advances in Edge AI and Tiny Machine Learning (TinyML) have enabled the deployment of machine learning models on resource-constrained environments. However, deploying these models on edge devices, such as micro-controllers, requires significant model footprint reduction through a variety of techniques such as quantization, pruning, and clustering. While these optimization methods offer considerable advantages, they potentially introduce AI-related security vulnerabilities, particularly concerning model robustness with respect to adversarial AI attacks. Prior research has extensively examined the impact of quantization on adversarial robustness; however, the effects of alternative reduction techniques and their combinations remain understudied. This paper investigates the impact of model size reduction techniques on adversarial robustness, when applied individually and combined. We utilized Fast Gradient Sign Method (FGSM) and Projected Gradient Descent (PGD) attacks to generate adversarial perturbations for both training and testing data, and then evaluated the models' accuracy under adversarial training conditions. Our findings revealed that reduction techniques generally diminished robustness; although, combining techniques was not found to make robustness any worse than when applied individually. Moreover, specific techniques can potentially enhance resistance to small size perturbations. This research provides insights into the trade-offs between model size reduction and security, establishing a foundation for future investigations into improving adversarial training techniques and methodologies for maintaining robustness while preserving memory footprint benefits.

Austria, Phillipe [ORNL] (ORCID:0000000236223973)↗
DOE OSTI2024

Resilient Control of Networked Microgrids Using Vertical Federated Reinforcement Learning: Designs and Real-Time Test-Bed Validations

Improving system-level resiliency of networked microgrids against adversarial cyber-attacks is an important aspect in the current regime of increased inverter-based resources (IBRs). To achieve that, this paper contributes in designing a hierarchical control layer, in conjunction with the existing control layers, resilient to adversarial attack signals. Considering model complexities, unknown dynamical behaviors of IBRs, and privacy issues regarding data sharing in multi-party-owned microgrids, designing such a control layer is non-trivial. Here, to tackle these issues, a novel federated reinforcement learning (Fed-RL) method is proposed. To grasp the interconnected dynamics of networked microgrids, the paper develops Federated Soft Actor-Critic (FedSAC) algorithm following the vertical structure of implementing Fed-RL. Next, utilizing the OpenAI Gym interface, we built a custom set-up in GridLAB-D/HELICS co-simulation platform, named Resilient RL Co-simulation (ResRLCoSIM), to train the RL agents with IEEE 123-bus benchmark comprising 3 interconnected microgrids. Finally, the learned policies in the simulation are transferred to the real-time hardware-in-the-loop (HIL) test-bed developed using the high-fidelity Hypersim platform. Finally, experiments show that the simulator-trained RL controllers achieve desirable performance with the test-bed platform, validating the minimization of the sim-to-real gap.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2022

Autonomous System Subversion Tactics: Prototypes and Recommended Countermeasures

One of the fielding requirements for Advanced and Small Modular Reactors (AR/SMR) is the ability to support remote and autonomous operations. Autonomous Control Systems (ACS) are found on platforms such as Autonomous Space Vehicles, Cruise Missiles, and advanced driver-assistance systems. Each of these ACS implementations depends upon a set of decision support subsystems responsible for supporting Autonomous Mission Managers (names vary based upon field and author preferences). These Autonomous Mission Managers receive inputs from system sensors (e.g., LIDAR collection from an automobile travelling down a street; transients from a nuclear reactor), and perform a set of classifications (e.g., Red Traffic Light; Small Pedestrian at 10m; Load Rejection; Single Coolant Pump Trip), and then use these classifications in combination with recommendation algorithms to achieve platform goals (e.g., Stop the Vehicle at the Traffic Light, Avoid the Small Pedestrian; Trip the Reactor to prevent a Safety Event). The design, implementation, and fielding of an ACS capability will alter the cyber-attack surface such that existing risk management plans will need to be updated to include how to protect and defend against data-science and decision-support-system attack classes. These attack classes would include protection of the design and training environments where algorithm selection and testing and training data would be obvious attack vectors. These attack classes would also require an informed set of detection and response procedures to identify anomalous behaviors and document best practices for anomaly assessment and vulnerability mitigation and remediation. Last year we published a Cyber Threat Assessment Methodology for Autonomous and Remote Operations for AR/SMRs along with a companion publication on Cyber Attack and Defense Use Cases. The focus of the methodology was on describing and enumerating ACS processes, components, and functions such that security engineers could: evaluate subversion options against the target; identify threat actor attributes and capabilities derived from each subversion option; and identify security controls and response countermeasures. The Use Cases document offered detailed methodology examples including an assessment of a Military Base SMR, an Autonomous System Decision Loop, and implementation of AR/SMR Machine Learning algorithms. Our proposal at the end of last year was to focus on implementation of subversion prototypes related to the last Use Case area: AR/SMR Machine Learning (ML) Algorithms. We included six attack scenarios in our Use Cases paper: a Poisoning Attack against ML functions implemented using an FPGA; a Trojaning Attack against ML classifiers exploiting the excitability of Nuclear Engineers; a Backdooring Attack against ML Training environments to ensure persistence of an attack vector; a False Positive Evasion Attack against multi-factor Access Control Systems using clever inputs; an Inference Attack against ML models by an Insider with access to the Operational environment; and an Adversarial Reprogramming Attack against a Material Access Control Video Surveillance System. At the beginning of this year these six attack scenarios were provided to our research teams at Georgia Tech and Idaho State University and each team successfully implemented a subversion attack against a ML implementation to include transient misclassifications. While this is a notable outcome from this type of research, this paper offers the reader insight into not only how to structure and execute these types of attacks, but into the thought process behind how the researcher investigated the problem space, performed initial algorithm implementation, and the trial-and-error behind arriving at the successful subversion prototypes. We include in this paper a set of associated Scenarios on how these subversion prototypes could be implemented and an initial set of guidance for AR/SMR architects, Nuclear Regulators, and Cyber Defenders to implement awareness and defense capabilities into their current operational portfolios.

42 ENGINEERING↗
DOE OSTI2022

ASK: Adversarial Soft k-Nearest Neighbor Attack and Defense

K-Nearest Neighbor (kNN)-based deep learning methods have been applied to many applications due to their simplicity and geometric interpretability. However, the robustness of kNN-based deep classification models has not been thoroughly explored and kNN attack strategies are underdeveloped. In this paper, we first propose an Adversarial Soft kNN (ASK) loss for developing more effective kNN-based deep neural network attack strategies and designing better defense methods against them. Our ASK loss provides a differentiable surrogate of the expected kNN classification error. It is also interpretable as it preserves the mutual information between the perturbed input and the in-class-reference data. We use the ASK loss to design a novel attack method called the ASK-Attack (ASK-Atk), which shows superior attack efficiency and accuracy degradation relative to previous kNN attacks on hidden layers. We then derive an ASK-Defense (ASK-Def) method that optimizes the worst-case ASK training loss. Experiments on CIFAR-10 (ImageNet) show that (i) ASK-Atk achieves ≥13% (≥ 13% ) improvement in attack success rate over previous kNN attacks, and (ii) ASK-Def outperforms the conventional adversarial training method by ≥ 6.9% (≥ 3.5% ) in terms of robustness improvement. Relevant codes are available at https://github.com/wangren09/ASK .

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

The Delicate Balance Redux: The Role of Nuclear Forces, Damage Limitation and Uncertainty in Future U.S.-China Crises

What is the impact of damage limitation capabilities like counterforce and missile defenses on deterrence, when their efficacy in stopping an adversary nuclear attack is uncertain? This is a key unanswered question to understand “how much is enough” for the United States to deter China and Russia in future nuclear crises. In this paper we extend an established, single move game theory model to capture the dynamics of two players in a nuclear crisis having varying damage limitation capabilities with uncertain effectiveness. Our model formalizes the logic of the “delicate balance” school of deterrence, which states that leverage in a crisis is driven by the risk each player can take with their combined strategic forces, and that those risks carry uncertainty as nuclear forces are hard to deliver against technologically advanced adversaries. Our model shows that damage limitation capabilities—even those with significant uncertainty around them like cyber or electronic warfare—can drive bargaining outcomes in an array of nuclear crises. We then apply these bargaining outcomes to the expected U.S.-China strategic balance as China builds out its nuclear force through 2035. We apply published force exchange models to determine the expected damage each side will be able to deliver, and we use these values to determine the likelihood that the U.S. can prevail in crises of varying stakes. Last, we show that U.S. policymakers have an array of options to improve future bargaining outcomes, evaluating how additional nuclear forces trade against improvements in damage limitation.

45 MILITARY TECHNOLOGY, WEAPONRY, AND NATIONAL DEF↗
DOE OSTI2025

Provable Security and Resilience in Critical Infrastructure – Next Steps

With the conclusion of the Laboratory Directed Research and Development (LDRD) project on Provable Security and Resilience (PSaR) in Critical Infrastructure, we present forward-looking technical concepts and strategies that build on the project’s outcomes and INL’s long-standing expertise in infrastructure protection. The challenge is to protect critical infrastructure and functions much more efficiently at scale than capable adversaries can attack at scale. After summarizing progress and ongoing work we’ll discuss what are the challenges that remain and what are new/emerging technologies, strategies, and processes to meet those challenges. Finally, we’ll layout concepts that integrate with other protection work in the coming year and beyond. For example, building secure function-specific platforms based on the seL4 microkernel, and considering the successes of Cyber-Informed Engineering as a model for engage, collaboration, and adoption. We look forward to your feedback and collaboration as we refine and expand this vision.

97 - MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Digital risk analysis in nuclear engineering projects: Designing for safety, performance, reliability, and security

Cyber-informed engineering and security-by-design frameworks are important in promoting the need to identify cybersecurity concerns early in the systems engineering lifecycle so risks from adversarial cyber-attacks can be eliminated or reduced through engineering design practices. In addition to adversarial risk, risk in operational technology systems also includes non-adversarial and unintentional risk from other factors such as human performance errors, environmental conditions, design flaws, and device degradation or failure. This paper introduces a new concept for characterizing digital risk, both adversarial and non-adversarial, and provides the basis for initial research into a novel digital risk analysis approach focused on incorporating attack difficulty into a multi-attribute analysis technique using robust decision-making. This digital risk characterization is also used to frame a discussion on the challenges of competing objectives and competing stakeholder requirements in an integrated energy system project that incorporates a small modular reactor and industrial facility.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2024

Attack Surface of Wind Energy Technologies in the United States

Low cost, reliable electrical energy production from wind relies upon automation and control systems, arguably more so than traditional thermal generation. These same systems, however, can serve as the target of adversaries’ cyber-attacks. Idaho National Laboratory (INL), at the request of the Department of Energy’s (DOE’s) Office of Cybersecurity, Energy Security, and Emergency Response (CESER) and Energy Efficiency and Renewable Energy’s (EERE’s) Wind Energy Technologies Office (WETO), evaluated a generalized wind plant architecture to understand the classes of potential threat actors and the vectors that could enable a cyber-attack. This evaluation explores the attack surface of a representative wind plant, identifying potential methods and vectors that an adversary could leverage to conduct a cyber-attack. Included in this assessment are some recommended mitigations and approaches. Each recommendation requires a full security evaluation, cost/benefit analysis, and risk analysis by each owner and operator.

17 WIND ENERGY↗
DOE OSTI2023

Introduction to ALERT: A User-Interface Tool for Resilient Optimization

As the cyber-physical systems grow in complexity, there is a need for proactive resilience strategies that involve online, adaptive control actions to best prepare for any impending adversarial events. In this technical effort, supported by RD2C LDRD initiative, the project team designed and demonstrated online strategies – referred to as ALERT controls – for proactive and adaptive tuning of existing optimal controls in a microgrid, with quantifiably assured margins of resilience to various cyber-physical adversarial events. This ALERT functionality is made available to the end-users, e.g., the system operators, via an interactive user-interface. The end-users will not only be able to use the interface to visualize the system’s operation under various cyber-physical adversarial scenarios, but also evaluate the amount of tolerance the system has against selected adversarial perturbations of interest (e.g., malfunctioning sensors, suspected attacked measurements) via the adversarial plots. In this technical report, we briefly outline the algorithmic modules of the developed ALERT control technology, and introduce the user-interface tool that allows end-users (e.g., microgrid operators) to enter their system description, specify various operational and resilience requirements, and evaluate the impact of the control decisions via illustrative plots.

97 MATHEMATICS AND COMPUTING↗