Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “adversarial evaluation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Attack Surface of Wind Energy Technologies in the United States

Low cost, reliable electrical energy production from wind relies upon automation and control systems, arguably more so than traditional thermal generation. These same systems, however, can serve as the target of adversaries’ cyber-attacks. Idaho National Laboratory (INL), at the request of the Department of Energy’s (DOE’s) Office of Cybersecurity, Energy Security, and Emergency Response (CESER) and Energy Efficiency and Renewable Energy’s (EERE’s) Wind Energy Technologies Office (WETO), evaluated a generalized wind plant architecture to understand the classes of potential threat actors and the vectors that could enable a cyber-attack. This evaluation explores the attack surface of a representative wind plant, identifying potential methods and vectors that an adversary could leverage to conduct a cyber-attack. Included in this assessment are some recommended mitigations and approaches. Each recommendation requires a full security evaluation, cost/benefit analysis, and risk analysis by each owner and operator.

17 WIND ENERGY↗
DOE OSTI2026

Evaluating lightweight unsupervised online IDS for masquerade attacks in CAN

Vehicular controller area networks (CANs) are susceptible to masquerade attacks by malicious adversaries. In masquerade attacks, adversaries silence a targeted ID and then send malicious frames with forged content at the expected timing of benign frames. As masquerade attacks could seriously harm vehicle functionality and are the stealthiest attacks to detect in CAN, recent work has devoted attention to compare frameworks for detecting masquerade attacks in CAN. However, most existing works report offline evaluations using CAN logs already collected using simulations that do not comply with the domain’s real-time constraints. Here we contribute to advance the state of the art by presenting a comparative evaluation of four different non-deep learning (DL)-based unsupervised online intrusion detection systems (IDS) for masquerade attacks in CAN. Our approach differs from existing comparative evaluations in that we analyze the effect of controlling streaming data conditions in a sliding window setting. In doing so, we use realistic masquerade attacks being replayed from the ROAD dataset. We show that although evaluated IDS are not effective at detecting every attack type, the method that relies on detecting changes in the hierarchical structure of clusters of time series produces the best results at the expense of higher computational overhead. We discuss limitations, open challenges, and how the evaluated methods can be used for practical unsupervised online CAN IDS for masquerade attacks.

Anomaly detection↗
DOE OSTI2023

Testing and Evaluation of Security Response at High-Consequence Nuclear Facilities

High-consequence nuclear facilities are required to implement a physical protection system (PPS) capable of preventing a design basis threat (DBT) adversary from completing radiological sabotage or theft of nuclear material. Response forces are a core component of the PPS. A rigorous testing and evaluation program is required to ensure that the response forces are appropriately trained and equipped and are capable of meeting the regulatory and mission requirements.

98 NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL P↗
NASA NTRSDate not supplied

Data Augmentation for Intelligent Contingency Management Using Generative Adversarial Neural Networks

Artificial intelligence (AI)-based techniques for intelligent contingency management (ICM) require that intelligent agents learn various aspects of system dynamics to create and execute contingencies. For high assurance contingency management, agents achieve the most compelling results through supervised or semi-supervised machine learning, for which agents require large datasets to learn the dynamics of the system. Unfortunately, data collection in aerospace applications can be costly, due to both time and resources. Presented work describes a framework for data augmentation of ICM databases containing training data for machine learning models. This framework populates the database with the outputs of generative adversarial network (GAN) models that were trained on flight data. Methods for evaluating the suitability of these models based on the equations of motion, as well as other physical constraints, are discussed. The paper demonstrates the utility of this database for training intelligent agents on the NASA T2 generic transport aircraft model and experimental vertical takeoff and landing (VTOL) simulation model.

Generative Machine Learning↗
NASA NTRSDate not supplied

Data Augmentation for Intelligent Contingency Management Using Generative Adversarial Neural Networks

Artificial intelligence (AI)-based techniques for intelligent contingency management (ICM) require that intelligent agents learn various aspects of system dynamics to create and execute contingencies. For high assurance contingency management, agents achieve the most compelling results through supervised or semi-supervised machine learning, for which agents require large datasets to learn the dynamics of the system. Unfortunately, data collection in aerospace applications can be costly, due to both time and resources. Presented work describes a framework for data augmentation of ICM databases containing training data for machine learning models. This framework populates the database with the outputs of generative adversarial network (GAN) models that were trained on flight data. Methods for evaluating the suitability of these models based on the equations of motion, as well as other physical constraints, are discussed. The paper demonstrates the utility of this database for training intelligent agents on the NASA T2 generic transport aircraft model and experimental vertical takeoff and landing (VTOL) simulation model.

Generative Machine Learning↗
DOE OSTI2024

Report on the AAPM grand challenge on deep generative modeling for learning medical image statistics

Abstract Background The findings of the 2023 AAPM Grand Challenge on Deep Generative Modeling for Learning Medical Image Statistics are reported in this Special Report. Purpose The goal of this challenge was to promote the development of deep generative models for medical imaging and to emphasize the need for their domain‐relevant assessments via the analysis of relevant image statistics. Methods As part of this Grand Challenge, a common training dataset and an evaluation procedure was developed for benchmarking deep generative models for medical image synthesis. To create the training dataset, an established 3D virtual breast phantom was adapted. The resulting dataset comprised about 108 000 images of size 512 512. For the evaluation of submissions to the Challenge, an ensemble of 10 000 DGM‐generated images from each submission was employed. The evaluation procedure consisted of two stages. In the first stage, a preliminary check for memorization and image quality (via the Fréchet Inception Distance [FID]) was performed. Submissions that passed the first stage were then evaluated for the reproducibility of image statistics corresponding to several feature families including texture, morphology, image moments, fractal statistics, and skeleton statistics. A summary measure in this feature space was employed to rank the submissions. Additional analyses of submissions was performed to assess DGM performance specific to individual feature families, the four classes in the training data, and also to identify various artifacts. Results Fifty‐eight submissions from 12 unique users were received for this Challenge. Out of these 12 submissions, 9 submissions passed the first stage of evaluation and were eligible for ranking. The top‐ranked submission employed a conditional latent diffusion model, whereas the joint runners‐up employed a generative adversarial network, followed by another network for image superresolution. In general, we observed that the overall ranking of the top 9 submissions according to our evaluation method (i) did not match the FID‐based ranking, and (ii) differed with respect to individual feature families. Another important finding from our additional analyses was that different DGMs demonstrated similar kinds of artifacts. Conclusions This Grand Challenge highlighted the need for domain‐specific evaluation to further DGM design as well as deployment. It also demonstrated that the specification of a DGM may differ depending on its intended use.

Radiology, Nuclear Medicine & Medical Imaging↗
DOE OSTI2021

Side-channel Leakage Assessment Metrics: A Case Study of GIFT Block Ciphers

Determination of an adequate level of security and providing subsequent mechanisms to achieve it, is one of the most pressing problems regarding embedded computing devices. While there are some solutions available for resource-rich computer systems, direct application of these solutions to resource-constrained environments are often unfeasible. The fundamental problem for such resource-constrained systems is the fact that current cryptographic algorithms utilize significant energy consumption and storage overhead. Both the cryptographic algorithm and its physical implementation affect the resilience of a cryptosystem against side-channel attacks. A side-channel attack represents a process that exploits leakages in order to extract sensitive information such as the key. This paper focuses on Correlation Power Analysis (CPA) which is side-channel attack based on the power consumption leakage. In 2016 the U.S. Commerce Department’s National Institute of Standards and Technology (NIST) initiated the call for proposals of new cryptographic algorithms to strengthen the cryptographic defense of networked devices against cyberattacks and to protect the data created by those innumerable device. This work evaluates S-boxes used by NIST candidates PICCOLO, GIFT, and PRESENT, as well as several S-box variants that demonstrated sufficient weaknesses against classical cryptanalysis, for a quantitative comparison in terms of resiliency to CPA attack. Three well-known theoretical metrics are evaluated: transparency order (TO and RTO), nonlinearity, and signal-to-noise (SNR) ratio, aiming to characterize the resistance of these S-boxes against adversaries exploiting physical leakages. Experimental results from attacks on an 8- bit XMEGA were obtained via the ChipWhisperer platform and of all the S-boxes evaluated, GIFT64 with a PICCOLO S-box was found to be the most susceptible to CPA. Results showed that variations in TO and RTO were not sufficient to ensure practical CPA resistance and that among S-boxes with equal non-linearity there were no significant differences in the TO and SNR variants.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Side-channel Leakage Assessment Metrics: A Case Study of GIFT Block Ciphers

Determination of an adequate level of security and providing subsequent mechanisms to achieve it, is one of the most pressing problems regarding embedded computing devices. While there are some solutions available for resource-rich computer systems, direct application of these solutions to resource-constrained environments are often unfeasible. The fundamental problem for such resource-constrained systems is the fact that current cryptographic algorithms utilize significant energy consumption and storage overhead. Both the cryptographic algorithm and its physical implementation affect the resilience of a cryptosystem against side-channel attacks. A side-channel attack represents a process that exploits leakages in order to extract sensitive information such as the key. This paper focuses on Correlation Power Analysis (CPA) which is side-channel attack based on the power consumption leakage. In 2016 the U.S. Commerce Department’s National Institute of Standards and Technology (NIST) initiated the call for proposals of new cryptographic algorithms to strengthen the cryptographic defense of networked devices against cyberattacks and to protect the data created by those innumerable device. This work evaluates S-boxes used by NIST candidates PICCOLO, GIFT, and PRESENT, as well as several S-box variants that demonstrated sufficient weaknesses against classical cryptanalysis, for a quantitative comparison in terms of resiliency to CPA attack. Three well-known theoretical metrics are evaluated: transparency order (TO and RTO), nonlinearity, and signal-to-noise (SNR) ratio, aiming to characterize the resistance of these S-boxes against adversaries exploiting physical leakages. Experimental results from attacks on an 8- bit XMEGA were obtained via the ChipWhisperer platform and of all the S-boxes evaluated, GIFT64 with a PICCOLO S-box was found to be the most susceptible to CPA. Results showed that variations in TO and RTO were not sufficient to ensure practical CPA resistance and that among S-boxes with equal non-linearity there were no significant differences in the TO and SNR variants.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Assessment of Physical Security Modeling and Simulation in the Vulnerability Assessment Process

This report provides a comprehensive assessment of physical security modeling and simulation tools available for use in the vulnerability assessment (VA) process for nuclear facilities. It outlines the historical evolution of VA methodologies, emphasizing the transition from traditional layer-based approaches to a more holistic framework that integrates detection probabilities directly into combat simulations. The document details the critical components of the VA process, including the characterization of targets, threats, and protective measures, as well as the development of adversary scenarios that reflect both insider and outsider threats. It highlights the importance of performance assurance programs, emphasizing the need for continuous evaluation and testing of security systems to ensure their effectiveness against evolving threats. Additionally, the report discusses the significance of utilizing accredited modeling and simulation tools in accredited areas to accurately represent adversary actions and the corresponding responses of protective forces. By establishing a systematic approach to VA, this document aims to enhance the overall security posture of nuclear facilities, ensuring compliance with regulatory standards while effectively mitigating risks associated with potential adversarial actions.

45 MILITARY TECHNOLOGY, WEAPONRY, AND NATIONAL DEF↗
DOE OSTI2021

Quantum-assisted associative adversarial network: applying quantum annealing in deep learning

Abstract Generative models have the capacity to model and generate new examples from a dataset and have an increasingly diverse set of applications driven by commercial and academic interest. In this work, we present an algorithm for learning a latent variable generative model via generative adversarial learning where the canonical uniform noise input is replaced by samples from a graphical model. This graphical model is learned by a Boltzmann machine which learns low-dimensional feature representation of data extracted by the discriminator. A quantum processor can be used to sample from the model to train the Boltzmann machine. This novel hybrid quantum-classical algorithm joins a growing family of algorithms that use a quantum processor sampling subroutine in deep learning, and provides a scalable framework to test the advantages of quantum-assisted learning. For the latent space model, fully connected, symmetric bipartite and Chimera graph topologies are compared on a reduced stochastically binarized MNIST dataset, for both classical and quantum sampling methods. The quantum-assisted associative adversarial network successfully learns a generative model of the MNIST dataset for all topologies. Evaluated using the Fréchet inception distance and inception score, the quantum and classical versions of the algorithm are found to have equivalent performance for learning an implicit generative model of the MNIST dataset. Classical sampling is used to demonstrate the algorithm on the LSUN bedrooms dataset, indicating scalability to larger and color datasets. Though the quantum processor used here is a quantum annealer, the algorithm is general enough such that any quantum processor, such as gate model quantum computers, may be substituted as a sampler.

Wilson, Max (ORCID:0000000207983391)↗
DOE OSTI2026

A Technical Evaluation of Self-Protection Dose Rates in U.S. Domestic Nuclear Security Policy

U.S. nuclear security policy uses radiation self-protection as a basis for reducing material attractiveness, but existing dose-rate criteria may not reflect the time scales of theft or sabotage. This report evaluates whether the commonly cited 1 Gy/h at 1 m criterion can plausibly cause adversary task failure during short-duration malicious acts.

Fritchie, Jacob Wesley [Sandia National Laboratori↗
DOE OSTI2020

TomoGAN: low-dose synchrotron x-ray tomography with generative adversarial networks: discussion

Synchrotron-based x-ray tomography is a noninvasive imaging technique that allows for reconstructing the internal structure of materials at high spatial resolutions from tens of micrometers to a few nanometers. In order to resolve sample features at smaller length scales, however, a higher radiation dose is required. Therefore, the limitation on the achievable resolution is set primarily by noise at these length scales. We present TomoGAN, a denoising technique based on generative adversarial networks, for improving the quality of reconstructed images for low-dose imaging conditions. We evaluate our approach in two photon-budget-limited experimental conditions: (1) sufficient number of low-dose projections (based on Nyquist sampling), and (2) insufficient or limited number of high-dose projections. In both cases, the angular sampling is assumed to be isotropic, and the photon budget throughout the experiment is fixed based on the maximum allowable radiation dose on the sample. Evaluation with both simulated and experimental datasets shows that our approach can significantly reduce noise in reconstructed images, improving the structural similarity score of simulation and experimental data from 0.18 to 0.9 and from 0.18 to 0.41, respectively. Furthermore, the quality of the reconstructed images with filtered back projection followed by our denoising approach exceeds that of reconstructions with the simultaneous iterative reconstruction technique, showing the computational superiority of our approach.

42 ENGINEERING↗
DOE OSTI2023

Clean Energy Cybersecurity Accelerator Cohort 1: Authentication and Authorization

In the 2023 National Cybersecurity Strategy, the Biden-Harris Administration defines the need for a "defensible, resilient digital ecosystem where it is costlier to attack systems than defend them." The strategy cites the Clean Energy Cybersecurity Accelerator (CECA) as an exemplary effort to bolster the security and resilience of clean energy generation. These efforts help "secure the clean energy grid of the future and [generate] security best practices that extend to other critical infrastructure sectors" and promise broad and far-reaching impacts to bridge the capabilities of private industry and the needs of energy production. Cohort 1 of CECA launched in the fall of 2022 with a focus on solutions that provide strong authentication and authorization for industrial control systems to mitigate attacks on the energy grid. Authentication and authorization verify that the identity (authentication) and permissions (authorization) of a user or device are aligned with their assigned roles. Weaknesses in either can have serious repercussions. To assess the strength of Cohort 1's solutions, CECA devised threat scenarios grounded in historical precedents: the CECA team reviewed exploits from real-world case studies of state-sponsored actors to match the assessment's attack paths and targets. Cohort 1 results provided the energy industry, product vendors, and related agencies valuable insights into the efficacy and applicability of solutions in common system configurations under realistic threat scenarios. The results of the assessment highlight points for interrogation and improvement in subsequent technology iterations. CECA's evaluations are part of an ongoing conversation and collaboration to bolster U.S. cyber resilience against adversaries today and in the future.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2021

CCE and Resilience Strategies

Consequence-driven Cyber-informed Engineering (CCE) is a methodology focused on securing the nation’s critical infrastructure systems. Developed at Idaho National Laboratory, CCE starts with the assumption that if a critical infrastructure system is targeted by a skilled and determined adversary, the targeted network can and will be penetrated. This think like the adversary approach provides critical infrastructure owners, operators, vendors and manufacturers with a disciplined methodology to: evaluate complex systems; determine what must be fully safeguarded; and apply proven engineering strategies to isolate and protect an industry’s most critical assets.

42 ENGINEERING↗
DOE OSTI2026

Enhancing Operational Safety via Agentic Dialogue Hazard Identification Analysis

Operational safety in high-stakes domains such as industrial process control, autonomous, and safety-critical systems demand reliable hazard identification. While large language models (LLMs) have shown promise in automating safety analysis tasks, single-turn, monolithic inference is brittle: it lacks the self-correction, deliberation, and contextual refinement that safety engineers apply iteratively. In this paper, we introduce HAZDIAL, a framework that investigates whether structured agentic dialogue (multi-agent, multi-turn interactions) improves the quality of NLP-based hazard identification over single-pass baselines. We systematically compare two dialogue modalities: adversarial debate and constructive discussion, and propose an genetic algorithm-based agentic interaction optimization. We evaluate all configurations against a curated golden dataset using standard classification metrics (accuracy, precision, recall, F1) and a novel dialogue metrics. This work advances the intersection of dialogue systems, multi-agent reasoning, and AI safety, providing empirical evidence for dialogue-driven hazard analysis.

Das, Sanjay [ORNL] (ORCID:0009000542591915)↗
NASA NTRS2020

CyberGAN: Generating High-fidelity Cybersecurity Data With Generative Adversarial Networks

Machine learning for cyber defense offers the promise of detecting adversarial activity against the ground data systems managing critical space assets. A fundamental challenge facing machine learning research in cybersecurity is the lack of high-fidelity, shareable datasets for robust evaluation and testing of machine learning-based solutions. High-fidelity, real-world datasets are necessary for reliable benchmarking of nominal system behavior and malicious activity. Unfortunately, such realistic datasets of both nominal and adversarial activity are rarely shared publicly by data owners due to security and privacy concerns. Besides, the available adversarial data is sparse, which makes training models on malicious activity much harder. This situation has impeded and continues to impede the research and successful adoption of machine learning methods for cyber defense. Researchers have dealt with this problem by generating data within a low-fidelity lab environment, using classified and thus unshareable datasets, or downloading low-fidelity public datasets made available by others. We propose an innovative solution to the problem by employing machine learning methods to generate high-fidelity data. Specifically, we propose the use of Generative Adversarial Networks (GANs) to generate high-fidelity data for cybersecurity purposes. GANs have found successful image processing and natural language applications, but have not yet been investigated for cyber data generation. Our proposed approach first involves training the `discriminator' network of the GAN with a sample of real-world data consisting of malicious and nominal samples. We then use the `generator' network to generate new high-fidelity data samples consisting of an appropriate mix of malicious and nominal activity. We demonstrate applications of our architecture by generating high-fidelity cybersecurity data containing both malicious and nominal samples. We thoroughly evaluate the fidelity of our generated data using heuristics and evaluate its usefulness for machine learning applications using three different datasets. Overall, our approach results in high-fidelity, shareable datasets.

Zhang, Yuening↗
DOE OSTI2021

Bennett-Brassard 1984 quantum key distribution using conjugate homodyne detection

Optical homodyne detection has been widely used in continuous-variable (CV) quantum information processing for measuring field quadrature. In this paper we explore the possibility of operating a conjugate homodyne detection system in “photon counting” mode to implement discrete-variable (DV) quantum key distribution (QKD). A conjugate homodyne detection system, which consists of a beam splitter followed by two optical homodyne detectors, can simultaneously measure a pair of conjugate quadratures X and P of the incoming quantum state. In classical electrodynamics, X 2 + P 2 is proportional to the energy (the photon number) of the input light. In quantum optics, X and P do not commute and thus the above photon-number measurement is intrinsically noisy. This implies that a blind application of standard security proofs of QKD could result in pessimistic performance. We overcome this obstacle by taking advantage of two special features of the proposed detection scheme. First, the fundamental detection noise associated with vacuum fluctuations cannot be manipulated by an external adversary. Second, the ability to reconstruct the photon number distribution at the receiver's end can place additional constraints on possible attacks from the adversary. As an example, we study the security of the BB84 QKD using conjugate homodyne detection and evaluate its performance through numerical simulations. This study may open the door to a family of QKD protocols, complementary to the well-established DV-QKD based on single-photon detection and CV-QKD based on coherent detection.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2021

RADA

The code for regularized adversarial domain adaption is provided here. The RADA methodology is developed for correcting climate model biases using an adversarial learning strategy. The code here covers the data processing, the algorithm, the baseline models, and the evaluation.

Anderson, GemmaJ.↗