Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Adversarial machine learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

Quantifying the robustness of deep multispectral segmentation models against natural perturbations and data poisoning

In overhead image segmentation tasks, including additional spectral bands beyond the traditional RGB channels can improve model performance. However, it is still unclear how incorporating this additional data impacts model robustness to adversarial attacks and natural perturbations. For adversarial robustness, the additional in-formation could improve the model’s ability to distinguish malicious inputs, or simply provide new attack avenues and vulnerabilities. For natural perturbations, the additional information could better inform model decisions and weaken perturbation effects or have no significant influence at all. In this work, we seek to characterize the performance and robustness of a multispectral (RGB and near infrared) image segmentation model subjected to adversarial attacks and natural perturbations. While existing adversarial and natural robustness research has focused primarily on digital perturbations, we prioritize on creating realistic perturbations designed with physical world conditions in mind. For adversarial robustness, we focus on data poisoning attacks whereas for natural robustness, we focus on extending ImageNet-C common corruptions for fog and snow that coherently and self-consistently perturbs the input data. Overall, we find both RGB and multispectral models are vulnerable to data poisoning attacks regardless of input or fusion architectures and that while physically-realizable natural perturbations still degrade model performance, the impact differs based on fusion architecture and input data.

Deep learning, multispectral images, multimodal fu↗
DOE OSTI2022

Autonomous System Subversion Tactics: Prototypes and Recommended Countermeasures

One of the fielding requirements for Advanced and Small Modular Reactors (AR/SMR) is the ability to support remote and autonomous operations. Autonomous Control Systems (ACS) are found on platforms such as Autonomous Space Vehicles, Cruise Missiles, and advanced driver-assistance systems. Each of these ACS implementations depends upon a set of decision support subsystems responsible for supporting Autonomous Mission Managers (names vary based upon field and author preferences). These Autonomous Mission Managers receive inputs from system sensors (e.g., LIDAR collection from an automobile travelling down a street; transients from a nuclear reactor), and perform a set of classifications (e.g., Red Traffic Light; Small Pedestrian at 10m; Load Rejection; Single Coolant Pump Trip), and then use these classifications in combination with recommendation algorithms to achieve platform goals (e.g., Stop the Vehicle at the Traffic Light, Avoid the Small Pedestrian; Trip the Reactor to prevent a Safety Event). The design, implementation, and fielding of an ACS capability will alter the cyber-attack surface such that existing risk management plans will need to be updated to include how to protect and defend against data-science and decision-support-system attack classes. These attack classes would include protection of the design and training environments where algorithm selection and testing and training data would be obvious attack vectors. These attack classes would also require an informed set of detection and response procedures to identify anomalous behaviors and document best practices for anomaly assessment and vulnerability mitigation and remediation. Last year we published a Cyber Threat Assessment Methodology for Autonomous and Remote Operations for AR/SMRs along with a companion publication on Cyber Attack and Defense Use Cases. The focus of the methodology was on describing and enumerating ACS processes, components, and functions such that security engineers could: evaluate subversion options against the target; identify threat actor attributes and capabilities derived from each subversion option; and identify security controls and response countermeasures. The Use Cases document offered detailed methodology examples including an assessment of a Military Base SMR, an Autonomous System Decision Loop, and implementation of AR/SMR Machine Learning algorithms. Our proposal at the end of last year was to focus on implementation of subversion prototypes related to the last Use Case area: AR/SMR Machine Learning (ML) Algorithms. We included six attack scenarios in our Use Cases paper: a Poisoning Attack against ML functions implemented using an FPGA; a Trojaning Attack against ML classifiers exploiting the excitability of Nuclear Engineers; a Backdooring Attack against ML Training environments to ensure persistence of an attack vector; a False Positive Evasion Attack against multi-factor Access Control Systems using clever inputs; an Inference Attack against ML models by an Insider with access to the Operational environment; and an Adversarial Reprogramming Attack against a Material Access Control Video Surveillance System. At the beginning of this year these six attack scenarios were provided to our research teams at Georgia Tech and Idaho State University and each team successfully implemented a subversion attack against a ML implementation to include transient misclassifications. While this is a notable outcome from this type of research, this paper offers the reader insight into not only how to structure and execute these types of attacks, but into the thought process behind how the researcher investigated the problem space, performed initial algorithm implementation, and the trial-and-error behind arriving at the successful subversion prototypes. We include in this paper a set of associated Scenarios on how these subversion prototypes could be implemented and an initial set of guidance for AR/SMR architects, Nuclear Regulators, and Cyber Defenders to implement awareness and defense capabilities into their current operational portfolios.

42 ENGINEERING↗
DOE OSTI2023

Test and Evaluation of Systems with Embedded Machine Learning Components

As Machine Learning (ML) continues to advance, it is being integrated into more systems. Often, the ML component represents a significant portion of the system that reduces the burden on the end user or significantly improves task performance. However, the ML component represents an unknown complex phenomenon that is learned from collected data without the need to be explicitly programmed. Despite the improvement in task performance, the models are often black boxes. Evaluating the credibility and the vulnerabilities of ML models poses a gap in current test and evaluation practice. For high consequence applications, the lack of testing and evaluation procedures represents a significant source of uncertainty and risk. To help reduce that risk, here we present considerations to evaluate systems embedded with an ML component within a red-teaming inspired methodology. We focus on (1) cyber vulnerabilities to an ML model, (2) evaluating performance gaps, and (3) adversarial ML vulnerabilities.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

XSub: Explanation-Driven Adversarial Attack against Blackbox Classifiers via Feature Substitution

Despite its significant benefits in enhancing the transparency and trustworthiness of artificial intelligence (AI) systems, explainable AI (XAI) can unintentionally provide adversaries with insights into blackbox models, increasing their vulnerability to various attacks. In this paper, we develop a novel explanation-driven adversarial attack against blackbox classifiers based on feature substitution, called XSub. The key idea of XSub is to strategically replace important features (identified via XAI) in the original sample with corresponding important features of a different label, thereby increasing the likelihood of the model misclassifying the perturbed sample. XSub only requires a minimal number of queries and can be easily extended to launch backdoor attacks in case the attacker has access to the model's training data. Our evaluation shows that XSub is not only effective and stealthy but also low-cost, showcasing its feasibility across a wide range of AI applications.

adversarial attack↗
DOE OSTI2025

Model Residuals as Shields: A Two-Level Formulation to Defend Smart Grids From Poisoning Attacks

The advancement of smart grids presents both vast opportunities and heightened cybersecurity risks. Data-driven defense mechanisms, though designed as a shield against these threats, can fall prey to poisoning attacks. We delve into regression settings, underscoring the imperative to fortify defenses against a spectrum of poison ratios, notably those above 0.5—an issue scarcely addressed in prior studies. Recognizing the susceptibilities of smart grids and their manipulable sensors, we exploit the very intent of poisoning attacks, compromising model accuracy, as our defense mechanism. Our proposed two-level optimization framework discerns between poisoned and authentic data based on model residuals, outperforming or matching existing methods in 72% to 77% of precision and 75% to 80% of recalls across various poisoning attacks, poison ratios, and datasets. Once the authentic data are identified, the trained model is adaptable for a variety of applications. Comprehensive evaluations on different smart grid datasets, pitted against myriad poisoning schemes, validate our methodology’s edge over existing methods. Here, we also shed light on the implications of model misspecification originating from temporal auto-correlation, a common feature in Internet of Things and smart grid data.

Adversarial machine learning (ML)↗
DOE OSTI2024

Universal Fourier Attack for Time Series

A wide variety of adversarial attacks have been proposed and explored using image and audio data. These attacks are notoriously easy to generate digitally when the attacker can directly manipulate the input to a model, but are much more difficult to implement in the real world. In this paper we present a universal, time invariant attack for general time series data such that the attack has a frequency spectrum primarily composed of the frequencies present in the original data. The universality of the attack makes it fast and easy to implement as no computation is required to add it to an input, while time invariance is useful for real world deployment. Additionally, the frequency constraint ensures the attack can withstand filtering defenses. We demonstrate the effectiveness of the attack on two different classification tasks through both digital and real world experiments, and show that the attack is robust against common transform-and-compare defense pipelines.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

BioADAPT-MRC: adversarial learning-based domain adaptation improves biomedical machine reading comprehension task

ABSTRACT Motivation Biomedical machine reading comprehension (biomedical-MRC) aims to comprehend complex biomedical narratives and assist healthcare professionals in retrieving information from them. The high performance of modern neural network-based MRC systems depends on high-quality, large-scale, human-annotated training datasets. In the biomedical domain, a crucial challenge in creating such datasets is the requirement for domain knowledge, inducing the scarcity of labeled data and the need for transfer learning from the labeled general-purpose (source) domain to the biomedical (target) domain. However, there is a discrepancy in marginal distributions between the general-purpose and biomedical domains due to the variances in topics. Therefore, direct-transferring of learned representations from a model trained on a general-purpose domain to the biomedical domain can hurt the model’s performance. Results We present an adversarial learning-based domain adaptation framework for the biomedical machine reading comprehension task (BioADAPT-MRC), a neural network-based method to address the discrepancies in the marginal distributions between the general and biomedical domain datasets. BioADAPT-MRC relaxes the need for generating pseudo labels for training a well-performing biomedical-MRC model. We extensively evaluate the performance of BioADAPT-MRC by comparing it with the best existing methods on three widely used benchmark biomedical-MRC datasets—BioASQ-7b, BioASQ-8b and BioASQ-9b. Our results suggest that without using any synthetic or human-annotated data from the biomedical domain, BioADAPT-MRC can achieve state-of-the-art performance on these datasets. Availability and implementation BioADAPT-MRC is freely available as an open-source project at https://github.com/mmahbub/BioADAPT-MRC. Supplementary information Supplementary data are available at Bioinformatics online.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2023

Super-Resolution for Renewable Energy Resource Data with Climate Change Impacts (Sup3rCC)

The Super-Resolution for Renewable Energy Resource Data with Climate Change Impacts (Sup3rCC) data is a collection of 4km hourly wind, solar, temperature, humidity, and pressure fields for the contiguous United States under various climate change scenarios. Sup3rCC is downscaled Global Climate Model (GCM) data. The downscaling process was performed using a generative machine learning approach called sup3r: Super-Resolution for Renewable Energy Resource Data (linked below as "Sup3r GitHub Repo"). The data includes both historical and future weather years, although the historical years represent the historical climate, not the actual historical weather that we experienced. You cannot use Sup3rCC data to study historical weather events, although other sup3r datasets may be intended for this. The Sup3rCC data is intended to help researchers study the impact of climate change on energy systems with high levels of wind and solar capacity. Please note that all climate change data is only a representation of the possible future climate and contains significant uncertainty. Analysis of multiple climate change scenarios and multiple climate models can help quantify this uncertainty.

Array↗
DOE OSTI2025

Robustness of Deep Learning Classification to Adversarial Input on GPUs: Asynchronous Parallel Accumulation Is a Source of Vulnerability

The ability of machine learning (ML) classification models to resist small, targeted input perturbations—known as adversarial attacks—is a key measure of their safety and reliability. We show that floating-point non associativity (FPNA) coupled with asynchronous parallel programming on GPUs is sufficient to result in misclassification, without any perturbation to the input. Additionally, we show that this misclassification is particularly significant for inputs close to the decision boundary and that standard adversarial robustness results may be overestimated up to 4.6 when not considering machine-level details. We first study a linear classifier, before focusing on standard Graph Neural Network (GNN) architectures and datasets used in robustness assessments. We develop a novel black-box attack using Bayesian optimization to discover external workloads that can change the instruction scheduling which bias the output of reductions on GPUs and reliably lead to misclassification. Motivated by these results, we present a new learnable permutation (LP) gradient-based approach to learning floating-point operation orderings that lead to misclassifications. The LP approach provides a worst-case estimate in a computationally efficient manner, avoiding the need to run identical experiments tens of thousands of times over a potentially large set of possible GPU states or architectures. Finally, using instrumentation-based testing, we investigate parallel reduction ordering across different GPU architectures under external background workloads, when utilizing multi-GPU virtualization, and when applying power capping. Our results demonstrate that parallel reduction ordering varies significantly across architectures under the first two conditions, substantially increasing the search space required to fully test the effects of this parallel scheduler-based vulnerability. These results and the methods developed here can help to include machine-level considerations into adversarial robustness assessments, which can make a difference in safety and mission critical applications.

Shanmugavelu, Sanjif [Maxeler Technologies, a Groq↗
DOE OSTI2020

Limited-view Cone Beam CT reconstruction using 3D Patch-based Supervised and Adversarial Learning [Slides]

We present a novel machine learning CNN architecture that can learn from limited data combined appropriately with physics and statistical priors (e.g., forward models and noise models). To address the limited availability of training data we adopt a 3D patch-based approach for our models. Patch-based learning is central to several image reconstruction methods and demands fewer training data than DL approaches, as a single data volume can be broken into several millions of overlapping 3D sub-volumes or patches. This creates a very large number of training sub-volumes from a limited number of overall image volumes. A 3D Generative Adversarial Networks (GAN) is then trained to remove artifacts at the sub-volume level. The combination of a sub-volume-based approach with DL allows us to exploit the richness of the latter in extracting and representing image features, while avoiding risks associated with overfitting due to limited training data.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2022

Synthetic Data Generation Using Machine Learning

Robust machine learning techniques for image analysis require a substantial amount of data to yield confident results. In the nuclear domain, data scarcity is a substantial challenge because there are so few facilities worldwide. This research focuses on being able alleviate the data scarcity problem by generating synthetic data to bridge the gap between large and small datasets. This work achieves that goal using a Generative Adversarial Network (GAN) architectural approach, by training a model on real-world data and expands that small dataset through synthetic data amendments. Model performance is impacted by the size of the real-world dataset and the number of training epochs utilized. This means that 1) It is important to develop your GAN to be optimized with the specific data type, and 2) approaches taken when training the GAN should be specialized to encompass important aspects of the dataset that it is generating. By taking a step to improve dataset sizes in this way, the gap between models trained by parties with significant amount of data and those without access to large data, closes, allowing for robust analyses of satellite imagery for nuclear domain applications.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2025

Improving microstructures segmentation via pretraining with synthetic data

Image analysis of material microstructures through microscopy is an integral capability in the field of materials science. The topological and chemical information obtained through microscopy allow us to draw vital connections between material microstructures, properties, and processing. While scanning electron microscopy (SEM) is able to yield a considerable wealth of information interpretable by the intuition of experts, there has been considerable interest in using machine learning, convolutional neural networks (CNNs) in particular, for such image analysis task. Training CNNs for an image analysis task requires a large annotated dataset. However, in many materials science applications, obtaining a large annotated dataset is cost and labor intensive. In this work, we study the use of synthetic data to enlarge the available annotated experimental data of uranium oxide. We utilize a modified Potts model to simulate uranium oxide particles with morphologies similar to those observed experimentally. We then leverage an image-to-image translation model to synthesize the simulated particles as if they are acquired with SEM. Through this process, we obtain pairs of particle images and their corresponding SEM representations, which corresponds to pairs of annotations and images. Unlike previous works, we leverage synthetic data for pretraining a CNN model prior, and finetune that model further with experimental data. We experimentally demonstrate that using synthetic data as incremental learning process benefits the overall performance compared to training a model on combined synthetic and experimental data.

36 MATERIALS SCIENCE↗
DOE OSTI2020

Loosely Conditioned Emulation of Global Climate Models With Generative Adversarial Networks

Climate models encapsulate our best understanding of the Earth system, allowing research to be conducted on its future under alternative assumptions of how human-driven climate forces are going to evolve. An important application of climate models is to provide metrics of mean and extreme climate changes, particularly under these alternative future scenarios, as these quantities drive the impacts of climate on society and natural systems. Because of the need to explore a wide range of alternative scenarios and other sources of uncertainties in a computationally efficient manner, climate models can only take us so far, as they require significant computational resources, especially when attempting to characterize extreme events, which are rare and thus demand long and numerous simulations in order to accurately represent their changing statistics. Here we use deep learning in a proof of concept that lays the foundation for emulating global climate model output for different scenarios. We train two "loosely conditioned" Generative Adversarial Networks (GANs) that emulate daily precipitation output from a fully coupled Earth system model: one GAN modeling Fall-Winter behavior and the other Spring-Summer. Our GANs are trained to produce spatiotemporal samples: 32 days of precipitation over a 64x128 regular grid discretizing the globe. We evaluate the generator with a set of related performance metrics based upon KL divergence, and find the generated samples to be nearly as well matched to the test data as the validation data is to test. We also find the generated samples to accurately estimate the mean number of dry days and mean longest dry spell in the 32 day samples. Our trained GANs can rapidly generate numerous realizations at a vastly reduced computational expense, compared to large ensembles of climate models, which greatly aids in estimating the statistics of extreme events.

climate emulation, extreme climate, impacts, machi↗
DOE OSTI2022

Finding Hidden Patterns in High Resolution Wind Flow Model Simulations

Wind flow data is critical in terms of investment decisions and policy making. High resolution data from wind flow model simulations serve as a supplement to the limited resource of original wind flow data collection. Given the large size of data, finding hidden patterns in wind flow model simulations are critical for reducing the dimensionality of the analysis. In this work, we first perform dimension reduction with two autoencoder models: the CNN-based autoencoder (CNN-AE) [1], and hierarchical autoencoder (HIER-AE) [2], and compare their performance with the Principal Component Analysis (PCA). We then investigate the super-resolution of the wind flow data. By training a Generative Adversarial Network (GAN) with 300 epochs, we obtained a trained model with 2× resolution enhancement. We compare the results of GAN with Convolutional Neural Network (CNN), and GAN results show finer structure as expected in the data field images. Also, the kinetic energy spectra comparisons show that GAN outperforms CNN in terms of reproducing the physical properties for high wavenumbers and is critical for analysis where high-wavenumber kinetics play an important role.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Evaluating generative networks using Gaussian mixtures of image features

We develop a measure for evaluating the performance of generative networks given two sets of images. A popular performance measure currently used to do this is the Fréchet Inception Distance (FID). However, FID assumes that images featurized using the penultimate layer of Inception follow a Gaussian distribution. This assumption allows FID to be easily computed, since FID uses the 2-Wasserstein distance of two Gaussian distributions fitted to the featurized images. However, we show that Inception features of the ImageNet dataset are not Gaussian; in particular, each marginal is not Gaussian. To remedy this problem, we model the featurized images using Gaussian mixture models (GMMs) and compute the 2-Wasserstein distance restricted to GMMs. We define a performance measure, which we call WaM, on two sets of images by using inception (or another classifier) to featurize the images, estimate two GMMs, and use the restricted 2-Wasserstein distance to compare the GMMs. We experimentally show the advantages of WaM over FID, including how FID is more sensitive than WaM to image perturbations. By modelling the non-Gaussian features obtained from inception as GMMs and using a GMM metric, we can more accurately evaluate generative network performance.

machine learning, genrative adversarial networks↗
NASA NTRSDate not supplied

Combining Data with Physical Knowledge for Uncertainty Quantification in Certification and Reliability Analysis

Unifying empirical data with predictive models can enable engineering cost-savings through certification by analysis and reliability-based design. Both concepts require rigorous uncertainty quantification (UQ) and robust understanding and treatment of relevant physics. Combining sampling-based UQ algorithms with high-fidelity simulations creates a computational bottleneck that is often alleviated through the use of machine learning (ML). ML can be used to create computationally efficient surrogates for simulations of complex or high-dimensional physical interactions (e.g., multi-phase interactions associated with melt pools in laser powder bed fusion or spatially-dependent material properties in functionally graded materials). However, negative side effects of ML may include a lack of interpretability and negative correlation between event rarity and simulation accuracy due to a lack of training data. As such, it is important to infuse ML algorithms with physics-based guardrails to provide confidence in their predictions. This talk will provide a brief review of recent NASA research at this intersection of physics-based simulation, ML, and UQ with a focus on certification and reliability analysis.

uncertainty quantification↗