Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Adversarial data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 235 records · Page 13
All sourcesNASA NTRSDOE OSTI
DOE OSTI2019

Information-Theoretically Secure Distributed Machine Learning

A previously obscure area of cryptography known as Secure Multiparty Computation (MPC) is enjoying increased attention in the field of privacy-preserving machine learning (ML), because ML models implemented using MPC can be uniquely resistant to capture or reverse engineering by an adversary. In particular, an adversary who captures a share of a distributed MPC model provably cannot recover the model itself, nor data evaluated by the model, even by observing the model in operation. We report on our small project to survey current MPC software and judge its practicality for fielding mission-relevant distributed machine learning models.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Application of Cyber-Informed Engineering for Protecting BESS

This white paper synthesizes an array of crucial grid services provided by BESS technology, assesses its architecture and communications, and presents a case study for analysis against the principles introduced by Cyber-Informed Engineering (CIE). Furthermore, in walking through the analysis, this paper presents a framework to evaluate risks and solutions when considering BESS components. Asset owners and buyers could perform this analysis to assess their BESS product implementations, alternative inverter-based resources (IBR), and energy management systems (EMS). Battery systems fulfill various roles contingent on the unique market demands and the specific challenges presented by regional grid infrastructures. These roles also vary due to the differing utility models for ownership and operation, which are adapted to meet regional and local capabilities and requirements. Concerns have been raised regarding the potential for adversaries to exploit knowledge of battery operational patterns to orchestrate decisive attacks. However, the security of operational data for these systems may not be the primary vulnerability, as much of this information is already well-understood within the community. Applying a modest degree of subject matter expertise can often yield valuable predictions regarding how a battery will respond under certain conditions, such as grid emergencies, high or low-temperature days, Public Safety Power Shutoff (PSPS) events, and outages. The operational characteristics of batteries are well-documented, and their capabilities, including the risks associated with misoperation and the resulting consequences, are published and understood within the industry. CIE practices represent the next step in gaining functional assurance and providing an acceptable level of risk, regardless of whether a battery vendor can support a trusted and validated supply chain. While this issue has exacerbated supply chain challenges, it is not an isolated condition. This foreign supply route is the primary source of BESS for the U.S. market. Significant efforts are underway through the Bipartisan Infrastructure Law (BIL) to change that. Still, strategic short-term operational mitigations are needed to ensure the security of our operational technology (OT) systems, which are enhanced by instilling trust and are separate from vendors implementing CIE principles.

25 ENERGY STORAGE↗
DOE OSTI2023

ProvSec: Open Cybersecurity System Provenance Analysis Benchmark Dataset with Labels

System provenance forensic analysis has been studied by a large body of research work. This area needs fine granularity data such as system calls along with event fields to track the dependencies of events. While prior work on security datasets has been proposed, we found a useful dataset of realistic attacks and details that are needed for high-quality provenance tracking is lacking. We created a new dataset of eleven vulnerable cases for system forensic analysis. It includes the full details of system calls including syscall parameters. Realistic attack scenarios with real software vulnerabilities and exploits are used. For each case, we created two sets of benign and adversary scenarios which are manually labeled for supervised machine-learning analysis. In addition, we present an algorithm to improve the data quality in the system provenance forensic analysis. We demonstrate the details of the dataset events and dependency analysis of our dataset cases.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

A data analytical approach for assessing the efficacy of Operational Technology active defenses against insider threats

In recent years, the need for Operational Technology (OT) defenses has been recognized, serving as an additional line of defense when Information Technology (IT) defenses are bypassed. This is no longer considered an uncommon possibility when dealing with advanced persistent threat (APT) actors expected to be state-sponsored and receiving insider assistance. In these extreme adversarial situations, OT defenses aim to provide another layer of defense for the system, introduced directly at the physical process level, as described by the sensors data, the system model, and control actions. Just like IT defenses, two schools of thought, i.e., passive and active defenses, have emerged to address this challenge. In active defenses, representing the focus of this paper, known signatures, synthesized based on the system's unique characteristics, are inserted into the system. In contradistinction, passive methods rely solely on observing system behavior in search of patterns of normal behavior with deviations thereof representing abnormal behavior. In their most sophisticated implementations, both passive and active defenses rely on the use of data analytics to identify the patterns and synthesize the observed and/or inserted signatures. Past research has shown that passive defenses may be bypassed by APT actors relying on data analytics and their intimate knowledge of the system to evade detection by respecting the patterns identified by the defenders. Here this manuscript explores the use of active defenses under the assumption that the attacker has privileged access to the system, including access to the system's model and sensors data. Specifically, this manuscript assesses the ability of active defenses to remain invisible to the attackers, and discusses the associated challenges that must be addressed to ensure their resiliency against APT actors.

98 NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL P↗
NASA NTRSDate not supplied

Creating Benchmark Data for Artificial Intelligence and Machine Learning Space Biology Research

To identify an appropriate AI/ML approach for a specific problem, the best practice is to measure algorithm performance through the benchmarking process. A scientific benchmark consists of an AI-ready dataset and a reference implementation on a specific scientific question. The NASA Science Mission Directorate (SMD) has started the “Benchmark Initiative for AI/ML to create scientific benchmark datasets in three applications: 1) scientific benchmarking, which finds the best algorithm for a specific problem; 2) application benchmarking, which measures algorithm performance against a set of parameters; and 3) system benchmarking, which evaluates performance of hardware and software architecture. Currently, there are no standardized datasets available to benchmark AI/ML algorithms in the domain of space biology. In this work, we constructed two AI/ML-ready biological datasets from experiments in space-flown mice: cellular imaging and RNA-seq. First, radiation-exposed immune cells harbor DNA damage foci that can be fluorescently marked to visualize the amount of damage following exposure to ionizing radiation. However, such large datasets are difficult to analyze visually, due to imaging inconsistencies and human bias, and classical image processing approaches can fail on imaging artifacts. AI/ML are therefore exciting alternative, providing the speed of machines and the accuracy of humans. We have made this dataset available at https://registry.opendata.aws/bps_microscopy/. Second, high-throughput nucleic acid sequencing (DNA-seq, RNA-seq) has become widespread in biomedical research due to the growing availability and affordability of these assays. However, most sequencing datasets suffer from high dimensionality and low sample count. In this work, we used a generative adversarial network to synthesize a standardized, AI-ready, publicly available benchmark dataset for space biology RNA-seq data with sufficient space-flown and ground control mouse liver samples from NASA GeneLab. This dataset is available at https://registry.opendata.aws/bps_rnaseq/. These datasets are now fully open the Space Biology community to test their favorite AI/ML approaches.

James Casaletto↗
DOE OSTI2021

Improve Learning from Crowds via Generative Augmentation

Crowdsourcing provides an efficient label collection schema for supervised machine learning. However, to control annotation cost, each instance in the crowdsourced data is typically annotated by a small number of annotators. This creates a sparsity issue and limits the quality of machine learning models trained on such data. In this paper, we study how to handle sparsity in crowdsourced data using data augmentation. Specifically, we propose to directly learn a classifier by augmenting the raw sparse annotations. We implement two principles of high-quality augmentation using Generative Adversarial Networks: 1) the generated annotations should follow the distribution of authentic ones, which is measured by a discriminator; 2) the generated annotations should have high mutual information with the ground-truth labels, which is measured by an auxiliary network. Extensive experiments and comparisons against an array of state-of-the-art learning from crowds methods on three real-world datasets proved the effectiveness of our data augmentation framework. It shows the potential of our algorithm for low-budget crowdsourcing in general.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
DOE OSTI2024

CO 2 storage site characterization using ensemble-based approaches with deep generative models

Estimating spatially distributed properties such as permeability from available sparse measurements is a great challenge in efficient subsurface CO 2 storage operations. In this paper, a deep generative model that can accurately capture complex subsurface structure is tested with an ensemble-based inversion method for accurate and accelerated characterization of CO 2 storage sites. We chose Wasserstein Generative Adversarial Network with Gradient Penalty (WGAN-GP) for its realistic reservoir property representation and Ensemble Smoother with Multiple Data Assimilation (ES-MDA) for its robust data fitting and uncertainty quantification capability. WGAN-GP are trained to generate high-dimensional permeability fields from a low-dimensional latent space and ES-MDA then updates the latent variables by assimilating available measurements. Several subsurface site characterization examples including Gaussian, channelized, and fractured reservoirs are used to evaluate the accuracy and computational efficiency of the proposed method and the main features of the unknown permeability fields are characterized accurately with reliable uncertainty quantification. Furthermore, the estimation performance is compared with a widely-used variational, i.e., optimization-based, inversion approach, and the proposed approach outperforms the variational inversion method in several benchmark cases. We explain such superior performance by visualizing the objective function in the latent space: because of nonlinear and aggressive dimension reduction via generative modeling, the objective function surface becomes extremely complex while the ensemble approximation can smooth out the multi-modal surface during the minimization. This suggests that the ensemble-based approach works well over the variational approach when combined with deep generative models at the cost of forward model runs unless convergence-ensuring modifications are implemented in the variational inversion.

42 ENGINEERING↗
DOE OSTI2018

Anomaly Detection for Power System Generation Control based on Hierarchical DBSCAN

The generation level of a generator in power grid is under control of multiple control loops such as governor action, Automatic Generation Control (AGC), and manual control. These control loops are vulnerable to cyber attacks and could become potential targets for the malicious adversary. In this paper, we investigated the detection of abnormal generation controls induced by cyber attacks. One data-driven anomaly detection methodology is presented based on the behavior conformity of generation units located in the same balancing authority. A semi-supervised clustering algorithm with Hierarchical Density based Spatial Clustering of Application with Noise (HDBSCAN) is proposed for the detection model training. In our previous work, we obtained a synthetic dataset that covers scenarios including normal operation, generation control under ramp attack, switching attack, AGC integrity attacks, etc. and it is used to evaluate the proposed clustering algorithm. Experimental results show that the proposed algorithm provides better detection accuracy than K-means clustering and can distinguish not only between normal and abnormal generation controls but also among various anomaly scenarios.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2023

EPiC-GAN: Equivariant point cloud generation for particle jets

With the vast data-collecting capabilities of current and future high-energy collider experiments, there is an increasing demand for computationally efficient simulations. Generative machine learning models enable fast event generation, yet so far these approaches are largely constrained to fixed data structures and rigid detector geometries. In this paper, we introduce EPiC-GAN - equivariant point cloud generative adversarial network - which can produce point clouds of variable multiplicity. This flexible framework is based on deep sets and is well suited for simulating sprays of particles called jets. The generator and discriminator utilize multiple EPiC layers with an interpretable global latent vector. Crucially, the EPiC layers do not rely on pairwise information sharing between particles, which leads to a significant speed-up over graph- and transformer-based approaches with more complex relation diagrams. We demonstrate that EPiC-GAN scales well to large particle multiplicities and achieves high generation fidelity on benchmark jet generation tasks.

Buhmann, Erik↗
DOE OSTI2020

Mass agnostic jet taggers

Searching for new physics in large data sets needs a balance between two competing effects—signal identification vs background distortion. In this work, we perform a systematic study of both single variable and multivariate jet tagging methods that aim for this balance. The methods preserve the shape of the background distribution by either augmenting the training procedure or the data itself. Multiple quantitative metrics to compare the methods are considered, for tagging 2-, 3-, or 4-prong jets from the QCD background. This is the first study to show that the data augmentation techniques of Planing and PCA based scaling deliver similar performance as the augmented training techniques of Adversarial NN and uBoost, but are both easier to implement and computationally cheaper.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2021

DeepMerge – II. Building robust deep learning algorithms for merging galaxy identification across domains

In astronomy, neural networks are often trained on simulation data with the prospect of being used on telescope observations. Unfortunately, training a model on simulation data and then applying it to instrument data leads to a substantial and potentially even detrimental decrease in model accuracy on the new target dataset. Simulated and instrument data represent different data domains, and for an algorithm to work in both, domain-invariant learning is necessary. Here we employ domain adaptation techniques— Maximum Mean Discrepancy (MMD) as an additional transfer loss and Domain Adversarial Neural Networks (DANNs)— and demonstrate their viability to extract domain-invariant features within the astronomical context of classifying merging and non-merging galaxies. Additionally, we explore the use of Fisher loss and entropy minimization to enforce better in-domain class discriminability. We show that the addition of each domain adaptation technique improves the performance of a classifier when compared to conventional deep learning algorithms. We demonstrate this on two examples: between two Illustris-1 simulated datasets of distant merging galaxies, and between Illustris-1 simulated data of nearby merging galaxies and observed data from the Sloan Digital Sky Survey. The use of domain adaptation techniques in our experiments leads to an increase of target domain classification accuracy of up to ~20%. With further development, these techniques will allow astronomers to successfully implement neural network models trained on simulation data to efficiently detect and study astrophysical objects in current and future large-scale astronomical surveys.

galaxies: interactions↗
NASA NTRS2022

Neurosymbolic Hybrid Approach to Driver Collision Warning

There are two main algorithmic approaches to autonomous driving systems: (1) An end-to-end system in which a single deep neural network learns to map sensory input directly into appropriate warning and driving responses. (2) A mediated hybrid recognition system in which a system is created by combining independent modules that detect each semantic feature. While some researchers believe that deep learning can solve any problem, others believe that a more engineered and symbolic approach is needed to cope with complex environments with less data. Deep learning alone has achieved state-of-the-art results in many areas, from complex gameplay to predicting protein structures. In particular, in image classification and recognition, deep learning models have achieved accuracies as high as humans. But sometimes it can be very difficult to debug if the deep learning model doesn't work. Deep learning models can be vulnerable and are very sensitive to changes in data distribution. Generalization can be problematic. It's usually hard to prove why it works or doesn't. Deep learning models can also be vulnerable to adversarial attacks. Here, we combine deep learning-based object recognition and tracking with an adaptive neurosymbolic network agent, called the Non-Axiomatic Reasoning System (NARS), that can adapt to its environment by building concepts based on perceptual sequences. We achieved an improved intersection-over-union (IOU) object recognition performance of 0.65 in the adaptive retraining model compared to IOU 0.31 in the COCO data pre-trained model. We improved the object detection limits using RADAR sensors in a simulated environment, and demonstrated the weaving car detection capability by combining deep learning-based object detection and tracking with a neurosymbolic model.

Wang, Pei↗
DOE OSTI2020

CCE Phase 3: Consequence-based Targeting

While Phase 2 was a data collection effort, Phase 3 is a targeting effort. In Phase 3, organizations systematically identify the necessary steps for adversary success - all from the adversary’s perspective. A key component to this approach is identifying the critical information needs, targets, access, and actions required for the adversary to achieve the desired effect. These Critical Needs are tied to accomplishing the HCE, such as the technical requirements for the payload (Development), or the access required to deliver a payload (Deployment).

99 GENERAL AND MISCELLANEOUS↗
DOE OSTI2020

Multimodal imaging and machine learning to enhance microscope images of shale

A machine learning based image processing workflow is presented to enhance shale source rock microscopic images obtained using diverse imaging platforms. Images were acquired from a 30 μm diameter cylindrical Vaca Muerta shale sample using both nondestructive Transmission X-Ray Microscopy (TXM, alternately referred to as nano computed tomography) and destructive Focused Ion Beam-Scanning Electron Microscopy (FIB-SEM). Output cross-sectional images from each modality were aligned using a combination of manual and automated registration techniques to create a registered image dataset. We then apply this dataset for two image processing tasks: prediction of image cross sections with SEM-like resolution from nondestructive TXM data and repair of charged region artifacts (localized accumulation of electrons) within SEM images. The image processing algorithms for both tasks use deep learning models, specifically image-to-image Convolutional Neural Networks (CNNs) and conditional Generative Adversarial Networks (cGANs). In the image enhancement tasks, we are able to achieve significant qualitative and quantitative improvement in TXM images. Here, the best model reaches an average Peak Signal to Noise Ratio (PSNR) of 15.8 dB. Conditioning on TXM data is also shown to reduce artifacts from SEM charging, achieving an average PSNR of 25.8 dB. Furthermore, our results suggest that properly trained and validated networks are capable of significant enhancement of images obtained using nondestructive techniques, thereby improving interpretation of two- and three-dimensional images while preserving samples for future use.

58 GEOSCIENCES↗
DOE OSTI2025

Resilient information and inference networks under mixed-trust sensing

With ubiquitous digitization, sensing, and computational intelligence deployed in increasingly more and broader domains, including critical infrastructure, potentially misleading and destabilizing effects of multimodal anomalies and adversarial behavior are growing in importance. Here, we develop randomized and reinforcement learning-based strategies for strategically recruiting and utilizing deployed (and, thus, vulnerable and potentially faulty and/or compromised) nodes from information and inference networks, while defending against adversaries that attempt to misguide assessments of inferred variables. Recognizing that, besides communication and other costs, sampling from any observable node can either provide true data or dangerously expose our inference to misinformation (without being easily distinguishable what actually happens), the proposed strategies proceed by progressively recruiting nodes and cautiously scaling their information contribution based on assumed, or, in our reinforcement learning approach, intelligently weighed trustworthiness, with the learning approach also considering network-wide, threat-inclusive risk/value tradeoffs. While avoiding the hardware, communication, analytical and computational burden of explicit redundancy, the proposed defensive schemes enable on-the-fly assessments of underlying processes, and system-wide situational awareness with demonstrable resilience against adversarial activities.

97 - MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Quantification of morphological change in materials based on image data utilizing machine learning techniques

Computed tomography (CT) resolution has become high enough to monitor morphological changes due to aging in materials in long-term applications. We explored the utility of the critic of a generative adversarial network (GAN) to automatically detect such changes. The GAN was trained with images of pristine Pharmatose, which is used as a surrogate energetic material. It is important to note that images of the material with altered morphology were only used during the test phase. The GAN-generated images visually reproduced the microstructure of Pharmatose well, although some unrealistic particle fusion was seen. Calculated morphological metrics (volume fraction, interfacial line length, and local thickness) for the synthetic images also showed good agreement with the training data, albeit with signs of mode collapse in the interfacial line length. While the critic exposed changes in particle size, it showed limited ability to distinguish images by particle shape. The detection of shape differences was also a more challenging task for the selected morphological metrics that related to energetic material performance. We further tested the critic with images of aged Pharmatose. Subtle changes due to aging are difficult for the human analyst to detect. Both critic and morphological metrics analysis showed image differentiation.

36 MATERIALS SCIENCE↗
DOE OSTI2022

DER Cybersecurity Detection and Response Suite

SAND2024-08475O The Distributed Energy Resource (DER) Cybersecurity Detection and Response Suite is a solution for distributed energy resource (DER) systems. The DER Security Orchestration, Automation, and Response (SOAR) solution that uses alerts from signature- and behavior-based Intrusion Detection Systems are intended to be deployed as bump-in-the-wire (BITW) devices in front of DER equipment. The fielded application would use multiple intrusion detection systems that report data to SOAR to respond to cyberattacks. The suite consists of two software components: • The proactive intrusion detection and mitigation system (PIDMS) secures grid-edge photovoltaic smart inverters and other equipment in distributed energy resource systems. It is a distributed BITW solution; cyber and physical data are automatically processed using network inspection tools and custom machine learning algorithms to detect abnormal events and correlate cyber-physical events. • The Security Orchestration, Automation, and Response for Distributed Energy Resources (SOAR4DER) application ingests data from several intrusion detection systems to quickly block attacks and revert DER systems to good states. Using a collection of intrusion detection system technologies on a BITW device, it incorporates physical and cyber data to detect abnormal and potential malicious behaviors. Multiple SOAR playbooks then use the intrusion detection system data streams to automatically defend the system. SOAR4DER system testing showed detection and response times under 30 seconds for all adversary reconnaissance, denial-of-service attacks, malicious Modbus commands, brute-force logins, and machine-in-the-middle attacks. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525.

Johnson, Jay↗