Scalable uncertainty quantification for deep operator networks using randomized priors
Not provided.
Engineering↗
SEARCH · Engineering Papers
Results for “Adversarial data”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
PMU Data Quality and Sensor Health Monitoring
Phasor Measurement Units (PMUs) play a critical role in the evolution of the electric power industry by providing high-precision, real-time monitoring of essential power system metrics. However, effectively detecting abnormalities and critical events from PMU data is a complex task, complicated by intricate temporal patterns, a scarcity of labeled data for training algo- rithms, and constraints on online computational power. In this study, we apply TranAD, an innovative algorithm that combines transformer architectures with the refinement of adversarial learning, to both synthetic and real-world PMU datasets for developing a data quality and sensor online health monitoring platform for utilities. Our findings reveal that TranAD not only provides efficient detection and localization but also enhances the detail with which abnormalities are detected, marking a a significant step forward in the field of clean data acquisition processes for power system monitoring
deep neural network, machine learning (ML)↗
Geometrical-Based Generative Adversarial Network to Enhance Digital Rock Image Quality
X-ray microcomputed tomography (micro-CT) is a common tool for the study of porous media structures and properties. High-quality micro-CT data are required to accurately capture pore structures. Acquiring high-quality micro-CT data, however, is not always possible, owing to application limitations and experimental constraints. Therefore, we propose a geometrical-based generative adversarial network (GAN) to rapidly restore noisy micro-CT images to their clean counterparts. The training data and related ground-truth (GT) data are scanned for 7 min and 9.5 h, respectively. To evaluate the performance of the geometrical-based GAN, a 6003 voxel image that has never been used for training is reconstructed and compared with the corresponding GT image. Histogram matching and linear normalization are implemented to adjust the histogram of the reconstructed image to that of the GT image. A watershed-based segmentation method is then applied to delineate pore and solid phases. Lastly, we measure the Minkowski functionals and petrophysical properties, including absolute permeability, pore size distribution, drainage capillary pressure-saturation curve, and imbibition relative permeability, to estimate the physical accuracy of the denoised image. The results show that the proposed geometrical-based GAN can accurately restore noisy micro-CT data. By reducing the scanning time from 9.5 h to 7 min, the expenditure of collecting micro-CT can be decreased significantly. This is particularly important for applications where time-lapse images of a dynamic process are required, high-throughput imaging is necessary for real-time data analysis, or where the quantification of large sample volumes is required.
58 GEOSCIENCES↗
Real-time Intrusion Detection for High-bandwidth Research Networks using Unsupervised Deep Learning
Nowadays, data generated by large-scale scientific experiments are on the scale of petabytes per month. These data are transferred through dedicated high-bandwidth networks across distributed collaboration sites for processing, storage, and analysis. Like general purpose networks, research networks experience attacks. However, monitoring traffic in such high-bandwidth network environments is challenging with conventional deep packet inspection (DPI)- based network intrusion detection systems (NIDS). Recently, machine learning (ML) algorithms have shown potential in detecting anomalous patterns with better generalizability and faster speed. Compared to the general purpose networks, traffic movement in research network is less volatile and more likely to benefit from the ML-based anomaly analysis. We present ThunderSecure, an efficient AI-powered system for detecting anomalies in 100G research networks. ThunderSecure implements a high-throughput packet processing infrastructure using multi-cores and GPUs. It takes time-series statistics from a variety of network measurements and uses 2D Convolutional Neural Network (CNN) to explore both spatial and temporal dependencies in network data stream. Patterns of science data traffic are learned with a one-class neural network, which blends an Adversarial Autoencoder (AAE) with Gaussian mixture density estimation. Testing traffic flows exhibiting significant deviations from the learned baseline of normality are marked as anomalies. We trained ThunderSecure on hundreds of billions of science data packets recorded from a 100G research network at Fermi National Accelerator Laboratory. The detection performance was evaluated on traffic captured from the same research network days and weeks after the training with different types of attack flows injected. To the best of our knowledge, this is the first ML work in the area of network anomaly detection that has been validated on such extreme scale datasets. Results show that ThunderSecure can recognize science data traffic that are captured long after the training and make nearly certain detection on those with anomalous flows injected, even in the case when the anomaly-to-normal mixing ratio is 0.1%.
Gong, Qian↗
Generating realistic building electrical load profiles through the Generative Adversarial Network (GAN)
Building electrical load profiles can improve understanding of building energy efficiency, demand flexibility, and building-grid interactions. Current approaches to generating load profiles are time-consuming and not capable of reflecting the dynamic and stochastic behaviors of real buildings; some approaches also trigger data privacy concerns. In this study, we proposed a novel approach for generating realistic electrical load profiles of buildings through the Generative Adversarial Network (GAN), a machine learning technique that is capable of revealing an unknown probability distribution purely from data. The proposed approach has three main steps: (1) normalizing the daily 24-hour load profiles, (2) clustering the daily load profiles with the k-means algorithm, and (3) using GAN to generate daily load profiles for each cluster. The approach was tested with an open-source database – the Building Data Genome Project. We validated the proposed method by comparing the mean, standard deviation, and distribution of key parameters of the generated load profiles with those of the real ones. The KL divergence of the generated and real load profiles are within 0.3 for majority of parameters and clusters. Additionally, results showed the load profiles generated by GAN can capture not only the general trend but also the random variations of the actual electrical loads in buildings. We report the proposed GAN approach can be used to generate building electrical load profiles, verify other load profile generation models, detect changes to load profiles, and more importantly, anonymize smart meter data for sharing, to support research and applications of grid-interactive efficient buildings.
32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
Data-scarce surrogate modeling of shock-induced pore collapse process
Understanding the mechanisms of shock-induced pore collapse is of great interest in various disciplines in sciences and engineering, including materials science, biological sciences, and geophysics. However, numerical modeling of the complex pore collapse processes can be costly. To this end, a strong need exists to develop surrogate models for generating economic predictions of pore collapse processes. Here, in this work, we study the use of a data-driven reduced-order model, namely dynamic mode decomposition, and a deep generative model, namely conditional generative adversarial networks, to resemble the numerical simulations of the pore collapse process at representative training shock pressures. Since the simulations are expensive, the training data are scarce, which makes training an accurate surrogate model challenging. To overcome the difficulties posed by the complex physics phenomena, we make several crucial treatments to the plain original form of the methods to increase the capability of approximating and predicting the dynamics. In particular, physics information is used as indicators or conditional inputs to guide the prediction. In realizing these methods, the training of each dynamic mode composition model takes only around 30 s on CPU. In contrast, training a generative adversarial network model takes 8 h on GPU. Moreover, using dynamic mode decomposition, the final-time relative error is around 0.3% in the reproductive cases. We also demonstrate the predictive power of the methods at unseen testing shock pressures, where the error ranges from 1.3 to 5% in the interpolatory cases and 8 to 9% in extrapolatory cases.
97 MATHEMATICS AND COMPUTING↗
Cybersecurity and Digital Components: Supply Chain Deep Dive Assessment
The report “America’s Strategy to Secure the Supply Chain for a Robust Clean Energy Transition” lays out the challenges and opportunities faced by the United States in the energy supply chain as well as the federal government plans to address these challenges and opportunities. It is accompanied by several issue-specific deep dive assessments, including this one, in response to Executive Order 14017 “America’s Supply Chains,” which directs the Secretary of Energy to submit a report on supply chains for the energy sector industrial base. The Executive Order is helping the federal government to build more secure and diverse U.S. supply chains, including energy supply chains. As the energy sector has become more globalized and increasingly complex, digitized, and even virtualized, its supply chain risk for digital components – the software, virtual platforms and services, and data – in energy systems has evolved and expanded. All digital components in U.S. energy sector systems are vulnerable and may be subject to cyber supply cha in risks stemming from a variety of threats, vulnerabilities, and impacts. This includes digital components in all systems within the ESIB, namely those systems operated by asset owners across different energy subsectors (e.g., electricity, oil and natural gas, and renewables) and the systems operated by a worldwide industrial complex with capabilities to perform research and development and design, produce, operate, and maintain energy sector systems, subsystems, components, or parts to meet U.S. energy requirements. Supply chain risks for digital components including software, virtual platforms and services, and data have grown in recent years as increasingly sophisticated cyber adversaries have targeted exploiting vulnerabilities in these digital assets. Supply chain risks for digital components in energy sector systems will continue to evolve and likely increase as these systems are increasingly interconnected, digitized, and remotely operated.
29 ENERGY PLANNING, POLICY, AND ECONOMY↗
CBRN Situational Awareness Tools for the Modern Age
CBRN Situational awareness (SA) should be a key operational requirement in the CNI area of discussion, not only for the “boots-on-the-ground” warfighter, but at a strategic level. For example, maneuver plans will need to be developed for contamination avoidance and battlefield survivability. Large datasets from instruments dispersed across the battlefield will thus need to be collected and processed into a holistic assessment of the radiological hazard. From there, hazard maps then must be distributed at the tactical level, all while continuing to effectively engage a peer adversary. This vision represents a network of sensors, software tools, data processing algorithms, communications suites, and common operational picture (COP) platforms working in concert to inform warfighters and command echelons alike of evolving nuclear and radiological hazards. This complex system-of-systems and the ultimate goal of effective engagement in a nuclear or radiological-hazard battlefield can be achieved through pursuit of improvement in collaborative SA tools.
Becker, Eric M.↗
Generative $β$-hairpin design using a residue-based physicochemical property landscape
De novo peptide design is a new frontier that has broad application potential in the biological and biomedical fields. Most existing models for de novo peptide design are largely based on sequence homology that can be restricted based on evolutionarily derived protein sequences and lack the physicochemical context essential in protein folding. Generative machine learning for de novo peptide design is a promising way to synthesize theoretical data that are based on, but unique from, the observable universe. In this study, we created and tested a custom peptide generative adversarial network intended to design peptide sequences that can fold into the -hairpin secondary structure. This deep neural network model is designed to establish a preliminary foundation of the generative approach based on physicochemical and conformational properties of 20 canonical amino acids, for example, hydrophobicity and residue volume, using extant structure-specific sequence data from the PDB. The beta generative adversarial network model robustly distinguishes secondary structures of hairpin from α helix and intrinsically disordered peptides with an accuracy of up to 96% and generates artificial -hairpin peptide sequences with minimum sequence identities around 31% and 50% when compared against the current NCBI PDB and nonredundant databases, respectively. These results highlight the potential of generative models specifically anchored by physicochemical and conformational property features of amino acids to expand the sequence-to-structure landscape of proteins beyond evolutionary limits.
37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
Data Security Defense: Modeling and Detection of Synchrophasor Data Spoofing Attack for Grid Edge
Data security and cyberattack have become critical issues in the distributed power system where adversaries can swap the source information of sensors or even spoof and alter measurements. However, the cyber security of the power system is challenged by the unpredictability and stealth of the spoofing attacks. Here, to protect the data security at the grid edge, this paper developed a synchrophasor data spoofing attack detection framework based on the time-frequency feature extraction techniques including the short-time Fourier transform (STFT) and object detection network for real-time synchrophasor data categorization and spoofing attack localization. The proposed approach outperforms earlier work in terms of spoofing attack detection and offers a vital localization function employing distributed synchrophasor sensors.
24 POWER TRANSMISSION AND DISTRIBUTION↗
Universal Fourier Attack for Time Series
A wide variety of adversarial attacks have been proposed and explored using image and audio data. These attacks are notoriously easy to generate digitally when the attacker can directly manipulate the input to a model, but are much more difficult to implement in the real world. In this paper we present a universal, time invariant attack for general time series data such that the attack has a frequency spectrum primarily composed of the frequencies present in the original data. The universality of the attack makes it fast and easy to implement as no computation is required to add it to an input, while time invariance is useful for real world deployment. Additionally, the frequency constraint ensures the attack can withstand filtering defenses. We demonstrate the effectiveness of the attack on two different classification tasks through both digital and real world experiments, and show that the attack is robust against common transform-and-compare defense pipelines.
97 MATHEMATICS AND COMPUTING↗
Attribute-Guided Adversarial Training for Robustness to Natural Perturbations
We report while existing work in robust deep learning has focused on small pixel-level norm-based perturbations, this may not account for perturbations encountered in several real world settings. In many such cases although test data might not be available, broad specifications about the types of perturbations (such as an unknown degree of rotation) may be known. We consider a setup where robustness is expected over an unseen test domain that is not i.i.d. but deviates from the training domain. While this deviation may not be exactly known, its broad characterization is specified a priori, in terms of attributes. We propose an adversarial training approach which learns to generate new samples so as to maximize exposure of the classifier to the attributes-space, without having access to the data from the test domain. Our adversarial training solves a min-max optimization problem, with the inner maximization generating adversarial perturbations, and the outer minimization finding model parameters by optimizing the loss on adversarial perturbations generated from the inner maximization. We demonstrate the applicability of our approach on three types of naturally occurring perturbations --- object-related shifts, geometric transformations, and common image corruptions. Our approach enables deep neural networks to be robust against a wide range of naturally occurring perturbations. We demonstrate the usefulness of the proposed approach by showing the robustness gains of deep neural networks trained using our adversarial training on MNIST, CIFAR-10, and a new variant of the CLEVR dataset.
97 MATHEMATICS AND COMPUTING↗
FL-DISCO: Federated Generative Adversarial Network for Graph-based Molecule Drug Discovery: Special Session Paper
The outbreak of the global COVID-19 pandemic emphasizes the importance of collaborative drug discovery for high effectiveness; however, due to the stringent data regulation, data privacy becomes an imminent issue needing to be addressed to enable collaborative drug discovery. In addition to the data privacy issue, the efficiency of drug discovery is another key objective since infectious diseases spread exponentially and effectively conducting drug discovery could save lives. Advanced Artificial Intelligence (AI) techniques are promising to solve these problems: (1) Federated Learning (FL) is born to keep data privacy while learning data from distributed clients; (2) graph neural network (GNN) can extract structural properties of molecules whose underlying architecture is the connected atoms; and (3) generative adversarial network (GAN) can generate novel molecules while retaining the properties learned from the training data. In this work, we make the first attempt to build a holistic collaborative and privacy-preserving FL framework, namely FL- DISCO, which integrates GAN and GNN to generate molecular graphs. Experimental results demonstrate the effectiveness of FL- DISCO on: (1) IID data for ESOL and QM9, where FL-DISCO can generate highly novel compounds with high drug-likeliness, uniqueness and LogP scores compared to the baseline; (2) non- IID data for ESOL and QM9, where FL-DISCO generates 100% novel compounds with high validity and LogP scores compared to the baseline. We also demonstrate how different fractions of clients, generator and discriminator architectures affect our evaluation scores.
Manu, Daniel↗
How Low Can You Go? Using Synthetic 3D Imagery to Drastically Reduce Real-World Training Data for Object Detection
Deep convolutional neural networks (DCNNs) currently provide state-of-the-art performance on image classification and object detection tasks, and there are many global security mission areas where such models could be extremely useful. Crucially, the success of these models is driven in large part by the widespread availability of high-quality open source data sets such as Image Net, Common Objects in Context (COCO), and KITTI, which contain millions of images with thousands of unique labels. However, global security relevant objects-of-interest can be difficult to obtain: relevant events are low frequency and high consequence; the content of relevant images is sensitive; and adversaries and proliferators seek to obscure their activities. For these cases where exemplar data is hard to come-by, even fine-tuning an existing model with available data can be effectively impossible. Recent work demonstrated that models can be trained using a combination of real-world and synthetic images generated from 3D representations; that such models can exceed the performance of models trained using real-world data alone; and that the generated images need not be perfectly realistic (Tremblay, et al., 2018). However, this approach still required hundreds to thousands of real-world images for training and fine tuning, which for sparse, global security-relevant datasets can be an unrealistic hurdle. In this research, we validate the performance and behavior of DCNN models as we drive the number of real-world images used for training object detection tasks down to a minimal set. We perform multiple experiments to identify the best approach to train DCNNs from an extremely small set of real-world images. In doing so, we: Develop state-of-the-art, parameterized 3D models based on real-world images and sample from their parameters to increase the variance in synthetic image training data; Use machine learning explainability techniques to highlight and correct through targeted training the biases that result from training using completely synthetic images; and Validate our results by comparing the performance of the models trained on synthetic data to one another, and to a control model created by fine-tuning an existing ImageNet-trained model with a limited number (hundreds) of real-world images.
97 MATHEMATICS AND COMPUTING↗
Large Scale Unsupervised Domain Adaptation of Segmentation Networks with Adversarial Learning
Most current state-of-the-art methods for semantic segmentation on remote sensing imagery require large labeled data, which is scarcely available. Due to the distribution shifting phenomenon inherent in remote sensing imagery, the reuse of pre-trained models on new areas of interest rarely yield satisfactory results. In this paper, we approach this problem from an adversarial learning perspective toward unsupervised domain adaptation. The core concept is to infuse fully convolutional neural networks and adversarial networks for semantic segmentation assuming the structures in the scene and objects of interest are similar in two set of images. Models are trained on a source dataset where ground truth is available and adapted to new target dataset iteratively via a adversarial loss on unlabeled samples. We use two real large scale datasets to validate the framework: 1) cross city road extraction and 2) cross country building extraction. The preliminary results show the usefulness of considering adversarial learning for indirect re-use of the pre-trained models. Experimental validation suggests significant benefits over models without adaptation.
Deng, Xueqing↗
Deep generative models for vehicle speed trajectories
Generating realistic vehicle speed trajectories is a crucial component in evaluating vehicle fuel economy and in predictive control of self-driving cars. Traditional generative models rely on Markov chain methods and can produce accurate synthetic trajectories but are subject to the curse of dimensionality. They do not allow to include conditional input variables into the generation process. In this paper, we show how extensions to deep generative models allow accurate and scalable generation. Proposed architectures involve recurrent and feed-forward layers and are trained using adversarial techniques. Our models are shown to perform well on generating vehicle trajectories using a model trained on GPS data from Chicago metropolitan area.
33 ADVANCED PROPULSION SYSTEMS↗
Breaking the barrier of human-annotated training data for machine learning-aided plant research using aerial imagery
Machine learning (ML) can accelerate biological research. However, the adoption of such tools to facilitate phenotyping based on sensor data has been limited by (i) the need for a large amount of human-annotated training data for each context in which the tool is used and (ii) phenotypes varying across contexts defined in terms of genetics and environment. This is a major bottleneck because acquiring training data is generally costly and time-consuming. This study demonstrates how a ML approach can address these challenges by minimizing the amount of human supervision needed for tool building. A case study was performed to compare ML approaches that examine images collected by an uncrewed aerial vehicle to determine the presence/absence of panicles (i.e. “heading”) across thousands of field plots containing genetically diverse breeding populations of 2 Miscanthus species. Automated analysis of aerial imagery enabled the identification of heading approximately 9 times faster than in-field visual inspection by humans. Leveraging an Efficiently Supervised Generative Adversarial Network (ESGAN) learning strategy reduced the requirement for human-annotated data by 1 to 2 orders of magnitude compared to traditional, fully supervised learning approaches. The ESGAN model learned the salient features of the data set by using thousands of unlabeled images to inform the discriminative ability of a classifier so that it required minimal human-labeled training data. This method can accelerate the phenotyping of heading date as a measure of flowering time in Miscanthus across diverse contexts (e.g. in multistate trials) and opens avenues to promote the broad adoption of ML tools.
59 BASIC BIOLOGICAL SCIENCES↗
pixelvar79/ESGAN-Flowering-Detection-paper
Machine learning (ML) can accelerate biological research. However, the adoption of such tools to facilitate phenotyping based on sensor data has been limited by (i) the need for a large amount of human-annotated training data for each context in which the tool is used and (ii) phenotypes varying across contexts defined in terms of genetics and environment. This is a major bottleneck because acquiring training data is generally costly and time-consuming. This study demonstrates how a ML approach can address these challenges by minimizing the amount of human supervision needed for tool building. A case study was performed to compare ML approaches that examine images collected by an uncrewed aerial vehicle to determine the presence/absence of panicles (i.e. “heading”) across thousands of field plots containing genetically diverse breeding populations of 2 Miscanthus species. Automated analysis of aerial imagery enabled the identification of heading approximately 9 times faster than in-field visual inspection by humans. Leveraging an Efficiently Supervised Generative Adversarial Network (ESGAN) learning strategy reduced the requirement for human-annotated data by 1 to 2 orders of magnitude compared to traditional, fully supervised learning approaches. The ESGAN model learned the salient features of the data set by using thousands of unlabeled images to inform the discriminative ability of a classifier so that it required minimal human-labeled training data. This method can accelerate the phenotyping of heading date as a measure of flowering time in Miscanthus across diverse contexts (e.g. in multistate trials) and opens avenues to promote the broad adoption of ML tools.
Varela, Sebastian↗