A Generative Adversarial Network-based Technique for Generating Synthetic Renewable Energy Data
Explore the source record for details and available documents.
Lopez Ramirez, Andres Stephen [Sandia National Lab↗
SEARCH · Engineering Papers
Results for “Network data”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Cross-Layered Cyber-Physical Power System State Estimation towards a Secure Grid Operation
In the Smart Grid paradigm, this critical infrastructure operation is increasingly exposed to cyber-threats due to the increased dependency on communication networks. An adversary can launch an attack on a power grid operation through False Data Injection into system measurements and/or through attacks on the communication network, such as flooding the communication channels with unnecessary data or intercepting messages. A cross-layered strategy that combines power grid data, communication grid monitoring and Machine Learning based processing is a promising solution for detecting cyberthreats. In this paper, an implementation of an integrated solution of a cross-layer framework is presented. The advantage of such a framework is the augmentation of valuable data that enhances the detection of anomalies in the operation of power grid. IEEE 118-bus system is built in Simulink to provide a power grid testing environment and communication network data is emulated using SimComponents. The performance of the framework is investigated under various FDI and communication attacks.
cyber security, network security, cyber-physical s↗
Hyperplane decision trees as piecewise linear surrogate models for chemical process design
Recent trends in chemical engineering research point towards an increasing reliance on data-driven modeling approaches. Neural networks, for instance, have proven to be accurate when data is plentiful and high-dimensional, but in many cases, they require computationally-intensive training procedures. Here, in this work, we describe hyperplane decision trees (HT) as a highly expressive and low-compute machine learning model architecture. These models are locally linear and have linear decision boundaries, resulting in a piecewise linear model of the data. This property allows them to be converted into mixed-integer linear constraints which can be globally optimized. Our open-source PyTorch implementation of this method is a fast, flexible, and accessible way to build accurate piecewise linear models of data.
Decision trees↗
pvlib iotools—Open-source Python functions for seamless access to solar irradiance data
Access to accurate solar resource data is critical for numerous applications, including estimating the yield of solar energy systems, developing radiation models, and validating irradiance datasets. However, lack of standardization in data formats and access interfaces across providers constitutes a major barrier to entry for new users. pvlib python’s iotools subpackage aims to solve this issue by providing standardized Python functions for reading local files and retrieving data from external providers. All functions follow a uniform pattern and return convenient data outputs, allowing users to seamlessly switch between data providers and explore alternative datasets. The pvlib package is community-developed on GitHub: https://github.com/pvlib/pvlib-python. As of pvlib python version 0.9.5, the iotools subpackage supports 12 different datasets, including ground measurement, reanalysis, and satellite-derived irradiance data. The supported ground measurement networks include the Baseline Surface Radiation Network (BSRN), NREL MIDC, SRML, SOLRAD, SURFRAD, and the US Climate Reference Network (CRN). Additionally, satellite-derived and reanalysis irradiance data from the following sources are supported: PVGIS (SARAH & ERA5), NSRDB PSM3, and CAMS Radiation Service (including McClear clear-sky irradiance).
14 SOLAR ENERGY↗
dCache: Inter-disciplinary storage system
The dCache project provides open-source software deployed internationally to satisfy ever more demanding storage requirements. Its multifaceted approach provides an integrated way of supporting different use-cases with the same storage, from high throughput data ingest, data sharing over wide area networks, efficient access from HPC clusters and long term data persistence on a tertiary storage. Though it was originally developed for the HEP experiments, today it is used by various scientific communities, including astrophysics, biomed, life science, which have their specific requirements. In this paper we describe some of the new requirements as well as demonstrate how dCache developers are addressing them.
Mkrtchyan, Tigran↗
Datashare
Datashare facilitates communication and data sharing within local networks in potentially dangerous situations such as an explosive ordnance disposal. During such events, there is a need to transmit information rapidly around the incident area. It is a distributed database that does not require an internet connection for operation. In addition, Datashare interfaces with XTK and other software applications, allowing for seamless integration and data management. Datashare supports video calls over the network, enabling real-time communication among users. This software serves to organize, package, and share between responders on location and export data to those off location. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525.
Eldridge, Bryce [Sandia National Lab. (SNL-CA), Li↗
Deep Hierarchical Super Resolution for Scientific Data
We present a novel technique for hierarchical super resolution (SR) with neural networks (NNs), which upscales volumetric data represented with an octree data structure to a high-resolution uniform gridwith minimal seam artifacts on octree node boundaries. Our method uses existing state-of-the-art SR models and adds flexibility to upscale input data with varying levels of detail across the domain, instead of only uniform grid data that are supported in previous approaches.The key is to use a hierarchy of SR NNs, each trained to perform 2x SR between two levels of detail, with a hierarchical SR algorithm that minimizes seam artifacts by starting from the coarsest level of detail and working up.We show that our hierarchical approach outperforms baseline interpolation and hierarchical upscaling methods, and demonstrate the usefulness of our proposed approach across three use cases including data reduction using hierarchical downsampling+SR instead of uniform downsampling+SR, computation savings for hierarchical finite-time Lyapunov exponent field calculation, and super-resolving low-resolution simulation results for a high-resolution approximation visualization.
97 MATHEMATICS AND COMPUTING↗
Pycheron: A Python-Based Seismic Waveform Data Quality Control Software Package
Supplementing an existing high-quality seismic monitoring network with openly available station data could improve coverage and decrease magnitudes of completeness; however, this can present challenges when varying levels of data quality exist. Without discerning the quality of openly available data, using it poses significant data management, analysis, and interpretation issues. Incorporating additional stations without properly identifying and mitigating data quality problems can degrade overall monitoring capability. If openly available stations are to be used routinely, a robust, automated data quality assessment for a wide range of quality control (QC) issues is essential. To meet this need, we developed Pycheron, a Python-based library for QC of seismic waveform data. Pycheron was initially based on the Incorporated Research Institutions for Seismology’s Modular Utility for STAtistical kNowledge Gathering but has been expanded to include more functionality. Pycheron can be implemented at the beginning of a data processing pipeline or can process stand-alone data sets. Its objectives are to (1) identify specific QC issues; (2) automatically assess data quality and instrumentation health; (3) serve as a basic service that all data processing builds on by alerting downstream processing algorithms to any quality degradation; and (4) improve our ability to process orders of magnitudes more data through performance optimizations. This article provides an overview of Pycheron, its features, basic workflow, and an example application using a synthetic QC data set.
58 GEOSCIENCES↗
Idaho National Laboratory Quality Of Service Dataset
The code is designed to run tests to generate and collect data from a Wi-Fi network using OPENWRT or a simulated a 5G network using Open5gs and UERANSIM. The tests simulate the network performing downloads or uploads of various files with a varying number of concurrent users. The tests use tcpdump to collect the network traffic but only stores the summarized data. The summarized datasets will be included.
Krome, Cameron [Idaho National Laboratory (INL), I↗
Unique Building Identifier (UBID): Public Sector Implementation Guide
Buildings generate data throughout their lifecycle – about ownership & taxation, usage, zoning, code compliance, energy use, and retrofits. State and local governments collect this data after it flows through growing networks of people and systems. But collecting data is only half the battle; what’s really needed is information – the actionable insights that lead to successful policy outcomes.
96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
Enhancing Drinking Water Quality Modeling: Leveraging Physics Informed Neural Networks for Learning with Imperfect Reaction Models and Partial Data
Chemical kinetics models, typically formulated as systems of ordinary or partial differential equations, are valuable tools for simulating drinking water quality. However, these models often face inaccuracies due to discrepancies between the laboratory and the real-world conditions, as well as limitations in experimental analytical methods, hindering the accurate representation of the true underlying chemical mechanisms. In this study, we propose a Physics Informed Neural Network (PINN), using the eXtreme Theory of Functional Connections, to improve the prediction of chemical concentrations over time. The PINN method accounts for imperfect chemical models and incorporates partial data to improve predictions. Focusing on reactions describing water disinfection residual and disinfectant byproduct formation, which are crucial for public health and regulatory compliance, we demonstrate that the PINN model is able to accurately predict the concentrations of chemical species across various pH values. Notably, the model extends its accuracy to predict concentrations of chemical species not originally included in its training data. The developed method can be extended to a variety of chemical systems, offering a wide array of potential applications.
13 HYDRO ENERGY↗
Multiscale Data-Driven Seismic Full-Waveform Inversion With Field Data Study
Seismic full-waveform inversion (FWI), which uses iterative methods to estimate high-resolution subsurface models from seismograms, is a powerful imaging technique in exploration geophysics. In recent years, the computational cost of FWI has grown exponentially due to the increasing size and resolution of seismic data. Moreover, it is a nonconvex problem and can encounter local minima due to the limited accuracy of the initial velocity models or the absence of low frequencies in the measurements. To overcome these computational issues, we develop a multiscale data-driven FWI method based on fully convolutional networks (FCNs). In preparing the training data, we first develop a real-time style transform method to create a large set of synthetic subsurface velocity models from natural images. We then develop two convolutional neural networks with encoder-decoder structures to reconstruct the low- and high-frequency components of the subsurface velocity models, separately. To validate the performance of our data-driven inversion method and the effectiveness of the synthesized training set, we compare it with conventional physics-based waveform inversion approaches using both synthetic and field data. Finally, these numerical results demonstrate that, once our model is fully trained, it can significantly reduce the computation time and yield more accurate subsurface velocity models in comparison with conventional FWI.
58 GEOSCIENCES↗
B/PS bulges in DESI Legacy edge-on galaxies – I. Sample building
ABSTRACT We present the biggest up-to-date sample of edge-on galaxies with boxy/peanut-shaped (B/PS) bulges and X-structures. The sample was prepared using images from the Dark Energy Spectroscopic Instrument (DESI) Legacy catalogue and contains about 2000 galaxies. To find suitable candidates in catalogue, we made the assumption that the residues (original images minus model) of galaxies with B/PS bulges should exhibit a characteristic X-shape. Galaxies with such features were selected by eye and then used as input data for a neural network training, which was applied to a bigger sample of edge-on galaxies. Using the available data and the photometric models from the literature, we investigated the observational and statistical properties of the sample created. Comparing the B/D ratios for galaxies with and without B/PS bulges, we found that the B/D ratio for galaxies from our sample is statistically higher, with typical values in the range ≈0.2–0.5 depending on the decomposition procedure. We studied how the opening angles φ of the X-structure and the length of its rays are distributed in the formed sample and found them to be consistent with previous measurements and predictions from N-body models, e.g. $\varphi \gtrsim 25{^\circ }$, but measured here for a much larger number of galaxies. We found a sharp increase in the B/PS bulge fraction for stellar masses log M⋆ ≳ 10.4, but for edge-on galaxies, which complements the results of previous works. The sample can be used in future work to test various bar models and their relationship with B/PS bulges, as well as to study their stability and evolution.
79 ASTRONOMY AND ASTROPHYSICS↗
Hybrid PDES Simulation of HPC Networks Using Zombie Packets
Although high-fidelity network simulations have proven to be reliable and cost-effective tools to peer into architectural questions for high-performance computing (HPC) networks, they incur a high resource cost. The time spent in simulating a single millisecond of network traffic in the highest detail can take hours, even for static, well-behaved traffic patterns such as uniform random. Surrogate models offer a significant reduction in runtime, yet they cannot serve as complete replacements and should only be used when appropriate. Thus, there is a need for hybrid modeling, where high-fidelity simulation and surrogates run side-by-side. Here, we present a surrogate model for HPC networks in which: packets bypass the network, while the network state is left untouched, i.e., suspended. To bypass the network, we use historical data to estimate the arrival time at which every packet should be scheduled at; to suspend the network, all in-flight packets are scheduled to arrive at their destinations, and are kept in the system to awaken as zombies when switching back to high-fidelity. Speedup for a hybrid model is relative to the proportion of surrogate to high-fidelity. This light-weight surrogate obtained up to 76× speedup. Keeping the zombies in the network showed an increase in the accuracy of the high-fidelity simulation on restart when compared to restarting the network from an empty state.
HPC networks↗
Convergence of Emerging Technologies - EAGL Test Information
The Emergency Automatic Gunshot Detection and Lockdown (EAGL) system provides automatic, autonomous, and timely gunshot detection in both indoor and outdoor environments. This system uses both wired and wireless devices. Self-contained wireless EAGL sensors passively “listen” for gunshot events. These devices also perform a single, daily supervisory heartbeat (HB) function to include a device self-check with reporting capability. Transmissions are received by an assigned EAGL Gateway, which translates the RF sensor data to a PoE network format solely for use by the EAGL system server. The server then performs additional processes after data receipt, which include but are not limited to: event validation and logging, GUI presentation, notifications, and other independent operations.
47 OTHER INSTRUMENTATION↗
A Systematic Study to Determine 5G Baseline Performance for Scientific Computing
The fifth-generation (5G) cellular networks envisions achieving higher data rates, improved connectivity, reduced latency, and better quality of service (QoS) than the fourthgeneration (4G) cellular networks. Such improved performance can be utilized to address the challenges in applications such as electricity generation in power systems. The traditional power grids responsible for electricity generation suffer from drawbacks such as life-threatening blackout crises, and energy storage proliferation as they are not robust to extreme climatic conditions. A recent study proposed the idea of extending the capabilities of advanced wireless technologies such as the current 5G to develop a robust, energy-efficient, and secure smart grids. However there are two main challenges associated with the integration of power systems and wireless technologies. First, it is imperative to understand the architecture and the enabling technologies of 5G to ensure that the performance requirements of the smart grids are met. Second, an end-to-end testbed is required to determine if the performance requirements are met by estimating the 5G characteristics such as latency, and throughput. Our proposed alleviates the aforementioned concerns in the following manner. To begin with, a systematic study of the 5G architecture including both the StandAlone (SA) and Non-Standalone (NSA) operations is presented. Furthermore, a detailed survey of the possible 5G enabling technologies is elicited. In addition to these, an end-toend testbed that can estimate the 5G characteristics is explained in detail with appropriate preliminary results.
5G, 5G Communication↗
Lessons Learned on the Interface Between Quantum and Conventional Networking
The future Quantum Internet is expected to be based on a hybrid architecture with core quantum transport capabilities complemented by conventional networking. Practical and foundational considerations indicate the need for conventional control and data planes that (i) utilize extensive existing telecommunications fiber infrastructure, and (ii) provide parallel conventional data channels needed for quantum networking protocols. We propose a quantum-conventional network (QCN) harness to implement a new architecture to meet these requirements. The QCN control plane carries the control and management traffic, whereas its data plane handles the conventional and quantum data communications. We established a local area QCN connecting three quantum laboratories over dedicated fiber and conventional network connections. We describe considerations and tradeoffs for layering QCN functionalities, informed by our recent quantum entanglement distribution experiments conducted over this network.
Alshowkan, Muneer↗
Identifying strengths and weaknesses of methods for computational network inference from single-cell RNA-seq data
Single-cell RNA-sequencing (scRNA-seq) offers unparalleled insight into the transcriptional programs of different cellular states by measuring the transcriptome of thousands of individual cells. An emerging problem in the analysis of scRNA-seq is the inference of transcriptional gene regulatory networks and a number of methods with different learning frameworks have been developed to address this problem. Here, we present an expanded benchmarking study of eleven recent network inference methods on seven published scRNA-seq datasets in human, mouse, and yeast considering different types of gold standard networks and evaluation metrics. We evaluate methods based on their computing requirements as well as on their ability to recover the network structure. We find that, while most methods have a modest recovery of experimentally derived interactions based on global metrics such as Area Under the Precision Recall curve, methods are able to capture targets of regulators that are relevant to the system under study. Among the top performing methods that use only expression were SCENIC, PIDC, MERLIN or Correlation. Addition of prior biological knowledge and the estimation of transcription factor activities resulted in the best overall performance with the Inferelator and MERLIN methods that use prior knowledge outperforming methods that use expression alone. We found that imputation for network inference did not improve network inference accuracy and could be detrimental. Comparisons of inferred networks for comparable bulk conditions showed that the networks inferred from scRNA-seq datasets are often better or at par with the networks inferred from bulk datasets. Our analysis should be beneficial in selecting methods for network inference. At the same time, this highlights the need for improved methods and better gold standards for regulatory network inference from scRNAseq datasets.
59 BASIC BIOLOGICAL SCIENCES↗