Results for “COMMUNICATIONS” — Page 32

DOE OSTI2022

Cyber-Physical Dynamic System (CPDS) Modeling for Frequency Regulation and AGC Services of Distributed Energy Resources

The substantial integration of renewable energy brings significant challenges to balance the system in real time because of the variability and intermittency of renewable power. For the reliable system operation, the frequency regulation service is used to stabilize the system frequency through automatically balancing the system generation and load. On one hand, with the substantially increasing deployment of renewable energy in electricity system, the requirement of frequency regulation (FR) services increases significantly. On the other hand, the current main resource of FR services, the controllable conventional generation, is continuously decreasing in the system generation mix. This means that in the future high renewable penetration power system, additional and alternative reliable FR services providers such as distributed energy storage (DES) resources should be explored. Although the capability of utility-scale energy storage to provide FR services has been demonstrated, the integrated control and dynamic modeling of distributed energy resource (DER) providing frequency regulation grid services has been rarely explored. There are several challenges to adopt DERs to provide reliable grid services as illustrated in FERC 755. First, the distributed installation of DES requires a comprehensive cyber-physical dynamic system (CPDS) modeling to fully consider the impacts of the communication latency variability on its real time FR provision capability. Unlike the conventional generators, there are two-layer communication between DERs and system operators which increase the communication delay. Second, unlike the conventional generators whose dynamic models have been comprehensively studied, the difference among individual DER components' power-dynamic characteristic brings challenges in the accurate dynamic AGC modeling of its power-frequency relationship. Third, the temporal dependent state of charge uncertainty of DERs challenges DERs' power and frequency regulation capacities scheduling in the look-ahead generation scheduling. Therefore, the aggregator should optimize the frequency regulation provision from individual DER in real time. To overcome these challenges, this project proposes a cyber-physical dynamic system (CPDS) model to handle the uncertainty of DERs two-layer communication latency and power dynamics. The variability of DERs' communication delay and dynamic constraints will be comprehensively modeled. The DERs' AGC model with communication delay is designed to validate DESs' frequency regulation services. Like current performance-based frequency regulation evaluation, the delivery of DERs' frequency regulation will be assessed through post-analysis of the actual AGC response with respect to the AGC control signal from system operators. Consequently, the reliability improvement with DERs providing reliable frequency regulation services can be evaluated from a comprehensive perspective considering all the dynamics of communication and power.

24 POWER TRANSMISSION AND DISTRIBUTION↗

DOE OSTI2024

Measuring Thread Timing to Assess the Feasibility of Early-Bird Message Delivery Across Systems and Scales

Early-bird communication is a communication/computation overlap technique that leverages fine-grained communication to improve application run-time. Communication is divided such that each individual thread can initiate transmission of its portion of the data upon completion rather than waiting for a dedicated communication phase. The benefit of early-bird communication depends on the completion timing of the individual threads: On the one hand, if all threads are complete at nearly the same time, the overheads of sending multiple messages will accumulate, leading to performance that is worse than if a single message had been sent. On the other hand, if thread completions are spread out in time, those that complete earlier can send data while others continue working, leading to performance that is better than if a single message had been sent. The challenge is that the completion times are currently unknown and can vary based on application, problem size, system software, and underlying hardware. In this paper, we address this lacuna by measuring and evaluating the potential overlap afforded by early-bird communication for a selection of proxy applications. These measurements help us understand whether a given application could benefit from early-bird communication. Here, we present our technique for gathering this data and evaluate data collected from three proxy applications: MiniFE, MiniMD, and MiniQMC. Each application is run on three systems with distinct CPU architectures and strong scales across three run sizes. To characterize the behavior of these workloads, we study the trends of thread timings at both a macro level, across all threads across all runs of an application, and a micro level, that is, within a single process of a single run. We observe that our tested applications exhibit significantly different thread arrival distributions. The machine used had a significant impact, with the window of potential overlap varying by as much as an order of magnitude.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2022

Real-Time Hardware-in-The-Loop Implementation of Protection and Self-Healing of Microgrids

The design of a reliable protection scheme for microgrids often requires communication between protective devices and microgrid controllers. The authors have developed such a communication assisted scalable protection scheme with a self healing feature to protect a microgrid with 100% Inverter Based Resources (IBRs). The communication assisted scheme must be validated in real time with cyber physical co-simulation for successful demonstration. In this regard, the paper presents a co-simulation platform between a simulated power system model using RTDS and physical protective devices. The primary protection of the scheme is programmed in SEL 421-7 relay and backup protection is programmed in MATLAB on a generic computer acting as a microgrid controller. The IEC 61850 models are used to communicate between SEL-421-7 relay and RTDS, whereas TCP/IP communication connects the microgrid controller to RTDS. The paper’s focus is to demonstrate the co- simulation platform with communication links established using both protocols and validate the proposed scheme in real-time on the IEEE 123 node distribution feeder. The paper explains the configuration of the IEC 61850 and TCP/IP communications as the interface requires proper hardware and software setup. Furthermore, the real time performance indicates the Hardware In the Loop (HIL) framework as a competent testing environment for the developed protection scheme for microgrids.

42 ENGINEERING↗

DOE OSTI2023

UPC++ v1.0 Programmer’s Guide (Rev. 2023.9.0)

UPC++ is a C++ library that supports Partitioned Global Address Space (PGAS) programming. It is designed for writing efficient, scalable parallel programs on distributed-memory parallel computers. The key communication facilities in UPC++ are one-sided Remote Memory Access (RMA) and Remote Procedure Call (RPC). The UPC++ control model is single program, multiple-data (SPMD), with each separate constituent process having access to local memory as it would in C++. The PGAS memory model additionally provides one-sided RMA communication to a global address space, which is allocated in shared segments that are distributed over the processes. UPC++ also features Remote Procedure Call (RPC) communication, making it easy to move computation to operate on data that resides on remote processes. UPC++ was designed to support exascale high-performance computing, and the library interfaces and implementation are focused on maximizing scalability. In UPC++, all communication operations are syntactically explicit, which encourages programmers to consider the costs associated with communication and data movement. Moreover, all communication operations are asynchronous by default, encouraging programmers to seek opportunities for overlapping communication latencies with other useful work. UPC++ provides expressive and composable abstractions designed for efficiently managing aggressive use of asynchrony in programs. Together, these design principles are intended to enable programmers to write applications using UPC++ that perform well even on hundreds of thousands of cores.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2022

UPC++ v1.0 Programmer’s Guide (Revision 2022.3.0)

UPC++ is a C++ library that supports Partitioned Global Address Space (PGAS) programming. It is designed for writing efficient, scalable parallel programs on distributed-memory parallel computers. The key communication facilities in UPC++ are one-sided Remote Memory Access (RMA) and Remote Procedure Call (RPC). The UPC++ control model is single program, multiple-data (SPMD), with each separate constituent process having access to local memory as it would in C++. The PGAS memory model additionally provides one-sided RMA communication to a global address space, which is allocated in shared segments that are distributed over the processes. UPC++ also features Remote Procedure Call (RPC) communication, making it easy to move computation to operate on data that resides on remote processes. UPC++ was designed to support exascale high-performance computing, and the library interfaces and implementation are focused on maximizing scalability. In UPC++, all communication operations are syntactically explicit, which encourages programmers to consider the costs associated with communication and data movement. Moreover, all communication operations are asynchronous by default, encouraging programmers to seek opportunities for overlapping communication latencies with other useful work. UPC++ provides expressive and composable abstractions designed for efficiently managing aggressive use of asynchrony in programs. Together, these design principles are intended to enable programmers to write applications using UPC++ that perform well even on hundreds of thousands of cores.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2023

UPC++ v1.0 Programmer’s Guide, Revision 2023.3.0

UPC++ is a C++ library that supports Partitioned Global Address Space (PGAS) programming. It is designed for writing efficient, scalable parallel programs on distributed-memory parallel computers. The key communication facilities in UPC++ are one-sided Remote Memory Access (RMA) and Remote Procedure Call (RPC). The UPC++ control model is single program, multiple-data (SPMD), with each separate constituent process having access to local memory as it would in C++. The PGAS memory model additionally provides one-sided RMA communication to a global address space, which is allocated in shared segments that are distributed over the processes. UPC++ also features Remote Procedure Call (RPC) communication, making it easy to move computation to operate on data that resides on remote processes. UPC++ was designed to support exascale high-performance computing, and the library interfaces and implementation are focused on maximizing scalability. In UPC++, all communication operations are syntactically explicit, which encourages programmers to consider the costs associated with communication and data movement. Moreover, all communication operations are asynchronous by default, encouraging programmers to seek opportunities for overlapping communication latencies with other useful work. UPC++ provides expressive and composable abstractions designed for efficiently managing aggressive use of asynchrony in programs. Together, these design principles are intended to enable programmers to write applications using UPC++ that perform well even on hundreds of thousands of cores.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2022

UPC++ v1.0 Programmer’s Guide, Revision 2022.9.0

UPC++ is a C++ library that supports Partitioned Global Address Space (PGAS) programming. It is designed for writing efficient, scalable parallel programs on distributed-memory parallel computers. The key communication facilities in UPC++ are one-sided Remote Memory Access (RMA) and Remote Procedure Call (RPC). The UPC++ control model is single program, multiple-data (SPMD), with each separate constituent process having access to local memory as it would in C++. The PGAS memory model additionally provides one-sided RMA communication to a global address space, which is allocated in shared segments that are distributed over the processes. UPC++ also features Remote Procedure Call (RPC) communication, making it easy to move computation to operate on data that resides on remote processes. UPC++ was designed to support exascale high-performance computing, and the library interfaces and implementation are focused on maximizing scalability. In UPC++, all communication operations are syntactically explicit, which encourages programmers to consider the costs associated with communication and data movement. Moreover, all communication operations are asynchronous by default, encouraging programmers to seek opportunities for overlapping communication latencies with other useful work. UPC++ provides expressive and composable abstractions designed for efficiently managing aggressive use of asynchrony in programs. Together, these design principles are intended to enable programmers to write applications using UPC++ that perform well even on hundreds of thousands of cores.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2022

Implementation Aspects of Smart Grids Cyber-Security Cross-Layered Framework for Critical Infrastructure Operation

Communication networks in power systems are a major part of the smart grid paradigm. It enables and facilitates the automation of power grid operation as well as self-healing in contingencies. Such dependencies on communication networks, though, create a roam for cyber-threats. An adversary can launch an attack on the communication network, which in turn reflects on power grid operation. Attacks could be in the form of false data injection into system measurements, flooding the communication channels with unnecessary data, or intercepting messages. Using machine learning-based processing on data gathered from communication networks and the power grid is a promising solution for detecting cyber threats. In this paper, a co-simulation of cyber-security for cross-layer strategy is presented. The advantage of such a framework is the augmentation of valuable data that enhances the detection as well as identification of anomalies in the operation of the power grid. The framework is implemented on the IEEE 118-bus system. The system is constructed in Mininet to simulate a communication network and obtain data for analysis. A distributed three controller software-defined networking (SDN) framework is proposed that utilizes the Open Network Operating System (ONOS) cluster. According to the findings of our suggested architecture, it outperforms a single SDN controller framework by a factor of more than ten times the throughput. This provides for a higher flow of data throughout the network while decreasing congestion caused by a single controller’s processing restrictions. Furthermore, our CECD-AS approach outperforms state-of-the-art physics and machine learning-based techniques in terms of attack classification. The performance of the framework is investigated under various types of communication attacks.

cross-layered↗

DOE OSTI2022

Turbocharger system with turbine bypass circuit to reduce emissions during engine cold start

An engine system includes an engine having first and second exhaust manifolds each having outlet ports and a bypass runner. A first turbocharger is in fluid communication with the outlet port of the first manifold, and a second turbocharger in fluid communication with the outlet port of the second manifold. An exhaust aftertreatment device is in fluid communication with the first and second turbochargers. A turbocharger bypass circuit includes a valve assembly having an inlet side connected in fluid communication with the bypass runners of the first and second manifolds, an outlet side in fluid communication with the aftertreatment device, and a valve having an open position in which the inlet and outlet sides are in fluid communication and a closed position in which the inlet and outlet sides are not in fluid communication.

Plagens, Keith Michael↗

DOE OSTI2025

Hong-Ou-Mandel interference with a coexisting clock using transceivers for synchronization over deployed fiber

Interference between independently generated photons is a key step towards distributing entanglement over long distances, but it requires synchronization between the distantly-located photon sources. Synchronizing the clocks of such photon sources using coexisting two-way classical optical communications over the same fiber that transports the quantum photonic signals is a promising approach for achieving photon-photon interference over long distances, enabling entanglement distribution for quantum networking using the deployed fiber infrastructure. Here, we demonstrate photon-photon interference by observing the Hong-Ou-Mandel dip between two distantly-located sources: a weak coherent-state source obtained by attenuating the output of a laser and a heralded single-photon source. We achieve a maximum dip visibility of 0.58 +/- 0.04 when the two sources are connected via 4.3 km of deployed fiber. Dip visibilities > 0.5 are nonclassical and a first step towards achieving teleportation over the deployed fiber infrastructure. In our experiment, the classical optical communication is achieved with - 21 dBm of optical signal launch power, which is used to synchronize the clocks in the two independent, distantly-located photon sources. The impact of spontaneous Raman scattering from the classical optical signals is mitigated by appropriate choice of the quantum- and classical-channel wavelengths. All equipment used in our experiment (the photon sources and the synchronization setup) is commercially available. Finally, our experiment represents a scalable approach to enabling practical quantum networking with commercial equipment and coexistence with classical communications in optical fiber.

47 OTHER INSTRUMENTATION↗

DOE OSTI2020

Scalable line and plane relaxation in a parallel structured multigrid solver

The efficient solution of sparse, linear systems that arise through the discretization of partial differential equations remains a key challenge for a range of high performance scientific simulations. One approach for reducing data movement and improving performance is by exposing and exploiting structure in a problem through the use of robust structured multilevel solvers. By choosing coarsening that preserves the structure of the problem, these methods maintain efficient structured computation and communication throughout the multigrid hierarchy. However, when coarsening is not permitted to be dependent on the operator, anisotropy must be addressed by the smoother — producing error compatible for coarse-grid correction with structured coarsening. Here, the components required in a scalable parallel structured solver are described with a focus on memory and communication efficiency of robust smoothers. While the implementation of communication and memory reduction techniques in smoothers integrated in a complete 3D solver present a significant engineering challenge, a novel approach is proposed that addresses these challenges systematically through a change to the solver’s execution model. Enabled by user-level threading paired with a set of data and communication abstractions, this approach permits seamless aggregation of communication in plane smoothers — directly reusing code for a 2D distributed multilevel cycle. Results show an effective reduction in communication costs for coarse-grid problems, and result in a speedup of 8.7x in smoothing routines shown in Fig. 12 using this approach. This produces a significant improvement to strong scalability while maintaining favorable weak scaling behavior. Finally, a parallel scaling study using a series of refined meshes is included that demonstrates the effectiveness of this approach in an application of interest.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2021

Transmission of Images on High-Temperature Nuclear-Grade Metallic Pipe with Ultrasonic Elastic Waves

Transmission of information using elastic ultrasonic waves on existing metallic pipes provides an alternative communication option for a nuclear facility. The advantages of this approach consist of transmitting information through barriers, such as the containment building wall, with minimal modification of the existing hardware. Because bit rates on the order of kilobits per second are achievable, relatively large volumes of data, such as images, can be transmitted. A viable candidate for an ultrasonic communication channel is a stainless steel pipe of the chemical volume control system (CVCS) that penetrates through the reactor containment building wall through a sealed tunnel. To study ultrasonic communication under simulated nuclear facility conditions of high temperature, a test article was developed by installing heating tapes, temperature controllers, and thermal insulation on a laboratory CVCS-like stainless steel pipe. High temperature and radiation-resilient lithium niobate ultrasonic transducers were utilized for information transmission on the heated pipe. The amplitude shift keying (ASK) digital communication protocol was developed and implemented in a GNU Radio software-defined radio environment. A root-raised-cosine filter was introduced to suppress ultrasonic transducer ringing and thus reduce inter-symbol interference. This resulted in the enhancement of the data transmission bit rate compared to information encoding with square pulses. Demonstrations of communication at high temperature included transmission of a 90-KB image at the bit rate of 10 Kbps with a bit error rate of 10 -3 across a 6-ft-long straight pipe heated up to 230 degrees C. Additional preliminary studies were conducted to evaluate ultrasonic communication system resilience to environmental degradation and damage.

42 ENGINEERING↗

DOE OSTI2021

Impact of Blockchain Delay on Grid-Tied Solar Inverter Performance

This paper investigates the impact of the delay resulting from a blockchain, a promising security measure, for a hierarchical control system of inverters connected to the grid. The blockchain communication network is designed at the secondary control layer for resilience against cyberattacks. To represent the latency in the communication channel, a model is developed based on the complexity of the blockchain framework. Taking this model into account, this work evaluates the plant’s performance subject to communication delays, introduced by the blockchain, among the hierarchical control agents. In addition, this article considers an optimal model-based control strategy that performs the system’s internal control loop. The work shows that the blockchain’s delay size influences the convergence of the power supplied by the inverter to the reference at the point of common coupling. In the results section, real-time simulations on OPAL-RT are performed to test the resilience of two parallel inverters with increasing blockchain complexity.

41 EE - Solar Energy Technologies Office (EE-4S)↗

DOE OSTI2022

Cross-Layered Cyber-Physical Power System State Estimation towards a Secure Grid Operation

In the Smart Grid paradigm, this critical infrastructure operation is increasingly exposed to cyber-threats due to the increased dependency on communication networks. An adversary can launch an attack on a power grid operation through False Data Injection into system measurements and/or through attacks on the communication network, such as flooding the communication channels with unnecessary data or intercepting messages. A cross-layered strategy that combines power grid data, communication grid monitoring and Machine Learning based processing is a promising solution for detecting cyberthreats. In this paper, an implementation of an integrated solution of a cross-layer framework is presented. The advantage of such a framework is the augmentation of valuable data that enhances the detection of anomalies in the operation of power grid. IEEE 118-bus system is built in Simulink to provide a power grid testing environment and communication network data is emulated using SimComponents. The performance of the framework is investigated under various FDI and communication attacks.

cyber security, network security, cyber-physical s↗

DOE OSTI2022

Employing Interacting Qubits for Distributed Microgrid Control

To empower flexible and scalable operations, distributed control of multi-inverter microgrids, based on classical communication networks among distributed energy resources, has attracted considerable attention as it can guarantee synchronization and provide suitable remedies to the problem of improper power sharing. Notwithstanding this, resilience of the current schemes on classical communication makes microgrids vulnerable to cyber attacks. Inspired by recent revolutionary breakthroughs in quantum communication, in this paper, we devise a novel synchronization mechanism. We extend the synchronization framework utilized in distributed control algorithms to networks of quantum systems by generating pinning terms and coupling mechanism for the new synchronization rule via exploiting proper quantum jump operators and observables, and show that the quantum system will converge to a time-variant target state. Our devised quantum distributed controller (QDC) gives rise to a novel quantum communication scheme for distributed control of microgrids and enables microgrids to exploit the state-of-the-art quantum communication frameworks as communication infrastructure. Finally, test results on two representative AC and DC networked microgrids validate the efficacy and universality of the quantum distributed control.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

DOE OSTI2024

Designing and prototyping extensions to the Message Passing Interface in MPICH

As HPC system architectures and the applications running on them continue to evolve, the MPI standard itself must evolve. The trend in current and future HPC systems toward powerful nodes with multiple CPU cores and multiple GPU accelerators makes efficient support for hybrid programming critical for applications to achieve high performance. However, the support for hybrid programming in the MPI standard has not kept up with recent trends. The MPICH implementation of MPI provides a platform for implementing and experimenting with new proposals and extensions to fill this gap and to gain valuable experience and feedback before the MPI Forum can consider them for standardization. Here, in this work, we detail six extensions implemented in MPICH to increase MPI interoperability with other runtimes, with a specific focus on heterogeneous architectures. First, the extension to MPI generalized requests lets applications integrate asynchronous tasks into MPI’s progress engine. Second, the iovec extension to datatypes lets applications use MPI datatypes as a general-purpose data layout API beyond just MPI communications. Third, a new MPI object, MPIX_Stream, can be used by applications to identify execution contexts beyond MPI processes, including threads and GPU streams. MPIX stream communicators can be created to make existing MPI functions thread-aware and GPU-aware, thus providing applications with explicit ways to achieve higher performance. Fourth, MPIX Streams are extended to support the enqueue semantics for offloading MPI communications onto a GPU stream context. Fifth, thread communicators allow MPI communicators to be constructed with individual threads, thus providing a new level of interoperability between MPI and on-node runtimes such as OpenMP. Lastly, we present an extension to invoke MPI progress, which lets users spawn progress threads with fine-grained control to adapt the communication performance to their application designs. We describe the design and implementation of these extensions, provide usage examples, and highlight their expected benefits with performance results.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2022

Energy Resilience for Mission Assurance: Agile Co-simulation for Cyber Energy System Security (ACCESS), Model Advancements for Resilience Analysis

Agile Co-simulation for Cyber Energy System Security (ACCESS) is a co-simulation platform developed by Lawrence Livermore National Laboratory (LLNL). The primary high-level use-case for ACCESS is to study existing or new cyber-physical critical infrastructure systems, with a heavy emphasis on 1) systems that utilize communication networks, and 2) studies that seek to understand cyber-related system impacts. ACCESS is currently used for several energy system resilience projects at LLNL. In the Energy Resilience for Mission Assurance (ERMA) project, ACCESS is used in the Modeling for Metric Calculation task (specifically, subtask 4.3, Communications and Cyber Modeling) to model and simulate the cyber and communication system aspects of Defense Critical Electric Infrastructure (DCEI) systems, with a focus on computing specific communication system metrics that can impact system resilience and mission performance. Simulated communication system performance will be fed back to other ERMA system components so that mission performance can be evaluated holistically. This report describes several enhancements to the ACCESS platform that were implemented during the execution of the ERMA project in support of reslience analysis. This includes the addition of new models and subsystems, enhancements to existing models, and integration with external systems. The remainder of this report is structured as follows. In Section 2, a brief background description of the ACCESS platform is provided, including an outline of ACCESS components, example usecases, and a set of communication network resilience metrics that can be computed with ACCESS. Section 3 describes the ACCESS model enhancements for ERMA in detail. Finally, Section 4 briefly outlines future integration opportunities between ACCESS and project participant capabilities identified during the progression of the project.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2023

Energy Resilience for Mission Assurance, Agile Co-simulation for Cyber Energy System Security (ACCESS) Model Advancements for Resilience Analysis (Version 3, July 2023)

Agile Co-simulation for Cyber Energy System Security (ACCESS) is a co-simulation platform developed by Lawrence Livermore National Laboratory (LLNL). The primary high-level use-case for ACCESS is to study existing or new cyber-physical critical infrastructure systems, with a heavy emphasis on 1) systems that utilize communication networks, and 2) studies that seek to understand cyber-related system impacts. ACCESS is currently used for several energy system resilience projects at LLNL. In the Energy Resilience for Mission Assurance (ERMA) project, ACCESS is used in the Mod eling for Metric Calculation task (specifically, subtask 4.3, Communications and Cyber Modeling) to model and simulate the cyber and communication system aspects of Defense Critical Electric Infrastructure (DCEI) systems, with a focus on computing specific communication system metrics that can impact system resilience and mission performance. Simulated communication system per formance will be fed back to other ERMA system components so that mission performance can be evaluated holistically. This report describes several enhancements to the ACCESS platform that were implemented during the execution of the ERMA project in support of reslience analysis. This includes the addition of new models and subsystems, enhancements to existing models, and integration with external systems. The remainder of this report is structured as follows. In Section 2, a brief background description of the ACCESS platform is provided, including an outline of ACCESS components, example use cases, and a set of communication network resilience metrics that can be computed with ACCESS. Section 3 describes the ACCESS model enhancements for ERMA in detail. Finally, Section 4 briefly outlines future integration opportunities between ACCESS and project participant capabilities identified during the progression of the project.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗