Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Communication architecture”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 397 records · Page 22
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

A Review of Distributed Secondary Control Architectures in Islanded-Inverter-Based Microgrids

The increasing energy demand, the shortage of energy resources, and the environmental challenges faced by conventional power-generation systems are some of the ongoing challenges faced by modern power systems. Therefore, many efforts have been made by the scientific community to develop comprehensive solutions to overcome these issues. For instance, current technological advances have allowed the integration of distributed generators into the power systems, promoting the use of microgrids to overcome these issues. However, the use of renewable distributed generators have introduced new challenges to the traditional control system schemes. To overcome these challenges, a hierarchical control approach has been proposed for distributed renewable sources. In other words, the control scheme have been divided into three hierarchical levels, primary, secondary, and tertiary, to overcome the new challenges present in modern power systems. Due to extensiveness of this topic, this overview is focused on secondary control systems, mainly for AC isolated microgrids. To improve the power quality of modern systems, several secondary control schemes have been proposed to overcome the well-known problem of frequency and voltage deviation. Some of these schemes have also introduced adequate active/reactive power sharing techniques to optimize the utilization of resources. Additionally, other secondary control schemes have also focused on reducing the communication load, to lower the network cost and adding robustness against communication problems. This article presents an insight of the different control techniques used to overcome power quality and communication problems. A comprehensive overview of distributed secondary control techniques for islanded microgrids is presented. In addition, the implementation of these techniques is explained in an orderly and sequential manner.

14 SOLAR ENERGY↗
DOE OSTI2023

A Hybrid Optimization and Deep Learning Algorithm for Cyber-Resilient DER Control

With the proliferation of distributed energy resources (DERs) in the distribution grid, it is a challenge to effectively control a large number of DERs resilient to the communication and security disruptions, as well as to provide the online grid services, such as voltage regulation and virtual power plant (VPP) dispatch. To this end, a hybrid feedback-based optimization algorithm along with deep learning forecasting technique is proposed to specifically address the cyber-related issues. The online decentralized feedback-based DER optimization control requires timely, accurate voltage measurement from the grid. However, in practice such information may not be received by the control center or even be corrupted. Therefore, the long short-term memory (LSTM) deep learning algorithm is employed to forecast delayed/missed/attacked messages with high accuracy. The IEEE 37-node feeder with high penetration of PV systems is used to validate the efficiency of the proposed hybrid algorithm. The results show that 1) the LSTM-forecasted lost voltage can effectively improve the performance of the DER control algorithm in the practical cyber-physical architecture; and 2) the LSTM forecasting strategy outperforms other strategies of using previous message and skipping dual parameter update.

cyber-resilient algorithm↗
DOE OSTI2024

P38 heterogeneous multi-tiled system with support for message queues (MoSAIC) v0.1

The proposed system is written in the hardware description language (HDL) verilog targeting an FPGA board. It is intended as a testbed to explore architecture tradeoffs in multi-tiled heterogeneous architectures. Although we target FPGAs, the system can be implemented as a monolithic SoC or a package comprised of many chiplets that are interconnected in the same package using a NoC. The proposed NoC is lightweight and follows an axi-lite interface. The endpoints of the NoC are a heterogeneous mix of "tiles" as endpoints that are general purpose processors, fixed function accelerators, and programmable accelerators. We assume that the network interfaces for the NoC endpoints are all addressable in a global name-space in that they represent an address range (for memory addresses) or a range of unique identifiers that are associated with each individual tile. This makes the functionality abstract from the standpoint of the NoC design details. Message queues offer a direct inter-processor interface between peer general purpose cores and diverse accelerators that comprise an SoC. Although they share the same NoC infrastructure for inter-tile communication within an SoC or SiP, the hardware message queues bypass the memory hierarchy and thus do not pollute the memory state or invoke the cache coherence mechanism.

Gonzalez, LouisaPatricia↗
DOE OSTI2025

Scaling Uintah on the Aurora Exascale System up to 122,880 Intel Ponte Vecchio Xe Stacks

The challenge of being able to scale application codes based on the Asynchronous Many-Task (AMT) Uintah framework on the Department of Energy (DOE) Aurora exascale system is addressed in this work by considering a challenging Reverse Monte Carlo Ray Tracing radiation benchmark calculation. This benchmark involves potentially global all-to-all communication and uses adaptive mesh refinement and ray tracing to achieve scalability. This benchmark has been used as part of previous scalability studies on a number of pre-exascale systems and on the DOE Frontier exascale system. This paper describes steps taken to enable this benchmark to run successfully on up to 10,240 nodes and 122,880 Intel® Ponte Vecchio Xe stacks on the DOE Aurora exascale system. This scalability was achieved through a limited number of experiments on Aurora, given machine loads and its uniqueness. These experiments constitute valuable lessons learned to achieve scalability at this level. The resulting scalability runs, while few in number, demonstrate relatively good strong-scaling characteristics. A detailed analysis of these results provides important indications about the path to scalability on Aurora for future work. Overall, these results continue the remarkable ability of this AMT approach to produce scalable solutions for challenging problems at extreme scale on heterogeneous architectures.

Garcia, Marta [Argonne National Laboratory (ANL)]↗
DOE OSTI2024

Achieving Cyber-Resilience for Power Systems using a Learning, Model-Assisted Blockchain Framework

The secure integration and management of distributed energy resources (DER) and power aggregators in the electric grid requires secure communications and a physics-aware Command and Control (C2) strategy. A Blockchain (BC)-based overlay network was developed to provide a security layer for the existing power grid network that mitigates risks in current and legacy network and C2 protocols. By integrating a Model-Assisted Machine Learning (MAML) framework with a Secure Blockchain Overlay Network (SBON) a defense-in-depth strategy was achieved. In our approach, the MAML framework leveraged a smart contract framework to gather network data and learn the dynamics of DER to develop detection strategies for attacks targeting sensors and actuators used by DER. The MAML framework learned dynamical systems models for individual DERs to detect sensor attacks. For DER we utilized a Digital Twin (DT) to accelerate the learning process for a model resistant to stealthy attacks. The project created DT for PV inverters and BESS. The DTs were coupled with a model-assisted, data-driven learning of DER behavior. Specifically, we evaluated architectures for model-based learning with model-free fine-tuning. Additionally, differential privacy techniques were used to obfuscate data, while still allowing the computation of attack detection results based on obfuscated data. The SBON developed leverages a private permissioned blockchain network orchestrated with the Hyperledger Fabric framework. To connect the cyber world, which orchestrates the blockchain fabric, and the physical world where the power network resides, we developed a system implementation to enable the secure interaction of the physical world and the abstracted blockchain.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Data Quality Monitoring for the Hadron Calorimeters Using Transfer Learning for Anomaly Detection

The proliferation of sensors brings an immense volume of spatio-temporal (ST) data in many domains, including monitoring, diagnostics, and prognostics applications. Data curation is a time-consuming process for a large volume of data, making it challenging and expensive to deploy data analytics platforms in new environments. Transfer learning (TL) mechanisms promise to mitigate data sparsity and model complexity by utilizing pre-trained models for a new task. Despite the triumph of TL in fields like computer vision and natural language processing, efforts on complex ST models for anomaly detection (AD) applications are limited. In this study, we present the potential of TL within the context of high-dimensional ST AD with a hybrid autoencoder architecture, incorporating convolutional, graph, and recurrent neural networks. Motivated by the need for improved model accuracy and robustness, particularly in scenarios with limited training data on systems with thousands of sensors, this research investigates the transferability of models trained on different sections of the Hadron Calorimeter of the Compact Muon Solenoid experiment at CERN. The key contributions of the study include exploring TL’s potential and limitations within the context of encoder and decoder networks, revealing insights into model initialization and training configurations that enhance performance while substantially reducing trainable parameters and mitigating data contamination effects.

47 OTHER INSTRUMENTATION↗
DOE OSTI2025

Room‐Temperature Mid‐Infrared Detection Using Metasurface‐Absorber‐Integrated Phononic Crystal Oscillator

Mid-infrared (MIR) detectors find extensive applications in chemical sensing, spectroscopy, communications, biomedical diagnosis, and space exploration. Alternative to semiconductor MIR photodiodes and bolometers, mechanical-resonator-based MIR detectors show advantages in higher sensitivity and lower noise at room temperature, especially toward longer wavelength infrared. Here, uncooled room-temperature MIR detectors based on lithium niobate surface acoustic wave phononic crystal (PnC) resonators integrated with wavelength-and-polarization-selective metasurface absorber arrays are demonstrated. The detection is based on the resonant frequency shift induced by the local temperature change due to MIR absorptions. The PnC resonator is configured in an oscillating mode, enabling active readout and low-frequency noise. The 1-GHz oscillator-based MIR detector shows a relative frequency deviation of 5.24 × 10 −10 Hz −1/2 at an integration time of 50 µs, leading to an incident noise equivalent power of 197 pW Hz −1/2 when input 6-µm MIR light is modulated at 1.8 kHz, and a large dynamic range of 10 7 in incident MIR power. The device architecture is compatible with the scalable manufacturing process and can be readily extended to a broader spectral range by tailoring the absorbing wavelengths of metasurface absorbers.

Xi, Zichen [Virginia Polytechnic Inst. and State U↗
DOE OSTI2020

Finding of Effect and Mitigation Documentation for the Power and Communications System in Mercury, Area 23, Nevada National Security Site, Nye County, Nevada

The U.S. Department of Energy, National Nuclear Security Administration Nevada Field Office (NNSA/NFO) proposes to make multiple upgrades and additions to the power and communications (P&C) system related to modernization activities in the town of Mercury at the Nevada National Security Site (NNSS) in Nye County, Nevada. The purpose of this letter report is to submit documentation related to the finding of effect and mitigation for the undertaking of modifications to and mitigation of the P&C system in Mercury. This submission is intended to comply with the stipulations in the Programmatic Agreement between the National Nuclear Security Administration Nevada Field Office and the Nevada State Historic Preservation Officer Regarding Modernization and Operational Maintenance of the Nevada National Security Site, at Mercury in Nye County, Nevada, hereafter referred to as the Mercury PA. The town of Mercury and the immediate surrounding area have been designated the Mercury Historic District (MHD; SHPO Resource #D230), which is a property eligible to the National Register of Historic Places under Criteria A and C for its importance in supporting nuclear testing and scientific research from 1951 through 1992. During the initial architectural survey (Reno et al. 2018), the P&C system was identified as contributing to the eligibility of the district.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

A sub-5 ps jitter time-to-digital converter ASIC with back-gate delay tuning in 22 nm CMOS

We present the design of a Time-to-Digital Converter (TDC) ASIC together with performance characterization results at cryogenic temperature (6-8 K), and at room temperature using emulated Low-Gain Avalanche Detector (LGAD) signals. The TDC design uses a two-step architecture with a ring-oscillator based counter and a Vernier delay line fine TDC. The TDC is implemented in an FD-SOI process, and back-gate tuning is used not only to correct threshold variation due to cryogenic operation, but also as a novel way to tune TDC delay elements with very little overhead. Using this technique, we demonstrate a design with very low power (0.5 mW) and area (0.003 mm 2 ). We present test results using the Fermilab Constant Fraction Discriminator (FCFD) ASIC to produce discriminated signals from an internal charge injection mechanism that mimics the waveform and signal amplitude of minimum ionizing particles impinging on LGAD sensors. We characterize the time precision of the full system and verify that the TDC ASIC contributes a negligible amount to the total system time precision, fully consistent with the expected jitter contribution of less than 5 ps. In conclusion, the results presented here demonstrate the utility of our TDC for applications in physics and quantum communications.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2024

TrustDER: Trusted, Private and Scalable Coordination of Distributed Energy Resources

In this project, the Stanford and SLAC Teams have developed a Trusted, Private and Scalable platform for coordinating Coordination of Distributed Energy Resources (TrustDER). This is a layered system that ensures private, trusted and scalable coordination and monitoring of DERs. It accommodates a variety of resources, such as solar generation, gensets and loads, with a particular focus on battery systems-based resources, as they are a transformational technology experiencing fast growth in adoption by large critical facilities. The platform can be used as standalone or added to existing aggregation systems to enable trust, privacy and resilience. TrustDER consists of layers that address each of the shortcomings of the existing state of the art. Each layer in the platform can operate independently but provides information to the layers above it to enable a novel form of overall coordination architecture. The project consists of several tasks, with each task dedicated to the design of each layer. Task 2 Resource Virtualization defined a software abstraction layer for distributed energy resources (DERs). The goal of this abstraction was to simplify the implementation of algorithms utilizing cooperation of DERs resources in a variety of use cases. Task 3 is on Secure ID for Asset Authentication. Identity Management Systems (IDMS) are a foundational infrastructure for interactions between entities (organizations, users, devices, and services). Secure ID is blockchain-based a distributed identity management system allowing (1) identity provisioning, (2) authentication, (3) authorization, and (4) identity data sharing for IoT-enabled assets on the electricity grid. In this project, the SLAC team focused on designing and testing Keymaker, a protocol for authenticating device identity managed by Secure ID. Task 5 Private and Safe Integration is focused on the design and evaluation of a DER cooperation scheme which allows for the aggregation of DERs without impacting network reliability. The approach is designed based on realistic assumptions regarding data availability, communication infrastructure limitations, and privacy. Task 6 Scalable Distributed Privacy for Information explored how virtualized batteries could be managed privately. Specifically, it examined the case in which a principal provides a partitioned battery to multiple clients. Task 7 Use Cases was to ensure that this technology was applied in relevant situations and scenarios. Primarily, this means that virtualization needed to be employed in a manner that either improved flexibility, bolstered security or privacy, or decreased costs.

25 ENERGY STORAGE↗
DOE OSTI2021

Deterministic multi-qubit entanglement in a quantum network

High-fidelity deterministic quantum state transfer and multi-qubit entanglement are demonstrated in a quantum network comprising two superconducting quantum nodes one metre apart, with each node including three interconnected qubits. The generation of high-fidelity distributed multi-qubit entanglement is a challenging task for large-scale quantum communication and computational networks. The deterministic entanglement of two remote qubits has recently been demonstrated with both photons and phonons. However, the deterministic generation and transmission of multi-qubit entanglement has not been demonstrated, primarily owing to limited state-transfer fidelities. Here we report a quantum network comprising two superconducting quantum nodes connected by a one-metre-long superconducting coaxial cable, where each node includes three interconnected qubits. By directly connecting the cable to one qubit in each node, we transfer quantum states between the nodes with a process fidelity of 0.911 +/- 0.008. We also prepare a three-qubit Greenberger-Horne-Zeilinger (GHZ) state in one node and deterministically transfer this state to the other node, with a transferred-state fidelity of 0.656 +/- 0.014. We further use this system to deterministically generate a globally distributed two-node, six-qubit GHZ state with a state fidelity of 0.722 +/- 0.021. In this work, the GHZ state fidelities are clearly above the threshold of 1/2 for genuine multipartite entanglement, showing that this architecture can be used to coherently link together multiple superconducting quantum processors, providing a modular approach for building large-scale quantum computers.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2024

MODAQ-BB (Modular Offshore Data Acquisition System - Blackbox) [SWR-24-72]

MODAQ-BB is a compact, rapid deployment data acquisition system that can withstand water depths up to 400m. Codenamed "BlackBox" (or simply BB), since its initial purpose was to track marine assets and record vital data streams that could later be recovered in the event of a mishap - much like a traditional black box used in aviation and shipping, the name has stuck. MODAQ BlackBox is a battery-operable microcontroller platform with internal inertial sensing, GPS, satellite communications, and additional I/O (input/output) support in a depth-rated pressure enclosure. Since BB is self-contained and relatively compact, it can be quickly deployed with minimal effort. The enclosure can be clamped to a tube (such as part of a railing) or mast in a location with unobstructed view of the sky using the available clamp accessory or a common hose clamp. While BB is designed to operate unattended, users can configure what data are uploaded and the frequency of satellite transmissions. Once data are uploaded, they can be relayed to an email distribution list, the MODAQ:Web operational dashboard, or a custom destination. BB has found utility in the National Laboratory of the Rockies' (NLR's) Waterpower projects beyond its original vision and has been configured and successfully deployed in more traditional data-gathering applications where a simple, battery-operated solution was indicated. As part of the MODAQ family, BB fills a space in the spectrum of missions that can be supported that were previously impractical using the traditional MODAQ hardware architecture due to factors such as weight, size, and cost.

Raye, Robert↗
DOE OSTI2024

Multilevel analysis between Physcomitrium patens and Mortierellaceae endophytes explores potential long‐standing interaction among land plants and fungi

SUMMARY The model moss species Physcomitrium patens has long been used for studying divergence of land plants spanning from bryophytes to angiosperms. In addition to its phylogenetic relationships, the limited number of differential tissues, and comparable morphology to the earliest embryophytes provide a system to represent basic plant architecture. Based on plant–fungal interactions today, it is hypothesized these kingdoms have a long‐standing relationship, predating plant terrestrialization. Mortierellaceae have origins diverging from other land fungi paralleling bryophyte divergence, are related to arbuscular mycorrhizal fungi but are free‐living, observed to interact with plants, and can be found in moss microbiomes globally. Due to their parallel origins, we assess here how two Mortierellaceae species, Linnemannia elongata and Benniella erionia , interact with P. patens in coculture. We also assess how Mollicute ‐related or Burkholderia ‐related endobacterial symbionts (MRE or BRE) of these fungi impact plant response. Coculture interactions are investigated through high‐throughput phenomics, microscopy, RNA‐sequencing, differential expression profiling, gene ontology enrichment, and comparisons among 99 other P. patens transcriptomic studies. Here we present new high‐throughput approaches for measuring P. patens growth, identify novel expression of over 800 genes that are not expressed on traditional agar media, identify subtle interactions between P. patens and Mortierellaceae, and observe changes to plant–fungal interactions dependent on whether MRE or BRE are present. Our study provides insights into how plants and fungal partners may have interacted based on their communications observed today as well as identifying L. elongata and B. erionia as modern fungal endophytes with P. patens .

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2021

INTEGRATE - Inverse Network Transformations for Efficient Generation of Robust Airfoil and Turbine Enhancements

The INTEGRATE (Inverse Network Transformations for Efficient Generation of Robust Airfoil and Turbine Enhancements) project is developing a new inverse-design capability for the aerodynamic design of wind turbine rotors using invertible neural networks. This AI-based design technology can capture complex non-linear aerodynamic effects while being 100 times faster than design approaches based on computational fluid dynamics. This project enables innovation in wind turbine design by accelerating time to market through higher-accuracy early design iterations to reduce the levelized cost of energy. INVERTIBLE NEURAL NETWORKS Researchers are leveraging a specialized invertible neural network (INN) architecture along with the novel dimension-reduction methods and airfoil/blade shape representations developed by collaborators at the National Institute of Standards and Technology (NIST) learns complex relationships between airfoil or blade shapes and their associated aerodynamic and structural properties. This INN architecture will accelerate designs by providing a cost-effective alternative to current industrial aerodynamic design processes, including: - Blade element momentum (BEM) theory models: limited effectiveness for design of offshore rotors with large, flexible blades where nonlinear aerodynamic effects dominate - Direct design using computational fluid dynamics (CFD): cost-prohibitive - Inverse-design models based on deep neural networks (DNNs): attractive alternative to CFD for 2D design problems, but quickly overwhelmed by the increased number of design variables in 3D problems AUTOMATED COMPUTATIONAL FLUID DYNAMICS FOR TRAINING DATA GENERATION - MERCURY FRAMEWORK The INN is trained on data obtained using the University of Marylands (UMD) Mercury Framework, which has with robust automated mesh generation capabilities and advanced turbulence and transition models validated for wind energy applications. Mercury is a multi-mesh paradigm, heterogeneous CPU-GPU framework. The framework incorporates three flow solvers at UMD, 1) OverTURNS, a structured solver on CPUs, 2) HAMSTR, a line based unstructured solver on CPUs, and 3) GARFIELD, a structured solver on GPUs. The framework is based on Python, that is often used to wrap C or Fortran codes for interoperability with other solvers. Communication between multiple solvers is accomplished with a Topology Independent Overset Grid Assembler (TIOGA). NOVEL AIRFOIL SHAPE REPRESENTATIONS USING GRASSMAN SPACES We developed a novel representation of shapes which decouples affine-style deformations from a rich set of data-driven deformations over a submanifold of the Grassmannian. The Grassmannian representation as an analytic generative model, informed by a database of physically relevant airfoils, offers (i) a rich set of novel 2D airfoil deformations not previously captured in the data , (ii) improved low-dimensional parameter domain for inferential statistics informing design/manufacturing, and (iii) consistent 3D blade representation and perturbation over a sequence of nominal shapes. TECHNOLOGY TRANSFER DEMONSTRATION - COUPLING WITH NREL WISDEM Researchers have integrated the inverse-design tool for 2D airfoils (INN-Airfoil) into WISDEM (Wind Plant Integrated Systems Design and Engineering Model), a multidisciplinary design and optimization framework for assessing the cost of energy, as part of tech-transfer demonstration. The integration of INN-Airfoil into WISDEM allows for the design of airfoils along with the blades that meet the dynamic design constraints on cost of energy, annual energy production, and the capital costs. Through preliminary studies, researchers have shown that the coupled INN-Airfoil + WISDEM approach reduces the cost of energy by around 1% compared to the conventional design approach. This page will serve as a place to easily access all the publications from this work and the repositories for the software developed and released through this pr...

aerodynamics↗
DOE OSTI2025

Casing Annulus Monitoring of CO 2 Injection Using Wireless Autonomous Distributed Sensor Networks

Effective and secure carbon subsurface storage, involving the deep underground injection of CO 2 into geological formations where it is permanently trapped, is paramount to mitigating CO 2 emissions (Figure I). Ensuring the integrity of these storage sites and detecting potential leakage through the casing annulus necessitates robust monitoring. This work provides the first integrated demonstration of a wireless casing-annulus monitoring architecture that can operate in highly attenuating cement-brine environments relevant to CO 2 storage. This project focused on developing and validating a novel sensor system for integration with autonomous monitoring near the cement reservoir interface. The goal was a fully integrated Technology Readiness Level (TRL) 4/5 field validation of a distributed wireless intelligent sensor system providing real-time, direct subsurface formation measurements to enhance fluid movement monitoring in the cemented casing annulus. Achieving this objective required the development and integration of 1) wireless autonomous microsensor technology by California Institute of Technology (Caltech); 2) sensor packaging and emplacement technology by Research Triangle Institute (RTI); and 3) smart well completions using wireless active casing collars and NOV pipe by the Sandia National Lab (SNL). The collaboration with the Caltech team in this project aimed to develop millimeter-scale radio frequency identification (RFID) sensors capable of detecting CO 2 , pH, and/or methane levels. These sensors are engineered to be impervious to fluids, allowing them to be mixed with cement and installed within the casing annulus. They operate using RFID protocols at frequencies of 902–928 MHz for both power and communication. A Sandia National Laboratories’ team engaged their expertise in the development of a Smart Collar system designed for the wireless data collection from these RFID sensors embedded in the cement annulus and transmission of this information to the ground surface via IntelliPipe/IntelliServ NOV drill pipe. This is accomplished through inductive coupling at the collar, which facilitates data transfer through each segment of the pipe. Because the system cannot transmit a direct current signal to power the Smart Collar, both power and communication were implemented using alternating current and electromagnetic signals at varying frequencies. Furthermore, the developed microsensor technology had to be demonstrated and validated in comparison with reference transducer measurements in a field test site at The University of Texas at Austin (UT-Austin). Although the full sensor suite did not reach field-deployment readiness, the system-level integration achieved in this project establishes a validated pathway for future incorporation of advanced microsensors.

47 OTHER INSTRUMENTATION↗
DOE OSTI2020

Optimizing the hypre solver for manycore and GPU architectures

The solution of large-scale combustion problems with codes such as Uintah on modern computer architectures requires the use of multithreading and GPUs to achieve performance. Uintah uses a low-Mach number approximation that requires iteratively solving a large system of linear equations. The Hypre iterative solver has solved such systems in a scalable way for Uintah, but the use of OpenMP with Hypre leads to at least slowdown due to OpenMP overheads. The proposed solution uses the MPI Endpoints within Hypre, where each team of threads acts as a different MPI rank. This approach minimizes OpenMP synchronization overhead and performs as fast or (up to 1.44) faster than Hypre's MPI-only version, and allows the rest of Uintah to be optimized using OpenMP. The profiling of the GPU version of Hypre shows the bottleneck to be the launch overhead of thousands of micro-kernels. The GPU performance was improved by fusing these micro-kernels and was further optimized by using Cuda-aware MPI, resulting in an overall speedup of 1.16—1.44 compared to the baseline GPU implementation. The above optimization strategies were published in the International Conference on Computational Science 2020 [1]. This work extends the previously published research by carrying out the second phase of communication-centered optimizations in Hypre to improve its scalability on large-scale supercomputers. Additionally, this includes an efficient non-blocking inter-thread communication scheme, communication-reducing patch assignment, and expression of logical communication parallelism to a new version of the MPICH library that utilizes the underlying network parallelism [2]. The above optimizations avoid communication bottlenecks previously observed during strong scaling and improve performance by up to 2 on 256 nodes of Intel Knight's Landing processor.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

SHARP-Net: Platform for Self-Healing and Attack Resilient PMU Networks

Synchrophasor technology plays a pivotal role in developing the next generation of wide-area monitoring, protection, and control in the smart grid environment. As technology and communications infrastructures evolve, however, so do the attack surfaces in the synchrophasor network that can be exploited by advanced persistent threat (APT) actors to affect power system stability and reliability. In this paper, we propose a novel platform for developing a self-healing and attack-resilient PMU network (SHARP-Net) by instituting a state-of-the-art intrusion detection system (IDS) with an intrusion mitigation system (IMS) and an alert management system (AMS). In particular, the proposed platform detects anomalies during cyberattacks on phasor data concentrators (PDCs) based on the rules defined in the IDS, then the generated alerts are published to the IMS through the AMS. The proposed IMS proceeds to take automated corrective responses to mitigate cyberattacks by reconfiguring the synchrophasor network to isolate the compromised PDCs, and it orchestrates new PDCs to prevent the future propagation of attacks. Further, the IMS restores the system's observability by reconnecting the new PDCs to make the grid attack-resilient. In this work, the SHARP-Net platform is developed by using Python-based libraries, minimega's software-defined network, and virtual machine orchestration. We implement and validate the proposed SHARP-Net architecture by testing a PMU network in the smart grid environment. SHARP-Net showed promising performance in detecting cyberattacks and mitigating them through the network reconfiguration.

computer architecture↗
DOE OSTI2021

Educating HPC Users in the use of advanced computing technology

We examine a multi-modal approach to educating and training users of an advanced computing technology testbed at the Institute for Advanced Computational Science at Stony Brook University. Ookami provides researchers worldwide with access to 176 Fujitsu A64FX compute nodes, this being the same processor technology powering the Japanese Fugaku supercomputer, the fastest computer in the world since June 2020. However, achieving high-performance on this Arm-based, leadership computing technology requires that users be familiar with details of computer architecture, performance analysis and modeling, and high-performance programming models that are commonly omitted in introductory programming courses. Indeed, regardless of their seniority, many of the testbed users are surprisingly unfamiliar with basic concepts such as vectorization, pipelining, latency/bandwidth, roofline models, computing energy/power, threads, and non-uniform memory access. These same concepts also pervade mainstream x86 technologies, so this is of widespread concern. Due to the national/global nature of our user community that is also very diverse in both discipline and experience, the inability to offer formal classes, and our experience that most people do not tend to read online documentation or training materials in sufficient depth, we have consciously employed multiple approaches that heavily emphasize (online) personal interactions and transfer of skills. Online documentation has been organized around best-practices and FAQs; twice-weekly hackathons and office hours via Zoom enable deep dives by both the team and the user community with multiple broad benefits; a Slack channel provides both real time and archived answers and discussions; and workshops, training and webinars target community needs as they arise. Furthermore, the perspective that these tools are being used in an educational setting rather than just for project communication makes them more effective and contributes to community success.

A64FX↗