Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “parallel communication”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

261 records · Page 15
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

Isochronous Architecture-Based Voltage-Active Power Droop for Multi-Inverter Systems

This article proposes an isochronous architecture for parallel inverters with only voltage-active power droop (VP-D) control for improving active power sharing as well as plug-and-play of multi-inverter-based distributed energy resources (DERs). The isochronous framework obviates the need for an explicit regulation of the frequency while allowing for sharing of reactive power. The article shows that the detrimental effects of circulating currents between inverters can be addressed in the framework developed. The isochronous architecture is implemented by employing a GPS to disseminate clock timing signals that enable the microgrid to maintain the nominal system frequency. Small signal eigenvalue analysis of a Multi-inverter Microgrid system near the steady-state operating point is presented to evaluate the system stability. Moreover, unlike traditional droop-based methods, the isochronous strategy lends itself to analytical guarantees which are developed in the article. Furthermore, the effect of delays in receiving GPS clock signals by an inverter is experimentally characterized and an inverter isolation mechanism is proposed that disconnects inverters facing large GPS communication delays. Hardware experiments on an 1.2 kVA-prototype and controller-hardware-in-the-loop validation on the CIGRE distribution network are conducted to demonstrate the effectiveness of the proposed architecture towards active and reactive power sharing between inverters with load scenarios.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2020

Multi-GPU immersed boundary method hemodynamics simulations

Large-scale simulations of blood flow that resolve the 3D deformation of each comprising cell are increasingly popular owing to algorithmic developments in conjunction with advances in compute capability. Among different approaches for modeling cell-resolved hemodynamics, fluid structure interaction (FSI) algorithms based on the immersed boundary method are frequently employed for coupling separate solvers for the background fluid and the cells within one framework. GPUs can accelerate these simulations; however, both current pre-exascale and future exascale CPU-GPU heterogeneous systems face communication challenges critical to performance and scalability. In this paper, we describe, to our knowledge, the largest distributed GPU-accelerated FSI simulations of high hematocrit cell-resolved flows with over 17 million red blood cells. We compare scaling on a fat node system with six GPUs per node and on a system with a single GPU per node. Through comparison between the CPU- and GPU-based implementations, we identify the costs of data movement in multiscale multi-grid FSI simulations on heterogeneous systems and show it to be the greatest performance bottleneck on the GPU.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Versatile soil gas concentration and isotope monitoring: optimization and integration of novel soil gas probes with online trace gas detection

Abstract. Gas concentrations and isotopic signatures can unveil microbial metabolisms and their responses to environmental changes in soil. Currently, few methods measure in situ soil trace gases such as the products of nitrogen and carbon cycling or volatile organic compounds (VOCs) that constrain microbial biochemical processes like nitrification, methanogenesis, respiration, and microbial communication. Versatile trace gas sampling systems that integrate soil probes with sensitive trace gas analyzers could fill this gap with in situ soil gas measurements that resolve spatial (centimeters) and temporal (minutes) patterns. We developed a system that integrates new porous and hydrophobic sintered polytetrafluoroethylene (sPTFE) diffusive soil gas probes that non-disruptively collect soil gas samples with a transfer system to direct gas from multiple probes to one or more central gas analyzer(s) such as laser and mass spectrometers. Here, we demonstrate the feasibility and versatility of this automated multiprobe system for soil gas measurements of isotopic ratios of nitrous oxide (δ18O, δ15N, and the 15N site preference of N2O), methane, carbon dioxide (δ13C), and VOCs. First, we used an inert silica matrix to challenge probe measurements under controlled gas conditions. By changing and controlling system flow parameters, including the probe flow rate, we optimized recovery of representative soil gas samples while reducing sampling artifacts on subsurface concentrations. Second, we used this system to provide a real-time window into the impact of environmental manipulation of irrigation and soil redox conditions on in situ N2O and VOC concentrations. Moreover, to reveal the dynamics in the stable isotope ratios of N2O (i.e., 14N14N16O, 14N15N16O, 15N14N16O, and 14N14N18O), we developed a new high-precision laser spectrometer with a reduced sample volume demand. Our integrated system – a tunable infrared laser direct absorption spectrometry (TILDAS) in parallel with Vocus proton transfer reaction mass spectrometry (PTR-MS), in line with sPTFE soil gas probes – successfully quantified isotopic signatures for N2O, CO2, and VOCs in real time as responses to changes in the dry–wetting cycle and redox conditions. Broadening the collection of trace gases that can be monitored in the subsurface is critical for monitoring biogeochemical cycles, ecosystem health, and management practices at scales relevant to the soil system.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

Trust: Triangle Counting Reloaded on GPUs

Triangle counting is a building block for a wide range of graph applications. Here, traditional wisdom suggests that i) hashing is not suitable for triangle counting, ii) edge-centric triangle counting beats vertex-centric design, and iii) communication-free and workload balanced graph partitioning is a grand challenge for triangle counting. On the contrary, we advocate that i) hashing can help the key operations for scalable triangle counting on Graphics Processing Units (GPUs), i.e., list intersection and graph partitioning, ii) vertex-centric option reduces both hash table construction cost and memory consumption, which is limited on GPUs. In addition, iii) we exploit graph and workload collaborative, and hash-based 2D partitioning to scale vertex-centric triangle counting over 1,000 GPUs with sustained scalability. In this work, we present TRUST, which performs triangle counting with the hash operation and vertex-centric paradigm. To the best of our knowledge, TRUST is the first work that achieves over one trillion Traversed Edges Per Second (TEPS) rate for triangle counting.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Flow regime and Reynolds number variation effects on the mixing behavior of parallel flows

The hydraulic single-phase mixing of three parallel rectangular channels is experimentally investigated at various Reynolds numbers (Re) and flow regime combinations. Particle Image Velocimetry results for seven mixing cases are presented and discussed with varying Re combinations ranging from 1,824 to 20,844. While all cases result in the same Re ratio of ~0.69 between the inner and outer flows, two cases represent multi-regime mixing with the inner-outer regime pair of laminar-transitional and transitional-turbulent, while the other 5 cases are all characteristic of turbulent mixing with varying levels of turbulence. The outer channels initially share characteristics with a backward facing step. The center channel is found to initially behave like a slot jet, but then sees a significant increase in velocity decay. This inner flow velocity decay increased dramatically in the laminar-transitional mixing case, whose centerline velocity decay was ~6 times larger than the decay in the turbulent mixing cases. Second order statistics revealed a consistent mixing layer thickness of ~0.1 hydraulic diameters for all the cases but showed more intense shearing in the multi-regime mixing cases. The combined point and thereby the mixing layer length is determined using centerline velocity decay profiles, which show a much more aggressive mixing in multi-regime flows. Multi-regime mixing demonstrated superior characteristics relative to turbulent mixing due to a more dramatic velocity decay in the inner flow and a shorter mixing length. The contributions of this work include communicating the benefits of multi-regime mixing and providing detailed characterization efforts that can serve future efforts for validating computational models. Here this research also lays the groundwork for future studies aimed at achieving high levels of mixing without a severe penalty in pressure drop.

42 ENGINEERING↗
DOE OSTI2024

Research Reactors Division Infrastructure Investment Plan for the High Flux Isotope Reactor

The High Flux Isotope Reactor (HFIR) is a unique national asset. Operational for nearly 60 years, continued investment into the aging infrastructure is necessary to ensure operation for another 6 decades. Additionally, growing missions require HFIR as well as important upgrades. Consequently, carefully integrated planning is required to ensure that infrastructure investments are timely executed to ensure long-term, reliable operation of HFIR. Concerns about challenges to the operational reliability of HFIR resulted in a recommendation from the 2023 Operations Review by the US Department of Energy (DOE) Office of Basic Energy Sciences that a HFIR management strategy be developed to address the infrastructure needs. This report defines the investment needs, which are evolving as new upgrade efforts are better defined. HFIR is part of the three-source strategy within the Neutron Sciences Directorate (NScD) and contributes to the five strategic science areas outlined in the NScD 10 Year Strategic Science Plan: quantum materials, soft matter, materials and engineering, chemistry, and biosciences. Fundamental to this strategy are three core values: operational excellence, responsible stewardship, and servant leadership. These values guide our mission of safe and reliable operation of the reactor and require a strong and just nuclear safety culture, a solemn respect for responsible care of the facility, good workforce development, robust procedures and processes, an effective communication strategy, world-class asset management, a determined customer focus, and a commitment to protecting the environment, the safety and health of the public and our people, and the quality of work performed within our facility. These principles are all essential to operate HFIR at a world-class level. The Research Reactors Division (RRD) will lead a new era of neutron science and isotope production at HFIR through responsible and purposeful leadership and unwavering support of the science community. The approach outlined in this plan highlights the direction leadership is taking to ensure that HFIR is ready to support the science challenges and national needs of the future and that the United States maintains world leadership in neutron sciences. The plan is in alignment with the DOE’s desire to continue operating HFIR and with the NScD strategic science goals for the future. HFIR is an aging facility with numerous infrastructure challenges and needs. It has an aging workforce in relation to the general population of Oak Ridge National Laboratory (ORNL), with many expected retirements over the next 5–10 years. With an increase in work scope caused by changing national priorities and science goals, several critical hires have been identified. To manage HFIR’s infrastructure needs, a prioritized list of equipment upgrades has been identified along with an analysis of future staffing requirements. A desire to operate HFIR at eight cycles per year will necessarily require some significant changes to procedures and processes currently in place as well as targeted staffing additions. Many of the equipment upgrades identified in this plan will significantly increase the reliability of the plant, thus contributing to the effort to reach the goal of safely operating eight cycles per year. A plan to attain eight-cycle operation is being prepared in parallel with the activities identified in this plan, although the actions identified to satisfy both plans will overlap. This plan identifies new infrastructure needs—for both plant equipment and staffing—thus necessitating formulation of future budget requests to fund the increased work scope and improvement activities. Some activities are currently being scheduled with the expectation that funding will be received. Any delays to funding or reductions of funding from the identified cost estimations will directly and negatively affect the plan’s implementation.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
DOE OSTI2024

Adaptive Protection and Validated Models to Enable Deployment of High Penetrations of Solar PV (PV-MOD)

The availability and validation of various PV models in commercial tools differ, with some models not yet thoroughly validated for advanced inverter functionalities and reliable performance under weak system conditions. Many existing models do not fully incorporate new inverter control functions, which can affect system stability. The increasing deployment of solar PV and other inverter-based resources (IBRs), including distributed energy resources (DERs), is influencing the reliable operation of protection schemes in distribution systems and microgrids. Emerging adaptive protection schemes (APS) offer new opportunities for protecting these systems during varying configurations and DER operating conditions, though their demonstration and validation remain limited. Adaptive protection schemes face similar challenges, as they are typically designed for specific configurations. There is a growing need for tools and methodologies to streamline the deployment of adaptive protection for safe and reliable DER integration. The project main objective was to develop and validate high-fidelity generic models of solar PV facilities for stability, protection, EMT, and QSTS analyses. This objective was achieved, and these models can now be integrated into commercial software tools, enabling utilities, vendors, and developers to study high-penetration PV systems more confidently. The project also demonstrated advanced applications of these models, including the design and deployment of adaptive protection schemes in high-penetration field applications and microgrids, supporting grid safety and reliability. Several milestones were reached by the end of the project. A sophisticated inverter test plan was developed, and inverters representative of the North American marketplace were selected. EPRI and NREL tested various inverters, conforming to IEEE standards. Improvements were made to existing generic models of IBR units, IBR plants, and aggregated feeders for various analyses. The first generic electromagnetic transient (EMT) model for a solar PV plant was developed, conforming to IEEE Std 2800™-2022 and validated against laboratory measurements of a 2.2 MVA large-scale battery energy storage system (BESS) inverter. That model was then used to produce reference responses illustrating examples of validated and verified IBR plant models that pass or fail tests for technical minimum capability and performance as specified in the IEEE standard. The developed, tested, and validated generic models can be used for transmission planning, stability assessments, expansion planning, and evaluating potential future IBR interconnection requirements. They can also support interconnection screens and conformity assessments of IBR plants, including solar PV. The project significantly contributed to the ongoing standardization and model-based representation and verification of IBR responses. The project further addressed challenges of common distribution protection schemes with increasing deployment of DER by developing, validating, and demonstrating adaptive protection schemes (APS) that can improve the reliable and safe integration of DER into distribution systems. New APS were designed using improved DER models for three common distribution systems: a radial feeder, a meshed network, and a microgrid. Modeling and hardware-in-the-loop (HIL) testing of the APS were conducted, successfully showing their effectiveness and selectivity. Proof-of-concept field demonstration was achieved for two APS, i.e., one on a radial feeder and another one in a microgrid. Field demonstration could not be achieved for the APS on a meshed network, primarily due apprehension of one utility partner and also due to limited access to the protective algorithms in the network protectors. Guidelines developed from the lessons learned in the project lay out the general process followed in the design, installation, and commissioning of APS for various distribution systems. Distribution utility partners’ apprehension about field demonstration of the new APS were addressed—with varying success—by taking a stepped risk-management approach of modeling of a wide range of sensitivities first, performing in-depth proof-of-concept testing in the laboratory including HIL next, and finally deliberately implementing and commissioning the actual protection equipment and algorithms into parts of—or in parallel operation to—the three real distribution systems. Future work should include pilot projects that further show the acceptable performance of the developed APS before these schemes be rolled out more widely. Inclusion of both utility and original equipment manufacturers (OEMs) in future projects could increase chances of successful field demonstration. Despite challenges in achieving the field demonstration goal of the project for all three APS, the research significantly contributed to the innovation of adaptive protection solutions for scalable and reliable DER integration into distribution systems. This project significantly enhances the understanding of the impact of using appropriate inverter models on distribution and transmission (T&D) systems. By addressing the limitations of existing generic models, the project introduces high-fidelity models for stability, protection, electromagnetic transient (EMT), and quasi-static time series (QSTS) analyses. These models, integrated into commercial software tools, enable utilities, vendors, and developers to confidently study high-penetration PV systems. The project also demonstrates advanced applications, including adaptive protection schemes (APS) for distribution systems and microgrids, ensuring grid safety and reliability. The technical effectiveness and economic feasibility of the methods are evident through the development and validation of sophisticated inverter test plans and the selection of representative inverters. Testing by EPRI and NREL on retail, commercial, and utility-scale inverters, conforming to IEEE standards, underscores the robustness of the models. Improvements to existing generic models for various analyses further enhance their validity and applicability. The project also identifies gaps in common distribution protection schemes and designed new APS using improved DER models, demonstrating their effectiveness through modeling and hardware-in-the-loop (HIL) testing. The project’s benefits to the public are manifold. By advancing the standardization and model-based representation of IBR response, it supports transmission planning, stability assessments, and future IBR interconnection requirements. The generic models can facilitate better communication between transmission planners and developers, supporting expected IBR plant capability and performance. Additionally, the development of APS for radial feeders, meshed networks, and microgrids supports the integration of distributed energy resources (DERs) into distribution systems, enhancing grid reliability and safety. The project’s emphasis on thorough testing and simplicity in design ensures practical and scalable solutions for DER integration.

14 SOLAR ENERGY↗
DOE OSTI2024

Developing And Scaling an OpenFOAM Model to Study Turbulent Flow in a HFIR Coolant Channel

Improving the understanding of how computational fluid dynamics (CFD) direct numerical simulations (DNS) of flows in the High Flux Isotope Reactor (HFIR) perform when run in parallel using the high performance computing (HPC) platform Summit at the Oak Ridge Leadership Computing Facility (OLCF) is of particular importance to boost the computational tools used to support HFIR conversion to low enriched fuel (LEU). Evaluation of scaling performance was driven by the increasing importance of graphics processing unit (GPU) usage in HPC, which is becoming the standard for modern supercomputers such as Summit. The desired results are to obtain a strong positive correlation between the computational resources dedicated to a problem and the relative speed-up of the simulation in comparison to a benchmark. This capability will allow substantially improvement in HFIR flow analytical capabilities, specifically when predicting turbulence properties at high Reynolds numbers. The study leverages previous simulation results performed with code PHASTA (finite element) on HPC platforms Cori (NERSC) and Theta (ALCF) [1] with computing options provided in the computing platform OpenFOAM (finite volume) at OLCF. Transitioning from PHASTA to OpenFOAM will (1) eliminate dependence on third-party software for mesh generation and manipulation, (2) reduce resource needs by employing modern architectures, and (3) build expertise for future modeling of HFIR-specific problems like heat transfer in involute geometry, entrance effects, flow structure in channel corners, and so on—all important issues when defining the available thermal margins in the transition to LEU. CPUs and GPUs differ significantly in their architecture and utilization, as discussed in the literature [2]. The most important differences are in the approach to computations and their memory. A single GPU contains a large quantity of cores, enabling it to perform with a much higher throughput than a CPU, but execution requires a different approach. GPU codes execute instructions using the Single-Instruction Multiple-Thread (SIMT) approach in which a single instruction is used for groups of threads called warps. A warp typically consists of 32 threads which must execute the same set of instructions, although on separate threads. Alternately, a CPU has far fewer cores that are much more flexible in their operation, excelling at quickly performing more complex serial computations. This is why GPUs have greater throughput when properly utilized. The second important difference is seen when comparing their memory spaces. Limited memory allocations and CPU–GPU communications cause a significant bottleneck in GPU-accelerated programs. Further study was required to properly take advantage of GPU resources. A comprehensive analysis of code performance and the model-specific features of turbulence constitutes the core of this work. In this study, a DNS simulation of HFIR channel turbulence was performed with the finite volume CFD code OpenFOAM v2112 and CUDA v11.0 on Red Hat Enterprise Linux v8.2. The OpenFOAM installation had AMGx integrated to enable GPU acceleration and utilizes the PETSc4FOAM library. The computational resources and the problem size were scaled on CPU and CPU + GPU architectures to gain a better understanding of the performance of a DNS problem on modern computing hardware. The study aimed to analyze the scaling of the code exclusively on CPUs and then to examine the scaling of the codes with GPU acceleration enabled. Scaling studies included CPU and GPU acceleration on a mesh of varying resolution to analyze the impact of problem size relative to computational resources. In the course of preparing the GPU configuration on Summit, mainly using the AMGX solvers, difficulties were encountered stemming from constant changes resulting from extensive ongoing development activities and the changing environment. This resulted in the inability to complete the GPU portion of the work. The code was compiled and tested, but production runs to assess acceleration were not performed because the used discretional compute time allocation expired as year-end approached. The Summit HPC platform is scheduled for decommissioning in 2024, making it unattractive for future use with Nvidia-based GPUs. Therefore, the work will be moved onto NERSC machines in FY24. An application was prepared and submitted, and sufficient node-hours were awarded to continue the research in the next calendar year. This report summarizes work performed thus far, which mostly focused on CPU OpenFOAM computing.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
DOE OSTI2020

Research Roadmap on Grid-Forming Inverters

This research roadmap is intended to fill the knowledge gap by providing a system view of grid-forming inverter-based resource controls and their impact on grid stability, which we believe is central to meeting some of the challenges to operating the future North American electric power system. This includes the roles and requirements of grid-forming inverter-based resources—including solar photovoltaics, wind generators, and energy storage. For this roadmap, we focus on a specific family of grid-forming inverter control approaches that do not rely on an external voltage source (i.e., no phase-locked loop) and that can share load without explicit communications. Although the roadmap is focused narrowly on system challenges for grid-forming controls and power system stability, including interactions with protection, we hope it serves as a foundational element for future system-of-systems roadmapping needed in a broader grid modernization effort with increasing deployments of inverter-based resources. The roadmap first introduces formal definitions for the grid stability topics and then describes the differences between grid-forming and traditional grid-following control approaches for inverter-based resources. The core of the roadmap consists of a review of current research and an outline of research needs related to five grid-forming inverter topics: frequency control, voltage control, system protection, fault ride-through and voltage recovery, and modeling and simulation. The review both delineates contemporary advances and highlights open research questions that must be addressed to enable the widespread adoption of inverter-based resources across the grid. Feedback from industry on these research questions is incorporated, including discussions during the Workshop on Grid-forming Inverters for Low-inertia Power Systems. The workshop included industry presentations and discussion of ongoing research, technology gaps, and piloting needs. This roadmap concludes by offering a multiyear perspective on the gradual field validation of grid-forming inverters. This perspective recognizes that the scale and scope of the types of power systems that inverters will be called on to provide grid-forming services will and should begin modestly. Specifically, this roadmap recognizes that inverter controls today are predominantly grid-following and that future power systems will involve a mix of inverter-based resources with both grid-following and grid-forming control capabilities. Growth over time will depend on how well grid-forming inverters perform and what advantages they bring as penetration levels (instantaneous and average) of inverter-based resources increases. This recognition, in turn, establishes a natural sequence of priorities for the research questions that must be addressed. Following this multiyear perspective, the roadmap offers short descriptions of two specific near-term research priorities: the review of regulatory and technical standards and the development of advanced modeling techniques. These priorities are foundational. We recommend immediate pursuit of them in parallel with and in direct support of the research outlined by our multiyear perspective.

24 POWER TRANSMISSION AND DISTRIBUTION↗