Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “interconnection options”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3

Assessing DER Network Cybersecurity Defences in a Power-Communication Co-Simulation Environment

Increasing penetrations of interoperable distributed energy resources (DER) in the electric power system are expanding the power system attack surface. Maloperation or malicious control of DER equipment can now cause substantial disturbances to grid operations. Fortunately, many options exist to defend and limit adversary impact on these newly-created DER communication networks, which typically traverse the public internet. However, implementing these security features will increase communication latency, thereby adversely impacting real-time DER grid support service effectiveness. In this work, a collection of software tools called SCEPTRE were used to create a co-simulation environment where SunSpec-compliant PV inverters were deployed as virtual machines and interconnected to simulated communication network equipment. Network segmentation, encryption, and moving target defence security features were deployed on the control network to evaluate their influence on cybersecurity metrics and power system performance. The results indicated that adding these security features did not impact DER-based grid control systems but improved the cybersecurity posture of the network when implemented appropriately.

97 MATHEMATICS AND COMPUTING↗

Atlantic Offshore Wind Transmission Literature Review and Gaps Analysis

In support of the national goal to deploy 30 gigawatts of offshore wind by 2030, DOE’s Wind Energy Technologies Office has published an Atlantic Offshore Wind Transmission Literature Review and Gaps Analysis. Transmission constraints— including both land-based transmission interconnections and offshore transmission—are expected to be one of the central challenges to meeting offshore wind deployment goals. Given current trends in offshore wind development, available technology, and state-level procurement goals, most of the first 30 GW of offshore wind are anticipated to be developed along the East Coast. This report summarizes current publicly available transmission analyses along the Atlantic Coast, as well as gaps in existing analyses. Improving understanding of offshore wind transmission options, establishing coordination and aligning processes among states and regions, and quantifying the long-term impacts of offshore wind transmission options will catalyze offshore wind energy’s growth and ensure adequate transmission and a reliable grid.

17 WIND ENERGY↗

A method for assessing economic, environmental, and reliability tradeoffs of interregional transmission connecting ERCOT (the Texas grid) to the eastern and western grids

Reliable development of the power grid is an evolving concern for humanity due to extreme weather that frequently threatens power sector infrastructure. The state of Texas is a uniquely structured testbed for grid planners to study when looking for solutions to development, innovation, and overcoming such challenges. Because of its size and islanded structure, Texas is small enough to model, but big enough to matter. Texas is a global leader in energy production, energy consumption, and maintains an unusually diverse fuel mix. In addition, the state has experienced winter freezes, heat waves, wind storms, droughts and floods that have threatened power sector infrastructure or caused recent blackouts and calls for demand side conservation. One of the most devastating of these events was the North American winter storm, dubbed “Winter Storm Uri” by the Weather Channel, that froze the region in February 2021 and led to an extended power outage event that put the majority of Texan residents in darkness for days. While preparing to avoid such outage events in the future, various tools have been proposed to improve grid reliability, including energy efficiency, demand response, and distributed energy resources. An additional option would be to develop interregional transmission that connects the Texas grid to other national grids. To assess the merits of this idea, we developed a novel, universally-applicable and internationally-relevant framework to study how the Texas grid would evolve alongside access to various interregional ties. This method allows us to stress the synthetic grid structure and analyze how it would respond to the shock of a simulated winter storm event. Our method leverages open-source modeling tools, such as PowerGenome, pyGRETA, and GenX to synthesize unique zonal grid data, construct a consolidated network of model regions, and simulate different developmental pathways of capacity expansion and operational dispatch. We demonstrate our method with an analysis connecting the Electric Reliability Council of Texas (ERCOT), the grid that serves most of Texas, the Western Electricity Coordinating Council (WECC), the grid that serves the western half of the contiguous U.S., and the Eastern Interconnect, the grid that serves the eastern half of the contiguous U.S. Our results indicate that the cost-optimal capacity of interregional transmission connecting the ERCOT grid to other grids lies between 9–13 GW assuming baseline conditions. Building this amount of connecting capacity in one or multiple directions lowers the costs and emissions of development and operation by up to $16 billion and 257 million metric tonnes (MMT) respectively. Additionally, our results show that the interregional connections between ERCOT and other national grids reduce the amount of total load shed required through mild winter storm events. However, our results also show that there is a threshold of very extreme winter storm conditions, spanning multiple service areas, above which the connections exacerbate resource adequacy problems. Therefore, the results indicate that the connections need to be carefully planned alongside the rest of the grid infrastructure to avoid over-reliance on specific resources or technology options.

24 POWER TRANSMISSION AND DISTRIBUTION↗

MoorDyn V2: New Capabilities in Mooring System Components and Load Cases

MoorDyn, an open-source mooring dynamics model, is being expanded with capabilities for additional mooring system features and load cases. As floating wind turbine technology matures, mooring systems are becoming more sophisticated and more complex scenarios need to be considered in the design process. Mooring systems may have synthetic line materials, ballast/buoyancy bodies along the lines, or interconnections between platforms. Failure modes may involve multiple cascading line failures that depend on mooring system dynamics. Features recently added to MoorDyn aim to address these emerging needs. MoorDyn’s linear elasticity model has been supplemented to support user-defined stress-strain curves, which can be adjusted to represent synthetic mooring materials. Rigid six-degree-of-freedom bodies in the mooring system can now be modeled using two new model objects. “Rod” objects provide an option for rigid cylindrical bodies. They use the existing Morison equation-based hydrodynamics model and can be connected to mooring lines at either end. “Body” objects provide a generic six-degree-of-freedom rigid-body representation based on a lumped-parameter model of translational and rotational properties. Rod objects can be added to Body objects and mooring lines can be attached at any location, allowing a wide variety of submerged structures to be integrated into the mooring system. Lastly, a means of dynamically simulating mooring line failures has been implemented. These new features, currently in the C++ version of MoorDyn, are described and then demonstrated on a two-turbine shared-mooring array. A qualitative view of the results suggests the new features are functioning as expected.

49 EE - Wind and Water Power Program - Wind (EE-4W↗

MoorDyn V2: New Capabilities in Mooring System Components and Load Cases: Preprint

MoorDyn, an open-source mooring dynamics model, is being expanded with capabilities for additional mooring system features and load cases. As floating wind turbine technology matures, mooring systems are becoming more sophisticated and more complex scenarios need to be considered in the design process. Mooring systems may have synthetic line materials, ballast/buoyancy bodies along the lines, or interconnections between platforms. Failure modes may involve multiple cascading line failures that depend on mooring system dynamics. Features recently added to MoorDyn aim to address these emerging needs. MoorDyn’s linear elasticity model has been supplemented to support user-defined stress-strain curves, which can be adjusted to represent synthetic mooring materials. Rigid six-degree-of-freedom bodies in the mooring system can now be modeled using two new model objects. “Rod” objects provide an option for rigid cylindrical bodies. They use the existing Morison equation-based hydrodynamics model and can be connected to mooring lines at either end. “Body” objects provide a generic six-degree-of-freedom rigid-body representation based on a lumped-parameter model of translational and rotational properties. Rod objects can be added to Body objects and mooring lines can be attached at any location, allowing a wide variety of submerged structures to be integrated into the mooring system. Lastly, a means of dynamically simulating mooring line failures has been implemented. These new features, currently in the C++ version of MoorDyn, are described and then demonstrated on a two-turbine shared-mooring array. A qualitative view of the results suggests the new features are functioning as expected.

49 EE - Wind and Water Power Program - Wind (EE-4W↗

Tritiated Water Disposition Strategies

Tritium, a radioactive isotope of hydrogen, is typically confined within one or more confinement barriers. Tritium readily reacts with oxygen to form tritiated water or can exchange with protium or deuterium atoms in water to create tritiated water which is around 10,000 times more toxic than elemental tritium. Primary tritium confinement systems can create small volumes of highly tritiated water. Secondary confinement systems, like inert gloveboxes, will produce larger volumes of lower concentration tritiated water. Much larger volumes of tritiated water of very low tritium content are produced by tritium permeation/diffusion. Tritium from inside primary and secondary confinement barriers will diffuse from these systems into cooling water systems. Closed loop, recirculating cooling water systems will slowly accumulate tritium and create concentrations which exceed drinking water (or permitted discharge) concentrations. The cooling water systems designed as series of interconnected cooling loops that can slowly transfer tritium from one cooling loop to another cooling loop through heat exchangers. These large volumes of tritiated water may be above federal discharge limits and also lack convenient disposition pathways. This document will discuss tritiated water management strategies from the perspective of large volume, low tritium concentrations that exceed drinking water or permitted discharge limits. Applicable US regulatory requirements for a Molybdenum-99 (Mo-99) production facility concerning tritiated water are also cited. Tritiated water disposition options, based on typical waste minimization strategies will guide dispositions options. A Case Study example is presented to illustrate the pros and cons of several disposition options. The report does not recommend any particular disposition pathway but discusses strategies to be explored by a producer to guide selection of processes which best fits the needs of a particular waste stream.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

The Critical Need for Innovations in Instruments and Techniques to Enable In-Situ Cold Ion and Cold Electron Measurements within the Magnetosphere

Relatively few studies have focused on the cold plasma populations of the magnetosphere due to difficulties associated with obtaining measurements. The cold particle populations are defined here as those with total energy approximately <100 eV which is an energy range for which measurements are difficult (regardless of species), but which often make up the bulk of the plasma density. These populations have known and suspected impacts on the structure and dynamics of the magnetosphere. Unfortunately, the lack of accurate measurements of the cold ion and cold electron populations through the magnetosphere makes closure of these science questions extremely difficult if not impossible. In a complex system of interconnected parts like the Earth’s magnetosphere, a lack of understanding of one part of the system (the cold electron and ion populations) is disabling. Hence, until cold electron and cold ion measurements are obtained robustly with every mission, the magnetospheric plasma environment cannot be completely understood. This will require innovations in plasma spectrometers and associated techniques required to obtain high-fidelity in-situ measurements of the cold ion and cold electron populations in the magnetosphere. This paper seeks to review the instruments and techniques that have been used to date and present possible options for future missions.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Understanding the Capabilities and Limitations of the Controls and Operations of High Voltage Direct Current (HVdc) Converters in Interconnected Electric Power Systems

This project will seek to build a fundamental understanding, and capability, to model and simulate the controls and operations of high voltage direct current (HVdc) converter stations in an electromagnetic simulation environment. While HVdc stations have been operated in the United States for over 50 years, these are typically simple two terminal point-to-point systems. Recently multi-terminal systems have begun to be deployed. The challenge with these new multi-terminal systems is that they often use different control schemes on the different terminals. The interactions of existing and new controls, and the fact that HVdc systems are rates in the 1,000’s of MWs means that small control instabilities can have dramatic impacts to bulk power systems. Despite these challenges, the operational capabilities of HVdc make them an attractive option for the transfer of the large amounts of renewable electricity that will be necessary for decarbonization of the nation’s electrical infrastructure and other sectors.

42 ENGINEERING↗

Measuring Mobility Potential: A Novel Metric to Quantify Mobility Energy Productivity (MEP) of Transportation Systems

For nearly a century, the automobile has been the primary mode of personal transportation in American life. This remains true today as millions of people rely heavily on cars to connect suburbs with cities or to travel long distances - often out of routine or convenience. However, advances in technology are fueling an era of transportation transformation, with the potential to transform a system that has remained virtually unchanged for decades. The challenges of interconnecting our cities and creating a cross-continental transportation system for military purposes spawned the interstate highway system, generating the age of the automobile. In this century, congestion and mobility challenges of rising urban populations are spawning ever-evolving mobility and communications technologies to connect people to goods, services, and employment within a metropolitan and national context - all of which define a high quality of life. Aspiring smart cities are wrestling with questions such as: How does mobility impact a person's quality of life? Would people make different travel choices if they were presented with better information about their mobility options? Would businesses make different location decisions if they could assess the quality of mobility in that area? The ability to quantify the quality of mobility at a given location is the first step toward answering these questions. In response, an interdisciplinary team at the National Renewable Energy Laboratory (NREL) has developed the Mobility-Energy Productivity (MEP) metric. The MEP metric provides an avenue to not only measure the quality of mobility at a specific location in its current configuration, but also to test how various technological advances (e.g., connected and automated vehicles, plug-in electric vehicles, shared mobility) and infrastructure investments (e.g., building an additional highway lane, constructing a new shopping mall, implementing a transit-oriented development) impact the mobility of that location over time.

30 DIRECT ENERGY CONVERSION↗

Energy–Performance Trade-offs in Privacy-Preserving Federated Learning on SmartNIC-Enabled HPC Systems

Federated learning (FL) is increasingly deployed on accelerator-rich high-performance computing (HPC) systems, yet the system-level energy cost of privacy-aware FL remains poorly understood, particularly across heterogeneous networking and server-placement options. We present a measurement-driven study of energy–performance trade-offs for FL on GH200-class nodes across three deployment configurations: CPU-Ethernet, CPU-InfiniBand (RDMA-capable), and a DPU-hosted FL server over InfiniBand using a BlueField-3 SmartNIC/DPU. Using NVIDIA FLARE (NVFLARE), we align node-level power telemetry with per-round timing extracted from NVFLARE logs to quantify time-to-solution (TTS), energy-to-solution (ETS), energy-delay product (EDP), and synchronization behavior for three transformer models (ALBERT, DistilBERT, BERT), trained with and without differential privacy (DP). We find that interconnect choice is the dominant driver of runtime and energy: host-managed InfiniBand consistently reduces communication overhead versus Ethernet, yielding lower TTS/ETS/EDP. In contrast, in our NVFLARE deployment, placing the FL server on the DPU does not consistently match CPU-InfiniBand performance and can be slower—especially for larger models—highlighting that server placement alone is not sufficient to guarantee end-to-end gains. Finally, under our fixed-round protocol, DP increases per-round cost and runtime variance; ETS increases largely in proportion to TTS because average node power remains relatively stable across configurations.

Kotevska, Olivera [ORNL] (ORCID:0000000316772243)↗

The Impacts on California of Expanded Regional Cooperation to Operate the Western Grid (Final Report)

Policy makers in California and other western states are exploring options for greater regional cooperation in managing the western power grid to better achieve public policy goals. This report, requested by the California State Assembly and commissioned by the California Independent System Operator (CAISO), reviews and summarizes recent proposals, studies, and papers addressing various modes of regional cooperation.

24 POWER TRANSMISSION AND DISTRIBUTION↗

West Coast Offshore Wind Transmission Study

As a primary federal investigation of the utility of transmission to support offshore wind emergence in the Western Interconnect, the West Coast Offshore Wind Transmission study evaluates five distinct West Coast-wide transmission scenarios and trades the economic benefits and costs of six development pathways from 2035 to 2050. In this time, 33 GW of OSW generation and supporting transmission are explored within the context of 350 additional GW of generation to support 100% clean electricity. Geospatial analysis informed plausible transmission topologies and cost estimates. All pathways to intraregional or interregional coordination transmission builds showed more than $\$$14 billion in present value cost savings, with a maximum of $\$$25 billion of savings in the interregional case. These savings are primarily driven by fuel costs and increased availability of power flows enabled by new transmission networks. Key considerations of clean, dispatchable generation options, geographic disaggregation of costs, system reliability reinforcements, resilience, and community values are also presented and discussed.

17 WIND ENERGY↗

Tandem Photovoltaics Core Program Final Technical Report

The Tandem Photovoltaics Core Program was a multi-year initiative aimed at advancing hybrid tandem solar cell technologies to enhance solar module efficiency beyond the limits of single junction devices. This project focused on the development, testing, and scaling of prototype photovoltaic devices, with the goal of achieving commercial relevance and driving industry adoption. The work was divided into three tasks: 1) Comparative Analysis of Tandem Technologies: This task focused on quantifying energy yield under real-world conditions and assessing economic viability of tandems relative to silicon-based modules. The project's modeling framework incorporated performance data, cost of materials, and manufacturing process impacts to optimize tandem designs 2) Tandem Integration and Prototyping: In this task, we developed innovative tandem designs by combining metal halide perovskite (MHP) top cells and silicon (Si) bottom cells. The project focuses on both mechanical integration and direct deposition techniques to enable compatibility with commercially relevant Si technologies, such as passivated contact or PERC cells. 3) Scale-up and Reliability: This task addressed the challenges of large-area fabrication by developing scalable deposition methods and robust interconnection schemes for tandems. The project looked at different accelerated testing such as thermal cycling, damp heat exposure, and potential induced degradation, to ensure long-term stability of devices in field conditions. Tandem solar cells can greatly increase module efficiency beyond conventional single junction (SJ) devices, which are approaching their theoretical limit. There are many ways to fabricate a tandem cell or module in terms of materials used, configuration, and terminal connection. This SETO core project focused critical factors in enabling tandems to enter the market, including hardware integration, technoeconomic analysis (TEA), and energy yield analysis. We focused on MHP/Si hybrid tandem solar cells and modules as a model system for their versatility in module design comparisons, providing valuable insights for other tandem options. While champion cells with areas <1cm2 are regularly demonstrated by groups around the world, it is significantly more challenging to translate these advances into modules, and fewer institutions and companies are working at the module level. This project addressed questions about module fabrication, testing, and reliability that are hard to answer without actually fabricating prototypes. We also performed analysis and road-mapping activities to understand the potential for a wider variety of tandems, including all-perovskite tandems fabricated in collaboration with the Perovskite PV core program. Detailed technical results from this project are described for each task in Section 7.

14 SOLAR ENERGY↗

Building the Efficiency Workforce: Preprint

Demand for high-performance homes and buildings is growing due to their marketability, environmental benefits, and increasing affordability. Growth for energy efficient technologies and building upgrades has driven expansion across many traditional industries including construction trades (which added almost 21,000 jobs) and professional services (which added 35,000 employees) in 2018, according to the 2019 U.S. Energy Employment Report (USEER). As buildings become more automated, digitized and interconnected, the workforce that supports the design, build, operations and maintenance of these buildings must evolve. The U.S. Department of Energy’s Building Technologies Office (BTO) has engaged NREL to provide a framework upon which BTO can develop their next generation workforce development strategy. The strategy will be designed to increase quantity of workers with the skills to adequately design, install, maintain and operate high- performance buildings across all building sectors (residential, commercial, industrial). This paper will summarize past BTO workforce development efforts, identify current skills gaps and hiring challenges, and present anticipated future workforce needs based on new technologies currently being developed by labs and industry. Through a literature review and a series of industry and academic workshops, NREL will present a framework that includes multiple options to best prepare the next generation workforce.

buildings↗

Modifications to Solar Titan-130 Combustion Systems for Efficient, High Turndown Operation

The project team of Southwest Research Institute® (SwRI®), Solar Turbines Incorporated (Solar), the Electric Power Research Institute (EPRI), the University of California, Irvine (UCI), and the Georgia Institute of Technology (Georgia Tech) investigated methods to allow higher efficiency part-load operation of a Solar Titan 130 gas turbine. The objective was to develop a low-emission combustion system capable of sustaining combustion and avoiding lean blowout during high turndown operation, which would allow the gas turbine to operate as efficiently as possible at part load. Currently, electric utility markets are beginning to experience substantial increases in renewable energy generation. Some of these renewable energy sources have highly variable output in an uncontrolled manner. In order to maintain grid stability, there is a need for power plants to ramp up power to the grid rapidly to make up for drops in renewable generation. This is often termed spinning reserve, but the size of this reserve may need to increase as renewable penetration into the electric utility market increases. Small combined heat and power (CHP) power plants provide a promising option for meeting this spinning reserve requirement. In order to operate in spinning reserve while still meeting the heat requirements for the CHP, the gas turbine needs to operate efficiently at very low loads. Efficient, high turndown operations in this engine are limited by the lean flammability limit of the premixed combustion system. This project sought enhance the lean operability range of the Titan 130 combustor. First, the project team participated in a brainstorming activity and ultimately selected two concepts to explore: fuel augmentation with hydrogen (H2) to improve the stability at lean operating conditions and modifications to the fuel nozzle to improve the emissions performance at lean operating conditions. Analytical and laboratory investigations were accomplished by UCI to investigate the efficacy of H2 addition at improving lean blow out (LBO) limits and the resulting emissions. These investigations used a variety of chemical reactor network (CRN) and CFD models, validated against laboratory data, to model the impact of H 2 and inform the experimental efforts accomplished by SwRI and Solar. Ultimately, both the CRN and CFD models yielded generally good agreement with the experimental data below a particular temperature threshold. Atmospheric tests of a full-scale T130 annular combustor were performed at SwRI facilities in San Antonio, Texas, to investigate the use of H 2 addition. For these tests, the T130 combustion system remained largely unchanged; minor modifications were performed to the fuel ducting to allow for the safe use of H 2 . The test ultimately demonstrated that the addition of H 2 to the fuel mixture significantly increased the AFR ratio at which the combustor could operate. This improvement to the LBO limit should allow for less use of compressor bleed and less throttling needed by the inlet guide vanes (IGV). This in turn could result in more efficient operation of the gas turbine at lower load points. The second modification explored in this work was a direct modification to the T130 injector. The project team hypothesized that modifications to the pilot of the T130 injector could provide lower emissions at high turn-down operations. These modifications were manufactured and explored by the team at Solar. High pressure rig tests, originally slated to occur at SwRI, were ultimately accomplished by Solar to maintain overall project budget and mitigate cost growth attributable to supply chain issues and inflation. The pressurized rig tests ultimately showed that the SwRI Project No. 18.24153 - DE-EE0008415 Page 2 Final Technical Report January 24, 2024 modifications did not significantly alter the performance of the combustion system at the high turn-down conditions; both the modified injectors and the baseline configuration exhibited elevated emissions comparted to the full-load operating condition. A final set of studies performed by EPRI investigated the benefit-cost of flexible CHP as well as a grid interconnection study for the California Independent System Operator (CAISO) grid. These studies considered: traditional CHP with no spinning reserve available for on-demand grid support, 50% flexible CHP where 50% of the machine’s capacity is consumed by on-site baseload operations while providing an additional 50% capacity for on-demand grid support, and 70% flexible CHP where 70% of capacity is consumed on-site by baseload operations and 30% is available for on-demand grid support. In all cases, the analyses showed a benefit-to-cost ratio greater than unity implying a positive net present value for all configurations. However, the traditional CHP showed the most economic benefit. These results are sensitive to several factors, many of which are not fully known and may vary over time. Thus site owners must be convinced that taking up the increased costs and risks from flexible CHP would be worth implementing. As the grid in California and across the country transition to incorporate larger renewable energy generation, flexible CHP can provide much needed operating reserves and dispatchability. Alternative fuel options, such as hydrogen blending and biofuels, may also lower carbon intensities of CHP. Flexible CHP should be examined in the evolving market to understand innovative business models, changes market rules and services, and new technologies.

20 FOSSIL-FUELED POWER PLANTS↗

Multisectoral analysis of drought impacts and management responses to the 2008–2015 record drought in the Colorado Basin, Texas

Abstract. Drought has long posed an existential threat to society. Engineering and technological advancements have enabled the development of complex, interconnected water supply systems that buffer societies from the impacts of drought, enabling growth and prosperity. However, increasing water demand from population growth and economic development, combined with more extreme and prolonged droughts due to climate change, poses significant challenges for governments in the 21st century. Improved understanding of the cascading multisectoral impacts and adaptive responses resulting from extreme drought can aid in adaptive planning and highlight key processes in modeling drought impacts. The record drought spanning 2008 to 2015 in the Colorado Basin in the state of Texas, United States, serves as an outstanding illustration to assess multisectoral impacts and responses to severe, multi-year drought. The basin faces similar water security challenges to those across the western US, such as groundwater depletion and sustainability, resource competition between agriculture and growing urban populations, limited options for additional reservoir expansion, and the heightened risk of more severe and frequent droughts due to climate change. By analyzing rich, high-quality data sourced from nine different local, state, and federal sources, we demonstrate that characterizing regional multisector dynamics is crucial to predicting and understanding future vulnerability and possible approaches to reduce impacts to human and natural systems in the face of extreme drought conditions. This review reveals that, despite the severe hydrometeorological conditions of the drought, the region's advanced economy and existing water infrastructure effectively mitigated economic and societal impacts.

54 ENVIRONMENTAL SCIENCES↗

A Versatile Simulated Data Transport Layer for in Situ Workflows Performance Evaluation

In situ processing does not only allow scientific applications to face the explosion in data volume and velocity but also to address the time constraints of many simulation-analysis workflows by providing scientists with early insights about their applications at runtime. Multiple frameworks implement the concept of a data transport layer (DTL) to enable such in situ workflows. These tools are very versatile, directly or indirectly access the data generated on the same node, another node of the same compute cluster, or a completely distinct node, and allow data publishers and subscribers to run on the same computing resources or not. This versatility puts on researchers the onus of taking key decisions related to resource allocation and how to transport data to ensure the most efficient execution of their in situ workflows. However, domain scientists and workflow practitioners lack the appropriate tools to assess the respective performance of particular design and deployment options. In this paper we introduce a versatile simulated DTL designed to provide researchers with insights on the respective performance of different execution scenarios of in situ workflows. This open-source, standalone library builds on the SimGrid toolkit and can be linked to any SimGrid-based simulator. It facilitates the evaluation of the performance behavior, at scale, of different data transport configurations and the study of the effects of resource allocation strategies. We demonstrate the scalability, versatility, and accuracy of this simulated DTL by reproducing the execution of two synthetic benchmarks and of a real-world in situ workflow composed of an MPI application and a parallel data analysis. Results of simulations run on a single core show that the proposed library can simulate the interactions of tens of thousands of simulated processes deployed on two interconnected commodity clusters in a few seconds, and the execution by a thousand simulated processes of an in situ workflow in less than three minutes.

Suter, Fred [ORNL] (ORCID:0000000319021955)↗