Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “network resilience”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 325 records · Page 18
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

Scalable, Physical Effects Measurable Microgrid for Cyber Resilience Analysis (SPEMMCRA)

The ability to advance state of the art automated protections for industrial control systems (ICS) has as a precursor in the ability to understand the tradeoff space. That is, to enable a cyber feedback loop in a control system environment you must first consider both the security mitigation available, the benefits and the impacts to the control system functionality when the mitigation is used. More damaging impacts could be precipitated that the mitigation was intended to rectify. This paper details networked ICS that controls a simulation of the frequency response represented with the swing equation. The microgrid loads and base generation can be balanced through the control of an emulated battery and power inverter. The simulated plant, which is implemented in Raspberry Pi computers, provides an inexpensive platform to realize the physical effects of cyber attacks to show the tradeoffs of available mitigatoins. This network design can include a commercial ICS controller to introduce real world implementation of feedback controls, and provides a scalable, physical effects measurable Microgrid for cyber resilience analysis (SPEMMCRA).

42 ENGINEERING↗
DOE OSTI2021

Topology-Agnostic, Scalable, Self-Healing, and Cost-Aware Protection of Microgrids

Most microgrid protection schemes found in published literature suffer from a lack of generality in that they work well for the assumed topology, including type and placement of sources. Other generic protection schemes tend to be too complicated, too expensive, or both. To overcome these draw- backs, a topology-agnostic, scalable, and cost-aware protection based on fundamental principles that work in the presence of high penetration of inverter-based resources (IBRs) is developed and tested in this paper. Here, the protection system also implements stable automatic reconfiguration of the healthy sections of the system after clearance of fault, thus increasing resilience by self- healing. To achieve this ambitious goal, stable inverter models are developed that operate in unbalanced networks in grid-connected and islanded modes, even with 100% IBRs, share power without conflicting controls, and can ride through faults while limiting fault currents. The scheme is tested for primary and backup protection and reconfiguration on the IEEE 123-node feeder in grid-connected and islanded modes with 15 IBRs connected to the system.

42 ENGINEERING↗
DOE OSTI2026

Constitutive and inducible oleoresin defenses share genetic architectures and mechanisms in Pinus taeda

The oleoresin defense system of loblolly pine (Pinus taeda) protects trees from insects and pathogens and is an important source of renewable biofuels and chemicals, but the genetic basis of oleoresin production is poorly understood. We characterized the genetic architecture of oleoresin flow, resin canal number, stem wood terpene content, and monoterpene composition in two clonal populations of P. taeda. We used quantitative genetic analyses, genome-wide association studies (GWASs), multiplex network learning, and gene expression profiling to elucidate shared gene networks underlying defense traits and to identify high-quality candidates for breeding and engineering loblolly pine. Genetic analyses revealed polygenic inheritance and trait-to-trait correlations provide strong evidence for shared genes regulating constitutive and induced oleoresin flow. We identified 236 single nucleotide polymorphisms associated with oleoresin flow, resin canal number, and terpene composition and highlight candidate genes likely involved in terpene biosynthesis, cambial meristem reprogramming, and pathogen perception and immune signaling. Fourteen GWAS candidates were methyl jasmonate-responsive in tissues where resin canals initiate and terpene production occurs. Integrating quantitative genetics, GWAS, gene expression, and multiplex network analyses enabled the prioritization of high-quality candidate genes. This work advances the development of more resilient loblolly pine optimized for ecological performance, renewable chemical, and biofuel production.

genome-wide association study↗
DOE OSTI2022

ASPIRES: Airport Shuttle Planning and Improved Routing Event-driven Simulation

Most of the existing traffic simulation packages require significant calibration work to be able to reflect reality. To evaluate special operations including emerging technologies, a microscopic simulation that tracks detailed interactions of all the elements of the traffic systems is usually needed. This type of simulation is usually computationally demanding. This work developed an Airport Shuttle Planning and Improved Routing Event-driven Simulation (ASPIRES) package to simulate and evaluate current, potential, and future airport shuttle operations. Here, the simulation was driven by data and thus did not require much calibration effort. The discrete-event simulation nature of ASPIRES makes the simulation computationally efficient. Simulating 1 day of shuttle operations takes less than 2s. The study site of this work is the Dallas/Fort Worth International Airport in the U.S. The shuttle service that connects the five terminals of the airport and the rental car center was studied. Travel times, dwell times, and passenger arrivals were simulated using empirical distributions derived mainly from real data to capture the stochastic nature of the rental car center shuttle bus operations. Data on bus miles traveled, bus energy consumption, passenger wait times, and passengers left behind at stops were collected to study the trade-off between energy use and passenger experience. Electric bus and on-demand bus operations were also included. The simulation outputs can show passengers statistics at terminals, shuttles statistics, and charging station statistics. ASPIRES cannot be used to model a generic traffic system but is well-suited for fleet systems.

33 ADVANCED PROPULSION SYSTEMS↗
DOE OSTI2026

SuperLab 2.0 Showcase: Connecting Five Labs to Tackle Grid Complexity and Unlock Unique Grid Asset Potential

SuperLab 2.0 (5-Lab Demo) is a collaborative, national-scale experiment showcasing the coordination of geographically distributed energy assets in real time. The demonstration integrates 25 physical and digital assets, spanning wind, PV, batteries, electrolyzers, DC fast chargers, microgrid controllers, building automation systems, small modular reactor (SMR), control centers, and gas turbines, across five DOE national laboratories-NLR, INL, NETL, LBNL, and SNL. These assets are unified using Energy Sciences Network (ESnet), a low-latency, high-performance U.S. Department of Energy's (DOE) network, and controlled via a centralized energy controller hosted at NLR's ARIES facility. The demonstration validates the ability to stress-test hybrid energy systems under dynamic scenarios to de-risk advanced control strategies for greater resilience and flexibility. SuperLab 2.0 (5-Lab Demo) showcased a major advancement in federated national laboratory collaboration, enabling real-time, cross-laboratory experimentation to coordinate geographically dispersed distributed energy resources (DERs) using various communication protocols and networks. SuperLab 2.0 (5-Lab Demo) built on previous demonstrations conducted between NLR-PNNL and NLR-INL connecting diverse assets including distant protection devices, a SMR simulator, and a high temperature electrolyzer (HTE). Previous demos were based on a single connection between two labs with minimal coordination challenges. The 5-Lab demo with a centralized controller, distributed testbeds across different geographical locations, and use of protocols-based communication represents a scenario closer to real-world grid operations that coordinate resources across a region to meet system needs. This experiment studied how local DER controllers interact with a centralized energy controller during normal and abnormal events to maintain reliability. The SuperLab team across the five labs implemented a notional power system model equivalent of transmission and distribution lines, represented by the data networks interconnecting the labs. Each lab continuously exchanged local parameters (such as P and Q) from its Hardware-In-Loop (CHIL) and Power Hardware-In-Loop (PHIL) assets through centralized energy controller at NLR, enabling real-time interaction and coordination across sites. By leveraging ESnet as the communication backbone, the team successfully operated the distributed assets as a unified power system, with each bus represented by a different laboratory. This setup mirrors how assets interact in real-world power systems across dispersed locations with various protocols and latencies. At each lab site, assets were operated using their own local controllers which were coordinated through an overarching operation and control layer of centralized energy controller, equivalent to how an energy management system (EMS) orchestrates assets across a regional or national grid. SuperLab's federated connectivity utilized a Digital Real-Time Simulators (DRTS)-type gateway to connect Controller Hardware-In-Loop (CHIL) and PHIL assets between labs. To enable this federated connection through ESnet, a deterministic network was established where latency variations were consistent. This consistency allowed the development of digital filters for the power system assets across CHIL and PHIL interfaces to avoid unstable and unreliable grid conditions. This report provides an overview of the cross-laboratory configuration and offers insights into interconnecting geographically distributed research assets to test them as if they were co-located. This experiment represents a step toward linking nine DOE national laboratories, enabling nation-wide simulations that can address utility-driven challenges with grid resilience, flexibility, and modernization.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2024

Application Guide for the Cyber-Resilient Design Framework for Hybrid Systems

As the energy landscape evolves, hybrid power plants—integrating multiple renewable energy sources (e.g., solar, wind, and battery storage) with the bulk power electric system—play a crucial role in meeting growing energy demands. However, with a hybrid power plant’s increased number of components, complex network connectivity and digitization of controls, these systems face growing cybersecurity risks. To help mitigate these risks, Idaho National Laboratory (INL) presents this application guide specifically designed for operators and systems engineers to evaluate the cybersecurity resilience of their hybrid power plant’s design.

14 SOLAR ENERGY↗
DOE OSTI2023

Establishment of an Intertribal Technical Energy Providers Network (Final Technical Report)

The Midwest Tribal Energy Resources Association (MTERA) embarked on a 5-year (extended to 7-year) initiative aimed at advancing Tribal energy resilience and sustainability through intertribal collaboration. By bringing together Tribes in Minnesota, Wisconsin, and Michigan (EPA Region 5 Tribes), the project's scope encompassed organizational growth, governance and management formalization and enhancement, and the planning and deployment of energy-related services to Member Tribes. MTERA strategically pursued four key objectives to fulfill its mission: 1. Clearinghouse Development: Creation of a centralized hub offering valuable information, education, and outreach for Tribal communities and their neighbors. 2. Policy Liaison: Serving as a vital link between federal and state agencies to contribute to the formulation of inclusive energy policies that ensure both Tribal and regional energy success. 3. Energy Planning and Research: Providing essential support for energy planning, researching financing opportunities, and exploring effective procurement strategies. 4. Training Support: Facilitating training and apprenticeship initiatives to empower the Tribal workforce and enhance community capabilities.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2023

Robust Decentralized Secondary Control Scheme for Inverter-based Power Networks

Inverter-dominated microgrids are quickly becoming a key building block of future power systems. They rely on centralized controllers that can provide reliability and resiliency in extreme events. Nonetheless, communication failures due to cyber-physical attacks or natural disasters can make autonomous operation of islanded microgrids challenging. This paper examines a unified decentralized secondary control scheme that is robust to inverter clock synchronization errors and can be seamlessly applied to grid-following or grid-forming control architectures. The proposed scheme overcomes the well-known stability problem that arises from parallel operation of local integral controllers. Theoretical guarantees for stability are provided along with criteria to appropriately tune the secondary control gains to achieve good frequency regulation performance while ensuring fair power sharing. The efficacy of our approach is demonstrated through simulations on a 5-bus microgrid with four grid-forming inverters.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2020

Power Grid Contingency Analysis with Machine Learning: A Brief Survey and Prospects

We briefly review previous applications of machine learning (ML) in power grid analyses and introduce our ongoing effort toward developing a generative-adversarial (GA) model for fast and reliable grid contingency analyses. According to our review, the persisting limitation of traditional ML techniques in grid analyses is the need for an exhaustive amount of training data for model generalization and accurate predictions. GA models overcome this limitation by first learning true data distribution from a small training set, from which new samples assimilating true data are generated with some variations. Subsequently, GA models can transfer learn or super-generalize with increased accuracy, that is, accurately predict n - (k + 2) contingencies from a small n - k training set and generated n - (k + 1) data. The joint effort between Idaho National Lab and Florida State University strives to develop a zero-shot and deep learning-based contingency analysis tool, named Smart Contingency Analysis Neural Network (SCANN), by leveraging the aforementioned advantages of GA models. The basic architecture of SCANN stems from the Latent Encoding of Atypical Perturbations network combined with an adversarial network, and it is designed to generate imbalanced power flow data from learned true data distributions for prediction purposes. Here we also introduce the abstract concept of resilience-chaos plots, a new resilience characterization tool proposed to complement SCANN by aiding in the assessment of large amounts of high-order contingency predictions.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2022

Energy Resilient Recovery in Puerto Rico: Hurricane Maria Recovery Support

This report covers microgrids as a special application of distributed energy resources (DERs) in Puerto Rico. This is one of a series of reports describing the U.S. Department of Energy (DOE) multi-laboratory efforts undertaken. To ensure the sustainable, long-term recovery of Puerto Rico's electric power grid from hurricanes Maria and Irma and to build capacity to manage future potential natural disasters in the most secure and resilient way, DOE convened experts from multiple national laboratories to develop a comprehensive set of data, models, analytic tools, and studies, considering inputs from a wide variety of stakeholder groups, to support technically sound recommendations for Puerto Rico's energy investment decisions. In Phase 1 of the multi-laboratory effort to support Puerto Rico's recovery, the National Renewable Energy Laboratory (NREL) provided the utility company Puerto Rico Electric Power Authority (PREPA) recommendations for a new framework of interconnection standards to accelerate the integration of utility-scale, transmission-connected, renewable electrical generation and energy storage that ensure cross-technology compatibility and enable high deployment levels without compromising grid reliability, safety, or security (https://www.nrel.gov/docs/fy19osti/73848.pdf). In Phase 2, NREL published a report focused on the interconnection of distributed energy resources (DERs) to the electric distribution system in Puerto Rico (https://www.nrel.gov/docs/fy21osti/77127.pdf). This report familiarized the reader with Puerto Rico's distribution infrastructure and operational practices and procedures that are relevant to DER interconnection. The report also provided considerations for streamlining the interconnection process given the expected increase in deployments resulting from Puerto Rico's renewable portfolio standard goal of 100% renewables by 2050. Accordingly, the report identifies considerations and concerns associated with the increase in intermittent generation, strategies for DER interconnection best practices, and the potential use of the latest technological solutions identified in the latest revision of the Institute of Electrical and Electronics Engineers 1547-2018 interconnection standard. Additionally, the report identified ways to improve the physical resiliency of installed DERs. This task describes how distributed energy systems may participate in and contribute to improving the resiliency of Puerto Rico's energy infrastructure in the future. This report presents an analysis of the Puerto Rico grid to illustrate the possible different microgrid and minigrid scenarios. The capability to form sustained microgrids and minigrids has the potential to provide a high level of resiliency during disturbances on the grid. DERs, on the other hand, have the capability to aid in the microgrid operation by providing grid support as well as grid-forming functionality. Other tasks in Phase 3 include voltage regulation in distribution networks using DERs and transmission-and-distribution co-simulation.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2025

Unified Universal Control and Coordination of Inverter-Based Resources, and Validation for a PV + Battery Hybrid Plant

As renewable energy deployment grows, hybrid power plants (HPPs) combining photovoltaic (PV) and battery systems must evolve to offer both energy and grid stability services. These systems typically include a mix of grid-following (GFL) and grid-forming (GFM) inverters, presenting unique coordination and control challenges. This Department of Energy–funded project developed and validated a Unified Universal Control and Coordination (UUCC) framework for such PV + battery hybrid plants, enabling seamless and stable operation, including ultrafast black start, autonomous synchronization, and robust frequency and voltage regulation, under different grid conditions. The project significantly advanced the understanding of inverter-based resource (IBR) control by developing and validating three complementary system-level approaches for hybrid GFL/GFM operation: 1. A combined Virtual Resistance (VR)-based GFL and Virtual Oscillator Control (VOC)-based GFM method, where each inverter type is governed by a specialized control strategy. Together, these achieve stable, fast-response coordination, eliminating inrush current and enabling smooth black start and grid synchronization across a wide range of grid strengths. 2. A Deadbeat-based UUCC strategy, which uses discrete-time, switching-cycle-level control for both GFL and GFM inverters. This approach replaces traditional PI/PLL control with a control parameter-free, high-bandwidth framework that supports stable LVRT and instantaneous synchronization under all conditions. 3. A benchmark comparison with Siemens’ commercial GFM microgrid controller, which provided a fast baseline platform. The commercial approach decoupled v & f control was implemented on a commercial microgrid controller.The baseline commercial benchmark helped highlight superior transient response and black start performance offered by the deadbeat and VOC approaches. These technical contributions offer substantial improvements over conventional inverter control schemes, which often rely on slow phase-locked loop (PLL)-based synchronization, require careful control parameters tuning, and prone to unstable in weak grids with GFL inverters and in stiff grid with GFM inverters therefore challenging for hybrid GFL+GFM under all grid conditions. The deadbeat-based UUCC framework enables simpler, faster, and more robust operation of hybrid IBR systems using wide-bandgap (WBG) devices such as SiC power semiconductors. The rapid expansion of hybrid distributed energy resources (DERs), including residential and commercial PV-BESS installations such as Tesla Powerwall, PV with vehicle-to-grid (V2G) capability, and other integrated configurations, presents complex operational challenges for medium-voltage radial distribution feeders. These networks are subject to frequent disturbances such as faults, switching operations, rapid reclosing sequences, and feeder reconfigurations, all of which introduce dynamic stress on IBRs. In addition, planned feeder segmentation and deliberate islanding for resilience will require DERs that can autonomously perform blackstart, establish voltage and frequency references, and resynchronize with the main grid. The advanced deadbeat-based UUCC control and blackstart functionalities developed in this project directly address these requirements, enabling decentralized and autonomous operation of inverter-dominated DERs in distribution systems under a wide range of fault and reconfiguration scenarios. From a public benefit perspective, these innovations enable more reliable and cost-effective integration of renewable energy into distribution networks. The ability to autonomously black start and stabilize grids under varying grid conditions support accelerates recovery from outages and support decentralized resilient energy systems. By reducing system complexity and improving performance, this project lays critical groundwork for future inverter-dominated power grids that are clean, reliable, and accessible to all.

14 SOLAR ENERGY↗
DOE OSTI2025

Enhancing Power Resilience for Remote Communities: A Comprehensive Renewable Energy Solution for Itbayat Island

The island of Itbayat, Philippines, faces significant challenges in maintaining a reliable and resilient power supply due to its current reliance on a vulnerable power distribution system managed by a local electric cooperative. The existing infrastructure, which includes diesel generators and a radial network configuration with some above-ground lines, is highly susceptible to frequent typhoons and adverse weather conditions. These factors, combined with inadequate staffing and high operational costs, result in frequent power outages that disrupt daily life and hinder economic development. This white paper proposes a comprehensive solution to enhance the resilience and reliability of Itbayat's power system by integrating renewable energy sources, specifically solar photovoltaic (PV) systems, battery storage, and a microgrid controller. The proposed solution aims to reduce dependency on diesel fuel, optimize energy use, and provide a sustainable and robust power supply for the island. Key components of the solution include: 1. Solar PV Installation: Deploying solar PV panels to harness abundant solar energy, reducing reliance on diesel fuel. 2. Battery Storage Systems: Installing battery storage to store excess solar energy and ensure a continuous power supply during low solar generation periods. 3. Microgrid Controller: Implementing a microgrid controller to manage and optimize the integration of solar PV, battery storage, and existing diesel generators. The proposed solution addresses several critical issues, including system vulnerability, generator dependency, and operational inefficiencies. By adopting this innovative approach, Itbayat Island can achieve a more resilient, efficient, and sustainable energy infrastructure, ensuring a stable power supply for its residents and enhancing overall energy security.

14 SOLAR ENERGY↗
DOE OSTI2024

Capturing Infrastructure Interdependencies for Power Outages Prediction During Extreme Events

As extreme weather events such as hurricanes, severe thunderstorms, and floods grow in frequency and intensity, the disruption of power grid systems poses significant challenges, including widespread electrical outages, economic losses, and threats to public safety. This paper presents a forward-looking approach that leverages geographical graph-based machine learning models to predict county-level maximum power outages during such events. By capturing the intricate interdependencies within power system networks, our approach aims to provide precise and actionable predictions that can optimize emergency response efforts and enhance grid resilience. Through the integration of real-world data, including hurricane advisories and power outage records, we have trained and benchmarked multiple machine learning models, demonstrating the feasibility and potential of this method. While our initial results are promising, this paper also charts a course for advancing these models, addressing the remaining challenges, and ultimately transforming how we anticipate and respond to the impacts of extreme weather on power systems.

Lee, Sangkeun (Matt) [ORNL] (ORCID:000000021317511↗
DOE OSTI2023

Autonomous Restoration of Networked Microgrids Using Communication-Free Smart Sensing and Protection Units

Here this paper presents autonomous restoration of networked microgrids using distributed energy resources (DERs), inverter-based resources (IBRs) in general, and controllable assets of intelligence without communication to achieve a extremely resilient power system restoration. The main innovation of this work lies in use of real-time smart sensing and decision making that enable autonomous recombination of islanded microgrids, which are individually black-started by local grid-forming (GFM) inverter units. The smart circuit breakers (SCBs) designed and demonstrated here play critical roles in the restoration process to achieve autonomy; the autonomous restoration scheme is not pre-engineered and may be subject to encounter overloading, live or downed segments as the boundary of the microgrids dynamically expands without prior knowledge and only with local measurement. We demonstrate autonomous microgrid restoration both in simulation of a modified IEEE 13-bus system, and in a hardware testbed comprising 24 GFM-IBRs and 6-SCBs, depicting several scenarios to evaluate their functionalities and interoperability. This work validates high potential of extremely resilient microgrid using novel approaches for inverter-based bottom-up system restoration in a communication-free paradigm.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2022

The nitrogen gap in soil health concepts and fertility measurements

Soil nitrogen (N) often limits productivity in agroecosystems, prompting fertilizer applications that increase crop yields but can degrade the environment. Nitrogen's dual role in both productivity and environmental quality should center it in soil health frameworks. We use recent evidence to argue that N availability is an emergent property of the integrated soil biogeochemical system and is strongly influenced by plant traits and their interactions with microbes and minerals. Building upon this, we theorize that the sources of plant and microbial N shift across soil health gradients, from inorganic N dependence in ecologically simple systems with poor soil health to a highly networked supply of organic N in healthy soils; ergo, investments in soil health should increase ecological complexity and the pathways by which plants can access N, leading to more resilient nutrient supplies and yields in a variable climate. However, current N assessment methods derive from a historical emphasis on inorganic N pool sizes and are unable to capture the shifting drivers of N availability across soil health gradients. We highlight the need to better understand the plant-microbial-mineral interactions that regulate bioavailable N as a first step to improving our ability to measure it. We conclude it will be necessary to harness agroecosystem complexity, account for plant and microbial drivers, and carefully integrate external N inputs into soils' internal N network to expand the routes by which N from organic pools can be made bioavailable. Finally, by emphasizing N in soil health concepts, we argue that researchers can accelerate advances in N use efficiency and resiliency.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2024

Evolving Multi-hazard Machine Learning Modeling for Advanced Risk-Informed Infrastructure Resilience Assessment

The socioeconomic impacts of pipeline incidents have escalated over the past three decades, revealing the limitation of traditional risk modeling methods when applied to extensive pipeline networks. This research aims to develop machine learning (ML) models that effectively identify, rank, and predict the diverse hazards and socioeconomic consequences associated with pipeline incidents. Utilizing historical data on pipeline incidents alongside weather and oceanographic data from the 1980s onward, the Houston metropolitan area serves as a testbed for the proposed methodologies. The research segments the combined datasets into three consecutive periods, demonstrating the efficacy of the updated model in predicting future events, particularly concerning precipitation rate data. Despite the challenges posed by a relatively limited dataset, local-level ML modeling offers valuable insights into the spatial and temporal dynamics of multiple hazards that contribute to pipeline incidents. These findings hold significant implications for future research, particularly in understanding and mitigating risks in various locations across the Gulf Coast and other coastal regions.

42 ENGINEERING↗
DOE OSTI2023

Multi-Agent Reinforcement Learning for Distribution System Critical Load Restoration

Grid resilience has become a critical topic recently because of the increasing occurrence of extreme events and the growing integration of intermittent renewable energy sources. To build a resilient distribution system, this paper develops a multiagent reinforcement learning-based (MARL) method to coordinate distribution energy resources (DERs) dispatch, load pickup, and network reconfiguration for load restoration after a system outage. With the help of two types of control agents, namely critical load restoration (CLR) and coordination (COR) agents, system loads can be restored efficiently, given available resources. The effectiveness and superiority of the proposed algorithm are demonstrated through simulations and comparative studies on a real distribution feeder in Western Colorado.

distribution system↗
DOE OSTI2026

JANUS: Resilient and Adaptive Data Transmission for Enabling Timely and Efficient Cross-Facility Scientific Workflows

In modern science, the growing complexity of large-scale scientific projects has led to an increasing reliance on cross-facility scientific workflows, where resources and expertise from multiple institutions and geographic locations are leveraged to accelerate scientific discovery. These workflows often require transmitting huge amounts of scientific data through wide-area networks. Although high-speed networks like ESnet and transfer services such as Globus have improved data mobility, several challenges remain. The sheer volume of data can overwhelm network bandwidth, widely used transport protocols such as TCP suffer from inefficiencies due to retransmissions triggered by packet loss, and existing fault-tolerance mechanisms like erasure coding introduce substantial overhead. In this paper, we propose Janus, a resilient and adaptable data transmission approach designed for cross-facility scientific workflows. Unlike traditional TCP-based methods, Janus leverages UDP, integrates erasure coding for fault tolerance, and combines it with error-bounded lossy compression to reduce overhead. This novel design allows users to balance data transmission time and accuracy, optimizing transfer performance based on specific scientific requirements. Additionally, Janus dynamically adjusts erasure coding parameters in response to real-time network conditions, ensuring efficient data transfers even in fluctuating environments. We develop optimization models for determining ideal configurations and implement adaptive data transfer protocols to enhance reliability. Through extensive simulations and real-network experiments, we demonstrate that Janus significantly improves transfer efficiency while maintaining data fidelity.

Esaulov, Vladislav [Georgia State University, Atla↗