Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “STREAMS”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6

Stream Chemistry, Synoptic Surveys, East Fork Poplar Creek Watershed, TN, USA; April 2023 to February 2025

Impacts of developed land cover on stream chemistry can be difficult to discern from natural variability, particularly in carbonate watersheds where weathering of urban infrastructure and lithology generate similar signatures. We evaluated how spatial patterns of stream chemistry varied across perennial and non-perennial tributaries spanning an urban-to-forested gradient in a mid-order, carbonate-dominated watershed. This data package contains a processed and compiled summary of stream chemistry and properties obtained from 12 synoptic surveys of 54 stream sites across the East Fork Poplar Creek watershed located near Oak Ridge, TN, United States. The sites include non-perennial tributaries, perennial tributaries, and the main stem and span forested to urban (highly developed) land cover gradients. The data package includes the processed and flagged chemical data (WaDE_SynopticSummary_FinalChemistry), metadata describing data flagging and analysis (WaDE_SynopticSummary_Metadata), information about each site and its contributing subcatchment (WaDE_SynopticSummary_SiteInformation), and a comparison of instrument and field detection limits used to determine method detection limits for the study (WaDE_SynopticSummary_DetectionLimitComparison). Stream chemistry includes stream parameters measured in situ using multiparameter probes (dissolved oxygen, pH, specific conductance, temperature) and solutes including nutrients (nitrate, ammonium, soluble reactive phosphorus), dissolved organic carbon, dissolved inorganic carbon, major cations (calcium, magnesium, potassium, sodium), major anions (chloride, sulfate), and a broad suite of minor and trace elements.

EARTH SCIENCE > TERRESTRIAL HYDROSPHERE > SURFACE ↗

Analysis of DWPF Sludge Batch 6 (Macrobatch 7): Pour Stream Glass Samples

The Defense Waste Processing Facility (DWPF) began processing Sludge Batch 6 (SB6), also referred to as Macrobatch 7 (MB7), in June 2010. SB6 is a blend of the heel of Tank 40 from Sludge Batch 5 (SB5), H-Canyon Np transfers and SB6 that was transferred to Tank 40 from Tank 51.1 SB6 was processed using Frit 418. Sludge is received into the DWPF Chemical Processing Cell (CPC) and is processed through the Sludge Receipt and Adjustment Tank (SRAT) and Slurry Mix Evaporator Tank (SME). The treated sludge slurry is then transferred to the Melter Feed Tank (MFT) and fed to the melter. During processing of each sludge batch, the DWPF is required to take at least one glass sample to meet the objectives of the Glass Product Control Program (GPCP) and to complete the necessary Production Records so that the final glass product may be disposed of at a Federal Repository. The DWPF requested various analyses of radioactive glass samples obtained from the melter pour stream during processing of SB6 as well as reduction/oxidation (REDOX) analysis of MFT samples to determine the impact of Argon bubbling. Sample analysis followed the Task Technical and Quality Assurance Plan (TTQAP) and an Analytical Study Plan (ASP). Four Pour Stream (PS) glass samples and two MFT slurry samples were delivered to the Savannah River National Laboratory (SRNL) from the DWPF. Table 1-1 lists the sample information for each pour stream glass sample. SB6 PS3 (S03472) was selected as the official pour stream sample for SB6 and full analysis was requested. This report details the visual observations of the as-received SB6 PS No.3 glass sample as well as results for the chemical composition, Product Consistency Test (PCT), radionuclide content, noble metals, and glass density. REDOX results will be provided for all four pour stream samples and vitrified samples of MFT-558 and MFT-568A. Where appropriate, data from other pour stream samples will be provided.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

SAMPA Based Streaming Readout Data Acquisition Prototype

We have assembled a small-scale streaming data acquisition system based on the SAMPA front-end ASIC. The 32-channel SAMPA chip was designed for the high-luminosity upgrade of the ALICE Time Projection Chamber (TPC) detector at the CERN Large Hadron Collider. The goals of the prototype system are to determine if the SAMPA chip is appropriate for use in detector systems at Jefferson Lab, and to gain experience with the hardware and software required to deploy streaming data acquisition systems in nuclear physics experiments. The 800 channel system is composed of components used in the ALICE TPC data acquisition upgrade. Five front-end cards (FEC) support five SAMPA chips each. SAMPA data streams on an FEC are concentrated into two high-speed (4.48 Gb/s) serial data streams by a pair Gigabit Transceiver ASICs (GBTx). These ten streams (44.8 Gb/s) are transmitted from the FECs over fibers to a PCIe based readout unit. The FPGA engine on the readout unit compresses data for transmission to a server via 100 Gb ethernet. Components on the FECs are radiation tolerant. High data rates can be handled. The system is by design scalable and thus provides a functional prototype for high-rate streaming readout at Jefferson Lab and the future Electron Ion Collider. We have made fundamental measurements (noise, linearity, time resolution) on the SAMPA ASIC. We have also coupled the readout system to a small Gas Electron Multiplier (GEM) detector and have studied its response to cosmic rays. A beam test of the system is planned.

ABBOTT, David↗

Probability of Forming Gaps in the GD-1 Stream by Close Encounters of Globular Clusters

One of the most intriguing properties of the GD-1 stellar stream is the existence of three gaps. If these gaps were formed by close encounters with dark matter subhalos, the GD-1 stream opens an exciting window through which we can see the size, mass, and velocity distributions of the dark matter subhalos in the Milky Way. However, in order to use the GD-1 stream as a probe of the dark matter substructure, we need to disprove that these gaps are not due to the perturbations from baryonic components of the Milky Way. Here we ran a large number of test-particle simulations to investigate the probability that each of the known globular clusters (GCs) can form a GD-1-like gap, by using the kinematical data of the GD-1 stream and GCs from Gaia early data release 3 and by fully taking the observational uncertainty into account. We found that the probability that all of the three gaps were formed by GCs is as low as 1.7 × 10 -5 , and the expected number of gaps formed by GCs is only 0.057 in our fiducial model. Our result highly disfavors a scenario in which GCs form the gaps. Given that other baryonic perturbers (e.g., giant molecular clouds) are even less likely to form a gap in the retrograde-moving GD-1 stream, we conclude that at least one of the gaps in the GD-1 stream was formed by dark matter subhalos if the gaps were formed by flyby perturbations.

79 ASTRONOMY AND ASTROPHYSICS↗

Phase-space Properties and Chemistry of the Sagittarius Stellar Stream Down to the Extremely Metal-poor ([Fe/H] ≲ -3) Regime

In this work, we study the phase-space and chemical properties of the Sagittarius (Sgr) stream, the tidal tails produced by the ongoing destruction of the Sgr dwarf spheroidal (dSph) galaxy, focusing on its very metal-poor (VMP; [Fe/H] < -2) content. We combine spectroscopic and astrometric information from SEGUE and Gaia EDR3, respectively, with data products from a new large-scale run of the StarHorse spectrophotometric code. Our selection criteria yield ~1600 stream members, including >200 VMP stars. We find the leading arm (b > 0°) of the Sgr stream to be more metal-poor, by ~0.2 dex, than the trailing one (b < 0°). With a subsample of turnoff and subgiant stars, we estimate this substructure's stellar population to be ~1 Gyr older than the thick disk's. With the aid of an N-body model of the Sgr system, we verify that simulated particles stripped earlier (>2 Gyr ago) have present-day phase-space properties similar to lower metallicity stream stars. Conversely, those stripped more recently (<2 Gyr) are preferentially akin to metal-rich ([Fe/H] > -1) members of the stream. Such correlation between kinematics and chemistry can be explained by the existence of a dynamically hotter, less centrally concentrated, and more metal-poor population in Sgr dSph prior to its disruption, implying that this galaxy was able to develop a metallicity gradient before its accretion. Finally, we identified several carbon-enhanced metal-poor ([C/Fe] > +0.7 and [Fe/H] ≤ -1.5) stars in the Sgr stream, which might be in tension with current observations of its remaining core where such objects are not found.

79 ASTRONOMY AND ASTROPHYSICS↗

Stellar Population Properties in the Stellar Streams around SPRC047

Abstract We have investigated the properties (e.g., age, metallicity) of the stellar populations of a ringlike tidal stellar stream (or streams) around the edge-on galaxy SPRC047 (z= 0.031) using spectral energy distribution (SED) fits to integrated broadband aperture flux densities. We used visual images in six different bands and Spitzer/IRAC 3.6μm data. We have attempted to derive best-fit stellar population parameters (metallicity, age) in three noncontiguous segments of the stream. Due to the very low surface brightness of the stream, we have performed a deconvolution with a Richardson–Lucy–type algorithm of the low spatial resolution 3.6μm IRAC image, thereby reducing the effect of the point-spread function aliasedemissionfrom the bright edge-on central galaxy at the locations of our three stream segments. Our SED fits that used several different star formation (SF) history priors, from an exponentially decaying SF burst to continuous SF, indicate that the age–metallicity–dust degeneracy is not resolved, most likely because of inadequate wavelength coverage and low signal-to-noise ratios of the low surface brightness features. We also discuss how future deep visual–near-infrared observations, combined with absolute flux calibration uncertainties at or below the 1% level, complemented by equally well absolute flux-calibrated observations in ultraviolet and mid-infrared bands, would improve the accuracy of broadband SED fitting results for low surface brightness targets, such as stellar streams around nearby galaxies that are not resolved into stars.

Astronomy & Astrophysics↗

Method for removing a foulant from a gas stream without external refrigeration

A process for removing a foulant from a gas stream is disclosed. The gas stream, containing a foulant, is cooled across a first heat exchanger and a second heat exchanger, producing a solid foulant entrained in cryogenic liquid as a foulant slurry, and a foulant-depleted gas stream. The foulant-depleted gas stream is passed through a cryogenic turbine and a first separation vessel, producing a light gas stream and further solid foulant. The solid foulants are recovered by a combination of pressurization, melting, and distillation to produce a liquid foulant product. Heat is recovered from the various streams in the various heat exchangers and the melter.

Baxter, Larry↗

STREAM: A Scalable Federated HPC Telemetry Platform

Obtaining and analyzing high performance computing (HPC) telemetry in real time is a complex task that can impact algo- rithmic performance, operating costs, and ultimately scientific outcomes. If your organization operates multiple HPC systems, filesystems, and clusters, telemetry streams can be synthesized in order to ease operational and analytics burden. In order to collect this telemetry, the Oak Ridge Leadership Computing Facility (OLCF) has deployed STREAM (Streaming Telemetry for Resource Events, Analytics, and Monitoring), which is a distributed and high-performance message bus based on Apache Kafka. STREAM collects center-wide performance information and must interface with many sources, including five HPE deployed supercomputers, each with their own Kafka cluster which is managed by HPCM. OLCF Supercomputers and their attached scratch filesystems currently send more than 300 million messages to over 200 topics producing around 1.3 Terabytes per day of telemetry data to STREAM. This paper describes the architectural principles that enable STREAM to be both resilient and highly performant while supporting multiple upstream Kafka clusters and other data sources. It also discusses the design challenges and decisions faced in adapting our existing system- monitoring infrastructure to support the first Exascale computing platform.

Adamson, Ryan↗

Copper and bismuth-based sorbent characterization in simulated iodine off-gas streams

The effective capture of volatile radioiodine, a fission product present in used nuclear fuel (UNF), is of paramount importance for development of used fuel reprocessing schemes to prevent release of radioiodine during unit operations and to meet regulatory standards for air emissions. A well-studied method for iodine capture in off-gas streams is the use of silver-functionalized zeolite phases (AgZ), which exploit chemisorption of I to Ag. Advances into other Ag-functionalized materials, including aerogels and metal organic frameworks (MOFs), are underway [1]. Additional metals with the capability to chemisorb iodine, including Cu, Bi, and Sn, [1] are being evaluated as alternatives to Ag for potential applicability to iodine management in off-gas systems. The design of novel functionalized sorbents with Cu and Bi, including composites with metal particles embedded in PAN substrates [2] and composites with metal sulfides embedded in PAN [3], is an ongoing area of study for improved iodine capture. Previously reported work on novel PAN-based metal sorbents has provided the synthesis, characterization, and iodine capture efficiency of this new class of sorbent. Specifically, the metal sulfide PAN composites are found to be easy to produce and reproduce, as well as having a high iodine loading potential under static conditions [3]. Due to the favorable testing previously performed with metal sulfide PAN composites, further testing into the performance of these composites under gas streams containing I2(g) in combination with NO2(g) and H2O(g) is needed. Humid streams of NO2(g) may arise from dissolver off-gas streams, when used fuel is dissolved in HNO3(aq) [4]. NO2(g) has been found to reduce AgZ sorption capacity for I because of oxidation of Ag, the chemisorbing agent, to Ag2O [5]. It follows that performance evaluation of novel sorbents under highly oxidizing conditions such as NO2(g) streams is critical. Therefore, the objective of this study is to determine the effect of flowing NO2(g) and H2O(g) streams on iodine sorption capacity and sorbent performance. This work utilizes custom-built gas handling capabilities for sorbent exposure along with solid-state characterization techniques to assess the physical and chemical properties of sorbents before and after exposure.

copper, bismuth, iodine capture, iodine sorbent, p↗

Watersheds and stream networks viewed longitudinally: Example insights from novel spatial portrayals of watershed characteristics

Abstract Longitudinal depictions of watershed structure and characteristics, including topography, stream networks, wetlands, ground water levels, and land use, can provide watershed knowledge and understanding unavailable from standard plan view maps. Three case studies provide examples of knowledge gained by applying longitudinal views of stream networks, watershed hydrologic behavior, and land use distributions. Longitudinal views of mountain stream networks show extreme variability in the slope‐area relationships of low Strahler order streams, large discontinuities in drainage area (large parts of drainage area space are absent in networks), and large variations in network curvature. Longitudinal views of a groundwater‐dominated headwater watershed increase the inference available from limited groundwater observations and clearly reveal how groundwater connections affect the permanence of surface water features and the distribution of vadose zone storage in the landscape. Plotting land uses longitudinally illuminates and allows a quantitative analysis of how land uses are distributed relative to topographic position. Viewing watersheds and stream networks longitudinally can provide new insights into watershed forms and processes and motivate new questions and research.

60 APPLIED LIFE SCIENCES↗

Rapid organic carbon spiraling in a headwater stream linked with streamflow, biogeochemistry, and canopy phenology

Headwater streams are abundant worldwide and important to global biogeochemical cycles, serving as critical processors and transporters of C. C spiraling is a useful way to understand the retention and mineralization of organic C (OC) in streams. However, analyses of seasonal and interannual variability in OC spiraling are currently limited. In this study, we aimed to understand the temporal patterns and driving mechanisms of OC spiraling, which will inform our understanding of future OC changes under climate change. We used 7 y of daily data in a small headwater stream (Walker Branch, Tennessee, USA) to assess seasonal and interannual variability in OC spiraling length (S OC ) and mineralization velocity (v fOC ), as well as their potential related variables. On average, S OC in Walker Branch was ~10× shorter than in previously studied small streams, indicating strong connections between the water column and the benthic environment where OC mineralization mostly takes place. OC spiraling was faster during the more biologically active periods of spring and autumn compared with more elongated OC spiraling in summer and winter, when OC retention was lower and downstream transport was higher. Gross primary production (GPP) was most strongly related to S OC and v fOC . Photosynthetically active radiation (PAR) and NO 3 − were also positively and negatively related to v fOC , respectively. Trends toward earlier and longer canopy cover and reduced GPP and PAR may result in longer S OC and slower v fOC , reducing localized instream processing of OC and potentially shunting more OC downstream. However, long-term observations indicate reduced NO 3 − at Walker Branch, suggesting opposing effects to those of GPP and PAR, leading to faster v fOC and greater OC retention. Time-series analyses of OC spiraling in streams can enhance our understanding of current and future responses of OC processing and downstream transport to climate change, as well as implications for downstream OC dynamics.

biological activity↗

Walker Branch Watershed: Daily Stream Metabolism and Organic Carbon Spiraling Metrics in the West Fork of Walker Branch, Tennessee, USA, 2004-2010

This dataset contains daily metabolism estimates of gross primary production (GPP), ecosystem respiration (ER), and net ecosystem production (NEP), in addition to organic carbon spiraling length (SOC) and mineralization velocity (VfOC) estimates at the West Fork of Walker Branch, a small headwater stream, in the Walker Branch Watershed, Tennessee, USA. Observations were made from 2004-2010 (2004-01-01 to 2010-12-31). These data were generated to assess seasonal and interannual variability in metabolism and organic carbon spiraling and to explore potential driver variables, as analyses of intra- and interannual variability in metabolism and organic carbon spiraling are currently limited, leaving knowledge gaps in the driving mechanisms of and future changes to stream metabolism and carbon processing under climate change. Additionally, measurements of discharge (Q), stream width, stream- and canopy-level photosynthetically active radiation (PAR), water temperature, and precipitation from this time frame are included. This dataset contains one data file in comma separated (*.csv) format.

54 ENVIRONMENTAL SCIENCES↗

Riparian vegetation shade restoration and loss effects on recent and future stream temperatures

River temperatures are expected to increase this century harming species requiring cold-water habitat unless restoration activities protect or improve habitat availability. Local shading by riparian vegetation can cool water temperatures, but uncertainty exists over the scaling of this local effect to larger spatial extents. Here, we evaluate this issue using a regional spatial stream network temperature model with covariates representing shade effects to predict mean August stream temperatures across 78,195 km of tributaries flowing into the Columbia River in the northwestern United States. We evaluate nine scenarios predicting stream temperatures for three riparian shade conditions (current, restored, and no riparian vegetation) within three different climate periods (2000s, 2040s, and 2080s). Results suggest riparian shade restoration (2000s climate) could decrease mean August stream temperatures by 0.62°C across the study network. Under the same restored shade conditions, temperature predictions for tributaries at their confluence with the Columbia River range from 0.02 to 2.08°C cooler than under current shade conditions. The climate warming effect predicted for the 2040s and 2080s, however, is greater than the cooling effect from restoring riparian shade. Streams less than 10-m bankfull width cooled more frequently with riparian shade restoration. In Oregon, the proportion of fish habitat for salmon and trout rearing and migration that meet temperature numeric water quality criteria could be increased by 20% under restored shade conditions although net habitat declines may still occur in the future. We conclude riparian vegetation restoration could partially mitigate future warming and help maintain cold-water habitats that function as thermal refuges if implemented strategically.

54 ENVIRONMENTAL SCIENCES↗

Toward a persistent event-streaming system for high-performance computing applications

High-performance computing (HPC) applications have traditionally relied on parallel file systems and file transfer services to manage data movement and storage. Alternative approaches have been proposed that use direct communications between application components, trading persistence and fault tolerance for speed. Event-driven architectures, as popularized in enterprise contexts, present a compelling middle ground, avoiding the performance cost and API constraints of parallel file systems while retaining persistence and offering impedance matching between application components. However, adapting streaming frameworks to HPC workloads requires addressing challenges unique to HPC systems. This paper investigates the potential for a streaming framework designed for HPC infrastructures and use cases. We introduce Mofka, a persistent event-streaming framework designed specifically for HPC environments. Mofka combines the capabilities of a traditional streaming service with optimizations tailored to the HPC context, such as support for massively multicore nodes, efficient scaling for large producer-consumer workflows, RDMA-enabled high-performance network communications, specialized network fabrics with multiple links per node, and efficient handling of large scientific data payloads. Built using the Mochi suite of HPC data service components, Mofka provides a lightweight, modular, and high-performance solution for persistent streaming in HPC systems. We present the architecture of Mofka and evaluate its performance against Kafka and Redpanda using benchmarks on diverse platforms, including Argonne's Polaris and Oak Ridge's Frontier supercomputers, showing up to 8× improvement in throughput in some scenarios. We then demonstrate its utility in several real-world applications: a tomographic reconstruction pipeline, a workflow for the discovery of metal-organic frameworks for carbon capture, and the instrumentation of Dask workflows for provenance tracking and performance analysis.

HPC↗

The understudied winter: Evidence of how precipitation differences affect stream metabolism in a headwater

Climate change is causing pronounced shifts during winter in the US, including shortening the snow season, reducing snowpack, and altering the timing and volume of snowmelt-related runoff. These changes in winter precipitation patterns affect in-stream freeze-thaw cycles, including ice and snow cover, and can trigger direct and indirect effects on in-stream physical, chemical, and biological processes in ~60% of river basins in the Northern Hemisphere. We used high-resolution, multi-parameter data collected in a headwater stream and its local environment (climate and soil) to determine interannual variability in physical, chemical, and biological signals in a montane stream during the winter of an El Niño and a La Niña year. We observed ~77% greater snow accumulation during the El Niño year, which caused the formation of an ice dam that shifted the system from a primarily lotic to a lentic environment. Water chemistry and stream metabolism parameters varied widely between years. They featured anoxic conditions lasting over a month, with no observable gross primary production (GPP) occurring under the ice and snow cover in the El Niño year. In contrast, dissolved oxygen and GPP remained relatively high during the winter months of the La Niña year. These redox and metabolic changes driven by changes in winter precipitation have significant implications for water chemistry and biological functioning beyond the winter. Our study suggests that as snow accumulation and hydrologic conditions shift during the winter due to climate change, hot-spots and hot-moments for biogeochemical processing may be reduced, with implications for the downstream movement of nutrients and transported materials.

54 ENVIRONMENTAL SCIENCES↗

StOKeDMD: Streaming Occupation kernel dynamic mode decomposition

Dynamic mode decomposition (DMD) has become a common technique for constructing surrogate models for dynamical systems from observed system states. The Occupation Kernel DMD (OKDMD) method proposed in (Rosenfeld et al., 2022) and (Rosenfeld et al., 2024) is a Liouville operator based method that builds surrogate models from system state trajectories. Here, this paper proposes an extension of OKDMD to the case when the system states are observed in a streaming fashion, i.e., only a small fraction of the state trajectory is available at a given time. The developed method, Streaming Occupation Kernel DMD (StOKeDMD), accommodates the streaming data input by leveraging properties of specific choices of kernel functions and occupation kernels. We apply the StoKeDMD method as a compression method for streaming data, analyze the memory complexity, and demonstrate the performance of StoKeDMD in the compression of streaming data generated from a Lorenz system and a fluid flow simulation.

97 MATHEMATICS AND COMPUTING↗

Ion Exchange Processes for CO 2 Mineralization Using Industrial Waste Streams: Pilot Plant Demonstration and Life Cycle Assessment

Abstract An attractive technique for removing CO 2 from the environment is sequestration within stable carbonate solids (e. g., calcite). However, continuous addition of alkalinity is required to achieve favorable conditions for carbonate precipitation (pH>8) from aqueous streams containing dissolved CO 2 (pH<4.5) and Ca 2+ ions. In this study, a pH‐swing process using ion exchange was demonstrated to process 300 L of produced water brine per day for CO 2 mineralization. Proton titration capacities were quantified for aqueous streams in equilibrium with gas streams at various concentrations of CO 2 (pCO 2 =0.03–0.20 atm) and at various flow rates (0.5–2.0 L min −1 ). Energy intensities for the process were determined to be between 30 and 65 kWh per tonne of CO 2 sequestered depending on the composition of the brine stream. A life cycle assessment was performed to analyze the net carbon emissions of the technology which indicated a net CO 2 reduction for pCO 2 ≥0.12 atm (−0.06–−0.39 kg CO 2 e per kg precipitated CaCO 3 ) utilizing calcium‐rich brines. The results from this study indicate the ion exchange process can be used as a scalable method to provide alkalinity necessary for the capture and storage of CO 2 in Ca‐rich waste streams.

Chemistry↗

Transitioning from File-Based HPC Workflows to Streaming Data Pipelines with openPMD and ADIOS2

This paper aims to create a transition path from file-based IO to streaming-based workflows for scientific applications in an HPC environment. By using the openPMP-api, traditional workflows limited by filesystem bottlenecks can be overcome and flexibly extended for in situ analysis. The openPMD-api is a library for the description of scientific data according to the Open Standard for Particle-Mesh Data (openPMD). Its approach towards recent challenges posed by hardware heterogeneity lies in the decoupling of data description in domain sciences, such as plasma physics simulations, from concrete implementations in hardware and IO. The streaming backend is provided by the ADIOS2 framework, developed at Oak Ridge National Laboratory. This paper surveys two openPMD-based loosely-coupled setups to demonstrate flexible applicability and to evaluate performance. In loose coupling, as opposed to tight coupling, two (or more) applications are executed separately, e.g. in individual MPI contexts, yet cooperate by exchanging data. This way, a streaming-based workflow allows for standalone codes instead of tightly-coupled plugins, using a unified streaming-aware API and leveraging high-speed communication infrastructure available in modern compute clusters for massive data exchange. We determine new challenges in resource allocation and in the need of strategies for a flexible data distribution, demonstrating their influence on efficiency and scaling on the Summit compute system. The presented setups show the potential for a more flexible use of compute resources brought by streaming IO as well as the ability to increase throughput by avoiding filesystem bottlenecks.

Poeschel, Franz↗