SEARCH · Engineering Papers

Results for “Scale down”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 559 records · Page 31

All sources NASA NTRS DOE OSTI

DOE OSTI2023

Updating and Evaluating Anthropogenic Emissions for NOAA’s Global Ensemble Forecast Systems for Aerosols (GEFS-Aerosols): Application of an SO 2 Bias-Scaling Method

We updated the anthropogenic emissions inventory in NOAA’s operational Global Ensemble Forecast for Aerosols (GEFS-Aerosols) to improve the model’s prediction of aerosol optical depth (AOD). We used a methodology to quickly update the pivotal global anthropogenic sulfur dioxide (SO 2 ) emissions using a speciated AOD bias-scaling method. The AOD bias-scaling method is based on the latest model predictions compared to NASA’s Modern-Era Retrospective analysis for Research and Applications, version 2 (MERRA2). The model bias was subsequently applied to the CEDS 2019 SO 2 emissions for adjustment. The monthly mean GEFS-Aerosols AOD predictions were evaluated against a suite of satellite observations (e.g., MISR, VIIRS, and MODIS), ground-based AERONET observations, and the International Cooperative for Aerosol Prediction (ICAP) ensemble results. The results show that transitioning from CEDS 2014 to CEDS 2019 emissions data led to a significant improvement in the operational GEFS-Aerosols model performance, and applying the bias-scaled SO 2 emissions could further improve global AOD distributions. The biases of the simulated AODs against the observed AODs varied with observation type and seasons by a factor of 3~13 and 2~10, respectively. The global AOD distributions showed that the differences in the simulations against ICAP, MISR, VIIRS, and MODIS were the largest in March–May (MAM) and the smallest in December–February (DJF). When evaluating against the ground-truth AERONET data, the bias-scaling methods improved the global seasonal correlation (r), Index of Agreement (IOA), and mean biases, except for the MAM season, when the negative regional biases were exacerbated compared to the positive regional biases. The effect of bias-scaling had the most beneficial impact on model performance in the regions dominated by anthropogenic emissions, such as East Asia. However, it showed less improvement in other areas impacted by the greater relative transport of natural emissions sources, such as India. The accuracies of the reference observation or assimilation data for the adjusted inputs and the model physics for outputs, and the selection of regions with less seasonal emissions of natural aerosols determine the success of the bias-scaling methods. A companion study on emission scaling of anthropogenic absorbing aerosols needs further improved aerosol prediction.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2024

How the Galaxy–Halo Connection Depends on Large-scale Environment

We investigate the connection between galaxies, dark matter halos, and their large-scale environments at z = 0 with Illustris TNG300 hydrodynamic simulation data. We predict stellar masses from subhalo properties to test two types of machine learning (ML) models: explainable boosting machines (EBMs) with simple galaxy environment features and E(3)-invariant graph neural networks (GNNs). The best-performing EBM models leverage spherically averaged overdensity features on 3 Mpc scales. Interpretations via SHapley Additive exPlanations also suggest that in the context of the TNG300 galaxy–halo connection, simple spherical overdensity on ∼3 Mpc scales is more important than cosmic web distance features measured using the DisPerSE algorithm. Meanwhile, a GNN with connectivity defined by a fixed linking length, L, outperforms the EBM models by a significant margin. As we increase the linking length scale, GNNs learn important environmental contributions up to the largest scales we probe (L = 10 Mpc). We conclude that 3 Mpc distance scales are most critical for describing the TNG galaxy–halo connection using the spherical overdensity parameterization, but that information on larger scales, which is not captured by simple environmental parameters or cosmic web features, can further augment these models. Our study highlights the benefits of using interpretable ML algorithms to explain models of astrophysical phenomena, and the power of using GNNs to flexibly learn complex relationships directly from data while imposing constraints from physical symmetries.

79 ASTRONOMY AND ASTROPHYSICS↗

NASA NTRS2013

State of the Art in Large-Scale Soil Moisture Monitoring

Soil moisture is an essential climate variable influencing land atmosphere interactions, an essential hydrologic variable impacting rainfall runoff processes, an essential ecological variable regulating net ecosystem exchange, and an essential agricultural variable constraining food security. Large-scale soil moisture monitoring has advanced in recent years creating opportunities to transform scientific understanding of soil moisture and related processes. These advances are being driven by researchers from a broad range of disciplines, but this complicates collaboration and communication. For some applications, the science required to utilize large-scale soil moisture data is poorly developed. In this review, we describe the state of the art in large-scale soil moisture monitoring and identify some critical needs for research to optimize the use of increasingly available soil moisture data. We review representative examples of 1) emerging in situ and proximal sensing techniques, 2) dedicated soil moisture remote sensing missions, 3) soil moisture monitoring networks, and 4) applications of large-scale soil moisture measurements. Significant near-term progress seems possible in the use of large-scale soil moisture data for drought monitoring. Assimilation of soil moisture data for meteorological or hydrologic forecasting also shows promise, but significant challenges related to model structures and model errors remain. Little progress has been made yet in the use of large-scale soil moisture observations within the context of ecological or agricultural modeling. Opportunities abound to advance the science and practice of large-scale soil moisture monitoring for the sake of improved Earth system monitoring, modeling, and forecasting.

Review↗

DOE OSTI2021

Validation of Scale-Derived Ages in Wild Juvenile and Adult Steelhead Using Parental-Based Tagging

Abstract Accurate age information is a critical component in fisheries monitoring, management, and research. The ability to assign accurate ages to fish by using nonlethal structures is vital in calculating cohort productivity for fish species with low abundances and variable life histories. A common nonlethal method to assign ages is by interpretation of patterns in scales. Validation of ages is not easily obtained for free-ranging fishes; however, the development of genetic “tags” has expanded age validation opportunities. In this study, we used parental-based genetic tagging to validate ages determined from scales collected from juvenile and adult steelhead Oncorhynchus mykiss in two streams within the Snake River basin. Juvenile scale ages were in agreement with 93% of the known ages. Adult scale ages were in agreement with 89% of the known ages. Although overall bias in juvenile ages was very low, we saw a slight positive bias in younger fish (young of the year and age 1) and a small negative bias in older juveniles (ages 2–4). A small negative bias in ages of adults was a consequence of errors in freshwater age. The errors observed did not significantly bias age compositions because the 90% confidence intervals about the age proportions based on scales contained the known proportions; therefore, scale analysis is an acceptable method for assigning ages to Snake River steelhead. We discuss the value of validated known-age scales as a reference collection, and as an illustrative use, we constructed a circulus count guide to aid technicians in identifying missing annuli and distinguishing true from false annuli. To demonstrate the use of the circulus guidelines, we reanalyzed samples with age discrepancies, applying the circulus count data and the known age to identify mistakes.

Reinhardt, Leslie↗

DOE OSTI2024

Advanced Energy Scale Correction Techniques for the X-ray Transition Edge Sensors of the Athena mission

The X-ray Integral Field Unit (X-IFU) onboard the future European X-ray telescope Athena will be the first space instrument carrying an array of more than a thousand transition edge sensors. One of the key challenges of the X-IFU is the measurement of narrow X-ray atomic lines to determine velocity shifts at an unprecedented level of accuracy. For this reason, the energy scale of the instrument needs to be known with extreme accuracy, of 0.4 eV (1σ) up to 7 keV. The energy scale will be measured on the ground through a dedicated calibration campaign using fiducial X-ray sources. Though calibrated, the energy scale is extremely sensitive to the environmental conditions around the TES array, and drifts in the readout chain electronics. Uncorrected, the energy scale can naturally drift up to hundreds of eVs. Changes of the TES gain will be monitored via onboard X-ray calibration sources, and the energy scale will be corrected either per pixel, or within a small groups of pixels. Although simulations show that a 0.4 eV level can be achieved, the very high accuracy required by the X-IFU calls for experimental validation. A dedicated measurement campaign has been performed by NASA Goddard Space Flight Center to characterize the energy scale of a prototype kilo-pixel array of X-IFU-representative TESs. The analysis of the data demonstrated the ability to correct for various drifts using two fiducial lines to track the temporal gain variation. In this paper, we propose to extend this study on the same data set by investigating multi-parameter correction techniques based on both the pulse-height of the fiducial line and the prepulse baseline level, using the knowledge of the TES energy scale at reference temperature/magnetic field set points acquired on the ground. Investigations on the co-adding of pixels to perform a joint correction over pools of pixels is also explored.

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2025

Universal scaling of electron transmission for nearly ballistic and quantum dragon nanodevices

Here, we predict two different universal scaling regimes for the quantum transmission of metallic nanodevices following the addition of a small amount of uncorrelated disorder. A nanodevice is connected to two thin semi-infinite uniform leads, and the Non-Equilibrium Green’s Function (NEGF) methodology yields the electron transmission $\mathscr{T}(E)$ as a function of the injected electron energy $E$. Ballistic nanodevices have no disorder and have $\mathscr{T}(E)=1$for all $E$ that allow electron propagation in the leads. Quantum dragon nanodevices can have extremely strong properly correlated disorder, and still have $\mathscr{T}(E)=1$for all $E$. Additional uncorrelated site disorder leads to Fano resonances in $\mathscr{T}(E)$. Averaging over the uncorrelated disorder we predict using perturbation theory two universal scaling regimes for $\mathscr{T}_{ave}(E)$. The functional form of both universal scaling regimes depend on the device length and width, energy, and variance of the uncorrelated disorder. The second scaling regime, valid for small but somewhat larger uncorrelated disorder than the first scaling regime, also has the form dependent on the density of states of the system. These two scaling regimes are demonstrated to be valid via large scale computer calculations.

Ballistic transport↗

DOE OSTI2026

Interfacial Void Formation and Self-Healing in Oxide Scales on Al-containing High-Entropy Alloy

The exceptional high-temperature oxidation resistance of Al-containing high-entropy alloys (HEAs) is often attributed to the formation of a protective α-Al 2 O 3 scale. However, the dynamic, atomic-scale mechanisms governing the stability of this scale—including interfacial void formation and the often-postulated but rarely visualized “self-healing” capacity—remain poorly understood. Herein, we reveal the complex evolution of the triple-layer oxide scale on an Al 10 CoCrFeNi HEA through combined electron microscopy and diffraction study. We show that interfacial voids are an inherent consequence of the scaling process, originating from two distinct mechanisms: the Kirkendall effect at the interface between the γ-Al 2 O 3 /α-Al 2 O 3 and alloy driven by cationic diffusion imbalance and volumetric contraction due to phase transformations at the spinel/Cr 2 O 3 interface. Crucially, we provide microstructural evidence consistent with an intrinsic self-healing response. This process is driven by coupled inward diffusion of oxygen and outward diffusion of metal cations, leading to the in-situ formation of transient θ-Al 2 O 3 and spinel phases that partially fill and seal the voids. Here, these results provide atomic-scale insights into the phase evolution, defect formation, and self-repair of oxide scales in HEAs—highlighting pathways to enhance their oxidation resistance in extreme environments.

High-entropy alloy↗

DOE OSTI2021

Morphological and chemical characteristics of oxide scales formed on δ-phase plutonium metal alloys II: 2.0 at% Ga

The focused ion-beam scanning electron microscopy (FIB-SEM) and three-dimensional (3D) microscopy were applied to characterize the subsurface morphological features of oxide scales formed on an ~2.0 at. % Ga plutonium (Pu) metal alloy. Using the FIB-SEM technique, a number of morphological features formed in the interior of the oxide scale from Pu metal’s environmental exposure were observed and identified. Auger electron spectroscopy (AES) was utilized to characterize the cross-sectional composition and chemistry of the oxide scale. The oxide scale formed during inert storage and operational environments was found to be highly variable in thickness and morphology, presenting some regions with a thin (<400 nm), dense oxide layer and others with a thick (>2 µm) scale with substantial lateral cracking. After subsequent exposure to dry air environment, the oxide scale became thicker (~4 µm) and slightly more porous. The changes following aging in a moist air environment were observed to be more severe, with the formation of a highly porous internal structure containing significant lateral and transverse cracking. In comparison to the scale formed on an ~3.5 at. % Ga-Pu metal alloy, the oxide morphology of the lower gallium alloy investigated here exhibited greater variation in thickness and a noteworthy dependence on the presence of water vapor, particularly in terms of the internal porosity formed during growth of the oxide.

36 MATERIALS SCIENCE↗

DOE OSTI2023

Effect of the integral length scales of turbulent inflows on wind turbine loads

As wind turbines become larger, the fluctuations in the inflow become increasingly influential in the turbine structural loading. These fluctuations are characterized by the integral length scale, which represents the average size of the largest energy-containing turbulent eddies. Current design standards neglect the varying integral length scales that characterize inflows of wind turbines in operation. Using large-eddy simulations, we generate turbulent inflows of varying integral length scales and quantify the loads of the IEA 15-MW reference wind turbine. Results illustrate that the impact of turbulence on rotor and tower loads is up to 10 times greater than the impact of the mean shear profile. Increasing integral length scales from 0.3x to 0.5x the rotor diameter reduces blade root flapwise moments and rotor and tower loads. Increasing integral length scales from 0.5x to 1.7x the rotor diameter increases the rotor aerodynamic thrust force and the blade root flapwise shear loads and decreases the tower base torsional moment and the tilting and yawing rotor aerodynamic moments. Additionally, turbulence intensity has a greater impact on wind turbine loads than integral length scales. In conclusion, findings indicate that design standards should consider varying integral length scales for accurate wind turbine loading characterization in turbulent inflow conditions.

17 WIND ENERGY↗

DOE OSTI2020

Time Scale Dependence of the Meridional Coherence of the Atlantic Meridional Overturning Circulation

The Atlantic Meridional Overturning Circulation (AMOC) variability is suggested to be incoherent between the subpolar and subtropical gyres in the Atlantic on interannual and even decadal time scales, questioning the representativeness of AMOC variability at a single latitude in modern observation and paleoreconstruction. Paleoreconstructions of the Florida Current transport suggest that Florida Current variability is associated with the AMOC on the millennial time scale, but the Rapid Climate Change (RAPID) mooring array suggests a weak correlation between the Florida Current and the AMOC. In this study, we investigate the meridional coherence of AMOC variability and the relationship between the Florida Current variability and the AMOC variability on different time scales in a transient 20,000-year simulation. We find that with the increase of time scales, the meridional coherence of the AMOC increases. On decadal and longer time scales, the coherent subtropical and subpolar AMOC is caused by the coherent buoyancy forcing in the subpolar gyre. Also, the Florida Current transport is highly correlated with AMOC variability on decadal and longer time scales, suggesting that observations of the Florida Current can be used to indicate AMOC variability on long time scales.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2020

Relationships Between Precipitation Properties and Large-Scale Conditions During Subsidence at the Eastern North Atlantic Observatory

Three years of reanalysis and ground-based observations collected at the Eastern North Atlantic (ENA) observatory are analyzed to document the properties of rain and boundary layer clouds, and their relationship with the large-scale environment during general subsidence conditions and following cold front passages. Clouds in the wake of cold fronts exhibit on average a 10% higher propensity to precipitate and higher rain-to-cloud fraction than cloud found in general subsidence conditions. Similarities in the seasonal cycle of rain and of large-scale properties suggest that the large-scale conditions created by the cold front passage are responsible for the unique properties of the rain forming in its wake. The identification of monotonic relationships between rain-to-cloud fraction and rain rate with surface forcing and boundary layer stability parameters as well as between virga base height with stability and humidity measures further supports that large-scale conditions impact precipitation variability. Likewise, these relationships between the large-scale and rain properties are less clear than those established between cloud and rain properties suggesting that cloud macrophysics have a more direct impact on the properties of rain than the large-scale environment. The applicability of previously documented relationships between rain properties and cloud thickness is tested and the relationships adjusted to accommodate the complex shallow clouds and melting precipitation observed to occur in the ENA region. Furthermore, establishing these relationships opens up opportunities for parametrization development and suggest that a realistic representation of precipitation properties in models relies on the accurate representation of both clouds and the large-scale environment.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2023

Multipoint Observations of Dispersive Scale Alfvénic Field‐Line Resonances Associated With Substorm Auroral Beads

We present a case study of the field-aligned current (FAC) systems that transpire within the high-altitude auroral acceleration region of an “auroral bead” initiated double oval substorm observed on 23 February 2001 by the Cluster fleet. Conjunctive Cluster measurements and auroral images from IMAGE reveal that auroral bead current system formation and evolution is a multi-scale, injection-mediated process. The FACs at large scales vary on substorm evolution time scales (~minutes) in response to the injection and evolution of hotter denser magnetospheric plasma. Embedded within the large-scale FACs are intense short-scale (≲ few 10s of km) currents comprising dispersive scale Alfvén wave (DAW) fluctuations. The DAWs are a complex mixture of ingoing and reflected components that regularly interfere to form a broad spectrum of kinetic (dispersive) scale Alfvénic field-line resonances (KFLRs). The Alfvénic currents appear as a nested series of upward and downward FAC densities with amplitudes reaching a few 100 nA/m 2 . Energized field-aligned or counterstreaming electrons near keV energies and below are observed with parallel skews that vary in concert with variations in the DAW current sense. Positive correlations between DAW electric field energy densities and the energies of energized H + , He + , and O + outflow are observed, indicative of ion energization within the DAW fields. Due to their L-shell location (L~5.8–7.0) and associations with injections, the KFLRs are interpreted as the high-altitude auroral zone analog of KFLRs observed in the equatorial inner magnetosphere.

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2024

A Scale‐Dependent Analysis of the Barotropic Vorticity Budget in a Global Ocean Simulation

Abstract The climatological mean barotropic vorticity budget is analyzed to investigate the relative importance of surface wind stress, topography, planetary vorticity advection, and nonlinear advection in dynamical balances in a global ocean simulation. In addition to a pronounced regional variability in vorticity balances, the relative magnitudes of vorticity budget terms strongly depend on the length‐scale of interest. To carry out a length‐scale dependent vorticity analysis in different ocean basins, vorticity budget terms are spatially coarse‐grained. At length‐scales greater than 1,000 km, the dynamics closely follow the Topographic‐Sverdrup balance in which bottom pressure torque, surface wind stress curl and planetary vorticity advection terms are in balance. In contrast, when including all length‐scales resolved by the model, bottom pressure torque and nonlinear advection terms dominate the vorticity budget (Topographic‐Nonlinear balance), which suggests a prominent role of oceanic eddies, which are of km in size, and the associated bottom pressure anomalies in local vorticity balances at length‐scales smaller than 1,000 km. Overall, there is a transition from the Topographic‐Nonlinear regime at scales smaller than 1,000 km to the Topographic‐Sverdrup regime at length‐scales greater than 1,000 km. These dynamical balances hold across all ocean basins; however, interpretations of the dominant vorticity balances depend on the level of spatial filtering or the effective model resolution. On the other hand, the contribution of bottom and lateral friction terms in the barotropic vorticity budget remains small and is significant only near sea‐land boundaries, where bottom stress and horizontal viscous friction generally peak.

Khatri, Hemant↗

DOE OSTI2024

A Machine Learning Bias Correction on Large–Scale Environment of High–Impact Weather Systems in E3SM Atmosphere Model

Large–scale dynamical and thermodynamical processes are common environmental drivers of high–impact weather systems causing extreme weather events. However, such large–scale environmental conditions often display systematic biases in climate simulations, posing challenges to evaluating high–impact weather systems and extreme weather events. In this paper, a machine learning (ML) approach was employed to bias correct the large–scale wind, temperature, and humidity simulated by the atmospheric component of the Energy Exascale Earth System Model (E3SM) at ~1° resolution. The usefulness of the ML approach for extreme weather analysis was demonstrated with a focus on three high–impact weather systems, including tropical cyclones (TCs), extratropical cyclones (ETCs), and atmospheric rivers (ARs). We show that the ML model can effectively reduce climate bias in large–scale wind, temperature, and humidity while preserving their responses to imposed climate change perturbations. The bias correction is found to directly improve water vapor transport associated with ARs, and representations of thermodynamical flows associated with ETCs. When the bias–corrected large–scale winds are used to drive a synthetic TC track forecast model over the Atlantic basin, the resulting TC track density agrees better with that of the TC track model driven by observed winds. In addition, the ML model insignificantly interferes with the mean climate change signals of large–scale storm environments as well as the occurrence and intensity of three weather systems. This study suggests that the proposed ML approach can be used to improve the downscaling of extreme weather events by providing more realistic large–scale storm environments simulated by low–resolution climate models.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2023

Exploring the parameter space of MagLIF implosions using similarity scaling. I. Theoretical framework

Magneto-inertial fusion concepts, such as the magnetized liner inertial fusion (MagLIF) platform, constitute an alternative path for achieving ignition and significant fusion yields in the laboratory. The space of experimental input parameters defining a MagLIF load is highly multi-dimensional, and the implosion itself is a complex event involving many physical processes. In the first paper of this series, we develop a simplified analytical model that identifies the main physical processes at play during a MagLIF implosion. Using non-dimensional analysis, we determine the most important dimensionless parameters characterizing MagLIF implosions and provide estimates of such parameters using typical fielded or experimentally observed quantities for MagLIF. Here, we then show that MagLIF loads can be “incompletely” similarity scaled, meaning that the experimental input parameters of MagLIF can be varied such that many (but not all) of the dimensionless quantities are conserved. Based on similarity-scaling arguments, we can explore the parameter space of MagLIF loads and estimate the performance of the scaled loads. Then, in the follow-up papers of this series, we test the similarity-scaling theory for MagLIF loads against simulations for two different scaling “vectors,” which include current scaling and rise-time scaling.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

DOE OSTI2020

Nonlinear modeling of the scaling law for the m/n = 3/2 error field penetration threshold

The scaling law for the m/n=3/2 error field (EF) penetration threshold is predicted numerically based on nonlinear single-fluid and two-fluid modeling using the TM1 code. The simulated penetration threshold of radial magnetic field b r at the plasma edge is scaled to the electron density n e , temperature T e , viscous time τ μ , toroidal field B t and the natural frequency ω in the form of b r /B t ∝n e αn T e αT τ μ αμ B t αB ω αω by scanning these parameters separately. Here, α n , α T , α μ , α B and α ω are the scaling coefficients on n e , T e , τ μ , B t and ω, respectively. Single-fluid modeling shows that the 3/2 EF threshold scales as b r /B t ∝n e 0.56 T e 0.6 τ μ -0.59 B t -1.15 ω, which is similar with the analytical scaling law in both the Rutherford and visco-resistive regimes. Yet, two-fluid modeling shows that the scaling law differs significantly in particular regarding the dependence on plasma rotation. In detail, the scaling coefficient α n on density decreases from 0.67 to 0.56 and α T on temperature decreases from 0.67 to 0.32, while α μ on viscous time is around -0.45 and α B on toroidal field decreases slightly from -1.15 to -1, when the ratio |ω E /ω *e | between plasma rotation frequency ω E and diamagnetic drift frequency ω *e varies from 0 to 10. Scans of the plasma rotation reveals that the penetration threshold linearly depends on the perpendicular rotation frequency (or natural frequency) ω ⊥e =ω E +ω *e , and there is a minimum in the required field amplitude when ω ⊥e 0. In addition, the enduring mystery of non-zero penetration threshold at zero plasma natural frequency in EF experiments is resolved by two-fluid simulations. We report that the very small island and smooth bifurcation in EF penetration near zero frequency is hard to detect in the experiment, leading to a finite penetration threshold within the capability of the experimental measurements.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

DOE OSTI2025

Electron-Only Magnetic Reconnection and Inverse Magnetic-Energy Transfer at Subion Scales

We derive, and validate numerically, an analytical model for electron-only magnetic reconnection applicable to strongly magnetized plasmas. Our model predicts subion-scale reconnection rates significantly higher than those pertaining to large-scale reconnection, aligning with recent observations and simulations. Here, we apply this reconnection model to the problem of inverse magnetic energy transfer at subion scales. We derive time-dependent scaling laws for the magnetic energy decay and the typical magnetic structure dimensions that differ from those previously found in the magnetohydrodynamics regime. These scaling laws are validated via two- and three-dimensional simulations, demonstrating that subion-scale magnetic fields can reach large, system-size scales via successive coalescence.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

DOE OSTI2022

Running Ensemble Workflows at Extreme Scale: Lessons Learned and Path Forward

The ever-increasing volumes of scientific data combined with sophisticated techniques for extracting information from them have led to the increasing popularity of ensemble workflows which are a collection of runs of individual workflows. A traditional approach followed by scientists to run ensembles is to rely on simple scripts to execute different runs and manage resources. This approach is not scalable and is error-prone, thereby motivating the development of workflow management systems that specialize in executing ensembles on HPC clusters. However, when the size of both the ensemble and the target system reach extreme scales, existing workflow management systems face new challenges that hamper their efficient execution. In this paper, we describe our experience scaling an ensemble workflow from the computational biology domain from the early design stages to the execution at extreme scale on Summit, a leadership class supercomputer at the Oak Ridge National Laboratory. We discuss challenges that arise when scaling ensembles to several million runs on thousands of HPC nodes. We identify challenges with composition of the ensemble itself, its execution at large scale, post-processing of the generated data, and scalability of the file system. Based on the experience acquired, we develop a generic vision of the capabilities and abstractions to add to existing workflow management systems to enable the execution of ensemble workflows at extreme scales. We believe that the understanding of these fundamental challenges will help application teams along with workflow system developers with designing the next generation of infrastructure for composing and executing extreme-scale ensemble workflows.

Mehta, Kshitij↗