Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Extreme scale”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 595 records · Page 33
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

Historical evaluation and future projections of compound heatwave and drought extremes over the conterminous United States in CMIP6 *

Abstract Independently, both droughts and heatwaves can induce severe impacts on human and natural systems. However, when these two climate extremes occur concurrently in a given region, their compound impacts are often more pronounced. With the improvement in both the spatiotemporal resolution and representation of complex climate processes in the global climate models (GCMs), they are increasingly used to study future changes in these extremes and associated regional impacts. However, GCM selection for such impact assessments is generally based on historical performance and/or future mean changes, without considering individual or compound extremes. In contrast, this study evaluates historical performance and projected changes in heatwaves, droughts, and compound heatwave-droughts using an ensemble of GCMs from the latest Phase 6 of Coupled Models Intercomparison Project at a regional scale across the conterminous United States. Additionally, we explore the inter-model differences in the projected changes that are associated with various characteristics of extremes and the choice of drought indices. Our analysis reveals considerable variation among the GCMs, as well as substantial differences in the projected changes based on the choice of drought indices and region of interest. For example, the projected increases in both the frequency and intensity of drought and associated compound extreme days, based on the standardized precipitation evapotranspiration index far exceed those derived from the standard precipitation index. Further, the largest changes in the frequency of compound extremes are projected over the Southwest, South Central, and parts of the Southeast while the smallest changes are projected over the Northeast. Overall, this study provides important insights for the interpretation and selection of GCMs for future assessment studies that are crucial for the development of regional adaptation strategies in the face of climate change.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

The detection and attribution of extreme reductions in vegetation growth across the global land surface

Abstract Negative extreme anomalies in vegetation growth (NEGs) usually indicate severely impaired ecosystem services. These NEGs can result from diverse natural and anthropogenic causes, especially climate extremes (CEs). However, the relationship between NEGs and many types of CEs remains largely unknown at regional and global scales. Here, with satellite‐derived vegetation index data and supporting tree‐ring chronologies, we identify periods of NEGs from 1981 to 2015 across the global land surface. We find 70% of these NEGs are attributable to five types of CEs and their combinations, with compound CEs generally more detrimental than individual ones. More importantly, we find that dominant CEs for NEGs vary by biome and region. Specifically, cold and/or wet extremes dominate NEGs in temperate mountains and high latitudes, whereas soil drought and related compound extremes are primarily responsible for NEGs in wet tropical, arid and semi‐arid regions. Key characteristics (e.g., the frequency, intensity and duration of CEs, and the vulnerability of vegetation) that determine the dominance of CEs are also region‐ and biome‐dependent. For example, in the wet tropics, dominant individual CEs have both higher intensity and longer duration than non‐dominant ones. However, in the dry tropics and some temperate regions, a longer CE duration is more important than higher intensity. Our work provides the first global accounting of the attribution of NEGs to diverse climatic extremes. Our analysis has important implications for developing climate‐specific disaster prevention and mitigation plans among different regions of the globe in a changing climate.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2024

Bilevel Nodal Behind-the-meter Solar Disaggregation Under Unexpected Extreme Weather Conditions

As the power grid undergoes significant paradigm shift due to the increasing penetration of renewable generation, the ever-growing installation of behind-the-meter (BTM) solar generation in the power grid also has a significant impact on nodal loads, posing challenges on transmission operators. Furthermore, increasing frequent and severe extreme weather events intertwine with ubiquitous BTM solar generations and have amplified the challenges of accurately model nodal load profiles, especially under the lack of ground-truth information for verification. To tackle these challenges, this paper introduces a bilevel model that utilizes year-long data (e.g., proxy solar, zonal load, and individual node load profiles) to disaggregate metered profiles into actual demand and BTM solar generation at each transmission node. The proxy solar not only scales the BTM solar generation of individual nodes but also create a compensation term for enhancing performance on days with unexpected extreme weather events. The proposed algorithm is validated with real-world PJM Interconnection data during unexpected events like the recent Winter Storm Elliott. For quantitative evaluations, a novel Score error is introduced, which is based on mean percentages and load scales and offers a universal assessment method suitable for all nodes and different data formats (e.g., normalized or raw values).

behind-the-meter solar, load disaggregations, load↗
DOE OSTI2023

Significant charge transport effects due to subtle molecular changes in nitroxide radical single crystals

Radical-containing materials have received renewed interest due to their charge-conducting properties. However, most work focuses on macromolecules, while less attention has been paid to radical-containing small molecules. Here, we quantify the electrical conductivity in two organic radical single crystals and demonstrate that a subtle change in atomic connectivity drastically alters the macroscopic electronic properties of the materials. One radical crystal has an electrical conductivity of 3S m –1 , which is the among the highest values for nonconjugated radical conductors over a 1 μm scale. However, the other radical crystal has a 1,000-fold lower conductivity, despite their extremely similar molecular structures. The temperature-dependent conductivity follows the variable range hopping mechanism for both radical crystals, and the difference in effective charge mobility is the reason for the conductivity difference. These results present a clear picture of the design rules and charge transport mechanism for radical-based small-molecule materials.

36 MATERIALS SCIENCE↗
DOE OSTI2020

Understanding the Spatial Organization of Simultaneous Heavy Precipitation Events Over the Conterminous United States

We introduce the idea of simultaneous heavy precipitation events (SHPEs) to understand whether extreme precipitation has a spatial organization manifested as specified tracks or contiguous fields with inherent scaling relationships. For this purpose, we created a database of SHPEs using ground-based precipitation observations recorded by the daily Global Historical Climatology Network across the conterminous United States during 1900–2014. SHPEs are examined for their seasonality, spatial manifestation, orientation, and areal extent. We quantified the spatial distribution of the centroids and principal axes of SHPEs and their quasi-elliptical manifestations, azimuthal orientations, and areal extents on the ground. Four seasons, December-January-February (DJF), March-April-May (MAM),June-July-August (JJA), and September-October-November (SON) are considered to examine the spatial patterns and associated large-scale atmospheric circulations. Results indicate that there are 54, 58, 103, and204 SHPEs in DJF, MAM, JJA, and SON seasons, and their longest stretches range on average between 650and 1,600, between 850 and 1,500, between 950 and 1,550, and between 750 and 1,450 km, respectively. SHPEs in the DJF, MAM, and JJA seasons occur mostly over the Pacific coast and central and midwestern United States, respectively. The atmospheric circulation patterns and mechanisms of precipitable water vapor and moisture transport in the atmosphere are also discussed in relation to these SHPEs. Power laws explain SHPEs' underlying area scaling behavior in all the four seasons, with stronger evidence in DJF and MAM. A seasonal spatial risk model is developed to predict the likelihood of SHPEs. Quantifying the characteristics of SHPEs and modeling their footprints can improve the projections of flood risk and understanding of damages to interconnected infrastructure systems.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Liquid-liquid phase separation in supercooled water from ultrafast heating of low-density amorphous ice

Recent experiments continue to find evidence for a liquid-liquid phase transition (LLPT) in supercooled water, which would unify our understanding of the anomalous properties of liquid water and amorphous ice. These experiments are challenging because the proposed LLPT occurs under extreme metastable conditions where the liquid freezes to a crystal on a very short time scale. Here, we analyze models for the LLPT to show that coexistence of distinct high-density and low-density liquid phases may be observed by subjecting low-density amorphous (LDA) ice to ultrafast heating. We then describe experiments in which we heat LDA ice to near the predicted critical point of the LLPT by an ultrafast infrared laser pulse, following which we measure the structure factor using femtosecond x-ray laser pulses. Consistent with our predictions, we observe a LLPT occurring on a time scale <100 ns and widely separated from ice formation, which begins at times >1 μs.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2025

Photon acceleration of high-intensity vector vortex beams into the extreme ultraviolet

Extreme ultraviolet (XUV) light sources allow for the probing of bound electron dynamics on attosecond scales, interrogation of high-energy-density matter, and access to novel regimes of strong-field quantum electrodynamics. Despite the importance of these applications, coherent XUV sources remain relatively rare, and those that do exist are limited in their peak intensity and spatio-polarization structure. Here, we demonstrate that photon acceleration of an optical vector vortex pulse in the moving density gradient of an electron beam–driven plasma wave can produce a high-intensity, tunable-wavelength XUV pulse with the same vector vortex structure as the original pulse. Quasi-3D, boosted-frame particlein- cell simulations show the transition of optical vector vortex pulses with 800-nm wavelengths and intensities below 10 18 W/cm 2 to XUV vector vortex pulses with 36-nm wavelengths and intensities exceeding 10 20 W/cm 2 over a distance of 1.2 cm. The XUV pulses have sub-femtosecond durations and nearly flat phase fronts. The production of such high-quality, high-intensity XUV vector vortex pulses could expand the utility of XUV light as a diagnostic and driver of novel light–matter interactions.

43 PARTICLE ACCELERATORS↗
DOE OSTI2023

Impact of Soret effect on hydrogen and helium retention in PFC tungsten under ELM-like conditions

Abstract In our previous work, we have demonstrated using nonequilibrium molecular-dynamics simulations that the fluxes of helium and self-interstitial atoms in the presence of a thermal gradient in tungsten are directed opposite to the heat flux, indicating that species transport is governed by a Soret effect, namely, thermal-gradient-driven diffusion, characterized by a negative heat of transport that drives species transport uphill, i.e. from the cooler to the hot regions of the tungsten sample. In this work, the findings of our thermal and species transport analysis have been implemented in our cluster-dynamics code, Xolotl, which has been used to compute temperature and species profiles over spatiotemporal scales representative of plasma-facing component (PFC) tungsten under typical reactor operating conditions, including extreme heat loads at the plasma-facing surface characteristic of plasma instabilities that induce edge localized modes (ELMs). We demonstrate that the steady-state species profiles, when properly accounting for the Soret effect, vary significantly from those where temperature-gradient-driven transport is not accounted for and discuss the implications of such a Soret effect on the response to plasma exposure of plasma-facing tungsten. Although our cluster-dynamics simulations do not yet include self-clustering of helium or hydrogen blister formation, our simulation results show that the Soret effect substantially reduces helium and hydrogenic species retention inside PFC tungsten.

36 MATERIALS SCIENCE↗
DOE OSTI2022

The role of supernova convection for the lower mass gap in the isolated binary formation of gravitational wave sources

ABSTRACT Understanding the astrophysical phenomena involving compact objects requires an insight about the engine behind the core-collapse supernovae (SNe) and the fate of the stellar collapse of massive stars. In particular, this insight is crucial in developing an understanding of the origin and formation channels of the growing populations of the detected black hole–black hole, black hole–neutron star, and neutron star–neutron star mergers. The time-scale of convection growth may have a large effect on the strength of SN explosion and therefore also on the mass distribution of stellar remnants. We adopt new formulas for the relation between the pre-SN star properties and their remnants and check how they impact the population of double compact object (DCO) mergers formed via the isolated binary evolution. The new formulas give one the ability to test a wide spectrum of assumptions on the convection growth time. In particular, the different variants allow for a smooth transition between having a deep mass gap and a remnant mass distribution filled by massive neutron stars and low-mass black holes. We present the distribution of masses, mass ratios, and the local merger rate densities of DCO for the different variants of new formulas and test them together with different approaches to other highly uncertain processes. We find that the mass distribution of DCO mergers is sensitive to the adopted assumption on the SN convection growth time-scale up to $m_1+m_2 \lesssim 35 \, \mathrm{M}_{\odot }$. Between the two extreme tested variants the probability of compact object formation within the mass gap may differ by up to approximately two orders of magnitude.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2023

Dimensionality-driven metal to Mott insulator transition in two-dimensional 1T-TaSe2

Abstract Two-dimensional materials represent a major frontier for research into exotic many-body quantum phenomena. In the extreme two-dimensional limit, electron-electron interaction often dominates over other electronic energy scales, leading to strongly correlated effects such as quantum spin liquid and unconventional superconductivity. The dominance is conventionally attributed to the lack of electron screening in the third dimension. Here, we discover an intriguing metal to Mott insulator transition in 1T-TaSe2 that defies conventional wisdom. Specifically, we find that dimensionality crossover, instead of reduced screening, drives the transition in atomically thin 1T-TaSe2. A dispersive band crossing the Fermi level is found to be responsible for the bulk metallicity in the material. Reducing the dimensionality, however, effectively quenches the kinetic energy of these initially itinerant electrons, and drives the material into a Mott insulating state. The dimensionality-driven metal to Mott insulator transition resolves the long-standing dichotomy between metallic bulk and insulating surface of 1T-TaSe2. Our work further reveals a new pathway for modulating two-dimensional materials that enables exploring strongly correlated systems across uncharted parameter space.

1T-TaSe2↗
DOE OSTI2022

Stream Temperature Predictions for River Basin Management in the Pacific Northwest and Mid-Atlantic Regions Using Machine Learning

Stream temperature (Ts) is an important water quality parameter that affects ecosystem health and human water use for beneficial purposes. Accurate Ts predictions at different spatial and temporal scales can inform water management decisions that account for the effects of changing climate and extreme events. In particular, widespread predictions of Ts in unmonitored stream reaches can enable decision makers to be responsive to changes caused by unforeseen disturbances. In this study, we demonstrate the use of classical machine learning (ML) models, support vector regression and gradient boosted trees (XGBoost), for monthly Ts predictions in 78 pristine and human-impacted catchments of the Mid-Atlantic and Pacific Northwest hydrologic regions spanning different geologies, climate, and land use. The ML models were trained using long-term monitoring data from 1980–2020 for three scenarios: (1) temporal predictions at a single site, (2) temporal predictions for multiple sites within a region, and (3) spatiotemporal predictions in unmonitored basins (PUB). In the first two scenarios, the ML models predicted Ts with median root mean squared errors (RMSE) of 0.69–0.84 °C and 0.92–1.02 °C across different model types for the temporal predictions at single and multiple sites respectively. For the PUB scenario, we used a bootstrap aggregation approach using models trained with different subsets of data, for which an ensemble XGBoost implementation outperformed all other modeling configurations (median RMSE 0.62 °C).The ML models improved median monthly Ts estimates compared to baseline statistical multi-linear regression models by 15–48% depending on the site and scenario. Air temperature was found to be the primary driver of monthly Ts for all sites, with secondary influence of month of the year (seasonality) and solar radiation, while discharge was a significant predictor at only 10 sites. The predictive performance of the ML models was robust to configuration changes in model setup and inputs, but was influenced by the distance to the nearest dam with RMSE <1 °C at sites situated greater than 16 and 44 km from a dam for the temporal single site and regional scenarios, and over 1.4 km from a dam for the PUB scenario. Our results show that classical ML models with solely meteorological inputs can be used for spatial and temporal predictions of monthly Ts in pristine and managed basins with reasonable (<1 °C) accuracy for most locations.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

Efforts and Innovations to Promote Data Sharing and Data Accessibility in the EGS Collab Experiments

Large-scale scientific research programs such as the EGS Collab experiments and the FORGE program play extremely important roles in advancing geothermal technologies. Such efforts involve a large number of researchers from multiple institutions, last multiple years, and generate large, complex datasets. The value of the research efforts is only realized when the datasets are used by a large community of researchers in the decades to come. A challenge is that due to the complexity of the data, it could require a user to devote a serious effort into understanding the data before the data can be effectively utilized. In the EGS Collab experiments, the team has devoted remarkable efforts and developed many innovative solutions to make the data more accessible to broader team members and future users. This paper documents the experience gained and lessons learned by the EGS Collab team in disseminating the data in the most informative and inspiring forms to maximize the value of this precious dataset. "Accessibility"in the title does not only mean making the raw data available for download. Particularly, we want to emphasize the importance of organizing, annotating, and presenting the data in ways to make it easy to digest by prospective consumers of the data.

circulation↗
DOE OSTI2024

Scalable Multiphysics Block Preconditioning for Low Mach Number Compressible Resistive MHD with Application to Magnetic Confinement Fusion

This study investigates multiphysics block preconditioners that are critical in devising scalable Newton–Krylov iterative solvers for longer time-scale fully implicit fluid plasma models. The specific model of interest is the visco-resistive, low Mach number, compressible magnetohydrodynamics (MHD) model. This model describes the dynamics of conducting fluids in the presence of electromagnetic fields and can be used to study aspects of astrophysical phenomena, important science and technology applications, and basic plasma physics. The specific application of interest that motivates this study is the macroscopic simulation of longer time-scale stability and disruptions of magnetic confinement fusion devices, specifically the ITER Tokamak. The computational solution of the governing balance equations for mass, momentum, heat transfer, and magnetic induction for resistive MHD systems can be extremely challenging. These difficulties arise from both the strong nonlinear, nonsymmetric coupling of fluid and electromagnetic phenomena as well as the significant range of time and length scales that the interactions of these physical mechanisms produce. To handle the range of time and spatial scales of interest, a fully implicit unstructured variational multiscale finite element formulation is employed. For the scalable solution of the Newton linearized systems, fully coupled block preconditioners are designed to leverage algebraic multigrid subsolves. In conclusion, results are presented for the strong and weak scaling of the method as well as the robustness of these techniques for a large range of Lundquist numbers.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

A Novel Machine Learning Workflow to Classify Mobile Home Parks at Scale

Understanding the built environment is essential to the overall study of population dynamics, grid infrastructure, emergency response, among others. In the United States there are multiple classifications for buildings within the built environment such as residential, signifying family homes while commercial buildings consist of apartments or larger structures which are multi-purpose. While there is a high level of understanding of where these aforementioned structures are located, there is a third class of structures, mobile home parks (MHP) which have been under-represented in the literature despite there being an estimated 2.7 million of them within the United States. Research has shown that individuals who reside in MHP are at higher risk to extreme events due to their location and structural integrity of residence. Attention must now turn to identifying MHP at scale to help first responders and policy makers understand where these at risk populations reside. To address for this gap, we develop a novel methodology to infer MHP at scale based off morphologies derived at a building level. Here, we show that across 3 million buildings in 6 states within the United States it is possible to identify MHP with 83% accuracy. This novel approach to identify MHP from other structures within the built environment using a machine learning approach provides a new tool to leverage in relation to helping at-risk populations.

Stipek, Clinton [Oak Ridge National Laboratory (OR↗
DOE OSTI2021

Reduced ecosystem resilience quantifies fine-scale heterogeneity in tropical forest mortality responses to drought

Sensitivity of forest mortality to drought in carbon-dense tropical forests remains fraught with uncertainty, while extreme droughts are predicted to be more frequent and intense. Here, in this study, the potential of temporal autocorrelation of high-frequency variability in Landsat Enhanced Vegetation Index (EVI), an indicator of ecosystem resilience, to predict spatial and temporal variations of forest biomass mortality is evaluated against in situ census observations for 64 site-year combinations in Costa Rican tropical dry forests during the 2015 ENSO drought. Temporal autocorrelation, within the optimal moving window of 24 months, demonstrated robust predictive power for in situ mortality (leave-one- out cross-validation R 2 = 0.54), which allows for estimates of annual biomass mortality patterns at 30 m resolution. Subsequent spatial analysis showed substantial fine-scale heterogeneity of forest mortality patterns, largely driven by drought intensity and ecosystem properties related to plant water use such as forest deciduousness and topography. Highly deciduous forest patches demonstrated much lower mortality sensitivity to drought stress than less deciduous forest patches after elevation was controlled. Our results highlight the potential of high-resolution remote sensing to “fingerprint” forest mortality and the significant role of ecosystem heterogeneity in forest biomass resistance to drought.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2024

Ad-Mat: Adaptations of Mature Manufacturing Strategies for Accelerated Redox Flow Battery Deployment

The concept of the Ad-Mat approach is to leverage existing adjacent markets across a broad scope of technologies in order to reduce the manufacturing learning curve and ultimately accelerate redox flow battery (RFB) deployment at scale. Lithium-ion batteries (LIBs) are currently the dominant energy storage technology, and they came to technological maturity under unique market conditions when there was no meaningful competition in the consumer electronic and electric vehicle (EV) space. Today, alternative chemistries that may be technologically better-suited for long-duration storage applications are experiencing a high barrier to entry. This is in large part due to the substantial bias towards the scaled-up production and supply chain that now exists for LIBs. In the case of RFBs in particular, numerous analyses have suggested that RFBs should theoretically have a much lower system cost than LIBs - however, this relies on a mature and competitive manufacturing landscape, which has been extremely challenging to achieve for both flow batteries and other LIB competitors. At the moment, LIB alternatives tend to have isolated small-scale manufacturing pathways, which preclude the economies of scale that would be required to compete with the mature LIB industry. In the present state of the industry, niche manufacturing tools and approaches have evolved to support each alternative technology, such that there is substantial replication and duplication in effort. Continuing to pursue a strategy of isolated manufacturing processes/approaches for each LIB-alternative may never allow for at-scale deployment. In order for RFBs to meaningfully compete with LIBs in the realm of LDES, a new disruptive approach based on cross-industry learning and coordination is needed - and this is exactly what our Ad-Mat concept aims to tackle. In this re-envisioned manufacturing landscape, tools and processes from mature industries can be adapted and deployed across the range of alternative energy storage technologies. Adapting tools, equipment, processes, and industrial learning from mature industries to meet the technological requirements of RFBs would open new markets for existing OEMs in adjacent industries, would prevent unnecessary duplication and re-development, would improve efficiency across the manufacturing chain, and would ultimately support reduced costs and accelerated deployment of RFBs at scale.

adaptive manufacturing↗
DOE OSTI2022

Simulation of ENSO Teleconnections to Precipitation Extremes over the United States in the High-Resolution Version of E3SM

Abstract We evaluate the simulated teleconnection of El Niño–Southern Oscillation (ENSO) to winter season precipitation extremes over the United States in a long (98 years) 1950 control high-resolution version (HR; 25-km nominal atmosphere model horizontal resolution) of the U.S. Department of Energy’s (DOE) Energy Exascale Earth System Model version 1 (E3SMv1). The model bias and spatial pattern of ENSO teleconnections to mean and extreme precipitation in HR overall are similar to the low-resolution model’s (LR; 110 km) historical simulation (four-member ensemble, 1925–59). However, over the southeastern United States (SE-U.S.), HR produces stronger El Niño–associated extremes, reducing LR’s model bias. Both LR and HR produce weaker than observed increase in storm track activity during El Niño events there, but HR improves the ENSO-associated variability of moisture transport over SE-U.S. During El Niño, stronger vertical velocities in HR produce stronger large-scale precipitation, causing larger latent heating of the troposphere that pulls in more moisture from the Gulf of Mexico into the SE-U.S. This positive feedback also contributes to the stronger mean and extreme precipitation response in HR. Over the Pacific Northwest, LR’s bias of stronger than observed La Niña associated extremes is amplified in HR. Both models simulate stronger than observed moisture transport from the Pacific Ocean into the region during La Niña years. The amplified HR bias there is due to stronger orographically driven vertical updrafts that create stronger large-scale precipitation, despite weaker La Niña–induced storm track activity. Significance Statement New high-resolution Earth system models (ESMs) solve mathematical equations of fluid flow at much smaller spatial scales than prevalent ESMs, and thus are prohibitively expensive to compute. However, they can be useful for simulating accurate details of regional climate extremes that are driven by naturally occurring climate oscillations like El Niño–Southern Oscillation (ENSO). Here, we evaluate the simulation of ENSO-driven precipitation extremes over the United States in the high-resolution version of the U.S. Department of Energy’s new Energy Exascale Earth System Model version 1. We find that the high-resolution model improves upon its low-resolution counterpart over the southeastern United States by producing a better transport of moisture into the region from the Gulf of Mexico during El Niño. Over the U.S. Pacific Northwest, the high-resolution model simulates the atmospheric flow in more detail over the complex mountainous terrain. However, it also brings in more moisture from the Pacific Ocean just like the low-resolution model. This causes it to produce precipitation extremes during La Niña years there that are stronger than that observed in the real world.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

Three practical workflow schedulers for easy maximum parallelism

Runtime scheduling and workflow systems are an increasingly popular algorithmic component in HPC because they allow full system utilization with relaxed synchronization requirements. There are so many special-purpose tools for task scheduling, one might wonder why more are needed. Use cases seen on the Summit supercomputer needed better integration with MPI and greater flexibility in job launch configurations. Preparation, execution, and analysis of computational chemistry simulations at the scale of tens of thousands of processors revealed three distinct workflow patterns. A separate job scheduler was implemented for each one using extremely simple and robust designs: file-based, task-list based, and bulk-synchronous. Comparing to existing methods shows unique benefits of this work, including simplicity of design, suitability for HPC centers, short startup time, and well-understood per-task overhead. All three new tools have been shown to scale to full utilization of Summit, and have been made publicly available with tests and documentation. This work presents a complete characterization of the minimum effective task granularity for efficient scheduler usage scenarios. Here, these schedulers have the same bottlenecks, and hence similar task granularities as those reported for existing tools following comparable paradigms.

97 MATHEMATICS AND COMPUTING↗