Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “system partition”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 865 records · Page 48
All sourcesNASA NTRSDOE OSTI
NASA NTRS2004

Using the GeoFEST Faulted Region Simulation System

GeoFEST (the Geophysical Finite Element Simulation Tool) simulates stress evolution, fault slip and plastic/elastic processes in realistic materials, and so is suitable for earthquake cycle studies in regions such as Southern California. Many new capabilities and means of access for GeoFEST are now supported. New abilities include MPI-based cluster parallel computing using automatic PYRAMID/Parmetis-based mesh partitioning, automatic mesh generation for layered media with rectangular faults, and results visualization that is integrated with remote sensing data. The parallel GeoFEST application has been successfully run on over a half-dozen computers, including Intel Xeon clusters, Itanium II and Altix machines, and the Apple G5 cluster. It is not separately optimized for different machines, but relies on good domain partitioning for load-balance and low communication, and careful writing of the parallel diagonally preconditioned conjugate gradient solver to keep communication overhead low. Demonstrated thousand-step solutions for over a million finite elements on 64 processors require under three hours, and scaling tests show high efficiency when using more than (order of) 4000 elements per processor. The source code and documentation for GeoFEST is available at no cost from Open Channel Foundation. In addition GeoFEST may be used through a browser-based portal environment available to approved users. That environment includes semi-automated geometry creation and mesh generation tools, GeoFEST, and RIVA-based visualization tools that include the ability to generate a flyover animation showing deformations and topography. Work is in progress to support simulation of a region with several faults using 16 million elements, using a strain energy metric to adapt the mesh to faithfully represent the solution in a region of widely varying strain.

Geophyical Finite Element Simulation Tool (GeoFEST↗
NASA NTRS1987

A dual-processor multi-frequency implementation of the FINDS algorithm

This report presents a parallel processing implementation of the FINDS (Fault Inferring Nonlinear Detection System) algorithm on a dual processor configured target flight computer. First, a filter initialization scheme is presented which allows the no-fail filter (NFF) states to be initialized using the first iteration of the flight data. A modified failure isolation strategy, compatible with the new failure detection strategy reported earlier, is discussed and the performance of the new FDI algorithm is analyzed using flight recorded data from the NASA ATOPS B-737 aircraft in a Microwave Landing System (MLS) environment. The results show that low level MLS, IMU, and IAS sensor failures are detected and isolated instantaneously, while accelerometer and rate gyro failures continue to take comparatively longer to detect and isolate. The parallel implementation is accomplished by partitioning the FINDS algorithm into two parts: one based on the translational dynamics and the other based on the rotational kinematics. Finally, a multi-rate implementation of the algorithm is presented yielding significantly low execution times with acceptable estimation and FDI performance.

Godiwala, Pankaj M.↗
NASA NTRS1999

Wheat Response to Differences In Water and Nutritional Status Between Zeoponic and Hydroponic Growth Systems

Hydroponic culture has traditionally been used for controlled environment life support systems (CELSS) because the optimal environment for roots supports high growth rates. Recent developments in zeoponic substrate and microporous tube irrigation (ZPT) also offer high control of the root environment. This study compared the effect of differences in water and nutrient status of ZPT or hydroponic culture on growth and yield of wheat (Triticum aestivum L., CV 'USU-Apogee'). In a side-by-side test in a controlled environment, wheat was grown in ZPT and recirculating hydroponics to maturity. Water use by plants grown in both culture systems peaked at 15-20 L per square meters per d up to day 40, after which it declined more rapidly for plants grown in ZPT culture due to earlier senescence of leaves. No consistent differences were noted in water status between plants grown in the two culture systems. Although yield was similar, harvest index was 28% lower for plants grown in ZPT versus hydroponic culture. Sterile green tillers made up 12% and 0% of the biomass of plants grown in ZPT and hydroponic culture, respectively. Differences in biomass partitioning were attributed primarily to NH4 -N nutrition of plants grown in ZPT as compared with NO3-N in hydroponic nutrient solution. It was likely that NH4-N induced Ca deficiency produced excess tillering and lower harvest index for plants grown in ZPT culture. These results suggest that further refinements in zeoponic substrate would make ZPT culture a viable alternative for achieving high productivity in a CELSS.

Steinberg, Susan L.↗
NASA NTRS1991

Introduction to IND and recursive partitioning, version 1.0

This manual describes the IND package for learning tree classifiers from data. The package is an integrated C and C shell re-implementation of tree learning routines such as CART, C4, and various MDL and Bayesian variations. The package includes routines for experiment control, interactive operation, and analysis of tree building. The manual introduces the system and its many options, gives a basic review of tree learning, contains a guide to the literature and a glossary, lists the manual pages for the routines, and instructions on installation.

Buntine, Wray↗
NASA NTRS1989

Proliferation of maize (Zea mays L.) roots in response to localized supply of nitrate

Maize (Zea mays L.) plants with two primary nodal root axes were grown for 8 d in flowing nutrient culture with each axis independently supplied with NO3-. Dry matter accumulation by roots was similar whether 1.0 mol m-3 NO3- was supplied to one or both axes. When NO3- was supplied to only one axis, however, accumulation of dry matter within the root system was significantly greater in the axis supplied with NO3-. The increased dry matter accumulation by the +N-treated axis was attributable entirely to increased density and growth of lateral branches and not to a difference in growth of the primary axis. Proliferation of lateral branches for the +N axis was associated with the capacity for in situ reduction and utilization of a portion of the absorbed NO3-, especially in the apical region where lateral primordia are initiated. Although reduced nitrogen was translocated to the -N axis, concentrations in the -N axis remained significantly lower than in the +N axis. The concentration of reduced nitrogen, as well as in vitro NO3- reductase activity, was greater in apical than in more basal regions of the +N axis. The enhanced proliferation of lateral branches in the +N axis was accompanied by an increase in total respiration rate of the axis. Part of the increased respiration was attributable to increased mass of roots. The specific respiration rate (micromoles CO2 evolved per hour per gram root dry weight) was also greater for the +N than for the -N axis. If respiration rate is taken as representative of sink demand, stimulation of initiation and growth of laterals by in situ utilization of a localized exogenous supply of NO3- establishes an increased sink demand through enhanced metabolic activity and the increased partitioning of assimilates to the +N axis responds to the difference in sink demand between +N and -N axes.

NASA Discipline Life Support Systems↗
NASA NTRS2011

Drop Size Distribution Measurements Supporting the NASA Global Precipitation Measurement Mission: Infrastructure and Preliminary Results

Global Precipitation Measurement Mission (GPM) retrieval algorithm validation requires datasets that characterize the 4-D structure, variability, and correlation properties of hydrometeor particle size distributions (PSD) and accumulations over satellite fields of view (5 -- 50 km). Key to this process is the combined use of disdrometer and polarimetric radar platforms. Here the disdrometer measurements serve as a reference for up-scaling dual-polarimetric radar observations of the PSD to the much larger volumetric sampling domain of the radar. The PSD observations thus derived provide a much larger data set for assessing DSD variability, and satellite-based precipitation retrieval algorithm assumptions, in all three spatial dimensions for a range of storm types and seasons. As one component of this effort, the GPM Ground Validation program recently acquired five 3rd generation 2D Video disdrometers as part of its Disdrometer and Radar Observations of Precipitation Facility (DROP), currently hosted in northern Alabama by the NASA Marshall Space Flight Center and the University of Alabama in Huntsville. These next-generation 2DVDs were operated and evaluated in different phases of data collection under the scanning domain of the UAH ARMOR C-band dual-polarimetric radar. During this period approximately 7500 minutes of PSD data were collected and processed to create gamma size distribution parameters using a truncated method of moments approach. After creating the gamma parameter datasets the DSDs were then used as input to T-matrix code for computation of polarimetric radar moments at C-band. The combined dataset was then analyzed with two basic objectives in mind: 1) the investigation of seasonal variability in the rain PSD parameters as observed by the 2DVDs; 2) the use of combined polarimetric moments and observed gamma distribution parameters in a functional form to retrieve PSD parameters in 4-D using the ARMOR radar for precipitation occurring in different seasons and for different rain system types. Preliminary results suggest that seasonal variations in the DSD parameters do occur, but are most pronounced when comparing tropical PSDs to either winter or summer convective precipitation. For example the previously documented shift to relatively smaller drop diameters in higher number concentrations for equivalent rain rate bins was observed in tropical storm rainbands occurring over Huntsville. On a more inter seasonal basis empirical fits between parameters such as D0 and ZDR do not appear to exhibit robust seasonal biases- i.e., one fit seems to work for all seasons within acceptable standard error (O[10%]) for estimates of D0. In polarimetric retrievals of the vertical variability in PSD (rain layer) for a tropical rainband we find that the Do varies with height when partitioned by specified precipitation categories (e.g., convective or stratiform, heavy and light stratiform etc.) but this variation is of order 10-20% and is smaller than the difference in D0 observed between the basic delineation of convective and stratiform precipitation types. Currently we are expanding our analysis of the vertical structure of the PSD to include several seasonally and/or dynamically-different storm system types (e.g., winter convection and stratiform events; summer mid-latitude convective etc.) sampled by ARMOR. The study will present the results of our combined analyses.

Petersen, Walter A.↗
DOE OSTI2021

Physical bioenergetics: Energy fluxes, budgets, and constraints in cells

Cells are the basic units of all living matter which harness the flow of energy to drive the processes of life. While the biochemical networks involved in energy transduction are well-characterized, the energetic costs and constraints for specific cellular processes remain largely unknown. In particular, what are the energy budgets of cells? What are the constraints and limits energy flows impose on cellular processes? Do cells operate near these limits, and if so how do energetic constraints impact cellular functions? Physics has provided many tools to study nonequilibrium systems and to define physical limits, but applying these tools to cell biology remains a challenge. Physical bioenergetics, which resides at the interface of nonequilibrium physics, energy metabolism, and cell biology, seeks to understand how much energy cells are using, how they partition this energy between different cellular processes, and the associated energetic constraints. In this work, we review recent advances and discuss open questions and challenges in physical bioenergetics.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2024

Distributed Multi-GPU Community Detection on Exascale Computing Platforms

Community detection is a fundamental operation in graph mining, and by uncovering hidden structures and patterns within complex systems it helps solve fundamental problems pertaining to social networks, such as information diffusion, epidemics, and recommender systems. Scaling graph algorithms for massive networks becomes challenging on modern distributed-memory multi-GPU (Graphics Processing Unit) systems due to limitations such as irregular memory access patterns, load imbalances, higher communication-computation ratios, and cross-platform support. We present a novel algorithm HiPDPL-GPU (distributed parallel Louvain) to address these challenges. We conduct experiments involving different partitioning techniques to achieve optimized performance of HiPDPL-GPU on the two largest supercomputers: Frontier and Summit. Remarkably, HiPDPL-GPU processes a graph with 4.2 billion edges in less than 3 minutes using 1024 GPUs. Qualitatively performance of HiPDPL-GPU is similar or better compared to other state-of-the-art CPU- and GPU-based implementations. While prior GPU implementations have predominantly employed CUDA, our first-of-its-kind implementation for community detection is cross-platform, accommodating both AMD and NVIDIA GPUs.

graph algorithms, high performance comptuing↗
DOE OSTI2026

Mountain Basin Controls on the Snow-to-Streamflow Signal: An AIC-Weighted Multiple Linear Regression Framework

A regression-based analysis quantifies how basin characteristics modulate the snow-to-streamflow signal. First, we use the ERA5-Land reanalysis gridded product (European Centre for Medium Range Weather Forecasts reanalysis 5 -Land component) for 4,655 hydrologic unit code - 10 (HUC10) mountain basins across the western United States (US) for water years 1987–2024. Linear regressions are performed for peak snow water equivalent (SWE) and annual streamflow for each mountain basin. Models use ordinary least squares in Python’s statsmodels package. After which, an Akaike Information Criterion (AIC)–weighted ensemble multiple linear regression (MLR) framework with 47 watershed traits is used to predict the linear regression coefficient of determination (r-squared) defining the ability of peak SWE to predict annual streamflow across all mountain basin. Predictor sets are constrained to avoid multicollinearity by excluding models with variance inflation factors (VIF) greater than 5. Mountain basin traits included in the MLR include seasonal climate, topography, vegetation type and structure, and bedrock geology. Accepted models are considered if their AIC is within 2.0 of the model with the minimum AIC, or best model. To compare predictor influence across acceptable models, we computed standardized regression coefficients. To evaluate structural redundancy among models, we constructed binary inclusion vectors for each acceptable model, denoting whether a predictor was present (1) or absent (0). Core predictor variables are defined as occurring in at least 67% of the acceptable models. For this regional analysis, only one model was found acceptable, with higher snow-to-streamflow translation (higher r-squared) occurring in colder mountain basins with higher relative winter precipitation, more snow accumulation and a lower fraction of annual precipitation that falls in the spring and summer. The second component of the data package uses previously published, high-resolution output from an integrated hydrological model of the East River watershed using the U.S. Geological Survey Groundwater and Surface water Flow model (GSFLOW, doi:10.15485/1998576). East River MLR expands upon the approach described above to explore the response of five streamflow metrics—annual streamflow, runoff efficiency, 7-day minimum flow, low-flow duration, and non-perennial stream fraction to snow system indicators including peak SWE, snow-covered area, snow disappearance date, and the fraction of basin area characterized by low-to-no snow, as well as seasonal precipitation and temperature, and annual hydrologic variables representing soil moisture, evapotranspiration (ET), the partitioning of incoming precipitation to evapotranspiration (ET/P), groundwater storage, and groundwater inflow to streams. MLR was done on all water years (P0: 1987-2024) and for each period as determined in the split analysis using pooled regression techniques (P1: 1987-2011 and P2: 2012-2024) to evaluate shifting predictor variable emphasis on streamflow generation. Results indicate that since 2012, peak SWE has lost statistical strength in its prediction of annual streamflow and runoff efficiency, and the indirect influence of spring temperature has emerged as critically important. Low-flow metrics remain largely influenced by soil moisture, vegetation water use and groundwater inflows with summer precipitation becoming a direct influence on minimum summer flow. Together, these data and Python-based analysis tools provide a framework for identifying the key watershed characteristics that control how streamflow responds to snow from year to year. The package also helps quantify uncertainty in statistical models and assess how snow–streamflow relationships vary across regions and over time. This dataset contains comma-separated values files (.csv), text files (.txt), python code files (.py), figure files (.png), and shapefiles (.cpg, .dbf, .prj, .sbn, .sbx, .shp, .xml). Further details on file contents and MLR execution can be found in the readme file and the FLMD files. Work was supported by the Watershed Function Science Focus Area at Lawrence Berkeley National Laboratory funded by the US Department of Energy, Office of Science, Biological and Environmental Research under Contract No. DE-AC02-05CH11231.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

Regime-based aerosol–cloud interactions from CALIPSO-MODIS and the Energy Exascale Earth System Model version 2 (E3SMv2) over the Eastern North Atlantic

This study investigates aerosol-cloud interactions in marine boundary layer (MBL) clouds using an advanced deep-learning-driven synoptic-regime-based framework, combining satellite data (CALIPSO vertically resolved aerosol extinction and MODIS cloud properties) with 1° nudged Energy Exascale Earth System Model version 2 (E3SMv2) simulation over the Eastern North Atlantic (ENA; ∼10°×10°, 2006–2014). The E3SMv2 captures observed seasonal variations in cloud droplet number concentrations (N d ) and liquid water path (LWP), though it systematically underestimates N d . We then partition ENA meteorology into four synoptic regimes (Pre-Trough, Post-Trough, Ridge, Trough) via a deep-learning clustering of ERA5 reanalysis fields, enabling regime-dependent aerosol-cloud interactions analyses. Both satellite and E3SMv2 exhibit an inverted-V LWP-N d relationship. In Post-Trough and Ridge regimes, the satellite shows stronger negative LWP-N d sensitivities than in Pre-Trough regime. The Trough regime displays a muted satellite LWP response. In comparison, the model predicts more exaggerated LWP responses across regimes, with LWP increasing too quickly at low N d and decreasing more sharply at high N d , especially in Pre-Trough and Trough regimes. These exaggerated model LWP sensitivities may stem from uncertainties in representing drizzle processes, entrainment, and turbulent mixing. As for N d susceptibility to aerosols, N d increases with MBL aerosol extinction in both datasets, but the simulated aerosol-cloud interactions appear oversensitive to meteorological conditions. Overall, E3SMv2 better captures aerosol effects under regimes that favor stratiform clouds (Post-Trough, Ridge), but performance deteriorates for regimes with deeper, dynamically complex clouds (Trough), highlighting the need for improved representations of those cloud processes in climate models.

Environmental sciences↗
NASA NTRS1974

Sequential square root filtering and smoothing of discrete linear systems

A square root information filter/smoother is derived using recursive least-squares arguments. The combined filter/smoother algorithm has the following attributes: (1) it has a square root structure, which enhances numerical accuracy; (2) filter and smoother mechanizations are identical in form, facilitating implementation of the smoother; and (3) storage and computation requirements are modest compared with other smoothing algorithms. Partitioning the results to separate bias parameters provides further computational economies and reduction of storage requirements.

Bierman, G. J.↗
NASA NTRS1996

Global Load Balancing with Parallel Mesh Adaption on Distributed-Memory Systems

Dynamic mesh adaption on unstructured grids is a powerful tool for efficiently computing unsteady problems to resolve solution features of interest. Unfortunately, this causes load imbalance among processors on a parallel machine. This paper describes the parallel implementation of a tetrahedral mesh adaption scheme and a new global load balancing method. A heuristic remapping algorithm is presented that assigns partitions to processors such that the redistribution cost is minimized. Results indicate that the parallel performance of the mesh adaption code depends on the nature of the adaption region and show a 35.5X speedup on 64 processors of an SP2 when 35% of the mesh is randomly adapted. For large-scale scientific computations, our load balancing strategy gives almost a sixfold reduction in solver execution times over non-balanced loads. Furthermore, our heuristic remapper yields processor assignments that are less than 3% off the optimal solutions but requires only 1% of the computational time.

Biswas, Rupak↗
NASA NTRS1996

Global Load Balancing with Parallel Mesh Adaption on Distributed-Memory Systems

Dynamic mesh adaptation on unstructured grids is a powerful tool for efficiently computing unsteady problems to resolve solution features of interest. Unfortunately, this causes load inbalances among processors on a parallel machine. This paper described the parallel implementation of a tetrahedral mesh adaption scheme and a new global load balancing method. A heuristic remapping algorithm is presented that assigns partitions to processors such that the redistribution coast is minimized. Results indicate that the parallel performance of the mesh adaption code depends on the nature of the adaption region and show a 35.5X speedup on 64 processors of an SP2 when 35 percent of the mesh is randomly adapted. For large scale scientific computations, our load balancing strategy gives an almost sixfold reduction in solver execution times over non-balanced loads. Furthermore, our heuristic remappier yields processor assignments that are less than 3 percent of the optimal solutions, but requires only 1 percent of the computational time.

Biswas, Rupak↗
NASA NTRS1999

Investigation of Next-Generation Earth Radiation Budget Radiometry

The current effort addresses two issues important to the research conducted by the Thermal Radiation Group at Virginia Tech. The first research topic involves the development of a method which can properly model the diffraction of radiation as it enters an instrument aperture. The second topic involves the study of a potential next-generation space-borne radiometric instrument concept. Presented are multiple modeling efforts to describe the diffraction of monochromatic radiant energy passing through an aperture for use in the Monte-Carlo ray-trace environment. Described in detail is a deterministic model based upon Heisenberg's uncertainty principle and the particle theory of light. This method is applicable to either Fraunhofer or Fresnel diffraction situations, but is incapable of predicting the secondary fringes in a diffraction pattern. Also presented is a second diffraction model, based on the Huygens-Fresnel principle with a correcting obliquity factor. This model is useful for predicting Fraunhofer diffraction, and can predict the secondary fringes because it keeps track of phase. NASA is planning for the next-generation of instruments to follow CERES (Clouds and the Earth's Radiant Energy System), an instrument which measures components of the Earth's radiant energy budget in three spectral bands. A potential next-generation concept involves modification of the current CERES instrument to measure in a larger number of wavelength bands. This increased spectral partitioning would be achieved by the addition of filters and detectors to the current CERES geometry. The capacity of the CERES telescope to serve for this purpose is addressed in this thesis.

Coffey, Katherine L.↗
NASA NTRS2000

Atmospheric Boundary Layer Wind Data During the Period January 1, 1998 Through January 21, 1999 at the Dallas-Fort Worth Airport: Data and Processing - Volume 2

The NASA Langley Research Center's Aircraft Vortex Spacing System (AVOSS) requires accurate winds and turbulence to determine aircraft wake vortex behavior near the ground. Volume 1 described the wind input and quality analysis process. This volume documents the data available during the period January 1998 through January 1999 and the partitioning and concatenation of files for time of day, turbulence, non duplication, cross wind profile quality and ceiling and visibility. It provides the resultant filtered files for the first three partitions as well as identification of ceiling/visibility conditions when they were below 5000 feet and 5 miles respectively. It also includes the wind profile quality flags to permit automatic selection of files for AVOSS application using selected ceiling/visibility and wind profile quality values and flags (or no flags).

Zak, J. Allen↗
NASA NTRS2014

The Messy Aerosol Submodel MADE3 (v2.0b): Description and a Box Model Test

We introduce MADE3 (Modal Aerosol Dynamics model for Europe, adapted for global applications, 3rd generation), an aerosol dynamics submodel for application within the MESSy framework (Modular Earth Submodel System). MADE3 builds on the predecessor aerosol submodels MADE and MADE-in. Its main new features are the explicit representation of coarse particle interactions both with other particles and with condensable gases, and the inclusion of hydrochloric acid (HCl)chloride (Cl) partitioning between the gas and condensed phases. The aerosol size distribution is represented in the new submodel as a superposition of nine lognormal modes: one for fully soluble particles, one for insoluble particles, and one for mixed particles in each of three size ranges (Aitken, accumulation, and coarse mode size ranges). In order to assess the performance of MADE3 we compare it to its predecessor MADE and to the much more detailed particle-resolved aerosol model PartMC-MOSAIC in a box model simulation of an idealized marine boundary layer test case. MADE3 and MADE results are very similar, except in the coarse mode, where the aerosol is dominated by sea spray particles. Cl is reduced in MADE3 with respect to MADE due to the HClCl partitioning that leads to Cl removal from the sea spray aerosol in our test case. Additionally, aerosol nitrate concentration is higher in MADE3 due to the condensation of nitric acid on coarse particles. MADE3 and PartMC- MOSAIC show substantial differences in the fine particle size distributions (sizes about 2 micrometers) that could be relevant when simulating climate effects on a global scale. Nevertheless, the agreement between MADE3 and PartMC-MOSAIC is very good when it comes to coarse particle size distribution, and also in terms of aerosol composition. Considering these results and the well-established ability of MADE in reproducing observed aerosol loadings and composition, MADE3 seems suitable for application within a global model.

Mixing state↗
NASA NTRS2016

A Spatiotemporal Indexing Approach for Efficient Processing of Big Array-Based Climate Data with MapReduce

Climate observations and model simulations are producing vast amounts of array-based spatiotemporal data. Efficient processing of these data is essential for assessing global challenges such as climate change, natural disasters, and diseases. This is challenging not only because of the large data volume, but also because of the intrinsic high-dimensional nature of geoscience data. To tackle this challenge, we propose a spatiotemporal indexing approach to efficiently manage and process big climate data with MapReduce in a highly scalable environment. Using this approach, big climate data are directly stored in a Hadoop Distributed File System in its original, native file format. A spatiotemporal index is built to bridge the logical array-based data model and the physical data layout, which enables fast data retrieval when performing spatiotemporal queries. Based on the index, a data-partitioning algorithm is applied to enable MapReduce to achieve high data locality, as well as balancing the workload. The proposed indexing approach is evaluated using the National Aeronautics and Space Administration (NASA) Modern-Era Retrospective Analysis for Research and Applications (MERRA) climate reanalysis dataset. The experimental results show that the index can significantly accelerate querying and processing (10 speedup compared to the baseline test using the same computing cluster), while keeping the index-to-data ratio small (0.0328). The applicability of the indexing approach is demonstrated by a climate anomaly detection deployed on a NASA Hadoop cluster. This approach is also able to support efficient processing of general array-based spatiotemporal data in various geoscience domains without special configuration on a Hadoop cluster.

big data↗
DOE OSTI2024

Distributed Multi-GPU Community Detection on Exascale Computing Platforms

Community detection is a fundamental operation in graph mining, and by uncovering hidden structures and patterns within complex systems it helps solve fundamental problems pertaining to social networks, such as information diffusion, epidemics, and recommender systems. Scaling graph algorithms for massive networks becomes challenging on modern distributed-memory multi-GPU (Graphics Processing Unit) systems due to limitations such as irregular memory access patterns, load imbalances, higher communication-computation ratios, and cross-platform support. We present a novel algorithm HiPDPL-GPU (Distributed Parallel Louvain) to address these challenges. We conduct experiments involving different partitioning techniques to achieve an optimized performance of HiPDPL-GPU on the two largest supercomputers: Frontier and Summit. Remarkably, HiPDPL-GPU processes a graph with 4.2 billion edges in less than 3 minutes using 1024 GPUs. Qualitatively, the performance of HiPDPL-GPU is similar or better compared to other state-of-the-art CPU- and GPU-based implementations. While prior GPU implementations have predominantly employed CUDA, our first-of-its-kind implementation for community detection is cross-platform, accommodating both AMD and NVIDIA GPUs.

Sattar, Naw Safrin↗