Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Community detection”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

DyG-DPCD: A Distributed Parallel Community Detection Algorithm for Large-Scale Dynamic Graphs

Dynamic (Temporal) graphs capture the valuable evolution of real-world systems, from the continuously evolving patterns of social interactions and genetic pathways to the dynamic fluctuations of economic forces. Detecting communities for such evolving networks poses unique challenges. Detecting and analyzing the evolution of communities within dynamic graphs unlocks valuable insights into the underlying structural and temporal patterns of real-world systems. However, the sheer volume of modern graph data and the inherent complexity of the temporal dimension pose significant challenges to scalable community detection algorithms. Addressing this gap, our work explores the limited landscape of scalable distributed-memory parallel methods specifically designed for dynamic network community detection. We propose a novel parallel algorithm, DyG-DPCD (Dynamic Graph Distributed Parallel Community Detection), to detect communities in dynamic networks using the Message Passing Interface (MPI) framework. We present a vertex-centric approach, allowing us to detect communities through local optimization. Furthermore, we enhance our baseline algorithm by incorporating three heuristics, which improve the algorithm’s performance significantly while maintaining the quality of the solutions. We demonstrate the efficiency of our algorithm by experimenting on several real-world large-scale networks with hundreds of millions of edges spanning diverse domains. Notably, DyG-DPCD achieves speedups between 25× and 30× for large networks that we experimented on using NERSC compute nodes. In conclusion, our algorithm outperforms the STINGER parallel re-agglomeration algorithm by 30×.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Distributed Multi-GPU Community Detection on Exascale Computing Platforms

Community detection is a fundamental operation in graph mining, and by uncovering hidden structures and patterns within complex systems it helps solve fundamental problems pertaining to social networks, such as information diffusion, epidemics, and recommender systems. Scaling graph algorithms for massive networks becomes challenging on modern distributed-memory multi-GPU (Graphics Processing Unit) systems due to limitations such as irregular memory access patterns, load imbalances, higher communication-computation ratios, and cross-platform support. We present a novel algorithm HiPDPL-GPU (distributed parallel Louvain) to address these challenges. We conduct experiments involving different partitioning techniques to achieve optimized performance of HiPDPL-GPU on the two largest supercomputers: Frontier and Summit. Remarkably, HiPDPL-GPU processes a graph with 4.2 billion edges in less than 3 minutes using 1024 GPUs. Qualitatively performance of HiPDPL-GPU is similar or better compared to other state-of-the-art CPU- and GPU-based implementations. While prior GPU implementations have predominantly employed CUDA, our first-of-its-kind implementation for community detection is cross-platform, accommodating both AMD and NVIDIA GPUs.

graph algorithms, high performance comptuing↗
DOE OSTI2026

Community detection robustness of graph neural networks

Graph neural networks (GNNs) are increasingly widely used for community detection in attributed networks. They combine structural topology with node attributes through message passing and pooling. However, their robustness or lack thereof with respect to different perturbations and targeted attacks in conjunction with community detection tasks is not well understood. To shed light on latent mechanisms behind GNN sensitivity on community detection tasks, we conduct a systematic computational evaluation of six widely adopted GNN architectures graph convolutional network, graph attention network, graph sample and aggregate (GraphSAGE), differentiable pooling (DiffPool), minimum cut pooling (MinCUT), and deep modularity networks (DMoN). The analysis covers three perturbation categories: node attribute manipulations, edge topology distortions, and adversarial attacks. We use element-centric similarity as the evaluation metric on synthetic benchmarks and real-world citation networks. Our findings indicate that supervised GNNs tend to achieve higher baseline accuracy, while unsupervised methods, particularly DMoN, maintain stronger resilience under targeted and adversarial perturbations. Furthermore, robustness appears to be strongly influenced by community strength, with well-defined communities reducing performance loss. Across all models, node attribute perturbations associated with targeted edge deletions and shifts in attribute distributions tend to cause the largest degradation in community recovery. These findings highlight important trade-offs between accuracy and robustness in GNN-based community detection and offer insights into selecting architectures resilient to noise and adversarial attacks.

Goel, Jaidev [Virginia Polytechnic Inst. and State↗
DOE OSTI2024

Exploring the Landscape of Distributed Graph Clustering on Leadership Supercomputers

The rapid growth of large-scale datasets in fields like biology and social networks has driven the need for advanced graph analytics techniques. Community detection, a fundamental task in graph analytics, identifies closely connected groups of nodes within a network, providing valuable insights across various disciplines. This study focuses on two classic community detection methods, the Louvain algorithm and Markov Clustering (MCL), and evaluates the performance of two prominent distributed community detection algorithms: HiPDPL-GPU, our prior implementation, and HipMCL. We conduct experiments on GPU-accelerated heterogeneous HPC systems, Summit and Frontier, to assess their performance under varying conditions. Our objective is to identify the strengths and weaknesses of these algorithms in terms of scalability, and quality of solutions. We evaluate these algorithms on a diverse set of 70+ networks spanning 13 domains, with sizes ranging up to 4.2 billion edges. Our results demonstrate that HiPDPL-GPU consistently outperforms HipMCL, especially for large-scale networks. HiPDPL-GPU achieves significantly faster runtimes (47x to 1439x), higher modularity scores, and improved scalability. These findings highlight HiPDPL-GPU as a promising solution for efficient and effective large-scale graph analytics in diverse application domains, and provide insights into the feasibility of using MCL-based approaches for certain application domains.

Community detection, graph algorithms↗
DOE OSTI2026

Simultaneous global and local clustering in multiplex networks with covariate information

Understanding both global and layer-specific group structures is useful for uncovering complex patterns in networks with multiple interaction types. In this work, we introduce a new model, the hierarchical multiplex stochastic blockmodel, which simultaneously detects communities within individual layers of a multiplex network while inferring a global node clustering across the layers. A stochastic blockmodel is assumed in each layer, with probabilities of layer-level group memberships determined by a node’s global group assignment. Our model uses a Bayesian framework, employing a probit stick-breaking process to construct node-specific mixing proportions over a set of shared Griffiths–Engen–McCloseky distributions. These proportions determine layer-level community assignment, allowing for an unknown and varying number of groups across layers, while incorporating nodal covariate information to inform the global clustering. We propose a scalable variational inference procedure with parallelisable updates for application to large networks. Extensive simulation studies demonstrate our model’s ability to accurately recover both global and layer-level clusters in complicated settings, and applications to real data showcase the model’s effectiveness in uncovering interesting latent network structure.

community detection↗
NASA NTRSDate not supplied

Constructing a Knowledge Graph & Applying Graph Algorithms to Draw Insights about GES-DISC Jira Tickets

In order to assess the complexities of Jira tickets created by NASA Goddard Earth Sciences Data and Information Services Center (GES-DISC), it was beneficial to create a knowledge graph. The knowledge graph receives ticket data through the Jira API. The creation of a knowledge graph will help to answer high-level questions about internal structure, knowledge gaps, and team organization within GES-DISC. To work towards this goal, the knowledge graph was constructed in adockerizedNeo4j graph database. Once the graph had been created, graph algorithms were applied to answer high-level questions, such as exploring the role of staff in relation to projects, which qualities of a ticket contribute to the formation of communities within the graph, etc. To answer these questions, centrality and community detection algorithms were applied using Cypher querying language. The analysis of the results of the algorithms indicated that, as expected, certain individuals were more connected to some projects, while others were serving as hub nodes between two or more projects. Similarly, specific keywords are more likely to increase a Jira ticket’s centrality in the graph. In terms of community detection, when tickets in a community have certain qualities, it is more probable for them to be grouped together. To best visualize which nodes had higher centrality scores or were grouped into certain communities, interactive graphs were created in Python using Plotly and Matplotlib. Ultimately, the project was successful in creating and deploying a knowledge graph to better understand the relationships between data in GES-DISC Jira tickets

Rebecca Lipton↗
NASA NTRS2022

Impacts of Climate Change on Global Food Trade Networks

Countries' reliance on global food trade networks implies that regionally different climate change impacts on crop yields will be transmitted across borders. This redistribution constitutes a significant challenge for climate adaptation planning and may affect how countries engage in cooperative action. This paper investigates the long-term (2070–2099) potential impacts of climate change on global food trade networks of three key crops: wheat, rice and maize. We propose a simple network model to project how climate change impacts on crop yields may be translated into changes in trade. Combining trade and climate impact data, our analysis proceeds in three steps. First, we use network community detection to analyse how the concentration of global production in present-day trade communities may become disrupted with climate change impacts. Second, we study how countries may change their network position following climate change impacts. Third, we study the total climate-induced change in production plus import within trade communities. Results indicate that the stability of food trade network structures compared to today differs between crops, and that countries' maize trade is least stable under climate change impacts. Results also project that threats to global food security may depend on production change in a few major global producers, and whether trade communities can balance production and import loss in some vulnerable countries. Overall, our model contributes a baseline analysis of cross-border climate impacts on food trade networks.

climate change↗
DOE OSTI2025

Toward computing bounds for Ramsey numbers using quantum annealing

Quantum annealing is a powerful tool for solving and approximating combinatorial optimization problems, such as graph partitioning, community detection, centrality, routing problems, and more. In this paper we explore the use of quantum annealing as a tool for use in exploring combinatorial mathematics research problems. We consider the monochromatic triangle problem and the Ramsey number problem, both examples of graph coloring. Conversion to quadratic unconstrained binary optimization (QUBO) form is required to run on quantum hardware. While the monochromatic triangle problem is quadratic by nature, the Ramsey number problem requires the use of order reduction methods for a quadratic formulation. The goal is to provide a method for producing special colorings of graphs which if successful would provide lower bounds for certain Ramsey numbers. We discuss implementations, limitations, and results when running on the D-Wave Advantage quantum annealer.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1997

Automated Test Systems for Toxic Vapor Detectors

The NASA Toxic Vapor Detection Laboratory (TVDL) at the Kennedy Space Center (KSC), Florida, has been using Personal Computer based Data Acquisition and Control Systems (PCDAS) for about nine years. These systems control the generation of toxic vapors of known concentrations under controlled conditions of temperature and humidity. The PCDAS also logs the test conditions and the test article responses in data files for analysis by standard spreadsheets or custom programs. The PCDAS was originally developed to perform standardized qualification and acceptance tests in a search for a commercial off-the-shelf (COTS) toxic vapor detector to replace the hydrazine detectors for the Space Shuttle launch pad. It has since become standard test equipment for the TVDL and is indispensable in producing calibration standards for the new hydrazine monitors at the 10 part per billion (ppb) level. The standard TVDL PCDAS can control two toxic vapor generators (TVG's) with three channels each and two flow/temperature/humidity (FIFH) controllers and it can record data from up to six toxic vapor detectors (TVD's) under test and can deliver flows from 5 to 50 liters per minute (L/m) at temperatures from near zero to 50 degrees Celsius (C) using an environmental chamber to maintain the sample temperature. The concentration range for toxic vapors depends on the permeation source installed in the TVG. The PCDAS can provide closed loop control of temperature and humidity to two sample vessels, typically one for zero gas and one for the standard gas. This is required at very low toxic vapor concentrations to minimize the time required to passivate the sample delivery system. Recently, there have been several requests for information about the PCDAS by other laboratories with similar needs, both on and off KSC. The purpose of this paper is to inform the toxic vapor detection community of the current status and planned upgrades to the automated testing of toxic vapor detectors at the Kennedy Space Center.

Mattson, C. B.↗
NASA NTRS1997

Automated Test Systems for Toxic Vapor Detectors

The NASA Toxic Vapor Detection Laboratory (TVDL) at the Kennedy Space Center (KSC), Florida, has been using Personal Computer based Data Acquisition and Control Systems (PCDAS) for about nine years. These systems control the generation of toxic vapors of known concentrations under controlled conditions of temperature and humidity. The PCDAS also logs the test conditions and the test article responses in data files for analysis by standard spreadsheets or custom programs. The PCDAS was originally developed to perform standardized qualification and acceptance tests in a search for a commercial off-the-shelf (COTS) toxic vapor detector to replace the hydrazine detectors for the Space Shuttle launch pad. It has since become standard test equipment for the TVDL and is indispensable in producing calibration standards for the new hydrazine monitors at the 10 part per billion (ppb) level. The standard TVDL PCDAS can control two toxic vapor generators (TVG's) with three channels each and two flow/ temperature / humidity (FTH) controllers and it can record data from up to six toxic vapor detectors (TVD's) under test and can deliver flows from 5 to 50 liters per minute (L/m) at temperatures from near zero to 50 degrees Celsius (C) using an environmental chamber to maintain the sample temperature. The concentration range for toxic vapors depends on the permeation source installed in the TVG. The PCDAS can provide closed loop control of temperature and humidity to two sample vessels, typically one for zero gas and one for the standard gas. This is required at very low toxic vapor concentrations to minimize the time required to passivate the sample delivery system. Recently, there have been several requests for information about the PCDAS by other laboratories with similar needs, both on and off KSC. The purpose of this paper is to inform the toxic vapor detection community of the current status and planned upgrades to the automated testing of toxic vapor detectors at the Kennedy Space Center.

Mattson, C. B.↗
NASA NTRS1985

Assessing LANDSAT TM and MSS Data for Detecting Submerged Plant Communities

The spectra, spacial, and radiometric characteristics of LANDSAT TM and MSS data for detecting and monitoring submerged plant communities were assessed. The following preliminary results focus upon the spectral aspects of the problem in which a submerged plant canopy is to be distinguished from a surrounding bottom of sand or mud. The effectiveness of an orbiting sensor in discriminating between submerged features and how strongly the bottom signal is attenuated by the water column. In optically shallow water the inherent contrast is the controlling factor. Thus, the optimum sensor band is that which correlates with the greatest inherent contrast between the submerged features. In optically deeper water, the optimum sensor band is that in which the bottom signal is attenuated the least.

Ackleson, S. G.↗
NASA NTRS1984

Parametric and Nonparametric Analysis of LANDSAT TM and MSS Imagery for Detecting Submerged Plant Communities

The spatial, spectral and radiometric characteristics of LANDSAT TM and MSS imagery for detecting submerged aquatic vegetation are assessed. The problem is approached from two perspectives; purely stochastic or nonparametric in a radiative sense and theoretical in which radiative transfer equations are used to predict upwelling radiance at satellite altitude. The spectral and radiometric aspects of the theoretical approach are addressed with which a submerged plant canopy is distinguished from a surrounding bottom of sand or mud.

Ackleson, S. G.↗
DOE OSTI2026

4th TDAMM Workshop White Paper

Time-Domain and Multi-Messenger Astrophysics (TDAMM) is entering a fundamentally new phase characterized by an unprecedented increase in the rate and diversity of astrophysical transient detections. The community is transitioning from a discovery-limited to a follow-up-limited era, driven by major investments across electromagnetic, gravitational-wave, and neutrino observatories. Upcoming facilities such as the Vera C. Rubin Observatory, the Nancy Grace Roman Space Telescope, and wide-field survey instruments will produce a deluge of time-domain alerts, reaching millions of events per night. Simultaneously, upgrades to the gravitational-wave network (LVK O5 and beyond) and neutrino observatories (IceCube Gen2) will significantly increase the detection rates of non-electromagnetic messengers. New high-energy missions and expansions of the InterPlanetary Network (IPN) will further enhance discovery capabilities across the gamma-ray and X-ray regimes. This convergence of capabilities represents a transformative opportunity: for the first time, the community will routinely detect rare and high-impact events across multiple messengers. However, the scientific return from these discoveries will depend critically on the ability to rapidly identify, prioritize, and coordinate follow-up observations across a heterogeneous and globally distributed set of facilities.

79 ASTRONOMY AND ASTROPHYSICS↗
NASA NTRS1972

Environmental applications activity at Marshall Space Flight Center

MSFC environmental applications demonstration projects have emphasized application of aerospace technology to community needs of southeastern U.S. Some of the typical projects underway are: hydrological parameter determination; land use surveys; agricultural stress detection; new community site surveys; pollution monitoring; urban transportation studies; and urban environmental quality.

Paludan, C. T. N.↗
DOE OSTI2024

Metatranscriptomic analysis reveals dissimilarity in viral community activity between an ice-free and ice-covered winter in Lake Erie

Winter is a relatively under-studied season in freshwater ecology. The paucity of wintertime surveys has led to a lack of knowledge regarding microbial community activity during the winter in Lake Erie, a North American Great Lake. Viruses shape microbial communities and regulate biogeochemical cycles by acting as top-down controls, yet very few efforts have been made to examine active virus populations during the winter in Lake Erie. Furthermore, climate change-driven declines in seasonal ice cover have been shown to influence microbial community structure, but no studies have compared viral community activity between different ice cover conditions. We surveyed surface water metatranscriptomes for viral hallmark genes as a proxy for active virus populations and compared activity metrics between ice-covered and ice-free conditions from two sampled winters. Transcriptionally active viral communities were detected in both winters, spanning diverse phylogenetic clades of putative bacteriophage (Caudoviricetes), giant viruses (Nucleocytoviricota, or NCLDV), and RNA viruses (Orthornavirae). However, viral community activity metrics revealed pronounced differences between the ice-covered and ice-free winters. Viral community composition was distinct between winters and viral hallmark gene richness was reduced in the ice-covered relative to the ice-free conditions. In addition, the observed differences in viral communities correlated with microbial community activity metrics. Overall, these findings contribute to our understanding of the viral populations that are active during the winter in Lake Erie and suggest that viral community activity may be associated with ice cover extent.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2024

Satellite-Based Assessment of Rocket Launch and Coastal Change Impacts on Cape Canaveral Barrier Island, Florida, USA

The Cape Canaveral Barrier Island, home to the National Aeronautics and Space Administration (NASA)’s Kennedy Space Center and the United States (U.S.) Space Force’s Cape Canaveral Space Force Station, is situated in a unique ecological transition zone that supports diverse wildlife. This study evaluates the recent changes in vegetation cover (2016–2023) and dune elevation (2007–2017) within the Cape Canaveral Barrier Island using high-resolution optical satellite and light detection and ranging (LiDAR) data. The study period was chosen to depict the time period of a recent increase in rocket launches. The study objectives include assessing changes in vegetation communities, identifying detectable impacts of liquid propellant launches on nearby vegetation, and evaluating dune elevation and tide level shifts near launchpads. The results indicate vegetation cover changes, including mangrove expansion in wetland areas and the conversion of coastal strands to denser scrubs and hardwood forests, which were likely influenced by mild winters and fire management. While detectable impacts of rocket launches on nearby vegetation were observed, they were less severe than those caused by solid rocket motors. Compounding challenges, such as rising tide levels, beach erosion, and wetland loss, potentially threaten the resilience of launch operations and the surrounding habitats. The volume and scale of launches continue to increase, and a balance between space exploration and ecological conservation is required in this biodiverse region. This study focuses on the assessment of barrier islands’ shorelines.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS2020

Searching for Hyperspectral Optical Proxies to Aid Chesapeake Bay Resource Managers in the Detection of Poor Water Quality

Shellfish aquaculture is a growing industry in the Chesapeake Bay. As population grows near the coast, extreme weather events cause a greater volume of pollutant runoff from impervious surfaces and agricultural lands. Resource managers who monitor shellfish beds need reliable information on a variety of water quality indicators at higher frequency than is possible through field monitoring programs and at a higher level of detail than current satellite products can provide. Although many factors causing degraded water quality that can impact human health are not currently discernable by traditional multispectral techniques, hyperspectral imagery offers a new opportunity to detect phytoplankton communities associated with harmful algal blooms and biotoxin production. Together with resource managers in their routine monitoring of sites around the bay from small boats, we have been exploring remotely sensed optical proxies for the detection of harmful algal blooms and sewage. Early warning by remote sensing could guide sampling and improve the efficiency of shellfish bed closures, ultimately improving health outcomes for humans and animals. An extensive network of routine sampling by Chesapeake Bay Program managers makes this is an ideal location to develop and test future satellite data products to support management decisions. Next generation hyperspectral measurements from the future Plankton Aerosol Cloud ocean Ecosystem (PACE) mission at nearly daily frequency, combined with the potential of higher spatial resolution from the Surface Biology and Geology (SBG) observing system recommended in the recent Decadal Survey, along with high frequency observations from the newly selected Geostationary Littoral Imaging and Monitoring Radiometer (GLIMR) Earth Venture Instrument make this a critical time for defining the needs of the aquaculture and resource management community to save lives, time, and money.

Schollaert Uz, Stephanie↗
DOE OSTI2024

Evaluating disease surveillance strategies for early outbreak detection in contact networks with varying community structure

Disease surveillance systems allow public health agencies to respond to emerging diseases before they become widespread. Developing such systems requires identifying optimal ways to monitor in the context of an epidemic outbreak; this problem is known as sensor selection. Contact networks represent the dynamics of interaction in a population and are used to model how a disease spreads in a population and to explore strategies of sensor selection. We evaluated five sensor selection strategies on their ability to provide an early warning of a COVID-like outbreak in synthetic contact networks encapsulated in four network scenarios. Three of these scenarios assessed different aspects of community structure. The fourth scenario employed a contact network representing the population and interactions of 6.8 million people in New York City, constructed from an agent-based simulation using census and transportation data. This scenario exemplifies how sensor selection strategies may perform in a real-world, urban context. Our findings suggest that the choice of the optimal strategy depends heavily on the community structure of the network. Strategies that select highly connected nodes or maximize network coverage are the optimal surveillance strategy for outbreak detection in many network community structures. However, a naive implementation of these strategies may fail to provide an early warning at all—including in the New York City scenario. Moreover, these methods are impractical for real-world use as they require knowledge of the underlying contact network. Instead, a selection strategy that starts with a set of random nodes and then performs a random walk through a chain of neighbors reliably provides early warnings without requiring prior knowledge of the network. We find this method, called “random chain”, to be the most pragmatic for implementation in a real-world disease surveillance context.

60 APPLIED LIFE SCIENCES↗