Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Community detection”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

DyG-DPCD: A Distributed Parallel Community Detection Algorithm for Large-Scale Dynamic Graphs

Dynamic (Temporal) graphs capture the valuable evolution of real-world systems, from the continuously evolving patterns of social interactions and genetic pathways to the dynamic fluctuations of economic forces. Detecting communities for such evolving networks poses unique challenges. Detecting and analyzing the evolution of communities within dynamic graphs unlocks valuable insights into the underlying structural and temporal patterns of real-world systems. However, the sheer volume of modern graph data and the inherent complexity of the temporal dimension pose significant challenges to scalable community detection algorithms. Addressing this gap, our work explores the limited landscape of scalable distributed-memory parallel methods specifically designed for dynamic network community detection. We propose a novel parallel algorithm, DyG-DPCD (Dynamic Graph Distributed Parallel Community Detection), to detect communities in dynamic networks using the Message Passing Interface (MPI) framework. We present a vertex-centric approach, allowing us to detect communities through local optimization. Furthermore, we enhance our baseline algorithm by incorporating three heuristics, which improve the algorithm’s performance significantly while maintaining the quality of the solutions. We demonstrate the efficiency of our algorithm by experimenting on several real-world large-scale networks with hundreds of millions of edges spanning diverse domains. Notably, DyG-DPCD achieves speedups between 25× and 30× for large networks that we experimented on using NERSC compute nodes. In conclusion, our algorithm outperforms the STINGER parallel re-agglomeration algorithm by 30×.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Distributed Multi-GPU Community Detection on Exascale Computing Platforms

Community detection is a fundamental operation in graph mining, and by uncovering hidden structures and patterns within complex systems it helps solve fundamental problems pertaining to social networks, such as information diffusion, epidemics, and recommender systems. Scaling graph algorithms for massive networks becomes challenging on modern distributed-memory multi-GPU (Graphics Processing Unit) systems due to limitations such as irregular memory access patterns, load imbalances, higher communication-computation ratios, and cross-platform support. We present a novel algorithm HiPDPL-GPU (distributed parallel Louvain) to address these challenges. We conduct experiments involving different partitioning techniques to achieve optimized performance of HiPDPL-GPU on the two largest supercomputers: Frontier and Summit. Remarkably, HiPDPL-GPU processes a graph with 4.2 billion edges in less than 3 minutes using 1024 GPUs. Qualitatively performance of HiPDPL-GPU is similar or better compared to other state-of-the-art CPU- and GPU-based implementations. While prior GPU implementations have predominantly employed CUDA, our first-of-its-kind implementation for community detection is cross-platform, accommodating both AMD and NVIDIA GPUs.

graph algorithms, high performance comptuing↗
DOE OSTI2026

Community detection robustness of graph neural networks

Graph neural networks (GNNs) are increasingly widely used for community detection in attributed networks. They combine structural topology with node attributes through message passing and pooling. However, their robustness or lack thereof with respect to different perturbations and targeted attacks in conjunction with community detection tasks is not well understood. To shed light on latent mechanisms behind GNN sensitivity on community detection tasks, we conduct a systematic computational evaluation of six widely adopted GNN architectures graph convolutional network, graph attention network, graph sample and aggregate (GraphSAGE), differentiable pooling (DiffPool), minimum cut pooling (MinCUT), and deep modularity networks (DMoN). The analysis covers three perturbation categories: node attribute manipulations, edge topology distortions, and adversarial attacks. We use element-centric similarity as the evaluation metric on synthetic benchmarks and real-world citation networks. Our findings indicate that supervised GNNs tend to achieve higher baseline accuracy, while unsupervised methods, particularly DMoN, maintain stronger resilience under targeted and adversarial perturbations. Furthermore, robustness appears to be strongly influenced by community strength, with well-defined communities reducing performance loss. Across all models, node attribute perturbations associated with targeted edge deletions and shifts in attribute distributions tend to cause the largest degradation in community recovery. These findings highlight important trade-offs between accuracy and robustness in GNN-based community detection and offer insights into selecting architectures resilient to noise and adversarial attacks.

Goel, Jaidev [Virginia Polytechnic Inst. and State↗
DOE OSTI2024

Exploring the Landscape of Distributed Graph Clustering on Leadership Supercomputers

The rapid growth of large-scale datasets in fields like biology and social networks has driven the need for advanced graph analytics techniques. Community detection, a fundamental task in graph analytics, identifies closely connected groups of nodes within a network, providing valuable insights across various disciplines. This study focuses on two classic community detection methods, the Louvain algorithm and Markov Clustering (MCL), and evaluates the performance of two prominent distributed community detection algorithms: HiPDPL-GPU, our prior implementation, and HipMCL. We conduct experiments on GPU-accelerated heterogeneous HPC systems, Summit and Frontier, to assess their performance under varying conditions. Our objective is to identify the strengths and weaknesses of these algorithms in terms of scalability, and quality of solutions. We evaluate these algorithms on a diverse set of 70+ networks spanning 13 domains, with sizes ranging up to 4.2 billion edges. Our results demonstrate that HiPDPL-GPU consistently outperforms HipMCL, especially for large-scale networks. HiPDPL-GPU achieves significantly faster runtimes (47x to 1439x), higher modularity scores, and improved scalability. These findings highlight HiPDPL-GPU as a promising solution for efficient and effective large-scale graph analytics in diverse application domains, and provide insights into the feasibility of using MCL-based approaches for certain application domains.

Community detection, graph algorithms↗
DOE OSTI2026

Simultaneous global and local clustering in multiplex networks with covariate information

Understanding both global and layer-specific group structures is useful for uncovering complex patterns in networks with multiple interaction types. In this work, we introduce a new model, the hierarchical multiplex stochastic blockmodel, which simultaneously detects communities within individual layers of a multiplex network while inferring a global node clustering across the layers. A stochastic blockmodel is assumed in each layer, with probabilities of layer-level group memberships determined by a node’s global group assignment. Our model uses a Bayesian framework, employing a probit stick-breaking process to construct node-specific mixing proportions over a set of shared Griffiths–Engen–McCloseky distributions. These proportions determine layer-level community assignment, allowing for an unknown and varying number of groups across layers, while incorporating nodal covariate information to inform the global clustering. We propose a scalable variational inference procedure with parallelisable updates for application to large networks. Extensive simulation studies demonstrate our model’s ability to accurately recover both global and layer-level clusters in complicated settings, and applications to real data showcase the model’s effectiveness in uncovering interesting latent network structure.

community detection↗
DOE OSTI2025

Toward computing bounds for Ramsey numbers using quantum annealing

Quantum annealing is a powerful tool for solving and approximating combinatorial optimization problems, such as graph partitioning, community detection, centrality, routing problems, and more. In this paper we explore the use of quantum annealing as a tool for use in exploring combinatorial mathematics research problems. We consider the monochromatic triangle problem and the Ramsey number problem, both examples of graph coloring. Conversion to quadratic unconstrained binary optimization (QUBO) form is required to run on quantum hardware. While the monochromatic triangle problem is quadratic by nature, the Ramsey number problem requires the use of order reduction methods for a quadratic formulation. The goal is to provide a method for producing special colorings of graphs which if successful would provide lower bounds for certain Ramsey numbers. We discuss implementations, limitations, and results when running on the D-Wave Advantage quantum annealer.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

4th TDAMM Workshop White Paper

Time-Domain and Multi-Messenger Astrophysics (TDAMM) is entering a fundamentally new phase characterized by an unprecedented increase in the rate and diversity of astrophysical transient detections. The community is transitioning from a discovery-limited to a follow-up-limited era, driven by major investments across electromagnetic, gravitational-wave, and neutrino observatories. Upcoming facilities such as the Vera C. Rubin Observatory, the Nancy Grace Roman Space Telescope, and wide-field survey instruments will produce a deluge of time-domain alerts, reaching millions of events per night. Simultaneously, upgrades to the gravitational-wave network (LVK O5 and beyond) and neutrino observatories (IceCube Gen2) will significantly increase the detection rates of non-electromagnetic messengers. New high-energy missions and expansions of the InterPlanetary Network (IPN) will further enhance discovery capabilities across the gamma-ray and X-ray regimes. This convergence of capabilities represents a transformative opportunity: for the first time, the community will routinely detect rare and high-impact events across multiple messengers. However, the scientific return from these discoveries will depend critically on the ability to rapidly identify, prioritize, and coordinate follow-up observations across a heterogeneous and globally distributed set of facilities.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2024

Metatranscriptomic analysis reveals dissimilarity in viral community activity between an ice-free and ice-covered winter in Lake Erie

Winter is a relatively under-studied season in freshwater ecology. The paucity of wintertime surveys has led to a lack of knowledge regarding microbial community activity during the winter in Lake Erie, a North American Great Lake. Viruses shape microbial communities and regulate biogeochemical cycles by acting as top-down controls, yet very few efforts have been made to examine active virus populations during the winter in Lake Erie. Furthermore, climate change-driven declines in seasonal ice cover have been shown to influence microbial community structure, but no studies have compared viral community activity between different ice cover conditions. We surveyed surface water metatranscriptomes for viral hallmark genes as a proxy for active virus populations and compared activity metrics between ice-covered and ice-free conditions from two sampled winters. Transcriptionally active viral communities were detected in both winters, spanning diverse phylogenetic clades of putative bacteriophage (Caudoviricetes), giant viruses (Nucleocytoviricota, or NCLDV), and RNA viruses (Orthornavirae). However, viral community activity metrics revealed pronounced differences between the ice-covered and ice-free winters. Viral community composition was distinct between winters and viral hallmark gene richness was reduced in the ice-covered relative to the ice-free conditions. In addition, the observed differences in viral communities correlated with microbial community activity metrics. Overall, these findings contribute to our understanding of the viral populations that are active during the winter in Lake Erie and suggest that viral community activity may be associated with ice cover extent.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2024

Satellite-Based Assessment of Rocket Launch and Coastal Change Impacts on Cape Canaveral Barrier Island, Florida, USA

The Cape Canaveral Barrier Island, home to the National Aeronautics and Space Administration (NASA)’s Kennedy Space Center and the United States (U.S.) Space Force’s Cape Canaveral Space Force Station, is situated in a unique ecological transition zone that supports diverse wildlife. This study evaluates the recent changes in vegetation cover (2016–2023) and dune elevation (2007–2017) within the Cape Canaveral Barrier Island using high-resolution optical satellite and light detection and ranging (LiDAR) data. The study period was chosen to depict the time period of a recent increase in rocket launches. The study objectives include assessing changes in vegetation communities, identifying detectable impacts of liquid propellant launches on nearby vegetation, and evaluating dune elevation and tide level shifts near launchpads. The results indicate vegetation cover changes, including mangrove expansion in wetland areas and the conversion of coastal strands to denser scrubs and hardwood forests, which were likely influenced by mild winters and fire management. While detectable impacts of rocket launches on nearby vegetation were observed, they were less severe than those caused by solid rocket motors. Compounding challenges, such as rising tide levels, beach erosion, and wetland loss, potentially threaten the resilience of launch operations and the surrounding habitats. The volume and scale of launches continue to increase, and a balance between space exploration and ecological conservation is required in this biodiverse region. This study focuses on the assessment of barrier islands’ shorelines.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Evaluating disease surveillance strategies for early outbreak detection in contact networks with varying community structure

Disease surveillance systems allow public health agencies to respond to emerging diseases before they become widespread. Developing such systems requires identifying optimal ways to monitor in the context of an epidemic outbreak; this problem is known as sensor selection. Contact networks represent the dynamics of interaction in a population and are used to model how a disease spreads in a population and to explore strategies of sensor selection. We evaluated five sensor selection strategies on their ability to provide an early warning of a COVID-like outbreak in synthetic contact networks encapsulated in four network scenarios. Three of these scenarios assessed different aspects of community structure. The fourth scenario employed a contact network representing the population and interactions of 6.8 million people in New York City, constructed from an agent-based simulation using census and transportation data. This scenario exemplifies how sensor selection strategies may perform in a real-world, urban context. Our findings suggest that the choice of the optimal strategy depends heavily on the community structure of the network. Strategies that select highly connected nodes or maximize network coverage are the optimal surveillance strategy for outbreak detection in many network community structures. However, a naive implementation of these strategies may fail to provide an early warning at all—including in the New York City scenario. Moreover, these methods are impractical for real-world use as they require knowledge of the underlying contact network. Instead, a selection strategy that starts with a set of random nodes and then performs a random walk through a chain of neighbors reliably provides early warnings without requiring prior knowledge of the network. We find this method, called “random chain”, to be the most pragmatic for implementation in a real-world disease surveillance context.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2025

Inter-Kingdom Viral Interactions

Please cite as : Josué A. Rodríguez-Ramos, Amy E. Zimmerman, Ruonan Wu, Sheryl Bell, Trinidad Alfaro, Kirsten Hofmockel, William C. Nelson. 2025. Inter-Kingdom Viral Interactions. [Data Set] PNNL DataHub. This data is published under a CC0 license. The authors encourage data reuse and request attribution by referencing the above citations for the data package and associated manuscript. Deciphering viral ecology in soils is challenging due to their high physiochemical and community complexity. To enhance detection of sub-communities of DNA and RNA viruses, we applied fractionation approaches to soils collected across a moisture gradient from a grassland field experiment. Analyses included metagenomics and metatranscriptomics of size-fractionated extracellular viruses (i.e., DNA and RNA viromes), metagenomics of bacteria/archaea- or eukaryote-enriched samples, and whole soil metatranscriptomes with rRNA-depletion or polyadenylation enrichment. While RNA virome and whole soil RNA methods captured similar viral diversity, RNA viromes identified longer, higher-quality genomes. Further, we showed that significantly more DNA viruses were active in higher moisture than lower moisture samples, whereas responses by overall diversity vary by genome type (DNA versus RNA genomes). Finally, we demonstrate the power of fractionation approaches for identifying distinct viral communities that infect unique hosts, which has significant implications for ecological investigations, particularly related to interkingdom interactions.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2024

Specialized protist communities on mycorrhizal fungal hyphae

Abstract Arbuscular mycorrhizal (AM) fungi not only play a crucial role in acquiring nutrients for plants but also serve as a habitat for soil microbes. Recent studies observed that AM fungal hyphae are colonized by specific bacterial communities. However, so far it has not been explored whether fungal hyphae and mycorrhizal networks also harbor specific communities of protists, a key group of microbes in the soil microbiome. Here, we characterized protist communities in soil in a compartment with plant roots and on hyphae collected from hyphal compartments without plant roots. We detected specific protist communities on fungal hyphae. Fourteen protistan amplicon sequences variants (ASVs) were significantly associated with fungal hyphae, half of which belonged to the Cercozoa group. This research, for the first-time detected specific protist ASVs directly associated with abundant AM fungus hyphae, highlighting the complexity of the hyphal food web.

Mycology↗
DOE OSTI2024

Point-of-use filtration units as drinking water distribution system sentinels

Abstract Municipal drinking water distribution systems (DWDSs) and associated premise plumbing (PP) systems are vulnerable to proliferation of opportunistic pathogens, even when chemical disinfection residuals are present, thus presenting a public health risk. Monitoring the structure of microbial communities of drinking water is challenging because of limited continuous access to faucets, pipes, and storage tanks. We propose a scalable household sampling method, which uses spent activated carbon and reverse osmosis (RO) membrane point-of-use (POU) filters to evaluate mid- to long-term occurrence of microorganisms in PP systems that are relevant to consumer exposure. As a proof of concept, POU filter microbiomes were collected from four different locations and analyzed with 16S rRNA gene amplicon sequencing. The analyses revealed distinct microbial communities, with occasional detection of potential pathogens. The findings highlight the importance of local, and if possible, continuous monitoring within and across distribution systems. The continuous operation of POU filters offers an advantage in capturing species that may be missed by instantaneous sampling methods. We suggest that water utilities, public institutions, and regulatory agencies take advantage of end-of-life POU filters for microbial monitoring. This approach can be easily implemented to ensure drinking water safety, especially from microbes of emerging concerns; e.g., pathogenic Legionella and Mycobacterium species.

42 ENGINEERING↗
DOE OSTI2025

Common practices for quantifying methane emissions from plumes detected by remote sensing

This document provides a set of community-accepted practices for quantifying methane emissions based on plumes detected via spectroscopic remote sensing. Its primary goal is to promote consistency in the generation, validation, reporting, and quality assessment of methane emission estimates derived from remote sensing radiances. Developed by subject matter experts with deep experience across all stages of the measurement process, this guidance reflects a critical evaluation of current methodologies and highlights key practices needed to produce reliable, interoperable, and traceable products. The focus is specifically on methane emissions quantified from distinct plumes originating from localized sources, rather than diffuse emissions spread over large regions, which are beyond the scope of this work. This document is intended to serve both data producers and users. For producers, it offers a framework for aligning with field-recognized standards to ensure their outputs meet rigorous quality and transparency criteria. For users, it provides a reference to assess dataset fitness-for-purpose by highlighting essential metadata, assumptions, and methodological choices that underpin emission estimates. By fostering a shared understanding of best practices, this work aims to enhance comparability, confidence, and utility of remotely sensed methane emission products.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

Longitudinal seroprevalence of Crimean-Congo hemorrhagic fever virus in Southern Uganda

Crimean-Congo hemorrhagic fever (CCHF) is a tick-borne disease endemic to many regions of Africa, the Middle East, Southeast Asia and the Balkans. Caused by the CCHF virus (CCHFV), CCHF has been a recognized cause of illness in Uganda since the 1950s and recently, more intensive surveillance suggests CCHFV is widely endemic within the country. Most surveillance has been focused on the Ugandan cattle corridor due to the risk of CCHFV exposure associated with livestock practices. Here we evaluated the seroprevalence of CCHFV in several Southern Ugandan communities outside the cattle corridor combined with longitudinal sample sets to measure the immune response to CCHFV for up to a decade. Interestingly, across three community types, agrarian, trading and fishing, we detected CCHFV seroprevalence in all three but found the highest seroprevalence in fishing communities. We also measured consistent CCHFV-specific antibody responses for up to a decade. Our findings support the conclusion that CCHFV is widely endemic in Uganda and highlight that additional communities may be at risk for CCHFV exposure.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2026

Extracellular DNA Alters Detection of Subtle Bacterial Responses to Soil Rewetting

Microbial communities are often characterized using DNA-based sequencing, but these approaches also capture extracellular DNA (exDNA) released from dead cells, potentially altering inference about microbial responses to environmental change. This may be especially important during pulse disturbances, such as soil drying–rewetting, which can increase microbial mortality and transient necromass pools. We assessed whether exDNA altered inference about bacterial responses to drying–rewetting (an 80 mm simulated rainfall event following a 28-day drought) in conventionally tilled corn and perennial switchgrass soils. We quantified bacterial abundance (16 S rRNA gene copies), alpha diversity, and community composition in paired soil samples with exDNA included (+ exDNA) and in samples treated with propidium monoazide (PMAxx) to reduce amplification of exDNA (− exDNA). At our level of replication (n = 4), PMAxx treatment did not significantly alter overall temporal response patterns (i.e., no significant main effect of DNA treatment or DNA × time interaction). However, PMAxx treatment increased sensitivity to detect some pairwise temporal changes in bacterial abundance and community composition in corn soils following rewetting. exDNA pools were proportionally highest immediately after rewetting in corn soils, suggesting transient extracellular DNA may contribute to masking during disturbance recovery. In contrast, PMAxx treatment had comparatively small effects in switchgrass soils, which exhibited weaker temporal responses overall. Inclusion of exDNA also changed which taxa appeared most responsive to rewetting. Together, our results suggest that exDNA does not uniformly bias soil microbial inference, but may reduce detectability of subtle disturbance-driven shifts in certain soils. Future studies should advance knowledge of microbial turnover and necromass dynamics, particularly using multiple complementary methods, to help predict when exDNA is most likely to influence ecological inference.

drying-rewetting↗
DOE OSTI2025

Cryogenic optical beam steering for superconducting device calibration

We have developed a calibration system based on a micro-electromechanical systems (MEMS) mirror that is capable of delivering an optical beam over a wavelength range of 180 -- 2000 nm (0.62 -- 6.89 eV) in a sub-Kelvin environment. This portable, integrated system can steer the beam over a $\sim$3 cm $\times$ 3 cm area on the surface of any sensor with a precision of $\sim$100 $μ$m, enabling characterization of device response as a function of position. This fills a critical need in the landscape of calibration tools for sub-Kelvin devices, including those used for dark matter detection and quantum computing. These communities have a shared goal of understanding the impact of ionizing radiation on device performance, which can be pursued with our system. This paper describes the design of the first-generation calibration system and the results from successfully testing its performance at room temperature and 20 mK.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2024

LumaCam: a novel class of position-sensitive event mode particle detectors using scintillator screens

A new type of position-sensitive detectors is gaining attention in the neutron community. They are scintillator based detectors that detect the scintillation light on an individual photon basis via an image intensifier and a fast image sensor. Their readout operates in event mode i.e. it produces information about individual neutron interactions, reconstructed from the sensor data, thus enabling to achieve superior spatial and temporal resolutions compared to regular detectors. Although the development of current detectors is focused on neutrons, the concept is also applicable to the detection of other particles such as high-energy photons. This document provides a description on how these detectors are built, how they operate, and what their characteristics are. An example of a detector implementation based on a Timepix3 chip is described to illustrate the detector concept. This includes a detailed description of the algorithm that reconstructs the neutron interactions from the sensor data, one of the core components that sets it apart from established scintillator-based imaging detectors. Energy-resolved epithermal neutron radiography was performed at the ISIS EMMA beamline with this detector, illustrating some of the fundamental differences in the data that can be produced with the new type of detector compared to more established types of scintillator based neutron detectors. The term LumaCam is proposed to refer to this new class of position-sensitive event-mode detectors.

47 OTHER INSTRUMENTATION↗