Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Hierarchical clustering”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 127 records · Page 7
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

Redshift inference from the combination of galaxy colours and clustering in a hierarchical Bayesian model – Application to realistic N -body simulations

ABSTRACT Photometric galaxy surveys constitute a powerful cosmological probe but rely on the accurate characterization of their redshift distributions using only broad-band imaging, and can be very sensitive to incomplete or biased priors used for redshift calibration. A hierarchical Bayesian model has recently been developed to estimate those from the robust combination of prior information, photometry of single galaxies, and the information contained in the galaxy clustering against a well-characterized tracer population. In this work, we extend the method so that it can be applied to real data, developing some necessary new extensions to it, especially in the treatment of galaxy clustering information, and we test it on realistic simulations. After marginalizing over the mapping between the clustering estimator and the actual density distribution of the sample galaxies, and using prior information from a small patch of the survey, we find the incorporation of clustering information with photo-z’s tightens the redshift posteriors and overcomes biases in the prior that mimic those happening in spectroscopic samples. The method presented here uses all the information at hand to reduce prior biases and incompleteness. Even in cases where we artificially bias the spectroscopic sample to induce a shift in mean redshift of $\Delta \bar{z} \approx 0.05,$ the final biases in the posterior are $\Delta \bar{z} \lesssim 0.003.$ This robustness to flaws in the redshift prior or training samples would constitute a milestone for the control of redshift systematic uncertainties in future weak lensing analyses.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2022

Hierarchical Division and Clustering of Group Structures

When using multigroup neutron transport models, it is important to choose a suitable group structure due to the impact on runtime and accuracy. Group structures at LANL have been largely pared down to a handful of commonly used group structures, such as the LANL-30 and LANL-70 group structures. By identifying the key boundaries for a given problem, more insight can be gained into why the commonly used group structures are accurate and when they are inaccurate. This work focuses on using hierarchical division and agglomeration methods to identify the most critical boundaries in these group structures, using the ICSBEP HEU-MET-FAST-001-001 benchmark as a case study.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2021

Propagation of priors for more accurate and efficient spectroscopic functional fits and their application to ferroelectric hysteresis

Multi-dimensional spectral-imaging is a mainstay of the scanning probe and electron microscopies, micro-Raman, and various forms of chemical imaging. In many cases, individual spectra can be fit to a specific functional form, with the model parameter maps, providing direct insight into material properties. Since spectra are often acquired across a spatial grid of points, spatially adjacent spectra are likely to be similar to one another; yet, this fact is almost never used when considering parameter estimation for functional fits. On datasets tried here, we show that by utilizing proximal information, whether it be in the spatial or spectral domains, it is possible to improve the reliability and increase the speed of such functional fits by ~2-3x, as compared to random priors. We explore and compare three distinct new methods: (1) spatially averaging neighborhood spectra, and propagating priors based on functional fits to the averaged case, (2) hierarchical clustering-based methods where spectra are grouped hierarchically based on response, with the priors propagated progressively down the hierarchy, and (3) regular clustering without hierarchical methods with priors propagated from fits to cluster means. Our results highlight that utilizing spatial and spectral neighborhood information is often critical for accurate parameter estimation in noisy environments, which we show for ferroelectric hysteresis loops acquired on a prototypical PbTiO3 thin film with piezoresponse spectroscopy. This method is general and applicable to any spatially measured spectra where functional forms are available. Examples include exploring the superconducting gap with tunneling spectroscopy, using the Dynes formula, or current-voltage curve fits in conductive atomic force microscopy mapping. Here we explore the problem for ferroelectric hysteresis, which, given its large parameter space, constitutes a more difficult task than, for example, fitting current-voltage curves with a Schottky emission formula.

42 ENGINEERING↗
DOE OSTI2022

Defensive Islanding to Enhance the Resilience of Distribution Systems Against Cyber-Induced Failures

The extensive integration of communication, computation, and control technologies into cyber-physical power systems (CPPSs) has increased the vulnerabilities of CPPSs to cyberattacks. This calls for developing solutions that assess and reduce the impacts of cyber-induced failures on CPPSs. This paper proposes a defensive islanding strategy to isolate impacted parts of the CPPS and form self-sufficient islanded grids with an objective of minimum load curtailment. The defensive islanding aims to split a power system into smaller grids to improve its resilience against a potential extreme event. A clustering approach that leverages the hierarchical spectral clustering method is utilized for the optimal defensive islanding. The proposed approach captures the fragility behavior and loading conditions of power system components due to cyber-induced failures. A graphical-based coupling framework is used to map the impacts of cyber failures into operation of power system components. The proposed method is demonstrated on a modified 33-node distribution feeder system integrated with distributed energy resources. The amount of load curtailment and radiality constraints have been used to evaluate the performance of the proposed clustering strategies. The results show the capability of the proposed algorithm to create islands considering the cyber-induced failures for enhanced resilience.

cyber-induced failures↗
DOE OSTI2025

Scalable edge clustering of dynamic graphs via weighted line graphs

Timestamped relational datasets consisting of records (or connections) between pairs of entities are ubiquitous in network science. For applications like peer-to-peer communication, email, various social network interactions, and computer network security, it is useful to organize these records into groups based on how and when they are occurring. Weighted line graphs offer a natural way to model how records are related in such datasets but for large real-world graph topologies, building and utilizing the line graph is prohibitively expensive. Here, we present the framework to cluster the edges of a dynamic graph via the associated line graph that contains two major contributions. The first is a method to work with the line graph implicitly and the second is a distributed scale implementation of an agglomerative hierarchical graph clustering algorithm. We outline a novel hierarchical dynamic graph edge clustering approach that efficiently breaks massive relational datasets into small sets of edges containing events at various timescales. This is in stark contrast to traditional graph clustering algorithms that prioritize highly connected (clique-like) community structures. Our approach relies on constructing a sufficient subgraph of a weighted line graph and applying a hierarchical agglomerative clustering. This approach is related to scalable techniques from spatial clustering, nonlinear-dimension reduction, topological data analysis, and draws particular inspiration from HDBSCAN. As an edge clustering, this method yields an overlapping node clustering. Our algorithm is parallelizable and we demonstrate efficient clustering of a billion-scale, real-world dynamic graph into small edge sets that correlate in topology and time. The entire clustering process for a graph with tens of billions of edges takes just a few minutes of run time on 256 nodes of a distributed compute environment. We argue how the output of the edge clustering is useful for a multitude of data visualization and powerful machine learning tasks, both involving the original massive dynamic graph data and metadata associated with the nodes and edges. Finally, we describe how this approach can be extended to dynamic hypergraphs and dynamic graphs/hypergraphs with unstructured data living on vertices and edges.

Data Analysis↗
NASA NTRS2004

Hierarchical Velocity Structure in the Core of Abell 2597

We present XMM-Newton RGS and EPIC data of the putative cooling flow cluster Abell 2597. Velocities of the low-ionization emission lines in the spectrum are blue shifted with respect to the high-ionization lines by 1320 (sup +660) (sub -210) kilometers per second, which is consistent with the difference in the two peaks of the galaxy velocity distribution and may be the signature of bulk turbulence, infall, rotation or damped oscillation in the cluster. A hierarchical velocity structure such as this could be the direct result of galaxy mergers in the cluster core, or the injection of power into the cluster gas from a central engine. The uniform X-ray morphology of the cluster, the absence of fine scale temperature structure and the random distribution of the the galaxy positions, independent of velocity, suggests that our line of sight is close to the direction of motion. These results have strong implications for cooling flow models of the cluster Abell 2597. They give impetus to those models which account for the observed temperature structure of some clusters using mergers instead of cooling flows.

Still, Martin↗
NASA NTRS2002

Recursive Hierarchical Image Segmentation by Region Growing and Constrained Spectral Clustering

This paper describes an algorithm for hierarchical image segmentation (referred to as HSEG) and its recursive formulation (referred to as RHSEG). The HSEG algorithm is a hybrid of region growing and constrained spectral clustering that produces a hierarchical set of image segmentations based on detected convergence points. In the main, HSEG employs the hierarchical stepwise optimization (HS WO) approach to region growing, which seeks to produce segmentations that are more optimized than those produced by more classic approaches to region growing. In addition, HSEG optionally interjects between HSWO region growing iterations merges between spatially non-adjacent regions (i.e., spectrally based merging or clustering) constrained by a threshold derived from the previous HSWO region growing iteration. While the addition of constrained spectral clustering improves the segmentation results, especially for larger images, it also significantly increases HSEG's computational requirements. To counteract this, a computationally efficient recursive, divide-and-conquer, implementation of HSEG (RHSEG) has been devised and is described herein. Included in this description is special code that is required to avoid processing artifacts caused by RHSEG s recursive subdivision of the image data. Implementations for single processor and for multiple processor computer systems are described. Results with Landsat TM data are included comparing HSEG with classic region growing. Finally, an application to image information mining and knowledge discovery is discussed.

Tilton, James C.↗
DOE OSTI2025

Web-based wide-area monitoring platform for ringdown and clustering analytics in power systems

This paper introduces an open-source research platform for monitoring the Mexican interconnected power grid, allowing real-time processing and information extraction of the grid’s dynamic condition. Moreover, the platform is a Python-based development that embeds different ringdown and clustering analytics tools. In the case of ringdown analysis, the modal information can be extracted using some of the most known algorithms, i.e., Prony analysis, eigensystem realization algorithm (ERA), and matrix pencil (MP). For clustering analysis, the coherent behaviour of generator and non-generator buses is provided by applying recent state-of-the-art techniques such as affinity propagation, K-means, hierarchical agglomerative clustering, and typicality data analysis. The results of up to 93 PMUs show that this open-source platform suits researchers’ and engineers’ power system dynamic analysis requirements.

Clustering↗
DOE OSTI2026

Mesh-based multiphysics coupling acceleration for fusion neutronics through clustering for fusion blanket applications

Accurate modeling of particle transport within fusion blankets is essential for predicting performance metrics such as heat deposition and the tritium breeding ratio (TBR). However, high-fidelity coupling of thermal fluids from computational fluid dynamics (CFD) to neutronics simulations often incurs significant computational costs due to the complexity of surface intersection calculations in Monte Carlo codes. This paper presents an accelerated multiphysics coupling method for neutronics that utilizes hierarchical agglomerative clustering to map complex material property distributions to a neutronics model. Implemented within the fusion reactor design and assessment (FREDA) framework, the method leverages existing Python packages to automate the creation of clustered geometries for OpenMC. The approach is demonstrated on a sector model of an ARC-class tokamak with an immersion molten salt blanket, and an simple geometry with varying isotopic concentrations. Results show that the clustering method significantly reduces computational burden without compromising fidelity, providing a foundation for agile iteration of neutronics simulations involving multiple coupled material properties.

Bae, Jin Whan [ORNL] (ORCID:0000000326548907)↗
DOE OSTI2026

Chronic obstructive pulmonary disease among former United States Department of Energy workers: comorbidities and lung function changes

Chronic Obstructive Pulmonary Disease (COPD) is a major cause of morbidity and mortality in the United States and is frequently associated with multiple comorbidities which lead to poor COPD outcomes in the general population. However, little is known regarding COPD comorbidities in occupational cohorts whose exposure experiences could result in differences in comorbidities compared to the general population. These differences may also be important for assessing COPD outcomes such as lung function changes or decline. Therefore, the objectives of this study were to: (1) identify and describe clusters of COPD comorbidities among Department of Energy (DOE) former workers; (2) assess if the attributes of the identified clusters differ from those identified among the general population based on the published literature, and (3) identify predictors of lung function changes and decline among DOE former workers.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2023

Estimating the electric vehicle charging demand of multi-unit dwelling residents in the United States

Abstract Early battery electric vehicle (EV) adopters can access home chargers for reliable charging. As the EV market grows, residents of multi-unit dwellings (MUDs) may face barriers in owning EVs and charging them without garage or parking availability. To investigate the mechanisms that can bridge existing disparities in home charging and station deployment, we characterized the travel behavior of MUD residents and estimated their EV residential charging demand. This study classifies the travel patterns of MUD residents by fusing trip diary data from the National Household Travel Survey and housing features from the American Housing Survey. A hierarchical agglomerative clustering method was used to cluster apartment complex residents’ travel profiles, considering attributes such as dwell time, daily vehicle miles traveled (VMT), income, and their residences’ US census division. We propose a charging decision model to determine the charging station placement demand in MUDs and the charging energy volume expected to be consumed, assuming that MUD drivers universally operate EVs in urban communities. Numerical experiments were conducted to gain insight into the charging demand of MUD residents in the US. We found that charging availability is indispensable for households that set out to meet 80% state of charge by the end of the day. When maintaining a 20% comfortable state of charge the entire day, the higher the VMT are, the greater the share of charging demand and the greater the energy use in MUD chargers. The upper-income group requires a greater share of MUD charging and greater daily kWh charged because of more VMT.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2022

A deep learning approach to real-time HIV outbreak detection using genetic data

Pathogen genomic sequence data are increasingly made available for epidemiological monitoring. A main interest is to identify and assess the potential of infectious disease outbreaks. While popular methods to analyze sequence data often involve phylogenetic tree inference, they are vulnerable to errors from recombination and impose a high computational cost, making it difficult to obtain real-time results when the number of sequences is in or above the thousands. Here, we propose an alternative strategy to outbreak detection using genomic data based on deep learning methods developed for image classification. The key idea is to use a pairwise genetic distance matrix calculated from viral sequences as an image, and develop convolutional neutral network (CNN) models to classify areas of the images that show signatures of active outbreak, leading to identification of subsets of sequences taken from an active outbreak. We showed that our method is efficient in finding HIV-1 outbreaks with R0 ≥ 2.5, and overall a specificity exceeding 98% and sensitivity better than 92%. We validated our approach using data from HIV-1 CRF01 in Europe, containing both endemic sequences and a well-known dual outbreak in intravenous drug users. Our model accurately identified known outbreak sequences in the background of slower spreading HIV. Importantly, we detected both outbreaks early on, before they were over, implying that had this method been applied in real-time as data became available, one would have been able to intervene and possibly prevent the extent of these outbreaks. This approach is scalable to processing hundreds of thousands of sequences, making it useful for current and future real-time epidemiological investigations, including public health monitoring using large databases and especially for rapid outbreak identification.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2022

AICCA: AI-Driven Cloud Classification Atlas

Clouds play an important role in the Earth’s energy budget, and their behavior is one of the largest uncertainties in future climate projections. Satellite observations should help in understanding cloud responses, but decades and petabytes of multispectral cloud imagery have to date received only limited use. This study describes a new analysis approach that reduces the dimensionality of satellite cloud observations by grouping them via a novel automated, unsupervised cloud classification technique based on a convolutional autoencoder, an artificial intelligence (AI) method good at identifying patterns in spatial data. Our technique combines a rotation-invariant autoencoder and hierarchical agglomerative clustering to generate cloud clusters that capture meaningful distinctions among cloud textures, using only raw multispectral imagery as input. Cloud classes are therefore defined based on spectral properties and spatial textures without reliance on location, time/season, derived physical properties, or pre-designated class definitions. We use this approach to generate a unique new cloud dataset, the AI-driven cloud classification atlas (AICCA), which clusters 22 years of ocean images from the Moderate Resolution Imaging Spectroradiometer (MODIS) on NASA’s Aqua and Terra instruments—198 million patches, each roughly 100 km × 100 km (128 × 128 pixels)—into 42 AI-generated cloud classes, a number determined via a newly-developed stability protocol that we use to maximize richness of information while ensuring stable groupings of patches. AICCA thereby translates 801 TB of satellite images into 54.2 GB of class labels and cloud top and optical properties, a reduction by a factor of 15,000. The 42 AICCA classes produce meaningful spatio-temporal and physical distinctions and capture a greater variety of cloud types than do the nine International Satellite Cloud Climatology Project (ISCCP) categories—for example, multiple textures in the stratocumulus decks along the West coasts of North and South America. We conclude that our methodology has explanatory power, capturing regionally unique cloud classes and providing rich but tractable information for global analysis. AICCA delivers the information from multi-spectral images in a compact form, enables data-driven diagnosis of patterns of cloud organization, provides insight into cloud evolution on timescales of hours to decades, and helps democratize climate research by facilitating access to core data.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1987

Star formation in molecular clouds - Observation and theory

Star-formation (SF) processes occurring on the scale of giant molecular clouds (10 to the 6th solar masses and 10 to the 20th cm) or smaller are discussed, reviewing the results of recent theoretical and observational investigations. Topics examined include the origin of stellar masses; bimodal SF; initial mass functions; binary stars, bound clusters, and hierarchical fragmentation; and the efficiency of SF. The properties of molecular clouds and the origin of substructures in molecular clumps are explored in detail, and consideration is given to gravitational collapse and protostars, bipolar outflows from young stellar objects, visible young stellar objects, and the implications for binary-star and planetary-system formation.

Shu, Frank H.↗
NASA NTRS1993

A statistical analysis of mesoscale rainfall as a random cascade

The paper considers the random cascade theory for spatial rainfall. Particular attention was given to the following four areas: (1) the relationship of the random cascade theory of rainfall to the simple scaling and the hierarchical cluster-point-process theories, (2) the mathematical foundations for some of the formalisms commonly applied in the develpment of statistical cascade theory, (3) the empirical evidence for a random cascade theory of rainfall, and (4) the way of using data for making estimates of parameters and for making statistical inference within this theoretical framework. An analysis of space-time rainfall data is presented. Cascade simulations are carried out to provide a comparison with methods of analysis that are applied to the rainfall data.

Gupta, Vijay K.↗
NASA NTRS1995

Constraints on cold dark matter theories from observations of massive x-ray-luminous clusters of galaxies at high redshift

During the course of a gravitational lensing survey of distant, X-ray selected Einstein Observatory Extended Medium Sensitivity Survey (EMSS) clusters of galaxies, we have studied six X-ray-luminous (L(sub x) greater than 5 x 10(exp 44)(h(sub 50)(exp -2))ergs/sec) clusters at redshifts exceeding z = 0.5. All of these clusters are apparently massive. In addition to their high X-ray luminosity, two of the clusters at z approximately 0.6 exhibit gravitationally lensed arcs. Furthermore, the highest redshift cluster in our sample, MS 1054-0321 at z = 0.826, is both extremely X-ray luminous (L(sub 0.3-3.5keV)=9.3 x 10(exp 44)(h(sub 50)(exp -2))ergs/sec) and exceedingly rich with an optical richness comparable to an Abell Richness Class 4 cluster. In this Letter, we discuss the cosmological implications of the very existence of these clusters for hierarchical structure formation theories such as standard Omega = 1 CDM (cold dark matter), hybrid Omega = 1 C + HDM (hot dark matter), and flat, low-density Lambda + CDM models.

Luppino, G. A.↗
DOE OSTI2021

A Kinematic Perspective on the Formation Process of the Stellar Groups in the Rosette Nebula

Stellar kinematics is a powerful tool for understanding the formation process of stellar associations. Here, we present a kinematic study of the young stellar population in the Rosette nebula using recent Gaia data and high-resolution spectra. We first isolate member candidates using the published mid-infrared photometric data and the list of X-ray sources. A total of 403 stars with similar parallaxes and proper motions are finally selected as members. The spatial distribution of the members shows that this star-forming region is highly substructured. The young open cluster NGC 2244 in the center of the nebula has a pattern of radial expansion and rotation. We discuss its implication on the cluster formation, e.g., monolithic cold collapse or hierarchical assembly. On the other hand, we also investigate three groups located around the border of the H ii bubble. The western group seems to be spatially correlated with the adjacent gas structure, but their kinematics is not associated with that of the gas. The southern group does not show any systematic motion relative to NGC 2244. These two groups might be spontaneously formed in filaments of a turbulent cloud. The eastern group is spatially and kinematically associated with the gas pillar receding away from NGC 2244. This group might be formed by feedback from massive stars in NGC 2244. Our results suggest that the stellar population in the Rosette Nebula may form through three different processes: the expansion of stellar clusters, hierarchical star formation in turbulent clouds, and feedback-driven star formation.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2026

Weak Lensing Mass Calibration of the ACT DR5 Galaxy Clusters with the DES Year 3 Weak Lensing Data

We use weak gravitational lensing measurements from Year 3 Dark Energy Survey data to calibrate the masses of 443 galaxy clusters selected via the Sunyaev-Zel'dovich effect from Atacama Cosmology Telescope Data Release 5 maps of the cosmic microwave background. We incorporate redshift and SZ measurements for individual clusters into a hierarchical model for the stacked lensing signals and perform Bayesian analyses to constrain the hydrostatic mass bias of the clusters. Our treatment of systematic uncertainties includes a prescription for measuring and accounting for the weak lensing boost factor, consideration of a miscentering effect, as well as marginalization over uncertainties in the source galaxy photometric redshift distributions and shear calibration. The resultant constraints on the normalization of the mass-observable relation have a precision of approximately 7%, with the mean WL halo mass of M $_{500c}$ = 5.4 × 10$^{14}$ M $_{⊙}$. We measure the bias between the true cluster mass and the mass estimated from the SZ signal based on an X-ray-calibrated scaling relation assuming hydrostatic equilibrium, to be 1 - b = 0.74$^{+0.06}$ $_{-0.05}$ over the full sample. When splitting the clusters into high (z = 0.43-0.70) and low (z = 0.15-0.43) redshift bins, we measure 1 - b = 0.58$^{+0.06}$ $_{-0.05}$ and 0.81$^{+0.08}$ $_{-0.06}$, respectively. When introducing additional freedom in redshift and mass to the hydrostatic bias model, we find that 1 - b decreases with redshift (with the power law of -1.8$^{+0.5}$ $_{-0.6}$, 99.95% confidence), consistent with findings from other recent studies, while we do not find any significant trend in mass. We also demonstrate that our result is robust against various systematics such as a scale cut, priors on baryonic and miscentering parameters, and degree of scatter in mass-observable relation. The weak-lensing mass calibration presented in this study will be a useful tool for using the ACT clusters as probes of astrophysics, and as a step towards using their abundance as a cosmological probe.

Shin, T. [Carnegie Mellon U.] (ORCID:0000000263895↗