Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Hierarchical clustering”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 145 records · Page 8
All sourcesNASA NTRSDOE OSTI
NASA NTRS1986

Statistics of voids in hierarchical universes

As one alternative to the N-point galaxy correlation function statistics, the distribution of holes or the probability that a volume of given size and shape be empty of galaxies can be considered. The probability of voids resulting from a variety of hierarchical patterns of clustering is considered, and these are compared with the results of numerical simulations and with observations. A scaling relation required by the hierarchical pattern of higher order correlation functions is seen to be obeyed in the simulations, and the numerical results show a clear difference between neutrino models and cold-particle models; voids are more likely in neutrino universes. Observational data do not yet distinguish but are close to being able to distinguish between models.

Fry, J. N.↗
NASA NTRS2006

Utilizing Hierarchical Segmentation to Generate Water and Snow Masks to Facilitate Monitoring Change with Remotely Sensed Image Data

The hierarchical segmentation (HSEG) algorithm is a hybrid of hierarchical step-wise optimization and constrained spectral clustering that produces a hierarchical set of image segmentations. This segmentation hierarchy organizes image data in a manner that makes the image's information content more accessible for analysis by enabling region-based analysis. This paper discusses data analysis with HSEG and describes several measures of region characteristics that may be useful analyzing segmentation hierarchies for various applications. Segmentation hierarchy analysis for generating landwater and snow/ice masks from MODIS (Moderate Resolution Imaging Spectroradiometer) data was demonstrated and compared with the corresponding MODIS standard products. The masks based on HSEG segmentation hierarchies compare very favorably to the MODIS standard products. Further, the HSEG based landwater mask was specifically tailored to the MODIS data and the HSEG snow/ice mask did not require the setting of a critical threshold as required in the production of the corresponding MODIS standard product.

Tilton, James C.↗
NASA NTRS2008

Parallel Implementation of the Recursive Approximation of an Unsupervised Hierarchical Segmentation Algorithm

The hierarchical image segmentation algorithm (referred to as HSEG) is a hybrid of hierarchical step-wise optimization (HSWO) and constrained spectral clustering that produces a hierarchical set of image segmentations. HSWO is an iterative approach to region grooving segmentation in which the optimal image segmentation is found at N(sub R) regions, given a segmentation at N(sub R+1) regions. HSEG's addition of constrained spectral clustering makes it a computationally intensive algorithm, for all but, the smallest of images. To counteract this, a computationally efficient recursive approximation of HSEG (called RHSEG) has been devised. Further improvements in processing speed are obtained through a parallel implementation of RHSEG. This chapter describes this parallel implementation and demonstrates its computational efficiency on a Landsat Thematic Mapper test scene.

Tilton, James C.↗
NASA NTRS1989

Decision net, directed graph, and neural net processing of imaging spectrometer data

A decision-net solution involving a novel hierarchical classifier and a set of multiple directed graphs, as well as a neural-net solution, are respectively presented for large-class problem and mixture problem treatments of imaging spectrometer data. The clustering method for hierarchical classifier design, when used with multiple directed graphs, yields an efficient decision net. New directed-graph rules for reducing local maxima as well as the number of perturbations required, and the new starting-node rules for extending the reachability and reducing the search time of the graphs, are noted to yield superior results, as indicated by an illustrative 500-class imaging spectrometer problem.

Casasent, David↗
NASA NTRS1981

Einstein imaging observations of clusters with a bimodal mass distribution

Einstein imaging observations of four X-ray clusters of galaxies characterized by a double X-ray surface brightness and thus mass distribution are presented. The clusters A98, A115, A1750 and SC 0627-54 were found to exhibit two enhancements in their X-ray surface brightness distributions in observations made with the Einstein Imaging Proportional Counter. Calculations of the probability that the clusters represent chance superpositions indicate that the double clusters are physically associated. The radial distributions of the components are inconsistent with those of single point sources, and have been used to derive cluster luminosities which are typical of rich clusters. Masses of the subclusters are also found to be typical of bound and virialized clusters with gas contributing 10%. Within the framework of the hierarchical theory of galactic clustering, the double clusters are suggested to represent an intermediate evolutionary stage before the merger of subclusters into a relaxed Coma-type cluster.

Forman, W.↗
DOE OSTI2021

Significant DBSCAN+: Statistically Robust Density-based Clustering

Cluster detection is important and widely used in a variety of applications, including public health, public safety, transportation, and so on. Given a collection of data points, we aim to detect density-connected spatial clusters with varying geometric shapes and densities, under the constraint that the clusters are statistically significant. The problem is challenging, because many societal applications and domain science studies have low tolerance for spurious results, and clusters may have arbitrary shapes and varying densities. As a classical topic in data mining and learning, a myriad of techniques have been developed to detect clusters with both varying shapes and densities (e.g., density-based, hierarchical, spectral, or deep clustering methods). However, the vast majority of these techniques do not consider statistical rigor and are susceptible to detecting spurious clusters formed as a result of natural randomness. On the other hand, scan statistic approaches explicitly control the rate of spurious results, but they typically assume a single “hotspot” of over-density and many rely on further assumptions such as a tessellated input space. To unite the strengths of both lines of work, we propose a statistically robust formulation of a multi-scale DBSCAN, namely Significant DBSCAN+, to identify significant clusters that are density connected. As we will show, incorporation of statistical rigor is a powerful mechanism that allows the new Significant DBSCAN+ to outperform state-of-the-art clustering techniques in various scenarios. We also propose computational enhancements to speed-up the proposed approach. Experiment results show that Significant DBSCAN+ can simultaneously improve the success rate of true cluster detection (e.g., 10–20% increases in absolute F1 scores) and substantially reduce the rate of spurious results (e.g., from thousands/hundreds of spurious detections to none or just a few across 100 datasets), and the acceleration methods can improve the efficiency for both clustered and non-clustered data.

Computer Science↗
DOE OSTI2026

Evaluating lightweight unsupervised online IDS for masquerade attacks in CAN

Vehicular controller area networks (CANs) are susceptible to masquerade attacks by malicious adversaries. In masquerade attacks, adversaries silence a targeted ID and then send malicious frames with forged content at the expected timing of benign frames. As masquerade attacks could seriously harm vehicle functionality and are the stealthiest attacks to detect in CAN, recent work has devoted attention to compare frameworks for detecting masquerade attacks in CAN. However, most existing works report offline evaluations using CAN logs already collected using simulations that do not comply with the domain’s real-time constraints. Here we contribute to advance the state of the art by presenting a comparative evaluation of four different non-deep learning (DL)-based unsupervised online intrusion detection systems (IDS) for masquerade attacks in CAN. Our approach differs from existing comparative evaluations in that we analyze the effect of controlling streaming data conditions in a sliding window setting. In doing so, we use realistic masquerade attacks being replayed from the ROAD dataset. We show that although evaluated IDS are not effective at detecting every attack type, the method that relies on detecting changes in the hierarchical structure of clusters of time series produces the best results at the expense of higher computational overhead. We discuss limitations, open challenges, and how the evaluated methods can be used for practical unsupervised online CAN IDS for masquerade attacks.

Anomaly detection↗
DOE OSTI2022

Weak lensing trispectrum and Kurt-spectra

We introduce two kurt-spectra to probe fourth-order statistics of weak lensing convergence maps. Using state-of-the-art numerical simulations, we study the shapes of these kurt-spectra as a function of source redshifts and smoothing angular scales. We employ a pseudo-Cℓ approach to estimate the spectra from realistic convergence maps in the presence of an observational mask and noise for stage-IV large-scale structure surveys. We compare these results against theoretical predictions calculated using the FFTLog formalism, and find that a simple nonlinear clustering model — the hierarchical ansatz — can reproduce the numerical trends for the kurt-spectra in the nonlinear regime. In addition, we provide estimators for beyond fourth-order spectra where no definitive analytical results are available, and present corresponding results from numerical simulations.

79 ASTRONOMY AND ASTROPHYSICS↗
NASA NTRS1994

Gravitational collapse of small-scale structure as the origin of the Lyman-alpha forest

If gravitational clustering is a hierarchical process, the present large-scale structure of the galaxy distribution implies that structures on smaller scales must have formed at high redshift. We simulate the formation of small-scale structure (average cell mass: (delta) bar m(sub b) = 10(sup 4.2) solar mass) and the evolution of photoionized gas, in the specific case of the CDM + Lambda model. The photoionized gas has a natural minimal scale of collapse, the Jeans scale (m(sub b,J) is approximately equal to 10(exp 9) solar mass). We find that low column density (N(sub HI) is less than or equal to 10(exp 14)/sq cm) lines originate in regions resembling Zel'dovich pancakes, where gas with overdensities in the range 3-30 is enclosed by two shocks but is typically reexpanding at approximately the Hubble velocity. However, higher column density lines stem from more overdense regions where the shocked gas is cooling. We show that this model can probably account for the observed number of lines, their distribution in column density and b-parameters, as well as the cloud physical sizes as observed in gravitationally lensed quasars. We find a redshift evolution that is too steep; however, this may be due to insufficient dynamical range in the simulation or because the specific model is incorrect. The model predicts that high signal-to-noise observations should find systematic deviations from Voigt profiles, mainly in the form of broad wings in the line profiles, and that a fluctuating Gunn-Peterson effect will be detected, which can be modeled as a superposition of weak lines with a wide range of b-parameters.

Cen, Renyue↗
DOE OSTI2023

Efficient Network Partitioning: Application for Decentralized State Estimation in Power Distribution Grids: Preprint

Increase in the proliferation of DERs requires real-time situational awareness for efficient grid operations. State estimation plays an important role for real time control and management of the power grid. As the sensing infrastructure grows, aggregating and handling high volumes of data at a centralized location is extremely difficult. To address this challenge, this paper first proposes a novel and efficient hierarchical spectral clustering-based network partition algorithm followed by a decentralized compressive sensing (DCS) based state estimation. The applicability of the proposed network partitioning algorithm is tested on IEEE-123 bus, IEEE-8500 node, and a 6204-node distribution network. The results shows that the proposed approach efficiently divides the network into multiple sub-networks with the minimum edge connections among the neighbors. Then, we perform DCS-based state estimation on the 6204-node distribution network after dividing the network into 18 optimal partitions. Simulation results show that DCS-based state estimation recovers the system states with high accuracy and low complexity.

ADMM↗
NASA NTRS2021

Atacama Cosmology Telescope measurements of a large sample of candidates from the Massive and Distant Clusters of WISE Survey: Sunyaev-Zeldovich effect confirmation of MaDCoWS candidates using ACT

Context. Galaxy clusters are an important tool for cosmology, and their detection and characterization are key goals for current and future surveys. Using data from the Wide-field Infrared Survey Explorer (WISE), the Massive and Distant Clusters of WISE Survey (MaDCoWS) located 2839 significant galaxy overdensities at redshifts 0.7 . z . 1.5, which included extensive follow-up imaging from the Spitzer Space Telescope to determine cluster richnesses. Concurrently, the Atacama Cosmology Telescope (ACT) has produced large area millimeter-wave maps in three frequency bands along with a large catalog of Sunyaev-Zeldovich (SZ)-selected clusters as part of its Data Release 5 (DR5). Aims. We aim to verify and characterize MaDCoWS clusters using measurements of, or limits on, their thermal SZ effect signatures. We also use these detections to establish the scaling relation between SZ mass and the MaDCoWS-defined richness. Methods. Using the maps and cluster catalog from DR5, we explore the scaling between SZ mass and cluster richness. We do this by comparing cataloged detections and extracting individual and stacked SZ signals from the MaDCoWS cluster locations. We use complementary radio survey data from the Very Large Array, submillimeter data from Herschel, and ACT 224 GHz data to assess the impact of contaminating sources on the SZ signals from both ACT and MaDCoWS clusters. We use a hierarchical Bayesian model to fit the mass-richness scaling relation, allowing for clusters to be drawn from two populations: one, a Gaussian centered on the mass-richness relation, and the other, a Gaussian centered on zero SZ signal. Results. We find that MaDCoWS clusters have submillimeter contamination that is consistent with a gray-body spectrum, while the ACT clusters are consistent with no submillimeter emission on average. Additionally, the intrinsic radio intensities of ACT clusters are lower than those of MaDCoWS clusters, even when the ACT clusters are restricted to the same redshift range as the MaDCoWS clusters. We find the best-fit ACT SZ mass versus MaDCoWS richness scaling relation has a slope of p1 = 1.84+0.15 −0.14, where the slope is defined as M ∝ λ p1 15 and λ15 is the richness. We also find that the ACT SZ signals for a significant fraction (∼57%) of the MaDCoWS sample can statistically be described as being drawn from a noise-like distribution, indicating that the candidates are possibly dominated by low-mass and unvirialized systems that are below the mass limit of the ACT sample. Further, we note that a large portion of the optically confirmed ACT clusters located in the same volume of the sky as MaDCoWS are not selected by MaDCoWS, indicating that the MaDCoWS sample is not complete with respect to SZ selection. Finally, we find that the radio loud fraction of MaDCoWS clusters increases with richness, while we find no evidence that the submillimeter emission of the MaDCoWS clusters evolves with richness. Conclusions. We conclude that the original MaDCoWS selection function is not well defined and, as such, reiterate the MaDCoWS collaboration’s recommendation that the sample is suited for probing cluster and galaxy evolution, but not cosmological analyses. We find a best-fit mass-richness relation slope that agrees with the published MaDCoWS preliminary results. Additionally, we find that while the approximate level of infill of the ACT and MaDCoWS cluster SZ signals (1–2%) is subdominant to other sources of uncertainty for current generation experiments, characterizing and removing this bias will be critical for next-generation experiments hoping to constrain cluster masses at the sub-percent level.

large↗
DOE OSTI2024

Search for interacting galaxy clusters from SDSS DR-17 employing optimized friends-of-friends algorithm and multimessenger tracers

ABSTRACT In the theoretical framework of hierarchical structure formation, galaxy clusters evolve through continuous accretion and mergers of substructures. Cosmological simulations have revealed the best picture of the universe as a 3D filamentary network of dark-matter distribution called the cosmic web. Galaxy clusters are found to form at the nodes of this network and are the regions of high merging activity. Such mergers being highly energetic, contain a wealth of information about the dynamical evolution of structures in the Universe. Observational validation of this scenario needs a colossal effort to identify numerous events from all-sky surveys. Therefore, such efforts are sparse in literature and tend to focus on individual systems. In this work, we present an improved search algorithm for identifying interacting galaxy clusters and have successfully produced a comprehensive list of systems from SDSS DR-17. By proposing a set of physically motivated criteria, we classified these interacting clusters into two broad classes, ‘merging’ and ‘pre-merging/postmerging’ systems. Interestingly, as predicted by simulations, we found that most cases show cluster interaction along the prominent cosmic filaments of galaxy distribution (i.e. the proxy for dark matter filaments), with the most violent ones at their nodes. Moreover, we traced the imprint of interactions through multiband signatures, such as diffuse cluster emissions in radio or X-rays. Although we could not find direct evidence of diffuse emission from connecting filaments and ridges; our catalogue of interacting clusters will ease locating such faintest emissions as data from sensitive telescopes such as eROSITA or SKA, becomes accessible.

Oak, Tejas↗
NASA NTRS2003

Hierarchical Image Segmentation of Remotely Sensed Data using Massively Parallel GNU-LINUX Software

A hierarchical set of image segmentations is a set of several image segmentations of the same image at different levels of detail in which the segmentations at coarser levels of detail can be produced from simple merges of regions at finer levels of detail. In [1], Tilton, et a1 describes an approach for producing hierarchical segmentations (called HSEG) and gave a progress report on exploiting these hierarchical segmentations for image information mining. The HSEG algorithm is a hybrid of region growing and constrained spectral clustering that produces a hierarchical set of image segmentations based on detected convergence points. In the main, HSEG employs the hierarchical stepwise optimization (HSWO) approach to region growing, which was described as early as 1989 by Beaulieu and Goldberg. The HSWO approach seeks to produce segmentations that are more optimized than those produced by more classic approaches to region growing (e.g. Horowitz and T. Pavlidis, [3]). In addition, HSEG optionally interjects between HSWO region growing iterations, merges between spatially non-adjacent regions (i.e., spectrally based merging or clustering) constrained by a threshold derived from the previous HSWO region growing iteration. While the addition of constrained spectral clustering improves the utility of the segmentation results, especially for larger images, it also significantly increases HSEG s computational requirements. To counteract this, a computationally efficient recursive, divide-and-conquer, implementation of HSEG (RHSEG) was devised, which includes special code to avoid processing artifacts caused by RHSEG s recursive subdivision of the image data. The recursive nature of RHSEG makes for a straightforward parallel implementation. This paper describes the HSEG algorithm, its recursive formulation (referred to as RHSEG), and the implementation of RHSEG using massively parallel GNU-LINUX software. Results with Landsat TM data are included comparing RHSEG with classic region growing.

Tilton, James C.↗
NASA NTRS1984

Globular cluster origin of X-ray bursters

X-ray bursters and galactic bulge X-ray sources, or the most luminous X-ray sources in the Galaxy, are reasonably well constrained in their basic nature but not in their origin. It is suggested they may all have been produced by tidal capture in high density cores of globular clusters, which have now largely been disrupted by tidal stripping and shocking in the galactic plane. General arguments are presented for cluster disruption by the possible ring of giant molecular clouds in the Galaxy. Tests of the cluster disruption hypothesis are in progress and preliminary results are summarized here. The G-K star 'companions' previously noted for at least four bursters have spectra (in the two cases observed) consistent with metal-rich cluster giants. Several possibilities are discussed, including the formation of hierarchical triples in the dissolving cluster or in the galactic plane.

Grindlay, J. E.↗
NASA NTRS1983

Filamentary galaxy clustering - A mapping algorithm

A simple and objective algorithm is presented which not only accurately identifies the filamentary structures in the Shane-Wirtanen galaxy count catalog, but also finds a set of visually less impressive filaments in a static hierarchical model of the clustering conducted by Soneira and Peebles (1978). The statistical properties of the elements in the model, while very similar to those in the data, show a significant excess of long and bright filaments in the data relative to the model. Two possible interpretations of these results are presented and discussed.

Gott, J. R., III↗
DOE OSTI2021

Hierarchical median narrow band for level set segmentation of cervical cell nuclei

This paper presents a novel hierarchical nuclei segmentation algorithm for isolated and overlapping cervical cells based on a narrow band level set implementation. Our method applies a new multiscale analysis algorithm to estimate the number of clusters in each image region containing cells, which turns into the input to a narrow band level set algorithm. We assess the nuclei segmentation results on three public cervical cell image databases. Overall, our segmentation method outperformed six state-of-the-art methods concerning the number of correctly segmented nuclei and the Dice coefficient reached values equal to or higher than 0.90. We also carried out classification experiments using features extracted from our segmentation results and the proposed pipeline achieved the highest average accuracy values equal to 0.89 and 0.77 for two-class and three-class problems, respectively. Furthermore, these results demonstrated the suitability of the proposed segmentation algorithm to integrate decision support systems for cervical cell screening.

47 OTHER INSTRUMENTATION↗
DOE OSTI2023

Interrupted DNA and Slow Silver Cluster Luminescence

A DNA–silver cluster conjugate is a hierarchical chromophore with a partly reduced silver core embedded within the DNA nucleobases that are covalently linked by the phosphodiester backbone. Specific sites within a polymeric DNA can be targeted to spectrally tune the silver cluster. Here, the repeated (C 2 A) 6 strand is interrupted with a thymine, and the resulting (C 2 A) 2 -T-(C 2 A) 4 forms only Ag 10 6+ , a chromophore with both prompt (~1 ns) green and sustained (~10 2 μs) red luminescence. Thymine is an inert placeholder that can be removed, and the two fragments (C 2 A) 2 and (C 2 A) 4 also produce the same Ag 10 6+ adduct. In relation to (C 2 A) 2 T(C 2 A) 4 , the (C 2 A) 2 + (C 2 A) 4 pair is distinguished because the red Ag 10 6+ luminescence is ~6× lower, relaxes ~30% faster, and is quenched ~2× faster with O 2 . These differences suggest that a specific break in the phosphodiester backbone can regulate how a contiguous vs broken scaffold wraps and better protects its cluster adduct.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
NASA NTRS1988

On hierarchical solutions to the BBGKY hierarchy

It is thought that the gravitational clustering of galaxies in the universe may approach a scale-invariant, hierarchical form in the small separation, large-clustering regime. Past attempts to solve the Born-Bogoliubov-Green-Kirkwood-Yvon (BBGKY) hierarchy in this regime have assumed a certain separable hierarchical form for the higher order correlation functions of galaxies in phase space. It is shown here that such separable solutions to the BBGKY equations must satisfy the condition that the clustered component of the solution has cluster-cluster correlations equal to galaxy-galaxy correlations to all orders. The solutions also admit the presence of an arbitrary unclustered component, which plays no dyamical role in the large-clustering regime. These results are a particular property of the specific separable model assumed for the correlation functions in phase space, not an intrinsic property of spatially hierarchical solutions to the BBGKY hierarchy. The observed distribution of galaxies does not satisfy the required conditions. The disagreement between theory and observation may be traced, at least in part, to initial conditions which, if Gaussian, already have cluster correlations greater than galaxy correlations.

Hamilton, A. J. S.↗