SEARCH · Engineering Papers

Results for “data clustering”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 379 records · Page 21

All sources NASA NTRS DOE OSTI

DOE OSTI2018

Mining data in a dynamic PRA framework

Computational (also known as Dynamic) Probabilistic Risk Assessment (PRA) methods employs system simulation codes coupled with stochastic analysis tools in order to determine probabilities of certain outcomes such as system failure. In contrast to classical PRA methods (i.e., Event-Tree and Fault-Tree) in which timing and sequencing of events is set by the analyst, accident progression is dictated by the system control logic and its interaction with system temporal evolution. Due to the nature of the problem, Simulation-Based PRA methods require large amount of computationally expensive simulation runs. In addition, these methods usually generate a large number of simulation runs (database storage may be on the order of gigabytes or higher). In this paper we investigate and apply several methods and algorithms to analyze these large amounts of time-dependent data. Furthermore, the scope of this article is to present a broad overview of methods and algorithms that can be used to improve data quality and to analyze and extract information from large data sets containing time dependent data. In this context, “extracting information” means constructing input-output correlations, finding commonalities, and identifying outliers.

97 MATHEMATICS AND COMPUTING↗

NASA NTRS1989

Remote sensing of soils in the eastern Palouse region with Landsat Thematic Mapper

Soils of the Palouse region of eastern Washington State were investigated using Landsat Thematic Mapper (TM) band ratios to discriminate areas where erosion has caused paleosols to be exposed. Ratioed data were clustered and plotted to show soil lines which could be subdivided into various levels of organic matter and iron oxides. Successfully classified scenes of a summer fallow (bare soil) field were obtained with band ratios 1/4, 3/4, and 5/4 to map organic carbon and 3/4, 5/4, and 5/3 for the iron/carbon ratio indicator of erosion. Regression models were made with 5/4 data and organic carbon and 5/3 data and the iron/carbon ratio. Based on this analysis, 21 percent of the test field soils are exposed or nearly exposed paleosols.

Frazier, B. E.↗

NASA NTRS1993

Variations in Venus cloud particle properties: A new view of Venus's cloud morphology as observed by Galileo Near-Infrared Mapping Spectrometer

Using Venus nightside data obtained by the Galileo Near-Infrared Mapping Spectrometer (NIMS), we have studied the correlation of 1.74 and 2.30 micrometer radiation which is transmitted through the clouds. Since the scattering and absorption properties of the cloud particles are different at these two wavelengths, one can distinguish between abundance variations and variations in the properties of the cloud particles themselves. The correlation of intensities shows a clustering of data into five distinct branches. Using radiative transfer calculations, we interpret these branches as regions of distinct but different mixes of Mode 2' and 3 particles. The data and calculations indicate large differences in these modal ratios, the active cloud regions varying in content from nearly pure Mode 2' particles to almost wholly Mode 3. The spatial distribution of these branches shows large scale sizes and both hemispheric symmetries and asymmetries. High-latitude concentrations of large particles are seen in both hemispheres and there is banded structure of small particles seen in both the North and South which may be related. The mean particle size in the Northern Hemisphere is greater than found in the South. If these different branch regions are due to mixing of vertically stratified source regions (e.g. photochemical and condensation source mechanisms), then the mixing must be coherent over very large spatial scales.

Carlson, R. W.↗

DOE OSTI2014

Intracluster light at the Frontier: A2744

The ultra-deep multiwavelength Hubble Space Telescope Frontier Fields coverage of the Abell Cluster 2744 is used to derive the stellar population properties of its intracluster light (ICL). The restframe colors of the ICL of this intermediate redshift (z = 0.3064) massive cluster are bluer (g – r = 0.68 ± 0.04; i – J = 0.56 ± 0.01) than those found in the stellar populations of its main galaxy members (g – r = 0.83 ± 0.01; i – J = 0.75 ± 0.01). Based on these colors, we derive the following mean metallicity Z = 0.018 ± 0.007 for the ICL. The ICL age is 6 ± 3 Gyr younger than the average age of the most massive galaxies of the cluster. The fraction of stellar mass in the ICL component comprises at least 6% of the total stellar mass of the galaxy cluster. Our data are consistent with a scenario where the bulk of the ICL of A2744 has been formed relatively recently (z < 1). The stellar population properties of the ICL suggest that this diffuse component is mainly the result of the disruption of infalling galaxies with similar characteristics in mass (M {sub *} ∼ 3 × 10{sup 10} M {sub ☉}) and metallicity than our own Milky Way. The amount of ICL mass in the central part of the cluster (<400 kpc) is equivalent to the disruption of 4-6 Milky-Way-type galaxies.

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2015

A statistical study of EMIC waves observed by Cluster. 1. Wave properties. EMIC Wave Properties

Electromagnetic ion cyclotron (EMIC) waves are an important mechanism for particle energization and losses inside the magnetosphere. In order to better understand the effects of these waves on particle dynamics, detailed information about the occurrence rate, wave power, ellipticity, normal angle, energy propagation angle distributions, and local plasma parameters are required. Previous statistical studies have used in situ observations to investigate the distribution of these parameters in the magnetic local time versus L-shell (MLT-L) frame within a limited magnetic latitude (MLAT) range. In our study, we present a statistical analysis of EMIC wave properties using 10 years (2001–2010) of data from Cluster, totaling 25,431 min of wave activity. Due to the polar orbit of Cluster, we are able to investigate EMIC waves at all MLATs and MLTs. This allows us to further investigate the MLAT dependence of various wave properties inside different MLT sectors and further explore the effects of Shabansky orbits on EMIC wave generation and propagation. Thus, the statistical analysis is presented in two papers. OUr paper focuses on the wave occurrence distribution as well as the distribution of wave properties. The companion paper focuses on local plasma parameters during wave observations as well as wave generation proxies.

58 GEOSCIENCES↗

DOE OSTI2015

Characterization and classification of vegetation canopy structure and distribution within the Great Smoky Mountains National Park using LiDAR

Vegetation canopy structure is a critically important habit characteristic for many threatened and endangered birds and other animal species, and it is key information needed by forest and wildlife managers for monitoring and managing forest resources, conservation planning and fostering biodiversity. Advances in Light Detection and Ranging (LiDAR) technologies have enabled remote sensing-based studies of vegetation canopies by capturing three-dimensional structures, yielding information not available in two-dimensional images of the landscape pro- vided by traditional multi-spectral remote sensing platforms. However, the large volume data sets produced by airborne LiDAR instruments pose a significant computational challenge, requiring algorithms to identify and analyze patterns of interest buried within LiDAR point clouds in a computationally efficient manner, utilizing state-of-art computing infrastructure. We developed and applied a computationally efficient approach to analyze a large volume of LiDAR data and to characterize and map the vegetation canopy structures for 139,859 hectares (540 sq. miles) in the Great Smoky Mountains National Park. This study helps improve our understanding of the distribution of vegetation and animal habitats in this extremely diverse ecosystem.

Clustering↗

DOE OSTI2018

Cluster Sampling Filters for Non-Gaussian Data Assimilation

This paper presents a fully non-Gaussian filter for sequential data assimilation. The filter is named the “cluster sampling filter”, and works by directly sampling the posterior distribution following a Markov Chain Monte-Carlo (MCMC) approach, while the prior distribution is approximated using a Gaussian Mixture Model (GMM). Specifically, a clustering step is introduced after the forecast phase of the filter, and the prior density function is estimated by fitting a GMM to the prior ensemble. Using the data likelihood function, the posterior density is then formulated as a mixture density, and is sampled following an MCMC approach. Four versions of the proposed filter, namely C ℓ MCMC , C ℓ HMC , MC- C ℓ HMC , and MC- C ℓ HMC are presented. C ℓ MCMC uses a Gaussian proposal density to sample the posterior, and C ℓ HMC is an extension to the Hamiltonian Monte-Carlo (HMC) sampling filter. MC- C ℓ MCMC and MC- C ℓ HMC are multi-chain versions of the cluster sampling filters C ℓ MCMC and C ℓ HMC respectively. The multi-chain versions are proposed to guarantee that samples are taken from the vicinities of all probability modes of the formulated posterior. The new methodologies are tested using a simple one-dimensional example, and a quasi-geostrophic (QG) model with double-gyre wind forcing and bi-harmonic friction. Here, numerical results demonstrate the usefulness of using GMMs to relax the Gaussian prior assumption especially in the HMC filtering paradigm.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2016

A novel procedure on next generation sequencing data analysis using text mining algorithm

Background: Next-generation sequencing (NGS) technologies have provided researchers with vast possibilities in various biological and biomedical research areas. Efficient data mining strategies are in high demand for large scale comparative and evolutional studies to be performed on the large amounts of data derived from NGS projects. Topic modeling is an active research field in machine learning and has been mainly used as an analytical tool to structure large textual corpora for data mining. Methods: We report a novel procedure to analyse NGS data using topic modeling. It consists of four major procedures: NGS data retrieval, preprocessing, topic modeling, and data mining using Latent Dirichlet Allocation (LDA) topic outputs. The NGS data set of the Salmonella enterica strains were used as a case study to show the workflow of this procedure. The perplexity measurement of the topic numbers and the convergence efficiencies of Gibbs sampling were calculated and discussed for achieving the best result from the proposed procedure. Results: The output topics by LDA algorithms could be treated as features of Salmonella strains to accurately describe the genetic diversity of fliC gene in various serotypes. The results of a two-way hierarchical clustering and data matrix analysis on LDA-derived matrices successfully classified Salmonella serotypes based on the NGS data. The implementation of topic modeling in NGS data analysis procedure provides a new way to elucidate genetic information from NGS data, and identify the gene-phenotype relationships and biomarkers, especially in the era of biological and medical big data. Conclusion: The implementation of topic modeling in NGS data analysis provides a new way to elucidate genetic information from NGS data, and identify the gene-phenotype relationships and biomarkers, especially in the era of biological and medical big data.

59 BASIC BIOLOGICAL SCIENCES↗

NASA NTRS2004

Fossil Signatures Using Elemental Abundance Distributions and Bayesian Probabilistic Classification

Elemental abundances (C6, N7, O8, Na11, Mg12, Al3, P15, S16, Cl17, K19, Ca20, Ti22, Mn25, Fe26, and Ni28) were obtained for a set of terrestrial fossils and the rock matrix surrounding them. Principal Component Analysis extracted five factors accounting for the 92.5% of the data variance, i.e. information content, of the elemental abundance data. Hierarchical Cluster Analysis provided unsupervised sample classification distinguishing fossil from matrix samples on the basis of either raw abundances or PCA input that agreed strongly with visual classification. A stochastic, non-linear Artificial Neural Network produced a Bayesian probability of correct sample classification. The results provide a quantitative probabilistic methodology for discriminating terrestrial fossils from the surrounding rock matrix using chemical information. To demonstrate the applicability of these techniques to the assessment of meteoritic samples or in situ extraterrestrial exploration, we present preliminary data on samples of the Orgueil meteorite. In both systems an elemental signature produces target classification decisions remarkably consistent with morphological classification by a human expert using only structural (visual) information. We discuss the possibility of implementing a complexity analysis metric capable of automating certain image analysis and pattern recognition abilities of the human eye using low magnification optical microscopy images and discuss the extension of this technique across multiple scales.

Hoover, Richard B.↗

DOE OSTI2015

The effect of halo mass on the H i content of galaxies in groups and clusters

We combine data from the Sloan Digital Sky Survey (SDSS) and the Arecibo Legacy Fast ALFA Survey (ALFALFA) to study the cold atomic gas content of galaxies in groups and clusters in the local universe. A careful cross-matching of galaxies in the SDSS, ALFALFA, and SDSS group catalogs provides a sample of group galaxies with stellar masses 10{sup 8.4}M{sub ⊙}⩽M{sub ∗}⩽10{sup 10.6}M{sub ⊙} and group halo masses 10{sup 12.5}h{sup −1}M{sub ⊙}⩽M{sub h}⩽10{sup 15.0}h{sup −1}M{sub ⊙}. Controlling our sample in stellar mass and redshift, we find no significant radial variation in the galaxy H i gas-to-stellar mass ratio for the halo mass range in our sample. However, the fraction of galaxies detected in ALFALFA declines steadily toward the centers of groups, with the effect being most prominent in the most massive halos. In the outskirts of massive halos a hint of a depressed detection fraction for low-mass galaxies suggests pre-processing that decreases the H i in these galaxies before they fall into massive clusters. We interpret the decline in the ALFALFA detection of galaxies in the context of a threshold halo mass for ram pressure stripping for a given galaxy stellar mass. The lack of an observable decrease in the galaxy H i gas-to-stellar mass ratio with the position of galaxies within groups and clusters highlights the difficulty of detecting the impact of environment on the galaxy H i content in a shallow H i survey.

79 ASTRONOMY AND ASTROPHYSICS↗

NASA NTRS1980

Distorted radio sources in Abell 2255 - Evidence of intergalactic gas 2.5 to 5 megaparsecs from the cluster center

Spectroscopic data are presented for radio galaxies in the A2255 region. The results indicate that the galaxies associated with distorted radio sources 2.5-5 Mpc projected radius from the cluster center are indeed cluster members. This implies the presence of substantial intergalactic gas 10-20 core radii from the cluster center. Reasonable assumptions concerning the gas distribution result in an estimated minimum mass for the A2255 intracluster gas of 4 x 10 to the 14th solar masses.

Hintzen, P.↗

DOE OSTI2025

Enhancing Cluster Identification in Atom Probe Tomography Data Using Transfer Learning

Atom probe tomography (APT) has enabled the direct visualization of solute clusters, providing valuable insights into material structures. This clustering is crucial for understanding the nanoscale composition and behavior of materials, which can significantly influence their mechanical and physical properties. However, the widely used clustering methods in the APT community face challenges such as subjective parametric selection and limited applicability, particularly in dealing with overlapping clusters, nested clusters, and artifacts across different scales, such as precipitates and dislocations. To address these challenges, we present a framework based on density-based cluster analysis that aims to be less dependent on user input, reproducible, and robust.

Density-based clustering↗

DOE OSTI2018

Visualizing Clustering and Uncertainty Analysis with Multivariate Longitudinal Data

Multivariate time-series datasets are intrinsic to the study of dynamic, naturalistic behavior, such as in the applications of finance and motion video analysis. Statistical models provide the ability to identify event patterns in these data under conditions of uncertainty, but researchers must be able to evaluate how well a model uses available information in a dataset for clustering decisions and for uncertainty information. The Hidden Markov Model (HMM) is an established method for clustering time-series data, where the hidden states of the HMM are the clusters. We develop novel methods for quantifying the uncertainty of the performance of and for visualizing the clustering performance and uncertainty of fitting a HMM to multivariate time-series data. We explain the usefulness of uncertainty quantification and visualization with evaluating the performance of clustering models, as well as how information exploitation of time-series datasets can be enhanced. We implement our methods to cluster patterns of scanpaths from raw eye tracking data.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2016

Parallel k-means++ for Multiple Shared-Memory Architectures

In recent years k-means++ has become a popular initialization technique for improved k-means clustering. To date, most of the work done to improve its performance has involved parallelizing algorithms that are only approximations of k-means++. In this paper we present a parallelization of the exact k-means++ algorithm, with a proof of its correctness. We develop implementations for three distinct shared-memory architectures: multicore CPU, high performance GPU, and the massively multithreaded Cray XMT platform. We demonstrate the scalability of the algorithm on each platform. In addition we present a visual approach for showing which platform performed k-means++ the fastest for varying data sizes.

data clustering↗

DOE OSTI2015

Solvent-mediated internal conversion in diphenoxyethane-(H{sub 2}O){sub n}clusters, n = 2-4

1,2-diphenoxyethane (DPOE) is a flexible bichromophore whose excited states come in close-lying pairs whose splitting and vibronic coupling can be modulated by solvent. Building on the ground state infrared spectroscopy of DPOE-(H{sub 2}O){sub n} clusters with n = 2-4 from the adjoining paper [Walsh et al., J. Chem. Phys. 142, 154303 (2015)], the present work focuses on the vibronic and excited state infrared spectroscopies of the clusters. The type and degree of asymmetry of the water cluster binding to DPOE is reflected in the variation in the magnitude of the S{sub 1}/S{sub 2} splitting with cluster size. Excited state resonant ion-dip infrared spectroscopy was performed at the electronic origins of the first two excited states in order to explore how the water clusters’ OH stretch spectra report on the nature of the two excited states, and the interaction of the S{sub 2} state with nearby S{sub 1} vibronic levels mediated by the water clusters. The data set, when taken as a whole, provides a state-to-state view of internal conversion and the role of solvent in mediating conversion of electronic excitation between two chromophores, providing a molecular-scale view of Kasha’s rule.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

DOE OSTI2019

The Application of the OPTICS Algorithm to Cluster Analysis in Atom Probe Tomography Data

Atom probe tomography (APT) is a powerful technique to characterize buried 3D nanostructures in a variety of materials. Accurate characterization of those nanometer-scale clusters and precipitates is of great scientific significance to understand the structure-property relationships and the microstructural evolution. The current widely used cluster analysis method, a variant of the DBSCAN algorithm, can only accurately extract clusters of the same atomic density, neglecting several experimental realities, such as density variations within and between clusters and the non-uniformity of the atomic density in the APT reconstruction itself (e.g. crystallographic poles and other field evaporation artifacts). The clustering results of DBSCAN-like method rely heavily on multiple input parameters, but ideal selection of those parameters is challenging and oftentimes ambiguous. In this study, we present a new cluster analysis method consisting of a well-known algorithm, called OPTICS, and a cluster extraction algorithm we developed to enable accurate detection of clusters of varying atomic density in APT data. The new method requires only one free parameter, and other inputs can be estimated based on physical parameters of the material. The effectiveness of this method is demonstrated by application to several small-scale model datasets and a real APT dataset obtained from an oxide-dispersion strengthened ferritic alloy specimen. The method’s sensitivity with respect to parameter selection is discussed and compared with the current widely used methods.

atom probe tomography, cluster analysis, density-b↗

DOE OSTI2018

Parallel multiple-chain DRAM MCMC for large-scale geosteering inversion and uncertainty quantification

Geosteering is the proactive control of a wellbore placement based on the downhole measurements, aiming at maximizing economic production from the well. We report the recent development of azimuthal resistivity logging-whiledrilling (LWD) tools have a much larger depth of detection, thus sets higher demands for geosteering inversion as more unknown parameters need to be taken into account in the earth model to be inverted. For complicated nonlinear problems, traditional deterministic inversion methods are more likely to be trapped near local optima. In general, Markov chain Monte Carlo (MCMC) inversion methods are more capable of finding the global optimal solution and providing additional uncertainty analyses by sampling from the target distribution. However, MCMC methods usually have the problem of slow convergence. Even though optimization methods like delayed rejection (DR) and adaptive Metropolis (AM) were proposed to speed up the convergence, using MCMC methods to solve geosteering inversion problems may still incur unacceptable time cost. To reduce the sampling cost of MCMC methods, we use parallel multiple-chain DRAM MCMC methods to solve geosteering inverse problems and estimate the corresponding uncertainty. A clustering method, density-based spatial clustering of applications with noise (DBSCAN), is applied to select the best solution among many results. Lastly, the simulation results show that running many relatively short MCMC chains for one problem can obtain a similar result as running a long single chain. Besides, avoiding communications between multiple Markov chains during the sampling process yields almost linear scalability.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2023

Six More Ultra-faint Milky Way Companions Discovered in the DECam Local Volume Exploration Survey

We report the discovery of six ultra-faint Milky Way satellites identified through matched-filter searches conducted using Dark Energy Camera (DECam) data processed as part of the second data release of the DECam Local Volume Exploration (DELVE) survey. Leveraging deep Gemini/GMOS-N imaging (for four candidates) as well as follow-up DECam imaging (for two candidates), we characterize the morphologies and stellar populations of these systems. We find that these candidates all share faint absolute magnitudes (M$_{V}$ ≥ −3.2 mag) and old, metal-poor stellar populations (τ > 10 Gyr, [Fe/H] < −1.4 dex). Three of these systems are more extended (r$_{1/2}$ > 15 pc), while the other three are compact (r$_{1/2}$ < 10 pc). From these properties, we infer that the former three systems (Boötes V, Leo Minor I, and Virgo II) are consistent with ultra-faint dwarf galaxy classifications, whereas the latter three (DELVE 3, DELVE 4, and DELVE 5) are likely ultra-faint star clusters. Using data from the Gaia satellite, we confidently measure the proper motion of Boötes V, Leo Minor I, and DELVE 4, and tentatively detect a proper-motion signal from DELVE 3 and DELVE 5; no signal is detected for Virgo II. We use these measurements to explore possible associations between the newly discovered systems and the Sagittarius dwarf spheroidal, the Magellanic Clouds, and the Vast Polar Structure, finding several plausible associations. Our results offer a preview of the numerous ultra-faint stellar systems that will soon be discovered by the Vera C. Rubin Observatory and highlight the challenges of classifying the faintest stellar systems.

79 ASTRONOMY AND ASTROPHYSICS↗