Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data analysis cluster”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
NASA NTRS2004

Clustering with Missing Values: No Imputation Required

Clustering algorithms can identify groups in large data sets, such as star catalogs and hyperspectral images. In general, clustering methods cannot analyze items that have missing data values. Common solutions either fill in the missing values (imputation) or ignore the missing data (marginalization). Imputed values are treated as just as reliable as the truly observed data, but they are only as good as the assumptions used to create them. In contrast, we present a method for encoding partially observed features as a set of supplemental soft constraints and introduce the KSC algorithm, which incorporates constraints into the clustering process. In experiments on artificial data and data from the Sloan Digital Sky Survey, we show that soft constraints are an effective way to enable clustering with missing values.

constraints↗
NASA NTRS1977

Alteration mapping at Goldfield, Nevada, by cluster and discriminant analysis of LANDSAT digital data

The ability of Landsat multispectral digital data to differentiate among 62 combinations of rock and alteration types at the Goldfield mining district of Western Nevada was investigated by using statistical techniques of cluster and discriminant analysis. Multivariate discriminant analysis was not effective in classifying each of the 62 groups, with classification results essentially the same whether data of four channels alone or combined with six ratios of channels were used. Bivariate plots of group means revealed a cluster of three groups including mill tailings, basalt and all other rock and alteration types. Automatic hierarchical clustering based on the fourth dimensional Mahalanobis distance between group means of 30 groups having five or more samples was performed. The results of the cluster analysis revealed hierarchies of mill tailings vs. natural materials, basalt vs. non-basalt, highly reflectant rocks vs. other rocks and exclusively unaltered rocks vs. predominantly altered rocks. The hierarchies were used to determine the order in which sets of multiple discriminant analyses were to be performed and the resulting discriminant functions were used to produce a map of geology and alteration which has an overall accuracy of 70 percent for discriminating exclusively altered rocks from predominantly altered rocks.

Ballew, G.↗
NASA NTRS1977

Alteration mapping at Goldfield, Nevada, by cluster and discriminant analysis of Landsat digital data

The ability of Landsat multispectral digital data to differentiate among 62 combinations of rock and alteration types at the Goldfield mining district of Western Nevada was investigated by using statistical techniques of cluster and discriminant analysis. Multivariate discriminant analysis was not effective in classifying each of the 62 groups, with classification results essentially the same whether data of four channels alone or combined with six ratios of channels were used. Bivariate plots of group means revealed a cluster of three groups including mill tailings, basalt and all other rock and alteration types. Automatic hierarchical clustering based on the fourth dimensional Mahalanobis distance between group means of 30 groups having five or more samples was performed using Johnson's HICLUS program. The results of the cluster analysis revealed hierarchies of mill tailings vs. natural materials, basalt vs. non-basalt, highly reflectant rocks vs. other rocks and exclusively unaltered rocks vs. predominantly altered rocks. The hierarchies were used to determine the order in which sets of multiple discriminant analyses were to be performed and the resulting discriminant functions were used to produce a map of geology and alteration which has an overall accuracy of 70 percent for discriminating exclusively altered rocks from predominantly altered rocks.

Ballew, G.↗
NASA NTRS2004

Fossil Signatures Using Elemental Abundance Distributions and Bayesian Probabilistic Classification

Elemental abundances (C6, N7, O8, Na11, Mg12, Al3, P15, S16, Cl17, K19, Ca20, Ti22, Mn25, Fe26, and Ni28) were obtained for a set of terrestrial fossils and the rock matrix surrounding them. Principal Component Analysis extracted five factors accounting for the 92.5% of the data variance, i.e. information content, of the elemental abundance data. Hierarchical Cluster Analysis provided unsupervised sample classification distinguishing fossil from matrix samples on the basis of either raw abundances or PCA input that agreed strongly with visual classification. A stochastic, non-linear Artificial Neural Network produced a Bayesian probability of correct sample classification. The results provide a quantitative probabilistic methodology for discriminating terrestrial fossils from the surrounding rock matrix using chemical information. To demonstrate the applicability of these techniques to the assessment of meteoritic samples or in situ extraterrestrial exploration, we present preliminary data on samples of the Orgueil meteorite. In both systems an elemental signature produces target classification decisions remarkably consistent with morphological classification by a human expert using only structural (visual) information. We discuss the possibility of implementing a complexity analysis metric capable of automating certain image analysis and pattern recognition abilities of the human eye using low magnification optical microscopy images and discuss the extension of this technique across multiple scales.

Hoover, Richard B.↗
NASA NTRS1984

A comparative evaluation of simulated SPOT, Thematic Mapper, and simulated Thematic Mapper data in a western ecozone

Spectral information content was quantitatively measured for simulated SPOT, Thematic Mapper, and simulated Thematic Mapper data acquired over Santa Cruz County, California. Two techniques were employed: spectral separability summarized through optimal wave band evaluation, and histogram cluster analysis. Using data from the three sources it is shown that three spectral bands can provide separability nearly as high as six bands provide, if the three bands are optimally chosen for the cover type. Analysis of the Santa Cruz scene emphasizes the importance of TM bands 1 and 5 in particular. The three bands of the SPOT scanner are shown to provide sufficient separability for cover types within the study area. The higher spatial resolution data (20 meters) provided by SPOT contain more distinct distributions than data from the other sources studied. Techniques should be developed to exploit this additional information.

Buis, J. S.↗
NASA NTRS2009

Spherical Harmonic Analysis of Particle Velocity Distribution Function: Comparison of Moments and Anisotropies using Cluster Data

This paper presents a spherical harmonic analysis of the plasma velocity distribution function using high-angular, energy, and time resolution Cluster data obtained from the PEACE spectrometer instrument to demonstrate how this analysis models the particle distribution function and its moments and anisotropies. The results show that spherical harmonic analysis produced a robust physical representation model of the velocity distribution function, resolving the main features of the measured distributions. From the spherical harmonic analysis, a minimum set of nine spectral coefficients was obtained from which the moment (up to the heat flux), anisotropy, and asymmetry calculations of the velocity distribution function were obtained. The spherical harmonic method provides a potentially effective "compression" technique that can be easily carried out onboard a spacecraft to determine the moments and anisotropies of the particle velocity distribution function for any species. These calculations were implemented using three different approaches, namely, the standard traditional integration, the spherical harmonic (SPH) spectral coefficients integration, and the singular value decomposition (SVD) on the spherical harmonic methods. A comparison among the various methods shows that both SPH and SVD approaches provide remarkable agreement with the standard moment integration method.

Gurgiolo, Chris↗
DOE OSTI2022

Cluster Analysis of Combined EDS and EBSD Data to Solve Ambiguous Phase Identifications

A common problem in analytical scanning electron microscopy (SEM) using electron backscatter diffraction (EBSD) is the differentiation of phases with distinct chemistry but the same or very similar crystal structure. X-ray energy dispersive spectroscopy (EDS) is useful to help differentiate these phases of similar crystal structures but different elemental makeups. However, open, automated, and unbiased methods of differentiating phases of similar EBSD responses based on their EDS response are lacking. This paper describes a simple data analytics-based method, using a combination of singular value decomposition and cluster analysis, to merge simultaneously acquired EDS + EBSD information and automatically determine phases from both their crystal and elemental data. I use hexagonal TiB 2 ceramic contaminated with multiple crystallographically ambiguous but chemically distinct cubic phases to illustrate the method. Code, in the form of a Python 3 Jupyter Notebook, and the necessary data to replicate the analysis are provided as Supplementary material.

47 OTHER INSTRUMENTATION↗
DOE OSTI2022

AutoEnRichness: A hybrid empirical and analytical approach for estimating the richness of galaxy clusters

ABSTRACT We introduce AutoEnRichness, a hybrid approach that combines empirical and analytical strategies to determine the richness of galaxy clusters (in the redshift range of 0.1 ≤ z ≤ 0.35) using photometry data from the Sloan Digital Sky Survey Data Release 16, where cluster richness can be used as a proxy for cluster mass. In order to reliably estimate cluster richness, it is vital that the background subtraction is as accurate as possible when distinguishing cluster and field galaxies to mitigate severe contamination. AutoEnRichness is comprised of a multistage machine learning algorithm that performs background subtraction of interloping field galaxies along the cluster line of sight and a conventional luminosity distribution fitting approach that estimates cluster richness based only on the number of galaxies within a magnitude range and search area. In this proof-of-concept study, we obtain a balanced accuracy of 83.20 per cent when distinguishing between cluster and field galaxies as well as a median absolute percentage error of 33.50 per cent between our estimated cluster richnesses and known cluster richnesses within r200. In the future, we aim for AutoEnRichness to be applied on upcoming large-scale optical surveys, such as the Legacy Survey of Space and Time and Euclid, to estimate the richness of a large sample of galaxy groups and clusters from across the halo mass function. This would advance our overall understanding of galaxy evolution within overdense environments as well as enable cosmological parameters to be further constrained.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2022

Estimating cluster masses from SDSS multiband images with transfer learning

ABSTRACT The total masses of galaxy clusters characterize many aspects of astrophysics and the underlying cosmology. It is crucial to obtain reliable and accurate mass estimates for numerous galaxy clusters over a wide range of redshifts and mass scales. We present a transfer-learning approach to estimate cluster masses using the ugriz-band images in the SDSS Data Release 12. The target masses are derived from X-ray or SZ measurements that are only available for a small subset of the clusters. We designed a semisupervised deep learning model consisting of two convolutional neural networks. In the first network, a feature extractor is trained to classify the SDSS photometric bands. The second network takes the previously trained features as inputs to estimate their total masses. The training and testing processes in this work depend purely on real observational data. Our algorithm reaches a mean absolute error (MAE) of 0.232 dex on average and 0.214 dex for the best fold. The performance is comparable to that given by redMaPPer, 0.192 dex. We have further applied a joint integrated gradient and class activation mapping method to interpret such a two-step neural network. The performance of our algorithm is likely to improve as the size of training data set increases. This proof-of-concept experiment demonstrates the potential of deep learning in maximizing the scientific return of the current and future large cluster surveys.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
NASA NTRS2011

Flood Inundation Mapping in the Logone Floodplain from Multi Temporal Landsat ETM+Imagery

Yearly flooding in the Logone floodplain makes an impact on agricultural, pastoral, and fishery systems in the Lake Chad Basin. Since the flooding extent and depth are highly variable, flood inundation mapping helps us make better use of water resources and prevent flood hazards in the Logone floodplain. The flood maps are generated from 33 multi temporal Landsat Enhanced Thematic Mapper Plus (ETM+) during three years 2006 to 2008. Flooded area is classified using a short-wave infrared band whereas open water is classified by Iterative Self-organizing Data Analysis (ISODATA) clustering. The maximum flooding extent in the study area increases up to approximately 5.8K km2 in late October 2008. The study also provides strong correlation of the flooding extents with water height variations in both the floodplain and the river based on a second polynomial regression model. The water heights are from ENIVSAT altimetry in the floodplain and gauge measurements in the river. Coefficients of determination between flooding extents and water height variations are greater than 0.91 with 4 to 36 days in phase lag. Floodwater drains back to the river and to the northeast during the recession period in December and January. The study supports understanding of the Logone floodplain dynamics in detail of spatial pattern and size of the flooding extent and assists the flood monitoring and prediction systems in the catchment.

Jung, Hahn Chul↗
DOE OSTI2020

Weak-lensing Analysis of X-Ray-selected XXL Galaxy Groups and Clusters with Subaru HSC Data

We present a weak-lensing analysis of X-ray galaxy groups and clusters selected from the XMM-XXL survey using the first-year data from the Hyper Suprime-Cam (HSC) Subaru Strategic Program. Our joint weaklensing and X-ray analysis focuses on 136 spectroscopically confirmed X-ray-selected systems at 0.031 6 z 6 1.033 detected in the 25 deg2 XXL-N region, which largely overlaps with the HSC-XMM field. With high-quality HSC weak-lensing data, we characterize the mass distributions of individual clusters and establish the concentration–mass (c–M) relation for the XXL sample, by accounting for selection bias and statistical effects and marginalizing over the remaining mass calibration uncertainty. We find the mass-trend parameter of the c–M relation to be β = -0.07 ± 0.28 and the normalization to be c200 = 4.8 ± 1.0 (stat) ± 0.8 (syst) at M 200 = 1014h -1M and z = 0.3. We find no statistical evidence for redshift evolution. Our weaklensing results are in excellent agreement with dark-matter-only c–M relations calibrated for recent ΛCDM cosmologies. The level of intrinsic scatter in c200 is constrained as σ(ln c 200 ) < 24% (99.7% CL), which is smaller than predicted for the full population of ΛCDM halos. This is likely caused in part by the X-ray selection bias in terms of the cool-core or relaxation state. We determine the temperature–mass (TX–M 500 ) relation for a subset of 105 XXL clusters that have both measured HSC lensing masses and X-ray temperatures. The resulting TX–M 500 relation is consistent with the self-similar prediction. Our TX–M 500 relation agrees with the XXL DR1 results at group scales but has a slightly steeper mass trend, implying a smaller mass scale in the cluster regime. The overall offset in the TX–M 500 relation is at the ~ 1.5σ level, corresponding to a mean mass offset of (34 ± 20)%. We also provide bias-corrected, weak-lensing-calibrated M 200 and M 500 mass estimates of individual XXL clusters based on their measured X-ray temperatures.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2021

Cosmological constraints from DES Y1 cluster abundances and SPT multiwavelength data

We perform a joint analysis of the counts of redMaPPer clusters selected from the Dark Energy Survey (DES) year 1 data and multiwavelength follow-up data collected within the 2500 deg2 South Pole Telescope (SPT) Sunyaev-Zel’dovich (SZ) survey. The SPT follow-up data, calibrating the richness-mass relation of the optically selected redMaPPer catalog, enable the cosmological exploitation of the DES cluster abundance data. To explore possible systematics related to the modeling of projection effects, we consider two calibrations of the observational scatter on richness estimates: a simple Gaussian model which account only for the background contamination (BKG), and a model which further includes contamination and incompleteness due to projection effects (PRJ). Assuming either a ΛCDM+∑mν or wCDM+∑mν cosmology, and for both scatter models, we derive cosmological constraints consistent with multiple cosmological probes of the low and high redshift Universe, and in particular with the SPT cluster abundance data. This result demonstrates that the DES Y1 and SPT cluster counts provide consistent cosmological constraints, if the same mass calibration data set is adopted. It thus supports the conclusion of the DES Y1 cluster cosmology analysis which interprets the tension observed with other cosmological probes in terms of systematics affecting the stacked weak lensing analysis of optically selected low–richness clusters. Finally, we analyze the first combined optically SZ selected cluster catalog obtained by including the SPT sample above the maximum redshift probed by the DES Y1 redMaPPer sample (z=0.65). Besides providing a mild improvement of the cosmological constraints, this data combination serves as a stricter test of our scatter models: the PRJ model, providing scaling relations consistent between the two abundance and multiwavelength follow-up data, is favored over the BKG model.

79 ASTRONOMY AND ASTROPHYSICS↗
NASA NTRS1984

Convective rain rates and their evolution during storms in a semiarid climate

The semiarid climate of the U.S. northern High Plains region has been studied with respect to rain rates and their evolution during summertime convective storms, using radar data from a total of 750 radar echo clusters. Analysis of this data suggests that the average rain rate R among storms is in a first approximation independent of the total rain volume, if the entire storm duration is considered in the averaging process. R primarily depends on the reflectivity threshold considered in calculating the area coverage integrated over the lifetime of the storm. R evolution during storms is analyzed by dividing each storm lifetime into 10 min, 1, 2, and 4 hours, as well as growing and decaying periods. The value of R remained independent of the total rain volume when the growing or decaying periods of storms were considered separately.

Doneaud, A. A.↗
DOE OSTI2025

A Parameter-masked Mock Data Challenge for Beyond-two-point Galaxy Clustering Statistics

The past few years have seen the emergence of a wide array of novel techniques for analyzing high-precision data from upcoming galaxy surveys, which aim to extend the statistical analysis of galaxy clustering data beyond the linear regime and the canonical two-point (2pt) statistics. We test and benchmark some of these new techniques in a community data challenge named “Beyond-2pt,” initiated during the Aspen 2022 Summer Program “Large-Scale Structure Cosmology beyond 2-Point Statistics,” whose first round of results we present here. The challenge data set consists of high-precision mock galaxy catalogs for clustering in real space, in redshift space, and on a light cone. Participants in the challenge have developed end-to-end pipelines to analyze mock catalogs and extract unknown (“masked”) cosmological parameters of the underlying ΛCDM models with their methods. The methods represented are density-split clustering, nearest neighbor statistics, BACCO power spectrum emulator, void statistics, LEFTfield field-level inference using effective field theory (EFT), and joint power spectrum and bispectrum analyses using both EFT and simulation-based inference. In this work, we review the results of the challenge, focusing on problems solved, lessons learned, and future research needed to perfect the emerging beyond-2pt approaches. The unbiased parameter recovery demonstrated in this challenge by multiple statistics and the associated modeling and inference frameworks supports the credibility of cosmology constraints from these methods. The challenge data set is publicly available, and we welcome future submissions from methods that are not yet represented.

Krause, Elisabeth [Univ. of Arizona, Tucson, AZ (U↗
DOE OSTI2025

SITCOMTN-161: PSF assessment in the field of Abell 360 and shapeHSM shear profile using LSSTComCam data

The Rubin LSSTComCam on-sky campaign performed at the end of 2024 provided observations of the Abell 360 galaxy cluster; these data allow a preliminary study of cluster weak lensing analysis using Rubin Data Preview 1 (DP1) data. Among all the steps required for such analyses, accurate modeling of the PSF is essential. This work uses several diagnostics, mostly based on the residuals between the second moments of stars and the PSF model, to characterize the accuracy of the PSF modeling in the A360 field. We find the level of the residuals to be sufficiently low not to hinder the measurement of the tangential shear profile around A360. With a simple source selection process, we demonstrate that outputs of the LSST Science Pipelines can be used to detect the tangential shear profile in Abell 360 at the 3.6σ level, and our analysis indicates that contamination from PSF modeling systematics is negligible.

Dell'Antonio, Ian [Brown University]↗
NASA NTRS1985

A cluster analysis method for identification of subpopulations of cells in flow cytometric list-mode arrays

A specialized program was developed for flow cytometric list-mode data using an heirarchical tree method for identifying and enumerating individual subpopulations, the method of principal components for a two-dimensional display of 6-parameter data array, and a standard sorting algorithm for characterizing subpopulations. The program was tested against a published data set subjected to cluster analysis and experimental data sets from controlled flow cytometry experiments using a Coulter Electronics EPICS V Cell Sorter. A version of the program in compiled BASIC is usable on a 16-bit microcomputer with the MS-DOS operating system. It is specialized for 6 parameters and up to 20,000 cells. Its two-dimensional display of Euclidean distances reveals clusters clearly, as does its 1-dimensional display. The identified subpopulations can, in suitable experiments, be related to functional subpopulations of cells.

Li, Z. K.↗
DOE OSTI2023

BiG-SLiCE 2 v1.0.0

BiG-SLiCE was originally an open source Python-based command line bioinformatics software that offers a highly scalable clustering analysis on biosynthetic gene clusters (BGC) data. It allows a simultaneous analysis of millions of BGCs, exceeding the capability of other existing tools (around one hundred thousands). As a tradeoff, the clustering accuracy is relatively lower and sometimes fall short in corner cases and specific BGC classes such as the RiPPs (Ribosomally-translated, Post-translationally modified Peptides). In BiG-SLiCE V2 (developed in LBNL), the clustering algorithm has been significantly improved to deliver a much accurate result even for RiPPs and other previous corner case classes. Moreover, the speed of the overall pipeline has been improved by 50-100%. Finally, additional features were implemented to support downstream analyses of BiG-SLiCE results, such as customized tabular (TSV/CSV) and columnar (Parquet) outputs.

Kautsar, Satria↗