Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data clustering”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3

Technologies for node-degree based clustering of data sets

Technologies for node-degree based clustering include a computing device to construct a graph that includes multiple vertices corresponding to the data points of a data set. The computing device inserts an edge between each pair of vertices that has a corresponding similarity metric that meets a predetermined threshold similarity metric. The computing device determines a node degree for each vertex in the graph and initializes a cutoff node degree as the lowest node degree of the vertices. The computing device selects a test subset of the graph that includes vertices having a node degree less than or equal to the cutoff node degree. The computing device determines whether the test subset covers the graph and if not increases the cutoff node degree. If the test subset covers the graph, the data points corresponding to the vertices of the test subset are the representative cluster. Other embodiments are described and claimed.

Sitik, Ahmet Can↗

Description of Sensor Assignment Optimization Method as Deployed on a Multi-Node Cluster

Data analytic methods are being developed to address the problem of how to assign a sensor set in a nuclear facility such that a requisite level of process monitoring capability is realized and that the sensor set is sufficiently rich to determine the status of the individual sensors with respect to need for calibration. There is an awareness in the nuclear industry that data analytics combined with rich sensor sets represent a means to improve operations and reduce costs. In the industry the calibration problem has been previously approached as an empirical data-driven problem with several methods having been developed. However, the experience of the utilities over the past ten years with these methods indicates that the absence of physics-based information renders the data-driven approach less reliable. Complicating factors such as the inherent variability of operation (both equipment alignment and operating condition) can confound a pure data-driven approach while there are no rigorous guidelines for determining what constitutes an adequate sensor set. The solution under development to overcome these shortcomings supplements the data analytic method with process information in a so-called process-constrained data-analytic approach. Simple balance equations are written for generic components (e.g., mechanical pump, valve, and heat exchanger). These do not require a priori knowledge of process parameters, such as heat transfer coefficients or friction factors. All that is needed on the part of the utility user is to identify the components and how they are connected. This report describes the development of a parallel computing capability for determining the optimal sensor set. The optimal sensor set problem suffers from the curse of dimensionality. Computation time increases exponentially as the size of the system grows. To overcome this difficulty a pre-conditioner algorithm is developed to find an approximate solution close the actual solution. This serves as a seed for the full-blown algorithm and acts to constrain the space that must searched. The optimization algorithms are described and the implementation on a parallel computing platform is described. The application of the method to a use case we are solving in collaboration with our utility partner served to illustrate how the default sensor set in a nuclear plant may not provide sufficient coverage to infer sensor calibration status.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Clustering high dimensional data using RIA

Clustering may simply represent a convenient method for organizing a large data set so that it can easily be understood and information can efficiently be retrieved. However, identifying cluster in high dimensionality data sets is a difficult task because of the curse of dimensionality. Another challenge in clustering is some traditional functions cannot capture the pattern dissimilarity among objects. In this article, we used an alternative dissimilarity measurement called Robust Influence Angle (RIA) in the partitioning method. RIA is developed using eigenstructure of the covariance matrix and robust principal component score. We notice that, it can obtain cluster easily and hence avoid the curse of dimensionality. It is also manage to cluster large data sets with mixed numeric and categorical value.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Evaluation of the radiometric quality of the TM data using clustering and multispectral distance measures

Radiometrically and geometrically corrected TM data from three different geographic locations were examined. Histograms were inspected for each band to determine the dynamic range of the data, the shape of the distributions, and to verify whether empty bins were introduced by the radiometric correction process. The effect of geometric correction on the radiometry of the resampled pixels was determined. The information content between TM and MSS data sets were compared and the TM data were used to map the thermal effluent discharge into a river ecosystem from a nuclear thermal power plant, and application only possible previously only possible through the acquisition of thermal infrared scanner data from aircraft altitudes.

Bartolucci, L. A.↗

Stratification of Landsat data by clustering

Full realization of the potential advantages of the synoptic coverage provided by Landsat will require the development and use of data analysis techniques which take into account the large variation and diversity of patterns found over many Landsat scenes. Stratification of the scene into units which are internally homogeneous is recommended as a first step in the analysis of data for whole or multiple frames of Landsat data. The use of clustering as an objective and efficient method of dividing scenes into areas which are spectrally similar (strata) is discussed and initial results, including classification performances and comparisons of spectral strata with major physical factors, are presented.

Bauer, M. E.↗

Dynamics of cD clusters of galaxies. I - Redshift data for seven clusters

We present the initial results of a spectroscopic study of a sample of cD galaxy clusters. The goal of this program is to study the dynamics of the clusters, with emphasis on determining the nature and frequency of peculiar velocity cD galaxies. In this paper we present redshifts for 611 galaxies in the fields of the rich Abell clusters A193, A399, A401, A1795, A1809, A2063, and A2124. We discuss our data reduction techniques, and compare our derived velocities with previously published values. We find a frequency of emission line galaxies of about 6 percent in clusters, a factor of 2-3 less than the field.

Hill, John M.↗

Applications of TOMS Ozone Data and Clustering Techniques in Satellite Soundings

Two primary areas are being pursued: (1) Applications of TOMS total ozone data to estimate the tropopause height and consequently to improve the temperature retrievals from satellite measurements; and (2) Applications of cluster analysis to satellite soundings. The strategy is to either use regression retrievals stratified by tropopause or to use the tropopause information as a constraint on the solution to the physical retrieval method.

Munteanu, M. J.↗

Main-sequence Scatter is Real: The Joint Dependence of Galaxy Clustering on Star Formation and Stellar Mass

We present new measurements of the clustering of stellar-mass-complete samples of ∼40,000 SDSS galaxies at z ∼ 0.03 as a joint function of stellar mass and specific star formation rate (sSFR). Our results confirm what Coil et al. find at z ∼ 0.7: galaxy clustering is a stronger function of sSFR at fixed stellar mass than of stellar mass at fixed sSFR. We also find that galaxies above the star-forming main sequence (SFMS) with higher sSFR are less clustered than galaxies below the SFMS with lower sSFR, at a given stellar mass. A similar trend is present for quiescent galaxies. This confirms that main-sequence scatter, and scatter within the quiescent sequence, is physically connected to the large-scale cosmic density field. We compare the resulting galaxy bias versus sSFR, and relative bias versus sSFR ratio, for different galaxy samples across 0 < z < 1.2 to mock galaxy catalogs based on the empirical galaxy evolution model of Behroozi et al. This model fits PRIMUS and DEEP2 clustering data well at intermediate redshift, but agreement with SDSS is not as strong. We show that increasing the correlation between galaxy SFR and halo accretion rate at z ∼ 0 in the model substantially improves agreement with SDSS data. Mock catalogs suggest that central galaxies contribute substantially to the dependence of clustering on sSFR at a given stellar mass and that the signal is not simply an effect of satellite galaxy fraction differences with sSFR. Our results are highly constraining for galaxy evolution models and show that the stellar-to-halo mass relation depends on sSFR.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Predicting the Seawater Chemistry of an Ocean World Using Machine Learning on Isotopic Measurements of Volatile CO2

Introduction: Given the long time intervals required for data transmission to and from ocean worlds targets, low bandwidth for data transmission, time required for data processing and analysis, and potentially extreme radiation environments (e.g., Europa), it is clear that ocean worlds missions will need more autonomous flight instruments and software in order to achieve established science goals. Protracted time intervals for data analysis (e.g., Europa Lander) strongly motivates the development of rapid, consistent and streamlined methods for interpreting data from flight mass spectrometers to e.g., determine how mass spectra from a plume or surface liquid/ice relates to the surface/subsurface. Since mass spectrometry also has the potential to correctly identify biosignatures[1], it is imperative that such methods for interpreting data are consistent and accurate. We used 848 isotope ratio mass spectra from laboratory analyses of CO2 that interacted with ocean worlds-relevant seawaters as a ‘training’ dataset for ‘unsupervised’ machine learning. In unsupervised learning, characteristics of the data are not labeled or linked, and any similarities found only result from the neural network. CO2 isotopologues analyzed for this dataset mimic the remote measurements of CO2 by a flight mass spectrometer, and are detailed in Theiling [2]. From this dataset, we used measured features of the spectra, such as retention time, intensity, and (isotopologue) mass ratios as inputs for our autoencoder neural network. Our neural network was trained to find similarities in these and other spectral features for seawaters of a particular composition and amount of initial CO2. Successful training then created an output of these similarities for various seawaters, which included MgSO4, Na2SO4, NaCl, MgCl2, KCl, and NaHCO3, and combinations of these salts. We then applied dimensionality reduction techniques such as Principal Component Analysis (PCA), T-Distributed Stochastic Neighbor Embedding (TSNE), and Uniform Manifold Approximation and Projection (UMAP) to demonstrate latent data features as a two-dimensional projection in a unitless, high-dimensional space. In this projection, a data point represents the combined effect of spectral features such as intensity, retention time, and isotope ratio. Our initial UMAP demonstrates data clustering (organization of the data by the neural network) based on the amount of CO2 that had initially interacted with each seawater. Further training using more ‘supervised’ learning techniques demonstrate strong clustering of preliminary data based on initial CO2 concentration, seawater chemical composition, and ionic strength (salinity). Our preliminary work therefore suggests that machine learning has the potential to identify compositional variants of an ocean world seawater based on mass spectra from volatile CO2 measurements. Acknowledgments: This work was funded through a Strategic Task Group at NASA Goddard Space Flight Center. The training dataset was collected through funding from the Oklahoma Space Grant Consortium. References: [1] Pappalardo, R. et al. (2013) Astrobiology, 13, 740–773. [2] Theiling (2020) Icarus, 114216.

Europa↗

Analysis of RXTE data on Clusters of Galaxies

This grant provided support for the reduction, analysis and interpretation of of hard X-ray (HXR, for short) observations of the cluster of galaxies RXJO658--5557 scheduled for the week of August 23, 2002 under the RXTE Cycle 7 program (PI Vahe Petrosian, Obs. ID 70165). The goal of the observation was to search for and characterize the shape of the HXR component beyond the well established thermal soft X-ray (SXR) component. Such hard components have been detected in several nearby clusters. distant cluster would provide information on the characteristics of this radiation at a different epoch in the evolution of the imiverse and shed light on its origin. We (Petrosian, 2001) have argued that thermal bremsstrahlung, as proposed earlier, cannot be the mechanism for the production of the HXRs and that the most likely mechanism is Compton upscattering of the cosmic microwave radiation by relativistic electrons which are known to be present in the clusters and be responsible for the observed radio emission. Based on this picture we estimated that this cluster, in spite of its relatively large distance, will have HXR signal comparable to the other nearby ones. The planned observation of a relatively The proposed RXTE observations were carried out and the data have been analyzed. We detect a hard X-ray tail in the spectrum of this cluster with a flux very nearly equal to our predicted value. This has strengthen the case for the Compton scattering model. We intend the data obtained via this observation to be a part of a larger data set. We have identified other clusters of galaxies (in archival RXTE and other instrument data sets) with sufficiently high quality data where we can search for and measure (or at least put meaningful limits) on the strength of the hard component. With these studies we expect to clarify the mechanism for acceleration of particles in the intercluster medium and provide guidance for future observations of this intriguing phenomenon by instrument on GLAST. The details of the nonthermal particle population has important implications for the theories of cluster formation, mergers and evolution. The results of this work were first presented at the High Energy Division meeting of the American astronomical Society at Mt. Tremblene, Canada (Petrosian et al. 2003). and in an invited review talk at the General Assembly of the International Astronomical Union at Sydney, Australia (Petrosian, 2003). A paper describe the observations, the data analysis and its implication is being prepared for publication in the Astrophysical Journal.

Petrosian, Vahe↗

ASA-FTL: An adaptive separation aware flash translation layer for solid state drives

Here, the flash-memory based Solid State Drive (SSD) presents a promising storage solution for increasingly critical data-intensive applications due to its low latency (high throughput), high bandwidth, and low power consumption. Within an SSD, its Flash Translation Layer (FTL) is responsible for exposing the SSD’s flash memory storage to the computer system as a simple block device. The FTL design is one of the dominant factors determining an SSD’s lifespan and performance. To reduce the garbage collection overhead and deliver better performance, we propose a new, low-cost, adaptive separation-aware flash translation layer (ASA-FTL) that combines sampling, data clustering and selective caching of recency information to accurately identify and separate hot/cold data while incurring minimal overhead. We use sampling for light-weight identification of separation criteria, and our dedicated selective caching mechanism is designed to save the limited RAM resource in contemporary SSDs. Using simulations of ASA-FTL with both real-world and synthetic workloads, we have shown that our proposed approach reduces the garbage collection overhead by up to 28% and the overall response time by 15% compared to one of the most advanced existing FTLs. We find that the data clustering using a small sample size provides significant performance benefit while only incurring a very small computation and memory cost. In addition, our evaluation shows that ASA-FTL is able to adapt to the changes in the access pattern of workloads, which is a major advantage comparing to existing fixed data separation methods.

97 MATHEMATICS AND COMPUTING↗

Multi-scale visual analysis of time-varying electrocorticography data via clustering of brain regions

There exists a need for effective and easy-to-use software tools supporting the analysis of complex Electrocorticography (ECoG) data. Understanding how epileptic seizures develop or identifying diagnostic indicators for neurological diseases require the in-depth analysis of neural activity data from ECoG. Such data is multi-scale and is of high spatio-temporal resolution. Comprehensive analysis of this data should be supported by interactive visual analysis methods that allow a scientist to understand functional patterns at varying levels of granularity and comprehend its time-varying behavior. We introduce a novel multi-scale visual analysis system, ECoG ClusterFlow, for the detailed exploration of ECoG data. Our system detects and visualizes dynamic high-level structures, such as communities, derived from the time-varying connectivity network. The system supports two major views: 1) an overview summarizing the evolution of clusters over time and 2) an electrode view using hierarchical glyph-based design to visualize the propagation of clusters in their spatial, anatomical context. We present case studies that were performed in collaboration with neuroscientists and neurosurgeons using simulated and recorded epileptic seizure data to demonstrate our system's effectiveness. ECoG ClusterFlow supports the comparison of spatio-temporal patterns for specific time intervals and allows a user to utilize various clustering algorithms. Neuroscientists can identify the site of seizure genesis and its spatial progression during various the stages of a seizure. Our system serves as a fast and powerful means for the generation of preliminary hypotheses that can be used as a basis for subsequent application of rigorous statistical methods, with the ultimate goal being the clinical treatment of epileptogenic zones.

60 APPLIED LIFE SCIENCES↗

Substructure in the stellar halo near the Sun: I. Data-driven clustering in integrals-of-motion space

Context. Merger debris is expected to populate the stellar haloes of galaxies. In the case of the Milky Way, this debris should be apparent as clumps in a space defined by the orbital integrals of motion of the stars. Aims. Our aim is to develop a data-driven and statistics-based method for finding these clumps in integrals-of-motion space for nearby halo stars and to evaluate their significance robustly. Methods. We used data from Gaia EDR3, extended with radial velocities from ground-based spectroscopic surveys, to construct a sample of halo stars within 2.5 kpc from the Sun. We applied a hierarchical clustering method that makes exhaustive use of the single linkage algorithm in three-dimensional space defined by the commonly used integrals of motion energy E, together with two components of the angular momentum, L z and L ⊥ . To evaluate the statistical significance of the clusters, we compared the density within an ellipsoidal region centred on the cluster to that of random sets with similar global dynamical properties. By selecting the signal at the location of their maximum statistical significance in the hierarchical tree, we extracted a set of significant unique clusters. By describing these clusters with ellipsoids, we estimated the proximity of a star to the cluster centre using the Mahalanobis distance. Additionally, we applied the HDBSCAN clustering algorithm in velocity space to each cluster to extract subgroups representing debris with different orbital phases. Results. Our procedure identifies 67 highly significant clusters (> 3σ), containing 12% of the sources in our halo set, and 232 subgroups or individual streams in velocity space. In total, 13.8% of the stars in our data set can be confidently associated with a significant cluster based on their Mahalanobis distance. Inspection of the hierarchical tree describing our data set reveals a complex web of relations between the significant clusters, suggesting that they can be tentatively grouped into at least six main large structures, many of which can be associated with previously identified halo substructures, and a number of independent substructures. This preliminary conclusion is further explored in a companion paper, in which we also characterise the substructures in terms of their stellar populations. Conclusions. Our method allows us to systematically detect kinematic substructures in the Galactic stellar halo with a data-driven and interpretable algorithm. The list of the clusters and the associated star catalogue are provided in two tables available at the CDS.

79 ASTRONOMY AND ASTROPHYSICS↗

Determining the Number of Clusters in a Data Set Without Graphical Interpretation

Cluster analysis is a data mining technique that is meant ot simplify the process of classifying data points. The basic clustering process requires an input of data points and the number of clusters wanted. The clustering algorithm will then pick starting C points for the clusters, which can be either random spatial points or random data points. It then assigns each data point to the nearest C point where "nearest usually means Euclidean distance, but some algorithms use another criterion. The next step is determining whether the clustering arrangement this found is within a certain tolerance. If it falls within this tolerance, the process ends. Otherwise the C points are adjusted based on how many data points are in each cluster, and the steps repeat until the algorithm converges,

Aguirre, Nathan S.↗

Performance Analysis Tool for HPC and Big Data Applications on Scientific Clusters

Big data is prevalent in HPC computing. Many HPC projects rely on complex workflows to analyze terabytes or petabytes of data. These workflows often require running over thousands of CPU cores and performing simultaneous data accesses, data movements, and computation. It is challenging to analyze the performance involving terabytes or petabytes of workflow data or measurement data of the executions, from complex workflows over a large number of nodes and multiple parallel task executions. To help identify performance bottlenecks or debug the performance issues in large-scale scientific applications and scientific clusters, we have developed a performance analysis framework, using state-ofthe- art open-source big data processing tools. Our tool can ingest system logs and application performance measurements to extract key performance features, and apply the most sophisticated statistical tools and data mining methods on the performance data. It utilizes an efficient data processing engine to allow users to interactively analyze a large amount of different types of logs and measurements. To illustrate the functionality of the big data analysis framework, we conduct case studies on the workflows from an astronomy project known as the Palomar Transient Factory (PTF) and the job logs from the genome analysis scientific cluster. Our study processed many terabytes of system logs and application performance measurements collected on the HPC systems at NERSC. The implementation of our tool is generic enough to be used for analyzing the performance of other HPC systems and Big Data workows.

97 MATHEMATICS AND COMPUTING↗

How good are polarizable and flexible models for water: Insights from a many-body perspective

We present a systematic analysis of state-of-the-art polarizable and flexible water models from a many-body perspective, with a specific focus on their ability to represent the Born–Oppenheimer potential energy surface of water from the gas to the liquid phase. Using coupled cluster data in the completed basis set limit as a reference, we examine the accuracy of the polarizable models in reproducing individual many-body contributions to interaction energies and harmonic frequencies of water clusters and compare their performance with that of MB-pol, an explicit many-body model that has been shown to correctly predict the properties of water across the entire phase diagram. Based on these comparisons, we use MB-pol as a reference to analyze the ability of the polarizable models to reproduce the energy landscape of liquid water under ambient conditions. We find that, while correctly reproducing the energetics of minimum-energy structures, the polarizable models examined in this study suffer from inadequate representations of many-body effects for distorted configurations. To investigate the role played by geometry-dependent representations of 1-body charge distributions in reproducing coupled cluster data for both interaction and many-body energies, we introduce a simplified version of MB-pol that adopts fixed atomic charges and demonstrate that the new model retains the same accuracy as the original MB-pol model. Based on the analyses presented in this study, we believe that future developments of both polarizable and explicit many-body models should continue in parallel and would benefit from synergistic efforts aimed at integrating the best aspects of the two theoretical/computational frameworks.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Optical galaxy cluster mock catalogs with realistic projection effects: Validations with the SDSS clusters

Galaxy clusters identified in optical imaging surveys suffer from projection effects: Physically unassociated galaxies along a cluster’s line of sight can be counted as its members and boost the observed richness (the number of cluster members). To model the impact of projection on cluster cosmology analyses, we apply a halo occupation distribution model to 𝑁-body simulations to simulate the red galaxies contributing to cluster members, and we use the number of galaxies in a cylinder along the line of sight (counts in cylinders) to model the impact of projection on cluster richness. We compare three projection models: uniform, quadratic, and Gaussian, and we convert between them by matching their effective cylinder volumes. We validate our mock catalogs using SDSS redMaPPer clusters’ data vectors, including counts vs richness, stacked lensing signal, spectroscopic redshift distribution of member galaxies, and richness remeasured on a redshift grid. We find the former two are insensitive to the projection model, while the latter two favor a quadratic projection model with a width of ≈180 ℎ −1 Mpc (equivalent to the volume of a uniform model with a width of 100 ℎ −1 Mpc and a Gaussian model with a width of 110 ℎ −1 Mpc, or a Gaussian redshift error of 0.04). Furthermore, our framework provides an efficient and flexible way to model optical cluster data vectors, paving the way for a simulation-based joint analysis for clusters, galaxies, and shear.

79 ASTRONOMY AND ASTROPHYSICS↗

Neutrino constraints: what large-scale structure and CMB data are telling us?

We discuss the reliability of neutrino mass constraints, either active or sterile, from the combination of different low redshift Universe probes with measurements of CMB anisotropies. In our analyses we consider WMAP 9-year or Planck Cosmic Microwave Background (CMB) data in combination with Baryonic Acoustic Oscillations (BAO) measurements from BOSS DR11, galaxy shear measurements from CFHTLenS, SDSS Ly α forest constraints and galaxy cluster mass function from Chandra observations. At odds with recent similar studies, to avoid model dependence of the constraints we perform a full likelihood analysis for all the datasets employed. As for the cluster data analysis we rely on to the most recent calibration of massive neutrino effects in the halo mass function and we explore the impact of the uncertainty in the mass bias and re-calibration of the halo mass function due to baryonic feedback processes on cosmological parameters. We find that none of the low redshift probes alone provide evidence for massive neutrino in combination with CMB measurements, while a larger than 2σ detection of non zero neutrino mass, either active or sterile, is achieved combining cluster or shear data with CMB and BAO measurements. Yet, the significance of the detection exceeds 3σ if we combine all four datasets. For a three active neutrino scenario, from the joint analysis of CMB, BAO, shear and cluster data including the uncertainty in the mass bias we obtain ∑ m{sub ν} =0.29{sup +0.18}{sub -0.21} eV and ∑ m{sub ν} =0.22{sup +0.17}{sub -0.18} eV 95%CL) using WMAP9 or Planck as CMB dataset, respectively. The preference for massive neutrino is even larger in the sterile neutrino scenario, for which we get m{sub s}{sup eff}=0.44{sup +0.28}{sub -0.26} eV and Δ N{sub eff}=0.78{sup +0.60}{sub -0.59} 95%CL) from the joint analysis of Planck, BAO, shear and cluster datasets. For this data combination the vanilla ΛCDM model is rejected at more than 3σ and a sterile neutrino mass as motivated by accelerator anomaly is within the 2σ errors. Conversely, the Ly α data favour vanishing neutrino masses and from the data combination Planck+BAO+Ly α we get the tight upper limits ∑ m{sub ν} <0.14 eV and m{sub s}{sup eff}<0.22 eV—Δ N{sub eff}<1.11 95%CL) for the active and sterile neutrino model, respectively. Finally, results from the full data combination reflect the tension between the σ{sub 8} constraints obtained from cluster and shear data and that inferred from Ly α forest measurements; in the active neutrino scenario for both CMB datasets employed, the full data combination yields only an upper limits on ∑ m{sub ν}, while assuming an extra sterile neutrino we still get preference for non-vanishing mass, m{sub s}{sup eff}=0.26{sup +0.22}{sub -0.24} eV, and dark contribution to the radiation content, Δ N{sub eff}=0.82±0.55.

79 ASTRONOMY AND ASTROPHYSICS↗