Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data analysis cluster”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 163 records · Page 9
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Tracer-field cross-correlations with k -nearest neighbour distributions

ABSTRACT In astronomy and cosmology significant effort is devoted to characterizing and understanding spatial cross-correlations between points – e.g galaxy positions, high energy neutrino arrival directions, X-ray and AGN sources, and continuous fields – e.g. weak lensing meiand Cosmic Microwave Background maps. Recently, we introduced the k-nearest neighbour (kNN) formalism to better characterize the clustering of discrete (point) data sets. Here, we extend it to the point – field cross-correlations analysis. It combines kNN measurements of the point data set with measurements of the field smoothed at many scales. The resulting statistics are sensitive to all orders in the joint clustering of the points and the field. We demonstrate that this approach, unlike the 2-pt cross-correlation, can measure the statistical dependence of two data sets even when there are no linear (Gaussian) correlations between them. We further demonstrate that this framework is far more effective than the two point function in detecting cross-correlations when the continuous field is contaminated by high levels of noise. For a particularly high level of noise, the cross-correlation between haloes and the underlying matter field in a cosmological simulation, between 10 h−1 Mpc and 30 h−1 Mpc, is detected at >5σ significance using the technique presented here, when the two-point cross-correlation significance is ∼1σ. Finally, we show that kNN cross-correlations of haloes and the matter field can be well modelled on quasi-linear scales using the Hybrid Effective Field Theory (HEFT) framework, with the same set of bias parameters as are used for 2-pt cross-correlations. The substantial improvement in the statistical power of detecting cross-correlations using this method makes it a promising tool for various cosmological applications.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2023

Exploring the stellar populations of backsplash galaxies

Backsplash galaxies are those that traverse and overshoot cluster cores as they fall into these structures. They are affected by environment, and should stand out in contrast to the infalling population. We target galaxies in the vicinity of clusters (R≳R 200 ) and select a sample in projected phase space (PPS), from the compilation of Sampaio et al. based on SDSS data. We present a statistical analysis, comparing two regions in PPS, with the same projected distance to the cluster but different velocity. The analysis relies on the presence of variations in the stellar population content of backsplash galaxies. We find a lower limit in the fractional contribution of ∼5 per cent with respect to the general sample of infalling galaxies at similar group-centric distance when using single line strength analysis, or ∼15-30 per cent when adopting bivariate distributions. The stellar populations show a subtle but significant difference towards older ages, and a higher fraction of quiescent galaxies. We also compare this set with a general field sample, where a substantially larger difference in galaxy properties is found, with the field sample being consistently younger, metal poorer and with a lower fraction of quiescent galaxies. Noting that our ‘cluster’ sample is located outside of the virial radius, we expect this difference to be caused by pre-processing of the infalling galaxies in the overall higher density regions.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2025

SCULPT (Supervised Clustering and Uncovering Latent Patterns with Training) v1

SCULPT (Supervised Clustering and Uncovering Latent Patterns with Training) is a comprehensive data visualization and analysis application focused on working with COLTRIMS (COLd Target Recoil Ion Momentum Spectroscopy) data, which is used in atomic and molecular physics experiments. The application offers several powerful features: - Data uploading and processing capabilities for COLTRIMS files - Multiple visualization methods using UMAP (Uniform Manifold Approximation and Projection) for dimensionality reduction - Interactive selection of data points across multiple views - Feature engineering through various methods: - Manual feature selection from calculated physics parameters - Deep autoencoder for dimension reduction - Genetic programming for discovering meaningful features - Mutual information-based feature selection - Multiple clustering approaches (DBSCAN, KMeans, Agglomerative) - Quality metrics for evaluating clustering results - Export capabilities for selections and generated features

Daoud, Hazem [Lawrence Berkeley National Laborator↗
DOE OSTI2020

Phase identification using co‐association matrix ensemble clustering

Calibrating distribution system models to aid in the accuracy of simulations such as hosting capacity analysis is increasingly important in the pursuit of the goal of integrating more distributed energy resources. The recent availability of smart meter data is enabling the use of machine learning tools to automatically achieve model calibration tasks. This research focuses on applying machine learning to the phase identification task, using a co‐association matrix‐based, ensemble spectral clustering approach. The proposed method leverages voltage time series from smart meters and does not require existing or accurate phase labels. This work demonstrates the success of the proposed method on both synthetic and real data, surpassing the accuracy of other phase identification research.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2025

Validation of the DESI-DR1 3x2-pt analysis: scale cut and shear ratio tests

Combined survey analyses of galaxy clustering and weak gravitational lensing (3x2-pt studies) will allow new and accurate tests of the standard cosmological model. However, careful validation is necessary to ensure that these cosmological constraints are not biased by uncertainties associated with the modelling of astrophysical or systematic effects. In this study we validate the combined 3x2-pt analysis of the Dark Energy Spectroscopic Instrument Data Release 1 (DESI-DR1) spectroscopic galaxy clustering and overlapping weak lensing datasets from the Kilo-Degree Survey (KiDS), the Dark Energy Survey (DES), and the Hyper-Suprime-Cam Survey (HSC). By propagating the modelling uncertainties associated with the non-linear matter power spectrum, non-linear galaxy bias and baryon feedback, we design scale cuts to ensure that measurements of the matter density and the amplitude of the matter power spectrum are biased by less than 30% of the statistical error. We also test the internal consistency of the data and weak lensing systematics by performing new measurements of the lensing shear ratio. We demonstrate that the DESI-DR1 shear ratios can be successfully fit by the same model used to describe cosmic shear correlations, and analyse the additional information that can be extracted about the source redshift distributions and intrinsic alignment parameters. This study serves as crucial preparation for the upcoming cosmological parameter analysis of these datasets.

Emas, N. [Swinburne U., Ctr. Astrophys. Supercompu↗
DOE OSTI2022

An overview of the operation architectures and energy management system for multiple microgrid clusters

The emerging novel energy infrastructures, such as energy communities, smart building-based microgrids, electric vehicles enabled mobile energy storage units raise the requirements for a more interconnective and interoperable energy system. It leads to a transition from simple and isolated microgrids to relatively large-scale and complex interconnected microgrid systems named multi-microgrid clusters. In order to efficiently, optimally, and flexibly control multi-microgrid clusters, cross-disciplinary technologies such as power electronics, control theory, optimization algorithms, information and communication technologies, cyber-physical, and big-data analysis are needed. This paper introduces an overview of the relevant aspects for multi-microgrids, including the outstanding features, architectures, typical applications, existing control mechanisms, as well as the challenges.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2024

Topological Interpretability for Deep Learning

With the growing adoption of AI-based systems across everyday life, the need to understand their decision-making mechanisms is correspondingly increasing. The level at which we can trust the statistical inferences made from AI-based decision systems is an increasing concern, especially in high-risk systems such as criminal justice or medical diagnosis, where incorrect inferences may have tragic consequences. Despite their successes in providing solutions to problems involving real-world data, deep learning (DL) models cannot quantify the certainty of their predictions. These models are frequently quite confident, even when their solutions are incorrect. This work presents a method to infer prominent features in two DL classification models trained on clinical and non-clinical text by employing techniques from topological and geometric data analysis. We create a graph of a model's feature space and cluster the inputs into the graph's vertices by the similarity of features and prediction statistics. We then extract subgraphs demonstrating high-predictive accuracy for a given label. These subgraphs contain a wealth of information about features that the DL model has recognized as relevant to its decisions. We infer these features for a given label using a distance metric between probability measures, and demonstrate the stability of our method compared to the LIME and SHAP interpretability methods. This work establishes that we may gain insights into the decision mechanism of a DL model. This method allows us to ascertain if the model is making its decisions based on information germane to the problem or identifies extraneous patterns within the data.

Spannaus, Adam↗
DOE OSTI2025

Examining Clustered MCSs and Their Precipitation Significance Over Global Land MCS Hotspots

Multiple independent mesoscale convective systems (MCSs) can occur closely in space and time, subjecting common geographic regions to multiple precipitation events over short periods. Motivated by an earlier study highlighting significant hydrological impacts of clustered MCSs in the US, we extend the analysis to seven global continental MCS hotspots using a 20‐year global MCS data set. Our findings reveal that (a) 40%–80% of MCSs occur in clusters, depending on the region; (b) clustered MCSs exhibit larger rainfall areas and higher mean areal rainfall, with the highest values observed in Argentina and India; and (c) clustered MCSs contribute increasing fractions to more extreme 24‐ and 72‐hr precipitation events across all regions. Our results underscore the critical role of clustered MCSs in driving hydrological extremes over global land regions and highlight the need to better understand how their formation mechanisms differ from non‐clustered events across the globe.

Hu, Huancui [Pacific Northwest National Laboratory↗
DOE OSTI2021

The Completed SDSS-IV Extended Baryon Oscillation Spectroscopic Survey: N-body Mock Challenge for Galaxy Clustering Measurements

We develop a series of N-body data challenges, functional to the final analysis of the extended Baryon Oscillation Spectroscopic Survey (eBOSS) Data Release 16 (DR16) galaxy sample. The challenges are primarily based on high-fidelity catalogues constructed from the Outer Rim simulation - a large box size realization (3h(-1) Gpc) characterized by an unprecedented combination of volume and mass resolution, down to 1.85 x 10(9) h(-1)M(circle dot). We generate synthetic galaxy mocks by populating Outer Rim haloes with a variety of halo occupation distribution (HOD) schemes of increasing complexity, spanning different redshift intervals. We then assess the performance of three complementary redshift space distortion (RSD) models in configuration and Fourier space, adopted for the analysis of the complete DR16 eBOSS sample of Luminous Red Galaxies (LRG5). We find all the methods mutually consistent, with comparable systematic errors on the Alcock-Paczynski parameters and the growth of structure, and robust to different HOD prescriptions - thus validating the robustness of the models and the pipelines used for the baryon acoustic oscillation (BAO) and full shape clustering analysis. In particular, all the techniques are able to recover and alpha(11) to within 0.9 per cent, and f sigma(8) to within 1.5 per cent. As a by-product of our work, we are also able to gain interesting insights on the galaxy-halo connection. Our study is relevant for the final eBOSS DR16 'consensus cosmology', as the systematic error budget is informed by testing the results of analyses against these high-resolution mocks. In addition, it is also useful for future large-volume surveys, since similar mock-making techniques and systematic corrections can be readily extended to model for instance the Dark Energy Spectroscopic Instrument (DESI) galaxy sample.

cosmology: theory, large-scale structure of Univer↗
DOE OSTI2024

Genetic and Functional Diversity Help Explain Pathogenic, Weakly Pathogenic, and Commensal Lifestyles in the Genus Xanthomonas

The genus Xanthomonas has been primarily studied for pathogenic interactions with plants. However, besides host and tissue-specific pathogenic strains, this genus also comprises nonpathogenic strains isolated from a broad range of hosts, sometimes in association with pathogenic strains, and other environments, including rainwater. Based on their incapacity or limited capacity to cause symptoms on the host of isolation, nonpathogenic xanthomonads can be further characterized as commensal and weakly pathogenic. This study aimed to understand the diversity and evolution of nonpathogenic xanthomonads compared to their pathogenic counterparts based on their cooccurrence and phylogenetic relationship and to identify genomic traits that form the basis of a life history framework that groups xanthomonads by ecological strategies. We sequenced genomes of 83 strains spanning the genus phylogeny and identified eight novel species, indicating unexplored diversity. While some nonpathogenic species have experienced a recent loss of a type III secretion system, specifically the hrp2 cluster, we observed an apparent lack of association of the hrp2 cluster with lifestyles of diverse species. We performed association analysis on a large data set of 337 Xanthomonas strains to explain how xanthomonads may have established association with the plants across the continuum of lifestyles from commensals to weak pathogens to pathogens. Presence of distinct transcriptional regulators, distinct nutrient utilization and assimilation genes, transcriptional regulators, and chemotaxis genes may explain lifestyle-specific adaptations of xanthomonads.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2021

Globular Cluster Candidates in the Sagittarius Dwarf Galaxy

Recently, new Sagittarius (Sgr) dwarf-galaxy globular clusters were discovered, which opens the question of the actual size of the Sgr globular cluster population, and therefore on our understanding of the Sgr galaxy formation and accretion history of the Milky Way. Based on Gaia EDR3 and SDSS IV DR16 (APOGEE-2) data sets, we performed an analysis of the color–magnitude diagrams (CMDs) of the eight new Sgr globular clusters found by Minniti et al. from a sound cleaning of the contamination of Milky Way and Sgr field stars, complemented by available kinematic and metal abundance information. The cleaned CMDs and spatial stellar distibutions reveal the presence of stars with a wide range of cluster membership probabilities. Minni 332 turned out to be a younger (<9 Gyr) and more metal-rich ([M/H] ≳ -1.0 dex) globular cluster than M54, the nuclear Sgr globular cluster; as could also be the case of Minni 342, 348, and 349, although their results are less convincing. Minni 341 could be an open cluster candidate (age < 1 Gyr, [M/H] ~ -0.3 dex), while the analyses of Minni 335, 343, and 344 did not allow us to confirm their physical reality. We also built the Sgr cluster frequency (CF) using available ages of the Sgr globular clusters and compared it with that obtained from the Sgr star formation history. Both CFs are in excellent agreement. However, the addition of eight new globular clusters with ages and metallicities distributed according to the Sgr age–metallicity relationship turns out in a remarkably different CF.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2021

The WaZP galaxy cluster sample of the dark energy survey year 1

We present a new (2+1)D galaxy cluster finder based on photometric redshifts called Wavelet Z Photometric (WaZP) applied to DES first year (Y1A1) data. The results are compared to clusters detected by the South Pole Telescope (SPT) survey and the redMaPPer cluster finder, the latter based on the same photometric data. WaZP searches for clusters in wavelet-based density maps of galaxies selected in photometric redshift space without any assumption on the cluster galaxy populations. The comparison to other cluster samples was performed with a matching algorithm based on angular proximity and redshift difference of the clusters. It led to the development of a new approach to match two optical cluster samples, following an iterative approach to minimize incorrect associations. The WaZP cluster finder applied to DES Y1A1 galaxy survey (1511.13 deg^2 up to = 23 mag) led to the detection of 60 547 galaxy clusters with redshifts 0.05 < z < 0.9 and richness N_gals ≥ 5. Considering the overlapping regions and redshift ranges between the DES Y1A1 and SPT cluster surveys, all sz based SPT clusters are recovered by the WaZP sample. The comparison between WaZP and redMaPPer cluster samples showed an excellent overall agreement for clusters with richness N_gals (λ for redMaPPer) greater than 25 (20), with 95 per cent recovery on both directions. Based on the cluster cross-match, we explore the relative fragmentation of the two cluster samples and investigate the possible signatures of unmatched clusters.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2024

Atomically Precise Single-Site Catalysts via Exsolution in a Polyoxometalate–Metal–Organic-Framework Architecture

Single-site catalysts (SSCs) achieve a high catalytic performance through atomically dispersed active sites. A challenge facing the development of SSCs is aggregation of active catalytic species. Reducing the loading of these sites to very low levels is a common strategy to mitigate aggregation and sintering; however, this limits the tools that can be used to characterize the SSCs. Here we report a sintering-resistant SSC with high loading that is achieved by incorporating Anderson–Evans polyoxometalate clusters (POMs, MMo 6 O 24 , M = Rh/Pt) within NU-1000, a Zr-based metal–organic framework (MOF). The dual confinement provided by isolating the active site within the POM, then isolating the POMs within the MOF, facilitates the formation of isolated noble metal sites with low coordination numbers via exsolution from the POM during activation. The high loading (up to 3.2 wt %) that can be achieved without sintering allowed the local structure transformation in the POM cluster and the surrounding MOF to be evaluated using in situ X-ray scattering with pair distribution function (PDF) analysis. Notably, the Rh/Pt···Mo distance in the active catalyst is shorter than the M···M bond lengths in the respective bulk metals. Furthermore, models of the active cluster structure were identified based on the PDF data with complementary computation and X-ray absorption spectroscopy analysis.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2024

Bragg glass signatures in Pd x ErTe 3 with X-ray diffraction temperature clustering

The Bragg glass phase is a nearly perfect crystal with glassy features predicted to occur in vortex lattices and charge-density-wave systems in the presence of disorder. Detecting it has been challenging, despite its sharp theoretical definition in terms of diverging correlation lengths. Here we present bulk probe evidence supporting a Bragg glass phase in the systematically disordered charge-density-wave material of Pd x ErTe 3 . We do this by using comprehensive X-ray data and a machine-learning-based analysis tool called X-ray diffraction temperature clustering (X-TEC). We establish a diverging correlation length in samples with moderate intercalation over a wide temperature range. To enable this analysis, we introduced a high-throughput measure of inverse correlation length that we call peak spread. The detection of Bragg glass order and the resulting phase diagram advance our understanding of the complex interplay between disorder and fluctuations. Moreover, the use of our analysis technique to target fluctuations through a high-throughput measure of peak spread can revolutionize the study of fluctuations in scattering experiments.

36 MATERIALS SCIENCE↗
DOE OSTI2020

The pantropical response of soil moisture to El Niño

Abstract. The 2015–2016 El Niño event ranks as one of the most severe on record in terms of the magnitude and extent of sea surface temperature (SST) anomalies generated in the tropical Pacific Ocean. Corresponding global impacts on the climate were expected to rival, or even surpass, those of the 1997–1998 severe El Niño event, which had SST anomalies that were similar in size. However, the 2015–2016 event failed to meet expectations for hydrologic change in many areas, including those expected to receive well above normal precipitation. To better understand how climate anomalies during an El Niño event impact soil moisture, we investigate changes in soil moisture in the humid tropics (between ±25∘) during the three most recent super El Niño events of 1982–1983, 1997–1998 and 2015–2016, using data from the Global Land Data Assimilation System (GLDAS). First, we use in situ soil moisture observations obtained from 16 sites across five continents to validate and bias-correct estimates from GLDAS (r2=0.54). Next, we apply a k-means cluster analysis to the soil moisture estimates during the El Niño mature phase, resulting in four groups of clustered data. The strongest and most consistent decreases in soil moisture occur in the Amazon basin and maritime southeastern Asia, while the most consistent increases occur over eastern Africa. In addition, we compare changes in soil moisture to both precipitation and evapotranspiration, which showed a lack of agreement in the direction of change between these variables and soil moisture most prominently in the southern Amazon basin, the Sahel and mainland southeastern Asia. Our results can be used to improve estimates of spatiotemporal differences in El Niño impacts on soil moisture in tropical hydrology and ecosystem models at multiple scales.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

Data Science Techniques, Assumptions, and Challenges in Alloy Clustering and Property Prediction

Data analytics methods have been increasingly applied to understanding materials chemistry, processing due to the manufacturing approach, and uni-axial and cyclic property relationships in the highly complex space of alloy design. There are several benefits to applying data analytics to this space, including the ability to manage non-linearities in the responses of the alloy attributes and the resulting mechanical properties. However, key difficulties in applying and understanding the results of data analytics include the often lack of reported assumptions and data processing steps necessary to improve interpretation and reproducibility in derived results. In this work, the methods used to generate clustering and correlation analyses for experimental 9% Cr ferritic-martensitic steel data were investigated and the resulting implications for mechanical property predictions were assessed. This work uses principal component analysis, partitioning around medoids, t-SNE, and k-means clustering to investigate trends in composition, processing and microstructure information with creep and tensile properties, building on work done previously using a smaller version of the same dataset. The initial assumptions, preprocessing steps and methods are investigated and outlined in order to depict the fine level of detail required to convey the steps taken to process data and produce analytical results. Here, the variations in the resulting analyses are explored due to the influence of new and more varied data.

36 MATERIALS SCIENCE↗
DOE OSTI2021

The persistence of large scale structures. Part I. Primordial non-Gaussianity

Abstract We develop an analysis pipeline for characterizing the topology of large scale structure and extracting cosmological constraints based on persistent homology . Persistent homology is a technique from topological data analysis that quantifies the multiscale topology of a data set, in our context unifying the contributions of clusters, filament loops, and cosmic voids to cosmological constraints. We describe how this method captures the imprint of primordial local non-Gaussianity on the late-time distribution of dark matter halos, using a set of N-body simulations as a proxy for real data analysis. For our best single statistic, running the pipeline on several cubic volumes of size 40 (Gpc/h) 3 , we detect f NL loc =10 at 97.5% confidence on ~ 85% of the volumes. Additionally wetest our ability to resolve degeneracies betweenthe topological signature of f NL loc and variation of σ 8 and argue that correctly identifying nonzero f NL loc in this case is possible via an optimal template method. Our method relies on information living at $\mathcal{O}$(10) Mpc/h, a complementary scale with respect to commonly used methods such as the scale-dependent bias in the halo/galaxy power spectrum. Therefore, while still requiring a large volume, our method does not require sampling long-wavelength modes to constrain primordial non-Gaussianity. Moreover, our statistics are interpretable: we are able to reproduce previous results in certain limits and we make new predictions for unexplored observables, such as filament loops formed by dark matter halos in a simulation box.

Astronomy & Astrophysics↗
DOE OSTI2020

Clustering earthquake signals and background noises in continuous seismic data with unsupervised deep learning

The continuously growing amount of seismic data collected worldwide is outpacing our abilities for analysis, since to date, such datasets have been analyzed in a human-expert intensive, supervised fashion. Moreover, analyses that are conducted can be strongly biased by the standard models employed by seismologists. In response to both of these challenges, we develop a new unsupervised machine learning framework for detecting and clustering seismic signals in continuous seismic records. Our approach combines a deep scattering network and a Gaussian mixture model to cluster seismic signal segments and detect novel structures. To illustrate the power of the framework, we analyze seismic data acquired during the June 2017 Nuugaatsiaq, Greenland landslide. We demonstrate the blind detection and recovery of the repeating precursory seismicity that was recorded before the main landslide rupture, which suggests that our approach could lead to more informative forecasting of the seismic activity in seismogenic areas.

59 BASIC BIOLOGICAL SCIENCES↗