Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data analysis cluster”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 145 records · Page 8
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

Redshift Evolution of the Feedback–Cooling Equilibrium in the Core of 48 SPT Galaxy Clusters: A Joint Chandra–SPT–ATCA Analysis

We analyze the cooling and feedback properties of 48 galaxy clusters at redshifts 0.4 < z < 1.3 selected from the South Pole Telescope (SPT) catalogs to evolve like the progenitors of massive and well-studied systems at z ~ 0. We estimate the radio power at the brightest cluster galaxy (BCG) location of each cluster from an analysis of Australia Telescope Compact Array data. Assuming that the scaling relation between the radio power and active galactic nucleus (AGN) cavity power P cav observed at low redshift does not evolve with redshift, we use these measurements in order to estimate the expected AGN cavity power in the core of each system. We estimate the X-ray luminosity within the cooling radius L cool of each cluster from a joint analysis of the available Chandra X-ray and SPT Sunyaev–Zel'dovich (SZ) data. This allows us to characterize the redshift evolution of the P cav /L cool ratio. When combined with low-redshift results, these constraints enable investigations of the properties of the feedback–cooling cycle across 9 Gyr of cluster growth. We model the redshift evolution of this ratio measured for cool-core clusters by a log-normal distribution Log-N (α + βz, σ 2 ) and constrain the slope of the mean evolution to β = -0.05 ± 0.47. This analysis improves the constraints on the slope of this relation by a factor of two. We find no evidence of redshift evolution of the feedback–cooling equilibrium in these clusters, which suggests that the onset of radio-mode feedback took place at an early stage of cluster formation. High values of P cav /L cool are found at the BCG location of noncool-core clusters, which might suggest that the timescales of the AGN feedback cycle and the cool core–noncool core transition are different. This work demonstrates that the joint analysis of radio, SZ, and X-ray data solidifies the investigation of AGN feedback at high redshifts.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2026

Dark Energy Survey Year 6 Results: Weak Lensing and Galaxy Clustering Cosmological Analysis Framework

We present the methodology for the weak lensing and galaxy clustering analyses of the Dark Energy Survey (DES) Year 6 data set. In this work, we design and validate the analysis pipeline for the cosmic shear, galaxy clustering plus galaxy$-$galaxy lensing ($2 \times 2$pt), and the joint analysis in the $3 \times 2$pt. Our framework accounts for key theoretical uncertainties, such as baryonic feedback and galaxy bias, incorporating both linear and non-linear models. We apply scale cuts in regimes where theoretical modeling becomes unreliable. The robustness of the pipeline is validated using mock data and simulations, confirming unbiased cosmological constraints and highlighting the importance of posterior projection effects in the validation process. As a result, we deliver robust and validated analysis pipelines for cosmic shear, $2 \times 2$pt, and $3 \times 2$pt in $Λ$CDM and $w$CDM scenarios, including a well-defined set of scales suitable for real data analysis, a robust prescription for theoretical systematics, and the theoretical covariance of the signal. This comprehensive methodology also lays the groundwork for future galaxy surveys such as the Vera C. Rubin Observatory Legacy Survey of Space and Time.

Sanchez-Cid, D. [Zurich U.; Madrid, CIEMAT; Madrid↗
NASA NTRS2000

The Distance and Mass of the Galaxy Cluster Abell 1995 Derived From Sunyaev-Zel'dovich Effect and X-Ray Measurements

We present multi-wavelength observations of the Abell 1995 galaxy cluster. From analysis of x-ray spectroscopy and imaging data we derive the electron temperature, cluster core radius, and central electron number density. Using optical spectroscopy of 15 cluster members, we derive an accurate cluster redshift and velocity dispersion. Finally, the interferometric imaging of the SZE toward Abell 1995 at 28.5 GHz provides a measure of the integrated pressure through the cluster.

Patel, Sandeep K.↗
NASA NTRS1985

Method for evaluating wind turbine wake effects on wind farm performance

A method of testing the performance of a cluster of wind turbine units an data analysis equations are presented which together form a simple and direct procedure for determining the reduction in energy output caused by the wake of an upwind turbine. This method appears to solve the problems presented by data scatter and wind variability. Test data from the three-unit Mod-2 wind turbine cluster at Goldendale, Washington, are analyzed to illustrate the application of the proposed method. In this sample case the reduction in energy was found to be about 10 percent when the Mod-2 units were separated a distance equal to seven diameters and winds were below rated.

Neustadter, H. E.↗
DOE OSTI2022

Solving the structure of “single-atom” catalysts using machine learning – assisted XANES analysis

We show that "single-atom” catalysts (SACs) have demonstrated excellent activity and selectivity in challenging chemical transformations such as photocatalytic CO 2 reduction. For heterogeneous photocatalytic SAC systems, it is essential to obtain sufficient information of their structure at the atomic level in order to understand reaction mechanisms. In this work, a SAC was prepared by grafting a molecular cobalt catalyst on a light-absorbing carbon nitride surface. Due to the sensitivity of the X-ray absorption near edge structure (XANES) spectra to subtle variances in the Co SAC structure in reaction conditions, different machine learning (ML) methods, including principal component analysis, K-means clustering, and neural network (NN), were utilized for in situ Co XANES data analysis. As a result, we obtained quantitative structural information of the SAC nearest atomic environment thereby extending the NN-XANES approach previously demonstrated for nanoparticles and size-selective clusters.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
NASA NTRS1988

Microwave remote sensing of sea ice

The long term objectives are: (1) to understand the physics of the multispectral microwave radiative characteristics of sea ice as it goes through different phases; (2) to improve characterization of sea ice cover using satellite microwave sensors; and (3) to study ice/ocean physical and biological processes associated with polynya formations and variability of the marginal sea ice region. Two field experiments were conducted to pursue these objectives. One involved measurements of radiative and physical characteristics of sea ice from a ship during a 3-month long cruise through the Weddell Sea ice pack during the Austral winter of 1986. The other involved similar measurements from two aircrafts and a submarine over the Central Arctic and Greenland Sea region. Preliminary results have already led to an enhanced understanding of the microwave signatures of pancake ice, nilas, first year ice, multiyear ice and effects of snow cover. Coastal and deep ocean polynyas and their role in bottom water formation and ocean circulation were studied using a time series of ice images from SMMR. An unsupervised cluster analysis of Arctic sea ice using SMMR and THIR emissivity and brightness temperature data was implemented. The analysis indicates the existence of several unique and persistent clusters in the Central Arctic region during winter and that the sum of the area of these clusters excluding those of first year ice is about 20 percent less than minimum ice cover area inferred from a previous summer data. This result is consistent with saline surface for some multiyear ice floes as observed during MIZEZ and suggests that a significant fraction of multiyear ice floes in the Arctic have first year ice signatures.

Comiso, J. C.↗
NASA NTRS1986

Spectral characteristics and the extent of paleosols of the Palouse formation

Spectral relationships were investigated for several bare soil fields which were in summer fallow rotation on the date of the imagery. Printouts of each band were examined and compared to aerial photography. Bands with dissimilar reflectance patterns for known areas were then combined using ratio techniques which were proven useful in other studies (Williams, 1983). Selected ratios were Thematic Mapper (TM) 1/TM4, TM3/TM4, and TM5/TM4. Cluster analyses and Baysian and Fastclass classifier images were produced using the three ratio images. Plots of cluster analysis outputs revealed distinct groupings of reflectance data representing green crops, ripened crops, soil and green plants, and bare soil. Bare soil was represented by a line of clusters on plots of the ratios TM5/TM4 and TM3/TM4. The soil line was investigated further to determine factors involved in the distributin of clusters alone the line. The clusters representing the bare soil line were also studied by plotting the Tm5/TM4, TM1/TM4 dimension. A total of 76 soil samples were gathered and analyzed for organic carbon.

Frazier, B. E.↗
NASA NTRS2013

Hadoop for High-Performance Climate Analytics: Use Cases and Lessons Learned

Scientific data services are a critical aspect of the NASA Center for Climate Simulations mission (NCCS). Hadoop, via MapReduce, provides an approach to high-performance analytics that is proving to be useful to data intensive problems in climate research. It offers an analysis paradigm that uses clusters of computers and combines distributed storage of large data sets with parallel computation. The NCCS is particularly interested in the potential of Hadoop to speed up basic operations common to a wide range of analyses. In order to evaluate this potential, we prototyped a series of canonical MapReduce operations over a test suite of observational and climate simulation datasets. The initial focus was on averaging operations over arbitrary spatial and temporal extents within Modern Era Retrospective- Analysis for Research and Applications (MERRA) data. After preliminary results suggested that this approach improves efficiencies within data intensive analytic workflows, we invested in building a cyber infrastructure resource for developing a new generation of climate data analysis capabilities using Hadoop. This resource is focused on reducing the time spent in the preparation of reanalysis data used in data-model inter-comparison, a long sought goal of the climate community. This paper summarizes the related use cases and lessons learned.

analytics↗
NASA NTRS2012

Preliminary Evaluation of MapReduce for High-Performance Climate Data Analysis

MapReduce is an approach to high-performance analytics that may be useful to data intensive problems in climate research. It offers an analysis paradigm that uses clusters of computers and combines distributed storage of large data sets with parallel computation. We are particularly interested in the potential of MapReduce to speed up basic operations common to a wide range of analyses. In order to evaluate this potential, we are prototyping a series of canonical MapReduce operations over a test suite of observational and climate simulation datasets. Our initial focus has been on averaging operations over arbitrary spatial and temporal extents within Modern Era Retrospective- Analysis for Research and Applications (MERRA) data. Preliminary results suggest this approach can improve efficiencies within data intensive analytic workflows.

Duffy, Daniel Q.↗
NASA NTRS2000

The Distance and Mass of the Galaxy Cluster Abell 1995 Derived from Sunyaev-Zeldovich Effect and X-Ray Measurements

We present multiwavelength observations of the Abell 1995 galaxy cluster. From an analysis of X-ray spectroscopy and imaging data, we derive the electron temperature, cluster core radius, and central electron number density. Using optical spectroscopy of 15 cluster members, we derive an accurate cluster redshift and velocity dispersion. Finally, the interferometric imaging of the Sunyaev-Zeldovich effect toward Abell 1995 at 28.5 GHz provides a measure of the integrated pressure through the cluster. The X-ray and Sunyaev-Zeldovich effect observations are combined to determine the angular diameter distance to the cluster of D(sub A) = 1294(sup +294 +438, sub -283 -458) Mpc (Statistical followed by systematic uncertainty), implying a Hubble constant of H(sub 0) = 52.2(sup +11.4 +18.5, sub -11.9 -17.7) km/s.Mpc for Omega(sub M) = 0.3 and Omega(sub lambda) = 0.7. We find a best-fit H(sub 0) of 46 km/s.Mpc for the Omega(sub M) = 1 and Omega(sub lambda) = 0 cosmology, and 48 km/s.Mpc for Omega(sub M) = 0.3 and Omega(sub lambda) = 0.0. The X-ray data are also used to derive a total cluster mass of M(sup HSE, sub tot)(r(sub 500)) = 5.18(sup +0.62, sub -0.48) x 10(exp 14)/h solar mass; the optical velocity dispersion yields an independent and consistent estimate of M(sup virial, sub tot)(r(sub 500)) = 6.35(sup +1.51, sub -1.19) X 10(exp 14) /h solar mass. Both of the total mass estimates are evaluated at a fiducial radius, r(sub 500) = 830 /h kpc, where the overdensity is 500 times the critical density. The total cluster mass is then combined with gas mass measurements to determine a cluster gas mass fraction of F(sub g) = 0.056(sup +0.010, sub -0.013) /h(sup 3/2) in combination with recent baryon density constraints, the measured gas mass fraction yields an upper limit on the mass density parameter of Omega(sub M) h(sup 1/2) <= 0.34(sup +/0.06, sub 0.05.

Patel, Sandeep K.↗
DOE OSTI2020

Seismology in the Cloud: A New Streaming Workflow

Data-intensive research in seismology is experiencing a recent boom, driven in part by large volumes of available data and advances in the growing field of data science. However, there are significant barriers to processing large data volumes, such as long retrieval times from data repositories, complex data management, and limited computational resources. New tools and platforms have reduced the barriers to entry for scientific cluster computing, including the maturation of the commercial cloud as an accessible instrument for research. Here, we build a customized research cluster in the cloud to test a new workflow for large-scale seismic analysis, in which data are processed as a stream (retrieved on-the-fly and acted upon without storing), with data from the Incorporated Research Institutions for Seismology Data Management Center. We use this workflow to deploy a spectral peak detection algorithm over 5.6 TB of compressed continuous seismic data from 2074 stations of the USArray Transportable Array EarthScope network. Using a 50-node cluster in the cloud, we completed the noise survey in 80 hr, with an average data throughput of 1.7 GB per minute. By varying cluster sizes, we find the scaling of our analysis to be sublinear, due to a combination of algorithmic limitations and data center response times. The cloud-based streaming workflow represents an order-of-magnitude increase in acquisition and processing speed compared to a traditional download-store-process workflow, and offers the additional benefits of employing a flexible, accessible, and widely used computing architecture. It is limited, however, due to its reliance on Internet transfer speeds and data center service capacity, and may not work well for repeated analyses or those for which even higher data throughputs are needed. These research applications will require a new class of cloud-native approaches in which both data and analysis are in the cloud.

58 GEOSCIENCES↗
DOE OSTI2025

Optical galaxy cluster mock catalogs with realistic projection effects: Validations with the SDSS clusters

Galaxy clusters identified in optical imaging surveys suffer from projection effects: Physically unassociated galaxies along a cluster’s line of sight can be counted as its members and boost the observed richness (the number of cluster members). To model the impact of projection on cluster cosmology analyses, we apply a halo occupation distribution model to 𝑁-body simulations to simulate the red galaxies contributing to cluster members, and we use the number of galaxies in a cylinder along the line of sight (counts in cylinders) to model the impact of projection on cluster richness. We compare three projection models: uniform, quadratic, and Gaussian, and we convert between them by matching their effective cylinder volumes. We validate our mock catalogs using SDSS redMaPPer clusters’ data vectors, including counts vs richness, stacked lensing signal, spectroscopic redshift distribution of member galaxies, and richness remeasured on a redshift grid. We find the former two are insensitive to the projection model, while the latter two favor a quadratic projection model with a width of ≈180 ℎ −1 Mpc (equivalent to the volume of a uniform model with a width of 100 ℎ −1 Mpc and a Gaussian model with a width of 110 ℎ −1 Mpc, or a Gaussian redshift error of 0.04). Furthermore, our framework provides an efficient and flexible way to model optical cluster data vectors, paving the way for a simulation-based joint analysis for clusters, galaxies, and shear.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2022

Mitigate: An Adaptive Network Data Anonymization Tool Using Condensation-Based Differential Privacy

Modern network devices collect a large amount of data that can be analyzed to identify bottlenecks, anomalies, cyber-attacks, etc. Therefore, there is often a need to analyze such collections of network data quite often by an external expert or by the research community. However, these collections of data contain sensitive, proprietary information. In order for the network data to be shared, it must first be anonymized. The overall objective of this project is to develop an innovative privacy management tool to anonymize network data and achieve sufficient privacy, acceptable data utility, and efficient data analysis at the same time. No existing anonymization methods can achieve all of these at the same time. The core of this technology is a differential private clustering algorithm that provides strong privacy protection, preserves data properties important for subsequent analysis, and allows the party receiving the anonymized data to conduct analysis directly on anonymized data without the need of decryption or any extra processing. The research carried out was to design, implement and verify a solution to this problem by completing the following tasks: 1) developing the core technology; 2) developing a context based method that automatically recommends fields that must be anonymized; 3) conducted experiments showing superior results using our approach compared to existing tools, and 4) developed an intuitive but basic user interface. The research that was conducted generated novel algorithmic techniques that utilize state-of-the-art methods such as condensation, differential privacy preservation, clustering, automated tuning based on contextual awareness, and recommendation techniques to specify columns to users for anonymization leading to optimal privacy that allows research analysis on the dataset. Experiments were conducted to evaluate the efficacy of these novel algorithmic techniques by performing analysis on original non-anonymized datasets, then conducting analysis on the same yet anonymized datasets and comparing the results of the analyses. Overall, the anonymized analysis results were within 1% of the original results, verifying that the generated technology not only guarantees a high level of privacy but also enables research analysis as if it were conducted on the original dataset. Potential applications of this technology include anonymization of any type of structured network datasets that contain sensitive identifiers, such as IP addresses, that can be used in multiple applications. For example, to create an AI or machine learning model for cyber security, e.g., to detect attacks, or for performance analysis, e.g., identify bottlenecks or predict performance. In addition, a market analysis that was conducted for potential applications of this technology identified a broader range of applications of our anonymization technology beyond the network sector that includes healthcare, banking, insurance, securities, finance (FISB), data brokering, cloud services, ad sales, and government.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1988

Einstein Observatory solid state detector observations of cooling flows in clusters of galaxies

Einstein Observatory (EO) solid state spectrometer results for the cooling flows in nine clusters of galaxies are discussed. An analysis of the data using three separate models provides strong evidence for the existence of cool components in the centers of these clusters. The obtained distribution of emission versus temperature for M87 is found to agree well with that determined from data obtained by the EO focal Plane Crystal Spectrometer.

Mushotzky, R. F.↗
DOE OSTI2021

Galaxy clustering in harmonic space from the dark energy survey year 1 data: compatibility with real-space results

ABSTRACT We perform an analysis in harmonic space of the Dark Energy Survey Year 1 Data (DES-Y1) galaxy clustering photometric data using products obtained for the real-space analysis. We test our pipeline with a suite of lognormal simulations, which are used to validate scale cuts in harmonic space as well as to provide a covariance matrix that takes into account the DES-Y1 mask. We then apply this pipeline to DES-Y1 data taking into account survey property maps derived for the real-space analysis. We compare with real-space DES-Y1 results obtained from a similar pipeline. We show that the harmonic space analysis we develop yields results that are compatible with the real-space analysis for the bias parameters. This verification paves the way to performing a harmonic space analysis for the upcoming DES-Y3 data.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2024

GeoCricket

SAND2025-12229O Geospatial Critical Infrastructure and Census Data Stockpile Tool (GeoCricket) is a set of functions that collect critical infrastructure and census data for use in the Resilient Node Cluster Analysis Tool (ReNCAT) and Quantum Geographic Information System Social Burden Calculator. It can also act to inform other place-based work. The code queries public-facing Representational State Transfer (REST) servers to collect geospatial data related to a specific area. It then exports that data as standard geographic information system file types or as a .csv file. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525.

Haines, John↗
DOE OSTI2024

Fast mass spectrometry search and clustering of untargeted metabolomics data

The throughput of mass spectrometers and the amount of publicly available metabolomics data are growing rapidly, but analysis tools such as molecular networking and Mass Spectrometry Search Tool do not scale to searching and clustering billions of mass spectral data in metabolomics repositories. To address this limitation, we designed MASST+ and Networking+, which can process datasets that are up to three orders of magnitude larger than those processed by state-of-the-art tools.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2026

Unsupervised atomic data mining via multi-kernel graph autoencoders for machine learning force fields

Constructing a chemically diverse dataset while avoiding sampling bias is critical to training efficient and generalizable force fields. However, in computational chemistry and materials science, many common dataset generation techniques are prone to oversampling regions of the potential energy surface. Furthermore, these regions can be difficult to identify and isolate from each other or may not align well with human intuition, making it challenging to systematically remove bias in the dataset. While traditional clustering and pruning (down-sampling) approaches can be useful for this, they can often lead to information loss or a failure to properly identify distinct regions of the potential energy surface due to difficulties associated with the high dimensionality of atomic descriptors. In this work, we introduce the Multi-kernel Edge Attention-based Graph Autoencoder (MEAGraph) model, an unsupervised approach for analyzing atomic datasets. MEAGraph combines multiple linear kernel transformations with attention-based message passing to capture geometric sensitivity and enable effective dataset pruning without relying on labels or extensive training. Demonstrated applications on niobium, tantalum, and iron datasets show that MEAGraph efficiently groups similar atomic environments, allowing for the use of basic pruning techniques for removing sampling bias. This approach provides an effective method for representation learning and clustering that can be used for data analysis, outlier detection, and dataset optimization.

Materials science↗