Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data analysis cluster”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 235 records · Page 13
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Characterization of DESI fiber assignment incompleteness effect on 2-point clustering and mitigation methods for DR1 analysis

We present an in-depth analysis of the fiber assignment incompleteness in the Dark EnergySpectroscopic Instrument (DESI) Data Release 1 (DR1). This incompleteness is caused by therestricted mobility of the robotic fiber positioner in the DESI focal plane, which limits thenumber of galaxies that can be observed at the same time, especially at small angular separations.As a result, the observed clustering amplitude is suppressed in a scale-dependent manner, which,if not addressed, can severely impact the inference of cosmological parameters. We discuss themethods adopted for simulating fiber assignment on mocks and data. In particular, we introducethe fast fiber assignment (FFA) emulator, which was employed to obtain the power spectrumcovariance adopted for the DR1 full-shape analysis. We present the mitigation techniques,organised in two classes: measurement stage and model stage. We then use high fidelity mocks as areference to quantify both the accuracy of the FFA emulator and the effectiveness of the differentmeasurement-stage mitigation techniques. This complements the studies conducted in a parallelpaper for the model-stage techniques, namely the θ-cut approach. We find that pairwiseinverse probability (PIP) weights with angular upweighting recover the “true” clustering in allthe cases considered, in both Fourier and configuration space. Notably, we present the first everpower spectrum measurement with PIP weights from real data.

79 ASTRONOMY AND ASTROPHYSICS↗
NASA NTRS1988

Rocketdyne automated dynamics data analysis and management system

An automated dynamics data analysis and management systems implemented on a DEC VAX minicomputer cluster is described. Multichannel acquisition, Fast Fourier Transformation analysis, and an online database have significantly improved the analysis of wideband transducer responses from Space Shuttle Main Engine testing. Leakage error correction to recover sinusoid amplitudes and correct for frequency slewing is described. The phase errors caused by FM recorder/playback head misalignment are automatically measured and used to correct the data. Data compression methods are described and compared. The system hardware is described. Applications using the data base are introduced, including software for power spectral density, instantaneous time history, amplitude histogram, fatigue analysis, and rotordynamics expert system analysis.

Tarn, Robert B.↗
NASA NTRS2004

Gas Density Discontinuities in Merging Clusters

In a number of galaxy clusters, Chandra has discovered a new phenomenon, the sharp gas density edges. Depending on the sign of the temperature jump across the edge, these features may either be bow shocks or cold fronts. While the merger origin of the bow shocks is unambiguous, what causes cold fronts is not entirely clear, as they are observed both in mergers and in relaxed clusters. The purpose of the study of A3376, an interesting cluster with density edges, is to understand the origin of cold fronts and to look for possible shocks. This cluster also has a radio halo, and comparison of an X-ray temperature map and radio image may shed light on the nature of the cluster radio halos, which is our secondary goal. The XMM data for A3376 have been obtained, processed and cleaned. Most of the preliminary analysis, including deriving a temperature map, is complete. We also have Chandra data for this cluster, and intend to compare and combine the two datasets. We are well underway toward determining the dynamical state of A3376 and the nature of its gas density discontinuities.

Mushotzky, Richard↗
DOE OSTI2021

Interpreting internal consistency of DES measurements

ABSTRACT Bayesian evidence ratios are widely used to quantify the statistical consistency between different experiments. However, since the evidence ratio is prior dependent, the precise translation between its value and the degree of concordance/discordance requires additional information. The most commonly adopted metric, the Jeffreys scale, can falsely suggest agreement between data sets when priors are chosen to be sufficiently wide. This work examines evidence ratios in a DES-Y1 simulated analysis, focusing on the internal consistency between weak lensing and galaxy clustering. We study two scenarios using simulated data in controlled experiments. First, we calibrate the expected evidence ratio distribution given noise realizations around the best-fitting DES-Y1 ΛCDM cosmology. Secondly, we show the behaviour of evidence ratios for noiseless fiducial data vectors simulated using a modified gravity model, which generates internal tension in the LCDM analysis. We find that the evidence ratio of noise realizations generated at all confidence levels was biased towards agreement and show, with a modified gravity model, that the choice of prior could conceal the discrepancies between weak lensing and galaxy clustering induced by prior effects in unlike cosmological models, concluding that the evidence ratio in a DES-Y1 study is, indeed, biased towards agreement. Boundary effects can also influence conclusions about the inconsistency induced by modified gravity, even in a noiseless data vector simulation.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2024

A network approach for multiscale catchment classification using traits

Abstract. The classification of river catchments into groups with similar biophysical characteristics is useful to understand and predict their hydrological behavior. The increasing availability of remote sensing and other large-scale geospatial datasets has enabled the use of advanced data-driven approaches to classify catchments using traits such as topography, geology, climate, land cover, land use, and human influence. Unsupervised clustering algorithms based on the Euclidean distance are commonly used for trait-based classification but are not suitable for highly dimensional data. In this study we present a new network-based method for multi-scale catchment classification, which can be applied to large datasets and used to determine the traits associated with different catchment groups. In this framework, two networks are analyzed in parallel: the first being where the nodes are traits and the second being where the nodes are catchments. In both cases, edges represent pairwise similarity, and a network cluster detection algorithm is used for the classification. The trait network is used to investigate redundancy in the trait data and to condense this information into a small number of interpretable categories. The catchments network is used to classify the catchments into clusters and to identify representative catchments for the different groups using the degree centrality metric. We apply this method to classify 9067 river catchments across the contiguous United States at both regional and continental scales using 274 non-categorical traits. At the continental scale, we identify 25 interpretable trait categories and 34 catchment clusters of sizes greater than 50. We find that catchments with similar trait categories are typically located in the same region, with different spatial patterns emerging among clusters dominated by natural and anthropogenic traits. We also find that the catchment clusters exhibit distinct hydrological behavior based on an analysis of streamflow indices. This network approach provides several advantages over traditional means of classification, including better separation of clusters, the use of alternate similarity metrics that are more suitable for highly dimensional data, and reducing redundancy in the trait information. The paired catchment–trait networks enable analysis of hydrological behavior using the dominant trait categories for each catchment cluster. The approach can be used at multiple spatial scales since the network topologies adjust automatically to reflect the trait patterns at the scale of investigation. Finally, the representative catchments identified as hub nodes in the network can be used to guide transferable observational and modeling strategies. The method is broadly applicable beyond hydrology for classification of other complex systems that utilize different types of trait datasets.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2026

Clustering analysis of medium-band selected high-redshift galaxies

Next-generation large-scale structure spectroscopic surveys will probe cosmology at high redshifts (2.3 < z < 3.5), relying on abundant galaxy tracers such as Lyα emitters (LAEs) and Lyman break galaxies (LBGs). Medium-band photometry has emerged as a potential technique for efficiently selecting these high-redshift galaxies. In this work, we present clustering analysis of medium-band selected galaxies at high redshift, utilizing photometric data from the Intermediate Band Imaging Survey (IBIS) and spectroscopic data from the Dark Energy Spectroscopic Instrument (DESI). We interpret the clustering of such samples using both Halo Occupation Distribution (HOD) modeling and a perturbation theory description of large-scale structure. Our modeling indicates that the current target sample is composed from an overlapping mixture of LAEs and LBGs with emission lines. Despite differences in target selection, we find that the clustering properties are consistent with previous studies, with correlation lengths r 0 ≃ 3-4 h -1 Mpc and a linear bias of b ∼ 1.8-2.5. Finally, we discuss the simulation requirements implied by these measurements and demonstrate that the properties of the samples would make them excellent targets to enhance our understanding of the high-z universe.

79 ASTRONOMY AND ASTROPHYSICS↗
NASA NTRS1989

A color gradient in the globular cluster M15

A U-B color map of the central regions of the globular cluster M15 is constructed from data taken at the CFHT in subarcsec seeing. Analysis of the color distribution of the pixels in this map unambiguously demonstrates that the area within 6.6 arcsec of the center of the cluster is bluer than surrounding regions. The difference is apparently caused by a population of faint blue stars more prevalent in the core than elsewhere, and is consistent with post-core-collapse models which predict a centrally condensed distribution of cataclysmic variables.

Bailyn, C. D.↗
DOE OSTI2021

Critical Role of the Interfacial Layer in Associating Polymers with Microphase Separation

Polymers with dynamic (transient) bonds, often called associating polymers, have been attracting significant attention in recent years due to their unique viscoelastic properties, self-healing ability, and recyclability. Nevertheless, understanding the mechanisms and the factors controlling their macroscopic properties remains limited due to the higher complexity introduced by the dynamic bonds. Here in this study, small-angle X-ray scattering (SAXS), broadband dielectric spectroscopy (BDS), and rheology were applied to unravel the structure and dynamics of telechelic associating polymers with different molecular weights. SAXS measurements revealed phase separation of the functional end groups with an average cluster size of ~2–3 nm and the distance between clusters controlled by the chain length. Borrowing the interfacial layer model analysis of BDS data from the polymer nanocomposite field, we demonstrated the presence of an interfacial polymer layer with a thickness of ~0.7–0.9 nm surrounding these clusters. Rheological measurements showed quantitatively that the presence of the interfacial layer significantly alters the viscoelastic behavior of these materials, indicating the crucial role of the interfacial layer in defining the macroscopic mechanical properties of the studied telechelic materials. The presented results emphasize that phase separation of the functional groups in associating polymers leads to very significant changes of the viscoelastic properties, opening a promising avenue in the design of novel functional materials.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
NASA NTRS1982

Classification of simulated and actual NOAA-6 AVHRR data for hydrologic land-surface feature definition

An examination of the possibilities of using Landsat data to simulate NOAA-6 Advanced Very High Resolution Radiometer (AVHRR) data on two channels, as well as using actual NOAA-6 imagery, for large-scale hydrological studies is presented. A running average was obtained of 18 consecutive pixels of 1 km resolution taken by the Landsat scanners were scaled up to 8-bit data and investigated for different gray levels. AVHRR data comprising five channels of 10-bit, band-interleaved information covering 10 deg latitude were analyzed and a suitable pixel grid was chosen for comparison with the Landsat data in a supervised classification format, an unsupervised mode, and with ground truth. Landcover delineation was explored by removing snow, water, and cloud features from the cluster analysis, and resulted in less than 10% difference. Low resolution large-scale data was determined useful for characterizing some landcover features if weekly and/or monthly updates are maintained.

Ormsby, J. P.↗
NASA NTRS1984

Analysis methods for Thematic Mapper data of urban regions

Studies have indicated the difficulty in deriving a detailed land-use/land-cover classification for heterogeneous metropolitan areas with Landsat MSS and TM data. The major methodological issues of digital analysis which possibly have effected the results of classification are examined. In response to these methodological issues, a multichannel hierarchical clustering algorithm has been developed and tested for a more complete analysis of the data for urban areas.

Wang, S. C.↗
NASA NTRS1989

Segmentation of multifrequency polarimetric radar images to facilitate the inference of geophysical parameters

An unsupervised clustering algorithm is used to segment multifrequency polarimetric radar data from the NASA/JPL airborne SAR (synthetic aperture radar). Twenty-two parameters are evaluated for their discriminatory capability for each pixel of an image. A clustering analysis is then performed using different subsets of these parameters. This analysis relies on data taken as part of an intensive field experiment during the summer of 1988 in the vicinity of the Pisgah lava flow in the Mojave Desert in southern California. As part of the experiment, extensive ground truth was acquired, including dielectric constant and topography measurements. Segmentation results show good agreement with these measurements.

Burnette, C F.↗
NASA NTRS2011

High-Performance Data Analysis Tools for Sun-Earth Connection Missions

The data analysis tool of choice for many Sun-Earth Connection missions is the Interactive Data Language (IDL) by ITT VIS. The increasing amount of data produced by these missions and the increasing complexity of image processing algorithms requires access to higher computing power. Parallel computing is a cost-effective way to increase the speed of computation, but algorithms oftentimes have to be modified to take advantage of parallel systems. Enhancing IDL to work on clusters gives scientists access to increased performance in a familiar programming environment. The goal of this project was to enable IDL applications to benefit from both computing clusters as well as graphics processing units (GPUs) for accelerating data analysis tasks. The tool suite developed in this project enables scientists now to solve demanding data analysis problems in IDL that previously required specialized software, and it allows them to be solved orders of magnitude faster than on conventional PCs. The tool suite consists of three components: (1) TaskDL, a software tool that simplifies the creation and management of task farms, collections of tasks that can be processed independently and require only small amounts of data communication; (2) mpiDL, a tool that allows IDL developers to use the Message Passing Interface (MPI) inside IDL for problems that require large amounts of data to be exchanged among multiple processors; and (3) GPULib, a tool that simplifies the use of GPUs as mathematical coprocessors from within IDL. mpiDL is unique in its support for the full MPI standard and its support of a broad range of MPI implementations. GPULib is unique in enabling users to take advantage of an inexpensive piece of hardware, possibly already installed in their computer, and achieve orders of magnitude faster execution time for numerically complex algorithms. TaskDL enables the simple setup and management of task farms on compute clusters. The products developed in this project have the potential to interact, so one can build a cluster of PCs, each equipped with a GPU, and use mpiDL to communicate between the nodes and GPULib to accelerate the computations on each node.

Messmer, Peter↗
DOE OSTI2026

An evaluation of air quality in major urban areas of India

Rapid economic growth and burgeoning population have contributed to enhanced levels of PM 2.5 concentrations in urban regions of India. Evaluation of ambient air quality facilitates the assessment of effectiveness of emission control measures and early identification of new sources. This study provides a comprehensive statistical analysis of PM 2.5 concentrations in key urban areas across India, including Delhi, Kolkata, Mumbai, Chennai, Hyderabad, and several regional centers. Data from 2017 to 2023 was analyzed using trend analysis, cluster analysis, principal component analysis, and geostatistical interpolation to understand spatiotemporal variations and sources. The analysis reveals significant differences in spatial distribution of PM 2.5 concentrations with high annual averages in urban regions in Indo-Gangetic plain (82–123 μg m −3 ) and relatively lower concentrations (29–46 μg m −3 ) in southern urban areas of Kerala, Tamil Nadu and Andhra Pradesh. Delhi state had the highest 24-averaged PM 2.5 concentrations (112 μg m −3 ) followed by urban regions in Uttar Pradesh, Bihar and West Bengal (94 μg m −3 ). Trend analysis from 2017 to 2023 revealed an overall 2.5% decline in site-wide PM2.5 concentrations, with the exception of Ludhiana, which exhibited a consistent annual increase of 10%. Principal component analysis (PCA) attributes 30% of the variance to wintertime emissions, 13% to biomass burning, and 18% to the regional haze in the northern Indo-Gangetic Plain. Different analyses clearly demonstrates the contribution of biomass burning to pollution in Delhi and surrounding cities. Transboundary pollution to Kolkata is likely from the highly polluted region in Indo-Gangetic Plain. Coastal cities of Mumbai and Chennai has relatively lower pollution attributed to the influence of sea breeze dilution, with mostly local contribution and some potential transport from upwind industry clusters. Hyderabad also has local contribution due to high density of vehicular traffic and local small industries. This study shows that mitigation efforts targeting clusters of regions should be undertaken to curb the high PM2.5 pollution. Policy measures should be implemented both at local and the intra-state level to address shared sources and transport of pollution.

Hysplitbacktrajectories↗
DOE OSTI2023

BigNeuron: a resource to benchmark and predict performance of algorithms for automated tracing of neurons in light microscopy datasets

BigNeuron is an open community bench-testing platform with the goal of setting open standards for accurate and fast automatic neuron tracing. We gathered a diverse set of image volumes across several species that is representative of the data obtained in many neuroscience laboratories interested in neuron tracing. Here, we report generated gold standard manual annotations for a subset of the available imaging datasets and quantified tracing quality for 35 automatic tracing algorithms. The goal of generating such a hand-curated diverse dataset is to advance the development of tracing algorithms and enable generalizable benchmarking. Together with image quality features, we pooled the data in an interactive web application that enables users and developers to perform principal component analysis, t-distributed stochastic neighbor embedding, correlation and clustering, visualization of imaging and tracing data, and benchmarking of automatic tracing algorithms in user-defined data subsets. The image quality metrics explain most of the variance in the data, followed by neuromorphological features related to neuron size. Furthermore, we observed that diverse algorithms can provide complementary information to obtain accurate results and developed a method to iteratively combine methods and generate consensus reconstructions. The consensus trees obtained provide estimates of the neuron structure ground truth that typically outperform single algorithms in noisy datasets. However, specific algorithms may outperform the consensus tree strategy in specific imaging conditions. Finally, to aid users in predicting the most accurate automatic tracing results without manual annotations for comparison, we used support vector machine regression to predict reconstruction quality given an image volume and a set of automatic tracings.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Characterization of DESI fiber assignment incompleteness effect on 2-point clustering and mitigation methods for DR1 analysis

We present an in-depth analysis of the fiber assignment incompleteness in the Dark Energy Spectroscopic Instrument (DESI) Data Release 1 (DR1). This incompleteness is caused by the restricted mobility of the robotic fiber positioner in the DESI focal plane, which limits the number of galaxies that can be observed at the same time, especially at small angular separations. As a result, the observed clustering amplitude is suppressed in a scale-dependent manner, which, if not addressed, can severely impact the inference of cosmological parameters. We discuss the methods adopted for simulating fiber assignment on mocks and data. In particular, we introduce the fast fiber assignment (FFA) emulator, which was employed to obtain the power spectrum covariance adopted for the DR1 full-shape analysis. We present the mitigation techniques, organised in two classes: measurement stage and model stage. We then use high fidelity mocks as a reference to quantify both the accuracy of the FFA emulator and the effectiveness of the different measurement-stage mitigation techniques. This complements the studies conducted in a parallel paper for the model-stage techniques, namely the θ-cut approach. We find that pairwise inverse probability (PIP) weights with angular upweighting recover the “true” clustering in all the cases considered, in both Fourier and configuration space. Notably, we present the first ever power spectrum measurement with PIP weights from real data.

cosmological simulations↗
NASA NTRS1976

Study of systems and techniques for data base management

Data management areas were studied to identify pertinent problems and issues that will affect future NASA data users in terms of performance and cost. Specific topics discussed include the identifications of potential NASA data users other than those normally discussed, consideration affecting the clustering of minicomputers, low cost computer system for information retrieval and analysis, the testing of minicomputer based data base management systems, ongoing work related to the use of dedicated systems for data base management, and the problems of data interchange among a community of NASA data users.

Source record↗
DOE OSTI2020

Coarse-grain cluster analysis of tensors with application to climate biome identification

A tensor provides a concise way to codify the interdependence of complex data. Treating a tensor as a d-way array, each entry records the interaction between the different indices. Clustering provides a way to parse the complexity of the data into more readily understandable information. Clustering methods are heavily dependent on the algorithm of choice, as well as the chosen hyperparameters of the algorithm. However, their sensitivity to data scales is largely unknown. In this work, we apply the discrete wavelet transform to analyze the effects of coarse-graining on clustering tensor data. We are particularly interested in understanding how scale affects clustering of the Earth's climate system. The discrete wavelet transform allows classification of the Earth's climate across a multitude of spatial-temporal scales. The discrete wavelet transform is used to produce an ensemble of classification estimates, as opposed to a single classification. Each element of the ensemble is a clustering at a different spatial-temporal scale. Information theoretic approaches are used to identify important scale lengths in clustering the L15 Climate Datset. We also discover a sub-collection of the ensemble that spans the majority of the variance observed, allowing for efficient consensus clustering techniques that can be used to identify climate biomes.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

Characterizing the Effect of Capillary Heterogeneity on Multiphase Flow Pulsation in an Intermediate–Scale Beadpack Experiment using Time Series Clustering and Frequency Analysis

An intermediate-scale beadpack drainage experiment was conducted to investigate how simple layered lamination heterogeneity affects CO 2 flow. Two simple layers of capillary barriers are manually packed in the tank and slow drainage was carried out using analog fluids to mimic the capillary- and gravity-dominated CO 2 upward migration process in deep saline aquifers. Nonwetting phase saturation time series clustering analysis and frequency analysis have been conducted on the experimental data. Additionally, modified invasion percolation numerical simulations were done on a digital model of the beadpack to compare to experimental results. Results show that capillary barriers can lead to strong pulsation behavior, which in turn can cause unexpected early breaching through other barriers. The inlet pressure is found to be able to respond to saturation changes in far regions of the domain, indicating that the wetting phase can transmit pressure changes from the other phase. Although static simulations were not able to capture all the dynamic behavior observed in the experiment, Monte Carlo composite simulation results combining many different realizations can better illustrate how the nonwetting phase will behave in the heterogeneous domain. Furthermore, our results suggest the need for CO 2 storage site selection with preference given to aquifers with more capillary barriers with finer grain sizes to avoid flow pulsation and to retard plume upward migration.

58 GEOSCIENCES↗