Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data analysis cluster”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 127 records · Page 7

Classification of simulated and actual NOAA-6 AVHRR data for hydrologic land-surface feature definition

An examination of the possibilities of using Landsat data to simulate NOAA-6 Advanced Very High Resolution Radiometer (AVHRR) data on two channels, as well as using actual NOAA-6 imagery, for large-scale hydrological studies is presented. A running average was obtained of 18 consecutive pixels of 1 km resolution taken by the Landsat scanners were scaled up to 8-bit data and investigated for different gray levels. AVHRR data comprising five channels of 10-bit, band-interleaved information covering 10 deg latitude were analyzed and a suitable pixel grid was chosen for comparison with the Landsat data in a supervised classification format, an unsupervised mode, and with ground truth. Landcover delineation was explored by removing snow, water, and cloud features from the cluster analysis, and resulted in less than 10% difference. Low resolution large-scale data was determined useful for characterizing some landcover features if weekly and/or monthly updates are maintained.

Ormsby, J. P.↗

Analysis methods for Thematic Mapper data of urban regions

Studies have indicated the difficulty in deriving a detailed land-use/land-cover classification for heterogeneous metropolitan areas with Landsat MSS and TM data. The major methodological issues of digital analysis which possibly have effected the results of classification are examined. In response to these methodological issues, a multichannel hierarchical clustering algorithm has been developed and tested for a more complete analysis of the data for urban areas.

Wang, S. C.↗

Segmentation of multifrequency polarimetric radar images to facilitate the inference of geophysical parameters

An unsupervised clustering algorithm is used to segment multifrequency polarimetric radar data from the NASA/JPL airborne SAR (synthetic aperture radar). Twenty-two parameters are evaluated for their discriminatory capability for each pixel of an image. A clustering analysis is then performed using different subsets of these parameters. This analysis relies on data taken as part of an intensive field experiment during the summer of 1988 in the vicinity of the Pisgah lava flow in the Mojave Desert in southern California. As part of the experiment, extensive ground truth was acquired, including dielectric constant and topography measurements. Segmentation results show good agreement with these measurements.

Burnette, C F.↗

High-Performance Data Analysis Tools for Sun-Earth Connection Missions

The data analysis tool of choice for many Sun-Earth Connection missions is the Interactive Data Language (IDL) by ITT VIS. The increasing amount of data produced by these missions and the increasing complexity of image processing algorithms requires access to higher computing power. Parallel computing is a cost-effective way to increase the speed of computation, but algorithms oftentimes have to be modified to take advantage of parallel systems. Enhancing IDL to work on clusters gives scientists access to increased performance in a familiar programming environment. The goal of this project was to enable IDL applications to benefit from both computing clusters as well as graphics processing units (GPUs) for accelerating data analysis tasks. The tool suite developed in this project enables scientists now to solve demanding data analysis problems in IDL that previously required specialized software, and it allows them to be solved orders of magnitude faster than on conventional PCs. The tool suite consists of three components: (1) TaskDL, a software tool that simplifies the creation and management of task farms, collections of tasks that can be processed independently and require only small amounts of data communication; (2) mpiDL, a tool that allows IDL developers to use the Message Passing Interface (MPI) inside IDL for problems that require large amounts of data to be exchanged among multiple processors; and (3) GPULib, a tool that simplifies the use of GPUs as mathematical coprocessors from within IDL. mpiDL is unique in its support for the full MPI standard and its support of a broad range of MPI implementations. GPULib is unique in enabling users to take advantage of an inexpensive piece of hardware, possibly already installed in their computer, and achieve orders of magnitude faster execution time for numerically complex algorithms. TaskDL enables the simple setup and management of task farms on compute clusters. The products developed in this project have the potential to interact, so one can build a cluster of PCs, each equipped with a GPU, and use mpiDL to communicate between the nodes and GPULib to accelerate the computations on each node.

Messmer, Peter↗

Study of systems and techniques for data base management

Data management areas were studied to identify pertinent problems and issues that will affect future NASA data users in terms of performance and cost. Specific topics discussed include the identifications of potential NASA data users other than those normally discussed, consideration affecting the clustering of minicomputers, low cost computer system for information retrieval and analysis, the testing of minicomputer based data base management systems, ongoing work related to the use of dedicated systems for data base management, and the problems of data interchange among a community of NASA data users.

Source record↗

Reindeer range inventory in western Alaska from computer-aided digital classification of LANDSAT data

An inventory of reindeer-range resources was conducted for the USDA Soil Conservation Service of 1.6 million hectares of wildlands in western Alaska using clustering techniques with digital Landsat data. Computer-aided digital analysis produced a provisional map of rangeland types which was used to design the field collection of vegetation and soil types data. This field data facilitated refinement of the inventory map and was used to describe the map units. The informational classes important to range resources were wet, moist and alpine tundra, tidal marsh, brush and open spruce forest. A significant feature of the study was the extraction of acreage figures by administrative boundaries within the study area. In addition to soil and vegetation association map products (at scales of 1:250,000 and 1:63,360) acreage values were tallied from the digital data for each of the four grazing permit areas established by the Bureau of Land Management.

George, T. H.↗

Use of ordination and classification procedures to evaluate phytoplankton communities during Superflux II

Cluster analysis and an ordination procedure were performed on two data matrices to investigate real and environmental spatial relationships. Multiple regression analysis was used to relate the measured environmental variables to the phytoplankton community changes. Qualitative type phytoplankton data proved to be less structured in both of these spaces, relative to the biomass data. The salinity gradients of the northern transects covaried significantly with the phytoplankton association changes. In the southern transects the light variable was most important in explaining the variance in the ordination axes. These data suggest the close relationships between phytoplankton community changes and the physical hydrology of the area.

Rutledge, C. K.↗

An ALMA+ACA measurement of the shock in the Bullet Cluster

Context: The thermal Sunyaev-Zeldovich (SZ) effect presents a relatively new tool for characterizing galaxy cluster merger shocks, traditionally studied through X-ray observations. Widely regarded as the “textbook example” of a cluster merger bow shock, the western, most-prominent shock front in the Bullet Cluster (1E0657-56) represents the ideal test case for such an SZ study. Aims: We aim to characterize the shock properties using deep, high-resolution interferometric SZ effect observations in combination with priors from an independent X-ray analysis. Methods: Our analysis technique relies on the reconstruction of a parametric model for the SZ signal by directly and jointly fitting data from the Atacama Large Millimeter/submillimeter Array (ALMA) and Atacama Compact Array (ACA) in Fourier space. Results: The ALMA+ACA data are primarily sensitive to the electron pressure difference across the shock front. To estimate the shock Mach number M, this difference can be combined with the value for the upstream electron pressure derived from an independent Chandra X-ray analysis. In the case of instantaneous electron-ion temperature equilibration, we find M = 2.08 (+0.12,−0.12), in ≈ 2.4σ tension with the independent constraint from Chandra, M(X)= 2.74 ± 0.25. The assumption of purely adiabatic electron temperature change across the shock leads to M = 2.53 (+0.33,−0.25), in better agreement with the X-ray estimate M(X) = 2.57 ± 0.23 derived for the same heating scenario. Conclusion: We have demonstrated that interferometric observations of the thermal SZ effect provide constraints on the properties of the shock in the Bullet Cluster that are highly complementary to X-ray observations. The combination of X-ray and SZ data yields a powerful probe of the shock properties, capable of measuring M and addressing the question of electron-ion equilibration in cluster shocks. Our analysis is however limited by systematics related to the overall cluster geometry and the complexity of the post-shock gas distribution. To overcome these limitations, a simultaneous, joint-likelihood analysis of SZ and X-ray data is needed.

Luca Di Mascolo↗

Hybrid Collaborative Learning for Classification and Clustering in Sensor Networks

Traditionally, nodes in a sensor network simply collect data and then pass it on to a centralized node that archives, distributes, and possibly analyzes the data. However, analysis at the individual nodes could enable faster detection of anomalies or other interesting events as well as faster responses, such as sending out alerts or increasing the data collection rate. There is an additional opportunity for increased performance if learners at individual nodes can communicate with their neighbors. In previous work, methods were developed by which classification algorithms deployed at sensor nodes can communicate information about event labels to each other, building on prior work with co-training, self-training, and active learning. The idea of collaborative learning was extended to function for clustering algorithms as well, similar to ideas from penta-training and consensus clustering. However, collaboration between these learner types had not been explored. A new protocol was developed by which classifiers and clusterers can share key information about their observations and conclusions as they learn. This is an active collaboration in which learners of either type can query their neighbors for information that they then use to re-train or re-learn the concept they are studying. The protocol also supports broadcasts from the classifiers and clusterers to the rest of the network to announce new discoveries. Classifiers observe an event and assign it a label (type). Clusterers instead group observations into clusters without assigning them a label, and they collaborate in terms of pairwise constraints between two events [same-cluster (mustlink) or different-cluster (cannot-link)]. Fundamentally, these two learner types speak different languages. To bridge this gap, the new communication protocol provides four types of exchanges: hybrid queries for information, hybrid "broadcasts" of learned information, each specified for classifiers-to-clusterers, and clusterers-to-classifiers. The new capability has the potential to greatly expand the in situ analysis abilities of sensor networks. Classifiers seeking to categorize incoming data into different types of events can operate in tandem with clusterers that are sensitive to the occurrence of new kinds of events not known to the classifiers. In contrast to current approaches that treat these operations as independent components, a hybrid collaborative learning system can enable them to learn from each other.

Wagstaff, Kiri L.↗

A Global End-Member Approach to Derive aCDOM(440) from Near-Surface Optical Measurements

This study establishes an optical inversion scheme for deriving the absorption coefficient of colored (or chromophoric, depending on the literature) dissolved organic material (CDOM) at the 440 nm wavelength, which can be applied to global water masses with near-equal efficacy. The approach uses a ratio of diffuse attenuation coefficient spectral end members, i.e., a short and long wavelength pair. The global perspective is established by sampling "extremely" clear water plus a generalized extent in turbidity and optical properties that each span three decades of dynamic range. A unique data set was collected in oceanic, coastal, and inland waters (as shallow as 0.6 m) from the North Pacific Ocean, the Arctic Ocean, Hawaii, Japan, Puerto Rico, and the east and west coasts of the United States. The data were partitioned using subjective categorizations to define a validation quality subset of conservative water masses, i.e., the inflow and outflow of properties constrain the range in the gradient of a constituent, plus 15 subcategories of water masses that were not evolving conservatively. The dependence on subcategories was confirmed with an objective methodology based on cluster analysis techniques. The latter defined five distinct classes with validation quality data present in all classes, but which also decreased in percent composition as a function of increasing class number and optical complexity. Four different algorithms based on different validation quality end members were validated with accuracies of 1.–6.2 %, wherein pairs with the largest spectral span were most accurate. Although algorithm accuracy decreased with the inclusion of more subcategories containing non-conservative water masses, changes to the algorithm fit were small when a preponderance of subcategories were included. The high accuracy for all end-member algorithms was the result of data acquisition and data processing improvements, e.g., increased vertical sampling resolution to less than 1mm and a boundary constraint to mitigate wave focusing effects, respectively. An independent evaluation with a historical database confirmed the consistency of the algorithmic approach and its application to quality assurance, e.g., to flag data outside expected ranges, identify suspect spectra, and objectively determine the in-water extrapolation interval by converging agreement for all applicable end-member algorithms. The legacy data exhibit degraded performance (as 44 % uncertainty) due to a lack of high-quality near-surface observations, especially for clear waters wherein wave-focusing effects are problematic. The novel optical approach allows the in situ estimation of an in-water constituent in keeping with the accuracy obtained in the laboratory.

Stanford B Hooker↗

Telescope Scientist on the Advanced X-Ray Astrophysics Observatory

This period included many scientific observations made with the Chandra Observatory. The results, as is well known, are spectacular. Fortunately, the HRMA performance continues to be essentially identical to that predicted from ground calibration data. The Telescope Scientist Team has improved the mirror model to provide a more accurate description to the Chandra observers and enable them to reduce the systematic errors and uncertainties in their data reduction. There also has been progress in the scientific program. At this time 47 distant clusters of galaxies have been observed. We are performing a systematic analysis of this rather large data set for the purpose of determining absolute distances utilizing the Sunyaev Zel'dovich effect.

VanSpeybroeck, L.↗

Autonomous image data reduction by analysis and interpretation

Image data is a critical component of the scientific information acquired by space missions. Compression of image data is required due to the limited bandwidth of the data transmission channel and limited memory space on the acquisition vehicle. This need becomes more pressing when dealing with multispectral data where each pixel may comprise 300 or more bytes. An autonomous, real time, on-board image analysis system for an exploratory vehicle such as a Mars Rover is developed. The completed system will be capable of interpreting image data to produce reduced representations of the image, and of making decisions regarding the importance of data based on current scientific goals. Data from multiple sources, including stereo images, color images, and multispectral data, are fused into single image representations. Analysis techniques emphasize artificial neural networks. Clusters are described by their outlines and class values. These analysis and compression techniques are coupled with decision making capacity for determining importance of each image region. Areas determined to be noise or uninteresting can be discarded in favor of more important areas. Thus limited resources for data storage and transmission are allocated to the most significant images.

Eberlein, Susan↗

Autonomous image data reduction by analysis and interpretation

Image data is a critical component of the scientific information acquired by space missions. Compression of image data is required due to the limited bandwidth of the data transmission channel and limited memory space on the acquisition vehicle. This need becomes more pressing when dealing with multispectral data where each pixel may comprise 300 or more bytes. An autonomous, real time, on-board image analysis system for an exploratory vehicle such as a Mars Rover is developed. The completed system will be capable of interpreting image data to produce reduced representations of the image, and of making decisions regarding the importance of data based on current scientific goals. Data from multiple sources, including stereo images, color images, and multispectral data, are fused into single image representations. Analysis techniques emphasize artificial neural networks. Clusters are described by their outlines and class values. These analysis and compression techniques are coupled with decision-making capacity for determining importance of each image region. Areas determined to be noise or uninteresting can be discarded in favor of more important areas. Thus limited resources for data storage and transmission are allocated to the most significant images.

Eberlein, Susan↗

Cluster Observations for Combined X-Ray and Sunyaev-Zel'dovich Estimates of Peculiar Velocities and Distances

Measurements of the peculiar velocities of galaxy clusters with respect to the Hubble flow allow the determination of the gravitational field from all matter in the universe, not just the visible component. The Sunyaev-Zel'dovich (SZ) effect (the inverse-Compton scattering of cosmic microwave background photons by the hot gas in clusters of galaxies) allows these velocities to be measured without the use of empirical distance indicators. Additionally, because the magnitude of the SZ effect is independent of redshift, the technique can be used to measure velocities out to the epoch of cluster formation. The SZ technique requires a determination of the temperature of the hot cluster gas from X-ray observations, and measurements of the SZ effect at millimeter wavelengths to separate the contribution from the thermal motions within the gas from that of the cluster peculiax velocity. We have constructed a bolometric receiver, the Sunyaev-Zel'dovich Infrared Experiment, specifically to make measurements of the SZ effect at millimeter wavelengths in order to apply the SZ technique to peculiar velocity measurements. This receiver has already been used to set limits to the peculiar velocities of two galaxy clusters at z approx. 0.2. As a test of the SZ technique, the double cluster pair Abell 222 and 223 was selected for observation. Measurements of the redshifts of the two components suggest that, if the clusters are gravitationally bound, they should exhibit a relative velocity of 10OO km/ s, well above the expected precision of 200 km/ s (set by astrophysical confusion) that is expected from the SZ method. The temperature can be measured from ASCA data which we obtained for this cluster pair. However, in order to ensure that the temperature estimate from the ASCA data was not dominated by cooling flows within the cluster, we requested ROSAT HRI observations of this cluster pair. Analysis of the X-ray properties of the cluster pair is continuing by combining the ROSAT observations funded under this proposal, archival PSPC data and our own PI ASCA data. The imaging data shows that neither cluster has a strong cooling flow, although A223 is the more compact of the two components. A223 also has a sub-cluster projected 1 Mpc to the North. The observed properties of this cluster pair are strikingly similar to those of the well-studied pair of A399 and A401 which are widely believed to be interacting and are in the initial stages of merging. This strengthens our selection of this cluster pair as a test for cluster peculiar motions. Combining these X-ray observations with SUZIE measurements of the SZ effect will provide important constraints on cluster merger models. This part of the program has been heavily delayed by bad weather at Mauna Kea, however SZ observations were obtained in November 1997 as part of a separately funded effort. These data are currently being analyzed.

Lange, A. E.↗

A new method for mapping multidimensional data to lower dimensions

A multispectral mapping method is proposed which is based on the new concept of BEND (Bidimensional Effective Normalised Difference). The method, which involves taking one sample point at a time and finding the interrelationships between its features, is found very economical from the point of view of storage and processing time. It has good dimensionality reduction and clustering properties, and is highly suitable for computer analysis of large amounts of data. The transformed values obtained by this procedure are suitable for either a planar 2-space mapping of geological sample points or for making grayscale and color images of geo-terrains. A few examples are given to justify the efficacy of the proposed procedure.

Gowda, K. C.↗

Satellite Remote Sensing to Assess Cyanobacterial Bloom Frequency Across the United States at Multiple Spatial Scales

Cyanobacterial blooms can have negative effects on human health and local ecosystems. Field monitoring of cyanobacterial blooms can be costly, but satellite remote sensing has shown utility for more efficient spatial and temporal monitoring across the United States. Here, satellite imagery was used to assess the annual frequency of surface cyanobacterial blooms, defined for each satellite pixel as the percentage of images for that pixel throughout the year exhibiting detectable cyanobacteria. Cyanobacterial frequency was assessed across 2,196 large lakes in 46 states across the continental United States (CONUS) using imagery from the European Space Agency’s Ocean and Land Colour Imager for the years 2017 through 2019. In 2019, across all satellite pixels considered, annual bloom frequency had a median value of 4% and a maximum value of 100%, the latter indicating that for those satellite pixels, a cyanobacterial bloom was detected by the satellite sensor for every satellite image considered. In addition to annual pixel-scale cyanobacterial frequency, results were summarized at the lake- and state-scales by averaging annual pixel-scale results across each lake and state. For 2019, average annual lake-scale frequencies also had a maximum value of 100%, and Oregon and Ohio had the highest average annual state-scale frequencies at 65% and 52%. Pixel-scale frequency results can assist in identifying portions of a lake that are more prone to cyanobacterial blooms, while lake- and state-scale frequency results can assist in the prioritization of sampling resources and mitigation efforts. Satellite imagery is limited by the presence of snow and ice, as imagery collected in these conditions are quality flagged and discarded. Thus, annual bloom frequencies within nine climate regions were investigated to determine whether missing data biased results in climate regions more prone to snow and ice, given that their annual summaries would be weighted toward the summer months when cyanobacterial blooms tend to occur. Results were unbiased by the time period selected in most climate regions, but a large bias was observed for the Northwest Rockies and Plains climate region. Moderate biases were observed for the Ohio Valley and the Southeast climate regions. Finally, a clustering analysis was used to identify areas of high and low cyanobacterial frequency across CONUS based on average annual lake-scale cyanobacterial frequencies for 2019. Several clusters were identified that transcended state, watershed, and eco-regional boundaries. Combined with additional data, results from the clustering analysis may offer insight regarding large-scale drivers of cyanobacterial blooms.

remote sensing↗

An Algorithm for Hyperspectral Remote Sensing of Aerosols: 3. Application to the GEO-TASO Data in KORUS-AQ Field Campaign

This paper describes the third part of a series of investigations to develop algorithms for simultaneous retrieval of aerosol parameters and surface spectral reflectance from the data measured by GEOstationary Trace gas and Aerosol Sensor Optimization (GEO-TASO) instrument. Since the algorithm is designed for future hyperspectral and geostationary satellite sensors, such as Tropospheric Emissions: Monitoring of Pollution (TEMPO), it is applied to GEO-TASO data measured over the same area by different flights as part of the Korea-United Stated Air Quality Study (KORUS-AQ) field campaign in 2016. While GEO-TAOS has a spectral sampling interval of ~0.28 nm in the visible, its data is thinned through a band selection approach with consideration of atmospheric transmittance and different surface types, which yields 20 common spectral bands to be used by the algorithm. The algorithm starts with 4 common principal components (PCs) for surface spectral reflectance extracted from various spectral libraries, and the constraints of surface reflectance parameters and aerosol scattering models respectively from k- means clustering analysis of the Rayleigh-corrected GEO-TASO spectra and AERONET data. The algorithm then proceeds iteratively with an optimal estimation approach to update PCs and retrieve aerosol optical depth (AOD) until the best match between simulated and GEOS-TASO measured spectra is achieved. The spectral AODs are compared between those retrieved from GEO-TASO (x) and 7 AERONETS (y) in 440, 550, 550, 675 nm, respectively. The comparison reveals that iterative updates of surface spectral PCs (and so surface reflectance) yield significant enhancement in AOD retrievals, improving the mean linear fitting equations from y = 0.735x + 0.088 (without update of PCs) to y = 1.055x+ 0.01, and the Pearson correlation coefficient (𝑅𝑅2) from 0.54 to 0.76, respectively. Move case studies are need to further evaluate the algorithm for its application to TEMPO that carries a enhanced version of GEO-TASO instrument.

Tropospheric Emissions: Monitoring of Pollution (T↗

Analysis of multispectral data using an unsupervised classification technique: Application to VAS

A statistical classification method based on clustering of multidimensional histograms was applied to several channels of the VAS multispectral imagery. The method automatically discriminates and classifies atmospheric ground features such as cloud types, atmospheric moisture patterns, ocean, or ground. Such a clustering method has the advantage of forming natural data groupings, without a priori classification. Clusters are not limited by straight lines or plane surfaces as is the case in threshold methods. The method was applied to simultaneous full resolution images from channels 8 (11.2 micron), 10 (6.7 micron), and 12 (3.9 micron). Twenty image segments of 64 by 64, 12 image segments of 128 by 128, and 4 image segments of 254 by 254 picture elements were analyzed. In addition, normal VISSR mode images at 1800, 1830, and 2000 GMT were used to identify the classes. The gray levels measured along a scan line and the result of the classification scheme (dashed curves) for the three channels investigated are shown. Each point of the image is affected to a class. Each class is identified by a center of gravity that is represented by a vector in the three dimensional space of gray levels.

Szejwach, G.↗