Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data analysis cluster”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 235 records · Page 13
All sourcesNASA NTRSDOE OSTI
NASA NTRS2013

Reducing Earth Topography Resolution for SMAP Mission Ground Tracks Using K-Means Clustering

The K-means clustering algorithm is used to reduce Earth topography resolution for the SMAP mission ground tracks. As SMAP propagates in orbit, knowledge of the radar antenna footprints on Earth is required for the antenna misalignment calibration. Each antenna footprint contains a latitude and longitude location pair on the Earth surface. There are 400 pairs in one data set for the calibration model. It is computationally expensive to calculate corresponding Earth elevation for these data pairs. Thus, the antenna footprint resolution is reduced. Similar topographical data pairs are grouped together with the K-means clustering algorithm. The resolution is reduced to the mean of each topographical cluster called the cluster centroid. The corresponding Earth elevation for each cluster centroid is assigned to the entire group. Results show that 400 data points are reduced to 60 while still maintaining algorithm performance and computational efficiency. In this work, sensitivity analysis is also performed to show a trade-off between algorithm performance versus computational efficiency as the number of cluster centroids and algorithm iterations are increased.

ground tracks↗
NASA NTRS2007

Transport in the Subtropical Lowermost Stratosphere during CRYSTAL-FACE

We use in situ measurements of water vapor (H2O), ozone (O3), carbon dioxide (CO2), carbon monoxide (CO), nitric oxide (NO), and total reactive nitrogen (NO(y)) obtained during the CRYSTAL-FACE campaign in July 2002 to study summertime transport in the subtropical lowermost stratosphere. We use an objective methodology to distinguish the latitudinal origin of the sampled air masses despite the influence of convection, and we calculate backward trajectories to elucidate their recent geographical history. The methodology consists of exploring the statistical behavior of the data by performing multivariate clustering and agglomerative hierarchical clustering calculations, and projecting cluster groups onto principal component space to identify air masses of like composition and hence presumed origin. The statistically derived cluster groups are then examined in physical space using tracer-tracer correlation plots. Interpretation of the principal component analysis suggests that the variability in the data is accounted for primarily by the mean age of air in the stratosphere, followed by the age of the convective influence, and lastly by the extent of convective influence, potentially related to the latitude of convective injection [Dessler and Sherwuud, 2004]. We find that high-latitude stratospheric air is the dominant source region during the beginning of the campaign while tropical air is the dominant source region during the rest of the campaign. Influence of convection from both local and non-local events is frequently observed. The identification of air mass origin is confirmed with backward trajectories, and the behavior of the trajectories is associated with the North American monsoon circulation.

Pittman, Jasna V.↗
NASA NTRS2007

Transport in the Subtropical Lowermost Stratosphere during the Cirrus Regional Study of Tropical Anvils and Cirrus Layers-Florida Area Cirrus Experiment

We use in situ measurements of water vapor (H2O), ozone (O3), carbon dioxide (CO2), carbon monoxide (CO), nitric oxide (NO), and total reactive nitrogen (NOy) obtained during the CRYSTAL-FACE campaign in July 2002 to study summertime transport in the subtropical lowermost stratosphere. We use an objective methodology to distinguish the latitudinal origin of the sampled air masses despite the influence of convection, and we calculate backward trajectories to elucidate their recent geographical history. The methodology consists of exploring the statistical behavior of the data by performing multivariate clustering and agglomerative hierarchical clustering calculations and projecting cluster groups onto principal component space to identify air masses of like composition and hence presumed origin. The statistically derived cluster groups are then examined in physical space using tracer-tracer correlation plots. Interpretation of the principal component analysis suggests that the variability in the data is accounted for primarily by the mean age of air in the stratosphere, followed by the age of the convective influence, and last by the extent of convective influence, potentially related to the latitude of convective injection (Dessler and Sherwood, 2004). We find that high-latitude stratospheric air is the dominant source region during the beginning of the campaign while tropical air is the dominant source region during the rest of the campaign. Influence of convection from both local and nonlocal events is frequently observed. The identification of air mass origin is confirmed with backward trajectories, and the behavior of the trajectories is associated with the North American monsoon circulation.

Pittman, Jasna V.↗
NASA NTRSDate not supplied

Anomaly Detection for the Roman Space Telescope Wide Field Instrument’s Science Data Processing Pipeline

The Roman Space Telescope (RST) Wide Field Instrument (WFI) will be utilizing a preliminary Science Data Processing (SDP) pipeline during its Integration and Test, and to some extent during Operations, to track basic statistics and identify known features such as cosmic rays, snowballs as well as possible anomalies in raw detector data. In our detectors, these anomalies appear as jumps in the ramp of a readout and are classified as cosmic rays if they appear as a streak or snowballs if they’re more circular. The WFI employs an array of 18 H4RG-10 detectors that collect image samples. Each set of raw frames within a non-destructive exposure is packaged by the SDP pipeline into image cubes for each detector. Each cube is a time series of 4096 × 4096 accumulating pixel frames. The preliminary analysis pipeline is used to locate anomalies in these time-series accumulation frames and identify the type of anomaly, either natural phenomena or detector characteristic. To compare different methods, we’ve implemented both heuristic-based and data-driven methods to identify anomalies. For the heuristic-based approach, we identify snowballs and cosmic rays by the size and shape of outlier pixel clusters between consecutive frames. For data driven methods, we evaluated a Convolutional Neural Network (CNN) model, and more traditional methods like Principal Component Analysis (PCA). CNN is a supervised learning/classification method. Thus, we used a labeled dataset of anomalies to perform segmentation of the image and identify anomalies. We used previously identified cosmic rays and snowballs to measure the accuracy and efficiency of the mentioned approaches. In evaluating these methods, we aim to pick the best fit for the SDP pipeline’s anomaly detection in terms of both performance and runtime.

Paul Horton↗
NASA NTRS2014

Search for Cosmic-Ray-Induced Gamma-Ray Emission in Galaxy Clusters

Current theories predict relativistic hadronic particle populations in clusters of galaxies in addition to the already observed relativistic leptons. In these scenarios hadronic interactions give rise to neutral pions which decay into gamma rays that are potentially observable with the Large Area Telescope (LAT) on board the Fermi space telescope. We present a joint likelihood analysis searching for spatially extended gamma-ray emission at the locations of 50 galaxy clusters in four years of Fermi-LAT data under the assumption of the universal cosmic-ray (CR) model proposed by Pinzke & Pfrommer. We find an excess at a significance of 2.7 delta, which upon closer inspection, however, is correlated to individual excess emission toward three galaxy clusters: A400, A1367, and A3112. We discuss these cases in detail and conservatively attribute the emission to unmodeled background systems (for example, radio galaxies within the clusters).Through the combined analysis of 50 clusters, we exclude hadronic injection efficiencies in simple hadronic models above 21% and establish limits on the CR to thermal pressure ratio within the virial radius, R(sub 200), to be below 1.25%-1.4% depending on the morphological classification. In addition, we derive new limits on the gamma-ray flux from individual clusters in our sample.

Online-only material: color figures↗
NASA NTRS2019

Electra: A Modular-Based Expansion of NASA's Supercomputing Capability

NASA has increasingly relied on high-performance computing (HPC) re- sources for computational modeling, simulation, and data analysis to meet the science and engineering goals of its missions in space exploration, aeronautics, and Earth and space science. The NASA Advanced Supercomputing (NAS) Division at Ames Research Center in Silicon Valley, Calif., hosts NASA’s premier supercomputing resources, integral to achieving and enhancing the success of the agency’s missions. NAS provides a balanced environment, funded under the High-End Computing Capability (HECC) project, comprised of world-class supercomputers, including its flagship distributed-memory cluster, Pleiades; high-speed networking; and massive data storage facilities, along with multi-disciplinary support teams for user support, code porting and optimization, and large-scale data analysis and scientific visualization. However, as scientists have increased the fidelity of their simulations and engineers are conducting larger parameter-space studies, the requirements for supercomputing resources have been growing by leaps and bounds. With the facility housing the HECC systems reaching its power and cooling capacity, NAS undertook a prototype project to investigate an alternative approach for housing supercomputers. Modular supercomputing, or container-based computing, is an innovative concept for expanding NASA’s HPC capabilities. With modular supercomputing, additional containers—similar to portable storage pods—can be connected together as needed to accommodate the agency’s ever-increasing demand for computing resources. In addition, taking advantage of the local weather permits the use of cooling technologies that would additionally save energy and reduce annual water usage. The first stage of NASA’s Modular Supercomputing Facility (MSF) prototype, which resulted in a 1,000 square-foot module on a concrete pad with room for 16 compute racks, was completed in Fall 2016 and an SGI (now HPE) computer system, named Electra, was deployed there in early 2017. Cooling is performed via an evaporative system built into the module, and preliminary experience shows a Power Usage Effectiveness (PUE) measurement of 1.03. Electra achieved over a petaflop on the LINPACK benchmark, sufficient to rank number 96 on the November 2016 TOP500 list [14]. The system consists of 1,152 InfiniBand-connected Intel Xeon Broadwell-based nodes. Its users access their files on a facility-wide file system shared by all HECC compute assets via Mellanox MetroX InfiniBand extenders, which connect the Electra fabric to Lustre routers in the primary facility over fiber-optic links about 900 feet long. The MSF prototype has exceeded expectations and is serving as a blueprint for future expansions. In the remainder of this chapter, we detail how modular data center technology can be used to expand an existing compute resource. We begin by describing NASA’s requirements for supercomputing and how resources were provided prior to the integration of the Electra module-based system.

Biswas, Rupak↗
NASA NTRS1994

Dynamics of cD clusters of galaxies. II: Analysis of seven Abell clusters

We have investigated the dynamics of the seven Abell clusters A193, A399, A401, A1795, A1809, A2063, and A2124, based on redshift data reported previously by us (Hill & Oegerle, (1993)). These papers present the initial results of a survey of cD cluster kinematics, with an emphasis on studying the nature of peculiar velocity cD galaxies and their parent clusters. In the current sample, we find no evidence for significant peculiar cD velocities, with respect to the global velocity distribution. However, the cD in A2063 has a significant (3 sigma) peculiar velocity with respect to galaxies in the inner 1.5 Mpc/h, which is likely due to the merger of a subcluster with A2063. We also find significant evidence for subclustering in A1795, and a marginally peculiar cD velocity with respect to galaxies within approximately 200 kpc/h of the cD. The available x-ray, optical, and galaxy redshift data strongly suggest that a subcluster has merged with A1795. We propose that the subclusters which merged with A1795 and A2063 were relatively small, with shallow potential wells, so that the cooling flows in these clusters were not disrupted. Two-body gravitational models of the A399/401 and A2063/MKW3S systems indicate that A399/401 is a bound pair with a total virial mass of approximately 4 x 10(exp 15) solar mass/h, while A2063 and MKW3S are very unlikely to be bound.

Oegerle, William R.↗
NASA NTRS1973

An unsupervised classification technique for multispectral remote sensing data.

Description of a two-part clustering technique consisting of (a) a sequential statistical clustering, which is essentially a sequential variance analysis, and (b) a generalized K-means clustering. In this composite clustering technique, the output of (a) is a set of initial clusters which are input to (b) for further improvement by an iterative scheme. This unsupervised composite technique was employed for automatic classification of two sets of remote multispectral earth resource observations. The classification accuracy by the unsupervised technique is found to be comparable to that by traditional supervised maximum-likelihood classification techniques.

Su, M. Y.↗
NASA NTRS1975

Spectrophotometry of faint cluster galaxies and the Hubble diagram - An approach to cosmology

New spectrophotometric data are presented for large-redshift galaxies in clusters. A new approach to aperture corrections and the analysis of the Hubble diagram is outlined, including the explicit incorporation of evolutionary effects. The importance of selection effects on the usual methods of analysis are in principle overcome in the analysis, although the heterogeneity of the sample still makes conclusions about cosmology slightly suspect. Formal values of the deceleration parameter are derived under several sets of assumptions, yielding results between +0.33 and -1.27 with formal standard deviations of order 0.7.

Gunn, J. E.↗
NASA NTRS1998

A Survey of Variable Extragalactic Sources with XTE's All Sky Monitor (ASM)

The original goal of the project was the near real-time detection of AGN utilizing the SSC 3 of the ASM on XTE which does a deep integration on one 100 square degree region of the sky. While the SSC never performed sufficiently well to allow the success of this goal, the work on the project has led to the development of a new analysis method for coded aperture systems which has now been applied to ASM data for mapping regions near clusters of galaxies such as the Perseus Cluster and the Coma Cluster. Publications are in preparation that describe both the new method and the results from mapping clusters of galaxies.

Jernigan, Garrett↗
NASA NTRS2014

The Parallel System for Integrating Impact Models and Sectors (pSIMS)

We present a framework for massively parallel climate impact simulations: the parallel System for Integrating Impact Models and Sectors (pSIMS). This framework comprises a) tools for ingesting and converting large amounts of data to a versatile datatype based on a common geospatial grid; b) tools for translating this datatype into custom formats for site-based models; c) a scalable parallel framework for performing large ensemble simulations, using any one of a number of different impacts models, on clusters, supercomputers, distributed grids, or clouds; d) tools and data standards for reformatting outputs to common datatypes for analysis and visualization; and e) methodologies for aggregating these datatypes to arbitrary spatial scales such as administrative and environmental demarcations. By automating many time-consuming and error-prone aspects of large-scale climate impacts studies, pSIMS accelerates computational research, encourages model intercomparison, and enhances reproducibility of simulation results. We present the pSIMS design and use example assessments to demonstrate its multi-model, multi-scale, and multi-sector versatility.

crop modeling↗
NASA NTRS1981

The evaluation of alternate methodologies for land cover classification in an urbanizing area

The usefulness of LANDSAT in classifying land cover and in identifying and classifying land use change was investigated using an urbanizing area as the study area. The question of what was the best technique for classification was the primary focus of the study. The many computer-assisted techniques available to analyze LANDSAT data were evaluated. Techniques of statistical training (polygons from CRT, unsupervised clustering, polygons from digitizer and binary masks) were tested with minimum distance to the mean, maximum likelihood and canonical analysis with minimum distance to the mean classifiers. The twelve output images were compared to photointerpreted samples, ground verified samples and a current land use data base. Results indicate that for a reconnaissance inventory, the unsupervised training with canonical analysis-minimum distance classifier is the most efficient. If more detailed ground truth and ground verification is available, the polygons from the digitizer training with the canonical analysis minimum distance is more accurate.

Smekofski, R. M.↗
NASA NTRS1998

Binning for IC Quality: Experimental Studies on the SEMATECH Data

The earlier smaller bipolar study did not provide a high enough bin 0 population to directly observe test escapes and thereby estimate defect levels for the best bin. Results presented here indicate that the best bin can be reasonably expected to show a 2 - 5 factor improvement in defect levels over the average for the lot for moderate to high yields (the overall yield for these experiments was approximately 65%). The experiments also confirm the dependence of the best bin quality on test transparency. The defect level improvement is poorer for the case Of IDDQ escapes where the tests applied had a much higher escape rate. Overall experimental results are consistent with analytical projections for typical values of the clustering parameter in [9]. The final version of this paper will include extensive analysis to validate the analytical models based on this data.

Singh, Adit D.↗
NASA NTRS2000

Satellite Data Sets in the Polar Regions

We have generated about two decades of consistently derived geophysical parameters in the polar regions. The key parameters are sea ice concentration, surface temperature, albedo, and cloud cover statistics. Sea ice concentrations were derived from the Scanning Multichannel Microwave Radiometer (SMMR) data and the Special Scanning Cl Microwave Imager (SSM/I) data from several platforms using the enhanced Bootstrap Algorithm for the period 1978 through 1999. The new algorithm reduces the errors associated with spatial and temporal variations in the emissivity and surface temperatures of sea ice. Also, bad data at ocean/land interfaces are identified and deleted in an unsupervised manner. Surface ice temperature, albedo and cloud cover statistics are derived simultaneously from the Advanced Very High Resolution Radiometer (AVHRR) data from 1981 through 1999 and mapped at a higher resolution but the same format as the ice concentration data. The technique makes use these co-registered ice concentration maps to enable cloud masking to be done separately for open ocean, sea ice and land areas. The effect of inversion is minimized by taking into consideration the expected changes in the effect of inversion with altitude, especially in the Antarctic. A technique for ice type regional classification has also been developed using multichannel cluster analysis and a neural network. This provide a means to identify large areas of thin ice, first year ice, and older ice types. The data sets have been shown to be coherent with each other and provide a powerful tool for in depth studies of the currently changing Arctic and Antarctic environment.

Comiso, Josefino C.↗
NASA NTRS1972

Unsupervised classification of earth resources data.

A new clustering technique is presented. It consists of two parts: (a) a sequential statistical clustering which is essentially a sequential variance analysis and (b) a generalized K-means clustering. In this composite clustering technique, the output of (a) is a set of initial clusters which are input to (b) for further improvement by an iterative scheme. This unsupervised composite technique was employed for automatic classification of two sets of remote multispectral earth resource observations. The classification accuracy by the unsupervised technique is found to be comparable to that by existing supervised maximum liklihood classification technique.

Su, M. Y.↗
NASA NTRS1987

Spectral characteristics and the extent of paleosols of the Palouse formation

Thematic mapping data was analyzed and verified by comparison to previously gathered transect samples and to aerial photographs. A bare-soil field with exposed paleosols characterized by slight enrichment of iron was investigated. Spectral relationships were first investigated statistically by creating a data set with DN values spatially matched as nearly as possible to field sample points. Chemical data for each point included organic carbon, free iron oxide, and amorphous iron content. The chemical data, DN values, and various band ratios were examined with the program package Statistix in order to find the combinations of reflectance data most likely to show a relationship which would dependably separate the exposed paleosols from the other soils. Cluster analysis and Fastclas classification procedures were applied to the most promising of the band ratio combinations.

Frazier, B. E.↗
NASA NTRS2015

Earth Science Data Fusion with Event Building Approach

Objectives of the NASA Information And Data System (NAIADS) project are to develop a prototype of a conceptually new middleware framework to modernize and significantly improve efficiency of the Earth Science data fusion, big data processing and analytics. The key components of the NAIADS include: Service Oriented Architecture (SOA) multi-lingual framework, multi-sensor coincident data Predictor, fast into-memory data Staging, multi-sensor data-Event Builder, complete data-Event streaming (a work flow with minimized IO), on-line data processing control and analytics services. The NAIADS project is leveraging CLARA framework, developed in Jefferson Lab, and integrated with the ZeroMQ messaging library. The science services are prototyped and incorporated into the system. Merging the SCIAMACHY Level-1 observations and MODIS/Terra Level-2 (Clouds and Aerosols) data products, and ECMWF re- analysis will be used for NAIADS demonstration and performance tests in compute Cloud and Cluster environments.

Lukashin, C.↗
NASA NTRS1994

Multivariate Statistical Analysis Software Technologies for Astrophysical Research Involving Large Data Bases

We developed a package to process and analyze the data from the digital version of the Second Palomar Sky Survey. This system, called SKICAT, incorporates the latest in machine learning and expert systems software technology, in order to classify the detected objects objectively and uniformly, and facilitate handling of the enormous data sets from digital sky surveys and other sources. The system provides a powerful, integrated environment for the manipulation and scientific investigation of catalogs from virtually any source. It serves three principal functions: image catalog construction, catalog management, and catalog analysis. Through use of the GID3* Decision Tree artificial induction software, SKICAT automates the process of classifying objects within CCD and digitized plate images. To exploit these catalogs, the system also provides tools to merge them into a large, complex database which may be easily queried and modified when new data or better methods of calibrating or classifying become available. The most innovative feature of SKICAT is the facility it provides to experiment with and apply the latest in machine learning technology to the tasks of catalog construction and analysis. SKICAT provides a unique environment for implementing these tools for any number of future scientific purposes. Initial scientific verification and performance tests have been made using galaxy counts and measurements of galaxy clustering from small subsets of the survey data, and a search for very high redshift quasars. All of the tests were successful and produced new and interesting scientific results. Attachments to this report give detailed accounts of the technical aspects of the SKICAT system, and of some of the scientific results achieved to date. We also developed a user-friendly package for multivariate statistical analysis of small and moderate-size data sets, called STATPROG. The package was tested extensively on a number of real scientific applications and has produced real, published results.

Djorgovski, S. G.↗