Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data analysis cluster”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 181 records · Page 10
All sourcesNASA NTRSDOE OSTI
NASA NTRS2005

Testing the Archivas Cluster (Arc) for Ozone Monitoring Instrument (OMI) Scientific Data Storage

The Ozone Monitoring Instrument (OMI) launched on NASA's Aura Spacecraft, the third of the major platforms of the EOS program on July 15,2004. In addition to the long term archive and distribution of the data from OM1 through the Goddard Earth Science Distributed Active Archive Center (GESDAAC), we are evaluating other archive mechanisms that can archive the data in a more immediately available method where it can be used for futher data production and analysis. In 2004, Archivas, Inc. was selected by NASA s Small Business Innovative Research (SBIR) program for the development of their Archivas Cluster (ArC) product. Arc is an online disk based system utilizing self-management and automation on a Linux cluster. Its goal is to produce a low cost solution coupled with the ease of management. The OM1 project is an application partner of the SBIR program, and has deployed a small cluster (5TB) based on the beta Archwas software. We performed extensive testing of the unit using production OM1 data since launch. In 2005, Archivas, Inc. was funded in SBIR Phase II for further development, which will include testing scalability with the deployment of a larger (35TB) cluster at Goddard. We plan to include Arc in the OM1 Team Leader Computing Facility (TLCF) hosting OM1 data for direct access and analysis by the OMI Science Team. This presentation will include a brief technical description of the Archivas Cluster, a summary of the SBIR Phase I beta testing results, and an overview of the OMI ground data processing architecture including its interaction with the Phase II Archivas Cluster and hosting of OMI data for the scientists.

Tilmes, Curt↗
NASA NTRS2003

Cluster Masses Derived from X-ray and Sunyaev-Zeldovich Effect Measurements

We infer the gas mass and total gravitational mass of 11 clusters using two different methods; analysis of X-ray data from the Chandra X-ray Observatory and analysis of centimeter-wave Sunyaev-Zel'dovich Effect (SZE) data from the BIMA and OVRO interferometers. This flux-limited sample of clusters from the BCS cluster catalogue was chosen so as to be well above the surface brightness limit of the ROSAT All Sky Survey; this is therefore an orientation unbiased sample. The gas mass fraction, f_g, is calculated for each cluster using both X-ray and SZE data, and the results are compared at a fiducial radius of r_500. Comparison of the X-ray and SZE results for this orientation unbiased sample allows us to constrain cluster systematics, such as clumping of the intracluster medium. We derive an upper limit on Omega_M assuming that the mass composition of clusters within r_500 reflects the universal mass composition Omega_M h_100 is greater than Omega _B / f-g. We also demonstrate how the mean f_g derived from the sample can be used to estimate the masses of clusters discovered by upcoming deep SZE surveys.

Laroque, S.↗
NASA NTRS2010

Mimas: Preliminary Evidence For Amorphous Water Ice from VIMS

We have conducted a statistical clustering analysis (1,2) on a mosaic of VIMS data cubes obtained on February 13, 2010, for Saturn s satellite Mimas. Seven VIMS cubes were geometrically projected and re-sampled to a common spatial resolution. The clustering technique consists of a partitioning algorithm coupled to a criterion that prevents sub-optimal solutions and tests for the influence of random noise in the measurements. The clustering technique is agnostic about the meaning of the clusters, and scientific interpretation requires their a posteriori evaluation. The preliminary results yielded five clusters, demonstrating that spectral variability across Mimas surface is statistically significant. The ratios of the means calculated for each of the clusters show structure within the 1.6- micron water ice band, as well as the shape and the central wavelength of the strong ice band at 2 micron, that map spatially in patterns apparently related to the topography of Mimas, in particular certain regions in and around Herschel crater. The mean spectra of the five clusters, show similarities with laboratory spectra of amorphous and crystalline H2O ice (3) that are suggestive of the presence of an amorphous ice component in certain regions of Mimas, notably on the central peak of Herschel, on the crater floor, and in faults surrounding the crater. This may represent a mixture of both ice phases, or perhaps a layer of amorphous ice on a base of crystalline ice. Another possible occurrence of amorphous ice appears southwest of Herschel, close to the south pole.

Cruikshank, Dale P.↗
NASA NTRS1985

Space station ECLSS integration analysis: Simplified General Cluster Systems Model, ECLS System Assessment Program enhancements

The data base verification of the ECLS Systems Assessment Program (ESAP) was documented and changes made to enhance the flexibility of the water recovery subsystem simulations are given. All changes which were made to the data base values are described and the software enhancements performed. The refined model documented herein constitutes the submittal of the General Cluster Systems Model. A source listing of the current version of ESAP is provided in Appendix A.

Ferguson, R. E.↗
DOE OSTI2022

Neural network acceleration of large-scale structure theory calculations

Here, we make use of neural networks to accelerate the calculation of power spectra required for the analysis of galaxy clustering and weak gravitational lensing data. For modern perturbation theory codes, evaluation time for a single cosmology and redshift can take on the order of two seconds. In combination with the comparable time required to compute linear predictions using a Boltzmann solver, these calculations are the bottleneck for many contemporary large-scale structure analyses. Here, we construct neural network-based surrogate models for Lagrangian perturbation theory (LPT) predictions of matter power spectra, real and redshift space galaxy power spectra, and galaxy-matter cross power spectra that attain ~ 0.1% (at one sigma) accuracy over a broad range of scales in a ωCDM parameter space. The neural network surrogates can be evaluated in approximately one millisecond, a factor of 1000 times faster than the full Boltzmann code and LPT computations. In a simulated full-shape redshift space galaxy power spectrum analysis, we demonstrate that the posteriors obtained using our surrogates are accurate compared to those obtained using the full LPT model. We make our surrogate models public at https://github.com/sfschen/EmulateLSS, so that others may take advantage of the speed gains they provide to enable rapid iteration on analysis settings, something that is essential in complex contemporary large-scale structure analyses.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2022

The XMM Cluster Survey analysis of the SDSS DR8 redMaPPer catalogue: implications for scatter, selection bias, and isotropy in cluster scaling relations

ABSTRACT In this paper, we present the X-ray analysis of Sloan Digital Sky Survey DR8 redMaPPer (SDSSRM) clusters using data products from the XMM Cluster Survey (XCS). In total, 1189 SDSSRM clusters fall within the XMM–Newton footprint. This has yielded 456 confirmed detections accompanied by X-ray luminosity (LX) measurements. Of these clusters, 381 have an associated X-ray temperature measurement (TX). This represents one of the largest samples of coherently derived cluster TX values to date. Our analysis of the X-ray observable to richness scaling relations has demonstrated that scatter in the TX–λ relation is roughly a third of that in the LX–λ relation, and that the LX–λ scatter is intrinsic, i.e. will not be significantly reduced with larger sample sizes. Analysis of the scaling relation between LX and TX has shown that the fits are sensitive to the selection method of the sample, i.e. whether the sample is made up of clusters detected ‘serendipitously’ compared to those deliberately targeted by XMM. These differences are also seen in the LX–λ relation and, to a lesser extent, in the TX–λ relation. Exclusion of the emission from the cluster core does not make a significant impact on the findings. A combination of selection biases is a likely, but yet unproven, reason for these differences. Finally, we have also used our data to probe recent claims of anisotropy in the LX–TX relation across the sky. We find no evidence of anistropy, but stress this may be masked in our analysis by the incomplete declination coverage of the SDSS.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2022

Training data selection for event classification in a highly variable environment

A problem of interest for nuclear nonproliferation is monitoring activities at nuclear facilities, where proliferation events may only take place a few times and often under variable conditions. Machine learning has revolutionized data analytics by enabling the use of measurable signatures to generate predictive models of facility operations. However, traditional methods for training these models require large, reliable data sets with labeled observations, a challenge for nonproliferation. Highly variable conditions further complicate this as events from training data may have occurred in conditions quite different from the event of interest. Our hypothesis is that when events occur in a highly variable environment, careful training data selection for each test event could outperform the standard approach of using all available training data. We developed a method to optimize training data selection for the given test event and applied it to predicting the power level of the High Flux Isotope Reactor (HFIR) at Oak Ridge National Laboratory. In this study, the reactor startup exhibits variability between occurrences due to natural variability in environmental conditions and operational procedures. Using a combination of analysis techniques, a similitude assessment was performed on data collected from HFIR to isolate clusters that were optimal for training a predictive model. Concepts such as dynamic time warping and Jaccard similarity were used in conjunction with clustering analysis. In order to validate this approach, the model was trained on every combination of unique training events and the predictive performance was compared to the performance using a subset of the training data selected by isolated clusters found through the similitude assessment.

Iyer, A↗
NASA NTRS2004

Determination of Cluster Distances from Chandra Imaging Spectroscopy and Sunyaev-Zeldovich Effect Measurements: Analysis Methods and Initial Results - I

X-ray and Sunyaev-Zeldovich Effect data ca,n be combined to determine the distance to galaxy clusters. High-resolution X-ray data are now available from the Chandra Observatory, which provides both spatial and spectral information, and interferometric radio measurements of the Sunyam-Zeldovich Effect are available from the BIMA and 0VR.O arrays. We introduce a Monte Carlo Markov chain procedure for the joint analysis of X-ray and Sunyaev-Zeldovich Effect data. The advantages of this method are the high computational efficiency and the ability to measure the full probability distribution of all parameters of interest, such as the spatial and spectral properties of the cluster gas and the cluster distance. We apply this technique to the Chandra X-ray data and the OVRO radio data for the galaxy cluster Abell 611. Comparisons with traditional likelihood-ratio methods reveal the robustness of the method. This method will be used in a follow-up paper to determine the distance of a large sample of galaxy clusters for which high-resolution Chandra X-ray and BIMA/OVRO radio data are available.

Bonamente, Massimiliano↗
DOE OSTI2021

Displacement Analysis of Geothermal Field Based on PSInSAR And SOM Clustering Algorithms A Case Study of Brady Field, Nevada—USA

The availability of free and high temporal resolution satellite data and advanced SAR techniques allows us to analyze ground displacement cost-effectively. Our aim was to properly define subsidence and uplift areas to delineate a geothermal field and perform time-series analysis to identify temporal trends. A Persistent Scatterer Interferometry (PSI) algorithm was used to estimate vertical displacement in the Brady geothermal field located in Nevada by analyzing 70 Sentinel-1A Synthetic-Aperture Radar (SAR) images, between January 2017 and December 2019. To classify zones affected by displacement, an unsupervised Self-Organizing Map (SOM) algorithm was applied to classify points based on their behavior in time, and those clusters were used to determine subsidence, uplift, and stable regions automatically. Finally, time-series analysis was applied to the clustered data to understand the inflection dates. The maximum subsidence is –19 mm/yr with an average value of –6 mm/yr within the geothermal field. The maximum uplift is 14 mm/yr with an average value of 4 mm/yr within the geothermal field. The uplift occurred on the NE of the field, where the injection wells are located. On the other hand, subsidence is concentrated on the SW of the field where the production wells are located. The coupling of the PSInSAR and the SOM algorithms was shown to be effective in analyzing the direction and pattern of the displacements observed in the field.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2020

An Integrated Platform for Collaborative Data Analytics

While collaboration among data scientists is a key to organizational productivity, data analysts face significant barriers to achieving this end, including data sharing, accessing and configuring the required computational environment, and a unified method of sharing knowledge. Each of these barriers to collaboration is related to the fundamental question of knowledge management “how can organizations use knowledge more effectively?”. In this paper, we consider the problem of knowledge management in collaborative data analytics and present ShareAL, an integrated knowledge management platform, as a solution to that problem. The ShareAL platform consists of three core components: a full stack web application, a dashboard for analyzing streaming data and a High Performance Computing (HPC) cluster for performing real time analysis. Prior research has not applied knowledge management to collaborative analytics or developed a platform with the same capabilities as ShareAL. ShareAL overcomes the barriers data scientists face to collaboration by providing intuitive sharing of data and analytics via the web application, a shared computing environment via the HPC cluster and knowledge sharing and collaboration via a real time messaging application.

Oesch, T↗
DOE OSTI2020

Bi-cross validation of spectral clustering hyperparameters

One challenge impeding the analysis of terabyte scale X-ray scattering data from the Linac Coherent Light Source (LCLS) is determining the number of clusters required for the execution of traditional clustering algorithms. Here, we demonstrate that the previous work using bi-cross validation to determine the number of singular vectors directly maps to the spectral clustering problem of estimating both the number of clusters and hyperparameter values. Here by applying this method to LCLS X-ray scattering data enables the identification of dropped shots without manually setting boundaries on detector fluence and provides a path toward identifying rare and anomalous events.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2025

Erratum: Centrality-Dependent Modification of Jet-Production Rates in Deuteron-Gold Collisions at $\sqrt{𝑠_{𝑁⁢𝑁}}$ = 200 GeV [Phys. Rev. Lett. 116, 122301 (2016)]

We previously reported jet-production rates in 𝑝 + 𝑝 and 𝑑 + Au collisions at $\sqrt{𝑠_{𝑁⁢𝑁}}$ = 200 GeV recorded in 2008 with the PHENIX detector at the Relativistic Heavy Ion Collider. Subsequently, we found that electronic noise in the electromagnetic calorimeter was not completely removed by the tower-timing cuts used in the analysis. We have conducted an independent re-analysis of the data that eliminated noisy towers by removing them completely from the analysis. These cuts remove any contributions from instrumental noise, but reduce the overall acceptance for electromagnetic clusters by ≈16% and hence the overall statistical power of the data. Finally, this re-analysis found that instrumental noise in the electromagnetic calorimeter led to an artificial increase in the jet yield for both the 𝑝 + 𝑝 and 𝑑 + Au collisions.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2020

Metabolic Pathway Engineering

Modern microbial and enzyme engineering and their advancement are increasingly dependent on the marriage of a wide range of sophisticated technologies. For students entering the field of biotechnology, the outlook is indeed daunting. Expertise at levels beyond that of simple familiarity will be needed to conduct competitive research. It goes without saying that to be competitive, all new researchers in this field will need basic preparation in molecular biology, biochemistry, and genetics. Further, experience working with concepts and experimental tools in enzyme biochemistry and kinetics, gene editing, computational metabolic pathway modeling, experimental pathway flux analysis, and computational clustering tools to process complex data sets will be vital for success. We speculate that most biotechnology researchers in early career at this time will build teams of collaborators to address these disparate science fields rather than attempt to become experts in one lab.

BASIC BIOLOGICAL SCIENCES,BIOMASS FUELS↗
DOE OSTI2020

A Satellite-Based Estimate of Convective Vertical Velocity and Convective Mass Flux: Global Survey and Comparison with Radar Wind Profiler Observations

Convective vertical velocity (w c ) and convective mass flux (M c ) lie at the heart of GCM cumulus parameterizations, but few observations of these critical parameters are available. In this paper, we develop and evaluate a novel, satellite-based method for estimating profiles of w c and M c . Here, comparisons with collocated ground-based radar wind profiler (RWP) observations show that satellite estimated median w c is slightly greater than the RWP estimates, but they show solid agreement when compared at the 95th percentiles (intense updrafts). RWP-derived and satellite estimated M c are broadly comparable in the lower and middle troposphere, with some differences in the upper troposphere due to differences in convective core sampling. A k-means cluster analysis of multiple years of w c data shows that convective characteristics are distinctly different among extratropical convection, tropical land convection, and tropical oceanic convection. Tropical land convection is significantly more intense and more variable than the oceanic counterpart.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

ENSO Impact on Winter Precipitation in the Southeast United States through a Synoptic Climate Approach

The ENSO impact on winter precipitation in the Southeast United States was analyzed from the perspective of daily weather types (WTs). We calculated the dynamic contribution associated with the change in frequency of the WTs and the thermodynamic contribution due to changes in the spatial patterns of the environmental fields of the WTs. Six WTs were obtained using a k-means clustering analysis of 850 hPa winds in reanalysis data from November to February of 1948–2022. All the WTs can only persist for a few days. The most frequent winter weather type is WT1 (shallow trough in Eastern U.S.), which can persist or likely transfer to WT4 (Mississippi River Valley ridge). WT1 becomes less frequent in El Niño years, while the frequency of WT4 does not change much. WTs 2–6 correspond to a loop of eastward propagating waves with troughs and ridges in the mid-latitude westerlies. Three WTs with a deep trough in the Southeast U.S., which are WT2 (east coast trough), WT3 (off east coast trough) and WT6 (plains trough), become more frequent in El Niño years. The more frequent deep troughs (WTs 2, 3 and 6) and less frequent shallow trough (WT1) result in above-normal precipitation in the coastal Southeast U.S. in the winter of El Niño years. WT5 (off coast Carolina High), with maximum precipitation extending from Mississippi Valley to the Great Lakes, becomes less frequent in El Niño years, which corresponds to the below-normal precipitation from the Great Lakes to Upper Mississippi and Ohio River Valley in El Niño years, and vice versa in La Niña years. The relative contribution of the thermodynamic and dynamic contribution is location dependent. On the east coast, the two contributions are similar in magnitude.

Qian, Jian-Hua↗
NASA NTRS1974

Satellite detection of air pollutants

NASA's ERTS-1 satellite, with its high resolution and multispectral capabilities, has been found useful in the detection and analysis of smoke from large point sources (power plants, steel mills, etc.), and widespread atmospheric turbidity associated with atmospheric stagnations. Smoke plumes from the Chicago-Northern Indiana industrial complex have been tracked over Lake Michigan for over 100 km. Experience has shown that smoke plumes are relatively easy to detect over water (in the 0.6-0.7 micrometer band) but much more difficult over land surfaces. Pattern recognition techniques (cluster analysis) were applied to the digital ERTS data, and it was found that the smoke plumes indeed had a unique spectral signature. Studies are currently underway to use measured plume geometries to obtain quantitative estimates of diffusion over water surfaces.

Lyons, W. A.↗
NASA NTRS1976

Stratification of Landsat data by clustering

Full realization of the potential advantages of the synoptic coverage provided by Landsat will require the development and use of data analysis techniques which take into account the large variation and diversity of patterns found over many Landsat scenes. Stratification of the scene into units which are internally homogeneous is recommended as a first step in the analysis of data for whole or multiple frames of Landsat data. The use of clustering as an objective and efficient method of dividing scenes into areas which are spectrally similar (strata) is discussed and initial results, including classification performances and comparisons of spectral strata with major physical factors, are presented.

Bauer, M. E.↗