Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data clustering”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 307 records · Page 17
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

Asynchronous x-ray multiprobe data acquisition for x-ray transient absorption spectroscopy

Laser pump X-ray Transient Absorption (XTA) spectroscopy offers unique insights into photochemical and photophysical phenomena. X-ray Multiprobe data acquisition (XMP DAQ) is a technique that acquires XTA spectra at thousands of pump-probe time delays in a single measurement, producing highly self-consistent XTA spectral dynamics. In this work, we report two new XTA data acquisition techniques that leverage the high performance of XMP DAQ in combination with High Repetition Rate (HRR) laser excitation: HRR-XMP and Asynchronous X-ray Multiprobe (AXMP). HRR-XMP uses a laser repetition rate up to 200 times higher than previous implementations of XMP DAQ and proportionally increases the data collection efficiency at each time delay. This allows HRR-XMP to acquire more high-quality XTA data in less time. AXMP uses a frequency mismatch between the laser and x-ray pulses to acquire XTA data at a flexibly defined set of pump-probe time delays with a spacing down to a few picoseconds. AXMP introduces a novel pump-probe synchronization concept that acquires data in clusters of time delays. Further, the temporally inhomogeneous distribution of acquired data improves the attainable signal statistics at early times, making the AXMP synchronization concept useful for measuring sub-nanosecond dynamics with photon-starved techniques like XTA. In this paper, we demonstrate HRR-XMP and AXMP by measuring the laser-induced spectral dynamics of dilute aqueous solutions of Fe(CN) 6 4₋ and [Fe II (bpy) 3 ] 2+ (bpy: 2,2'-bipyridine), respectively.

47 OTHER INSTRUMENTATION↗
NASA NTRS2011

Compact near-IR and mid-IR cavity ring down spectroscopy device

This invention relates to a compact cavity ring down spectrometer for detection and measurement of trace species in a sample gas using a tunable solid-state continuous-wave mid-infrared PPLN OPO laser or a tunable low-power solid-state continuous wave near-infrared diode laser with an algorithm for reducing the periodic noise in the voltage decay signal which subjects the data to cluster analysis or by averaging of the interquartile range of the data.

Miller, J. Houston↗
DOE OSTI2018

Balancing computation and communication power in power constrained clusters

Systems, apparatuses, and methods for balancing computation and communication power in power constrained environments. A data processing cluster with a plurality of compute nodes may perform parallel processing of a workload in a power constrained environment. Nodes that finish tasks early may be power-gated based on one or more conditions. In some scenarios, a node may predict a wait duration and go into a reduced power consumption state if the wait duration is predicted to be greater than a threshold. The power saved by power-gating one or more nodes may be reassigned for use by other nodes. A cluster agent may be configured to reassign the unused power to the active nodes to expedite workload processing.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2025

Ocelot: An Interactive, Efficient Distributed Compression-As-a-Service Platform With Optimized Data Compression Techniques

Large volumes of data generated by scientific simulations, genome sequencing, and other applications need to be moved among clusters for data collection/analysis. Data compression techniques have effectively reduced data storage and transfer costs. However, users' requirements on interactively controlling both data quality and compression ratios are non-trivial to fulfill. Here, we propose a novel Compression-as-a-Service (CaaS) platform called Ocelot with four important contributions: (1) It offers real-time visualization, interactive compression, and transfer of scientific datasets. (2) It incorporates new strategies for compressing diverse types of datasets more effectively than traditional methods. (3) It provides an effective method for estimating the compression ratio and execution time of compression tasks. (4) Experiments on multiple real-world datasets on geographically distributed computers show that Ocelot can significantly improve data transfer efficiency with a performance gain of more than 10x in computing clusters with relatively slow networks.

compression as a service (CaaS)↗
DOE OSTI2017

Method for Continuous Monitoring of Electrospray Ion Formation

A method for continuously monitoring the performance of electrospray ionization without the addition of hardware or chemistry to the system is demonstrated. In the method, which we refer to as SprayDx, cluster ions with solvent vapor natively formed by electrospray are followed throughout the collection of liquid chromatography–selected reaction monitoring data. The cluster ion extracted ion chromatograms report on the consistency of the ion formation and detection system. The data collected by the SprayDx method resemble the data collected for postcolumn infusion of analyte. The response of the cluster ions monitored reports on changes in the physical parameters of the ion source such as voltage and gas flow. SprayDx is also observed to report on ion suppression in a fashion very similar to a postcolumn infusion of analyte. We anticipate the method finding utility as a continuous readout on the performance of electrospray and other atmospheric pressure ionization processes. .

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2019

Methods for cluster cosmology and application to the SDSS in preparation for DES Year 1 release

ABSTRACT We implement the first blind analysis of cluster abundance data to derive cosmological constraints from the abundance and weak lensing signal of redMaPPer clusters in the Sloan Digital Sky Survey (SDSS). We simultaneously fit for cosmological parameters and the richness–mass relation of the clusters. For a flat Λ cold dark matter cosmological model with massive neutrinos, we find $S_8 \equiv \sigma _{8}(\Omega _\mathrm{ m}/0.3)^{0.5}=0.79^{+0.05}_{-0.04}$. This value is both consistent and competitive with that derived from cluster catalogues selected in different wavelengths. Our result is also consistent with the combined probes analyses by the Dark Energy Survey (DES), the Kilo-Degree Survey (KiDS), and with the cosmic microwave background (CMB) anisotropies as measured by Planck. We demonstrate that the cosmological posteriors are robust against variation of the richness–mass relation model and to systematics associated with the calibration of the selection function. In combination with baryon acoustic oscillation data and big bang nucleosynthesis data (Cooke et al.), we constrain the Hubble rate to be h = 0.66 ± 0.02, independent of the CMB. Future work aimed at improving our understanding of the scatter of the richness–mass relation has the potential to significantly improve the precision of our cosmological posteriors. The methods described in this work were developed for use in the forthcoming analysis of cluster abundances in the DES. Our SDSS analysis constitutes the first part of a staged-unblinding analysis of the full DES data set.

79 ASTRONOMY AND ASTROPHYSICS↗
NASA NTRS1981

Astronomical Data Center Bulletin, volume 1, number 2

Work in progress on astronomical catalogs is presented in 16 papers. Topics cover astronomical data center operations; automatic astronomical data retrieval at GSFC; interactive computer reference search of astronomical literature 1950-1976; formatting, checking, and documenting machine-readable catalogs; interactive catalog of UV, optical, and HI data for 201 Virgo cluster galaxies; machine-readable version of the general catalog of variable stars, third edition; galactic latitude and magnitude distribution of two astronomical catalogs; the catalog of open star clusters; infrared astronomical data base and catalog of infrared observations; the Air Force geophysics laboratory; revised magnetic tape of the N30 catalog of 5,268 standard stars; positional correlation of the two-micron sky survey and Smithsonian Astrophysical Observatory catalog sources; search capabilities for the catalog of stellar identifications (CSI) 1979 version; CSI statistics: blue magnitude versus spectral type; catalogs available from the Astronomical Data Center; and status report on machine-readable astronomical catalogs.

Nagy, T. A.↗
NASA NTRS1995

Fast Multipole Methods for Three-Dimensional N-body Problems

We are developing computational tools for the simulations of three-dimensional flows past bodies undergoing arbitrary motions. High resolution viscous vortex methods have been developed that allow for extended simulations of two-dimensional configurations such as vortex generators. Our objective is to extend this methodology to three dimensions and develop a robust computational scheme for the simulation of such flows. A fundamental issue in the use of vortex methods is the ability of employing efficiently large numbers of computational elements to resolve the large range of scales that exist in complex flows. The traditional cost of the method scales as Omicron (N(sup 2)) as the N computational elements/particles induce velocities at each other, making the method unacceptable for simulations involving more than a few tens of thousands of particles. In the last decade fast methods have been developed that have operation counts of Omicron (N log N) or Omicron (N) (referred to as BH and GR respectively) depending on the details of the algorithm. These methods are based on the observation that the effect of a cluster of particles at a certain distance may be approximated by a finite series expansion. In order to exploit this observation we need to decompose the element population spatially into clusters of particles and build a hierarchy of clusters (a tree data structure) - smaller neighboring clusters combine to form a cluster of the next size up in the hierarchy and so on. This hierarchy of clusters allows one to determine efficiently when the approximation is valid. This algorithm is an N-body solver that appears in many fields of engineering and science. Some examples of its diverse use are in astrophysics, molecular dynamics, micro-magnetics, boundary element simulations of electromagnetic problems, and computer animation. More recently these N-body solvers have been implemented and applied in simulations involving vortex methods. Koumoutsakos and Leonard (1995) implemented the GR scheme in two dimensions for vector computer architectures allowing for simulations of bluff body flows using millions of particles. Winckelmans presented three-dimensional, viscous simulations of interacting vortex rings, using vortons and an implementation of a BH scheme for parallel computer architectures. Bhatt presented a vortex filament method to perform inviscid vortex ring interactions, with an alternative implementation of a BH scheme for a Connection Machine parallel computer architecture.

Koumoutsakos, P.↗
NASA NTRS1986

Analyzing Multidimensional Image Data

Six computer programs perform histogram cluster analysis. Histogram Cluster Analysis Procedure (HICAP) developed to perform unsupervised classification of multidimensional image data. Clustering approach used in HICAP based on algorithm which uses multidimensional histogram to perform unsupervised classification of four-dimensional Landsat multispectral-scanner data. HICAP generalizes this procedure to process up to 32-bit data with arbitrary number of dimensions. Also incorporates efficiency improvements so classification requires less computation than original algorithm. Computational savings afforded by HICAP increase with number of dimensions in data. HICAP programs written in FORTRAN 77 for batch or interactive execution.

Wharton, S. W.↗
NASA NTRS1978

Compact and diffuse X-ray sources in the core of the Perseus cluster /Abell 426/

Data on the X-ray structure of the Perseus cluster of galaxies in the energy range from 2 to 10 keV are analyzed which were obtained during a 12-day observation with the rotating modulation collimator aboard SAS 3. In particular, the nature of the compact X-ray source coincident with NGC 1275 is examined. The data are shown to confirm the presence of a diffuse region of X-ray emission surrounding the nucleus of NGC 1275; it is found that a model which also includes a point source at the nucleus is consistent with the data if the point contribution is no larger than 20% of the X-ray flux detected within 3 to 5 arcmin of the galaxy. The results suggest that NGC 1275 may have accreted a large amount of gas from the intracluster medium and that the flux from the possible point source is consistent with a synchro/Compton model.

Helmken, H.↗
NASA NTRS2004

Model-Biased, Data-Driven Adaptive Failure Prediction

This final report, which contains a research summary and a viewgraph presentation, addresses clustering and data simulation techniques for failure prediction. The researchers applied their techniques to both helicopter gearbox anomaly detection and segmentation of Earth Observing System (EOS) satellite imagery.

Leen, Todd K.↗
DOE OSTI2020

RedMaPPer: Evolution and Mass Dependence of the Conditional Luminosity Functions of Red Galaxies in Galaxy Clusters

In this work we characterize the luminosity distribution, halo-mass dependence, and redshift evolution of red galaxies in galaxy clusters using the SDSS Data Release 8 redMaPPer cluster sample. We propose a simple prescription for the relationship between the luminosity of both red central and red satellite galaxies and the mass of their host halos, and show that this model is well fit by the data. Using a larger galaxy cluster sample than previously employed in the literature, we find that the luminosities of red central galaxies scale as $\langle \mathrm{log}L\rangle \propto {A}_{L}\mathrm{log}({M}_{200b})$, with A L = 0.39 ± 0.04, and that the scatter of the red central galaxy luminosity at fixed M 200b (${\sigma }_{\mathrm{log}L| M}$) is ${0.23}_{-0.04}^{+0.05}$ dex, with the error bar including systematics due to mis-centering of the cluster finder, photometry, and photometric redshift estimation. Our data prefers a positive correlation between the luminosity of red central galaxies and the observed richness of clusters at a fixed halo mass, with an effective correlation coefficient ${d}_{\mathrm{eff}}={0.36}_{-0.16}^{+0.17}$. The characteristic luminosity of red satellites becomes dimmer from z = 0.3 to z = 0.1 by ~20% after accounting for passive evolution. We estimate the fraction of galaxy clusters where the brightest red galaxy is not the central to be P BNC ~ 20%. We discuss implications of these findings in the context of galaxy evolution and the galaxy–halo connection.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2014

Star formation and substructure in galaxy clusters

We investigate the relationship between star formation (SF) and substructure in a sample of 107 nearby galaxy clusters using data from the Sloan Digital Sky Survey. Several past studies of individual galaxy clusters have suggested that cluster mergers enhance cluster SF, while others find no such relationship. The SF fraction in multi-component clusters (0.228 ± 0.007) is higher than that in single-component clusters (0.175 ± 0.016) for galaxies with M{sub r}{sup 0.1}<−20.5. In both single- and multi-component clusters, the fraction of star-forming galaxies increases with clustercentric distance and decreases with local galaxy number density, and multi-component clusters show a higher SF fraction than single-component clusters at almost all clustercentric distances and local densities. Comparing the SF fraction in individual clusters to several statistical measures of substructure, we find weak, but in most cases significant at greater than 2σ, correlations between substructure and SF fraction. These results could indicate that cluster mergers may cause weak but significant SF enhancement in clusters, or unrelaxed clusters exhibit slightly stronger SF due to their less evolved states relative to relaxed clusters.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2015

Adding Data Management Services to Parallel File Systems

The objective of this project, called DAMASC for “Data Management in Scientific Computing”, is to coalesce data management with parallel file system management to present a declarative interface to scientists for managing, querying, and analyzing extremely large data sets efficiently and predictably. Managing extremely large data sets is a key challenge of exascale computing. The overhead, energy, and cost of moving massive volumes of data demand designs where computation is close to storage. In current architectures, compute/analysis clusters access data in a physically separate parallel file system and largely leave it scientist to reduce data movement. Over the past decades the high-end computing community has adopted middleware with multiple layers of abstractions and specialized file formats such as NetCDF-4 and HDF5. These abstractions provide a limited set of high-level data processing functions, but have inherent functionality and performance limitations: middleware that provides access to the highly structured contents of scientific data files stored in the (unstructured) file systems can only optimize to the extent that file system interfaces permit; the highly structured formats of these files often impedes native file system performance optimizations. We are developing Damasc, an enhanced high-performance file system with native rich data management services. Damasc will enable efficient queries and updates over files stored in their native byte-stream format while retaining the inherent performance of file system data storage via declarative queries and updates over views of underlying files. Damasc has four key benefits for the development of data-intensive scientific code: (1) applications can use important data-management services, such as declarative queries, views, and provenance tracking, that are currently available only within database systems; (2) the use of these services becomes easier, as they are provided within a familiar file-based ecosystem; (3) common optimizations, e.g., indexing and caching, are readily supported across several file formats, avoiding effort duplication; and (4) performance improves significantly, as data processing is integrated more tightly with data storage. Our key contributions are: SciHadoop which explores changes to MapReduce assumption by taking advantage of semantics of structured data while preserving MapReduce’s failure and resource management; DataMods which extends common abstractions of parallel file systems so they become programmable such that they can be extended to natively support a variety of data models and can be hooked into emerging distributed runtimes such as Stanford’s Legion; and Miso which combines Hadoop and relational data warehousing to minimize time to insight, taking into account the overhead of ingesting data into data warehousing.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
NASA NTRS1978

Compton scattering of microwave background radiation by gas in galaxy clusters

Based on data on the X-ray spectrum of the Coma cluster, interpreted as thermal bremsstrahlung, the expected brightness depletion from Compton scattering of the microwave background in the direction of the cluster is computed. The calculated depletion is about one-third that recently observed by Gull and Northover, and the discrepancy is discussed. In comparing the observed microwave depletion in the direction of other clusters which are X-ray sources it is found that there is no correlation with the cluster X-ray luminosity. Consequently, the microwave depletion observations cannot yet be taken as good evidence for a thermal bremsstrahlung origin for the X-ray emission. The perturbation from Compton scattering of photons on the high-frequency (Wien) tail of the blackbody distribution is computed and found to be much larger than predicted in previous calculations. In the Wien tail the effect is a relative increase in the blackbody intensity that is appreciably greater in magnitude than the depletion in the Rayleigh-Jeans domain.

Gould, R. J.↗
DOE OSTI2017

Comparison of Clustering Techniques for Residential Energy Behavior using Smart Meter Data

Current practice in whole time series clustering of residential meter data focuses on aggregated or subsampled load data at the customer level, which ignores day-to-day differences within customers. This information is critical to determine each customer’s suitability to various demand side management strategies that support intelligent power grids and smart energy management. Clustering daily load shapes provides fine-grained information on customer attributes and sources of variation for subsequent models and customer segmentation. In this paper, we apply 11 clustering methods to daily residential meter data. We evaluate their parameter settings and suitability based on 6 generic performance metrics and post-checking of resulting clusters. Finally, we recommend suitable techniques and parameters based on the goal of discovering diverse daily load patterns among residential customers. To the authors’ knowledge, this paper is the first robust comparative review of clustering techniques applied to daily residential load shape time series in the power systems’ literature.

24 POWER TRANSMISSION AND DISTRIBUTION↗
NASA NTRS1987

Narrow-angle tail radio sources and the distribution of galaxy orbits in Abell clusters

The present data on the orientations of the tails with respect to the cluster centers of a sample of 70 narrow-angle-tail (NAT) radio sources in Abell clusters show the distribution of tail angles to be inconsistent with purely radial or circular orbits in all the samples, while being consistent with isotropic orbits in (1) the whole sample, (2) the sample of NATs far from the cluster center, and (3) the samples of morphologically regular Abell clusters. Evidence for very radial orbits is found, however, in the sample of NATs near the cluster center. If these results can be generalized to all cluster galaxies, then the presence of radial orbits near the center of Abell clusters suggests that violent relaxation may not have been fully effective even within the cores of the regular clusters.

O'Dea, Christopher P.↗
DOE OSTI2016

HeCS-SZ: The hectospec survey of Sunyaev–Zeldovich-selected clusters

We estimate cluster masses and velocity dispersions for 123 clusters from optical spectroscopy to compare the Sunyaev–Zeldovich (SZ) mass proxy and dynamical masses. Our new survey, HeCS-SZ (Hectospec Cluster Survey of SZ-selected clusters), includes 7721 new or remeasured redshifts from MMT/Hectospec observations of 21 SZ-selected clusters at redshifts z = 0.05–0.20. We supplement the Hectospec data with spectra from the Sloan Digital Sky Survey (SDSS) and cluster data from the Cluster Infall Regions in SDSS project and the Hectospec Cluster Survey, our Hectospec survey of clusters selected by X-ray flux. We measure the scaling relation between velocity dispersion and SZ mass estimates from the integrated Compton parameter for an SZ-complete sample of 83 clusters. The observed relation agrees very well with a simple virial scaling from mass (based on SZ) to velocity dispersion. The SZ mass estimates (calibrated with hydrostatic X-ray mass estimates) are not significantly biased compared to dynamical mass estimates under the assumption of small velocity bias of galaxies compared to dark matter particles. Significant mass bias in SZ mass estimates could relieve tension between cosmological results from Planck SZ cluster counts and Planck CMB data. In principle, SZ mass bias and velocity bias of galaxies could conspire to yield good agreement, but the required velocity bias is σ{sub galaxy}≈0.77σ{sub DM}, outside the range of recent models of velocity bias in the literature. More likely, SZ mass bias and velocity bias are both small, and the tension between SZ cluster counts and CMB data requires another explanation.

79 ASTRONOMY AND ASTROPHYSICS↗