Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data clustering”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 415 records · Page 23
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Clusters of galaxies up to z = 1.5 identified from photometric data of the Dark Energy Survey and unWISE

ABSTRACT Using photometric data from the Dark Energy Survey and the Wide-field Infrared Survey Explorer, we estimate photometric redshifts for 105 million galaxies using the nearest-neighbour algorithm. From such a large data base, 151 244 clusters of galaxies are identified in the redshift range of 0.1 < z ≲ 1.5 based on the overdensity of the total stellar mass of galaxies within a given photometric redshift slice, among which 76 826 clusters are newly identified and 30 477 clusters have a redshift z > 1. We cross-match these clusters with those in the catalogues identified from the X-ray surveys and the Sunyaev–Zel’dovich (SZ) effect by the Planck, South Pole Telescope and Atacama Cosmology Telescope surveys, and get the redshifts for 45 X-ray clusters and 56 SZ clusters. More than 95 per cent SZ clusters in the sky region have counterparts in our catalogue. We find multiple optical clusters in the line of sight towards about 15 per cent of SZ clusters.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2025

Graph-Based Prediction of Spatio-Temporal Vaccine Hesitancy From Insurance Claims Data

Growing vaccine hesitancy is contributing to the decline in immunization rates for highly contagious, vaccine-preventable childhood diseases. Therefore, there has been a significant interest in understanding how hesitancy is spreading at higher spatio-temporal resolutions, enabling more targeted interventions. Motivated by this, we study the problem of prediction of vaccine hesitancy at the ZIP Code level, referred to as the VaxHesitancy problem. A significant challenge for this problem is the lack of high-resolution data that indicates hesitancy. Here, we develop a hybrid VaxHesSTL framework that combines a Graph Neural Network (GNN) and a Recurrent Neural Network (RNN) to address the VaxHesitancy problem. The GNN uses a ZIP Code-level network to capture spatial signals from neighboring areas, while the RNN models the temporal dynamics present in the data. We train and evaluate VaxHesSTL using a large dataset, namely the All-Payer Claims Databases (APCD), for Virginia, consisting of insurance claims from over five million individuals for six years. We find that an aggregated contact network or graph, developed from a detailed activity-based population network, plays an important role in the performance of VaxHesSTL, compared to graph models based solely on spatial proximity. Experiments demonstrate that VaxHesSTL outperforms a range of state-of-the-art baselines, which rely solely on historical time series data without accounting for spatial relationships. Since hesitancy data at higher spatial resolution is often unavailable or hard to get, we incorporate an active learning approach with our VaxHesSTL framework to optimize the training set without compromising the prediction performance. We find that hesitancy data for only 18% of ZIP Codes selected by active learning allows us to forecast hesitancy for all the ZIP Codes in the Virginia.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2013

Accelerating semantic graph databases on commodity clusters

We are developing a full software system for accelerating semantic graph databases on commodity cluster that scales to hundreds of nodes while maintaining constant query throughput. Our framework comprises a SPARQL to C++ compiler, a library of parallel graph methods and a custom multithreaded runtime layer, which provides a Partitioned Global Address Space (PGAS) programming model with fork/join parallelism and automatic load balancing over a commodity clusters. We present preliminary results for the compiler and for the runtime.

commodity clusters↗
DOE OSTI2017

Study of nuclear matter in hard proton-nuclei and nuclei-nuclei collisions at the U70 accelerator (FLUKTON project proposal)

A two-arm spectrometer FLUKTON for investigations in the field of relativistic nuclear physics at U70 energies is proposed to be constructed on base of the existing detector SPIN (IHEP, Protvino). The main objective is to obtain new data on clusters of cold superdense nuclear matter. Interaction of a high intensity proton beam with nuclear targets and an ion beam with liquid hydrogen and nuclear targets will be studied.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2024

Unveiling Temporal Performance Deviation: Leveraging Clustering in Microservices Performance Analysis

As the market for cloud computing continues to grow, an increasing number of users are deploying applications as microservices. The shift introduces unique challenges in identifying and addressing performance issues, particularly within large and complex infrastructures. To address this challenge, we propose a methodology that unveils temporal performance deviations in microservices by clustering containers based on their performance characteristics at different time intervals. Showcasing our methodology on the Alibaba dataset, we found both stable and dynamic performance patterns, providing a valuable tool for enhancing overall performance and reliability in modern application landscapes.

Clustering↗
DOE OSTI2019

ldlms_plugins_llnl

ldlms_plugins_llnl contains LLNL-developed plugins for LDMS (Lightweight Distributed Metric Service), which is software for low overhead monitoring, transport and storage of cluster metric data. These plugins add to LDMS the capability to monitor Lustre file system clients, metadata servers, and object storage servers.

Morrone, ChristopherJ↗
DOE OSTI2023

Subseasonal Representation and Predictability of North American Weather Regimes Using Cluster Analysis

Abstract This study focuses on assessing the representation and predictability of North American weather regimes, which are persistent large-scale atmospheric patterns, in a set of initialized subseasonal reforecasts created using the Community Earth System Model, version 2 (CESM2). The k -means clustering was used to extract four key North American (10°–70°N, 150°–40°W) weather regimes within ERA5 reanalysis, which were used to interpret CESM2 subseasonal forecast performance. Results show that CESM2 can recreate the climatology of the four main North American weather regimes with skill but exhibits biases during later lead times with overoccurrence of the West Coast high regime and underoccurrence of the Greenland high and Alaskan ridge regimes. Overall, the West Coast high and Pacific trough regimes exhibited higher predictability within CESM2, partly related to El Niño. Despite biases, several reforecasts were skillful and exhibited high predictability during later lead times, which could be partly attributed to skillful representation of the atmosphere from the tropics to extratropics upstream of North America. The high predictability at the subseasonal time scale of these case-study examples was manifested as an “ensemble realignment,” in which most ensemble members agreed on a prediction despite ensemble trajectory dispersion during earlier lead times. Weather regimes were also shown to project distinct temperature and precipitation anomalies across North America that largely agree with observational products. This study further demonstrates that unsupervised learning methods can be used to uncover sources and limits of subseasonal predictability, along with systematic biases present in numerical prediction systems. Significance Statement North American weather regimes are large-scale atmospheric patterns that can persist for several days. Their skillful subseasonal (2 weeks or greater) prediction can provide valuable lead time to prepare for temperature and precipitation anomalies that can stress energy and water resources. The purpose of this study was to assess the climatological representation and subseasonal predictability of four key North American weather regimes using a research subseasonal prediction system and clustering analysis. We found that the Pacific trough and West Coast high regimes exhibited higher predictability than other regimes and that skillful representation of conditions across the tropics and extratropics can increase predictability during later lead times. Future work will quantify causal pathways associated with high predictability.

58 GEOSCIENCES↗
DOE OSTI2025

Investigating Aerosol and Meteorological Influences on Convective Clouds in Houston, Texas, during the TRACER/ESCAPE Field Campaigns

Aerosols serve as cloud condensation nuclei, shaping the microphysical properties of cloud droplets. Aerosol effects on convective clouds are complex and remain controversial. The debate centers around the process of aerosol-induced invigoration of deep convection, a phenomenon that could significantly affect convective cloud properties but lacks robust evidence due to methodological limitations in observational approaches and questions about the robustness of modeling studies. Resolving these discrepancies is crucial for understanding how aerosols affect the atmosphere. Here, this study examines the effects of meteorological and aerosol parameters in a weakly synoptic-driven convective environment, where the influence of aerosols may be more pronounced and observable. Daily atmospheric soundings and aerosol concentrations from several ground instruments collected during the summer of 2022 in Houston, Texas, as part of the Tracking Aerosol Convection interactions Experiment (TRACER) and Experiment of Sea Breeze Convection, Aerosols, Precipitation, and Environment (ESCAPE) field campaigns are analyzed. Statistical learning methods are applied to uncover the complex relationships between aerosols, meteorology, and convective cloud characteristics, such as cell area and echo-top height. The findings reveal that higher aerosol concentrations are associated with narrower convective cells, which we argue contradicts the idea of stronger convection with increased aerosol loading. However, once the data are clustered by the synoptic environment, the relationship between aerosol loading and convective cell area diminishes, indicating that the covariablity between synoptic-scale weather patterns, local thermodynamics, and aerosol loading makes it challenging to draw definitive conclusions about the specific impacts of aerosols on convective cloud properties.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2018

The experimental study of the phase structure of strongly interacting matter

Over the duration of the project period, our research has been focused on the beam energy scan data conducted at the Relativistic Heavy Ion Collider (RHIC) at Brookhaven National Laboratory in order to search for the location of the Critical End Point (CEP) temperature or cross-over in the nuclear phase diagram using the STAR detector at RHIC and to determine the Equation of State (EOS) and the QGP transport coefficients. Working at the RHIC early on, we discovered the Long Range forward-backward multiplicity Correlations (LRC) which quantitatively confirmed the predictions of saturation models, in particular, the Color Glass Condensate (CGC) and the Color String Percolation Model (CSPM). We have used the CSPM to extract the initial temperature from the data. The clustering of color sources may be the way to achieve thermalization in A-A and N-N collisions. Recently we also used CSPM to measure the shear viscosity to entropy ratio at RHIC and LHC temperatures. The ratios are in agreement with values expected for a strongly interacting QGP. The thermodynamic quantities,temperature and the equation of state are in agreement with lattice QCD.

42 ENGINEERING↗
DOE OSTI2023

Improving Subsurface Stress Characterization for Carbon Dioxide Storage Projects by Incorporating Machine Learning Techniques

The overall objective of this project is to develop a framework for reliable characterization and prediction of the state of stress in the overburden and underburden (including the basement) in CO 2 storage reservoirs using machine learning and integrated geomechanics and geophysical methods. Specifically, we propose to develop workflow encompassing of technologies and/or methods to predict stress and pressure changes due to CO 2 injection in an active tertiary recovery site and their impacts on subtle fault activation, fractures and occurrence of microseismic events and compare responses to field observations. In this project, we anticipate using dataset from the Farnsworth field Unit (FWU) which is operated by Purdure Petroleum. A novel elastic-waveform VSP inversion technique will be used to estimate high-resolution spatial and temporal changes of elastic moduli in CO 2 storage reservoirs, which will be combined with velocity-stress relationship derived from laboratory tests to obtain subsurface pressure and stress. Clustered microseismic data will be jointly inverted for improved focal mechanisms. Least-squares reverse-time migration of microseismic waveform data will be performed to directly image fracture/fault zones. Additionally, a deep neural network machine learning technique with convolutional and recurrent layers will be used for learning the spectro-temporal structures in microseismic waveforms. The results of this geotechnical data analysis will be integrated to develop a high-resolution 3D mechanical earth model extending from the overburden sealing formations to the underburden including the basement. Mechanical properties will be derived through integration of mechanical logs, tests, available results from chemo-mechanical laboratory tests, and elastic inversion of seismic data using a combination of Bayesian and stochastic methods as well as machine learning technique. Failure features (faults/fractures) will be represented and/or modeled based on seismic and core data analysis. A transient hydrodynamic-geomechanical model will be developed through coupling with the calibrated FWU reservoir simulation model. The full physics coupled model will be used to train a reduced order proxy model using machine learning algorithm for estimating stress which will then be used with appropriate constitutive relationships and forward seismological models to simulate pressure changes and induced microseismicity. An advanced optimization framework will be developed to perform a history match to minimize error between field observations and simulated. The history matched proxy model will be verified against the full-physics equivalent. The field observations that will be used in the coupled model calibration process include pressure/stress inverted from VSP, moment magnitude from microseismic analysis, real time downhole pressure measurements, production and injection data. Parameter sensitivity and uncertainty analysis will be performed to characterize the impact of model parameter uncertainty on stress estimates. The proposed project will have significant impact on future field implementation of the proposed technology. Because the project field site is an ongoing CO 2 EOR development, the value of the new technology will be demonstrated in an operational context and evaluated as a viable risk mitigation strategy. Cost/benefit will be evaluated together with the various commercial incentives for CO 2 sequestration available to oil and gas operators. The extensive available dataset and ongoing data acquisition under the SWP Phase III work plan provides flexibility for investigation of multiple approaches and reduces technical risk.

58 GEOSCIENCES↗
DOE OSTI2023

Maximizing the impact of the Vera Rubin Observatory LSST, and DESI and CMB-S4 experiments in constraining gravity and dark sector physics

The work in this award has addressed our objectives on a number of fronts: Current graduate student Chris Wilson and Bean are determining how cosmic voids can be used to test the properties of gravity, leveraging simulations from DESI collaborator Baojiu Li and refining theoretical Gaussian Streaming Model approaches to predict galaxy dynamics in voids when the cosmology is not GR. This work is still underway, including analyses of the efficacy of void quadrupole correlation statistics and void size functions when studying dark matter halo statistics or the expected galaxy distributions from upcoming galaxy surveys; Current graduate student Yulin Gong, Bean with collaborators are refining techniques to extract out the kinetic Sunyaev Zel’dovich effect from galaxy cluster survey data and CMB temperature anisotropies, with the intent to further ready this pipeline development in readiness for spectroscopic data from DESI and LSST along with CMB-S4 measurements.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2024

Augmented Human Analysis (AHA)

Radio frequency (RF) signal monitoring generally emphasizes intentionally generated signals, such as WiFi, Bluetooth, or cellular transmissions. However, electronic devices also produce unintended radiated emissions (UREs), which could also be useful in RF spectrum analysis. In either case, deriving intelligence from RF signals is typically a human-intensive process requiring significant domain knowledge. In the Augmented Human Analysis (AHA) project, we investigate the utility of dimensionally aligned signal projection (DASP) and machine learning (ML) algorithms for accelerating RF analysis workflows. We find that while DASP algorithms can indeed highlight signal characteristics relevant for classification tasks, the choice of algorithmic hyperparameters greatly affects performance. To address this challenge, we evaluate the quality of DASP outputs using the silhouette score, which measures how well data points cluster; high silhouette scores indicate good clustering, and thus good hyperparameter values. This approach is critical for machine learning pipelines as the DASP parameters cannot be directly optimized during model training. By identifying good DASP parameters, and thus good DASP outputs, as a preprocessing step, we can decrease the amount of effort required for downstream ML model training. We demonstrate our workflow using a dataset of UREs from common household devices, showing that even without the aid of ML, proper selection of DASP parameters enables clustering by device type.

42 ENGINEERING↗
DOE OSTI2017

Hints on the Gradual Resizing of the Torus in AGNs through Decomposition of Spitzer/IRS Spectra

Several authors have claimed that less luminous active galactic nuclei (AGNs) are not capable of sustaining a dusty torus structure. Thus, a gradual resizing of the torus is expected when the AGN luminosity decreases. Our aim is to examine mid-infrared observations of local AGNs of different luminosities for the gradual resizing and disappearance of the torus. We applied the decomposition method described by Hernán-Caballero et al. to a sample of ∼100 Spitzer/IRS spectra of low-luminosity AGNs and powerful Seyferts in order to decontaminate the torus component from other contributors. We have also included Starburst objects to ensure secure decomposition of the Spitzer/IRS spectra. We have used the affinity propagation (AP) method to cluster the data into five groups within the sample according to torus contribution to the 5–15 μm range (C{sub torus}) and bolometric luminosity (L{sub bol}). The AP groups show a progressively higher torus contribution and an increase of the bolometric luminosity from Group 1 (C{sub torus}∼0% and log(L{sub bol})∼41) up to Group 5 (C{sub torus}∼80% and log(L{sub bol})∼44). We have fitted the average spectra of each of the AP groups to clumpy models. The torus is no longer present in Group 1, supporting its disappearance at low luminosities. We were able to fit the average spectra for the torus component in Groups 3 (C{sub torus}∼40% and log(L{sub bol})∼42.6), 4 (C{sub torus}∼60% and log(L{sub bol})∼43.7), and 5 to Clumpy torus models. We did not find a good fitting to Clumpy torus models for Group 2 (C{sub torus}∼18% and log(L{sub bol})∼42). This might suggest a different configuration and/or composition of the clouds for Group 2, which is consistent with the different gas content seen in Groups 1, 2, and 3, according to detections of H{sub 2} molecular lines. Groups 3, 4, and 5 show a trend of decreasing torus width (which leads to a likely decrease of the geometrical covering factor), although we cannot confirm it with the present data. Finally, Groups 3, 4, and 5 show an increase of the outer radius of the torus for higher luminosities, consistent with a resizing of the torus according to AGN luminosity.

79 ASTRONOMY AND ASTROPHYSICS↗
NASA NTRS1974

International Symposium on Remote Sensing of Environment, 9th, University of Michigan, Ann Arbor, Mich., April 15-19, 1974, Proceedings. Volumes 1, 2 & 3

The present work gathers together numerous papers describing the use of remote sensing technology for mapping, monitoring, and management of earth resources and man's environment. Studies using various types of sensing equipment are described, including multispectral scanners, radar imagery, spectrometers, lidar, and aerial photography, and both manual and computer-aided data processing techniques are described. Some of the topics covered include: estimation of population density in Tokyo districts from ERTS-1 data, a clustering algorithm for unsupervised crop classification, passive microwave sensing of moist soils, interactive computer processing for land use planning, the use of remote sensing to delineate floodplains, moisture detection from Skylab, scanning thermal plumes, electrically scanning microwave radiometers, oil slick detection by X-band synthetic aperture radar, and the use of space photos for search of oil and gas fields. Individual items are announced in this issue.

Source record↗
NASA NTRS1975

The effect of grazing flow on the steady state resistance of square-edged orifices

A simple inviscid interaction model has been developed which uses thin airfoil theory to account for pressure forces exerted at the interface between the orifice and grazing flow together with a one-dimensional discharge coefficient concept. The effect of grazing flow boundary layer thickness was also included in the model. Resistance measurements for a wide range of grazing flow speeds and orifice flow rates collapse into a single curve when plotted in terms of effective discharge coefficient versus orifice-to-grazing velocity ratio. The correlation curves for inflow and outflow are different. Data for clustered orifices collapse in the same way as the single orifice. The effect of boundary layer thickness is compared with model predictions. The effect of orifice length-diameter ratio is shown to be significant for inflow but negligible for outflow.

Rogers, T.↗
NASA NTRS1976

Effect of grazing flow on steady-state resistance of isolated square-edged orifices

Steady state diagnostic testing of an isolated orifice has shown the nature of the interaction between grazing and orifice flow causing the large increase in orifice resistance for both inflow and outflow. A simple inviscid interaction model is developed which uses thin aerofoil theory to account for pressure forces exerted at the interface between the orifice and grazing flow together with a one-dimensional discharge coefficient concept. The effect of grazing flow boundary layer thickness is also included in the model. Resistance measurements for each orifice tested, over a wide range of grazing flow speeds and flow rates, collapse into a single curve when plotted in terms of effective discharge coefficient against orifice to grazing velocity ratio. The correlation curves for inflow and outflow are different. Data for clustered orifices collapse in the same way as those for the single orifice. The effect of boundary layer thickness is compared with model predictions.

Rogers, T.↗