SEARCH · Engineering Papers

Results for “data analysis cluster”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 271 records · Page 15

All sources NASA NTRS DOE OSTI

DOE OSTI2024

Power Profile Monitoring and Tracking Evolution of System-Wide HPC Workloads

The power & energy demands of HPC machines have grown significantly. Modern exascale HPC systems require tens of megawatts of combined power for computing resources and cooling facilities at full capacity. The current energy trend is not sustainable for future HPC systems, and there is a need to work toward the energy efficiency aspect of HPC performance. Energy awareness of the HPC applications at the job level is essential for running an efficient HPC system. This work aims to develop a pipeline to provide a production-level system-wide overview of the HPC workloads' power profile while handling evolving workloads exhibiting new power trends. We developed an open-set classification model for HPC jobs based on the properties of power profiles to continuously provide a system-wide holistic view of recently completed jobs. The pipeline helps continuously monitor the job-level power usage pattern of HPC and enables us to capture the new trends in applications' power behavior. We employed a comprehensive set of techniques to generate job-level data, custom-designed feature extraction methods to extract critical features from jobs' power profiles, clustering techniques powered by generative modeling, and open-set classification for identifying job profiles into known classes or an unknown set. With extensive evaluations, we demonstrate the effectiveness of each component in our pipeline. We provide an analysis of the resulting clusters that characterize the power profile landscape of the Summit supercomputer from more than 60K jobs executed in a year. The open-set classification classifies the known data sets into known classes with high accuracy and identifies unknown data noints with over 85% accuracy.

Karimi, Ahmad Maroof↗

DOE OSTI2023

A review on recent machine learning applications for imaging mass spectrometry studies

Imaging mass spectrometry (IMS) is a powerful analytical technique widely used in biology, chemistry, and materials science fields that continue to expand. IMS provides a qualitative compositional analysis and spatial mapping with high chemical specificity. The spatial mapping information can be 2D or 3D depending on the analysis technique employed. Due to the combination of complex mass spectra coupled with spatial information, large high-dimensional datasets (hyperspectral) are often produced. Therefore, the use of automated computational methods for an exploratory analysis is highly beneficial. The fast-paced development of artificial intelligence (AI) and machine learning (ML) tools has received significant attention in recent years. These tools, in principle, can enable the unification of data collection and analysis into a single pipeline to make sampling and analysis decisions on the go. There are various ML approaches that have been applied to IMS data over the last decade. Here, in this review, we discuss recent examples of the common unsupervised (principal component analysis, non-negative matrix factorization, k-means clustering, uniform manifold approximation and projection), supervised (random forest, logistic regression, XGboost, support vector machine), and other methods applied to various IMS datasets in the past five years. The information from this review will be useful for specialists from both IMS and ML fields since it summarizes current and representative studies of computational ML-based exploratory methods for IMS.

47 OTHER INSTRUMENTATION↗

DOE OSTI2023

Hyper Suprime-Cam Year 3 results: Cosmology from galaxy clustering and weak lensing with HSC and SDSS using the minimal bias model

Here, we present cosmological parameter constraints from a blind joint analysis of three two-point correlation functions measured from the Year 3 Hyper Suprime-Cam (HSC-Y3) imaging data, covering about 416 deg 2 , and the SDSS DR11 spectroscopic galaxies spanning the redshift range [0.15, 0.70]. We subdivide the SDSS galaxies into three luminosity-cut, and therefore nearly volume-limited samples separated in redshift, each of which acts as a large-scale structure tracer characterized by the measurement of the projected correlation function, w p ⁡(R). We also use the measurements of the galaxy-galaxy weak-lensing signal Δ⁢Σ⁡(R) for each of these SDSS samples which act as lenses for a secure sample of source galaxies selected from the HSC-Y3 shape catalog based on their photometric redshifts. We combine these measurements with the cosmic shear correlation functions, ξ ± ⁡($\vartheta$) measured for our HSC source sample. We model these observables with the minimal bias model of the galaxy clustering observables in the context of a flat Λ⁢CDM cosmology. We use conservative scale cuts, R >12 and 8⁢ℎ -1 Mpc for Δ⁢Σ and w p , respectively, where the minimal bias model is valid, in addition to conservative prior on the residual bias in the mean redshift of the HSC photometric source galaxies. We present various validation tests of our model as well as analysis methods. Our baseline analysis yields S 8 = $0.77⁢5^{+0.043}_{-0.038}$ (68% C.I.) for the Λ⁢CDM model, after marginalizing over uncertainties in other parameters. Our value of S 8 is consistent with that from the Planck 2018 data, but the credible interval of our result is still relatively large. We show that various internal consistency tests based on different splits of the data are passed. Our results are statistically consistent with those of a companion paper, which extends this analysis to smaller scales with an emulator-based halo model, using Δ⁢Σ⁡(R) and w p ⁡(R) down to R >3 and 2⁢ℎ -1 Mpc, respectively.

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2021

Structural Changes during the Growth of Atomically Precise Metal Oxido Nanoclusters from Combined Pair Distribution Function and Small-Angle X-ray Scattering Analysis

The combination of in situ pair distribution function (PDF) analysis and small-angle X-ray scattering (SAXS) enables analysis of the formation mechanism of metal oxido nanoclusters and cluster–solvent interactions as they take place. Herein, we demonstrate the method for the formation of clusters with a [Bi 38 O 45 ] core. Upon dissolution of crystalline [Bi 6 O 5 (OH) 3 (NO 3 ) 5 ]∙3 H 2 O in DMSO, an intermediate rapidly forms, which slowly grows to stable [Bi 38 O 45 ] clusters. To identify the intermediate, we developed an automated modeling method, where smaller [Bi x O y ] structures based on the [Bi 38 O 45 ] framework are tested against the data. [Bi 22 O 26 ] was identified as the main intermediate species, illustrating how combined PDF and SAXS analysis is a powerful tool to gain insight into nucleation on an atomic scale. PDF also provides information on the interaction between nanoclusters and solvent, which is shown to depend on the nature of the ligands on the cluster surface.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

DOE OSTI2022

Accuracy of power spectra in dissipationless cosmological simulations

Abstract We exploit a suite of large N-body simulations (up to N = 40963) performed with abacus, of scale-free models with a range of spectral indices n, to better understand and quantify convergence of the matter power spectrum. Using self-similarity to identify converged regions, we show that the maximal wavenumber resolved at a given level of accuracy increases monotonically as a function of time. At 1 per cent level it starts at early times from a fraction of $k_\Lambda$, the Nyquist wavenumber of the initial grid, and reaches at most, if the force softening is sufficiently small, ${\sim}2{-}3 k_\Lambda$ at the very latest times we evolve to. At the $5{{\ \rm per\ cent}}$ level, accuracy extends up to wavenumbers of order $5k_\Lambda$ at late times. Expressed as a suitable function of the scale-factor, accuracy shows a very simple n-dependence, allowing a extrapolation to place conservative bounds on the accuracy of N-body simulations of non-scale-free models like LCDM. We note that deviations due to discretization in the converged range are not well modelled by shot noise, and subtracting it in fact degrades accuracy. Quantitatively our findings are broadly in line with the conservative assumptions about resolution adopted by recent studies using large cosmological simulations (e.g. Euclid Flagship) aiming to constrain the mildly non-linear regime. On the other hand, we remark that conclusions about small-scale clustering (e.g. concerning the validity of stable clustering) obtained using PS data at wavenumbers larger than a few $k_\Lambda$ may need revision in light of our convergence analysis.

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2022

DAAP (Data Analytics Application Profiling)

The Data Analytics Application Profiling (DAAP) utility is a software program comprised of a library and scripts to provide basic instrumenting of parallel scientific software applications while they are running on an HPC cluster, and transmit this data over a secure connection as individual records to aggregators. When used in conjunction with additional open source software components (Telegraf, FluentD, and RabbitMQ), the data collected by DAAP can be transmitted off-cluster to an analytics system of the user’s choice for further analysis.

Shereda, Charles↗

DOE OSTI2026

Fiducial-cosmology-dependent systematics for the DESI 2024 full-shape analysis

We assess the impact of the fiducial cosmology choice on cosmological inference from full-shape (FS) fits of the galaxy power spectrum in the DESI 2024 Data Release 1 (DR1). Using a suite of AbacusSummit DR1 mock catalogues based on the Planck 2018 best-fit cosmology, we quantify potential systematic shifts introduced by analysing the data under five secondary cosmologies — featuring variations in matter density, thawing dark energy, higher effective number of neutrino species, reduced clustering amplitude, and the DESI DR1 BAO best-fit w 0 w a CDM cosmology — relative to DESI's baseline Planck 2018 cosmology. We investigate two complementary FS analysis approaches: full-modelling (FM) and ShapeFit (SF), each with distinct sensitivities to the assumed fiducial model. Across all tracers, we find for FM that systematic shifts induced by fiducial cosmology mismatches remain well below the DESI DR1 statistical uncertainties, with maximum deviations of 0.22σ DR1 in ΛCDM scenarios and 0.12σ DR1+SN when including SN Ia mock data in extended w 0 w a CDM fits. For SF, the shifts in the compressed parameters remain below 0.45σ DR1 for all tracers and cosmologies.

dark energy experiments↗

DOE OSTI2023

Detection of the significant impact of source clustering on higher order statistics with DES Year 3 weak gravitational lensing data

We measure the impact of source galaxy clustering on higher order summary statistics of weak gravitational lensing data. By comparing simulated data with galaxies that either trace or do not trace the underlying density field, we show that this effect can exceed measurement uncertainties for common higher order statistics for certain analysis choices. We evaluate the impact on different weak lensing observables, finding that third moments and wavelet phase harmonics are more affected than peak count statistics. Using Dark Energy Survey (DES) Year 3 (Y3) data, we construct null tests for the source-clustering-free case, finding a p-value of p = 4 × 10 −3 (2.6σ) using third-order map moments and p = 3 × 10 −11 (6.5σ) using wavelet phase harmonics. The impact of source clustering on cosmological inference can be either included in the model or minimized through ad hoc procedures (e.g. scale cuts). We verify that the procedures adopted in existing DES Y3 cosmological analyses were sufficient to render this effect negligible. Failing to account for source clustering can significantly impact cosmological inference from higher order gravitational lensing statistics, e.g. higher order N-point functions, wavelet-moment observables, and deep learning or field-level summary statistics of weak lensing maps.

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2025

Dark energy survey year 3 results: Cosmological constraints from cluster abundances, weak lensing, and galaxy clustering

Galaxy clusters provide a unique probe of the late-time cosmic structure and serve as a powerful independent test of the Λ⁢CDM model. This work presents the first set of cosmological constraints derived with ∼16,000 optically selected redMaPPer clusters across nearly 5000 deg 2 using DES year 3 datasets. Our analysis leverages a consistent modeling framework for galaxy cluster cosmology and DES-Y3 joint analyses of galaxy clustering and weak lensing (3 × 2⁢pt), ensuring direct comparability with the DES-Y3 3 × 2⁢pt analysis. Here, we obtain constraints of 𝑆 8 = 0.864 ± 0.035 and Ω m = $0.26⁢5^{+0.019}_{−0.031}$ from the cluster-based data vector. We find that cluster constraints and 3 × 2⁢pt constraints are consistent under the Λ⁢CDM model with a posterior predictive distribution (PPD) value of 0.53. The consistency between clusters and 3 × 2⁢pt provides a stringent test of Λ⁢CDM across different mass and spatial scales. Jointly analyzing clusters with 3 × 2⁢pt further improves cosmological constraints, yielding 𝑆 8 = $0.81⁢1^{+0.022}_{−0.020}$ and Ω m = $0.29⁢4^{+0.022}_{−0.033}$, a 24% improvement in the Ω m − 𝑆 8 figure of merit over 3 × 2⁢pt alone. Moreover, we find no significant deviation from the Planck CMB constraints with a probability to exceed (PTE) value of 0.6, significantly reducing previous 𝑆 8 tension claims. Finally, combining DES 3 × 2⁢pt, DES clusters, and Planck CMB places an upper limit on the sum of neutrino masses of ∑𝑚 𝜈 < 0.26 eV at 95% confidence under the Λ⁢CDM model. These results establish optically selected clusters as a key cosmological probe and pave the way for cluster-based analyses in upcoming stage-IV surveys such as LSST, Euclid, and Roman.

Abbott, T. M. C. [NSF’s National Optical-Infrared ↗

DOE OSTI2022

Study of charged-particle emission in neutron reactions on 10 B

The 10 B + n system has been studied by measuring charged particles from neutron-induced reactions in the 1 to 20 MeV energy range. Protons, deuterons, tritons, and α particles were measured at four angles using the pulsed white neutron spectrum at the Los Alamos Neutron Science Center Weapons Neutron Research facility. Differential cross sections for each species are reported. These new data are combined with literature data in an R-matrix analysis of the 11 B compound system. Furthermore, a comparison is made to the molecular and cluster states candidates observed in these reactions.

43 PARTICLE ACCELERATORS↗

DOE OSTI2023

The Aemulus Project. V. Cosmological Constraint from Small-scale Clustering of BOSS Galaxies

We analyze clustering measurements of BOSS galaxies using a simulation-based emulator of two-point statistics. We focus on the monopole and quadrupole of the redshift-space correlation function, and the projected correlation function, at scales of 0.1 ~ 60 h -1 Mpc. Although our simulations are based on wCDM with general relativity (GR), we include a scaling parameter of the halo velocity field, γ f , defined as the amplitude of the halo velocity field relative to the GR prediction. We divide the BOSS data into three redshift bins. After marginalizing over other cosmological parameters, galaxy bias parameters, and the velocity scaling parameter, we find fσ 8 (z = 0.25) = 0.413 ± 0.031, fσ 8 (z = 0.4) = 0.470 ± 0.026, and fσ 8 (z = 0.55) = 0.396 ± 0.022. Compared with Planck observations using a flat Lambda cold dark matter model, our results are lower by 1.9σ, 0.3σ, and 3.4σ, respectively. These results are consistent with other recent simulation-based results at nonlinear scales, including weak lensing measurements of BOSS LOWZ galaxies, two-point clustering of eBOSS LRGs, and an independent clustering analysis of BOSS LOWZ. All these results are generally consistent with a combination of ${\gamma }_{f}^{1/2}{\sigma }_{8}\approx 0.75$. We note, however, that the BOSS data is well fit assuming GR, i.e., γf = 1. We cannot rule out an unknown systematic error in the galaxy bias model at nonlinear scales, but near-future data and modeling will enhance our understanding of the galaxy–halo connection, and provide a strong test of new physics beyond the standard model.

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2024

Oleaginous Yeast Biology Elucidated With Comparative Transcriptomics

ABSTRACT Extremophilic yeasts have favorable metabolic and tolerance traits for biomanufacturing‐ like lipid biosynthesis, flavinogenesis, and halotolerance – yet the connection between these favorable phenotypes and strain genotype is not well understood. To this end, this study compares the phenotypes and gene expression patterns of biotechnologically relevant yeasts Yarrowia lipolytica , Debaryomyces hansenii , and Debaryomyces subglobosus grown under nitrogen starvation, iron starvation, and salt stress. To analyze the large data set across species and conditions, two approaches were used: a “network‐first” approach where a generalized metabolic network serves as a scaffold for mapping genes and a “cluster‐first” approach where unsupervised machine learning co‐expression analysis clusters genes. Both approaches provide insight into strain behavior. The network‐first approach corroborates that Yarrowia upregulates lipid biosynthesis during nitrogen starvation and provides new evidence that riboflavin overproduction in Debaryomyces yeasts is overflow metabolism that is routed to flavin cofactor production under salt stress. The cluster‐first approach does not rely on annotation; therefore, the coexpression analysis can identify known and novel genes involved in stress responses, mainly transcription factors and transporters. Therefore, this work links the genotype to the phenotype of biotechnologically relevant yeasts and demonstrates the utility of complementary computational approaches to gain insight from transcriptomics data across species and conditions.

Weintraub, Sarah J. [Department of Bioinformatics ↗

DOE OSTI2024

Analysis of BOSS galaxy data with weighted skew-spectra

We present the first application of the weighted skew-spectra to analyze non-Gaussian information in galaxy survey data. Using the tree-level galaxy skew-spectra together with the one-loop power spectrum multipoles, we analyze the Sloan Digital Sky Survey (SDSS)-III Baryon Oscillation Spectroscopic Survey (BOSS) galaxy clustering data, and target our search towards the equilateral bispectrum shape of primordial non-Gaussianity. We use the Effective Field Theory model for the galaxy power spectrum and bispectrum, and account for systematic effects, such as the survey geometry. From our likelihood analysis, we find f NL equil = -34 +296 -334 at 68% CL, consistent with previous works, while systematic errors from our treatment of the survey geometry lead to an unreliable estimation of f NL ortho . As a result, we further constrain the bias and counterterm parameters, while keeping the cosmology fixed to Planck 2018 values. As a check, we also validate our analysis pipeline using theNseries simulation suite.

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2021

Understanding nanoscale structural distortions in Pb(Zr 0.2 Ti 0.8 )O 3 by utilizing X-ray nanodiffraction and clustering algorithm analysis

Hard X-ray nanodiffraction provides a unique nondestructive technique to quantify local strain and structural inhomogeneities at nanometer length scales. However, sample mosaicity and phase separation can result in a complex diffraction pattern that can make it challenging to quantify nanoscale structural distortions. In this work, a k-means clustering algorithm was utilized to identify local maxima of intensity by partitioning diffraction data in a three-dimensional feature space of detector coordinates and intensity. This technique has been applied to X-ray nanodiffraction measurements of a patterned ferroelectric PbZr 0.2 Ti 0.8 O 3 sample. The analysis reveals the presence of two phases in the sample with different lattice parameters. A highly heterogeneous distribution of lattice parameters with a variation of 0.02 Å was also observed within one ferroelectric domain. This approach provides a nanoscale survey of subtle structural distortions as well as phase separation in ferroelectric domains in a patterned sample.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

DOE OSTI2025

Poster Abstract: Leveraging Large Language Models to Reveal Interpretable Cooling Behaviors from Smart Thermostat Data

Frequent heatwaves and hot summers increasingly challenge occupant comfort, health, and energy grid stability. Addressing these challenges requires a detailed understanding of household cooling behaviors, such as thermostat adjustments and adaptive responses to extreme conditions. Traditional analyses often rely on aggregated numerical metrics that overlook subtle but important household-specific variations. In this study, we introduce a generalizable methodology that integrates large language models (LLMs) with vision capabilities to enable scalable and detailed analysis of residential thermostat data. Using Ecobee's Donate Your Data (DYD) dataset—which provides five-minute records of indoor temperatures, thermostat setpoints, and HVAC runtimes—we focus on two U.S. cities with contrasting summer climates : Austin (TX) and Phoenix (AZ). Because raw time-series data are not well suited for direct LLM analysis, we transform them into visual representations, such as daily indoor temperature trajectories and weekly runtime histograms, to better capture behavioral variations. Leveraging LLMs' visual interpretation, we extract descriptive behavioral features, including temperature preferences, time-of-day cooling orientation, anticipatory versus reactive heatwave responses, and behavioral consistency. These semantic features support unsupervised clustering to identify distinct occupant archetypes at scale, revealing differences—such as morning-centric anticipatory coolers versus households that shift toward warmer setpoints during heatwaves—that can inform demand response, resilience planning, and health-aware interventions. By converting raw numerical data into interpretable behavioral patterns, this methodology enables scalable and practical analysis of occupant behavior, supporting actionable insights for comfort, resilience, and energy management.

Nihar, Kopal↗

DOE OSTI2025

Source Analysis of Ozone Pollution in Liaoyuan City’s Atmosphere Based on Machine Learning Models and HYSPLIT Clustering Method

Firstly, this study investigates the spatiotemporal distribution characteristics of the ozone (O 3 ) pollution in Liaoyuan City using monitoring data from 2015 to 2024. Then, three machine learning models (ML)—random forest (RF), support vector machine (SVM), and artificial neural network (ANN)—are employed to quantify the influence of meteorological and non-meteorological factors on O 3 concentrations. Finally, the HYSPLIT clustering method and CMAQ model are utilized to analyze inter-regional transport characteristics, identifying the causes of O 3 pollution. The results indicate that O 3 pollution in Liaoyuan exhibits a distinct seasonal pattern, with the highest concentrations found in spring and summer, peaking in the afternoon. Among the three ML models, the random forest model demonstrates the best predictive performance (R 2 = 0.9043). Feature importance identifies NO 2 as the primary driving factor, followed by meteorological conditions in the second quarter and land surface characteristics. Furthermore, regional transport significantly contributes to O 3 pollution, with approximately 80% of air mass trajectories in heavily polluted episodes originating from adjacent industrial areas and the sea. The combined effects of transboundary precursors and O 3 transport with local emissions and meteorological conditions further increase the O 3 pollution level. This study highlights the need to strengthen coordinated NO X and VOCs emission reductions and enhance regional joint prevention and control strategies in China.

HYSPLIT clustering↗

DOE OSTI2020

Data Analytics Applied to Coal Fired Boilers for Detecting Leaks

Data analytics were used to detect boiler leaks from five different coal-fired boilers including both subcritical and supercritical systems. Discriminant functions were developed that detected leaks up to two weeks prior to forced plant shutdowns for repairs. The leaks were identified to occur at different sections of the boiler for each plant, including waterwalls, economizer and superheater using conventional process measurement data. Leaking conditions were detected with a high degree of confidence (&NestedLessLess; 1% misclassified observations) and were able to distinguish normal operations from those time periods with steam leaks even while operating the power plants in power cycling mode.Multivariable statistical analyses, including Principal Component (PCA), cluster, and Fischer Discriminant Analysis (FDA) were used to characterize the leak occurrence. Normal and operational states with steam leaks were provided in the original process datasets. These datasets were split into two different groups for training and validation purposes. The data were sorted chronologically, and every third observation was assigned to training the Discriminant Function Model (DFM) while the rest were reserved for validation. PCA was used to reduce dimensionality of the original datasets. Canonical and FDA analyses were used to investigate the relationship between process variables. The outcome of the analyses revealed that nearly 35,000 observations were classified correctly; less than 0.05% of total observations were misclassified to be leaking, i.e. both false positives and false negatives.

Indrawan, Natarianto↗

DOE OSTI2021

Stability of Cool Cores during Galaxy Cluster Growth: A Joint Chandra/SPT Analysis of 67 Galaxy Clusters along a Common Evolutionary Track Spanning 9 Gyr

We present the results of a joint analysis of Chandra X-ray and South Pole Telescope (SPT) Sunyaev–Zel’dovich (SZ) observations targeting the first sample of galaxy clusters at 0.3 < z < 1.3, selected to be the progenitors of well-studied nearby clusters based on their expected accretion rate. We develop a new procedure in order to tackle the analysis challenge that is estimating the intracluster medium (ICM) properties of low-mass and high-redshift clusters with ~150 X-ray counts. One of the dominant sources of uncertainty on the ICM density profile estimated with a standard X-ray analysis with such shallow X-ray data is due to the systematic uncertainty associated with the ICM temperature obtained through the analysis of the background-dominated X-ray spectrum. We show that we can decrease the uncertainty on the density profile by a factor varying between 2 and 8 with a joint deprojection of the X-ray surface brightness profile measured by Chandra and the SZ-integrated Compton parameter available in the SPT cluster catalog. We apply this technique to the whole sample of 67 clusters in order to track the evolution of the ICM core density during cluster growth. We confirm that the evolution of the gas density profile is well modeled by the combination of a fixed core and a self-similarly evolving non-cool-core profile. We show that the fraction of cool cores in this sample is remarkably stable with redshift although clusters have gained a factor of ~4 in total mass over the past ~9 Gyr.

79 ASTRONOMY AND ASTROPHYSICS↗