Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data analysis cluster”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 253 records · Page 14
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

The importance of accounting for landscape position when investigating grasslands: A multidisciplinary characterisation of a Californian coastal grassland

Data from the characterisation of the Point Reyes Field Site, published in AGU Earth's Future under the title: The importance of accounting for landscape position when investigating grasslands: A multidisciplinary characterisation of a Californian coastal grassland. This paper explored the effect of landscape position on the response of a Californian grassland to seasonal changes. All files are csv files. The EMI data contains 8 csv files with a metadata csv explaining the columns. The dataset also includes soil variables including total concentrations calculated from fused samples, then dissolved and measured on ICP-AES for whole-rock elements and ICP-MS for trace elements. Mineral composition was attained using X-ray diffraction at BL 11-3 at SSRL . Data was then run through the High Score database to characterise different mineral phases. total It also includes a table with bulk soil characteristics such as soil pH, cation exchange capacity, and soil textural data. Data from Teros 12 Meter soil moisture, electrical conductivity and temperature sensors are presented in SMS Csv file. While the WL bottom and top files contain data from Piezometers measuring the ground water table. We have included a csv file that contains soil CO2 efflux data from Feb 2021-Oct 2021 in the Point Reyes Grassland Experiment We have included the spatially orientated (easting northing) remotely sensed datasets that were used in the K-means clustering analysis conducted on our site with electrical conductivity, normalised difference vegetation index, elevation, slope, solar radiation, topographic position and wetness index, and a clustering score. Finally there is a list of all the identified grassland species at the site.For more information on flux data, please contact the corresponding author.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Studying the hadron structure with PANDA and CLAS using machine learning techniques

The hadron spectroscopy and structure are currently very active fields of research to study the non-perturbative regime of quantum chronodynamics. The first one studies the complex structure of excited hadrons by looking at their decay products, while the latter uses lepton scattering on nucleons. Both methods require reconstruction algorithms with great efficiency and good particle identification and background rejection rates. This work aims to provide these by either improving the existing methods or developing new ones. The first part of this document presents a feasibility study of a predicted hybrid charmonium state for the PANDA experiment. Lattice QCD calculations predict the ground state hybrid charmonium to be a spin exotic with quantum numbers of JP C = 1?+ at a mass of around 4.3 GeV with a width to be around 20 MeV. A machine learning based data analysis scheme is proposed to further improve the signal efficiency and the background reduction, alongside with improvements of the analysis software (PandaRoot), that are vital for this study. These improvements include a reworked clustering algorithm for the electromagnetic calorimeter (EMC) and an optimized monte carlo matching for neutral particles. The second part of this document is about studying the proton structure. A multidimensional study of the structure function ratio Fsin(?)LU /FUU has been performed for K±, based on the measurement of beam-spin asymmetries. It uses the high statistics data recorded with the CLAS12 spectrometer at Jefferson Laboratory. Fsin(?)LU is a twist-3 quantity that provides information about the quark gluon correlations in the proton. This document will present for the first time a simultaneous analysis of two kaon channels over a large kinematic range of z, xB , PT and Q2 with virtualities Q2 ranging from 1 GeV2 up to 8 GeV2 using machine learning techniques for improved particle identification.

Kripko, Aron↗
DOE OSTI2020

An Improved Boosting to Amplify Signal with Isobaric Labeling (iBASIL) Strategy for Precise Quantitative Single-cell Proteomics

Mass spectrometry (MS)-based proteomics has great potential for overcoming the limitations of antibody-based immunoassays for antibody-independent, comprehensive, and quantitative proteomic analysis of single cells. Indeed, recent advances in nanoscale sample preparation have enabled effective processing of single cells. In particular, the concept of using boosting/carrier channels in isobaric labeling to increase the sensitivity in MS detection has also been increasingly used for quantitative proteomic analysis of small-sized samples including single cells. However, the full potential of such boosting/carrier approaches has not been significantly explored, nor has the resulting quantitation quality been carefully evaluated. Herein, we have further evaluated and optimized our recent boosting to amplify signal with isobaric labeling (BASIL) approach, originally developed for quantifying phosphorylation in small number of cells, for highly effective analysis of proteins in single cells. This improved BASIL (iBASIL) approach enables reliable quantitative single-cell proteomics analysis with greater proteome coverage by carefully controlling the boosting-to-sample ratio (e.g. in general <100×) and optimizing MS automatic gain control (AGC) and ion injection time settings in MS/MS analysis (e.g. 5E5 and 300 ms, respectively, which is significantly higher than that used in typical bulk analysis). By coupling with a nanodroplet-based single cell preparation (nanoPOTS) platform, iBASIL enabled identification of ~2500 proteins and precise quantification of ~1500 proteins in the analysis of 104 FACS-isolated single cells, with the resulting protein profiles robustly clustering the cells from three different acute myeloid leukemia cell lines. This study highlights the importance of carefully evaluating and optimizing the boosting ratios and MS data acquisition conditions for achieving robust, comprehensive proteomic analysis of single cells.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2023

The cosmic web around the Coma cluster from constrained cosmological simulations

Galaxy clusters in the Universe occupy the important position of nodes of the cosmic web. They are connected among them by filaments, elongated structures composed of dark matter, galaxies, and gas. The connection of galaxy clusters to filaments is important, as it is related to the process of matter accretion onto the former. For this reason, investigating the connections to the cosmic web of massive clusters, especially well-known ones for which a lot of information is available, is a hot topic in astrophysics. In a previous work, we performed an analysis of the filament connections of the Coma cluster of galaxies, as detected from the observed galaxy distribution. In this work we resort to a numerical simulation whose initial conditions are constrained to reproduce the local Universe, including the region of the Coma cluster to interpret our observations in an evolutionary context. We detect the filaments connected to the simulated Coma cluster and perform an accurate comparison with the cosmic web configuration we detect in observations. We perform an analysis of the halos’ spatial and velocity distributions close to the filaments in the cluster outskirts. We conclude that, although not significantly larger than the average, the flux of accreting matter on the simulated Coma cluster is significantly more collimated close to the filaments with respect to the general isotropic accretion flux. This paper is the first example of such a result and the first installment in a series of publications which will explore the build-up of the Coma cluster system in connection to the filaments of the cosmic web as a function of redshift.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2022

Two-proton decay from α-cluster states in 10 C and 11 N

Two-proton decay from excited states in 10 C and 11 N has been characterized from an analysis of the 2p + 2α and 3p + 2α exit channels. Data from four previously published invariant-mass studies associated with inelastic excitation, multinucleon knockout, and neutron pickup reactions have been considered. A rotational band in 10 C was identified built on the second 0 + state which has strong α-cluster molecular structure. The members of this band were tentatively identified up to J π = 4 + and all identified states have a prompt 2p decay branch. The spins of the 0 + and 2 + members were deduced from comparisons of the momentum correlations in their 2p branches to those from known 2p emitters. The 2 + and the tentative 4 + members are both above the α threshold and exhibit large reduced decay widths for α emission. This band exhibits a similar moment of inertia to its analog in the mirror nucleus 10 Be. Evidence is also presented for a molecular band in 11 N built on the second 3/2 – state which 2p decays. Furthermore, tentative members of this band have been assigned up to J π = 9/2 – and their excitation energies also match their likely analogs in the mirror nucleus 11 Be.

6 ≤ A ≤ 19↗
DOE OSTI2025

DESI 2024 V: Full-Shape galaxy clustering from galaxies and quasars

We present the measurements and cosmological implications of the galaxy two-point clustering using over 4.7 million unique galaxy and quasar redshifts in the range 0.1 < z < 2.1 divided into six redshift bins over a ∼ 7,500 square degree footprint, from the first year of observations with the Dark Energy Spectroscopic Instrument (DESI Data Release 1). By fitting the full power spectrum, we extend previous DESI DR1 baryon acoustic oscillation (BAO) measurements to include redshift-space distortions and signals from the matter-radiation equality scale. For the first time, this Full-Shape analysis is blinded at the catalogue-level to avoid confirmation bias and the systematic errors are accounted for at the two-point clustering level, which automatically propagates them into any cosmological parameter. When analyzing the data in terms of compressed model-agnostic variables, we obtain a combined precision of 4.7% on the amplitude of the redshift space distortion (RSD) signal reaching a similar precision with just one year of DESI data than with twenty years of observation from the previous generation survey. We also analyze the data to directly constrain the cosmological parameters within the ΛCDM model using perturbation theory and combine this information with the reconstructed DESI DR1 galaxy BAO. Using a Big Bang Nucleosynthesis Gaussian prior on the baryon density parameter, ω b , and a weak Gaussian prior on the spectral index, n s , we constrain the matter density is Ω m = 0.296±0.010 and the Hubble constant H 0 = (68.63 ± 0.79)[km s -1 Mpc -1 ]. Additionally, we measure the amplitude of clustering σ 8 = 0.841±0.034. The DESI DR1 galaxy clustering results are in agreement with the ΛCDM model based on general relativity with parameters consistent with those from Planck. The cosmological interpretation of these results in combination with DESI DR1 Ly-α forest data and external datasets are presented in the companion paper [1].

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2024

MVP: a modular viromics pipeline to identify, filter, cluster, annotate, and bin viruses from metagenomes

While numerous computational frameworks and workflows are available for recovering prokaryote and eukaryote genomes from metagenome data, only a limited number of pipelines are designed specifically for viromics analysis. With many viromics tools developed in the last few years alone, it can be challenging for scientists with limited bioinformatics experience to easily recover, evaluate quality, annotate genes, dereplicate, assign taxonomy, and calculate relative abundance and coverage of viral genomes using state-of-the-art methods and standards. Here, we describe Modular Viromics Pipeline (MVP) v.1.0, a user-friendly pipeline written in Python and providing a simple framework to perform standard viromics analyses. MVP combines multiple tools to enable viral genome identification, characterization of genome quality, filtering, clustering, taxonomic and functional annotation, genome binning, and comprehensive summaries of results that can be used for downstream ecological analyses. Overall, MVP provides a standardized and reproducible pipeline for both extensive and robust characterization of viruses from large-scale sequencing data including metagenomes, metatranscriptomes, viromes, and isolate genomes. As a typical use case, we show how the entire MVP pipeline can be applied to a set of 20 metagenomes from wetland sediments using only 10 modules executed via command lines, leading to the identification of 11,656 viral contigs and 8,145 viral operational taxonomic units (vOTUs) displaying a clear beta-diversity pattern. Further, acting as a dynamic wrapper, MVP is designed to continuously incorporate updates and integrate new tools, ensuring its ongoing relevance in the rapidly evolving field of viromics. MVP is available at https://gitlab.com/ccoclet/mvp and as versioned packages in PyPi and Conda.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2025

Utah FORGE: Seismic DAS and Geophone Borehole Data Processing and 3D Imaging of Vp/Vs Ratio in the 2024 Stimulated Reservoir

This dataset includes a final report and a 3D velocity model derived from seismic DAS and geophone borehole data collected during the April 2024 stimulation of the reservoir at Utah FORGE. The report details the processing of over 50,000 P- and S-wave travel times used in a tomographic inversion to estimate the Vp/Vs ratio, revealing anomalies adjacent to wells 16A and 16B that may be associated with injected fracturing fluids. Additionally, Wadati analysis of more than 27,000 pairs of differential P- and S-wave travel times supports this interpretation with a high Vp/Vs estimate of 1.86, averaged over the dimensions of the earthquake cluster. The accompanying data file provides a 3D model of P- and S-wave velocities and Vp/Vs ratios, structured with spatial coordinates and velocity values.

15 GEOTHERMAL ENERGY↗
DOE OSTI2022

Gaussian mixture model clustering algorithms for the analysis of high-precision mass measurements

The development of the phase-imaging ion-cyclotron resonance (PI-ICR) technique for use in Penning trap mass spectrometry (PTMS) increased the speed and precision with which PTMS experiments can be carried out. In PI-ICR, data sets of the locations of individual ion hits on a detector are created showing how ions cluster together into spots according to their cyclotron frequency. Ideal data sets would consist of a single, 2D-spherical spot with no other noise, but in practice data sets typically contain multiple spots, non-spherical spots, or significant noise, all of which can make determining the locations of spot centers non-trivial. A method for assigning groups of ions to their respective spots and determining the spot centers is therefore essential for further improving precision and confidence in PI-ICR experiments. Here, we present the class of Gaussian mixture model (GMM) clustering algorithms as an optimal solution. We show that on simulated PI-ICR data, several types of GMM clustering algorithms perform better than other clustering algorithms over a variety of typical scenarios encountered in PI-ICR. The mass spectra of 163Gd, 163 mGd, 162 Tb, and 162 mTb measured using PI-ICR at the Canadian Penning trap mass spectrometer were checked using GMMs, producing results that were in close agreement with the previously published values.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2024

Remote Sensing and GIS data at 1km-grid over Chesapeake Bay used in “He et al. 2024, Effects of spatial variability in vegetation phenology, climate, landcover, biodiversity, topography, and soil property on soil respiration across a coastal ecosystem”

The package contains the data layers used in “He et al. 2024, Effects of spatial variability in vegetation phenology, climate, landcover, biodiversity, topography, and soil property on soil respiration across a coastal ecosystem”. The study aims to use multi-source remote sensing and GIS datasets to investigate the spatial heterogeneity and identify spatial zones with similar environmental characteristics and understand the primary driving factors affecting soil respiration within sub-ecosystems of the coastal ecosystem. We employed unsupervised hierarchical clustering analysis to identify spatial regions with distinct environmental characteristics, then determined the main driving factors using Random Forest regression and SHapley Additive exPlanations (SHAP). Spatial data layers include soil respiration, kernel Normalized Difference Vegetation Index (kNDVI) computed from Harmonized Landsat 8 and Sentinel-2 time series, climate variables from the Daymet dataset, land cover, biodiversity, topographical metrics, soil property, and tidal elevation.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2020

Integrated Proteomic and Glycoproteomic Characterization of Human High-Grade Serous Ovarian Carcinoma

Many gene products exhibit great structural heterogeneity because of an array of modifications. These modifications are not directly encoded in the genomic template but often affect the functionality of proteins. Protein glycosylation plays a vital role in proper protein functions. However, the analysis of glycoproteins has been challenging compared with other protein modifications, such as phosphorylation. Here, we perform an integrated proteomic and glycoproteomic analysis of 83 prospectively collected high-grade serous ovarian carcinoma (HGSC) and 23 non-tumor tissues. Integration of the expression data from global proteomics and glycoproteomics reveals tumor-specific glycosylation, uncovers different glycosylation associated with three tumor clusters, and identifies glycosylation enzymes that were correlated with the altered glycosylation. In addition to providing a valuable resource, these results provide insights into the potentialroles of glycosylation in the pathogenesis of HGSC, with the possibility of distinguishing pathological outcomes of ovarian tumors from non-tumors, as well as classifying tumor clusters.

Hu, Yingwei↗
DOE OSTI2024

Functional analysis of the methylerythritol phosphate pathway terminal enzymes IspG and IspH from Zymomonas mobilis

ABSTRACT Isoprenoids are a diverse family of compounds that are synthesized from two isomeric compounds, isopentenyl diphosphate and dimethylallyl diphosphate. In most bacteria, isoprenoids are produced from the essential methylerythritol phosphate (MEP) pathway. The terminal enzymes of the MEP pathway IspG and IspH are [4Fe-4S] cluster proteins, and in Zymomonas mobilis, the substrates of IspG and IspH accumulate in cells in response to O 2 , suggesting possible lability of their [4Fe-4S] clusters. Here, we show using complementation assays in Escherichia coli that even under anaerobic conditions, Z. mobilis IspG and IspH are not as functional as their E. coli counterparts, requiring higher levels of expression to rescue viability. A deficit of the sulfur utilization factor (SUF) Fe-S cluster biogenesis pathway did not explain the reduced function of Z. mobilis IspG and IspH since no improvement in viability was observed in E. coli expressing the Z. mobilis SUF pathway or having increased expression of the E. coli SUF pathway. Complementation of single and double mutants with various combinations of Z. mobilis and E. coli IspG and IspH indicated that optimal growth required the pairing of IspG and IspH from the same species. Furthermore, Z. mobilis IspH conferred an O 2 -sensitive growth defect to E. coli that could be partially rescued by co-expression of Z. mobilis IspG. In vitro analysis showed O 2 sensitivity of the [4Fe-4S] cluster of both Z. mobilis IspG and IspH. Altogether, our data indicate an important role of the cognate protein IspG in Z. mobilis IspH function under both aerobic and anaerobic conditions. IMPORTANCE Isoprenoids are one of the largest classes of natural products, exhibiting diversity in structure and function. They also include compounds that are essential for cellular life across the biological world. In bacteria, isoprenoids are derived from two precursors, isopentenyl diphosphate and dimethylallyl diphosphate, synthesized primarily by the methylerythritol phosphate pathway. The aerotolerant Z. mobilis has the potential for methylerythritol phosphate pathway engineering by diverting some of the glucose that is typically efficiently converted into ethanol to produce isoprenoid precursors to make bioproducts and biofuels. Our data revealed the surprising finding that Z. mobilis IspG and IspH need to be co-optimized to improve flux via the methyl erythritol phosphate pathway in part to evade the oxygen sensitivity of IspH.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2022

The Observed Evolution of the Stellar Mass–Halo Mass Relation for Brightest Central Galaxies

We quantify evolution in the cluster-scale stellar mass–halo mass (SMHM) relation's parameters using 2323 clusters and brightest central galaxies (BCGs) over the redshift range 0.03 ≤ z ≤ 0.60. The precision on the inferred SMHM parameters is improved by including the magnitude gap (m gap ) between the BCG and fourth-brightest cluster member (M14) as a third parameter in the SMHM relation. At fixed halo mass, accounting for m gap , through a stretch parameter, reduces the SMHM relation's intrinsic scatter. To explore this redshift range, we use clusters, BCGs, and cluster members identified using the Sloan Digital Sky Survey C4 and redMaPPer cluster catalogs and the Dark Energy Survey redMaPPer catalog. Through this joint analysis, we detect no systematic differences in BCG stellar mass, m gap , and cluster mass (inferred from richness) between the data sets. We utilize the Pareto function to quantify each parameter's evolution. We confirm prior findings of negative evolution in the SMHM relation's slope (3.5σ), and detect negative evolution in the stretch parameter (4.0σ) and positive evolution in the offset parameter (5.8σ). This observed evolution, combined with the absence of BCG growth, when stellar mass is measured within 50 kpc, suggests that this evolution results from changes in the cluster's m gap . For this to occur, late-term growth must be in the intracluster light surrounding the BCG. We also compare the observed results to IllustrisTNG 300-1 cosmological hydrodynamic simulations and find modest qualitative agreement. However, the simulations lack the evolutionary features detected in the real data.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2025

ROOT RNTuple and EOS: The Next Generation of Event Data I/O

For several years, the ROOT team is developing the new RNTuple I/O subsystem in preparation of the next generation of collider experiments. Both HL-LHC and DUNE are expected to start data taking by the end of this decade. They pose unprecedented challenges to event data I/O in terms of data rates, event sizes, and event complexity. At the same time, the I/O landscape is becoming more diverse. HPC cluster file systems and object stores, NVMe disk cache layers in analysis facilities, and S3 storage on cloud resources are mixing with traditional XRootD-managed spinning disk pools.The ROOT team will finalize a first production version of the RNTuple binary format by the end of 2024. After this point, ROOT will provide backward compatibility for RNTuple data. This contribution provides an overview of the RNTuple feature set, the related R&D activities and the long-term vision for RNTuple. We report on performance, interface design, tooling, robustness, integration with experiment frameworks, and validation results, as well as recent R&D on parallel reading and writing and exploitation of modern hardware and storage systems. We will give an outlook on possible future features after a first production release.Collaboratively, the IT and EP departments at CERN have launched a formal project within the Research and Computing sector to evaluate the novel data format for physics analysis data utilized in LHC experiments and other fields. This part of the project focuses on validating the scalability of the EOS storage backend during the transition from the over 25 years old TTree production format to the newly developed RNTuple format, using both replicated and erasure-coded storage profiles.

Blomer, Jakob [CERN]↗
DOE OSTI2020

Network Modeling of Complex Data Sets

We demonstrate a selection of network and machine learning techniques useful in the analysis of complex datasets, including 2-way similarity networks, Markov clustering, enrichment statistical networks, FCROS differential analysis, and random forests. We demonstrate each of these techniques on the Populus trichocarpa gene expression atlas.

Jones, Piet C.↗
DOE OSTI2022

Discovering hidden geothermal signatures using non-negative matrix factorization with customized k-means clustering

Discovery of hidden geothermal resources is challenging. It requires the mining of large datasets with diverse data attributes representing subsurface hydrogeological and geothermal conditions. The commonly used play fairway analysis approach typically incorporates subject-matter expertise to analyze regional data to estimate geothermal characteristics and favorability. We demonstrate an alternative approach based on machine learning (ML) to process a geothermal dataset from southwest New Mexico (SWNM). The study region includes low- and medium-temperature hydrothermal systems. Several of these systems are not well characterized because of insufficient existing data and limited past explorative work. This study discovers hidden patterns and relations in the SWNM geothermal dataset to improve our understanding of the regional hydrothermal conditions and energy-production favorability. This understanding is obtained by applying an unsupervised ML algorithm based on non-negative matrix factorization coupled with customized k-means clustering (NMFk). NMFk can automatically identify (1) hidden signatures characterizing analyzed datasets, (2) the optimal number of these signatures, (3) the dominant data attributes associated with each signature, and (4) the spatial distribution of the extracted signatures. Here, in this study, NMFk is applied to analyze 18 geological, geophysical, hydrogeological, and geothermal attributes at 44 locations in SWNM. Using NMFk, we find data patterns and identify the spatial associations of hydrothermal signatures within two physiographic provinces (Colorado Plateau and Basin and Range) and two sub-regions of these provinces (the Mogollon-Datil volcanic field and the Rio Grande rift) in SWNM. The ML algorithm extracted five hydrothermal signatures in the SWNM datasets that differentiate between low (<90°C) and medium (90-150°C)-temperature hydrothermal systems. The algorithm also suggests that the Rio Grande rift and northern Mogollon-Datil volcanic field are the most favorable regions for future geothermal resource discovery. NMFk also identified critical attributes to identify medium-temperature hydrothermal systems in the study area. The resulting NMFk model can be applied to predict geothermal conditions and their uncertainties at new SWNM locations based on limited data from unexplored regions. The code to execute the performed analyses as well as the corresponding data can be found at https://github.com/SmartTensors/GeoThermalCloud.jl.

15 GEOTHERMAL ENERGY↗
DOE OSTI2026

Dark Energy Survey Year 6 results: Clustering redshifts and importance sampling of self-organized-maps 𝑛⁡(𝑧) realizations for 3 × 2 ⁢pt samples

This work is part of a series establishing the redshift framework for the 3 × 2 ⁢pt analysis of the Dark Energy Survey Year 6 (DES Y6). For DES Y6, photometric redshift distributions are estimated using self-organizing maps (SOMs), calibrated with spectroscopic and many-band photometric data. To overcome limitations from color-redshift degeneracies and incomplete spectroscopic coverage, we enhance this approach by incorporating clustering-based redshift constraints (clustering-z, or WZ) from angular cross-correlations with BOSS and eBOSS galaxies and eBOSS quasar samples. We define a WZ likelihood and apply importance sampling to a large ensemble of SOM-derived 𝑛⁡(𝑧) realizations, selecting those consistent with the clustering measurements to produce a posterior sample for each lens and source bin. The analysis uses angular scales corresponding to 1.5–5 Mpc to optimize signal-to-noise ratio while mitigating modeling uncertainties and marginalizes over redshift-dependent galaxy bias and other systematics informed by the N-body simulation CARDINAL . While a sparser spectroscopic reference sample limits WZ constraining power at 𝑧 >1.1, particularly for source bins, we demonstrate that combining SOM with WZ improves redshift accuracy and enhances the overall cosmological constraining power of DES Y6. As a result, we estimate an improvement in 𝑆 8 of approximately 10% for cosmic shear and 3 ×2⁢pt analysis, primarily due to the WZ calibration of the source samples.

Cosmological parameters↗
DOE OSTI2024

Power Profile Monitoring and Tracking Evolution of System-Wide HPC Workloads

The power & energy demands of HPC machines have grown significantly. Modern exascale HPC systems require tens of megawatts of combined power for computing resources and cooling facilities at full capacity. The current energy trend is not sustainable for future HPC systems, and there is a need to work toward the energy efficiency aspect of HPC performance. Energy awareness of the HPC applications at the job level is essential for running an efficient HPC system. This work aims to develop a pipeline to provide a production-level system-wide overview of the HPC workloads' power profile while handling evolving workloads exhibiting new power trends. We developed an open-set classification model for HPC jobs based on the properties of power profiles to continuously provide a system-wide holistic view of recently completed jobs. The pipeline helps continuously monitor the job-level power usage pattern of HPC and enables us to capture the new trends in applications' power behavior. We employed a comprehensive set of techniques to generate job-level data, custom-designed feature extraction methods to extract critical features from jobs' power profiles, clustering techniques powered by generative modeling, and open-set classification for identifying job profiles into known classes or an unknown set. With extensive evaluations, we demonstrate the effectiveness of each component in our pipeline. We provide an analysis of the resulting clusters that characterize the power profile landscape of the Summit supercomputer from more than 60K jobs executed in a year. The open-set classification classifies the known data sets into known classes with high accuracy and identifies unknown data noints with over 85% accuracy.

Karimi, Ahmad Maroof↗