Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data analysis cluster”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 523 records · Page 29
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Galaxy–galaxy lensing with the DES-CMASS catalogue: measurement and constraints on the galaxy-matter cross-correlation

ABSTRACT The DMASS sample is a photometric sample from the DES Year 1 data set designed to replicate the properties of the CMASS sample from BOSS, in support of a joint analysis of DES and BOSS beyond the small overlapping area. In this paper, we present the measurement of galaxy–galaxy lensing using the DMASS sample as gravitational lenses in the DES Y1 imaging data. We test a number of potential systematics that can bias the galaxy–galaxy lensing signal, including those from shear estimation, photometric redshifts, and observing conditions. After careful systematic tests, we obtain a highly significant detection of the galaxy–galaxy lensing signal, with total S/N = 25.7. With the measured signal, we assess the feasibility of using DMASS as gravitational lenses equivalent to CMASS, by estimating the galaxy-matter cross-correlation coefficient rcc. By jointly fitting the galaxy–galaxy lensing measurement with the galaxy clustering measurement from CMASS, we obtain $r_{\rm cc}=1.09^{+0.12}_{-0.11}$ for the scale cut of $4 \, h^{-1}{\rm \,\,Mpc}$ and $r_{\rm cc}=1.06^{+0.13}_{-0.12}$ for $12 \, h^{-1}{\rm \,\,Mpc}$ in fixed cosmology. By adding the angular galaxy clustering of DMASS, we obtain rcc = 1.06 ± 0.10 for the scale cut of $4 \, h^{-1}{\rm \,\,Mpc}$ and rcc = 1.03 ± 0.11 for $12 \, h^{-1}{\rm \,\,Mpc}$. The resulting values of rcc indicate that the lensing signal of DMASS is statistically consistent with the one that would have been measured if CMASS had populated the DES region within the given statistical uncertainty. The measurement of galaxy–galaxy lensing presented in this paper will serve as part of the data vector for the forthcoming cosmology analysis in preparation.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2020

The completed SDSS-IV extended Baryon Oscillation Spectroscopic Survey: BAO and RSD measurements from the anisotropic power spectrum of the quasar sample between redshift 0.8 and 2.2

ABSTRACT We measure the clustering of quasars of the final data release (DR16) of eBOSS. The sample contains $343\, 708$ quasars between redshifts 0.8 ≤ z ≤ 2.2 over $4699\, \mathrm{deg}^2$. We calculate the Legendre multipoles (0,2,4) of the anisotropic power spectrum and perform a BAO and a Full-Shape (FS) analysis at the effective redshift zeff = 1.480. The errors include systematic errors that amount to 1/3 of the statistical error. The systematic errors comprise a modelling part studied using a blind N-body mock challenge and observational effects studied with approximate mocks to account for various types of redshift smearing and fibre collisions. For the BAO analysis, we measure the transverse comoving distance DM(zeff)/rdrag = 30.60 ± 0.90 and the Hubble distance DH(zeff)/rdrag = 13.34 ± 0.60. This agrees with the configuration space analysis, and the consensus yields: DM(zeff)/rdrag = 30.69 ± 0.80 and DH(zeff)/rdrag = 13.26 ± 0.55. In the FS analysis, we fit the power spectrum using a model based on Regularised Perturbation Theory, which includes redshift space distortions and the Alcock–Paczynski effect. The results are DM(zeff)/rdrag = 30.68 ± 0.90 and DH(zeff)/rdrag = 13.52 ± 0.51 and we constrain the linear growth rate of structure f(zeff)σ8(zeff) = 0.476 ± 0.047. Our results agree with the configuration space analysis. The consensus analysis of the eBOSS quasar sample yields: DM(zeff)/rdrag = 30.21 ± 0.79, DH(zeff)/rdrag = 3.23 ± 0.47, and f(zeff)σ8(zeff) = 0.462 ± 0.045 and is consistent with a flat ΛCDM cosmological model using Planck results.

Neveux, Richard↗
DOE OSTI2023

GPU acceleration of Swendsen–Wang dynamics

When simulating a lattice system near its critical temperature, local algorithms for modeling the system’s evolution can introduce very large autocorrelation times into sampled data. Here, this critical slowing down places restrictions on the analysis that can be completed in a timely manner of the behavior of systems around the critical point. Because it is often desirable to study such systems around this point, a new algorithm must be introduced. Therefore, we turn to cluster algorithms, such as the Swendsen–Wang algorithm and the Wolff clustering algorithm. They incorporate global updates which generate new lattice configurations with little correlation to previous states, even near the critical point. We look to accelerate the rate at which these algorithm are capable of running by implementing and benchmarking a parallel implementation of each algorithm designed to run on GPUs under NVIDIA’s CUDA framework. A 17 and 90 fold increase in the computational rate was, respectively, experienced when measured against the equivalent algorithm implemented in serial code.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2021

Scalar Field Comparison with Topological Descriptors: Properties and Applications for Scientific Visualization

In topological data analysis and visualization, topological descriptors such as persistence diagrams, merge trees, contour trees, Reeb graphs, and Morse–Smale complexes play an essential role in capturing the shape of scalar field data. Herein we present a state–of–the–art report on scalar field comparison using topological descriptors. We provide a taxonomy of existing approaches based on visualization tasks associated with three categories of data: single fields, time–varying fields, and ensembles. These tasks include symmetry detection, periodicity detection, key event/feature detection, feature tracking, clustering, and structure statistics. Our main contributions include the formulation of a set of desirable mathematical and computational properties of comparative measures, and the classification of visualization tasks and applications that are enabled by these measures.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Dark Energy Survey Year 3 results: optimized $w$CDM simulation-based inference with weak lensing map-level hybrid statistics

We present cosmological constraints from the Dark Energy Survey Year 3 (DES Y3) weak lensing data using hierarchical hybrid statistics within a Bayesian simulation-based inference framework that is based on the Gower Street simulations. To maximize the precision of the inference, we have developed a new, information-theory based, data compression of the weak lensing maps to just seven highly informative summary statistics. The hybrid scheme exploits the high information content of the power spectrum, compressing both the power spectrum and neural-based summaries that are designed to extract further information. Our simulation-based approach enables principled forward modelling of all major sources of systematic uncertainty and survey properties into realistic mock observations, including the survey mask, photometric redshift uncertainties, intrinsic galaxy alignments, multiplicative shear calibration bias, source galaxy clustering, non-Gaussian shape noise, and non-linear structure formation. The summary statistics are then used in a Bayesian simulation-based inference pipeline. The inference is validated through coverage tests and checks for robustness against baryonic feedback. Assuming a $w$CDM cosmology, our analysis yields $S_8 = 0.808 \pm 0.017$, $Ω_{\rm m} = 0.325 \pm 0.024$, and $w < -0.766$ (marginalized posterior 68 per cent credible intervals). This rigorous combination of information theory, physics- and neural network-based extreme data compression, and principled Bayesian analysis improves the figure of merit for $(Ω_{\rm m}, S_8, w)$ by 60 per cent over the previous state-of-the-art, and by almost a factor of 3 over two-point analyses of the same data. They are the most precise joint constraints on $(Ω_{\rm m}, S_8, w)$ from weak gravitational lensing data alone of any survey to date. We intend to apply this analysis to the more recent DES Y6 data.

Williamson, J. [University Coll. London]↗
DOE OSTI2023

Conquering Data Chaos: Research Data Management with Kubernetes

Managing massive volumes of data and effectively making it accessible to researchers poses significant challenges and is a barrier to scientific discovery. In many cases, critical data is locked up in unwieldy file formats or one-off databases and is too large to effectively process on a single machine. This talk explores the role of Kubernetes, an open-source container orchestration platform, in addressing research data management challenges. I will discuss how we are using a set of publicly available open-source and home-grown tools in the National Renewable Energy Lab (NREL) Data, Analysis, and Visualization (DAV) group to help researchers overcome data-related bottlenecks. The talk will begin by providing an overview of the data challenges faced in research data management, including data storage, processing, and analysis. I will highlight Kubernetes' ability to handle large-scale data by leveraging containerization and distributed computing, including distributed storage. Kubernetes allows researchers to encapsulate data processing infrastructure and workflows into portable containers, enabling reproducibility and ease of deployment. Kubernetes can then schedule and manage the resource allocation of these containers to enable efficient utilization of limited computing resources, leading to more efficient data processing and analysis. I will discuss some limitations of traditional, siloed approaches to dealing with data and emphasize the need for solutions which foster collaboration. I will highlight how we are using Kubernetes at NREL to facilitate data sharing and cooperation among research teams. Kubernetes' flexible architecture enables the deployment of shared computing environments, such as Apache Superset, where researchers can seamlessly access and analyze shared datasets. Providing the ability to have one research team easily consume data generated by another, utilizing Kubernetes' as a central data platform, is one of the major wins we've encountered by adopting the platform. Finally, I will showcase real-world use cases from NREL where we have used Kubernetes to solve some persistent data challenges involving large volumes of sensor and monitoring data. I will discuss the challenges we encountered when creating our cluster and making it available as a production-ready resource. I will also discuss the specific suite of tools, including Postgres and Apache Druid for columnar and timeseries data, and Redpanda Kafka for streaming data we have deployed in our infrastructure, and the process that went into the selection of these tools.

collaborative environment↗
DOE OSTI2022

Distilling physical origins of hardness in multi-principal element alloys directly from ensemble neural network models

Abstract Despite a plethora of data being generated on the mechanical behavior of multi-principal element alloys, a systematic assessment remains inaccessible via Edisonian approaches. We approach this challenge by considering the specific case of alloy hardness, and present a machine-learning framework that captures the essential physical features contributing to hardness and allows high-throughput exploration of multi-dimensional compositional space. The model, tested on diverse datasets, was used to explore and successfully predict hardness in Al x Ti y (CrFeNi) 1- x - y , Hf x Co y (CrFeNi) 1- x - y and Al x (TiZrHf) 1- x systems supported by data from density-functional theory predicted phase stability and ordering behavior. The experimental validation of hardness was done on TiZrHfAl x . The selected systems pose diverse challenges due to the presence of ordering and clustering pairs, as well as vacancy-stabilized novel structures. We also present a detailed model analysis that integrates local partial-dependencies with a compositional-stimulus and model-response study to derive material-specific insights from the decision-making process.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
DOE OSTI2025

The Importance of Being Adaptable: An Exploration of the Power and Limitations of Domain Adaptation for Simulation-Based Inference with Galaxy Clusters

The application of deep machine learning methods in astronomy has exploded in the last decade, with new models showing remarkably improved performance on benchmark tasks. Not nearly enough attention is given to understanding the models' robustness, especially when the test data are systematically different from the training data, or "out of domain." Domain shift poses a significant challenge for simulation-based inference, where models are trained on simulated data but applied to real observational data. In this paper, we explore domain shift and test domain adaptation methods for a specific scientific case: simulation-based inference for estimating galaxy cluster masses from X-ray profiles. We build datasets to mimic simulation-based inference: a training set from the Magneticum simulation, a scatter-augmented training set to capture uncertainties in scaling relations, and a test set derived from the IllustrisTNG simulation. We demonstrate that the Test Set is out of domain in subtle ways that would be difficult to detect without careful analysis. We apply three deep learning methods: a standard neural network (NN), a neural network trained on the scatter-augmented input catalogs, and a Deep Reconstruction-Regression Network (DRRN), a semi-supervised deep model engineered to address domain shift. Although the NN improves results by 17% in the Training Data, it performs 40% worse on the out-of-domain Test Set. Surprisingly, the Scatter-Augmented Neural Network (SANN) performs similarly. While the DRRN is successful in mapping the training and Test Data onto the same latent space, it consistently underperforms compared to a straightforward Yx scaling relation. These results serve as a warning that simulation-based inference must be handled with extreme care, as subtle differences between training simulations and observational data can lead to unforeseen biases creeping into the results.

Ntampaka, Michelle [Baltimore, Space Telescope Sci↗
DOE OSTI2020

Blanco DECam Bulge Survey (BDBS) II: project performance, data analysis, and early science results

ABSTRACT The Blanco DECam Bulge Survey (BDBS) imaged more than 200 sq deg of the Southern Galactic bulge using the ugrizY filters of the Dark Energy Camera, and produced point spread function photometry of approximately 250 million unique sources. In this paper, we present details regarding the construction and collation of survey catalogues, and also discuss the adopted calibration and dereddening procedures. Early science results are presented with a particular emphasis on the bulge metallicity distribution function and globular clusters. A key result is the strong correlation (σ ∼ 0.2 dex) between (u − i)o and [Fe/H] for bulge red clump giants. We utilized this relation to find that interior bulge fields may be well described by simple closed box enrichment models, but fields exterior to b ∼ −6° seem to require a secondary metal-poor component. Applying scaled versions of the closed box model to the outer bulge fields is shown to significantly reduce the strengths of any additional metal-poor components when compared to Gaussian mixture models. Additional results include: a confirmation that the u band splits the subgiant branch in M22 as a function of metallicity, the detection of possible extratidal stars along the orbits of M 22 and FSR 1758, and additional evidence that NGC 6569 may have a small but discrete He spread, as evidenced by red clump luminosity variations in the reddest bands. We do not confirm previous claims that FSR 1758 is part of a larger extended structure.

Johnson, Christian I.↗
DOE OSTI2025

Hiperclust

This software leverages transfer learning to analyze atom probe tomography (APT) data. It is trained on synthetic data and then applies this knowledge to predict the optimal number of clusters for a given APT dataset. Initially, the software used preliminary clustering to estimate the general structure of the data. Based on this, it provides suggestions for key parameters like minimum cluster size and minimum number of points. These parameters are critical for algorithms like HDBSCAN, ensuring accurate cluster formation without the need for trial-and-error testing. The software runs on High-Performance computing (HPC) systems, enabling fast, scalable analysis of large APT datasets, ultimately saving time and improving the reliability of clustering outcomes.

Tang, Yalei [Idaho National Laboratory (INL), Idah↗
DOE OSTI2021

Workflows Community Summit: Tightening the Integration between Computing Facilities and Scientific Workflows

Scientific workflows are used almost universally across science domains for solving complex and largescale computing and data analysis problems. The importance of workflows is highlighted by the fact that they have underpinned some of the most significant discoveries of the past decades. Many of these workflows have significant computational, storage, and communication demands, and thus must execute on a range of large-scale computer systems, from local clusters to public clouds and upcoming exascale HPC platforms. Managing these executions is often a significant undertaking, requiring a sophisticated and versatile software infrastructure. Historically, infrastructures for workflow execution consisted of complex, integrated systems, developed in-house by workflow practitioners with strong dependencies on a range of legacy technologies—even including sets of ad hoc scripts. Due to the increasing need to support workflows, dedicated workflow systems were developed to provide abstractions for creating, executing, and adapting workflows conveniently and efficiently while ensuring portability. While these efforts are all worthwhile individually, there are now hundreds of independent workflow systems. These workflow systems are created and used by thousands of researchers and developers, leading to a rapidly growing corpus of workflows research publications. The resulting workflow system technology landscape is fragmented, which may present significant barriers for future workflow users due to many seemingly comparable, yet usually mutually incompatible, systems that exist. In order to tackle some of the challenges described above, the DOE-funded ExaWorks and NSF-funded WorkflowsRI projects have organized in 2021 a series of events entitled the “Workflows Community Summit”. The third edition of the “Workflows Community Summit” explored workflows challenges and opportunities from the perspective of computing centers and facilities. This third summit builds on two prior summits (https://workflowsri.org/summits) that (i) established a high level vision for workflows research; and (ii) explored technical approaches for realizing that vision. The third summit brought together a small group of facilities representatives with the aim to understand how workflows are currently being used at each facility, how facilities would like to interact with workflow developers and users, how workflows fit with facility roadmaps, and what opportunities there are for tighter integration between facilities and workflows. This report documents and organizes the wealth of information provided by the participants before, during, and after the summit.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Building Alternative Indices of Socioeconomic Status for Population Modeling in Data-Sparse Contexts (Short Paper)

Population modeling requires clear definitions of socioeconomic status (SES) to ensure overall estimate accuracy and locate potentially underserved subpopulations. This presents a challenge as SES can be measured in myriad ways and for divergent purposes, and the data required to calculate these metrics may be lacking, particularly in low and middle income countries (LMICs). To support more refined SES measurement, we explore improvements upon the Demographic and Health Survey’s (DHS) Wealth Index (DHS-WI) using alternative characterizations of SES based on multiple correspondence analysis (MCA) and hierarchical clustering. We produce the MCA-derived metrics first on a full suite of household economic, demographic, and dwelling variables, then on a reduced set of occupant-only SES characteristics. We explore the utility of these metrics relative to DHS-WI based on their ability to 1) differentiate DHS household types and 2) identify mixtures of SES levels within DHS samples and mapped at high spatial resolution. We find that our full suite MCA yields more clearly defined SES segments and that our reduced MCA delineates occupant SES most clearly, suggesting potential pathways to improve upon the DHS-WI in future population modeling efforts for LMICs.

Cunningham, Angela↗
DOE OSTI2025

SIDM Concerto: Compilation and Data Release of Self-interacting Dark Matter Zoom-in Simulations

We present SIDM Concerto: 14 cosmological zoom-in simulations in cold dark matter (CDM) and self-interacting dark matter (SIDM) models based on the Symphony and Milky Way-est suites. SIDM Concerto includes one Large Magellanic Cloud– (LMC-) mass system (host mass ∼10 11 M ⊙ ), two Milky Way (MW) analogs (∼10 12 M ⊙ ), two group-mass hosts (∼10 13 M ⊙ ), and one low-mass cluster (∼10 14 M ⊙ ). Each host contains ≈2 × 10 7 particles and is run in CDM and one or more strong, velocity-dependent SIDM models. Our analysis of SIDM (sub)halo populations over seven subhalo mass decades reveals that (1) the fraction of core-collapsed isolated halos and subhalos peaks at a maximum circular velocity corresponding to the transition of the SIDM cross section from a v −4 to v 0 scaling; (2) SIDM subhalo mass functions are suppressed by ≈50% relative to CDM in LMC, MW, and group-mass hosts but are consistent with CDM in the low-mass cluster host; (3) subhalos’ inner density profile slopes, which are more diverse in SIDM than in CDM, are sensitive to both the amplitude and shape of the SIDM cross section. Our simulations provide a benchmark for testing SIDM predictions with astrophysical observations of field and satellite galaxies, strong lensing systems, and stellar streams. Data products are publicly available at doi:10.5281/zenodo.14933624.

dark matter↗
DOE OSTI2022

Applications of persistent homology in nuclear collisions

Here, we introduce a novel set of observables associated to the rapidly developing field of persistent homology for the quantitative characterization of nuclear collisions and their evolution. Persistent homology allows for the identification of topological and homological characteristics of distributions in multidimensional spaces. We demonstrate here how to apply the tool kit of persistent homology to the extraction of novel clustering signatures and the identification of long-range flow correlations in the particle production process of nuclear collisions.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2025

Time‐series multi‐omics analysis of micronutrient stress in Sorghum bicolor reveals iron and zinc crosstalk and regulatory network conservation

Micronutrient stress impacts growth, biomass production, and grain yield in crops. Multi-omics studies are valuable resources in identifying genes for functional studies and trait improvement, such as accumulation of Fe or Zn under deficient or excess conditions for bioenergy or grain agriculture. We conducted transcriptomics and ionomics analyses on Sorghum bicolor BTx623, grown under Fe and Zn limited and excess conditions over a 21-day period. To identify early and late transcriptional response in roots and leaves, 180 RNAseq libraries were sequenced for differential expression and co-expression network analyses. Fe and Zn accumulation was measured using ICP-MS at each time point, and a fluorometer was used to estimate chlorophyll content in leaves. Among the four treatments, Fe limitation and Zn excess resulted in the largest phenotypic effects and transcriptional response in roots and leaves. Several of the reduction (Strategy I) and chelation (Strategy II) strategy genes that improve bioavailability of Fe and Zn in plant roots often used by non-grass and grass species, respectively, were differentially expressed. Gene regulatory network (GRN) analysis of roots revealed enrichment of genes from Fe limiting and Zn excess which strongly connect to homologues of SbFIT, SbPYE, and SbBTS as hub genes. The GRN for leaf responses showed homologues of SbPYE and SbBTS as hubs connecting genes for chloroplast biosynthesis, Fe-S cluster assembly, photosynthesis, and ROS scavenging. Expression analyses suggest sorghum uses Strategy II genes for Fe and Zn uptake, as expected, but can also utilize Strategy I genes, which may be advantageous in variable moisture environments. We found strong overlap between Fe and Zn responsive GRNs, indicative of micronutrient crosstalk. We also found conservation of root and leaf GRNs, and known homologous genes suggest strong constraints on homeostasis networks in plants. These data will provide a resource for functional genetics to enhance micronutrient transport in sorghum, and opportunities to conduct further comparative GRN analysis across diverse crops species.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2025

High-speed Boulders and the Debris Field in DART Ejecta

On 2022 September 26 the Double Asteroid Redirection Test (DART) spacecraft collided with Dimorphos, the moon of the near-Earth asteroid 65803 Didymos, in a full-scale demonstration of a kinetic impactor concept. The companion Light Italian Cubesat for Imaging of Asteroids (LICIACube) spacecraft documented the aftermath, capturing images of the expansion and evolution of the ejecta from 29 to 243 s after the impact. We present results from our analyses of these observations, including an improved reduction of the data and new absolute calibration, an updated LICIACube trajectory, and a detailed description of the events and phenomena that were recorded throughout the flyby. One notable aspect of the ejecta was the existence of clusters of boulders, up to 3.6 m in radius, that were ejected at speeds of up to 52 m s −1 . Our analysis of the spatial distribution of 104 of these boulders suggests that they are likely the remnants of larger boulders shattered by the DART spacecraft in the first stages of the impact. The amount of momentum contained in these boulders is more than 3 times that of the DART spacecraft, and it is directed primarily to the south, almost perpendicular to the DART trajectory. Recoil of Dimorphos from the ejection of these boulders has the potential to change its orbital plane by up to a degree and to impart a non-principal-axis component to its rotation state. Damping timescales for these phenomena are such that the Hera spacecraft, arriving at the system in 2026, should be able to measure these effects.

asteroid dynamics↗
DOE OSTI2026

Jet fragmentation function and groomed substructure of bottom quark jets in proton-proton collisions at 5.02 TeV

A measurement of the substructure of bottom quark jets (b jets) in proton-proton (pp) collisions is presented. The measurement uses data collected in pp collisions at $\sqrt{s}=5.02$ TeV, with a low number of simultaneous interactions per bunch crossing, recorded by the CMS experiment in 2017, corresponding to an integrated luminosity of 301 pb −1 . An algorithm to identify and cluster the charged decay daughters of b hadrons is developed for this analysis, which facilitates the exposure of the gluon radiation pattern of b jets using iterative Cambridge-Aachen declustering. The soft-drop-groomed jet radius, R g , and momentum balance, z g , of b quark jets are presented. These observables can be used to test perturbative quantum chromodynamics predictions that account for mass effects. Because the b hadron is partially reconstructed from its charged decay daughters, only charged particles are used for the jet substructure studies. In addition, a jet fragmentation function, z b,ch , is measured, which is defined as the distribution of the ratio of the transverse momentum (p T ) of the partially reconstructed b hadron with respect to the charged-particle component of the jet p T . The substructure variable distributions are unfolded to the charged-particle level. The b jet substructure is compared to the substructure of jets in an inclusive jet sample that is dominated by light-quark and gluon jets in order to assess the role of the b quark mass. A strong suppression of emissions at small R g values is observed for b jets when compared to inclusive jets, consistent with the dead-cone effect. The measurement is also compared with theoretical predictions from Monte Carlo event generators. This is the first substructure measurement of b jets that clusters together the b hadron decay daughters independent of the b hadron species and decay channel.

boosted jets↗