Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data analysis cluster”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 451 records · Page 25
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

Robust design of semi-automated clustering models for 4D-STEM datasets

Materials discovery and design require characterizing material structures at the nanometer and sub-nanometer scale. Four-Dimensional Scanning Transmission Electron Microscopy (4D-STEM) resolves the crystal structure of materials, but many 4D-STEM data analysis pipelines are not suited for the identification of anomalous and unexpected structures. This work introduces improvements to the iterative Non-Negative Matrix Factorization (NMF) method by implementing consensus clustering for ensemble learning. We evaluate the performance of models during parameter tuning and find that consensus clustering improves performance in all cases and is able to recover specific grains missed by the best performing model in the ensemble. The methods introduced in this work can be applied broadly to materials characterization datasets to aid in the design of new materials.

Bruefach, Alexandra (ORCID:0000000209323477)↗
NASA NTRS1973

Analysis of data on absorption in the northern part of the milky way

The values of absorption for different galactic longitudes are obtained on the basis of star cluster data. It is found that the distance up to the point, where the straightening of the absorption distribution curve begins, depends on the limiting magnitude of the catalogue and the density of absorbing matter near the sun. The picture of absorbing matter distribution in the northern part of the Milky Way is given.

Polishchuk, E. P.↗
NASA NTRS2013

Reducing Earth Topography Resolution for SMAP Mission Ground Tracks Using K-Means Clustering

The K-means clustering algorithm is used to reduce Earth topography resolution for the SMAP mission ground tracks. As SMAP propagates in orbit, knowledge of the radar antenna footprints on Earth is required for the antenna misalignment calibration. Each antenna footprint contains a latitude and longitude location pair on the Earth surface. There are 400 pairs in one data set for the calibration model. It is computationally expensive to calculate corresponding Earth elevation for these data pairs. Thus, the antenna footprint resolution is reduced. Similar topographical data pairs are grouped together with the K-means clustering algorithm. The resolution is reduced to the mean of each topographical cluster called the cluster centroid. The corresponding Earth elevation for each cluster centroid is assigned to the entire group. Results show that 400 data points are reduced to 60 while still maintaining algorithm performance and computational efficiency. In this work, sensitivity analysis is also performed to show a trade-off between algorithm performance versus computational efficiency as the number of cluster centroids and algorithm iterations are increased.

ground tracks↗
NASA NTRS2007

Transport in the Subtropical Lowermost Stratosphere during CRYSTAL-FACE

We use in situ measurements of water vapor (H2O), ozone (O3), carbon dioxide (CO2), carbon monoxide (CO), nitric oxide (NO), and total reactive nitrogen (NO(y)) obtained during the CRYSTAL-FACE campaign in July 2002 to study summertime transport in the subtropical lowermost stratosphere. We use an objective methodology to distinguish the latitudinal origin of the sampled air masses despite the influence of convection, and we calculate backward trajectories to elucidate their recent geographical history. The methodology consists of exploring the statistical behavior of the data by performing multivariate clustering and agglomerative hierarchical clustering calculations, and projecting cluster groups onto principal component space to identify air masses of like composition and hence presumed origin. The statistically derived cluster groups are then examined in physical space using tracer-tracer correlation plots. Interpretation of the principal component analysis suggests that the variability in the data is accounted for primarily by the mean age of air in the stratosphere, followed by the age of the convective influence, and lastly by the extent of convective influence, potentially related to the latitude of convective injection [Dessler and Sherwuud, 2004]. We find that high-latitude stratospheric air is the dominant source region during the beginning of the campaign while tropical air is the dominant source region during the rest of the campaign. Influence of convection from both local and non-local events is frequently observed. The identification of air mass origin is confirmed with backward trajectories, and the behavior of the trajectories is associated with the North American monsoon circulation.

Pittman, Jasna V.↗
NASA NTRS2007

Transport in the Subtropical Lowermost Stratosphere during the Cirrus Regional Study of Tropical Anvils and Cirrus Layers-Florida Area Cirrus Experiment

We use in situ measurements of water vapor (H2O), ozone (O3), carbon dioxide (CO2), carbon monoxide (CO), nitric oxide (NO), and total reactive nitrogen (NOy) obtained during the CRYSTAL-FACE campaign in July 2002 to study summertime transport in the subtropical lowermost stratosphere. We use an objective methodology to distinguish the latitudinal origin of the sampled air masses despite the influence of convection, and we calculate backward trajectories to elucidate their recent geographical history. The methodology consists of exploring the statistical behavior of the data by performing multivariate clustering and agglomerative hierarchical clustering calculations and projecting cluster groups onto principal component space to identify air masses of like composition and hence presumed origin. The statistically derived cluster groups are then examined in physical space using tracer-tracer correlation plots. Interpretation of the principal component analysis suggests that the variability in the data is accounted for primarily by the mean age of air in the stratosphere, followed by the age of the convective influence, and last by the extent of convective influence, potentially related to the latitude of convective injection (Dessler and Sherwood, 2004). We find that high-latitude stratospheric air is the dominant source region during the beginning of the campaign while tropical air is the dominant source region during the rest of the campaign. Influence of convection from both local and nonlocal events is frequently observed. The identification of air mass origin is confirmed with backward trajectories, and the behavior of the trajectories is associated with the North American monsoon circulation.

Pittman, Jasna V.↗
NASA NTRSDate not supplied

Anomaly Detection for the Roman Space Telescope Wide Field Instrument’s Science Data Processing Pipeline

The Roman Space Telescope (RST) Wide Field Instrument (WFI) will be utilizing a preliminary Science Data Processing (SDP) pipeline during its Integration and Test, and to some extent during Operations, to track basic statistics and identify known features such as cosmic rays, snowballs as well as possible anomalies in raw detector data. In our detectors, these anomalies appear as jumps in the ramp of a readout and are classified as cosmic rays if they appear as a streak or snowballs if they’re more circular. The WFI employs an array of 18 H4RG-10 detectors that collect image samples. Each set of raw frames within a non-destructive exposure is packaged by the SDP pipeline into image cubes for each detector. Each cube is a time series of 4096 × 4096 accumulating pixel frames. The preliminary analysis pipeline is used to locate anomalies in these time-series accumulation frames and identify the type of anomaly, either natural phenomena or detector characteristic. To compare different methods, we’ve implemented both heuristic-based and data-driven methods to identify anomalies. For the heuristic-based approach, we identify snowballs and cosmic rays by the size and shape of outlier pixel clusters between consecutive frames. For data driven methods, we evaluated a Convolutional Neural Network (CNN) model, and more traditional methods like Principal Component Analysis (PCA). CNN is a supervised learning/classification method. Thus, we used a labeled dataset of anomalies to perform segmentation of the image and identify anomalies. We used previously identified cosmic rays and snowballs to measure the accuracy and efficiency of the mentioned approaches. In evaluating these methods, we aim to pick the best fit for the SDP pipeline’s anomaly detection in terms of both performance and runtime.

Paul Horton↗
NASA NTRS2014

Search for Cosmic-Ray-Induced Gamma-Ray Emission in Galaxy Clusters

Current theories predict relativistic hadronic particle populations in clusters of galaxies in addition to the already observed relativistic leptons. In these scenarios hadronic interactions give rise to neutral pions which decay into gamma rays that are potentially observable with the Large Area Telescope (LAT) on board the Fermi space telescope. We present a joint likelihood analysis searching for spatially extended gamma-ray emission at the locations of 50 galaxy clusters in four years of Fermi-LAT data under the assumption of the universal cosmic-ray (CR) model proposed by Pinzke & Pfrommer. We find an excess at a significance of 2.7 delta, which upon closer inspection, however, is correlated to individual excess emission toward three galaxy clusters: A400, A1367, and A3112. We discuss these cases in detail and conservatively attribute the emission to unmodeled background systems (for example, radio galaxies within the clusters).Through the combined analysis of 50 clusters, we exclude hadronic injection efficiencies in simple hadronic models above 21% and establish limits on the CR to thermal pressure ratio within the virial radius, R(sub 200), to be below 1.25%-1.4% depending on the morphological classification. In addition, we derive new limits on the gamma-ray flux from individual clusters in our sample.

Online-only material: color figures↗
DOE OSTI2025

Optimal reconstruction of baryon acoustic oscillations for DESI 2024

Baryon acoustic oscillations (BAO) provide a robust standard ruler to measure the expansion history of the Universe through galaxy clustering. Density-field reconstruction is now a widely adopted procedure for increasing the precision and accuracy of the BAO detection. With the goal of finding the optimal reconstruction settings to be used in the DESI 2024 galaxy BAO analysis, we assess the sensitivity of the post-reconstruction BAO constraints to different choices in our analysis configuration, performing tests on blinded data from the first year of DESI observations (DR1), as well as on mocks that mimic the expected clustering and selection properties of the DESI DR1 target samples. Overall, we find that BAO constraints remain robust against multiple aspects in the reconstruction process, including the choice of smoothing scale, treatment of redshift-space distortions, fiber assignment incompleteness, and parameterizations of the BAO model. We also present a series of tests that DESI followed in order to assess the maturity of the end-to-end galaxy BAO pipeline before the unblinding of the large-scale structure catalogs.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2020

Modeling regional precipitation over the Indus River basin of Pakistan using statistical downscaling

Complex processes govern spatiotemporal distribution of precipitation within the high-mountainous headwater regions (commonly known as the upper Indus basin (UIB)), of the Indus River basin of Pakistan. Reliable precipitation simulations particularly over the UIB present a major scientific challenge due to regional complexity and inadequate observational coverage. Here, we present a statistical downscaling approach to model observed precipitation of the entire Indus basin, with a focus on UIB within available data constraints. Taking advantage of recent high altitude (HA) observatories, we perform precipitation regionalization using K-means cluster analysis to demonstrate effectiveness of low-altitude stations to provide useful precipitation inferences over more uncertain and hydrologically important HA of the UIB. We further employ generalized linear models (GLM) with gamma and Tweedie distributions to identify major dynamic and thermodynamic drivers from a reanalysis dataset within a robust cross-validation framework that explain observed spatiotemporal precipitation patterns across the Indus basin. Final statistical models demonstrate higher predictability to resolve precipitation variability over wetter southern Himalayans and different lower Indus regions, by mainly using different dynamic predictors. The modeling framework also shows an adequate performance over more complex and uncertain trans-Himalayans and the northwestern regions of the UIB, particularly during the seasons dominated by the westerly circulations. However, the cryosphere-dominated trans-Himalayan regions, which largely govern the basin hydrology, require relatively complex models that contain dynamic and thermodynamic circulations. Furthermore, we also analyzed relevant atmospheric circulations during precipitation anomalies over the UIB, to evaluate physical consistency of the statistical models, as an additional measure of reliability. Overall, our results suggest that such circulation-based statistical downscaling has the potential to improve our understanding towards distinct features of the regional-scale precipitation across the upper and lower Indus basin. Additionally, such understanding should help to assess the response of this complex, data-scarce, and climate-sensitive river basin amid future climatic changes, to serve communal and scientific interests.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS2019

Electra: A Modular-Based Expansion of NASA's Supercomputing Capability

NASA has increasingly relied on high-performance computing (HPC) re- sources for computational modeling, simulation, and data analysis to meet the science and engineering goals of its missions in space exploration, aeronautics, and Earth and space science. The NASA Advanced Supercomputing (NAS) Division at Ames Research Center in Silicon Valley, Calif., hosts NASA’s premier supercomputing resources, integral to achieving and enhancing the success of the agency’s missions. NAS provides a balanced environment, funded under the High-End Computing Capability (HECC) project, comprised of world-class supercomputers, including its flagship distributed-memory cluster, Pleiades; high-speed networking; and massive data storage facilities, along with multi-disciplinary support teams for user support, code porting and optimization, and large-scale data analysis and scientific visualization. However, as scientists have increased the fidelity of their simulations and engineers are conducting larger parameter-space studies, the requirements for supercomputing resources have been growing by leaps and bounds. With the facility housing the HECC systems reaching its power and cooling capacity, NAS undertook a prototype project to investigate an alternative approach for housing supercomputers. Modular supercomputing, or container-based computing, is an innovative concept for expanding NASA’s HPC capabilities. With modular supercomputing, additional containers—similar to portable storage pods—can be connected together as needed to accommodate the agency’s ever-increasing demand for computing resources. In addition, taking advantage of the local weather permits the use of cooling technologies that would additionally save energy and reduce annual water usage. The first stage of NASA’s Modular Supercomputing Facility (MSF) prototype, which resulted in a 1,000 square-foot module on a concrete pad with room for 16 compute racks, was completed in Fall 2016 and an SGI (now HPE) computer system, named Electra, was deployed there in early 2017. Cooling is performed via an evaporative system built into the module, and preliminary experience shows a Power Usage Effectiveness (PUE) measurement of 1.03. Electra achieved over a petaflop on the LINPACK benchmark, sufficient to rank number 96 on the November 2016 TOP500 list [14]. The system consists of 1,152 InfiniBand-connected Intel Xeon Broadwell-based nodes. Its users access their files on a facility-wide file system shared by all HECC compute assets via Mellanox MetroX InfiniBand extenders, which connect the Electra fabric to Lustre routers in the primary facility over fiber-optic links about 900 feet long. The MSF prototype has exceeded expectations and is serving as a blueprint for future expansions. In the remainder of this chapter, we detail how modular data center technology can be used to expand an existing compute resource. We begin by describing NASA’s requirements for supercomputing and how resources were provided prior to the integration of the Electra module-based system.

Biswas, Rupak↗
NASA NTRS1994

Dynamics of cD clusters of galaxies. II: Analysis of seven Abell clusters

We have investigated the dynamics of the seven Abell clusters A193, A399, A401, A1795, A1809, A2063, and A2124, based on redshift data reported previously by us (Hill & Oegerle, (1993)). These papers present the initial results of a survey of cD cluster kinematics, with an emphasis on studying the nature of peculiar velocity cD galaxies and their parent clusters. In the current sample, we find no evidence for significant peculiar cD velocities, with respect to the global velocity distribution. However, the cD in A2063 has a significant (3 sigma) peculiar velocity with respect to galaxies in the inner 1.5 Mpc/h, which is likely due to the merger of a subcluster with A2063. We also find significant evidence for subclustering in A1795, and a marginally peculiar cD velocity with respect to galaxies within approximately 200 kpc/h of the cD. The available x-ray, optical, and galaxy redshift data strongly suggest that a subcluster has merged with A1795. We propose that the subclusters which merged with A1795 and A2063 were relatively small, with shallow potential wells, so that the cooling flows in these clusters were not disrupted. Two-body gravitational models of the A399/401 and A2063/MKW3S systems indicate that A399/401 is a bound pair with a total virial mass of approximately 4 x 10(exp 15) solar mass/h, while A2063 and MKW3S are very unlikely to be bound.

Oegerle, William R.↗
DOE OSTI2025

A Versatile Simulated Data Transport Layer for in Situ Workflows Performance Evaluation

In situ processing does not only allow scientific applications to face the explosion in data volume and velocity but also to address the time constraints of many simulation-analysis workflows by providing scientists with early insights about their applications at runtime. Multiple frameworks implement the concept of a data transport layer (DTL) to enable such in situ workflows. These tools are very versatile, directly or indirectly access the data generated on the same node, another node of the same compute cluster, or a completely distinct node, and allow data publishers and subscribers to run on the same computing resources or not. This versatility puts on researchers the onus of taking key decisions related to resource allocation and how to transport data to ensure the most efficient execution of their in situ workflows. However, domain scientists and workflow practitioners lack the appropriate tools to assess the respective performance of particular design and deployment options. In this paper we introduce a versatile simulated DTL designed to provide researchers with insights on the respective performance of different execution scenarios of in situ workflows. This open-source, standalone library builds on the SimGrid toolkit and can be linked to any SimGrid-based simulator. It facilitates the evaluation of the performance behavior, at scale, of different data transport configurations and the study of the effects of resource allocation strategies. We demonstrate the scalability, versatility, and accuracy of this simulated DTL by reproducing the execution of two synthetic benchmarks and of a real-world in situ workflow composed of an MPI application and a parallel data analysis. Results of simulations run on a single core show that the proposed library can simulate the interactions of tens of thousands of simulated processes deployed on two interconnected commodity clusters in a few seconds, and the execution by a thousand simulated processes of an in situ workflow in less than three minutes.

Suter, Fred [ORNL] (ORCID:0000000319021955)↗
DOE OSTI2022

Data augmentation for disruption prediction via robust surrogate models

The goal of this work is to generate large statistically representative data sets to train machine learning models for disruption prediction provided by data from few existing discharges. Such a comprehensive training database is important to achieve satisfying and reliable prediction results in artificial neural network classifiers. Here, we aim for a robust augmentation of the training database for multivariate time series data using Student t process regression. We apply Student t process regression in a state space formulation via Bayesian filtering to tackle challenges imposed by outliers and noise in the training data set and to reduce the computational complexity. Thus, the method can also be used if the time resolution is high. We use an uncorrelated model for each dimension and impose correlations afterwards via colouring transformations. We demonstrate the efficacy of our approach on plasma diagnostics data of three different disruption classes from the DIII-D tokamak. To evaluate if the distribution of the generated data is similar to the training data, we additionally perform statistical analyses using methods from time series analysis, descriptive statistics and classic machine learning clustering algorithms.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2023

A proteomic meta-analysis refinement of plasma extracellular vesicles

Extracellular vesicles play major roles in cell-to-cell communication and are excellent biomarker candidates. However, studying plasma extracellular vesicles is challenging due to contaminants. Here, we performed a proteomics meta-analysis of public data to refine the plasma EV composition by separating EV proteins and contaminants into different clusters. We obtained two clusters with a total of 1717 proteins that were depleted of known contaminants and enriched in EV markers with independently validated 71% true-positive. These clusters had 133 clusters of differentiation (CD) antigens and were enriched with proteins from cell-to-cell communication and signaling. We compared our data with the proteins deposited in PeptideAtlas, making our refined EV protein list a resource for mechanistic and biomarker studies. As a use case example for this resource, we validated the type 1 diabetes biomarker proplatelet basic protein in EVs and showed that it regulates apoptosis of β cells and macrophages, two key players in the disease development. Our approach provides a refinement of the EV composition and a resource for the scientific community.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2023

Measurement of substructure-dependent jet suppression in Pb + Pb collisions at 5.02 TeV with the ATLAS detector

The ATLAS detector at the Large Hadron Collider has been used to measure jet substructure modification and suppression in Pb + Pb collisions at a nucleon–nucleon center-of-mass energy $\sqrt{{^S}_{NN}}$ = 5.02 TeV in comparison with proton–proton (pp) collisions at $\sqrt{{^S}}$ = 5.02 TeV. The Pb + Pb data, collected in 2018, have an integrated luminosity of 1.72 nb -1 , while the pp data, collected in 2017, have an integrated luminosity of 260 pb -1 . Jets used in this analysis are clustered using the anti- k t algorithm with a radius parameter R = 0.4. The jet constituents, defined by both tracking and calorimeter information, are used to determine the angular scale r g of the first hard splitting inside the jet by reclustering them using the Cambridge–Aachen algorithm and employing the soft-drop grooming technique. The nuclear modification factor, R AA , used to characterize jet suppression in Pb + Pb collisions, is presented differentially in r g , jet transverse momentum, and in intervals of collision centrality. The R AA value is observed to depend significantly on jet r g . Jets produced with the largest measured r g are found to be twice as suppressed as those with the smallest r g in central Pb + Pb collisions. The R AA values do not exhibit a strong variation with jet p T in any of the r g intervals. The r g and p T dependence of jet R AA is qualitatively consistent with a picture of jet quenching arising from coherence and provides the most direct evidence in support of this approach.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2024

Pando

SAND2025-02006O Pando is a distributed data analysis software tool. It is designed to handle large-scale graph analysis problems, often with a specific focus on blockchain/cryptocurrency data. Pando handles scalability by running on a distributed cluster of servers. Users can customize the output using the program’s plugin/extension design methodology. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525.

Gabert, Kasimir↗
DOE OSTI2020

The Dynamics of the Wide-angle Tailed (WAT) Galaxy Cluster A562

We present the first in-depth dynamical analysis of the archetypal wide-angle tailed (WAT) cluster A562. We have combined Gemini observations with archival data from the literature to form a sample of 76 cluster members and derived a mean redshift of 0.1088 ± 0.0004 and a velocity dispersion of 919 ± 116 km s{sup −1}. This relatively large velocity dispersion suggests either a very massive cluster (M {sub dyn} > 6.9 × 10{sup 14} M {sub ☉}) and/or a merger system. The merger model is supported by a non-Gaussian galaxy velocity distribution, an elongated spatial distribution of likely cluster members, and an elongated X-ray emitting gas. This scenario would generate the bulk flow motion of the intracluster medium that can exert enough ram pressure to bend the radio jets. Thus, our observations support the model in which a recent off-axis merger event produced the cluster wide conditions needed to shape the WAT in A562.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2022

Data augmentation for disruption prediction via robust surrogate models

The goal of this work is to generate large statistically representative datasets to train machine learning models for disruption prediction provided by data from few existing discharges. Such a comprehensive training database is important to achieve satisfying and reliable prediction results in artificial neural network classifiers. Here, we aim for a robust augmentation of the training database for multivariate time series data using Student-t process regression. We apply Student-t process regression in a state space formulation via Bayesian filtering to tackle challenges imposed by outliers and noise in the training data set and to reduce the computational complexity. Thus, the method can also be used if the time resolution is high. We use an uncorrelated model for each dimension and impose correlations afterwards via coloring transformations. We demonstrate the efficacy of our approach on plasma diagnostics data of three different disruption classes from the DIII-D tokamak. To evaluate if the distribution of the generated data is similar to the training data, we additionally perform statistical analyses using methods from time series analysis, descriptive statistics, and classic machine learning clustering algorithms.

97 MATHEMATICS AND COMPUTING↗