Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data analysis cluster”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3
All sourcesNASA NTRSDOE OSTI
NASA NTRS1985

A cluster analysis method for identification of subpopulations of cells in flow cytometric list-mode arrays

A specialized program was developed for flow cytometric list-mode data using an heirarchical tree method for identifying and enumerating individual subpopulations, the method of principal components for a two-dimensional display of 6-parameter data array, and a standard sorting algorithm for characterizing subpopulations. The program was tested against a published data set subjected to cluster analysis and experimental data sets from controlled flow cytometry experiments using a Coulter Electronics EPICS V Cell Sorter. A version of the program in compiled BASIC is usable on a 16-bit microcomputer with the MS-DOS operating system. It is specialized for 6 parameters and up to 20,000 cells. Its two-dimensional display of Euclidean distances reveals clusters clearly, as does its 1-dimensional display. The identified subpopulations can, in suitable experiments, be related to functional subpopulations of cells.

Li, Z. K.↗
DOE OSTI2023

BiG-SLiCE 2 v1.0.0

BiG-SLiCE was originally an open source Python-based command line bioinformatics software that offers a highly scalable clustering analysis on biosynthetic gene clusters (BGC) data. It allows a simultaneous analysis of millions of BGCs, exceeding the capability of other existing tools (around one hundred thousands). As a tradeoff, the clustering accuracy is relatively lower and sometimes fall short in corner cases and specific BGC classes such as the RiPPs (Ribosomally-translated, Post-translationally modified Peptides). In BiG-SLiCE V2 (developed in LBNL), the clustering algorithm has been significantly improved to deliver a much accurate result even for RiPPs and other previous corner case classes. Moreover, the speed of the overall pipeline has been improved by 50-100%. Finally, additional features were implemented to support downstream analyses of BiG-SLiCE results, such as customized tabular (TSV/CSV) and columnar (Parquet) outputs.

Kautsar, Satria↗
NASA NTRS2005

Small-Scale Drop Size Variability: Impact on Estimation of Cloud Optical Properties

Most cloud radiation models and conventional data processing techniques assume that the mean number of drops of a given radius is proportional to volume. The analysis of microphysical data on liquid water drop sizes shows that, for sufficiently small volumes, this proportionality breaks down; the number of cloud drops of a given radius is instead proportional to the volume raised to a drop size-dependent nonunit power. The coefficient of proportionality, a generalized drop concentration, is a function of the drop size. For abundant small drops the power is unity as assumed in the conventional approach. However, for rarer large drops, it falls increasingly below unity. This empirical fact leads to drop clustering, with the larger drops exhibiting a greater degree of clustering. The generalized drop concentration shows the mean number of drops per cluster, while the power characterizes the occurrence frequency of clusters. With a fixed total number of drops in a cloud, a decrease in frequency of clusters is accompanied by a corresponding increase in the generalized concentration. This initiates a competing process missed in the conventional models: an increase in the number of drops per cluster enhances the impact of rarer large drops on cloud radiation while a decrease in the frequency suppresses it. Because of the nonlinear relationship between the number of clustered drops and the volume, these two opposite tendencies do not necessarily compensate each other. The data analysis suggests that clustered drops likely have a stronger radiative impact compared to their unclustered counterpart; ignoring it results in underestimation of the contribution from large drops to cloud horizontal optical path.

Knyazikhin, Y.↗
DOE OSTI2025

Web-based wide-area monitoring platform for ringdown and clustering analytics in power systems

This paper introduces an open-source research platform for monitoring the Mexican interconnected power grid, allowing real-time processing and information extraction of the grid’s dynamic condition. Moreover, the platform is a Python-based development that embeds different ringdown and clustering analytics tools. In the case of ringdown analysis, the modal information can be extracted using some of the most known algorithms, i.e., Prony analysis, eigensystem realization algorithm (ERA), and matrix pencil (MP). For clustering analysis, the coherent behaviour of generator and non-generator buses is provided by applying recent state-of-the-art techniques such as affinity propagation, K-means, hierarchical agglomerative clustering, and typicality data analysis. The results of up to 93 PMUs show that this open-source platform suits researchers’ and engineers’ power system dynamic analysis requirements.

Clustering↗
DOE OSTI2022

Estimating Cosmological Constraints from Galaxy Cluster Abundance using Simulation-Based Inference

Inferring the values and uncertainties of cosmological parameters in a cosmology model is of paramount importance for modern cosmic observations. In this paper, we use the simulation-based inference (SBI) approach to estimate cosmological constraints from a simplified galaxy cluster observation analysis. Using data generated from the Quijote simulation suite and analytical models, we train a machine learning algorithm to learn the probability function between cosmological parameters and the possible galaxy cluster observables. The posterior distribution of the cosmological parameters at a given observation is then obtained by sampling the predictions from the trained algorithm. Our results show that the SBI method can successfully recover the truth values of the cosmological parameters within the 2σ limit for this simplified galaxy cluster analysis, and acquires similar posterior constraints obtained with a likelihood-based Markov Chain Monte Carlo method, the current state-of the-art method used in similar cosmological studies.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
NASA NTRS2005

Impacts of the Variability of Ice Types on the Decline of the Arctic Perennial Sea Ice Cover

The observed rapid decline in the Arctic perennial ice cover is one of the most remarkable signal of change in the Arctic region. Updated data now show an even higher rate of decline of 9.8% per decade than the previous report of 8.9% per decade mainly because of abnormally low values in the last 4 years. To gain insights into this decline, the variability of the second year ice, which is the relatively thin component of the perennial ice cover, and other ice types is studied. The perennial ice cover in the 1990s was observed to be highly variable which might have led to higher production of second year ice and may in part explain the observed ice thinning during the period and triggered further decline. The passive microwave signature of second year ice is also studied and results show that while the signature is different from that of the older multiyear ice, it is surprisingly more similar to that of first year ice. This in part explains why previous estimates of the area of multiyear ice during the winter period are considerably lower than the area of the perennial ice cover during the preceding summer. Four distinct clusters representing radiometrically different types have been identified using multi-channel cluster analysis of passive microwave data. Data from two of these clusters, postulated to come from second year and older multiyear ice regions are also shown to have average thicknesses of 2.4 and 4.1 m, respectively, indicating that the passive microwave data may contain some ice thickness information that can be utilized for mass balance studies. The yearly anomaly maps indicate high gains of first year ice cover in the Arctic during the last decade which means higher production of second year ice and fraction of this type in the declining perennial ice cover. While not the only cause, the rapid decline in the perennial ice cover is in part caused by the increasing fractional component of the thinner second year ice cover that is very vulnerable to total melt due to warming in the Arctic, especially in spring.

Comiso, Josefino C.↗
DOE OSTI2025

Enhancing Cluster Identification in Atom Probe Tomography Data Using Transfer Learning

Atom Probe Tomography (APT) is a powerful technique for visualizing the atomic-scale distribution of solutes in materials, but quantitative cluster analysis of APT datasets remains a challenge due to the need for subjective parameter selection in clustering algorithms. While distance-based and density-based methods such as HDBSCAN are widely used, their performance is highly sensitive to user-defined parameters, which undermines reproducibility and accuracy. This study proposes an image-based, deep learning-aided workflow for automating parameter selection and cluster detection in APT data analysis. By projecting 3D APT point clouds onto 2D planes, we leverage pretrained convolutional neural networks (ConvNeXt-Tiny and ResNet-50) through transfer learning to predict the number of clusters present in synthetic datasets. The output is used to guide K-means clustering and estimate HDBSCAN parameters, specifically minimum cluster size and minimum sample points. This approach reduces reliance on manual parameter tuning, improving consistency and scalability. The methodology demonstrates the feasibility of using image-based deep learning for interpreting complex spatial patterns in APT data, enabling faster and more objective analysis. The complete workflow and code are made publicly available to support reproducibility and future research.

Density-based clustering↗
NASA NTRS1991

Monte Carlo Shower Counter Studies

Activities and accomplishments related to the Monte Carlo shower counter studies are summarized. A tape of the VMS version of the GEANT software was obtained and installed on the central computer at Gallaudet University. Due to difficulties encountered in updating this VMS version, a decision was made to switch to the UNIX version of the package. This version was installed and used to generate the set of data files currently accessed by various analysis programs. The GEANT software was used to write files of data for positron and proton showers. Showers were simulated for a detector consisting of 50 alternating layers of lead and scintillator. Each file consisted of 1000 events at each of the following energies: 0.1, 0.5, 2.0, 10, 44, and 200 GeV. Data analysis activities related to clustering, chi square, and likelihood analyses are summarized. Source code for the GEANT user subprograms and data analysis programs are provided along with example data plots.

Snyder, H. David↗
DOE OSTI2022

Machine-learning identification of the variability of mean velocity and turbulence intensity for wakes generated by onshore wind turbines: Cluster analysis of wind LiDAR measurements

Light detection and ranging (LiDAR) measurements of isolated wakes generated by wind turbines installed at an onshore wind farm are leveraged to characterize the variability of the wake mean velocity and turbulence intensity during typical operations, which encompass a breadth of atmospheric stability regimes and rotor thrust coefficients. The LiDAR measurements are clustered through the k-means algorithm, which enables identifying the most representative realizations of wind turbine wakes while avoiding the imposition of thresholds for the various wind and turbine parameters. Considering the large number of LiDAR samples collected to probe the wake velocity field, the dimensionality of the experimental dataset is reduced by projecting the LiDAR data on an intelligently truncated basis obtained with the proper orthogonal decomposition (POD). The coefficients of only five physics-informed POD modes are then injected in the k-means algorithm for clustering the LiDAR dataset. The analysis of the clustered LiDAR data and the associated supervisory control and data acquisition and meteorological data enables the study of the variability of the wake velocity deficit, wake extent, and wake-added turbulence intensity for different thrust coefficients of the turbine rotor and regimes of atmospheric stability. Furthermore, the cluster analysis of the LiDAR data allows for the identification of systematic off-design operations with a certain yaw misalignment of the turbine rotor with the mean wind direction.

17 WIND ENERGY↗
DOE OSTI2024

A Generative Model for Realistic Galaxy Cluster X-Ray Morphologies

Abstract The X-ray morphologies of clusters of galaxies display significant variations, reflecting their dynamical histories and the nonlinear dependence of X-ray emissivity on the density of the intracluster gas. Qualitative and quantitative assessments of X-ray morphology have long been considered a proxy for determining whether clusters are dynamically active or “relaxed.” Conversely, the use of circularly or elliptically symmetric models for cluster emission can be complicated by the variety of complex features realized in nature, spanning scales from megaparsecs down to the resolution limit of current X-ray observatories. In this work, we use mock X-ray images from simulated clusters from The Three Hundred project to define a basis set of cluster image features. We take advantage of the clusters’ approximate self-similarity to minimize the differences between images before encoding the remaining diversity through a distribution of high-order polynomial coefficients. Principal component analysis then provides an orthogonal basis for this distribution, corresponding to natural perturbations from an average model. This representation allows novel, realistically complex X-ray cluster images to be easily generated, and we provide code to do so. The approach provides a simple way to generate training data for cluster image analysis algorithms and could be straightforwardly adapted to generate clusters displaying specific types of features or selected by physical characteristics available in the original simulations.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2020

Hybrid deep learning architecture for general disruption prediction across tokamaks

In this paper, we present a new deep learning disruption prediction algorithm based on important findings from explorative data analysis which effectively allows knowledge transfer from existing devices to new ones, thereby predicting disruptions using very limited disruptive data from the new devices. Here, the explorative data analysis conducted via unsupervised clustering techniques confirms that time-sequence data are much better separators of disruptive and non-disruptive behavior than the instantaneous plasma state data with further advantageous implications for a sequence-based predictor. Based on such important findings, we have designed a new algorithm for multi-machine disruption prediction that achieves high predictive accuracy on the C-Mod (AUC=0.801), DIII-D (AUC=0.947) and EAST (AUC=0.973) tokamaks with limited hyperparameter tuning. Through numerical experiments, we show that boosted accuracy (AUC=0.959) is achieved on EAST predictions by including in the training only 20 disruptive discharges, thousands of non-disruptive discharges from EAST, and combining this with more than a thousand discharges from DIII-D and C-Mod. The improvement of predictive ability obtained by combining disruptive data from other devices is found to be true for all permutations of the three devices. Furthermore, by comparing the predictive performance of each individual numerical experiment, we find that non-disruptive data are machine-specific while disruptive data from multiple devices contain device-independent knowledge that can be used to inform predictions for disruptions occurring on a new device.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2021

Hybrid deep learning architecture for general disruption prediction across tokamaks

In this paper, we present a new deep learning disruption prediction algorithm based on important findings from explorative data analysis which effectively allows knowledge transfer from existing devices to new ones, thereby predicting disruptions using very limited disruptive data from the new devices. The explorative data analysis conducted via unsupervised clustering techniques confirms that time-sequence data are much better separators of disruptive and non-disruptive behavior than the instantaneous plasma state data with further advantageous implications for a sequence-based predictor. Based on such important findings, we have designed a new algorithm for multi-machine disruption prediction that achieves high predictive accuracy on the C-Mod (AUC=0.801), DIII-D (AUC=0.947) and EAST (AUC=0.973). tokamaks with limited hyperparameter tuning. Through numerical experiments, we show that boosted accuracy (AUC=0.959) is achieved on EAST predictions by including in the training only 20 disruptive discharges, thousands of non-disruptive discharges from EAST, and combining this with more than a thousand discharges from DIII-D and C-Mod. The improvement of predictive ability obtained by combining disruptive data from other devices is found to be true for all permutations of the three devices. Furthermore, by comparing the predictive performance of each individual numerical experiment, we find that non-disruptive data are machine-specific while disruptive data from multiple devices contain device-independent knowledge that can be used to inform predictions for disruptions occurring on a new device.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
NASA NTRS1975

Elemental composition of suspended particulates as functions of space and time in Cleveland, Ohio

An approach to the analysis of a very large elemental concentration data set. The particular data considered was generated by instrumental neutron activation and emission spectroscopy analyses of over 750 24-hour ambient air particulate samples collected at 16 sites in Cleveland, Ohio, during the 15 months from August 1971 thru October 1972. Examples are presented that show the use of multiple approaches to interpreting the data, including pairwise correlation statistics, selective data plotting and cluster analysis.

Neustadter, H. E.↗
DOE OSTI2020

Untangling the Galaxy. II. Structure within 3 kpc

We present the results of the hierarchical clustering analysis of the Gaia DR2 data to search for clusters, comoving groups, and other stellar structures. The current paper builds on the sample from the previous work, extending it in distance from 1 to 3 kpc and increasing the number of identified structures up to 8292. To aid in the analysis of the population properties, we developed a neural network called Auriga to robustly estimate the age, extinction, and distance of a stellar group based on the input photometry and parallaxes of the individual members. We apply Auriga to derive the properties of not only the structures found in this paper, but also previously identified open clusters. Through this work, we examine the temporal structure of the spiral arms. Specifically, we find that the Sagittarius Arm has moved by >500 pc in the last 100 Myr and the Perseus Arm has been experiencing a relative lull in star formation activity over the last 25 Myr. We confirm the findings of the previous paper on the transient nature of the spiral arms, with the timescale of transition of a few 100 Myr. Finally, we find a peculiar ~1 Gyr old stream of stars that appears to be heliocentric. Its origin is unclear.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2021

Code Artifact for: Clustering Analysis of Commercial Vehicles Using Automatically Extracted Features from Time Series Data [SWR-21-96]

This repository contains data ingestion, feature extraction, and analysis code used in NREL Technical report "Clustering Analysis of Commercial Vehicles Using Automatically Extracted Features from Time Series Data." The code is written in Python. The ETL and feature extraction code must be run in a Spark context. The analysis code can be run without Spark, provided you have pre-computed features in a CSV file. Analysis code related to the NREL Technical Report NREL/TP-2C00-74212. Includes PySpark functions to perform trip segmentation and feature extraction over big time series data in Apache Spark. Includes "domain specific" features such as Aerodynamic Speed (ft/s), Characteristic Acceleration (ft/s2), Percent Below 55 (%), Percent Zero (%), Stops Per Mile, Average Speed (mph), Maximum Speed (mph), and Speed Standard Deviation (mph). Includes Pyspark UDF to compute "domain agnostic" features using the TSFresh library. This software record also includes the analysis notebooks and code to generate the results in the previously mentioned technical report.

Perr-Sauer, Jordan↗
DOE OSTI2022

High-throughput investigation of structural evolution upon solid-state in Cu–Cr–Co combinatorial multilayer thin-film

Cu-Cr-Co combinatorial multilayer thin-films were prepared by a high-throughput ion beam sputtering system. Based on the thickness ratio among the individual nanoscale monolayers (Cu, Cr, Co), the resulting stoichiometry covered the entire phase diagram. The chemical composition and structure of Cu-Cr-Co combinatorial chip upon solid-state reaction were studied by lab-based micro-X-ray fluorescence (mu-XRF) and high-throughput synchrotron X-ray diffraction (XRD), respectively. A composition-structure map for Cu-Cr-Co combinatorial chip was developed through automated data analysis employing hierarchical clustering techniques. The structural evolution of Cu-Cr-Co combinatorial chip as a function of heat-treatment temperature, time, and modulation period were studied systematically. Furthermore, the effect of the elemental distribution in-depth direction was investigated to gain more insights regarding phase transformation. This work provides an efficient method and new perspectives for the design and optimization of the composition and structure of high-performance thin-films.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Nitrogen starvation causes lipid remodeling in Rhodotorula toruloides

Abstract Background The oleaginous yeast Rhodotorula toruloides is a promising chassis organism for the biomanufacturing of value-added bioproducts. It can accumulate lipids at a high fraction of biomass. However, metabolic engineering efforts in this organism have progressed at a slower pace than those in more extensively studied yeasts. Few studies have investigated the lipid accumulation phenotype exhibited by R. toruloides under nitrogen limitation conditions. Consequently, there have been only a few studies exploiting the lipid metabolism for higher product titers. Results We performed a multi-omic investigation of the lipid accumulation phenotype under nitrogen limitation. Specifically, we performed comparative transcriptomic and lipidomic analysis of the oleaginous yeast under nitrogen-sufficient and nitrogen deficient conditions. Clustering analysis of transcriptomic data was used to identify the growth phase where nitrogen-deficient cultures diverged from the baseline conditions. Independently, lipidomic data was used to identify that lipid fractions shifted from mostly phospholipids to mostly storage lipids under the nitrogen-deficient phenotype. Through an integrative lens of transcriptomic and lipidomic analysis, we discovered that R. toruloides undergoes lipid remodeling during nitrogen limitation, wherein the pool of phospholipids gets remodeled to mostly storage lipids. We identify specific mRNAs and pathways that are strongly correlated with an increase in lipid levels, thus identifying putative targets for engineering greater lipid accumulation in R. toruloides . One surprising pathway identified was related to inositol phosphate metabolism, suggesting further inquiry into its role in lipid accumulation. Conclusions Integrative analysis identified the specific biosynthetic pathways that are differentially regulated during lipid remodeling. This insight into the mechanisms of lipid accumulation can lead to the success of future metabolic engineering strategies for overproduction of oleochemicals.

59 BASIC BIOLOGICAL SCIENCES↗
NASA NTRS1993

Mass and spatial distribution of carbonaceous component in Comet Halley

Cometary grains containing large amounts of carbon and/or organic matter were discovered by in situ measurements of cometary dust composition during VEGA and GIOTTO fly-by missions. In accordance with the classification for the data of PUMA-1 and PUMA-2 mass-spectrometers on board the VEGA spacecraft, particles with a ratio of C to any rock-forming element (Mg, Si, Fe, Ca etc.) greater than 10, were categorized as CHON. There are 464 such particles in PUMA-1 data and 51 in PUMA-2 data. Application of cluster analysis to these grains revealed several distinct compositional classes, namely: (H,C,N,O), (H,C,N), (H,C), (H,C,O), (C,N), (C,O), (C,N,O), and (C). Similar classes were identified among particles analyzed by PIA. Also, about a third of all particles fell into groups (H) and (O) characterized by abundances of these elements beyond chemically reasonable limits.

Fomenkova, M.↗