Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “clustering algorithm”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 595 records · Page 33
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Combining Spectroscopic and Imaging Galaxy Surveys for Improved Measurements of Large-Scale Structure

Galaxy imaging surveys represent a promising opportunity for testing cosmological models but suffer from significant challenges associated with galaxy redshift uncertainty. I present improved techniques for the use of spectroscopic redshifts for measurements of large-scale structure with galaxy imaging surveys. I additionally present the results of the application of these techniques to data from the Sloan Digital Sky Survey (SDSS) and the Dark Energy Survey (DES). The results of my work are photometric redshift measurements of the approximately 100 million galaxies comprising the the DES Year 3 Weak Lensing Source Galaxy Catalog, constraints on parameters of the Lambda-CDM cosmological model based on the gravitational lensing and clustering of the DES Year 3 source and lens galaxy catalogs, an algorithm for propagating uncertainties measured in simulations to analogous measurements on data, and measurements of the incidence of projection effects in optically detected clusters of galaxies. I argue that continued development of the methods I have worked on is a promising path toward a conclusive test of Lambda-CDM with galaxy surveys.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2021

Global Optimization of Chemical Cluster Structures: Methods, Applications, and Challenges

Chemical clusters are relevant to many applications in catalysis, separations, materials, and energy sciences. Experimentally, the structure of clusters is difficult to determine, but it is very important in understanding their chemistry and properties. Computational methods can be used to examine cluster structure, however finding the most stable structure is not simple, particularly as the cluster size increases. Global optimization techniques have long been used to tackle the problem of the most stable structure, but such approaches would have to look for a global minimum, while sampling local minima over the whole potential energy surface as well. In this review, the state-of-the-art theory of global optimization theory is summarized. First, the definition, significance, relation to experiments, and a brief history of global optimization is presented. We then discuss, in more detail, three versatile global optimization methods: the basin hopping, the artificial bee colony algorithm, and the genetic algorithm. We close with some representative application examples of global optimization of clusters since 2016 and the challenges, open questions and opportunities in this field.

Global optimization, Chemical clusters, Artificial↗
DOE OSTI2020

Symbol alphabets from plabic graphs

Symbol alphabets of n-particle amplitudes in N = 4 super-Yang-Mills theory are known to contain certain cluster variables of Gr(4, n) as well as certain algebraic functions of cluster variables. In this paper we suggest an algorithm for computing these symbol alphabets from plabic graphs by solving matrix equations of the form C ∙ Z = 0 to associate functions on Gr(m, n) to parameterizations of certain cells of Gr(k, n) indexed by plabic graphs. For m = 4 and n = 8 we show that this association precisely reproduces the 18 algebraic symbol letters of the two-loop NMHV eight-particle amplitude from four plabic graphs.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2024

A mapped dataset of surface ocean acidification indicators in large marine ecosystems of the United States

Mapped monthly data products of surface ocean acidification indicators from 1998 to 2022 on a 0.25° by 0.25° spatial grid have been developed for eleven U.S. large marine ecosystems (LMEs). The data products were constructed using observations from the Surface Ocean CO 2 Atlas, co-located surface ocean properties, and two types of machine learning algorithms: Gaussian mixture models to organize LMEs into clusters of similar environmental variability and random forest regressions (RFRs) that were trained and applied within each cluster to spatiotemporally interpolate the observational data. The data products, called RFR-LMEs, have been averaged into regional timeseries to summarize the status of ocean acidification in U.S. coastal waters, showing a domain-wide carbon dioxide partial pressure increase of 1.4 ± 0.4 μatm yr -1 and pH decrease of 0.0014 ± 0.0004 yr -1 . RFR-LMEs have been evaluated via comparisons to discrete shipboard data, fixed timeseries, and other mapped surface ocean carbon chemistry data products. Regionally averaged timeseries of RFR-LME indicators are provided online through the NOAA National Marine Ecosystem Status web portal.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

Variational quantum eigensolver for approximate diagonalization of downfolded Hamiltonians using generalized unitary coupled cluster ansatz

In this paper we discuss the utilization of Variational Quantum Solver (VQE) and recently introduced Generalized Unitary Coupled Cluster (GUCC) formalism for the diagonalization of downfolded/effective Hamiltonians in active spaces. In addition to effective Hamiltonians defined by the downfolding of a subset of virtual orbitals we also consider their form defined by freezing core orbitals, which enables us to deal with larger systems. Furthermore, we also consider various solvers to identify solutions of the GUCC equations. We use N 2 , H 2 O, and C 2 H 4 , and benchmark systems to illustrate the performance of the combined framework.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2025

Fast and Scalable FFT-Based GPU-Accelerated Algorithms for Block-Triangular Toeplitz Matrices with Application to Linear Inverse Problems Governed by Autonomous Dynamical Systems

In this work, we present an efficient and scalable algorithm for performing matrix-vector multiplications (matvecs) for block Toeplitz matrices. Such matrices, which are shift-invariant with respect to their blocks, arise in the context of solving inverse problems governed by autonomous systems, and time-invariant systems in particular. In this article, we consider inverse problems that infer unknown parameters from observational data of a linear time-invariant dynamical system given in the form of partial differential equations (PDEs). Matrix-free Newton-conjugate-gradient methods are often the gold standard for solving these inverse problems, but they require numerous actions of the Hessian on a vector. Matrix-free adjoint-based Hessian matvecs require solution of a pair of linearized forward/adjoint PDE solves per Hessian action, which may be prohibitive for large-scale inverse problems. Time invariance of the forward PDE problem leads to a block Toeplitz structure of the discretized parameter-to-observable (p2o) map defining the mapping from inputs (parameters) to outputs (observables) of the PDEs. This block Toeplitz structure enables us to exploit two key properties: (1) compact storage of the p2o map and its adjoint, and (2) efficient fast Fourier transform–based Hessian matvecs. The proposed algorithm is mapped onto large multi-GPU clusters and achieves more than 80% of peak bandwidth on NVIDIA A100 GPUs. Excellent weak scaling is shown for up to 48 A100 GPUs. For the targeted problems, the implementation executes Hessian matvecs within fractions of a second, which is orders of magnitude faster than can be achieved by conventional matrix-free Hessian matvecs via forward/adjoint PDE solves.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Analysis of Superconducting Magnet Quench Antenna Data

Quenching poses a serious problem for superconducting magnets operating at high currents. It occurs when the material transitions from the superconducting to the normal state, which leads to heating and potential damage to the magnet. To understand and mitigate quenching, the Magnet Department at Fermilab is developing and testing superconducting magnet quench antenna arrays. This study delves into the anomalous events preceding the quench during magnet training by analyzing the collected data. With the moving average and Fast Fourier Transform techniques, we investigate the trends and frequency patterns of the data. Moreover, we introduce an unsupervised anomaly detection algorithm based on Principal Component Analysis and DBSCAN clustering. It can autonomously identify events within background noise, without relying on any predefined event features. Our analysis reveals that the spatio-temporal distribution of these anomalous events has little connection to the quench location, indicating that a majority of them bear no relation to the quenching process.

43 PARTICLE ACCELERATORS↗
DOE OSTI2023

Reclassification of ASFV into 7 Biotypes Using Unsupervised Machine Learning

In 2007, an outbreak of African swine fever (ASF), a deadly disease of domestic swine and wild boar caused by the African swine fever virus (ASFV), occurred in Georgia and has since spread globally. Historically, ASFV was classified into 25 different genotypes. However, a newly proposed system recategorized all ASFV isolates into 6 genotypes exclusively using the predicted protein sequences of p72. However, ASFV has a large genome that encodes between 150–200 genes, and classifications using a single gene are insufficient and misleading, as strains encoding an identical p72 often have significant mutations in other areas of the genome. We present here a new classification of ASFV based on comparisons performed considering the entire encoded proteome. A curated database consisting of the protein sequences predicted to be encoded by 220 reannotated ASFV genomes was analyzed for similarity between homologous protein sequences. Weights were applied to the protein identity matrices and averaged to generate a genome-genome identity matrix that was then analyzed by an unsupervised machine learning algorithm, DBSCAN, to separate the genomes into distinct clusters. We conclude that all available ASFV genomes can be classified into 7 distinct biotypes.

59 BASIC BIOLOGICAL SCIENCES↗
NASA NTRS1975

Evaluation of ERTS multispectral signatures in relation to ground control signatures using a nested-sampling approach

The author has identified the following significant results. Ground measured spectral signatures of wavelength bands matching ERTS MSS were collected using a radiometer at several Californian and Nevadan sites, and directly compared with similar data from ERTS CCTs. The comparison was tested at the highest possible spatial resolution for ERTS, using deconvoluted MSS data, and contrasted with that of ground measured spectra, originally from 1 meter squares. In the mobile traverses of the grassland sites, these one meter fields of view were integrated into eighty meter transects along the five km track across four major rock/soil types. Suitable software was developed to read the MSS CCT tapes, to shadeprint individual bands with user-determined greyscale stretching. Four new algorithms for unsupervised and supervised, normalized and unnormalized clustering were developed, into a program termed STANSORT. Parallel software allowed the field data to be calibrated, and by using concurrently continuously collected, upward- and downward-viewing, 4 band radiometers, bidirectional reflectances could be calculated.

Lyon, R. J. P.↗
NASA NTRS1993

A comparison using APPL and PVM for a parallel implementation of an unstructured grid generation program

Efforts to parallelize the VGRIDSG unstructured surface grid generation program are described. The inherent parallel nature of the grid generation algorithm used in VGRIDSG was exploited on a cluster of Silicon Graphics IRIS 4D workstations using the message passing libraries Application Portable Parallel Library (APPL) and Parallel Virtual Machine (PVM). Comparisons of speed up are presented for generating the surface grid of a unit cube and a Mach 3.0 High Speed Civil Transport. It was concluded that for this application, both APPL and PVM give approximately the same performance, however, APPL is easier to use.

Arthur, Trey↗
NASA NTRS1999

Shared Memory Parallelization of an Implicit ADI-type CFD Code

A parallelization study designed for ADI-type algorithms is presented using the OpenMP specification for shared-memory multiprocessor programming. Details of optimizations specifically addressed to cache-based computer architectures are described and performance measurements for the single and multiprocessor implementation are summarized. The paper demonstrates that optimization of memory access on a cache-based computer architecture controls the performance of the computational algorithm. A hybrid MPI/OpenMP approach is proposed for clusters of shared memory machines to further enhance the parallel performance. The method is applied to develop a new LES/DNS code, named LESTool. A preliminary DNS calculation of a fully developed channel flow at a Reynolds number of 180, Re(sub tau) = 180, has shown good agreement with existing data.

Hauser, Th.↗
NASA NTRS2001

Level 1 Processing of MODIS Direct Broadcast Data at the GSFC DAAC

The GSFC DAAC is working to test and package the MODIS Level 1 Processing software for Aqua Direct Broadcast data. This entails the same code base, but different lookup tables for Aqua and Terra. However, the most significant change is the use of ancillary attitude and ephemeris files instead of orbit/attitude information within the science data stream (as with Terra). In addition, we are working on Linux: ports of the algorithms, which could eventually enable processing on PC clusters. Finally, the GSFC DAAC is also working with the GSFC Direct Readout laboratory to ingest Level 0 data from the GSFC DB antenna into the main DAAC, enabling level 1 production in near real time in support of applications users, such as the Synergy project. The mechanism developed for this could conceivably be extended to other participating stations.

Lynnes, Christopher↗
DOE OSTI2022

Profiling the BLAST bioinformatics application for load balancing on high-performance computing clusters

Abstract Background The Basic Local Alignment Search Tool (BLAST) is a suite of commonly used algorithms for identifying matches between biological sequences. The user supplies a database file and query file of sequences for BLAST to find identical sequences between the two. The typical millions of database and query sequences make BLAST computationally challenging but also well suited for parallelization on high-performance computing clusters. The efficacy of parallelization depends on the data partitioning, where the optimal data partitioning relies on an accurate performance model. In previous studies, a BLAST job was sped up by 27 times by partitioning the database and query among thousands of processor nodes. However, the optimality of the partitioning method was not studied. Unlike BLAST performance models proposed in the literature that usually have problem size and hardware configuration as the only variables, the execution time of a BLAST job is a function of database size, query size, and hardware capability. In this work, the nucleotide BLAST application BLASTN was profiled using three methods: shell-level profiling with the Unix “time” command, code-level profiling with the built-in “profiler” module, and system-level profiling with the Unix “gprof” program. The runtimes were measured for six node types, using six different database files and 15 query files, on a heterogeneous HPC cluster with 500+ nodes. The empirical measurement data were fitted with quadratic functions to develop performance models that were used to guide the data parallelization for BLASTN jobs. Results Profiling results showed that BLASTN contains more than 34,500 different functions, but a single function, RunMTBySplitDB, takes 99.12% of the total runtime. Among its 53 child functions, five core functions were identified to make up 92.12% of the overall BLASTN runtime. Based on the performance models, static load balancing algorithms can be applied to the BLASTN input data to minimize the runtime of the longest job on an HPC cluster. Four test cases being run on homogeneous and heterogeneous clusters were tested. Experiment results showed that the runtime can be reduced by 81% on a homogeneous cluster and by 20% on a heterogeneous cluster by re-distributing the workload. Discussion Optimal data partitioning can improve BLASTN’s overall runtime 5.4-fold in comparison with dividing the database and query into the same number of fragments. The proposed methodology can be used in the other applications in the BLAST+ suite or any other application as long as source code is available.

59 BASIC BIOLOGICAL SCIENCES↗
NASA NTRS1990

Computation and parallel implementation for early vision

The problem of early vision is to transform one or more retinal illuminance images-pixel arrays-to image representations built out of such primitive visual features such as edges, regions, disparities, and clusters. These transformed representations form the input to later vision stages that perform higher level vision tasks including matching and recognition. Researchers developed algorithms for: (1) edge finding in the scale space formulation; (2) correlation methods for computing matches between pairs of images; and (3) clustering of data by neural networks. These algorithms are formulated for parallel implementation of SIMD machines, such as the Massively Parallel Processor, a 128 x 128 array processor with 1024 bits of local memory per processor. For some cases, researchers can show speedups of three orders of magnitude over serial implementations.

Gualtieri, J. Anthony↗
DOE OSTI2025

Replacing non-biomedical concepts improves embedding of biomedical concepts

Embeddings are semantically meaningful representations of words in a vector space, commonly used to enhance downstream machine learning applications. Traditional biomedical embedding techniques often replace all synonymous words representing biological or medical concepts with a unique token, ensuring consistent representation and improving embedding quality. However, the potential impact of replacing non-biomedical concept synonyms has received less attention. Embedding approaches often employ concept replacement to replace concepts that span multiple words, such as non-small-cell lung carcinoma, with a single concept identifier (e.g., D002289). Also, all synonyms of each concept are merged into the same identifier. Here, we additionally leveraged WordNet to identify and replace sets of non-biomedical synonyms with their most common representatives. This combined approach aimed to reduce embedding noise from non-biomedical terms while preserving the integrity of biomedical concept representations. We applied this method to 1,055 biomedical concept sets representing molecular signatures or medical categories and assessed the mean pairwise distance of embeddings with and without non-biomedical synonym replacement. A smaller mean pairwise distance was interpreted as greater intra-cluster coherence and higher embedding quality. Embeddings were generated using the Word2Vec algorithm applied to a corpus of 10 million PubMed abstracts. Our results demonstrate that the addition of non-biomedical synonym replacement reduced the mean intra-cluster distance by an average of 8%, suggesting that this complementary approach enhances embedding quality. Future work will assess its applicability to other embedding techniques and downstream tasks. Python code implementing this method is provided under an open-source license.

algorithms↗
NASA NTRS1988

The Einstein Observatory survey of stars in the Hyades cluster region

The paper reports the results of an extensive X-ray study of the Hyades region and improves on previous studies by using refined X-ray source detection algorithms and the complete set of Einstein Observatory IPC exposures covering the Hyades cluster region (a total of 63 1 x 1 deg images). Maximum likelihood integral X-ray luminosity functions were computed for the Hyades stars for given color index (B-V) ranges. The present results agree substantially with a previous survey about the ubiquity of the stellar activity in the Hyades cluster, especially among solar-type stars. The result of Stern et al. (1981) that stellar X-ray emission is dependent on stellar age is confrimed.

Micela, G.↗
DOE OSTI2024

Buzzard to Cardinal: Improved Mock Catalogs for Large Galaxy Surveys

Abstract We present the Cardinal mock galaxy catalogs, a new version of the Buzzard simulation that has been updated to support ongoing and future cosmological surveys, including the Dark Energy Survey (DES), DESI, and LSST. These catalogs are based on a one-quarter sky simulation populated with galaxies out to a redshift of z = 2.35 to a depth of m r = 27. Compared to the Buzzard mocks, the Cardinal mocks include an updated subhalo abundance matching model that considers orphan galaxies and includes mass-dependent scatter between galaxy luminosity and halo properties. This model can simultaneously fit galaxy clustering and group–galaxy cross-correlations measured in three different luminosity threshold samples. The Cardinal mocks also feature a new color assignment model that can simultaneously fit color-dependent galaxy clustering in three different luminosity bins. We have developed an algorithm that uses photometric data to further improve the color assignment model and have also developed a novel method to improve small-scale lensing below the ray-tracing resolution. These improvements enable the Cardinal mocks to accurately reproduce the abundance of galaxy clusters and the properties of lens galaxies in the DES data. As such, these simulations will be a valuable tool for future cosmological analyses based on large sky surveys.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2022

Velocity dispersions of clusters in the Dark Energy Survey Y3 redMaPPer catalogue

ABSTRACT We measure the velocity dispersions of clusters of galaxies selected by the red-sequence Matched-filter Probabilistic Percolation (redMaPPer) algorithm in the first three years of data from the Dark Energy Survey (DES), allowing us to probe cluster selection and richness estimation, λ, in light of cluster dynamics. Our sample consists of 126 clusters with sufficient spectroscopy for individual velocity dispersion estimates. We examine the correlations between cluster velocity dispersion, richness, X-ray temperature, and luminosity, as well as central galaxy velocity offsets. The velocity dispersion–richness relation exhibits a bimodal distribution. The majority of clusters follow scaling relations between velocity dispersion, richness, and X-ray properties similar to those found for previous samples; however, there is a significant population of clusters with velocity dispersions that are high for their richness. These clusters account for roughly 22 per cent of the λ < 70 systems in our sample, but more than half (55 per cent) of λ < 70 clusters at z > 0.5. A couple of these systems are hot and X-ray bright as expected for massive clusters with richnesses that appear to have been underestimated, but most appear to have high velocity dispersions for their X-ray properties likely due to line-of-sight structure. These results suggest that projection effects contribute significantly to redMaPPer selection, particularly at higher redshifts and lower richnesses. The redMaPPer determined richnesses for the velocity dispersion outliers are consistent with their X-ray properties, but several are X-ray undetected and deeper data are needed to understand their nature.

79 ASTRONOMY AND ASTROPHYSICS↗