Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “clustering algorithm”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 613 records · Page 34
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

via machinae : Searching for stellar streams using unsupervised machine learning

ABSTRACT We develop a new machine learning algorithm, via machinae, to identify cold stellar streams in data from the Gaia telescope. via machinae is based on ANODE, a general method that uses conditional density estimation and sideband interpolation to detect local overdensities in the data in a model agnostic way. By applying ANODE to the positions, proper motions, and photometry of stars observed by Gaia, via machinae obtains a collection of those stars deemed most likely to belong to a stellar stream. We further apply an automated line-finding method based on the Hough transform to search for line-like features in patches of the sky. In this paper, we describe the via machinae algorithm in detail and demonstrate our approach on the prominent stream GD-1. Though some parts of the algorithm are tuned to increase sensitivity to cold streams, the via machinae technique itself does not rely on astrophysical assumptions, such as the potential of the Milky Way or stellar isochrones. This flexibility suggests that it may have further applications in identifying other anomalous structures within the Gaia data set, for example debris flow and globular clusters.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2020

The parallelism motifs of genomic data analysis

Genomic datasets are growing dramatically as the cost of sequencing continues to decline and small sequencing devices become available. Enormous community databases store and share these data with the research community, but some of these genomic data analysis problems require large-scale computational platforms to meet both the memory and computational requirements. These applications differ from scientific simulations that dominate the workload on high-end parallel systems today and place different requirements on programming support, software libraries and parallel architectural design. For example, they involve irregular communication patterns such as asynchronous updates to shared data structures. We consider several problems in high-performance genomics analysis, including alignment, profiling, clustering and assembly for both single genomes and metagenomes. We identify some of the common computational patterns or ‘motifs’ that help inform parallelization strategies and compare our motifs to some of the established lists, arguing that at least two key patterns, sorting and hashing, are missing. This article is part of a discussion meeting issue ‘Numerical algorithms for high-performance computational science’.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2025

Application of Machine Learning and Data Augmentation Algorithms in the Discovery of Metal Hydrides for Hydrogen Storage

The development of efficient and sustainable hydrogen storage materials is a key challenge for realizing hydrogen as a clean and flexible energy carrier. Among various options, metal hydrides offer high volumetric storage density and operational safety, yet their application is limited by thermodynamic, kinetic, and compositional constraints. In this work, we investigate the potential of machine learning (ML) to predict key thermodynamic properties—equilibrium plateau pressure, enthalpy, and entropy of hydride formation—based solely on alloy composition using Magpie-generated descriptors. We significantly expand an existing experimental dataset from ~400 to 806 entries and assess the impact of dataset size and data augmentation, using the PADRE algorithm, on model performance. Models including Support Vector Machines and Gradient Boosted Random Forests were trained and optimized via grid search and cross-validation. Results show a marked improvement in predictive accuracy with increased dataset size, while data augmentation benefits are limited to smaller datasets and do not improve accuracy in underrepresented pressure regimes. Furthermore, clustering and cross-validation analyses highlight the limited generalizability of models across different material classes, though high accuracy is achieved when training and testing within a single hydride family (e.g., AB2). The study demonstrates the viability and limitations of ML for accelerating hydride discovery, emphasizing the importance of dataset diversity and representation for robust property prediction.

augmentation↗
DOE OSTI2021

Parallelizing the Unpacking and Clustering of Detector Data for Reconstruction of Charged Particle Tracks on Multi-core CPUs and Many-core GPUs

We present results from parallelizing the unpacking and clustering steps of the raw data from the silicon strip modules for reconstruction of charged particle tracks. Throughput is further improved by concurrently processing multiple events using nested OpenMP parallelism on CPU or CUDA streams on GPU. The new implementation along with earlier work in developing a parallelized and vectorized implementation of the combinatoric Kalman filter algorithm has enabled efficient global reconstruction of the entire event on modern computer architectures. We demonstrate the performance of the new implementation on Intel Xeon and NVIDIA GPU architectures.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2022

A novel cosmic filament catalogue from SDSS data

Here, in this work, we present a new catalogue of cosmic filaments obtained from the latest Sloan Digital Sky Survey (SDSS) public data. In order to detect filaments, we implement a version of the Subspace-Constrained Mean-Shift algorithm that is boosted by machine learning techniques. This allows us to detect cosmic filaments as one-dimensional maxima in the galaxy density distribution. Our filament catalogue uses the cosmological sample of SDSS, including Data Release 16, and therefore inherits its sky footprint (aside from small border effects) and redshift coverage. In particular, this means that, taking advantage of the quasar sample, our filament reconstruction covers redshifts up to z = 2.2, making it one of the deepest filament reconstructions to our knowledge. We follow a tomographic approach and slice the galaxy data in 269 shells at different redshift. The reconstruction algorithm is applied to 2D spherical maps. The catalogue provides the position and uncertainty of each detection for each redshift slice. The quality of our detections, which we assess with several metrics, show improvement with respect to previous public catalogues obtained with similar methods. We also detect a highly significant correlation between our filament catalogue and galaxy cluster catalogues built from microwave observations of the Planck Satellite and the Atacama Cosmology Telescope.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2021

Quantum Solvers for Plane-Wave Hamiltonians: Abridging Virtual Spaces Through the Optimization of Pairwise Correlations

For many-body methods such as MCSCF and CASSCF, in which the number of one-electron orbitals is optimized and independent of the basis set used, there are no problems with using plane-wave basis sets. However, for methods currently used in quantum computing such as select configuration interaction (CI) and coupled cluster (CC) methods, it is necessary to have a virtual space that is able to capture a significant amount of electron-electron correlation in the system. The virtual orbitals in a pseudopotential plane-wave Hartree–Fock calculation, because of Coulomb repulsion, are often scattering states that interact very weakly with the filled orbitals. As a result, very little correlation energy is captured from them. The use of virtual spaces derived from the one-electron operators has also been tried, and while some correlations are captured, the amount is quite low. To overcome these limitations, we have been developing new classes of algorithms to define virtual spaces by optimizing orbitals from small pairwise CI Hamiltonians, which we term as correlation optimized virtual orbitals with the abbreviation COVOs. With these procedures, we have been able to derive virtual spaces, containing only a few orbitals, which are able to capture a significant amount of correlation. The focus in this manuscript is on using these derived basis sets to target full CI (FCI) quality results for H2 on near-term quantum computers. However, the initial results for this approach were promising. We were able to obtain good agreement with FCI/cc-pVTZ results for this system with just 4 virtual orbitals, using both FCI and quantum simulations. The quality of the results using COVOs suggests that it may be possible to use them in other many-body approaches, including coupled cluster and Møller–Plesset perturbation theories, and open up the door to many-body calculations for pseudopotential plane-wave basis set methods.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2023

Solving the nuclear pairing model with neural network quantum states

In this work, we present a variational Monte Carlo method that solves the nuclear many-body problem in the occupation number formalism exploiting an artificial neural network representation of the groundstate wave function. A memory-efficient version of the stochastic reconfiguration algorithm is developed to train the network by minimizing the expectation value of the Hamiltonian. We benchmark this approach against widely used nuclear many-body methods by solving a model used to describe pairing in nuclei for different types of interaction and different values of the interaction strength. Despite its polynomial computational cost, our method outperforms coupled-cluster and provides energies that are in excellent agreement with the numerically-exact full configuration interaction values.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2026

NuLattice: Ab initio computations of atomic nuclei on lattices

Here, we introduce NuLattice, a Python software package for ab initio computations of atomic nuclei on lattices. The computational tools consist of Hartree Fock, the coupled cluster method, the in-medium similarity renormalization group, and full configuration interaction. At present, the employed interactions are from pion-less effective field theory at leading order and consist of two-body and three-body contacts. We present results for light nuclei 2 H, 3,4 He, 8 Be, 12 C, and 16 O. NuLattice algorithms exploit the sparsity and locality of lattice interactions, and as a result computations can be run on laptops.

Rothman, Maxwell [Univ. of Tennessee, Knoxville, T↗
DOE OSTI2020

Efficient Step-Merged Quantum Imaginary Time Evolution Algorithm for Quantum Chemistry

In this work, we develop a resource-efficient step-merged quantum imaginary time evolution approach (smQITE) to solve for the ground state of a Hamiltonian on quantum computers. This heuristic method features a fixed shallow quantum circuit depth along the state evolution path. We use this algorithm to determine the binding energy curves of a set of molecules, including H 2 , H 4 , H 6 , LiH, HF, H 2 O, and BeH 2 , and find highly accurate results. The required quantum resources of smQITE calculations can be further reduced by adopting the circuit form of the variational quantum eigensolver (VQE) technique, such as the unitary coupled cluster ansatz. We demonstrate that smQITE achieves a similar computational accuracy as VQE at the same fixed-circuit ansatz, without requiring a generally complicated high-dimensional nonconvex optimization. Finally, smQITE calculations are carried out on Rigetti quantum processing units, demonstrating that the approach is readily applicable on current noisy intermediate-scale quantum devices.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2020

ConnectIt: a framework for static and incremental parallel graph connectivity algorithms

Connected components is a fundamental kernel in graph applications. The fastest existing multicore algorithms for solving graph connectivity are based on some form of edge sampling and/or linking and compressing trees. However, many combinations of these design choices have been left unexplored. In this paper, we design the ConnectIt framework, which provides different sampling strategies as well as various tree linking and compression schemes. ConnectIt enables us to obtain several hundred new variants of connectivity algorithms, most of which extend to computing spanning forest. In addition to static graphs, we also extend ConnectIt to support mixes of insertions and connectivity queries in the concurrent setting. We present an experimental evaluation of ConnectIt on a 72-core machine, which we believe is the most comprehensive evaluation of parallel connectivity algorithms to date. Compared to a collection of state-of-the-art static multicore algorithms, we obtain an average speedup of 12.4x (2.36x average speedup over the fastest existing implementation for each graph). Using ConnectIt, we are able to compute connectivity on the largest publicly-available graph (with over 3.5 billion vertices and 128 billion edges) in under 10 seconds using a 72-core machine, providing a 3.1x speedup over the fastest existing connectivity result for this graph, in any computational setting. For our incremental algorithms, we show that our algorithms can ingest graph updates at up to several billion edges per second. To guide the user in selecting the best variants in ConnectIt for different situations, we provide a detailed analysis of the different strategies. Finally, we show how the techniques in ConnectIt can be used to speed up two important graph applications: approximate minimum spanning forest and SCAN clustering.

Computer Science↗
DOE OSTI2020

Automated Coupling of Nanodroplet Sample Preparation with Liquid Chromatography–Mass Spectrometry for High-Throughput Single-Cell Proteomics

Single-cell proteomics can provide critical biological insight into the cellular heterogeneity that is masked by bulk-scale analysis. Here, we have developed a nanoPOTS (nanodroplet processing in one pot for trace samples) platform and demonstrated its broad applicability for single-cell proteomics. However, because of nanoliter-scale sample volumes, the nanoPOTS platform is not compatible with automated LC-MS systems, which significantly limits sample throughput and robustness. To address this challenge, we have developed a nanoPOTS autosampler allowing fully automated sample injection from nanowells to LC-MS systems. We also developed a sample drying, extraction, and loading workflow to enable reproducible and reliable sample injection. The sequential analysis of 20 samples containing 10 ng tryptic peptides demonstrated high reproducibility with correlation coefficients of >0.995 between any two samples. The nanoPOTS autosampler can provide analysis throughput of 9.6, 16, and 24 single cells per day using 120, 60, and 30 min LC gradients, respectively. As a demonstration for single-cell proteomics, the autosampler was first applied to profiling protein expression in single MCF10A cells using a label-free approach. At a throughput of 24 single cells per day, an average of 256 proteins was identified from each cell and the number was increased to 731 when the Match Between Runs algorithm of MaxQuant was used. Using a multiplexed isobaric labeling approach (TMT-11plex), ~77 single cells could be analyzed per day. We analyzed 152 cells from three acute myeloid leukemia cell lines, resulting in a total of 2558 identified proteins with 1465 proteins quantifiable (70% valid values) across the 152 cells. These data showed quantitative single-cell proteomics can cluster cells to distinct groups and reveal functionally distinct differences.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2024

Correlated pair ansatz with a binary tree structure

We develop an efficient algorithm to implement the recently introduced binary tree state (BTS) ansatz on a classical computer. BTS allows a simple approximation to permanents arising from the computationally intractable antisymmetric product of interacting geminals and respects size-consistency. We show how to compute BTS overlap and reduced density matrices efficiently. We also explore two routes for developing correlated BTS approaches: Jastrow coupled cluster on BTS and linear combinations of BT states. Here, the resulting methods show great promise in benchmark applications to the reduced Bardeen–Cooper–Schrieffer Hamiltonian and the one-dimensional XXZ Heisenberg Hamiltonian.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2020

Resilient Autonomous Wind Farms: Preprint

With the advent of an increasing number of control strategies that seek to optimize wind turbine performance on a farm-level, taking account of individual wind turbine information to achieve wind farm-level objectives has become an increasingly important goal. Methods for controlling wind turbines on an individual and farm level have seen significant development, and an abundance of new implementations for gathering and using data from turbines have created potential for novel control mechanisms which can further optimize the performance and delivery characteristics of a wind farm. A key element of making these wind farms more efficient is to develop reliable algorithms that use local sensor information that is already being collected, such as supervisory control and data acquisition (SCADA) data, local meteorological stations, and nearby radars/sodars/lidars. Making use of information from all wind turbines in a wind farm can enable such approaches as determining the atmospheric conditions across the farm, improving fault-finding, and enabling more efficient overall control of farm-wide optimizations through mechanisms such as wake-steering. However, these approaches typically involve a centralized communications and control center. In order to ensure the resilient operation of the farm, it is necessary to develop an approach which distributes the calculation and communication amongst multiple nodes throughout the farm. In this fashion, a redundant, robust, and secure network can be created, which can tolerate faults in calculation, communication, and even external attacks which seek to disrupt the operation of the wind farm. This paper introduces the use of the Raft Byzantine Fault Tolerance algorithm in the implementation of autonomous control of a wind farm. This implementation will allow for fault tolerance for malfunctioning nodes, sensors, transmitters, and connectors. This approach is equally extensible to account for malicious actors. It will be shown to achieve overall consensus, provided the number of faults/malicious nodes is less than 3$n$+1, where $n$ is the number of turbine cluster faults which may occur, and to be robust in the face of multiple arbitrary faults.

autonomous↗
DOE OSTI2021

A Co-design Framework for Online Data Analysis and Reduction

Science applications preparing for the exascale era are increasingly exploring in situ computations comprising of simulation-analysis-reduction pipelines coupled in-memory. Efficient composition and execution of such complex pipelines for a target platform is a codesign process that evaluates the impact and tradeoffs of various application- and system-specific parameters. In this article, we describe a toolset for automating performance studies of composed HPC applications that perform online data reduction and analysis. We describe Cheetah, a new framework for composing parametric studies on coupled applications, and Savanna, a runtime engine for orchestrating and executing campaigns of codesign experiments. Furthermore, this toolset facilitates understanding the impact of various factors such as process placement, synchronicity of algorithms, and storage versus compute requirements for online analysis of large data. Ultimately, we aim to create a catalog of performance results that can help scientists understand tradeoffs when designing next-generation simulations that make use of online processing techniques. We illustrate the design of Cheetah and Savanna, and present application examples that use this framework to conduct codesign studies on small clusters as well as leadership class supercomputers.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Efficient Parameterization of Density Functional Tight-Binding for 5 f -Elements: A Th–O Case Study

Density functional tight binding (DFTB) models for f-element species are challenging to parametrize owing to the large number of adjustable parameters. The explicit optimization of the terms entering the semiempirical DFTB Hamiltonian related to f orbitals is crucial to generating a reliable parametrization for f-block elements, because they play import roles in bonding interactions. However, since the number of parameters grows quadratically with the number of orbitals, the computational cost for parameter optimization is much more expensive for the f-elements than for the main group elements. In this work we present a set of efficient approaches for mitigating the hurdle imposed by the large size of the parameter space. A novel group-by-orbital correction functions for two-center bond integrals was developed. With this approach the number of parameters is reduced, and it grows linearly with the number of elements, maintaining the accuracy and the number of parameters, in the case of f elements, by more than 40%. The parameter optimization step was accelerated by means of the mini-batch BFGS method. This method allows parameter optimizations with much larger training sets than other single batch methods. A stochastic optimizer was employed that helped overcome shallow local minima in the objective function. The proposed algorithm was used to parametrize the DFTB Hamiltonian for the Th–O system, which was subsequently applied to the study of ThO 2 nanoparticles. The training set consisted of 6322 unique structures, which is barely feasible with conventional optimization methods. The optimized parameter set, LANL-ThO, displays good agreement with DFT-calculated properties such as energies, forces, and structures for both clusters and bulk ThO 2 . Benefiting from the fewer number of parameters and lower computational costs for objective function evaluations, this new approach shows its potential applications in DFTB parametrization for elements with high angular momentum, which present a challenge to conventional methods.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2021

Peculiar velocity estimation from kinetic SZ effect using deep neural networks

ABSTRACT The Sunyaev–Zel’dolvich (SZ) effect is expected to be instrumental in measuring velocities of distant clusters in near future telescope surveys. We simplify the calculation of peculiar velocities of galaxy clusters using deep learning frameworks trained on numerical simulations to avoid the independent estimation of the optical depth. Images of distorted photon backgrounds are generated for idealized observations using one of the largest cosmological hydrodynamical simulations, the Magneticum simulations. The model is tested to determine its ability of estimating peculiar velocities from future kinetic SZ observations under different noise conditions. The deep learning algorithm displays robustness in estimating peculiar velocities from kinetic SZ effect by an improvement in accuracy of about 17 per cent compared to the analytical approach.

Wang, Yuyu (ORCID:0000000202458547)↗
DOE OSTI2024

Generalized representative structures for atomistic systems

A new method is presented to generate atomic structures that reproduce the essential characteristics of arbitrary material systems, phases, or ensembles. Previous methods allow one to reproduce the essential characteristics (e.g. the chemical disorder) of a large random alloy within a small crystal structure. The ability to generate small representations of random alloys, along with the restriction to crystal systems, results from using the fixed-lattice cluster correlations to describe structural characteristics. A more general description of the structural characteristics of atomic systems is obtained using complete sets of atomic environment descriptors. These are used within for generating representative atomic structures without restriction to fixed lattices. A general data-driven approach is provided here utilizing the atomic cluster expansion (ACE) basis. The N-body ACE descriptors are a complete set of atomic environment descriptors that span both chemical and spatial degrees of freedom and are used within for describing atomic structures. The generalized representative structure (GRS) method presented within generates small atomic structures that reproduce ACE descriptor distributions corresponding to arbitrary structural and chemical complexity. It is shown that systematically improvable representations of crystalline systems on fixed parent lattices, amorphous materials, liquids, and ensembles of atomic structures may be produced efficiently through optimization algorithms. With the GRS method, we highlight reduced representations of atomistic machine-learning training datasets that contain similar amounts of information and small 40–72 atom representations of liquid phases. The ability to use GRS methodology as a driver for informed novel structure generation is also demonstrated. The advantages over other data-driven methods and state-of-the-art methods restricted to high-symmetry systems are highlighted.

atomic cluster expansion↗