Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “clustering algorithm”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 199 records · Page 11
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Learning From User Behavior: A Survey-Assist Algorithm for Longitudinal Mobility Data Collection

GPS-based travel surveys are widely used in mobility studies to gather crucial qualitative data, like purpose, transportation mode and replaced mode. However, survey response still poses a burden to users, especially in long-term mobility studies, leading to response fatigue. We explore a survey-assist strategy to ease this burden by a novel, user-level modeling approach that leverages past responses from each user to predict responses for new trips, without relying on external data sources like GIS data. We investigate three main algorithms for predicting responses: (i) clustering trips and extrapolating responses for similar trips, (ii) using random forest classification, and (iii) clustering that uses a hybrid algorithm to determine spatial structure, which is then fed as input to a classic random forest classifier. The clustering approach can flexibly predict responses for even complex qualitative survey questions; it achieved F-scores of 65%. The random forest pipeline uses architecture that restricts it to predicting three predetermined survey questions: trip purpose, mode, and replaced mode. However, it achieved F-scores of 78%. While the survey-assist approach has been implemented by several proprietary systems, to our knowledge, this is the first exploration in the academic literature. It follows that this is also the first rigorous evaluation of multiple algorithms that can implement the approach. The evaluation uses a large scale, publicly available, longitudinal dataset consisting of ~ 92k trips from 235 users over a period of roughly one and a half years. With this approach, travel surveys can be pre-filled with the predicted responses for each trip, thus streamlining the survey process for users. Combined with an active learning system that requests user input on low-confidence predictions, models can be updated and improved over time to better support the long-term collection of longitudinal qualitative data.

clustering↗
DOE OSTI2023

Efficient Application of the Factorized form of the Unitary Coupled-Cluster Ansatz for the Variational Quantum Eigensolver Algorithm by Using Linear Combination of Unitaries

The variational quantum eigensolver is one of the most promising algorithms for near-term quantum computers. It has the potential to solve quantum chemistry problems involving strongly correlated electrons with relatively low-depth circuits, which are otherwise difficult to solve on classical computers. The variational eigenstate is constructed from a number of factorized unitary coupled-cluster terms applied onto an initial (single-reference) state. Current algorithms for applying one of these operators to a quantum state require a number of operations that scale exponentially with the rank of the operator. We exploit a hidden SU(2) symmetry to allow us to employ the linear combination of unitaries approach, Our Prepare subroutine uses n+2 ancilla qubits for a rank-n operator. Our Select (U) scheme uses O(n) Cnot gates. This results in a full algorithm that scales like the cube of the rank of the operator n 3 , a significant reduction in complexity for rank five or higher operators. This approach, when combined with other algorithms for lower-rank operators (when compared to the standard implementation), will make the factorized form of the unitary coupled-cluster approach much more efficient to implement on all types of quantum computers.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Continuous momentum dependence in the dynamical cluster approximation

The dynamical cluster approximation (DCA) is a quantum cluster extension to the single-site dynamical mean-field theory that incorporates spatially nonlocal dynamic correlations systematically and nonperturbatively. The DCA + algorithm addresses the cluster shape dependence of the DCA and improves the convergence with cluster size by introducing a lattice self-energy with continuous momentum dependence. However, we show that the DCA + algorithm is plagued by a fundamental problem when its self-consistency equations are formulated using the bare Green's function of the cluster. This problem is most severe in the strongly correlated regime at low doping, where the DCA + self-energy becomes overly metallic and local, and persists to cluster sizes where the standard DCA has long converged. In view of the failure of the DCA + algorithm, we propose to complement DCA simulations with a post-interpolation procedure for single-particle and two-particle correlation functions to preserve continuous momentum dependence and the associated benefits in the DCA. We demonstrate the effectiveness of this practical approach with results for the half-filled and hole-doped two-dimensional Hubbard model.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
DOE OSTI2021

Scalable, In-situ Data Clustering Data Analysis for Extreme Scale Scientific Computing (Final Report)

The objective of this project is to address challenges in the design and development of scalable in-situ data clustering and analytics algorithms and software. Our goal is to develop parallel software consisting of a set of spatio-temporal data clustering and anomaly detection functions, both of which are very important for large-scale analysis and have wide applicability for in-situ runs as well as post-processing analysis. Our design principles for in-situ analysis consider the following: (1) identify parts of the computation can be done close to the data within the nodes, while it is still in memory; (2) extract analysis components can (and should) be performed in remote staging and analysis nodes; (3) develop error-bound approximation methods for applications tolerable for small errors; (4) identify the type of derived distributions and statistics, for spatio-temporal data, that can be kept locally in order to both accelerate computations and meet energy constraints in subsequent iterations and phases; (5) use a self-describing data format so that data can be consistent and understood among local storage (memory and SSDs) and at staging and analysis nodes, thereby providing portability and flexibility; (6) develop service-oriented functions that can schedule in-situ and post-hoc analysis tasks based on the dynamic requirements of applications. Our development focus is to produce the parallel data analysis software/library that will be scalable, reusable, extensible, and generic for applications in different disciplines. The software will be able to run in-situ with the simulations as well as post-hoc analysis. This approach will satisfy many synergistic requirements for data intensive applications executed on data coming from instruments and experiments. In particular, the proposed multilevel approach is directly applicable to perform design tradeoffs for running part of the algorithms near the instruments and the rest on remote (analysis) systems.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Coupled Cluster Downfolding Theory: towards universal many-body algorithms for dimensionality reduction of composite quantum systems in chemistry and materials science

Abstract The recently introduced coupled cluster (CC) downfolding techniques for reducing the dimensionality of quantum many-body problems recast the CC formalism in the form of the renormalization procedure allowing, for the construction of effective (or downfolded) Hamiltonians in small-dimensionality sub-space, usually identified with the so-called active space, of the entire Hilbert space. The resulting downfolded Hamiltonians integrate out the external (out-of-active-space) Fermionic degrees of freedom from the internal (in-the-active-space) parameters of the wave function, which can be determined as components of the eigenvectors of the downfolded Hamiltonians in the active space. This paper will discuss the extension of non-Hermitian (associated with standard CC formulations) and Hermitian (associated with the unitary CC approaches) downfolding formulations to composite quantum systems commonly encountered in materials science and chemistry. The non-Hermitian formulation can provide a platform for developing local CC approaches, while the Hermitian one can serve as an ideal foundation for developing various quantum computing applications based on the limited quantum resources. We also discuss the algorithm for extracting the semi-analytical form of the inter-electron interactions in the active spaces.

Bauman, Nicholas P.↗
DOE OSTI2024

Search for interacting galaxy clusters from SDSS DR-17 employing optimized friends-of-friends algorithm and multimessenger tracers

ABSTRACT In the theoretical framework of hierarchical structure formation, galaxy clusters evolve through continuous accretion and mergers of substructures. Cosmological simulations have revealed the best picture of the universe as a 3D filamentary network of dark-matter distribution called the cosmic web. Galaxy clusters are found to form at the nodes of this network and are the regions of high merging activity. Such mergers being highly energetic, contain a wealth of information about the dynamical evolution of structures in the Universe. Observational validation of this scenario needs a colossal effort to identify numerous events from all-sky surveys. Therefore, such efforts are sparse in literature and tend to focus on individual systems. In this work, we present an improved search algorithm for identifying interacting galaxy clusters and have successfully produced a comprehensive list of systems from SDSS DR-17. By proposing a set of physically motivated criteria, we classified these interacting clusters into two broad classes, ‘merging’ and ‘pre-merging/postmerging’ systems. Interestingly, as predicted by simulations, we found that most cases show cluster interaction along the prominent cosmic filaments of galaxy distribution (i.e. the proxy for dark matter filaments), with the most violent ones at their nodes. Moreover, we traced the imprint of interactions through multiband signatures, such as diffuse cluster emissions in radio or X-rays. Although we could not find direct evidence of diffuse emission from connecting filaments and ridges; our catalogue of interacting clusters will ease locating such faintest emissions as data from sensitive telescopes such as eROSITA or SKA, becomes accessible.

Oak, Tejas↗
DOE OSTI2021

The HectoMAP Cluster Survey: Spectroscopically Identified Clusters and their Brightest Cluster Galaxies (BCGs)

We apply a friends-of-friends (FoF) algorithm to identify galaxy clusters and we use the catalog to explore the evolutionary synergy between brightest cluster galaxies (BCGs) and their host clusters. We base the cluster catalog on the dense HectoMAP redshift survey (2000 redshifts deg -2 ). The HectoMAP FoF catalog includes 346 clusters with 10 or more spectroscopic members within the range 0.05 < z < 0.55 and with a median z = 0.29. We list these clusters and their members. We also include central velocity dispersions (σ*, BCG ) for the FoF cluster BCGs, a distinctive feature of the HectoMAP FoF catalog. HectoMAP clusters with higher galaxy number density (80 systems) are all genuine clusters with a strong concentration and a prominent BCG in Subaru/Hyper Suprime-Cam images. The phase-space diagrams show the expected elongation along the line of sight. Lower-density systems include some low reliability systems. We establish a connection between BCGs and their host clusters by demonstrating that σ*, BCG /σ cl decreases as a function of cluster velocity dispersion (σ cl ), in contrast, numerical simulations predict a constant σ*, BCG /σ cl . Sets of clusters at two different redshifts show that BCG evolution in massive systems is slow over the redshift range z < 0.4. The data strongly suggest that minor mergers may play an important role in BCG evolution in clusters with σ cl ≳ 300 km s -1 . For lower mass systems (σ cl < 300 km s -1 ), major mergers may play a significant role. The coordinated evolution of BCGs and their host clusters provides an interesting test of simulations in high-density regions of the universe.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2022

High performance FPGA embedded system for machine learning based tracking and trigger in sPhenix and EIC

We present a comprehensive end-to-end pipeline to classify triggers versus background events in this paper. This pipeline makes online decisions to select signal data and enables the intelligent trigger system for efficient data collection in the Data Acquisition System (DAQ) of the upcoming sPHENIX and future EIC (Electron-Ion Collider) experiments. Starting from the coordinates of pixel hits that are lightened by passing particles in the detector, the pipeline applies three-stage of event processing (hits clustering, track reconstruction, and trigger detection) and labels all processed events with the binary tag of trigger versus background events. The pipeline consists of deterministic algorithms such as clustering pixels to reduce event size, tracking reconstruction to predict candidate edges, and advanced graph neural network-based models for recognizing the entire jet pattern. In particular, we apply the message-passing graph neural network to predict links between hits and reconstruct tracks and a hierarchical pooling algorithm (DiffPool) to make the graph-level trigger detection. We obtain an impressive performance (≥70% accuracy) for trigger detection with only 3200 neuron weights in the end-to-end pipeline. We deploy the end-to-end pipeline into a field-programmable gate array (FPGA) and accelerate the three stages with speedup factors of 1152, 280, and 21, respectively.

Instruments & Instrumentation↗
DOE OSTI2021

Analyzing acoustic emission data to identify cracking modes in cement paste using an artificial neural network

This research is focused on the identification of cracking mechanisms for cement paste using acoustic emission data, recorded from compression and notched four-point bending tests. A procedure is developed for analyzing the data by employing an agglomerative hierarchical clustering method, an artificial neural network, and a ray-tracing source location algorithm. An agglomerative hierarchical clustering method is utilized to cluster the AE data from a compression test using frequency-dependent features. A neural network is trained using the compression test data and applied to the AE data emitted during the four-point bending test. The clustered data from the four-point bending test is localized using a ray-tracing algorithm. Based on the occurrence and locations of the clustered events and signal feature analyses, potential cracking mechanisms are identified and assigned.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Implementation of Relativistic Coupled Cluster Theory for Massively Parallel GPU-Accelerated Computing Architectures

In this paper, we report reimplementation of the core algorithms of relativistic coupled cluster theory aimed at modern heterogeneous high-performance computational infrastructures. The code is designed for parallel execution on many compute nodes with optional GPU coprocessing, accomplished via the new ExaTENSOR back end. The resulting ExaCorr module is primarily intended for calculations of molecules with one or more heavy elements, as relativistic effects on the electronic structure are included from the outset. In the current work, we thereby focus on exact two-component methods and demonstrate the accuracy and performance of the software. The module can be used as a stand-alone program requiring a set of molecular orbital coefficients as the starting point, but it is also interfaced to the DIRAC program that can be used to generate these. We therefore also briefly discuss an improvement of the parallel computing aspects of the relativistic self-consistent field algorithm of the DIRAC program.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2021

Improving the Accuracy of Clustering Electric Utility Net Load Data using Dynamic Time Warping

Identifying patterns in electric utility net load data in a time-series format is very useful in preparing the operation for next day. Machine learning algorithms have been used in other domains and those concepts are applied in this paper on real-world net load measurement data. Clustering is the practice of grouping data with similar characteristics as determined by the distance measure. The K-means clustering algorithm is utilized here with actual electric utility data. The paper uses the standard distance measure, Euclidean distance (ED), and compares its performance against the dynamic time warping (DTW) measure. An actual case study with real data is presented, and DTW distance measure-based method observed to result better accuracy compared to the ED based method for substation net load measurements predominantly with residential customers.

clustering↗
DOE OSTI2022

Toward Discord : Code for Simulating Continuous Spin Systems

A new computational tool to simulate classical spin systems with frustrated crystal structures is presented. Complementary single- and cluster-spin flip algorithms are implemented to calculate the diffuse scattering patterns, spin-pair correlations, and thermodynamic quantities. Test cases of geometrically frustrated kagome, pyrochlore, and cubic systems are detailed. Two recent scientific cases are also shown here. This new method, together with recent developments of the rmc-discord package (https://github.com/zjmorgan/rmc-discord), represent integrated and strategic step in a complete forward and reverse Monte Carlo framework discord.

36 MATERIALS SCIENCE↗
DOE OSTI2023

Selecting representative geological realizations to model subsurface CO 2 storage under uncertainty

Carbon capture and storage (CCS) is one of the quickest and most effective solutions for reducing carbon emissions. The majority of subsurface storage occurs in saline aquifers, for which geological information is lacking which in turn results in geological uncertainty. To evaluate uncertainty in CO 2 injection projections, the use of multiple geological realizations (GRs) has been practiced very commonly. In this approach, hundreds or thousands of high-resolution GRs is used that quickly becomes computationally expensive. This issue can be addressed with representative geological realizations (RGRs) that preserve the uncertainty domain of the ensemble GRs. Here, in this study, we propose the use of unsupervised machine learning (UML) frameworks, including dissimilarity measurement, dimensionality reduction, clustering and sampling algorithms ta select a predetermined number of RGRs. We compare the simulation outputs of the RGR sets and the ensemble using the Kolmogorov–Smirnov (KS) test to select the best UML. The UML frameworks and their associated selection processes are evaluated using a saline aquifer with a single CO 2 injection well and 200 GRs with varying uncertain petrophysical characteristics. The best UML framework is selected to use only 5% of the GRs while maintaining the uncertainty domain of the ensemble GRs. In addition, the best UML framework is tested using a saline aquifer with three CO 2 injection wells and varied GRs. The results show that our proposed UML framework can be used to choose RGRs, capturing the whole uncertainty domain. Our approach leads to a significant reduction in the computational cost associated with scenario testing, decision-making, and development planning for CO 2 storage sites under geological uncertainty.

58 GEOSCIENCES↗
DOE OSTI2020

Landfalling Droughts: Global Tracking of Moisture Deficits From the Oceans Onto Land

Abstract Droughts threaten food, energy, and water security, causing death and displacement of millions of people and billions of dollars in damages. However, there are still important gaps in the understanding of drought mechanisms and behaviors, inhibiting the accuracy of early‐warning systems designed to protect communities worldwide. We use an object‐tracking algorithm to track clusters of precipitation‐minus‐evaporation moisture deficits across land and ocean areas of the globe from 1981–2018. This analysis reveals a new type of “landfalling drought” that originates over the ocean and “migrates” onto land. We find that 16% of droughts that affected the continents worldwide from 1981–2018 were landfalling droughts. These droughts were significantly larger (220–425%) and more intense (4–30%)—and grew (253–285%) and intensified (9–28%) faster—than droughts that developed solely over the land or ocean. To identify potential underlying mechanisms, we analyze moisture transport associated with landfalling droughts over western North America. We find that landfalling droughts in this region are associated with anomalously anticyclonic atmospheric pressure patterns that reduce moisture fluxes over the Pacific Ocean toward the continent. By advancing understanding of the spatiotemporal evolution of droughts, our findings offer the potential to improve seasonal‐scale prediction and long‐term projection of global drought risks.

Herrera‐Estrada, Julio E.↗
DOE OSTI2023

DeepAstroUDA: semi-supervised universal domain adaptation for cross-survey galaxy morphology classification and anomaly detection

Abstract Artificial intelligence methods show great promise in increasing the quality and speed of work with large astronomical datasets, but the high complexity of these methods leads to the extraction of dataset-specific, non-robust features. Therefore, such methods do not generalize well across multiple datasets. We present a universal domain adaptation method, DeepAstroUDA , as an approach to overcome this challenge. This algorithm performs semi-supervised domain adaptation (DA) and can be applied to datasets with different data distributions and class overlaps. Non-overlapping classes can be present in any of the two datasets (the labeled source domain, or the unlabeled target domain), and the method can even be used in the presence of unknown classes. We apply our method to three examples of galaxy morphology classification tasks of different complexities (three-class and ten-class problems), with anomaly detection: (1) datasets created after different numbers of observing years from a single survey (Legacy Survey of Space and Time mock data of one and ten years of observations); (2) data from different surveys (Sloan Digital Sky Survey (SDSS) and DECaLS); and (3) data from observing fields with different depths within one survey (wide field and Stripe 82 deep field of SDSS). For the first time, we demonstrate the successful use of DA between very discrepant observational datasets. DeepAstroUDA is capable of bridging the gap between two astronomical surveys, increasing classification accuracy in both domains (up to 40 % on the unlabeled data), and making model performance consistent across datasets. Furthermore, our method also performs well as an anomaly detection algorithm and successfully clusters unknown class samples even in the unlabeled target dataset.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2020

Using modularity to segment binary code

We consider the problem of recovering program structure from compiled binary code. We first extract the call graph and layout of functions in memory from the compiled code and represent this information in a graphical format. We then employ Louvain's modularity algorithm to identify clusters of functions that are considered to be related. We find that the quality and properties of clusters extracted by our technique are greatly impacted by the relative importance we assign to the call graph and the ordering of functions in memory.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

A VXS [VITA41] Trigger Processor for the 12GEV Experimental Programs at Jefferson Lab

The VXS_Trigger_Processor [VTP] was developed and commissioned for CLAS12 in the fall of 2016. This board is a VITA41 switch card and it collects data from a variety of front-end TDC and Flash ADC modules. The VTP has since been used in several experiments at Jefferson Lab serving as the L1 trigger module for a variety of detector types, such as stacked calorimeters, strip calorimeters, time-of-flight, Cerenkov, hodoscopes, drift chambers, and silicon strips. Trigger algorithms implemented include cluster finding (1D, 2D), drift chamber segment and road finding, geometry matching between various detectors, particle counting, and general global trigger bit processing. The VTP is also capable of reading out each front-end crate with up to 40Gbps Ethernet which is an enormous increase compared to the currently used 200MB/s VME bus. Recent progress has been made to show that a firmware and software upgrade can enable existing Jefferson Lab front-end crates to operate in a streaming DAQ mode. In February 2020, tests will be performed on a full calorimeter and matched hodoscope which are components of the CLAS12 Forward Tagger detector system with beam in Hall B. This paper details the hardware performance, triggered, and streaming applications that have been implemented using the VTP for several experiments at Jefferson Lab.

ABBOTT, David↗