Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “clustering algorithm”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 505 records · Page 28
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

DESI peculiar velocity survey – Fundamental Plane

The Dark Energy Spectroscopic Instrument (DESI) peculiar velocity survey aims to measure the peculiar velocities of early- and late-type galaxies within the DESI footprint using both the Fundamental Plane and optical Tully–Fisher relations. Direct measurements of peculiar velocities can significantly improve constraints on the growth rate of structure, reducing uncertainty by a factor of approximately 2.5 at redshift 0.1 compared to the DESI Bright Galaxy Survey’s redshift space distortion measurements alone. We assess the quality of stellar velocity dispersion measurements from DESI spectroscopic data. These measurements, along with photometric data from the Legacy Survey, establish the Fundamental Plane relation and determine distances and peculiar velocities of early-type galaxies. During survey validation, we obtain spectra for 6698 unique early-type galaxies, up to a photometric redshift of 0.15. 64 per cent of observed galaxies (4267) have relative velocity dispersion errors below 10 per cent. This percentage increases to 75 per cent if we restrict our sample to galaxies with spectroscopic redshifts below 0.1. We use the measured central velocity dispersion, along with photometry from the DESI Legacy Imaging Surveys, to fit the Fundamental Plane parameters using a 3D Gaussian maximum likelihood algorithm that accounts for measurement uncertainties and selection cuts. In addition, we conduct zero-point calibration using the absolute distance measurements to the Coma cluster, leading to a value of the Hubble constant, H 0 = 76.05 ± 0.35 (statistical) ±0.49 (systematic Fundamental Plane) ±4.86 (statistical due to calibration) km s –1 Mpc –1 ⁠. This H 0 value is within 2σ of Planck cosmic microwave background results and within 1σ of other low-redshift distance indicator-based measurements.

cosmological parameters↗
DOE OSTI2025

ICAT: The Interactive Corpus Analysis Tool

The Interactive Corpus Analysis Tool (ICAT) is a Python library for creating dashboards to explore textual datasets and build simple binary classification models to help filter through them and focus on entries of interest. This tool uses a form of interactive machine learning (IML), a paradigm of “machine teaching” (Simard et al., 2017) that sits at the intersection of the fields of human computer interaction (HCI), visual analytics, and machine learning. The intent of ICAT is to allow subject matter experts (SME) with limited to no experience in machine learning to benefit from an iterative human-in-the-loop (HITL) approach to building their own model without needing to understand the details of the underlying algorithm. This interactivity is achieved by allowing the user to create features, label data points, and visually manipulate a representation of the features to manually cluster and investigate data, while a model is trained on the fly based on these actions. ICAT is built on top of the Panel (Holoviz, 2018) library, using a combination of Vega, a custom IPyWidget using D3, and ipyvuetify, and is intended to be used inside of a Jupyter environment.

Martindale, Nathan [Oak Ridge National Laboratory ↗
DOE OSTI2021

Productive Programming of Distributed Systems with the SHAD C++ Library

High-performance computing (HPC) is often perceived as a matter of making large-scale systems (e.g., clusters) run as fast as possible, regardless the required programming effort. However, the idea of "bringing HPC to the masses" has recently emerged. Inspired by this vision, we have designed SHAD, the Scalable High-performance Algorithms and Data-structures library. SHAD is open source software, written in C++, for C++ developers. Unlike other HPC libraries for distributed systems, which rely on SPMD models, SHAD adopts a shared-memory programming abstraction, to make C++ programmers feel at home. Underneath, SHAD manages tasking and data-movements, moving the computation where data resides and taking advantage of asynchrony to tolerate network latency. At the bottom of his stack, SHAD can interface with multiple runtime systems: this not only improves developer’s productivity, by hiding the complexity of such software and of the underlying hardware, but also greatly enhance code portability. Thanks to its abstraction layers, SHAD can indeed target different systems, ranging from laptops to HPC clusters, without any need for modifying the user-level code. We have prototyped and open-sourced the implementation of (a subset of) the C++ standard library (STL) targeting multi-node HPC clusters. Our work allows plain STL-based C++ code to scale on HPC systems, with no need for rewriting the code to exploit the complex hardware. SHAD is available under Apache v2 License at https://github.com/pnnl/SHAD. In this paper we overview the design of the SHAD library, depicting its main components: runtime systems abstractions for tasking; parallel and distributed data-structures; STL-compliant interfaces and algorithms.

Castellana, Vito G.↗
DOE OSTI2020

The Architecture of Metabolism Maximizes Biosynthetic Diversity in the Largest Class of Fungi

Abstract Ecological diversity in fungi is largely defined by metabolic traits, including the ability to produce secondary or “specialized” metabolites (SMs) that mediate interactions with other organisms. Fungal SM pathways are frequently encoded in biosynthetic gene clusters (BGCs), which facilitate the identification and characterization of metabolic pathways. Variation in BGC composition reflects the diversity of their SM products. Recent studies have documented surprising diversity of BGC repertoires among isolates of the same fungal species, yet little is known about how this population-level variation is inherited across macroevolutionary timescales. Here, we applied a novel linkage-based algorithm to reveal previously unexplored dimensions of diversity in BGC composition, distribution, and repertoire across 101 species of Dothideomycetes, which are considered the most phylogenetically diverse class of fungi and known to produce many SMs. We predicted both complementary and overlapping sets of clustered genes compared with existing methods and identified novel gene pairs that associate with known secondary metabolite genes. We found that variation among sets of BGCs in individual genomes is due to nonoverlapping BGC combinations and that several BGCs have biased ecological distributions, consistent with niche-specific selection. We observed that total BGC diversity scales linearly with increasing repertoire size, suggesting that secondary metabolites have little structural redundancy in individual fungi. We project that there is substantial unsampled BGC diversity across specific families of Dothideomycetes, which will provide a roadmap for future sampling efforts. Our approach and findings lend new insight into how BGC diversity is generated and maintained across an entire fungal taxonomic class.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2020

Cooperative Load Scheduling for Multiple Aggregators Using Hierarchical ADMM

Demand response (DR) serves an important role in improving the efficiency and stability of power systems. In recent years, with advances in communication and smart device technologies, many aggregators have emerged to facilitate end customer participation in DR programs. These aggregators, equipped with customized optimal control algorithms, are capable of providing various grid services. Among them is load scheduling during DR events, namely following a load signal provided by the utility company while minimizing overall customer discomfort. However, as the number of aggregators keeps increasing, it becomes challenging for utility companies to conduct load scheduling for multiple aggregators and generate reference signals for each of them. This paper proposes an optimization framework using hierarchical alternating direction method of multipliers (H-ADMM) to optimally generate load following signals for multiple aggregators. Under this framework, utility and multiple aggregators work in a cooperative manner, aiming at minimizing an overall system cost from different levels of the power system hierarchy, while protecting user privacy. A case study has been conducted in a system with multiple aggregators, based on control of HVAC loads. Experimental results validate the effectiveness of the proposed algorithm.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

SHAPER: can you hear the shape of a jet?

The identification of interesting substructures within jets is an important tool for searching for new physics and probing the Standard Model at colliders. Many of these substructure tools have previously been shown to take the form of optimal transport problems, in particular the Energy Mover’s Distance (EMD). In this work, we show that the EMD is in fact the natural structure for comparing collider events, which accounts for its recent success in understanding event and jet substructure. We then present a Shape Hunting Algorithm using Parameterized Energy Reconstruction (SHAPER), which is a general framework for defining and computing shape-based observables. SHAPER generalizes N-jettiness from point clusters to any extended, parametrizable shape. This is accomplished by efficiently minimizing the EMD between events and parameterized manifolds of energy flows representing idealized shapes, implemented using the dual-potential Sinkhorn approximation of the Wasserstein metric. We show how the geometric language of observables as manifolds can be used to define novel observables with built-in infrared-and-collinear safety. We demonstrate the efficacy of the SHAPER framework by performing empirical jet substructure studies using several examples of new shape-based observables.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2022

Adaptive Hierarchical Cyber Attack Detection and Localization in Active Distribution Systems

Development of a cyber security strategy for the active distribution systems is challenging due to the inclusion of distributed renewable energy generations. Here this paper proposes an adaptive hierarchical cyber attack detection and localization framework for distributed active distribution systems via analyzing electrical waveforms. Cyber attack detection is based on a sequential deep learning model, via which even minor cyber attacks can be identified. The two-stage cyber attack localization algorithm first estimates the cyber attack sub-region, and then localize the specified cyber attack within the estimated subregion. We propose a modified spectral clustering-based network partitioning method for the hierarchical cyber attack ‘coarse’ localization. Next, to further narrow down the cyber attack location, a normalized impact score based on waveform statistical metrics is proposed to obtain a ‘fine’ cyber attack location by characterizing different waveform properties. Finally, compared with classical and state-of-art methods, a comprehensive quantitative evaluation with two case studies shows promising estimation results of the proposed framework.

42 ENGINEERING↗
DOE OSTI2024

Augmented Human Analysis (AHA)

Radio frequency (RF) signal monitoring generally emphasizes intentionally generated signals, such as WiFi, Bluetooth, or cellular transmissions. However, electronic devices also produce unintended radiated emissions (UREs), which could also be useful in RF spectrum analysis. In either case, deriving intelligence from RF signals is typically a human-intensive process requiring significant domain knowledge. In the Augmented Human Analysis (AHA) project, we investigate the utility of dimensionally aligned signal projection (DASP) and machine learning (ML) algorithms for accelerating RF analysis workflows. We find that while DASP algorithms can indeed highlight signal characteristics relevant for classification tasks, the choice of algorithmic hyperparameters greatly affects performance. To address this challenge, we evaluate the quality of DASP outputs using the silhouette score, which measures how well data points cluster; high silhouette scores indicate good clustering, and thus good hyperparameter values. This approach is critical for machine learning pipelines as the DASP parameters cannot be directly optimized during model training. By identifying good DASP parameters, and thus good DASP outputs, as a preprocessing step, we can decrease the amount of effort required for downstream ML model training. We demonstrate our workflow using a dataset of UREs from common household devices, showing that even without the aid of ML, proper selection of DASP parameters enables clustering by device type.

42 ENGINEERING↗
DOE OSTI2025

Spectral Clustering-Based Partitioning of Large-Scale Power Electronics-Based Power Systems for Small-Signal Stability Analysis

The nodal admittance matrix (NAM)-based approach is well-suited for small-signal stability analysis of large-scale power electronics-based power systems (PEPSs), as it preserves the system structure through its admittance matrix. Previous studies have explored partitioning such systems into subareas and interconnections to reduce computational burden; however, they lacked a formal algorithmic procedure for determining feasible partitions. While several grid partitioning methods, such as those based on graph theory or machine learning, exist in the literature, they cannot be directly applied to NAM-based analysis due to differing objectives and constraints. Here, this paper addresses this gap by presenting a systematic, step-by-step procedure for applying a spectral partitioning algorithm that yields a division of the system into subareas suitable for NAM-based analysis. The computational complexity of the proposed method is also derived to demonstrate its efficiency and justify the practicality of the resulting subarea decomposition. The performance of the partitioning method is evaluated by applying the spectral clustering-derived subareas and interconnections to the NAM-based partitioning approach on a 140-bus system. Computational times for the full-system and partitioned NAM analyses are compared using MATLAB. Additionally, PSCAD simulations of the complete system and partitioned subareas are carried out to verify the effectiveness of the proposed method.

Nupur [Univ. of Tennessee, Knoxville, TN (United S↗
DOE OSTI2022

Using machine learning to identify extragalactic globular cluster candidates from ground-based photometric surveys of M87

Globular clusters (GCs) have been at the heart of many longstanding questions in many sub-fields of astronomy and, as such, systematic identification of GCs in external galaxies has immense impacts. In this study, we take advantage of M87’s well-studied GC system to implement supervised machine learning (ML) classification algorithms – specifically random forest and neural networks – to identify GCs from foreground stars and background galaxies, using ground-based photometry from the Canada–France–Hawaii Telescope (CFHT). We compare these two ML classification methods to studies of ‘human-selected’ GCs and find that the best-performing random forest model can reselect 61.2 per cent ± 8.0 per cent of GCs selected from HST data (ACSVCS) and the best-performing neural network model reselects 95.0 per cent ± 3.4 per cent. When compared to human-classified GCs and contaminants selected from CFHT data – independent of our training data – the best-performing random forest model can correctly classify 91.0 per cent ± 1.2 per cent and the best-performing neural network model can correctly classify 57.3 per cent ± 1.1 per cent. ML methods in astronomy have been receiving much interest as Vera C. Rubin Observatory prepares for first light. The observables in this study are selected to be directly comparable to early Rubin Observatory data and the prospects for running ML algorithms on the upcoming data set yields promising results.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2021

Neuromorphic learning with Mott insulator NiO

Habituation and sensitization (nonassociative learning) are among the most fundamental forms of learning and memory behavior present in organisms that enable adaptation and learning in dynamic environments. Emulating such features of intelligence found in nature in the solid state can serve as inspiration for algorithmic simulations in artificial neural networks and potential use in neuromorphic computing. In this work, we demonstrate nonassociative learning with a prototypical Mott insulator, nickel oxide (NiO), under a variety of external stimuli at and above room temperature. Similar to biological species such as Aplysia, habituation and sensitization of NiO possess time-dependent plasticity relying on both strength and time interval between stimuli. A combination of experimental approaches and first-principles calculations reveals that such learning behavior of NiO results from dynamic modulation of its defect and electronic structure. An artificial neural network model inspired by such nonassociative learning is simulated to show advantages for an unsupervised clustering task in accuracy and reducing catastrophic interference, which could help mitigate the stability–plasticity dilemma. Mott insulators can therefore serve as building blocks to examine learning behavior noted in biology and inspire new learning algorithms for artificial intelligence.

42 ENGINEERING↗
DOE OSTI2021

Cleaning Images with Gaussian Process Regression

Many approaches to astronomical data reduction and analysis cannot tolerate missing data: corrupted pixels must first have their values imputed. This paper presents astrofix, a robust and flexible image imputation algorithm based on Gaussian process regression. Through an optimization process, astrofix chooses and applies a different interpolation kernel to each image, using a training set extracted automatically from that image. It naturally handles clusters of bad pixels and image edges and adapts to various instruments and image types. For bright pixels, the mean absolute error of astrofix is several times smaller than that of median replacement and interpolation by a Gaussian kernel. We demonstrate good performance on both imaging and spectroscopic data, including the SBIG 6303 0.4 m telescope and the FLOYDS spectrograph of Las Cumbres Observatory and the CHARIS integral-field spectrograph on the Subaru Telescope.

42 ENGINEERING↗
DOE OSTI2025

Uncertainty estimation of bifurcated solutions in the Rayleigh–Bénard problem for advanced nuclear reactors applications

Multiphysics models of nuclear reactors frequently comprise nonlinear systems of equations. The nonlinear nature of these models could lead to solution bifurcations, where a small change in a certain parameter, e.g., the thermophysical properties of the coolant, can lead to a sudden change in the system’s behavior. At the point in parameter space where this happens, called a critical point, the Jacobian matrix of the model’s nonlinear operator becomes singular potentially permitting multiple solutions to coexist. In this paper, we perform uncertainty estimation (UE) in a parameter range that includes bifurcated solutions within the context of Rayleigh–Bénard problem. We perform this analysis assuming uncertain temperature difference, and tilt angle for the iterative solution algorithm with a unit Prandtl number (Pr = 1). Also, we perform this analysis under uncertain thermophysical properties for both FLiBe molten salt and liquid sodium as working fluid. We deploy two approaches to compute statistical moments for the resulting distributions of selected flow-field variables. The first approach is the blind computation of the mean and the standard deviation without any consideration of solution bifurcation, while the second approach utilizes k-means clustering to cluster each branch’s solutions together and compute separate statistical moments for each branch. The statistical distributions are obtained by perturbing the selected parameters about nominal values that correspond to a solution on one of the valid branches, and that solution is used as initial guess for the iterative solution algorithm. We found that perturbation of any parameter when its nominal value is close to its critical point always leads to branch jumping, i.e., the iterations converge to a solution on a branch different from the branch of the initial guess. This produces a statistical ensemble comprised of fundamentally different solutions leading to wrong mean values and uncertainty estimates, whereas clustering provides an efficient way to deal with this type of computation. This work is important for developing Gen IV nuclear systems because many of these systems rely on natural convection for cooling especially in accident conditions.

97 - MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Speedup of UEDGE Parameter Scans Using Machine-Learning Optimized OpenMP Parallelization and a Continuation Solver

This article presents the OpenMP parallelization of the preconditioning Jacobian assembly and right‐hand side residual evaluation in UEDGE. A continuation algorithm, utilizing the internal NKSOL implicit Jacobian‐Free Newton‐Krylov solver to efficiently scan physical parameters, is also presented. The implemented parallelization reduces the computational time for a benchmark scan run on 32 threads by compared to the serial version when using trained random forest regression models to identify the optimal decomposition of the system of equations. Random forest regression models applied to the UEDGE time‐dependent and continuation solver algorithms did not yield meaningful improvement in computational performance. A benchmark DIII‐D gas injection rate scan in the 0.35–0.75 kA interval, performed on a test cluster using the parallelized code and continuation solver, produced 1066 steady‐state solutions with a 22 s average wall‐clock computational time per steady‐state solution.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2024

Graph Analytics on Jellyfish topology

Because large unstructured datasets is important for many science domains, distributed graph analytics is critical to many scientists. Unfortunately, obtaining scaling and performance for irregular communication is challenging because contemporary network interconnects are primarily designed to maximize bandwidths of fixed-neighborhoods large-message exchanges (e.g., stencils). Although there is no consensus on the “best” network topologies for irregular communication, unstructured graph-based interconnects can be more suitable. We analyze three popular graph workloads – clustering, pattern enumeration, and traversal — on comparable networks (in terms of resources and costs) constructed from Jellyfish Random Regular, Dragonfly and Fat tree topologies, varying the routing algorithms. Using packet-level simulations, we demonstrate up to 60% improvement in communication time with Jellyfish due to diversity of the short paths between arbitrary endpoints, which can reduce overall network stalls and congestion.

Graph Analytics, network topology, interconnect, H↗
DOE OSTI2021

Planetary normal mode computation: Parallel algorithms, performance and reproducibility

This report is an extension of work entitled “Computing planetary interior normal modes with a highly parallel polynomial filtering eigensolver.” by Shi et al., [1] originally presented at the SC18 conference. A highly parallel polynomial filtered eigensolver was developed and exploited to calculate the planetary normal modes. The proposed method is ideally suited for computing interior eigenpairs for large-scale eigenvalue problems as it greatly enhances memory and computational efficiency. In this article, the second-order finite element method is used to further improve the accuracy as only the first-order finite element method was deployed in the previous work. The parallel algorithm, its parallel performance up to 20k processors, and the great computational accuracy are illustrated. The reproducibility of the previous work was successfully performed on the Student Cluster Competition at the SC19 conference by several participant teams using a completely different Mars-model dataset on different clusters. Both weak and strong scaling performances of the reproducibility by the participant teams were impressive and encouraging. The analysis and reflection of their results are demonstrated and future direction is discussed.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Kinetic Model of HoxEFU reduction by NADH [SWR-26-087]

This repository is used to release code generated for manuscripts on the Photosynthetic Energy Transduction core program. This code simulates the reduction of HoxEFU by NADH. The electro transfer rate constants for the simulation are specified in the .csv files. The two .csv files correspond tot he two models described in Dawson et al. Cell. Rep. Phys. Sci. 2026. The code utilizes a chemical master equation, a set of differential equations, defining the time evolution of the oxidation and reduction kinetics of NAD+, NADH, a FMN flavin, and a set of iron sulfur clusters. The kinetics of HoxEFU reduction by NADH are evaluated by numerical integration of the chemical master equation using a variable-time-step Runge-Kutta algorithm.

Dahl, Peter [National Laboratory of the Rockies (N↗
DOE OSTI2020

mpi-sppy: Optimization Under Uncertainty for Pyomo

We introduce a new Pyomo extension for optimization under uncertainty, mpi-sppy. This extension allows for the mixing of, and sharing of information between, various algorithmic approaches for optimization under uncertainty. We will discuss the underlying architecture and assess the performance and scalability of mpi-sppy on various systems, including HPC clusters.

27 ARPA - Advanced Research Projects Agency-Energy↗