Hierarchical Algorithms for Training Kolmogorov-Arnold Networks
Explore the source record for details and available documents.
Cyr, Eric Christopher [Sandia National Laboratorie↗
SEARCH · Engineering Papers
Results for “hierarchical algorithm”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
A Hierarchical OPF Algorithm with Improved Gradient Evaluation in Three-Phase Networks
Linear approximation commonly used in solving alternating-current optimal power flow (AC-OPF) simplifies the system models but incurs accumulated voltage errors in large power networks. Such errors will make the primal-dual type gradient algorithms converge to solutions with voltage violation. In this paper, we improve a recent hierarchical OPF algorithm that rested on primal-dual gradients evaluated with a linearized distribution power flow model. Specifically, we propose a more accurate gradient evaluation method based on an unbalanced three-phase nonlinear distribution power flow model to mitigate the errors arising from linearization. The resultant gradients feature a blocked structure that enables our development of an improved hierarchical primal-dual algorithm to solve the OPF problem. Numerical results on the IEEE 123-bus test feeder and a 4,518-node test feeder show that the proposed method can enhance voltage safety at comparable computational efficiency with the linearized algorithm.
approximation algorithms↗
A multiscale design method using interpretable machine learning for phononic materials with closely interacting scales
Manipulating the dispersive characteristics of vibrational waves is beneficial for many applications, e.g., high-precision instruments. architected hierarchical phononic materials have sparked promise tunability of elastodynamic waves and vibrations over multiple frequency ranges. In this article, hierarchical unit-cells are obtained, where features at each length scale result in a band gap within a targeted frequency range. Our novel approach, the ‘‘hierarchical unit-cell template method,’’ is an interpretable machine-learning approach that uncovers global unit-cell shape/topology patterns corresponding to predefined band-gap objectives. A scale-separation effect is observed where the coarse-scale band-gap objective is mostly unaffected by the fine-scale features despite the closeness of their length scales, thus enabling an efficient hierarchical algorithm. Moreover, the hierarchical patterns revealed are not predefined or self-similar hierarchies as common in current hierarchical phononic materials. Furthermore, our approach offers a flexible and efficient method for the exploration of new regions in the hierarchical design space, extracting minimal effective patterns for inverse design in applications targeting multiple frequency ranges.
Architected materials↗
A Hierarchical Optimization Method for Electric Vertical Takeoff and Landing Aircraft Network Design
Electric vertical takeoff and landing aircraft (eVTOLs) are expected to serve urban air mobility in a station-to-station configuration, which makes the optimal network design of eVTOL stations a critical question to explore. Existing approaches often face limitations, such as the inability to interact station locations with demand or difficulty in finding the optimal solution for large study regions. Here, this paper first proposes a mathematical model to generate optimal eVTOL station locations while considering associated potential eVTOL demand, and then proposes a heuristic algorithm, Hierarchical Optimization MEthod (HOME), to efficiently solve the model. With a case study of Southern California, HOME was compared to 1) directly solving the original integer linear programming-based network design problem, and 2) employing the widely used genetic algorithm. Results suggest that HOME can find optimal solutions with limited computational resources. The proposed framework powered by HOME provides a computationally efficient way to support urban air mobility planning.
97 MATHEMATICS AND COMPUTING↗
Hierarchical Analysis of Halo Center in Cosmology
Ever-increasing data size raises many challenges for scientific data analysis. Particularly in cosmological N-body simulation, finding the center of a dark matter halo suffers heavily from the large computational cost associated with the large number of particles (up to 20 million). In this work, we exploit the latent structure embed in a halo, and we propose a hierarchical approach to approximate the exact gravitational potential calculation for each particle in order to more efficiently find the halo center. Tests of our method on data from N-body simulations show that in many cases the hierarchical algorithm performs significantly faster than existing methods with a desirable accuracy.
97 MATHEMATICS AND COMPUTING↗
Optimizing Multi-Robot Placements for Wire Arc Additive Manufacturing
Wire arc additive manufacturing is a metal additive manufacturing process in which the material is deposited using arc welding technology. It is gaining popularity due to high material deposition rates and faster build time. It is en-abled using robotic manipulators and can build relatively large-scale parts faster when compared with other metal additive manufacturing processes. However, the size of the large-scale parts is limited by the size of the industrial manipulator being used for the process. This limitation is overcome by using a fixed configuration multi-robot cell in which manipulators work cooperatively to build large-scale parts quickly. A fixed multi-robot cell with closely spaced industrial manipulators has high flexibility, but it restricts the part size that can be built. If the manipulators are spread out, the cell loses its flexibility but can build relatively larger parts. This issue can be avoided by using larger size manipulators, which are expensive, or by moving the modest size manipulators based on the part geometries. This paper presents a novel algorithm to generate multi-robot placements for different part geometries to be built using wire arc additive manufacturing. Furthermore, the algorithm hierarchically optimizes the build time and the inverse kinematics consistency in robot paths to improve the process efficiency and part quality. We compare the results with fixed multi-robot cells and provide insights to users to make an informed decision on whether to use a fixed or a flexible multi-robot cell for wire arc additive manufacturing.
Bhatt, Prahar↗
CONGO²: Scalable Online Anomaly Detection and Localization in Power Electronics Networks
Rapid and accurate detection and localization of electronic disturbances simultaneously are important for preventing its potential damages and determining potential remedies. Existing anomaly detection methods are severely limited by the low accuracy, the expensive computational cost and the need for highly trained personnel. There is an urgent need for a scalable online algorithm for in-field analysis of large-scale power electronics networks. Here in this paper, we propose a fast and accurate algorithm for anomaly detection and localization of power electronics networks: stratified colored-node graph (CONGO2). This algorithm hierarchically models the change of correlated waveforms and then correlated sensors using the colored-node graph. By aggregating the change of each sensor with its neighbors’ inputs, we can spontaneously identify and localize the anomaly that cannot be detected by data collected from a single sensor. As our proposed method only focuses on the changes within a short time frame, it is highly computational efficient and only needs small data storage. Thus, our method is ideal for online and reliable anomaly detection and localization of large-scale power electronic networks. Compared to existing anomaly detection methods, our method is entirely data-driven without training data, highly accurate and reliable for wide-spectrum anomalies detection, and more importantly, capable of both detection and localization. Thus, it is ideal for infield deployment for large-scale power electronic networks. As illustrated by a distributed energy resources (DERs) power grid with 37-node, our method can effectively detect and localize various cyber and physical attacks.
42 ENGINEERING↗
Haar-Like Wavelets on Hierarchical Trees
Here, discrete wavelet methods, originally formulated in the setting of regularly sampled signals, can be adapted to data defined on a point cloud if some multiresolution structure is imposed on the cloud. A wide variety of hierarchical clustering algorithms can be used for this purpose, and the multiresolution structure obtained can be encoded by a hierarchical tree of subsets of the cloud. Prior work introduced the use of Haar-like bases defined with respect to such trees for approximation and learning tasks on unstructured data. This paper builds on that work in two directions. First, we present an algorithm for constructing Haar-like bases on general discrete hierarchical trees. Second, with an eye towards data compression, we present thresholding techniques for data defined on a point cloud with error controlled in the $L$ $\infty$ norm and in a Hölder-type norm. In a concluding trio of numerical examples, we apply our methods to compress a point cloud dataset, study the tightness of the $L$ $\infty$ error bound, and use thresholding to identify MNIST classifiers with good generalizability.
97 MATHEMATICS AND COMPUTING↗
Scalable edge clustering of dynamic graphs via weighted line graphs
Timestamped relational datasets consisting of records (or connections) between pairs of entities are ubiquitous in network science. For applications like peer-to-peer communication, email, various social network interactions, and computer network security, it is useful to organize these records into groups based on how and when they are occurring. Weighted line graphs offer a natural way to model how records are related in such datasets but for large real-world graph topologies, building and utilizing the line graph is prohibitively expensive. Here, we present the framework to cluster the edges of a dynamic graph via the associated line graph that contains two major contributions. The first is a method to work with the line graph implicitly and the second is a distributed scale implementation of an agglomerative hierarchical graph clustering algorithm. We outline a novel hierarchical dynamic graph edge clustering approach that efficiently breaks massive relational datasets into small sets of edges containing events at various timescales. This is in stark contrast to traditional graph clustering algorithms that prioritize highly connected (clique-like) community structures. Our approach relies on constructing a sufficient subgraph of a weighted line graph and applying a hierarchical agglomerative clustering. This approach is related to scalable techniques from spatial clustering, nonlinear-dimension reduction, topological data analysis, and draws particular inspiration from HDBSCAN. As an edge clustering, this method yields an overlapping node clustering. Our algorithm is parallelizable and we demonstrate efficient clustering of a billion-scale, real-world dynamic graph into small edge sets that correlate in topology and time. The entire clustering process for a graph with tens of billions of edges takes just a few minutes of run time on 256 nodes of a distributed compute environment. We argue how the output of the edge clustering is useful for a multitude of data visualization and powerful machine learning tasks, both involving the original massive dynamic graph data and metadata associated with the nodes and edges. Finally, we describe how this approach can be extended to dynamic hypergraphs and dynamic graphs/hypergraphs with unstructured data living on vertices and edges.
Data Analysis↗
Deep Hierarchical Super Resolution for Scientific Data
We present a novel technique for hierarchical super resolution (SR) with neural networks (NNs), which upscales volumetric data represented with an octree data structure to a high-resolution uniform gridwith minimal seam artifacts on octree node boundaries. Our method uses existing state-of-the-art SR models and adds flexibility to upscale input data with varying levels of detail across the domain, instead of only uniform grid data that are supported in previous approaches.The key is to use a hierarchy of SR NNs, each trained to perform 2x SR between two levels of detail, with a hierarchical SR algorithm that minimizes seam artifacts by starting from the coarsest level of detail and working up.We show that our hierarchical approach outperforms baseline interpolation and hierarchical upscaling methods, and demonstrate the usefulness of our proposed approach across three use cases including data reduction using hierarchical downsampling+SR instead of uniform downsampling+SR, computation savings for hierarchical finite-time Lyapunov exponent field calculation, and super-resolving low-resolution simulation results for a high-resolution approximation visualization.
97 MATHEMATICS AND COMPUTING↗
High performance FPGA embedded system for machine learning based tracking and trigger in sPhenix and EIC
We present a comprehensive end-to-end pipeline to classify triggers versus background events in this paper. This pipeline makes online decisions to select signal data and enables the intelligent trigger system for efficient data collection in the Data Acquisition System (DAQ) of the upcoming sPHENIX and future EIC (Electron-Ion Collider) experiments. Starting from the coordinates of pixel hits that are lightened by passing particles in the detector, the pipeline applies three-stage of event processing (hits clustering, track reconstruction, and trigger detection) and labels all processed events with the binary tag of trigger versus background events. The pipeline consists of deterministic algorithms such as clustering pixels to reduce event size, tracking reconstruction to predict candidate edges, and advanced graph neural network-based models for recognizing the entire jet pattern. In particular, we apply the message-passing graph neural network to predict links between hits and reconstruct tracks and a hierarchical pooling algorithm (DiffPool) to make the graph-level trigger detection. We obtain an impressive performance (≥70% accuracy) for trigger detection with only 3200 neuron weights in the end-to-end pipeline. We deploy the end-to-end pipeline into a field-programmable gate array (FPGA) and accelerate the three stages with speedup factors of 1152, 280, and 21, respectively.
Instruments & Instrumentation↗
Integrate Latimer Controls' Solution into RTAC (CRADA Final Report, CRD-23-24672)
Latimer Controls, Inc. was awarded two vouchers under the Department of Energy's American-Made Solar Prize Round 6 to conduct collaborative research at a national laboratory. The National Renewable Energy Laboratory (NREL) was selected as a partner to assist Latimer Controls in the performance evaluation of its photovoltaic (PV) control software. This collaboration focuses on developing a hardware-in-the-loop (HIL) testbed at NREL, which will be used to test and validate the Latimer PV control technology in a realistic yet de-risked environment. Both Latimer and NREL teams will work together to analyze the collected test data, derive insights, and disseminate the scientific findings. Recent studies underscore the potential of solar energy as a zero-marginal-cost and zero-emission flexibility resource within the bulk power system, particularly when integrated with advanced control systems. To enhance the performance of such systems, Latimer Controls has developed leading-edge technologies, including machine learning (ML) algorithms and hierarchical inverter set-point allocation methods. These innovations are designed to estimate the operational headroom of large PV plants for grid integration and control. However, comprehensive validation under real-world conditions remains necessary. To address this gap, the concurrent CRADA project proposes the real-world application and validation of the Latimer Control solution within a HIL environment. Initially, the Latimer algorithm was developed and tested within MATLAB Simulink, a platform suitable for research-level simulations and iterative development. However, transitioning this technology to a real solar site as an industry-ready solution necessitates implementation in a format compatible with widely used solar power plant controllers. In this additional CRADA work, the MATLAB Simulink-based logic will be translated into Structured Text, a programming language compliant with IEC 61131 standards, which is commonly used for custom logic implementations in industry-leading programmable logic controllers (PLCs), such as the Schweitzer SEL real-time automation controller (RTAC). This transition will facilitate the deployment of the Latimer Control solution in real-world solar power plants, thereby advancing the technology towards commercialization.
14 SOLAR ENERGY↗
Optimizing group structures using hierarchical division
Creating group structures with few groups that give low errors is a difficult problem in reactor analysis. In recent years, automated optimization techniques have been applied to this task. We continue this trend by applying the hierarchical division algorithm to generate optimized group structures that minimize a cost function. At each stage, the algorithm adds a single group boundary to an existing group structure, dividing one group into two to increase the resolution of the group structure. The location of the added boundary is the one that gives the lowest error over all possible new boundary locations. Our implementation requires a beginning group structure, a set of candidate new boundary locations, and a set of reference reaction rates. As a proof of concept, we used WIMS-69 as the initial group structure, XMAS-172 as the ending group structures, and a 344-group reference group structure. Testing on two simple, homogenized reactor problems, we found that hierarchical division was able to reduce the error by a factor of around 5 with an increase of only 15% in the number of groups. Because hierarchical division can get stuck in local minima, it often reaches a plateau in its error reduction capability as many groups are added. Nevertheless, we find hierarchical division has strong potential to make good group structures into great group structures at a modest increase in computational cost. (authors)
22 GENERAL STUDIES OF NUCLEAR REACTORS↗
Phlex: Parallel, Hierarchical, and Layered EXecution of data-processing algorithms
Phlex is a computing framework supporting the parallel, hierarchical, and layered execution of data-processing algorithms. It is based on the functional-programming paradigm, thus guaranteeing thread-safety when invoking user-defined pure functions. Phlex allows users to specify arbitrary graph-based hierarchies of data organization, enabling more flexible processing of data as required by the constraints of the program.
Knoepfel, KyleJ. [Fermi National Accelerator Labor↗
Enabling machine learning-ready HPC ensembles with Merlin
With the growing complexity of computational and experimental facilities, many scientific researchers are turning to machine learning (ML) techniques to analyze large scale ensemble data. With complexities such as multi-component workflows, heterogeneous machine architectures, parallel file systems, and batch scheduling, care must be taken to facilitate this analysis in a high performance computing (HPC) environment. Here, we present Merlin, a workflow framework to enable large ML-friendly ensembles of scientific HPC simulations. By augmenting traditional HPC with distributed compute technologies, Merlin aims to lower the barrier for scientific subject matter experts to incorporate ML into their analysis. As a producer–consumer workflow model, Merlin enables multi-machine, cross-batch job, dynamically allocated yet persistent workflows capable of utilizing surge-compute resources. Key features of Merlin are a flexible HPC-centric interface, low per-task overhead, multi-tiered fault recovery, and a hierarchical sampling algorithm that allows for $\mathscr{O}$(N) task execution and $\mathscr{O}$(N ln N) task queuing to ensembles of millions of tasks. In addition to Merlin’s design, we test the algorithm’s performance in an HPC center and demonstrate the ability to enqueue 40 million simulations in 100 s, with a 30 millisecond per-task overhead that is independent of ensemble size. Finally, we describe some example applications that Merlin has enabled on leadership-class HPC resources, such as the ML-augmented optimization of nuclear fusion experiments and the calibration of infectious disease models to study the progression of and possible mitigation strategies for COVID-19.
97 MATHEMATICS AND COMPUTING↗
Coherent correlation imaging for resolving fluctuating states of matter
Fluctuations and stochastic transitions are ubiquitous in nanometre-scale systems, especially in the presence of disorder. However, their direct observation has so far been impeded by a seemingly fundamental, signal-limited compromise between spatial and temporal resolution. Here we develop coherent correlation imaging (CCI) to overcome this dilemma. Our method begins by classifying recorded camera frames in Fourier space. Contrast and spatial resolution emerge by averaging selectively over same-state frames. Temporal resolution down to the acquisition time of a single frame arises independently from an exceptionally low misclassification rate, which we achieve by combining a correlation-based similarity metric with a modified, iterative hierarchical clustering algorithm. We apply CCI to study previously inaccessible magnetic fluctuations in a highly degenerate magnetic stripe domain state with nanometre-scale resolution. We uncover an intricate network of transitions between more than 30 discrete states. Our spatiotemporal data enable us to reconstruct the pinning energy landscape and to thereby explain the dynamics observed on a microscopic level. CCI massively expands the potential of emerging high-coherence X-ray sources and paves the way for addressing large fundamental questions such as the contribution of pinning and topology in phase transitions and the role of spin and charge order fluctuations in high-temperature superconductivity.
36 MATERIALS SCIENCE↗
Variability in terrestrial characteristics and erosion rates on the Alaskan Beaufort Sea coast
Abstract Arctic coastal environments are eroding and rapidly changing. A lack of pan-Arctic observations limits our ability to understand controls on coastal erosion rates across the entire Arctic region. Here, we capitalize on an abundance of geospatial and remotely sensed data, in addition to model output, from the North Slope of Alaska to identify relationships between historical erosion rates and landscape characteristics to guide future modeling and observational efforts across the Arctic. Using existing datasets from the Alaska Beaufort Sea coast and a hierarchical clustering algorithm, we developed a set of 16 coastal typologies that captures the defining characteristics of environments susceptible to coastal erosion. Relationships between landscape characteristics and historical erosion rates show that no single variable alone is a good predictor of erosion rates. Variability in erosion rate decreases with increasing coastal elevation, but erosion rate magnitudes are highest for intermediate elevations. Areas along the Alaskan Beaufort Sea coast (ABSC) protected by barrier islands showed a three times lower erosion rate on average, suggesting that barrier islands are critical to maintaining mainland shore position. Finally, typologies with the highest erosion rates are not broadly representative of the ABSC and are generally associated with low elevation, north- to northeast-facing shorelines, a peaty pebbly silty lithology, and glaciomarine deposits with high ice content. All else being equal, warmer permafrost is also associated with higher erosion rates, suggesting that warming permafrost temperatures may contribute to higher future erosion rates on permafrost coasts. The suite of typologies can be used to guide future modeling and observational efforts by quantifying the distribution of coastlines with specific landscape characteristics and erosion rates.
54 ENVIRONMENTAL SCIENCES↗
Small-Signal Angle Stability-Oriented False Data Injection Cyber-Attacks on Power Systems
The small-signal angle stability (SSAS) of a power system is determined by the property of operation points. The widely applied false data injection (FDI) cyber-attack, however, is able to stealthily mislead the optimal power flow (OPF) and thus compromise operation points, leading to damages to the SSAS margin. Here, to provide insights for cyber defenders, this paper proposes and investigates a stealthy SSAS-oriented FDI cyber-attack focusing on two attacking purposes, i.e., the SSAS margin and operation cost, with higher priority on the former one. First, this paper establishes a novel bi-level model with an implicit SSAS constraint based on a structure preserving model to compromise operation points. Then, for the SSAS interarea mode in a typical two-area system, this paper formulates closed-form expressions of how the SSAS margin and operation cost behave with respect to stealthy injections. By comparison, for the SSAS local mode in general power systems, this paper proposes a moving target cyber-attack-based hierarchical solution algorithm. Simulation results on a two-area system, a Kundur 11 bus system, and a modified IEEE 14 bus system demonstrate the significant damaging effects of the proposed SSAS-oriented FDI cyber-attack and the conflict between the two attacking purposes.
Benders decomposition↗