Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Scalable Algorithms”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 487 records · Page 27

Real Time Simulation of Power Grid Disruptions

DOE-OE and DOE-SC workshops (Reference 1-3) identified the key power grid problem that requires insight addressable by the next generation of exascale computing is coupling of real-time data streams (1-2 TB per hour) as the streams are ingested to dynamic models. These models would then identify predicted disruptions in time (2-4 seconds) to trigger the smart grid s self healing functions. This project attempted to establish the feasibility of this approach and defined the scientific issues, and demonstrated example solutions to important smart grid simulation problems. These objectives were accomplished by 1) using the existing frequency recorders on the national grid to establish a representative and scalable real-time data stream; 2) invoking ORNL signature identification algorithms; 3) modeling dynamically a representative region of the Eastern interconnect using an institutional cluster, measuring the scalability and computational benchmarks for a national capability; and 4) constructing a prototype simulation for the system s concept of smart grid deployment. The delivered ORNL enduring capability included: 1) data processing and simulation metrics to design a national capability justifying exascale applications; 2) Software and intellectual property built around the example solutions; 3) demonstrated dynamic models to design few second self-healing.

Chinthavali, Supriya↗

Improved particle-flow event reconstruction with scalable neural networks for current and future particle detectors

Abstract Efficient and accurate algorithms are necessary to reconstruct particles in the highly granular detectors anticipated at the High-Luminosity Large Hadron Collider and the Future Circular Collider. We study scalable machine learning models for event reconstruction in electron-positron collisions based on a full detector simulation. Particle-flow reconstruction can be formulated as a supervised learning task using tracks and calorimeter clusters. We compare a graph neural network and kernel-based transformer and demonstrate that we can avoid quadratic operations while achieving realistic reconstruction. We show that hyperparameter tuning significantly improves the performance of the models. The best graph neural network model shows improvement in the jet transverse momentum resolution by up to 50% compared to the rule-based algorithm. The resulting model is portable across Nvidia, AMD and Habana hardware. Accurate and fast machine-learning based reconstruction can significantly improve future measurements at colliders.

Physics↗

Final Report: Towards Optimal Petascale Simulations (TOPS), ER25785

Multiscale, multirate scientific and engineering applications in the SciDAC portfolio possess resolution requirements that are practically inexhaustible and demand execution on the highest-capability computers available, which will soon reach the petascale. While the variety of applications is enormous, their needs for mathematical software infrastructure are surprisingly coincident; moreover the chief bottleneck is often the solver. At their current scalability limits, many applications spend a vast majority of their operations in solvers, due to solver algorithmic complexity that is superlinear in the problem size, whereas other phases scale linearly. Furthermore, the solver may be the phase of the simulation with the poorest parallel scalability, due to intrinsic global dependencies. This project brings together the providers of some of the world's most widely distributed, freely available, scalable solver software and focuses them on relieving this bottleneck for many specific applications within SciDAC, which are representative of many others outside. Solver software directly supported under TOPS includes: hypre, PETSc, SUNDIALS, SuperLU, TAO, and Trilinos. Transparent access is also provided to other solver software through the TOPS interface. The primary goals of TOPS are the development, testing, and dissemination of solver software, especially for systems governed by PDEs. Upon discretization, these systems possess mathematical structure that must be exploited for optimal scalability; therefore, application-targeted algorithmic research is included. TOPS software development includes attention to high performance as well as interoperability among the solver components. Support for integration of TOPS solvers into SciDAC applications is also directly supported by this proposal. The role of the UCSD PI in this overall CET, is one of direct interaction between the TOPS software partners and various DOE applications scientists' specifically toward magnetohydrodynamics (MHD) simulations with the Center for Extended Magnetohydrodynamic Modeling (CEMM) SciDAC and Applied Partial Differential Equations Center (APDEC) SciDAC, and toward core-collapse supernova simulations with the previous Terascale Supernova Initiative (TSI) SciDAC and in continued work on INCITE projects headed by Doug Swesty, SUNY Stony Brook. In addition to these DOE applications scientists, the UCSD PI works to bring leading-edge DOE solver technology to applications scientists in cosmology and large-scale galactic structure formation. Unfortunately, the funding for this grant ended after only two years of its five-year duration, in August 2008, due to difficulties at DOE in transferring the grant to the PI's new faculty position at Southern Methodist University. Therefore, this report only describes two years' worth of effort.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Improving the Performance Scalability of the Community Atmosphere Model

The Community Atmosphere Model (CAM), which serves as the atmosphere component of the Community Climate System Model (CCSM), is the most computationally expensive CCSM component in typical configurations. On current and next-generation leadership class computing systems, the performance of CAM is tied to its parallel scalability. Improving performance scalability in CAM has been a challenge, due largely to algorithmic restrictions necessitated by the polar singularities in its latitude-longitude computational grid. Nevertheless, through a combination of exploiting additional parallelism, implementing improved communication protocols, and eliminating scalability bottlenecks, we have been able to more than double the maximum throughput rate of CAM on production platforms. We describe these improvements and present results on the Cray XT5 and IBM BG/P. The approaches taken are not specific to CAM and may inform similar scalability enhancement activities for other codes.

99 GENERAL AND MISCELLANEOUS↗

A Parallel Solver for Graph Laplacians

Problems from graph drawing, spectral clustering, network flow and graph partitioning can all be expressed in terms of graph Laplacian matrices. There are a variety of practical approaches to solving these problems in serial. However, as problem sizes increase and single core speeds stagnate, parallelism is essential to solve such problems quickly. We present an unsmoothed aggregation multigrid method for solving graph Laplacians in a distributed memory setting. We introduce new parallel aggregation and low degree elimination algorithms targeted specifically at irregular degree graphs. These algorithms are expressed in terms of sparse matrix-vector products using generalized sum and product operations. This formulation is amenable to linear algebra using arbitrary distributions and allows us to operate on a 2D sparse matrix distribution, which is necessary for parallel scalability. Our solver outperforms the natural parallel extension of the current state of the art in an algorithmic comparison. We demonstrate scalability to 576 processes and graphs with up to 1.7 billion edges.

Konolige, Tristan↗

HPS-RL: Hyperparameter tuning for deep RL applications (HPS-RL) v1

Genetic Algorithms meets Deep RL for Hyperparameters Hyperparameter optimization and architecture search can easily become cumbersome and finding the right hyperparameters can seriously impact the robustness of the deep RL application being developed. We use genetic algorithms to evolve optimum deep RL architectures in a scalable manner. HPS-RL is designed to work with multiple gym enviornments, allow users to test their own optimization functions and tune multi-objective parameters in multiple deep RL algorithms. HPS-RL uses multi-threading and is being extended with mpipy for distributed processing on HPC. https://arxiv.org/abs/2201.11182

Kiran, Mariam↗

Scalable quantum search using trapped ions

We propose a scalable implementation of Grover's quantum search algorithm in a trapped-ion quantum information processor. The system is initialized in an entangled Dicke state by using adiabatic techniques. The inversion-about-average and oracle operators take the form of single off-resonant laser pulses. This is made possible by utilizing the physical symmetries of the trapped-ion linear crystal. The physical realization of the algorithm represents a dramatic simplification: each logical iteration (oracle and inversion about average) requires only two physical interaction steps, in contrast to the large number of concatenated gates required by previous approaches. This not only facilitates the implementation but also increases the overall fidelity of the algorithm.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Towards Rapid Interactive Machine Learning: Evaluating Tradeoffs of Classification without Representation

Our contribution is the design and evaluation of an interac- tive machine learning interface that rapidly provides the user with model feedback after every interaction. To address visual scalability, this interface communicates with the user via a “tip of the iceberg” approach, where the user interacts with a small set of recommended instances for each class. To address computational scalability, we developed an O(n) classification algorithm that incorporates user feedback incrementally, and without consulting the underlying data’s representation matrix. Our computational evaluation showed that this algorithm has similar accuracy to several off-the-shelf classification algo- rithms with small amounts of labeled data. Empirical evalu- ation revealed that users performed better using our design compared to an equivalent active learning setup.

Arendt, Dustin L.↗

pnnl/ripples

A C++ Library for Influence Maximization Ripples is a software framework to study the Influence Maximization problem. Our goal with Ripples is to provide tools implementing fast and scalable state-of-the-art approximation algorithms to solve the influence maximization problem.

Minutoli, Marco↗

saSNP Approach for Scalable SNP Analyses of Multiple Bacterial or Viral Genomes

With the flood of whole genome finished and draft microbial sequences, we need faster, more scalable bioinformatics tools for sequence comparison. An algorithm is described to find single nucleotide polymorphisms (SNPs) in whole genome data. It scales to hundreds of bacterial or viral genomes, and can be used for finished and/or draft genomes available as unassembled contigs. The method is fast to compute, finding SNPs and building a SNP phylogeny in seconds to hours. We use it to identify thousands of putative SNPs from all publicly available Filoviridae, Poxviridae, foot-and-mouth disease virus, Bacillus, and Escherichia coli genomes and plasmids. The SNP-based trees that result are consistent with known taxonomy and trees determined in other studies. The approach we describe can handle as input hundreds of gigabases of sequence in a single run. The algorithm is based on k-mer analysis using a suffix array, so we call it saSNP.

59 BASIC BIOLOGICAL SCIENCES↗

Investigating the Role of Biogeochemical Processes in the Northern High Latitudes on Global Climate Feedbacks Using an Efficient Scalable Earth System Model

The overall objectives of this DOE funded project is to combine scientific and computational challenges in climate modeling by expanding our understanding of the biogeophysical-biogeochemical processes and their interactions in the northern high latitudes (NHLs) using an earth system modeling (ESM) approach, and by adopting an adaptive parallel runtime system in an ESM to achieve efficient and scalable climate simulations through improved load balancing algorithms.

54 ENVIRONMENTAL SCIENCES↗

Computing properties of hadrons, nuclei and nuclear matter from quantum chromodynamics

The goal of the research of the William and Mary group (PI from Computer Science and Co-PI from Physics) is to develop algorithms that address the limitations in computational scalability of current methods in lattice QCD. Of particular interest is the problem of computing traces of certain functions of a matrix, which arises in nuclear physics calculations of disconnected diagrams and requires the solution of a large number of linear systems of equations.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

k-merSNP discovery: Software for alignment-and reference-free scalable SNP discovery, phylogenetics, and annotation for hundreds of microbial genomes

With the flood of whole genome finished and draft microbial sequences, we need faster, more scalable bioinformatics tools for sequence comparison. An algorithm is described to find single nucleotide polymorphisms (SNPs) in whole genome data. It scales to hundreds of bacterial or viral genomes, and can be used for finished and/or draft genomes available as unassembled contigs or raw, unassembled reads. The method is fast to compute, finding SNPs and building a SNP phylogeny in minutes to hours, depending on the size and diversity of the input sequences. The SNP-based trees that result are consistent with known taxonomy and trees determined in other studies. The approach we describe can handle many gigabases of sequence in a single run. The algorithm is based on k-mer analysis.

Source record↗

GraphMeta: Managing HPC Rich Metadata in Graphs

High-performance computing (HPC) systems face increasingly critical metadata management challenges, especially in the approaching exascale era. These challenges arise not only from exploding metadata volumes, but also from increasingly diverse metadata, which contains data provenance and arbitrary user-defined attributes in addition to traditional POSIX metadata. This ‘rich’ metadata is becoming critical to supporting advanced data management functionality such as data auditing and validation. In our prior work, we identified a graph-based model as a promising solution to uniformly manage HPC rich metadata due to its flexibility and generality. However, at the same time, graph-based HPC rich metadata anagement also introduces significant challenges to the underlying infrastructure. In this study, we first identify the challenges on the underlying infrastructure to support scalable, high-performance rich metadata management. Based on that, we introduce GraphMeta, a graphbased engine designed for this use case. It achieves performance scalability by introducing a new graph partitioning algorithm and a write-optimal storage engine. We evaluate GraphMeta under both synthetic and real HPC metadata workloads, compare it with other approaches, and demonstrate its advantages in terms of efficiency and usability for rich metadata management in HPC systems.

Dai, Dong↗

NE-COST plug-in: Expanding ACCERT's Capabilities for Life-Cycle Cost Modeling

The Algorithm for the Capital Cost Estimation of Reactor Technologies (ACCERT) is a structured methodology and software tool designed to simplify and standardize cost estimation for nuclear reactor technologies [1]. By utilizing a relational database structure and modular cost estimation algorithms, ACCERT delivers a robust, flexible, and scalable framework for evaluating costs across various reactor types and configurations [2]. The recent integration of the NE-COST plugin further expands ACCERT’s scope by introducing detailed life-cycle cost modeling and probabilistic analysis of uncertainties. This addition enables users to evaluate costs across front-end processes such as uranium enrichment and fabrication, as well as back-end activities including waste disposal and geologic storage. Through Monte Carlo statistical cost simulations, the plugin provides probabilistic insights into cost ranges, offering critical decision-making support for stakeholders including reactor developers, policymakers, and researchers.

Zhou, Jia↗

Scalable distributed consensus to support MPI fault tolerance.

As system sizes increase, the amount of time in which an application can run without experiencing a failure decreases. Exascale applications will need to address fault tolerance. In order to support algorithm-based fault tolerance, communication libraries will need to provide fault-tolerance features to the application. One important fault-tolerance operation is distributed consensus. This is used, for example, to collectively decide on a set of failed processes. This paper describes a scalable, distributed consensus algorithm that is used to support new MPI fault-tolerance features proposed by the MPI 3 Forum's fault-tolerance working group. The algorithm was implemented and evaluated on a 4,096-core Blue Gene/P. The implementation was able to perform a full-scale distributed consensus in 305 {mu}s and scaled logarithmically.

99 GENERAL AND MISCELLANEOUS↗

Characterization of bandwidth requirements of algorithms for extreme scale science

Technology trends have resulted in widely different rates of improvement in computational throughput as compared to data movement rates in computer systems. With future systems, the cost of data movement through the memory hierarchy is expected to become even more dominant relative to the cost of performing arithmetic operations, both in terms of time and energy. Therefore, optimizing data access costs will become ever more critical in the coming years. However, unlike the well understood model of computational complexity in terms of the number of elementary operations to execute an algorithm, the inherent data-movement complexity of computations is unknown for most algorithms. Given the crucial importance of optimizing data access costs in scalable parallel systems, it is of great interest to develop techniques for understanding the inherent data movement complexity of an algorithm. In this project, we focus on this fundamental and significant problem.

97 MATHEMATICS AND COMPUTING↗