SEARCH · Engineering Papers

Results for “parallelize algorithm computation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,279 records · Page 71

All sources NASA NTRS DOE OSTI

DOE OSTI2018

DeepHyper: Asynchronous Hyperparameter Search for Deep Neural Networks

Hyperparameters employed by deep learning (DL) methods play a substantial role in the performance and reliability of these methods in practice. Unfortunately, finding performance-optimizing hyperparameter settings is a notoriously difficult task. Hyperparameter search methods typically have limited production-strength implementations or do not target scalability within a highly parallel machine, portability across different machines, experimental comparison between different methods, and tighter integration with workflow systems. In this paper, we present DeepHyper, a Python package that provides a common interface for the implementation and study of scalable hyperparameter search methods. It adopts the Balsam workflow system to hide the complexities of running large numbers of hyperparameter configurations in parallel on high-performance computing (HPC) systems. We implement and study asynchronous model-based search methods that consist of sampling a small number of input hyperparameter configurations and progressively fitting surrogate models over the input-output space until exhausting a user-defined budget of evaluations. We evaluate the efficacy of these methods relative to approaches such as random search, genetic algorithms, Bayesian optimization, and hyperband on DL benchmarks on CPU- and GPU-based HPC systems.

Balaprakash, Prasanna↗

DOE OSTI2018

The Future of Software and Computing for HEP: Pushing the Boundaries of the Possible

The nature of computing is changing. Driven by the solid state physics of CPU technology, industry is moving to multi-core systems with less memory, power, and memory bandwidth per core. The result is that the pleasantly parallel HEP event processing paradigm has to adjust to this new reality. New techniques such as machine learning and algorithms capable of exploiting vector processors, will be needed. Advances in instrumentation enabling fine-granularity high-precision measurements, have driven a data revolution. In this paper I will capture the needs, and where we are in addressing these data and compute challenges.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

DOE OSTI2022

Profiling the BLAST bioinformatics application for load balancing on high-performance computing clusters

Abstract Background The Basic Local Alignment Search Tool (BLAST) is a suite of commonly used algorithms for identifying matches between biological sequences. The user supplies a database file and query file of sequences for BLAST to find identical sequences between the two. The typical millions of database and query sequences make BLAST computationally challenging but also well suited for parallelization on high-performance computing clusters. The efficacy of parallelization depends on the data partitioning, where the optimal data partitioning relies on an accurate performance model. In previous studies, a BLAST job was sped up by 27 times by partitioning the database and query among thousands of processor nodes. However, the optimality of the partitioning method was not studied. Unlike BLAST performance models proposed in the literature that usually have problem size and hardware configuration as the only variables, the execution time of a BLAST job is a function of database size, query size, and hardware capability. In this work, the nucleotide BLAST application BLASTN was profiled using three methods: shell-level profiling with the Unix “time” command, code-level profiling with the built-in “profiler” module, and system-level profiling with the Unix “gprof” program. The runtimes were measured for six node types, using six different database files and 15 query files, on a heterogeneous HPC cluster with 500+ nodes. The empirical measurement data were fitted with quadratic functions to develop performance models that were used to guide the data parallelization for BLASTN jobs. Results Profiling results showed that BLASTN contains more than 34,500 different functions, but a single function, RunMTBySplitDB, takes 99.12% of the total runtime. Among its 53 child functions, five core functions were identified to make up 92.12% of the overall BLASTN runtime. Based on the performance models, static load balancing algorithms can be applied to the BLASTN input data to minimize the runtime of the longest job on an HPC cluster. Four test cases being run on homogeneous and heterogeneous clusters were tested. Experiment results showed that the runtime can be reduced by 81% on a homogeneous cluster and by 20% on a heterogeneous cluster by re-distributing the workload. Discussion Optimal data partitioning can improve BLASTN’s overall runtime 5.4-fold in comparison with dividing the database and query into the same number of fragments. The proposed methodology can be used in the other applications in the BLAST+ suite or any other application as long as source code is available.

59 BASIC BIOLOGICAL SCIENCES↗

DOE OSTI2014

Load Balancing Scientific Applications

The largest supercomputers have millions of independent processors, and concurrency levels are rapidly increasing. For ideal efficiency, developers of the simulations that run on these machines must ensure that computational work is evenly balanced among processors. Assigning work evenly is challenging because many large modern parallel codes simulate behavior of physical systems that evolve over time, and their workloads change over time. Furthermore, the cost of imbalanced load increases with scale because most large-scale scientific simulations today use a Single Program Multiple Data (SPMD) parallel programming model, and an increasing number of processors will wait for the slowest one at the synchronization points. To address load imbalance, many large-scale parallel applications use dynamic load balance algorithms to redistribute work evenly. The research objective of this dissertation is to develop methods to decide when and how to load balance the application, and to balance it effectively and affordably. We measure and evaluate the computational load of the application, and develop strategies to decide when and how to correct the imbalance. Depending on the simulation, a fast, local load balance algorithm may be suitable, or a more sophisticated and expensive algorithm may be required. We developed a model for comparison of load balance algorithms for a specific state of the simulation that enables the selection of a balancing algorithm that will minimize overall runtime.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2018

Modular Spiking Neural Circuits for Mapping Long Short-Term Memory on a Neurosynaptic Processor

Due to the distributed and asynchronous nature of neural computation through low-energy spikes, brain-inspired hardware systems offer high energy efficiency and massive parallelism. One such platform is the IBM TrueNorth neurosynaptic system. Recently, TrueNorth compatible representation learning algorithms have emerged, achieving close to the state-of-the-art performance in various data sets. However, its application in temporal sequence processing models, such as recurrent neural networks (RNNs), is still only at the proof of concept level. There is an inherent difficulty in capturing temporal dynamics of an RNN using spiking neurons, which is only exasperated by the hardware constraints in connectivity and synaptic weight resolution. This paper presents a design flow that overcomes these difficulties and maps a special case of recurrent networks called long short-term memory (LSTM) onto a spike-based platform. The framework utilizes various approximation techniques, such as activation discretization, weight quantization, and scaling and rounding, spiking neural circuits that implement the complex gating mechanisms, and a store-and-release technique to enable neuron synchronization and faithful storage. While the presented techniques can be applied to map LSTM to any spiking neural network (SNN) simulator/emulator, here we choose the TrueNorth chip as the target platform by adhering to its hardware constraints. Three LSTM applications, parity check, extended Reber grammar, and question classification, are evaluated. The tradeoffs among accuracy, performance, and energy tradeoffs achieved on TrueNorth are demonstrated. Finally, this is compared with the performance on an SNN platform without hardware constraints, which represents the upper bound of the achievable accuracy.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2018

Parallelizable adjoint stencil computations using transposed forward-mode algorithmic differentiation

Algorithmic differentiation (AD) is a tool for generating discrete adjoint solvers, which efficiently compute gradients of functions with many inputs, for example for use in gradient-based optimization. AD is often applied to large computations such as stencil operators, which are an important part of most structured-mesh PDE solvers. Stencil computations are often parallelized, for example by using OpenMP, and optimized by using techniques such as cache-blocking and tiling to fully utilize multicore CPUs and many-core accelerators and GPUs. Differentiating these codes with conventional reverse-mode AD results in adjoint codes that cannot be expressed as stencil operations and may not be easily parallelizable. They thus leave most of the compute power of modern architectures unused. We present a method that combines forward-mode AD and loop transformation to generate adjoint solvers that use the same memory access pattern as the original computation that they are derived from and can benefit from the same optimization techniques. The effectiveness of this method is demonstrated by generating a scalable adjoint CFD solver for multicore CPUs and Xeon Phi accelerators.

algorithmic differentiation↗

NASA NTRS1989

A survey on the design of multiprocessing systems for artificial intelligence applications

Some issues in designing computers for artificial intelligence (AI) processing are discussed. These issues are divided into three levels: the representation level, the control level, and the processor level. The representation level deals with the knowledge and methods used to solve the problem and the means to represent it. The control level is concerned with the detection of dependencies and parallelism in the algorithmic and program representations of the problem, and with the synchronization and sheduling of concurrent tasks. The processor level addresses the hardware and architectural components needed to evaluate the algorithmic and program representations. Solutions for the problems of each level are illustrated by a number of representative systems. Design decisions in existing projects on AI computers are classed into top-down, bottom-up, and middle-out approaches.

Wah, Benjamin W.↗

DOE OSTI2021

Efficient quadrature rules for finite element discretizations of nonlocal equations

Here, in this paper, we design efficient quadrature rules for finite element (FE) discretizations of nonlocal diffusion problems with compactly supported kernel functions. Two of the main challenges in nonlocal modeling and simulations are the prohibitive computational cost and the nontrivial implementation of discretization schemes, especially in three-dimensional settings. In this work, we circumvent both challenges by introducing a parametrized mollifying function that improves the regularity of the integrand, utilizing an adaptive integration technique, and exploiting parallelization. We first show that the “mollified” solution converges to the exact one as the mollifying parameter vanishes, then we illustrate the consistency and accuracy of the proposed method on several two- and three-dimensional test cases. Furthermore, we demonstrate the good scaling properties of the parallel implementation of the adaptive algorithm and we compare the proposed method with recently developed techniques for efficient FE assembly.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2021

Computable and Operationally Meaningful Multipartite Entanglement Measures

Multipartite entanglement is an essential resource for quantum communication, quantum computing, quantum sensing, and quantum networks. The utility of a quantum state |ψ$\rangle$ for these applications is often directly related to the degree or type of entanglement present in |ψ$\rangle$. Therefore, efficiently quantifying and characterizing multipartite entanglement is of paramount importance. Here, in this work, we introduce a family of multipartite entanglement measures, called concentratable entanglements. Several well-known entanglement measures are recovered as special cases of our family of measures, and hence we provide a general framework for quantifying multipartite entanglement. We prove that the entire family does not increase, on average, under local operations and classical communications. We also provide an operational meaning for these measures in terms of probabilistic concentration of entanglement into Bell pairs. Finally, we show that these quantities can be efficiently estimated on a quantum computer by implementing a parallelized SWAP test, opening up a research direction for measuring multipartite entanglement on quantum devices.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

NASA NTRS1996

I/O-Efficient Scientific Computation Using TPIE

In recent years, input/output (I/O)-efficient algorithms for a wide variety of problems have appeared in the literature. However, systems specifically designed to assist programmers in implementing such algorithms have remained scarce. TPIE is a system designed to support I/O-efficient paradigms for problems from a variety of domains, including computational geometry, graph algorithms, and scientific computation. The TPIE interface frees programmers from having to deal not only with explicit read and write calls, but also the complex memory management that must be performed for I/O-efficient computation. In this paper we discuss applications of TPIE to problems in scientific computation. We discuss algorithmic issues underlying the design and implementation of the relevant components of TPIE and present performance results of programs written to solve a series of benchmark problems using our current TPIE prototype. Some of the benchmarks we present are based on the NAS parallel benchmarks while others are of our own creation. We demonstrate that the central processing unit (CPU) overhead required to manage I/O is small and that even with just a single disk, the I/O overhead of I/O-efficient computation ranges from negligible to the same order of magnitude as CPU time. We conjecture that if we use a number of disks in parallel this overhead can be all but eliminated.

Vengroff, Darren Erik↗

DOE OSTI2023

Sparse Symmetric Format for Tucker Decomposition

Tensor-based methods are receiving renewed attention in recent years due to their prevalence in diverse real-world applications. There is considerable literature on tensor representations and algorithms for tensor decompositions, both for dense and sparse tensors. Many applications in hypergraph analytics, machine learning, psychometry, and signal processing result in tensors that are both sparse and symmetric, making them an important class for further study. Similar to the critical Tensor Times Matrix chain operation (TTM c ) in general sparse tensors, the $\underline{S}$ parse $\underline{S}$ ymmetric $\underline{T}$ ensor $\underline{T}$ imes $\underline{S}$ ame $\underline{M}$ atrix $\underline{c}$ hain (S 3 TTM c ) operation is compute and memory intensive due to high tensor order and the associated factorial explosion in the number of non-zeros. We present the novel Compressed Sparse Symmetric (CSS) format for sparse symmetric tensors, along with an efficient parallel algorithm for the S 3 TTM c operation. We theoretically establish that S 3 TTM c on CSS achieves a better memory versus run-time trade-off compared to state-of-the-art implementations, and visualize the variation of the performance gap over the parameter space. We demonstrate experimental findings that confirm these results and achieve up to 2.72× speedup on synthetic and real datasets. The scaling of the algorithm on different test architectures is also showcased to highlight the effect of machine characteristics on algorithm performance.

42 ENGINEERING↗

NASA NTRS1990

Transient finite element computations on the transputer system

The aim was to study the solution of transient finite element problems on the Transputer system of parallel processors. The central difference time integration rule was used so that no equation solving was necessary. Also investigated was subcycling time integration which uses different time steps in different subdomains of the finite element mesh. A one-dimensional bar problem was analyzed using the parallel time integration algorithm. This involves subdividing the bar into subproblems which are assigned to different processors. Results show that the significant speed-up can be obtained through parallel processing. Also subcycling can give an additional speed-up in certain classes of problems. A two-dimensional problem was also examined to evaluate the effect of the communication to computation ratio on solution time.

Smolinski, Patrick J.↗

DOE OSTI2017

Scaling deep learning workloads: NVIDIA DGX-1/Pascal and Intel Knights Landing

Deep Learning (DL) algorithms have become ubiquitous in data analytics. As a result, major computing vendors --- including NVIDIA, Intel, AMD, and IBM --- have architectural road-maps influenced by DL workloads. Furthermore, several vendors have recently advertised new computing products as accelerating large DL workloads. Unfortunately, it is difficult for data scientists to quantify the potential of these different products. This paper provides a performance and power analysis of important DL workloads on two major parallel architectures: NVIDIA DGX-1 (eight Pascal P100 GPUs interconnected with NVLink) and Intel Knights Landing (KNL) CPUs interconnected with Intel Omni-Path or Cray Aries. Our evaluation consists of a cross section of convolutional neural net workloads: CifarNet, AlexNet, GoogLeNet, and ResNet50 topologies using the Cifar10 and ImageNet datasets. The workloads are vendor-optimized for each architecture. Our analysis indicates that although GPUs provide the highest overall performance, the gap can close for some convolutional networks; and the KNL can be competitive in performance/watt. We find that NVLink facilitates scaling efficiency on GPUs. However, its importance is heavily dependent on neural network architecture. Furthermore, for weak-scaling --- sometimes encouraged by restricted GPU memory --- NVLink is less important.

Gawande, Nitin A.↗

DOE OSTI2014

The RAJA Portability Layer: Overview and Status

As computer architectures become increasingly complex and diverse, application developers face difficult challenges to achieve high performance while maintaining code portability. The problem is especially acute for large ASC multiphysics codes. Efficient parallel execution often requires tuning algorithms and data access to match processor and memory system constraints. Changing compiler directives and parallel programming model constructs on thousands of individual loops in a large code is disruptive and unwieldy. RAJA is a programming approach that we have been developing at Lawrence Livermore National Laboratory to encapsulate platform-specific concerns, related to both hardware and parallel programming models. The RAJA abstraction layer simplifies porting C/C++ codes to various programming models and architectures by reducing effort and developer disruption. In this report, we motivate and describe key aspects of RAJA. We also present a preliminary assessment of RAJA based on exploration in three ASC hydrodynamics codes at LLNL, which was one part of a three-part ASC Level 2 milestone, completed in September 2014.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2014

Stochastic simulation of electron avalanches on supercomputers

In the paper, we present a three-dimensional parallel Monte Carlo algorithm named ELSHOW which is developed for simulation of electron avalanches in gases. Parallel implementation of the ELSHOW was made on supercomputers with different architectures (massive parallel and hybrid ones). Using the ELSHOW, calculations of such integral characteristics as the number of particles in an avalanche, the coefficient of impact ionization, the drift velocity, and the others were made. Also, special precise computations were made to select an appropriate size of the time step using the technique of dependent statistical tests. Particularly, the algorithm consists of special methods of distribution modeling, a lexicographic implementation scheme for “branching” of trajectories, justified estimation of functionals. A comparison of the obtained results for nitrogen with previously published theoretical and experimental data was made.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

NASA NTRS1976

Modeling of three-dimensional mixing and reacting ducted flows

A computer code, based upon a finite element solution algorithm, was developed to solve the governing equations for three-dimensional, reacting boundary region, and constant area ducted flow fields. Effective diffusion coefficients are employed to allow analyses of turbulent, transitional or laminar flows. The code was used to investigate mixing and reacting hydrogen jets injected from multiple orifices, transverse and parallel to a supersonic air stream. Computational results provide a three-dimensional description of velocity, temperature, and species-concentration fields downstream of injection. Experimental data for eight cases covering different injection conditions and geometries were modeled using mixing length theory (MLT). These results were used as a baseline for examining the relative merits of other mixing models. Calculations were made using a two-equation turbulence model (k+d) and comparisons were made between experiment and mixing length theory predictions. The k+d model shows only a slight improvement in predictive capability over MLT. Results of an examination of the effect of tensorial transport coefficients on mass and momentum field distribution are also presented. Solutions demonstrating the ability of the code to model ducted flows and parallel strut injection are presented and discussed.

Zelazny, S. W.↗

DOE OSTI2022

Two-dimensional materials for bio-realistic neuronal computing networks

Two-dimensional (2D) van der Waals materials have found broad utility in a diverse range of applications including electronics, optoelectronics, renewable energy, and quantum information technologies. Meanwhile, exponentially growing digital data coupled with the ubiquity of artificial intelligence algorithms have generated significant interest in edge neuromorphic computing as an alternative to centralized cloud computing. The drive to incorporate neuroscience principles into computing hardware is motivated by the low power consumption, parallel processing, and reconfigurability of the human brain. The diverse library of 2D materials with atomic-level thicknesses, exceptional electrostatic tunability, and integration versatility is particularly well-suited for realizing bio-realistic synaptic and neuronal functionality. Here, we summarize past and present work in this field and outline the frontier challenges that have not yet been overcome. Here we also delineate potential solutions and suggest that the neuroscience principles of criticality and synchrony have the potential to inspire breakthrough applications of 2D materials in neuronal computing networks.

36 MATERIALS SCIENCE↗

DOE OSTI2020

Virtual-Voltage Partition-Based Approach to Optimal Transmission Switching

This article deals with optimal transmission switching (OTS) problems involving binary decisions about network topology and nonconvex power flow constraints. We adopt a semidefinite programming formulation for the optimal power flow (OPF) problem that, however, remains nonconvex due to the presence of discrete variables and bilinear products between the decision variables. To tackle the latter, we introduce a physically inspired, virtual-voltage approximation that leads to provable lower and upper bounds on the solution of the original problem. To deal with the exponential complexity caused by the discrete variables, we introduce a graph partition-based algorithm that breaks the problem into several parallel mixed-integer subproblems of smaller size. Simulations on the IEEE bus test cases demonstrate the high degree of accuracy and affordable computational requirements of our approach.

42 ENGINEERING↗