SEARCH · Engineering Papers

Results for “parallel algorithms”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,099 records · Page 61

All sources NASA NTRS DOE OSTI

DOE OSTI2014

Xyce TM Parallel Electronic Simulator Users’ Guide, Version 6.1

This manual describes the use of the Xyce Parallel Electronic Simulator. Xyce has been designed as a SPICE-compatible, high-performance analog circuit simulator, and has been written to support the simulation needs of the Sandia National Laboratories electrical designers. This development has focused on improving capability over the current state-of-the-art in the following areas; Capability to solve extremely large circuit problems by supporting large-scale parallel computing platforms (up to thousands of processors). This includes support for most popular parallel and serial computers; A differential-algebraic-equation (DAE) formulation, which better isolates the device model package from solver algorithms. This allows one to develop new types of analysis without requiring the implementation of analysis-specific device models; Device models that are specifically tailored to meet Sandia's needs, including some radiation-aware devices (for Sandia users only); and Object-oriented code design and implementation using modern coding practices. Xyce is a parallel code in the most general sense of the phrase-a message passing parallel implementation-which allows it to run efficiently a wide range of computing platforms. These include serial, shared-memory and distributed-memory parallel platforms. Attention has been paid to the specific nature of circuit-simulation problems to ensure that optimal parallel efficiency is achieved as the number of processors grows.

42 ENGINEERING↗

DOE OSTI2019

A Parallel-in-Time Multigrid Approach to Constrained Optimization

Valuable computer simulations such as weather prediction, traffic flow, and resource allocation are time dependent problems. One such example is the viscous Burgers’ equation which can model various systems such as fluids and traffic flow. Further, many of these models require optimization routines to meet the needs of many businesses. Unfortunately, traditional sequential-in-time algorithms have become bottlenecks in solution model generation. This is because computer architectures have begun to rely more and more on concurrency (parallel processing) as opposed to processor power to provide greater performance. To avoid this bottleneck, many researchers use parallelization methods to take advantage of the increasing number of processors in computers. This approach allows for variables in a problem to be solved for simultaneously. As such, parallelization is a crucial requirement to achieve high-performance computing.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2014

ENZO: AN ADAPTIVE MESH REFINEMENT CODE FOR ASTROPHYSICS

This paper describes the open-source code Enzo, which uses block-structured adaptive mesh refinement to provide high spatial and temporal resolution for modeling astrophysical fluid flows. The code is Cartesian, can be run in one, two, and three dimensions, and supports a wide variety of physics including hydrodynamics, ideal and non-ideal magnetohydrodynamics, N-body dynamics (and, more broadly, self-gravity of fluids and particles), primordial gas chemistry, optically thin radiative cooling of primordial and metal-enriched plasmas (as well as some optically-thick cooling models), radiation transport, cosmological expansion, and models for star formation and feedback in a cosmological context. In addition to explaining the algorithms implemented, we present solutions for a wide range of test problems, demonstrate the code's parallel performance, and discuss the Enzo collaboration's code development methodology.

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2015

A partially reflecting random walk on spheres algorithm for electrical impedance tomography

In this work, we develop a probabilistic estimator for the voltage-to-current map arising in electrical impedance tomography. This novel so-called partially reflecting random walk on spheres estimator enables Monte Carlo methods to compute the voltage-to-current map in an embarrassingly parallel manner, which is an important issue with regard to the corresponding inverse problem. Our method uses the well-known random walk on spheres algorithm inside subdomains where the diffusion coefficient is constant and employs replacement techniques motivated by finite difference discretization to deal with both mixed boundary conditions and interface transmission conditions. We analyze the global bias and the variance of the new estimator both theoretically and experimentally. Subsequently, the variance of the new estimator is considerably reduced via a novel control variate conditional sampling technique which yields a highly efficient hybrid forward solver coupling probabilistic and deterministic algorithms.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

DOE OSTI2018

Modular Spiking Neural Circuits for Mapping Long Short-Term Memory on a Neurosynaptic Processor

Due to the distributed and asynchronous nature of neural computation through low-energy spikes, brain-inspired hardware systems offer high energy efficiency and massive parallelism. One such platform is the IBM TrueNorth neurosynaptic system. Recently, TrueNorth compatible representation learning algorithms have emerged, achieving close to the state-of-the-art performance in various data sets. However, its application in temporal sequence processing models, such as recurrent neural networks (RNNs), is still only at the proof of concept level. There is an inherent difficulty in capturing temporal dynamics of an RNN using spiking neurons, which is only exasperated by the hardware constraints in connectivity and synaptic weight resolution. This paper presents a design flow that overcomes these difficulties and maps a special case of recurrent networks called long short-term memory (LSTM) onto a spike-based platform. The framework utilizes various approximation techniques, such as activation discretization, weight quantization, and scaling and rounding, spiking neural circuits that implement the complex gating mechanisms, and a store-and-release technique to enable neuron synchronization and faithful storage. While the presented techniques can be applied to map LSTM to any spiking neural network (SNN) simulator/emulator, here we choose the TrueNorth chip as the target platform by adhering to its hardware constraints. Three LSTM applications, parity check, extended Reber grammar, and question classification, are evaluated. The tradeoffs among accuracy, performance, and energy tradeoffs achieved on TrueNorth are demonstrated. Finally, this is compared with the performance on an SNN platform without hardware constraints, which represents the upper bound of the achievable accuracy.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2018

Tomography in Optically Axisymmetric Media

A generalization of the Cormac algorithm for the inversion of the Radon transformation in an optical medium with an axisymmetric index of refraction has been proposed. The distribution of rays is such that only one ray passes through any two points in the circle. A “parallel scanning scheme” of tomography has been considered: a cylindrical object is illuminated by a parallel beam of light, the rays of which are deflected into the cylinder. There is no refraction on the surface of the cylinder. The algorithm assumes the possibility of attenuation on a ray, which also has an axisymmetric character. Such a type of ray deflection occurs in problematic issues of the tomography of GRADANs, light guides, and plasma.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

DOE OSTI2014

fast-matmul

This software provides implementations of fast matrix multiplication algorithms. These algorithms perform fewer floating point operations than the classical cubic algorithm. The software uses code generation to automatically implement the fast algorithms based on high-level descriptions. The code serves two general purposes. The first is to demonstrate that these fast algorithms can out-perform vendor matrix multiplication algorithms for modest problem sizes on a single machine. The second is to rapidly prototype many variations of fast matrix multiplication algorithms to encourage future research in this area. The implementations target sequential and shared memory parallel execution.

Ballard, Grey↗

DOE OSTI2018

PyBioNetFit

Systems biology models written in the BioNetGen language often contain parameters that must be fit based on experimental data. PyBioNetFit is a software tool designed to fit parameters of BioNetGen models. The user is expected to provide a BioNetGen model that contains unknown parameters, along with one or more text files containing experimental data. PyBioNetFit will perform a series of calls to the BioNetGen simulator using different parameter sets, to identify the parameter set that best fits the data. The parameter search is dictated by the user's choice of one of several metaheuristic optimization algorithms, including differential evolution, particle swarm optimization, scatter search, and simulated annealing. The software supports parallelized computing on multi-core computers and on computing clusters. We expect this software to be a useful tool for researchers designing BioNetGen models in collaboration with experimentalists.

Mitra, Eshan↗

DOE OSTI2023

LCA

The Locally Competitive Algorithm (LCA) is a dynamical sparse solver that uses only local computations, allowing for massively parallel implementations on compatible neuromorphic architectures such as Intel's Loihi research chip. In this invention, we show how unsupervised dictionary learning with spiking LCA can be implemented on GPUs and Intel's Loihi research chip.

Parpart, Gavin↗

DOE OSTI2026

JAXtronomy: A JAX port of lenstronomy

Gravitational lensing is a phenomenon where light bends around massive objects, resulting in distorted images seen by an observer. Studying gravitationally lensed systems provides insights into cosmology and astrophysics, including constraints of the expansion rate of the Universe and the distribution of dark matter. Thus, we introduce JAXtronomy, a re-implementation of the gravitational lensing software package lenstronomy (Birrer, 2021; Birrer & Amara, 2018) using JAX (Bradbury et al., 2018). JAX is a Python library that uses an accelerated linear algebra (XLA) compiler to improve the performance of computing software. Our core design principle of JAXtronomy is to maintain an identical API to that of lenstronomy. The main JAX features utilized in JAXtronomy are just-in-time compilation, which can lead to significant reductions in execution time, and automatic differentiation, which allows for the implementation of gradient-based algorithms that were previously impossible. Additionally, JAX allows code to be run on GPUs or parallelized across CPU cores, further boosting the performance of JAXtronomy.

astronomy↗

DOE OSTI2023

Quantum-inspired tempering for ground state approximation using artificial neural networks

A large body of work has demonstrated that parameterized artificial neural networks (ANNs) can efficiently describe ground states of numerous interesting quantum many-body Hamiltonians. However, the standard variational algorithms used to update or train the ANN parameters can get trapped in local minima, especially for frustrated systems and even if the representation is sufficiently expressive. We propose a parallel tempering method that facilitates escape from such local minima. This methods involves training multiple ANNs independently, with each simulation governed by a Hamiltonian with a different "driver" strength, in analogy to quantum parallel tempering, and it incorporates an update step into the training that allows for the exchange of neighboring ANN configurations. We study instances from two classes of Hamiltonians to demonstrate the utility of our approach using Restricted Boltzmann Machines as our parameterized ANN. The first instance is based on a permutation-invariant Hamiltonian whose landscape stymies the standard training algorithm by drawing it increasingly to a false local minimum. The second instance is four hydrogen atoms arranged in a rectangle, which is an instance of the second quantized electronic structure Hamiltonian discretized using Gaussian basis functions. We study this problem in a minimal basis set, which exhibits false minima that can trap the standard variational algorithm despite the problem’s small size. We show that augmenting the training with quantum parallel tempering becomes useful to finding good approximations to the ground states of these problem instances.

Albash, Tameem↗

DOE OSTI2014

fast-matmul

Grey Ballard, Austin Benson↗

DOE OSTI2014

Application of the TITAN Hybrid Deterministic Transport Code to Medical Physics - Paper 118

The TITAN code is a 3-dimensional parallel deterministic transport code that solves the linear Boltzmann equation. The TITAN code has hybrid algorithms that are ideal for medical physics applications: i) SN and characteristics method and ii) SN with a fictitious quadrature set and ray-tracing. These hybrid algorithms have been used in the simulation of computed tomography (CT) and single photon emission computed tomography (SPECT) In this paper, the hybrid algorithms developed for medical physics applications in the TITAN code are discussed. First, the TITAN code's block-oriented approach for utilizing SN and characteristic solvers for CT is presented and results for a simple test case are given. Second, the paper describes a hybrid formulation for SPECT with an SN solver in the phantom, a fictitious quadrature set for directions from the phantom surface passing through the collimator, and a simplified ray-tracing algorithm for particle transport to the detector. Finally, ongoing work is described and concluding remarks are given. The TITAN code's hybrid algorithms allow it to efficiently solve problems with low-scatter regions, which are frequently found in medical physics applications. The TITAN code has been used to solve a simplified CT problem and generate projection images for SPECT simulations. The WCOS technique was developed to improve the TITAN code's collimator representation. For these applications, the TITAN code was shown to require significantly less computation time than MCNP5. Work is currently underway to implement an iterative image reconstruction algorithm in the TITAN code. A script that calls the TITAN code to complete the forward projection step in SPECT has already been used to reconstruction a 2-dimensional phantom. The ability of TITAN to fully model particle transport in the patient in short computation times is expected to be a great advantage to iterative reconstruction. (authors)

61 RADIATION PROTECTION AND DOSIMETRY↗

DOE OSTI2014

MCNP6 Monte Carlo Code Optimization [Slides]

The MCNP6.1 Monte Carlo code, released in 2013, offers many features not available in previous versions, but runs 20-30% slower for most problems and 2-5x slower for some problems. The MCNP 2020 initiative was established in mid-2013 to address code performance and other issues, with 3 basic components: (1) Improve MCNP performance, with the specific goal of achieving a 2x performance gain within 2 years; (2) Upgrade the core MCNP software and algorithms in an evolutionary fashion, with the goal of achieving more agile and sustainable code; (3) Prepare for future exascale systems that will include massive parallelism and limited memory per core, with the goal of providing flexible and adaptable coding for possible exascale systems. This talk reviews the initial performance improvements to MCNP6.1 that have been incorporated into the 2014 update, MCNP6.1.1. The performance improvements to date have included both classic code optimizations and algorithm improvements. Testing on a variety of problems has demonstrated that the performance improvements were effective, yielding speedups by factors of 1.2x - 4x compared to MCNP6.1, depending on the type of problem. For criticality problems, MCNP6.1.1 runs 1.5x - 1.7x faster than MCNP6.1. Much more work is planned for improving MCNP6 performance, structure, and algorithms.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

DOE OSTI2015

Modeling Rare Events in High Dimensional Stochastics Systems

The main issue addressed in this note is the study of an algorithm to accelerate the computation of kinetic rates in the context of molecular dynamics(MD). It is based on parallel simulations of short-time trajectories and its main components are: a decomposition of phase space into macrostates or cells, a resampling procedure that ensures that the number of parallel replicas (MD simulations) in each macro-state remains constant, the use of multiple populations (colored replicas) to compute multiple rates (e.g., forward and backward rates) in one simulation. The method leads to enhancing the sampling of macro-states associated to the transition states, since in traditional MD these are likely to be depleted even after short-time simulations. By allowing parallel replicas to carry different probabilistic weights, the number of replicas within each macro-state can be maintained constant without introducing any bias. The empirical variance of the estimated reaction rate, defined as a probability flux, is expectedly diminished. This note is a first attempt towards a better mathematical and numerical understanding of this method. It provides first a mathematical formalization of the notion of colors. Then, the link between this algorithm and a set of closely related methods having been proposed in the literature within the past few years is discussed. Lastly, numerical results are provided that illustrate the efficiency of the method.

59 BASIC BIOLOGICAL SCIENCES↗

DOE OSTI2016

A template-based approach for parallel hexahedral two-refinement

Here, we provide a template-based approach for generating locally refined all-hex meshes. We focus specifically on refinement of initially structured grids utilizing a 2-refinement approach where uniformly refined hexes are subdivided into eight child elements. The refinement algorithm consists of identifying marked nodes that are used as the basis for a set of four simple refinement templates. The target application for 2-refinement is a parallel grid-based all-hex meshing tool for high performance computing in a distributed environment. The result is a parallel consistent locally refined mesh requiring minimal communication and where minimum mesh quality is greater than scaled Jacobian 0.3 prior to smoothing.

2-refinement↗

DOE OSTI2020

Algorithms for Efficient Reproducible Floating Point Summation

We define “reproducibility” as getting bitwise identical results from multiple runs of the same program, perhaps with different hardware resources or other changes that should not affect the answer. Many users depend on reproducibility for debugging or correctness. However, dynamic scheduling of parallel computing resources, combined with nonassociative floating point addition, makes reproducibility challenging even for summation, or operations like the BLAS. We describe a “reproducible accumulator” data structure (the “binned number”) and associated algorithms to reproducibly sum binary floating point numbers, independent of summation order. We use a subset of the IEEE Floating Point Standard 754-2008 and bitwise operations on the standard representations in memory. Our approach requires only one read-only pass over the data, and one reduction in parallel, using a 6-word reproducible accumulator (more words can be used for higher accuracy), enabling standard tiling optimization techniques. Summing n words with a 6-word reproducible accumulator requires approximately 9 n floating point operations (arithmetic, comparison, and absolute value) and approximately 3 n bitwise operations. The final error bound with a 6-word reproducible accumulator and our default settings can be up to 2 29 times smaller than the error bound for conventional (recursive) summation on ill-conditioned double-precision inputs.

Computer Science↗

DOE OSTI2022

A Length Adaptive Algorithm-Hardware Co-design of Transformer on FPGA Through Sparse Attention and Dynamic Pipelining

Transformers are considered one of the most important deep learning models since 2018, in part because it establishes state-of-the-art (SOTA) records and could potentially replace existing Deep Neural Networks (DNNs). Despite the remarkable triumphs, the prolonged turnaround time of Transformer models is a widely recognized roadblock. The variety of sequence lengths imposes additional computing overhead where inputs need to be zero-padded to the maximum sentence length in the batch to accommodate the parallel computing platforms. This paper targets the field-programmable gate array (FPGA) and proposes a coherent sequence length adaptive algorithm–hardware co-design for Transformer acceleration. Particularly, we develop a hardware-friendly sparse attention operator and a length-aware hardware resource scheduling algorithm. The proposed sparse attention operator brings the complexity of attention-based models down to linear complexity and alleviates the off-chip memory traffic. The proposed length-aware resource hardware scheduling algorithm dynamically allocates the hardware resources to fill up the pipeline slots and eliminates bubbles for NLP tasks. Experiments show that our design has very small accuracy loss and has 80.2 × and 2.6 × speedup compared to CPU and GPU implementation, and 4 × higher energy efficiency than state-of-the-art GPU accelerator optimized via CUBLAS GEMM.

Peng, Hongwu↗