Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “cluster computing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 343 records · Page 19
All sourcesNASA NTRSDOE OSTI
DOE OSTI2019

State-based network similarity visualization

We introduce an approach for the interactive visual analysis of weighted, dynamic networks. These networks arise in areas such as computational neuroscience, sociology, and biology. Network analysis remains challenging due to complex time-varying network behavior. For example, edges disappear/reappear, communities grow/vanish, or overall network topology changes. Our technique, TimeSum, detects the important topological changes in graph data to abstract the dynamic network and visualize one summary representation for each temporal phase, a state. We define a network state as a graph with similar topology over a specific time interval. To enable a holistic comparison of networks, we use a difference network to depict edge and community changes. Here, we present case studies to demonstrate that our methods are effective and useful for extracting and exploring complex dynamic behavior of networks.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
DOE OSTI2019

MetaBAT 2: an adaptive binning algorithm for robust and efficient genome reconstruction from metagenome assemblies

We previously reported on MetaBAT, an automated metagenome binning software tool to reconstruct single genomes from microbial communities for subsequent analyses of uncultivated microbial species. MetaBAT has become one of the most popular binning tools largely due to its computational efficiency and ease of use, especially in binning experiments with a large number of samples and a large assembly. MetaBAT requires users to choose parameters to fine-tune its sensitivity and specificity. If those parameters are not chosen properly, binning accuracy can suffer, especially on assemblies of poor quality. Here, we developed MetaBAT 2 to overcome this problem. MetaBAT 2 uses a new adaptive binning algorithm to eliminate manual parameter tuning. We also performed extensive software engineering optimization to increase both computational and memory efficiency. Comparing MetaBAT 2 to alternative software tools on over 100 real world metagenome assemblies shows superior accuracy and computing speed. Binning a typical metagenome assembly takes only a few minutes on a single commodity workstation. We therefore recommend the community adopts MetaBAT 2 for their metagenome binning experiments. MetaBAT 2 is open source software and available at https://bitbucket.org/berkeleylab/metabat .

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2016

Architecture and method for a burst buffer using flash technology

A parallel supercomputing cluster includes compute nodes interconnected in a mesh of data links for executing an MPI job, and solid-state storage nodes each linked to a respective group of the compute nodes for receiving checkpoint data from the respective compute nodes, and magnetic disk storage linked to each of the solid-state storage nodes for asynchronous migration of the checkpoint data from the solid-state storage nodes to the magnetic disk storage. Each solid-state storage node presents a file system interface to the MPI job, and multiple MPI processes of the MPI job write the checkpoint data to a shared file in the solid-state storage in a strided fashion, and the solid-state storage node asynchronously migrates the checkpoint data from the shared file in the solid-state storage to the magnetic disk storage and writes the checkpoint data to the magnetic disk storage in a sequential fashion.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2015

Improving Block-level Efficiency with scsi-mq

Current generation solid-state storage devices are exposing a new bottlenecks in the SCSI and block layers of the Linux kernel, where IO throughput is limited by lock contention, inefficient interrupt handling, and poor memory locality. To address these limitations, the Linux kernel block layer underwent a major rewrite with the blk-mq project to move from a single request queue to a multi-queue model. The Linux SCSI subsystem rework to make use of this new model, known as scsi-mq, has been merged into the Linux kernel and work is underway for dm-multipath support in the upcoming Linux 4.0 kernel. These pieces were necessary to make use of the multi-queue block layer in a Lustre parallel filesystem with high availability requirements. We undertook adding support of the 3.18 kernel to Lustre with scsi-mq and dm-multipath patches to evaluate the potential of these efficiency improvements. In this paper we evaluate the block-level performance of scsi-mq with backing storage hardware representative of a HPC-targerted Lustre filesystem. Our findings show that SCSI write request latency is reduced by as much as 13.6%. Additionally, when profiling the CPU usage of our prototype Lustre filesystem, we found that CPU idle time increased by a factor of 7 with Linux 3.18 and blk-mq as compared to a standard 2.6.32 Linux kernel. Our findings demonstrate increased efficiency of the multi-queue block layer even with disk-based caching storage arrays used in existing parallel filesystems.

Cluster Computing↗
DOE OSTI2024

A new “gold standard”: Perturbative triples corrections in unitary coupled cluster theory and prospects for quantum computing

A major difficulty in quantum simulation is the adequate treatment of a large collection of entangled particles, synonymous with electron correlation in electronic structure theory, with coupled cluster (CC) theory being the leading framework for dealing with this problem. Augmenting computationally affordable low-rank approximations in CC theory with a perturbative account of higher-rank excitations is a tractable and effective way of accounting for the missing electron correlation in those approximations. This is perhaps best exemplified by the “gold standard” CCSD(T) method, which bolsters the baseline CCSD with the effects of triple excitations using considerations from many-body perturbation theory (MBPT). Despite this established success, such a synergy between MBPT and the unitary analog of CC theory (UCC) has not been explored. In this work, we propose a similar approach wherein converged UCCSD amplitudes are leveraged to evaluate energy corrections associated with triple excitations, leading to the UCCSD[T] method. In terms of quantum computing, this correction represents an entirely classical post-processing step that improves the energy estimate by accounting for triple excitation effects without necessitating new quantum algorithm developments or increasing demand for quantum resources. The rationale behind this choice is shown to be rigorous by studying the properties of finite-order UCC energy functionals, and our efforts do not support the addition of the fifth-order contributions as in the (T) correction. We assess the performance of these approaches on a collection of small molecules and demonstrate the benefits of harnessing the inherent synergy between MBPT and UCC theories.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2017

Investigation of the cluster formation in lithium niobate crystals by computer modeling method

The processes occurring upon the formation of energetically equilibrium oxygen-octahedral clusters in the ferroelectric phase of a stoichiometric lithium niobate (LiNbO{sub 3}) crystal have been investigated by the computer modeling method within the semiclassical atomistic model. An energetically favorable cluster size (at which a structure similar to that of a congruent crystal is organized) is shown to exist. A stoichiometric cluster cannot exist because of the electroneutrality loss. The most energetically favorable cluster is that with a Li/Nb ratio of about 0.945, a value close to the lithium-to-niobium ratio for a congruent crystal.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
DOE OSTI2020

Density Matrix Quantum Circuit Simulation via the BSP Machine on Modern GPU Clusters

As quantum computers evolve, simulations of quantum programs on classical computers will be essential in validating quantum algorithms, understanding the effect of system noise, and design applications for future quantum computers. In this paper, we propose a novel multi-GPU programming model called MG-BSP that constructs a virtual BSP machine on top of modern multi-GPU platforms, and tweak the programming model to build a multi-GPU density matrix quantum simulator. We propose and evaluated a new formulation minimizing communication and prove that the transformation conserves original semantics when noise is introduced. We build the tool-chain to support quantum assembly open standard, synthesize testing quantum circuit, and enable ultra-deep quantum simulation. We evaluated our design on four state-of-the-art multi-GPU platforms including the latest DGX-1 and DGX-2 systems. We demonstrate simulation of 1 million gates in 94 minutes, far deeper circuits than has been demonstrated in prior work. A roofline-model analysis show that we have reached near-optimal performance under memory bound.

Li, Ang↗
DOE OSTI2016

Breaking the bottleneck: Use of molecular tailoring approach for the estimation of binding energies at MP2/CBS limit for large water clusters

A pragmatic method based on the molecular tailoring approach (MTA) for estimating the complete basis set (CBS) limit at Møller-Plesset second order perturbation (MP2) theory accurately for large molecular clusters with limited computational resources is developed. It is applied to water clusters, (H{sub 2}O){sub n} (n = 7, 8, 10, 16, 17, and 25) optimized employing aug-cc-pVDZ (aVDZ) basis-set. Binding energies (BEs) of these clusters are estimated at the MP2/aug-cc-pVNZ (aVNZ) [N = T, Q, and 5 (whenever possible)] levels of theory employing grafted MTA (GMTA) methodology and are found to lie within 0.2 kcal/mol of the corresponding full calculation MP2 BE, wherever available. The results are extrapolated to CBS limit using a three point formula. The GMTA-MP2 calculations are feasible on off-the-shelf hardware and show around 50%–65% saving of computational time. The methodology has a potential for application to molecular clusters containing ∼100 atoms.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2015

STAR CLUSTER PROPERTIES IN TWO LEGUS GALAXIES COMPUTED WITH STOCHASTIC STELLAR POPULATION SYNTHESIS MODELS

We investigate a novel Bayesian analysis method, based on the Stochastically Lighting Up Galaxies (slug) code, to derive the masses, ages, and extinctions of star clusters from integrated light photometry. Unlike many analysis methods, slug correctly accounts for incomplete initial mass function (IMF) sampling, and returns full posterior probability distributions rather than simply probability maxima. We apply our technique to 621 visually confirmed clusters in two nearby galaxies, NGC 628 and NGC 7793, that are part of the Legacy Extragalactic UV Survey (LEGUS). LEGUS provides Hubble Space Telescope photometry in the NUV, U, B, V, and I bands. We analyze the sensitivity of the derived cluster properties to choices of prior probability distribution, evolutionary tracks, IMF, metallicity, treatment of nebular emission, and extinction curve. We find that slug's results for individual clusters are insensitive to most of these choices, but that the posterior probability distributions we derive are often quite broad, and sometimes multi-peaked and quite sensitive to the choice of priors. In contrast, the properties of the cluster population as a whole are relatively robust against all of these choices. We also compare our results from slug to those derived with a conventional non-stochastic fitting code, Yggdrasil. We show that slug's stochastic models are generally a better fit to the observations than the deterministic ones used by Yggdrasil. However, the overall properties of the cluster populations recovered by both codes are qualitatively similar.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2018

Optimizing Tensor Contractions in CCSD(T) for Efficient Execution on GPUs

Tensor contractions are higher dimensional analogs of matrix multiplications, used in many computational contexts such as high order models in quantum chemistry, deep learning, nite element methods etc. In contrast to the wide availability of high-performance libraries for matrix multiplication on GPUs, the same is not true for tensor contractions. In this paper, we address the optimization of a set of symmetrized tensor contractions that form the computational bottleneck in the CCSD(T) coupled-cluster method in computational chemistry suites like NWChem. Some of the challenges in optimizing tensor contractions that arise in practice from the variety of dimensionalities and shapes for tensors include effective mapping of the high-dimensional iteration space to threads, choice of data buffering in shared-memory and registers, and tile sizes for multi-level tiling. Furthermore, in the case of symmetrized tensor contractions in CCSD(T), it is also a challenge to fuse contractions to reduce data movement cost by exploiting reuse of intermediate tensors. In this paper, we developed an efficient GPU implementation of the tensor contractions in CCSD(T) using shared-memory buffering, register tiling, loop fusion and register transpose. Experimental results demonstrate significant improvement over the current state-of-the-art.

Kim, Jinsung↗
DOE OSTI2022

Spectral function for 4 He using the Chebyshev expansion in coupled-cluster theory

Here, we compute spectral function for 4 He by combining coupled-cluster theory with an expansion of integral transforms into Chebyshev polynomials. Our method allows us to estimate the uncertainty of spectral reconstruction. The properties of the Chebyshev polynomials make the procedure numerically stable and considerably lower in memory usage than the typically employed Lanczos algorithm. We benchmark our predictions with other calculations in the literature and with electron-scattering data in the quasi-elastic peak. The spectral function formalism allows one to extend ab initio lepton-nucleus cross sections into the relativistic regime. This makes it a promising tool for modeling this process at higher-energy transfers. The results we present open the door for studies of heavier nuclei, important for the neutrino oscillation programs.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2024

PANDORA: A Parallel Dendrogram Construction Algorithm for Single Linkage Clustering on GPU

This paper introduces Pandora, a parallel algorithm for computing dendrograms, the hierarchical cluster trees for single linkage clustering (SLC). Current parallel approaches construct dendrograms by partitioning a minimum spanning tree and removing edges. However, they struggle with skewed, hard-to-parallelize real-world dendrograms. Consequently, computing dendrograms is the sequential bottleneck in HDBSCAN*[21], a popular SLC variant. Pandora uses recursive tree contraction to address this limitation. Pandora contracts nodes to construct progressively smaller trees. It computes the smallest contracted dendrogram and expands it by inserting contracted edges. This recursive strategy is highly parallel, skew-independent, work-optimal, and well-suited for GPUs and multicores. We develop a performance portable implementation of Pandora in Kokkos[31] and evaluate its performance on multicore CPUs and multi-vendor GPUs (e.g., Nvidia, AMD) for dendrogram construction in HDBSCAN*. Multithreaded Pandora is 2.2x faster than the current best-multithreaded implementation. Our GPU version achieves 6-20x speedup on AMD GPUs and 10-37x on NVIDIA GPUs over multithreaded Pandora. Pandora removes HDBSCAN*’s sequential bottleneck, greatly boosting efficiency, particularly with GPUs.

Sao, Piyush↗
DOE OSTI2021

Dimensionality reduction of the many-body problem using coupled-cluster subsystem flow equations: classical and quantum computing perspective

We discuss reduced-scaling strategies employing recently introduced sub-system embedding sub-algebras coupled-cluster formalism (SES-CC) to describe many-body systems. These strategies utilize properties of the SES-CC formulations where the equations describing certain classes of sub- systems can be integrated into a computational flows composed coupled eigenvalue problems of reduced dimensionality. Additionally, these flows can be defined at the level of the CC Ansatz defined by selected classes of cluster amplitudes, which define the wave function ”memory” of possible partitionings of the many-body system into constituent sub-systems. One of the possible ways of solving these coupled problems is through implementing procedures, where the information is passed between the sub-systems in a self-consistent manner. As a special case, we consider local flow formulations where the so-called local character of correlation effects can be closely related to properties of sub-system embedding sub-algebras employing localized molecular basis. We also generalize flow equations to the time domain and to downfolding methods utilizing double exponential unitary CC Ansatz (DUCC), where reduced dimensionality of constituent sub-problems offer a possibility of efficient utilization of limited quantum resources in modeling realistic systems.

Electron correlation, quantum chemistry, quantum c↗