Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “high performance parallel computing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,531 records · Page 85
All sourcesNASA NTRSDOE OSTI
DOE OSTI2018

A two-phase mixing layer between parallel gas and liquid streams: multiphase turbulence statistics and influence of interfacial instability

The two-phase mixing layer formed between parallel gas and liquid streams is an important fundamental problem in turbulent multiphase flows. The problem is relevant to many industrial applications and natural phenomena, such as air-blast atomizers in fuel injection systems and breaking waves in the ocean. The velocity difference between the gas and liquid streams triggers an interfacial instability which can be convective or absolute depending on the stream properties and injection parameters. In the present study, a direct numerical simulation of a two-phase gas–liquid mixing layer that lie in the absolute instability regime is conducted. A dominant frequency is observed in the simulation and the numerical result agrees well with the prediction from viscous stability theory. As the interfacial wave plays a critical role in turbulence transition and development, the temporal evolution of turbulent fluctuations (such as the enstrophy) also exhibits a similar frequency. To investigate the statistical response of the multiphase turbulence flow, the simulation has been run for a long physical time so that time-averaging can be performed to yield the statistically converged results for Reynolds stresses and the turbulent kinetic energy (TKE) budget. An extensive mesh refinement study using from 8 million to about 4 billions cells has been performed. The turbulent dissipation is shown to be highly demanding on mesh resolution compared with other terms in TKE budget. The results obtained with the finest mesh are shown to be close to converged results of turbulent dissipation which allow us to obtain estimations of the Kolmogorov and Hinze scales. The estimated Kolmogorov scale is found to be similar to the cell size of the finest mesh used here. The computed Hinze scale is significantly larger than the size of droplets observed and does not seem to be a relevant length scale to describe the smallest size of droplets formed in atomization.

Mechanics↗
DOE OSTI2020

Scalable molecular dynamics on CPU and GPU architectures with NAMD

NAMD is a molecular dynamics program designed for high-performance simulations of very large biological objects on CPU- and GPU-based architectures. NAMD offers scalable performance on petascale parallel supercomputers consisting of hundreds of thousands of cores, as well as on inexpensive commodity clusters commonly found in academic environments. It is written in C++ and leans on Charm++ parallel objects for optimal performance on low-latency architectures. NAMD is a versatile, multipurpose code that gathers state-of-the-art algorithms to carry out simulations in apt thermodynamic ensembles, using the widely popular CHARMM, AMBER, OPLS, and GROMOS biomolecular force fields. Here, we review the main features of NAMD that allow both equilibrium and enhanced-sampling molecular dynamics simulations with numerical efficiency. We describe the underlying concepts utilized by NAMD and their implementation, most notably for handling long-range electrostatics; controlling the temperature, pressure, and pH; applying external potentials on tailored grids; leveraging massively parallel resources in multiple-copy simulations; and hybrid quantum-mechanical/molecular-mechanical descriptions. We detail the variety of options offered by NAMD for enhanced-sampling simulations aimed at determining free-energy differences of either alchemical or geometrical transformations and outline their applicability to specific problems. Last, we discuss the roadmap for the development of NAMD and our current efforts toward achieving optimal performance on GPU-based architectures, for pushing back the limitations that have prevented biologically realistic billion-atom objects to be fruitfully simulated, and for making large-scale simulations less expensive and easier to set up, run, and analyze. NAMD is distributed free of charge with its source code at www.ks.uiuc.edu.

high-performance computing↗
DOE OSTI2020

Methodology and Application of Physical Security Effectiveness Based on Dynamic Force-on-Force Modeling

This report describes the research and development being performed at INL towards a dynamic modeling and simulation framework to enable physical security optimization at commercial nuclear power plants. The framework is based on the dynamic modeling tool EMRALD and is demonstrated for applications that can result in physical security optimization. Two main applications are presented: 1. Integrating FLEX portable equipment performance with FOF models of a plant’s physical security posture, and 2. Location optimization of bullet resistant enclosure. The generic framework for modeling FLEX portable equipment is described in detail, followed by a case study modeling an adversarial attack aimed at causing a radiological release by sabotaging the plant’s power supply and its ultimate heat sink capabilities at a hypothetical PWR. Two distinct FLEX deployment strategies, series and parallel, are modeled with distinct timelines. The results of the adversarial attack modeled in a commercial FOF tool, AVERT, are integrated with the FLEX deployment model in EMRALD. Monte Carlo simulation is used to model the distribution of the timeline in FLEX deployment strategies. Thermal-hydraulic analysis of FLEX performance is performed in RELAP5 and integrated with the EMRALD simulations to provide more realistic timelines in the models. The results demonstrate that, even in the extreme case of a successful adversarial attack, deployment of FLEX equipment can result in a significantly high likelihood of preventing radiological release. The modeling and simulation framework of integrating FLEX equipment with FOF models enables the NPPs to credit FLEX portable equipment in the plant security posture, resulting in an efficient and optimized physical security. The objective of location optimization of BRE is to determine the best location in the plant for a new BRE being planned by the plant to enhance their physical security effectiveness. The plant physical security FOF model is integrated with EMRALD that performs Monte Carlo simulation to run different attack scenarios and a discrete set of potential BRE locations. Sensitivity analysis is used to determine the most effective location for the BRE. The optimization approach can be extended to wide applications such as location optimization of remotely operated weapons and other strategic fixed assets.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Efficient Multigrid Reduction-in-Time for Method-of-Lines Discretizations of Linear Advection

Parallel-in-time methods for partial differential equations (PDEs) have been the subject of intense development over recent decades, particularly for diffusion-dominated problems. It has been widely reported in the literature, however, that many of these methods perform quite poorly for advection-dominated problems. In this report we analyze the particular iterative parallel-in-time algorithm of multigrid reduction-in-time (MGRIT) for discretizations of constant-wave-speed linear advection problems. We focus on common method-of-lines discretizations that employ upwind finite differences in space and Runge-Kutta methods in time. Using a convergence framework we developed in previous work, we prove for a subclass of these discretizations that, if using the standard approach of rediscretizing the fine-grid problem on the coarse grid, robust MGRIT convergence with respect to CFL number and coarsening factor is not possible. This poor convergence and non-robustness is caused, at least in part, by an inadequate coarse-grid correction for smooth Fourier modes in space-time known as characteristic components. We propose an alternative coarse-grid operator that provides a better correction of these modes. This coarse-grid operator is related to previous work and uses a semi-Lagrangian discretization combined with an implicitly treated truncation error correction. Theory and numerical experiments show the proposed coarse-grid operator yields fast MGRIT convergence for many of the method-of-lines discretizations considered, including for both implicit and explicit discretizations of high order. Parallel results demonstrate speed-up over sequential time-stepping.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2015

Evaluation of SNS Beamline Shielding Configurations using MCNPX Accelerated by ADVANTG

Shielding analyses for the Spallation Neutron Source (SNS) at Oak Ridge National Laboratory pose significant computational challenges, including highly anisotropic high-energy sources, a combination of deep penetration shielding and an unshielded beamline, and a desire to obtain well-converged nearly global solutions for mapping of predicted radiation fields. The majority of these analyses have been performed using MCNPX with manually generated variance reduction parameters (source biasing and cell-based splitting and Russian roulette) that were largely based on the analyst's insight into the problem specifics. Development of the variance reduction parameters required extensive analyst time, and was often tailored to specific portions of the model phase space. We previously applied a developmental version of the ADVANTG code to an SNS beamline study to perform a hybrid deterministic/Monte Carlo analysis and showed that we could obtain nearly global Monte Carlo solutions with essentially uniform relative errors for mesh tallies that cover extensive portions of the model with typical voxel spacing of a few centimeters. The use of weight window maps and consistent biased sources produced using the FW-CADIS methodology in ADVANTG allowed us to obtain these solutions using substantially less computer time than the previous cell-based splitting approach. While those results were promising, the process of using the developmental version of ADVANTG was somewhat laborious, requiring user-developed Python scripts to drive much of the analysis sequence. In addition, limitations imposed by the size of weight-window files in MCNPX necessitated the use of relatively coarse spatial and energy discretization for the deterministic Denovo calculations that we used to generate the variance reduction parameters. We recently applied the production version of ADVANTG to this beamline analysis, which substantially streamlined the analysis process. We also tested importance function collapsing (in space and energy) capabilities in ADVANTG. These changes, along with the support for parallel Denovo calculations using the current version of ADVANTG, give us the capability to improve the fidelity of the deterministic portion of the hybrid analysis sequence, obtain improved weight-window maps, and reduce both the analyst and computational time required for the analysis process.

43 PARTICLE ACCELERATORS↗
DOE OSTI2012

Temporal Expression-based Analysis of Metabolism

Metabolic flux is frequently rerouted through cellular metabolism in response to dynamic changes in the intra- and extra-cellular environment. Capturing the mechanisms underlying these metabolic transitions in quantitative and predictive models is a prominent challenge in systems biology. Progress in this regard has been made by integrating high-throughput gene expression data into genome-scale stoichiometric models of metabolism. Here, we extend previous approaches to perform a Temporal Expression-based Analysis of Metabolism (TEAM). We apply TEAM to understanding the complex metabolic dynamics of the respiratorily versatile bacterium Shewanella oneidensis grown under aerobic, lactate-limited conditions. TEAM predicts temporal metabolic flux distributions using time-series gene expression data. Increased predictive power is achieved by supplementing these data with a large reference compendium of gene expression, which allows us to take into account the unique character of the distribution of expression of each individual gene. We further propose a straightforward method for studying the sensitivity of TEAM to changes in its fundamental free threshold parameter θ, and reveal that discrete zones of distinct metabolic behavior arise as this parameter is changed. By comparing the qualitative characteristics of these zones to additional experimental data, we are able to constrain the range of θ to a small, well-defined interval. In parallel, the sensitivity analysis reveals the inherently difficult nature of dynamic metabolic flux modeling: small errors early in the simulation propagate to relatively large changes later in the simulation. We expect that handling such “history-dependent” sensitivities will be a major challenge in the future development of dynamic metabolic-modeling techniques.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2020

Sage: parallel semi-asymmtric graph algorithms for NVRAMs

Non-volatile main memory (NVRAM) technologies provide an attractive set of features for large-scale graph analytics, including byte-addressability, low idle power, and improved memory-density. NVRAM systems today have an order of magnitude more NVRAM than traditional memory (DRAM). NVRAM systems could therefore potentially allow very large graph problems to be solved on a single machine, at a modest cost. However, a significant challenge in achieving high performance is in accounting for the fact that NVRAM writes can be much more expensive than NVRAM reads. In this paper, we propose an approach to parallel graph analytics using the Parallel Semi-Asymmetric Model (PSAM), in which the graph is stored as a read-only data structure (in NVRAM), and the amount of mutable memory is kept proportional to the number of vertices. Similar to the popular semi-external and semi-streaming models for graph analytics, the PSAM approach assumes that the vertices of the graph fit in a fast read-write memory (DRAM), but the edges do not. In NVRAM systems, our approach eliminates writes to the NVRAM, among other benefits. To experimentally study this new setting, we develop Sage, a parallel semi-asymmetric graph engine with which we implement provably-efficient (and often work-optimal) PSAM algorithms for over a dozen fundamental graph problems. We experimentally study Sage using a 48--core machine on the largest publicly-available real-world graph (the Hyperlink Web graph with over 3.5 billion vertices and 128 billion edges) equipped with Optane DC Persistent Memory, and show that Sage outperforms the fastest prior systems designed for NVRAM. Importantly, we also show that Sage nearly matches the fastest prior systems running solely in DRAM, by effectively hiding the costs of repeatedly accessing NVRAM versus DRAM.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

When Do Extended Physics-Informed Neural Networks (XPINNs) Improve Generalization?

Physics-informed neural networks (PINNs) have become a popular choice for solving high-dimensional partial differential equations (PDEs) due to their excellent approximation power and generalization ability. Recently, extended PINNs (XPINNs) based on domain decomposition methods have attracted considerable attention due to their effectiveness in modeling multiscale and multiphysics problems and their parallelization. However, theoretical understanding of their convergence and generalization properties remains unexplored. In this study, we take an initial step towards understanding how and when XPINNs outperform PINNs. Specifically, for general multilayer PINNs and XPINNs, we first provide a prior generalization bound via the complexity of the target functions in the PDE problem and a posterior generalization bound via the posterior matrix norms of the networks after optimization. Moreover, based on our bounds, we analyze the conditions under which XPINNs improve generalization. Concretely, our theory shows that the key building block of XPINN, namely, the domain decomposition, introduces a tradeoff for generalization. On the one hand, XPINNs decompose the complex PDE solution into several simple parts, which decreases the complexity needed to learn each part and boosts generalization. On the other hand, decomposition leads to less training data being available in each subdomain, and hence such a model is typically prone to overfitting and may become less generalizable. Empirically, we choose five PDEs to show when XPINNs perform better than, similar to, or worse than PINNs, hence demonstrating and justifying our new theory.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2012

Terascale High-Fidelity Simulations of Turbulent Combustion with Detailed Chemistry

The TSTC project is a multi-university collaborative effort to develop a high-fidelity turbulent reacting flow simulation capability utilizing terascale, massively parallel computer technology. The main paradigm of our approach is direct numerical simulation (DNS) featuring highest temporal and spatial accuracy, allowing quantitative observations of the fine-scale physics found in turbulent reacting flows as well as providing a useful tool for development of sub-models needed in device-level simulations. The code named S3D, developed and shared with Chen and coworkers at Sandia National Laboratories, has been enhanced with new numerical algorithms and physical models to provide predictive capabilities for spray dynamics, combustion, and pollutant formation processes in turbulent combustion. Major accomplishments include improved characteristic boundary conditions, fundamental studies of auto-ignition in turbulent stratified reactant mixtures, flame-wall interaction, and turbulent flame extinction by water spray. The overarching scientific issue in our recent investigations is to characterize criticality phenomena (ignition/extinction) in turbulent combustion, thereby developing unified criteria to identify ignition and extinction conditions. The computational development under TSTC has enabled the recent large-scale 3D turbulent combustion simulations conducted at Sandia National Laboratories.

33 ADVANCED PROPULSION SYSTEMS↗
DOE OSTI2014

Formation of Imploding Plasma Liners for HEDP and MIF Application

Plasma jets with high density and velocity have a number of important applications in fusion energy and elsewhere, including plasma refueling, disruption mitigation in tokamaks, magnetized target fusion, injection of momentum into centrifugally confined mirrors, plasma thrusters, and high energy density plasmas (HEDP). In Magneto-Inertial Fusion (MIF), for example, an imploding material liner is used to compress a magnetized plasma to fusion conditions and to confine the resulting burning plasma inertially to obtain the necessary energy gain. The imploding shell may be solid, liquid, gaseous, or a combination of these states. The presence of the magnetic field in the target plasma suppresses thermal transport to the plasma shell, thus lowering the imploding power needed to compress the target to fusion conditions. This allows the required imploding momentum flux to be generated electromagnetically using off-the-shelf pulsed power technology. Practical schemes for standoff delivery of the imploding momentum flux are required and are open topics for research. One approach for accomplishing this, called plasma jet driven magneto-inertial fusion (PJMIF), uses a spherical array of pulsed plasma guns to create a spherically imploding shell of very high velocity, high momentum flux plasma. This approach requires development of plasma jet accelerators capable of achieving velocities of 50-200 km/s with very precise timing and density profiles, and with high total mass and density. Low-Z plasma jets would require the higher velocities, whereas very dense high-Z plasma shells could achieve the goal at velocities of only 50-100 km/s. In this report, we describe our work to develop the pulsed plasma gun technology needed for an experimental scientific exploration of the PJMIF concept, and also for the other applications mentioned earlier. The initial goal of a few hundred of hydrogen at 200 km/s was eventually replaced with accelerating 8000 μg of argon or xenon to 50 km/s for the Plasma Liner Experiment (PLX) at Los Alamos National Laboratory (LANL). Initial work used existing computational and analytical tools to develop and refine a specific plasma gun concept having a novel tapered coaxial electromagnetic accelerator contour with an array of symmetric ablative plasma injectors. The profile is designed to suppress the main barrier to success in coaxial guns, namely the blow-by instability in which the arc slips past and outruns the bulk of the plasma mass. Efforts to begin developing a set of annular non-ablative plasma injectors for the coaxial gun, in order to accelerate pure gases, resulted in development of linear parallel-plate MiniRailguns that turned out to work well as plasma guns in their own right and we subsequently chose them for an initial plasma liner experiment on the PLX facility at LANL. This choice was mainly driven by cost and schedule for that particular experiment, while longer term goals still projected use of coaxial guns for reactor-relevant applications for reasons of better symmetry, lower impurities, more compact plasma jet formation, and higher gun efficiency. Our efforts have focused mainly on 1) developing various plasma injection systems for both coax and linear railguns and ensuring they work reliably with the accelerator section, 2) developing a suite of plasma and gun diagnostics, 3) performing computational modeling to design and refine the plasma guns, 4) establishing a research facility dedicated to plasma gun development, and finally, 5) developing plasma guns and associated pulse power systems capable of achieving these goals and installing and testing the first two gun sets on the PLX facility at LANL. During the second funding cycle for this program, HyperV joined in a collaborative effort with LANL, the University of Alabama at Huntsville, and the University of New Mexico to perform a plasma liner experiment (PLX) to investigate the physics and technology of forming spherically imploding plasma liners. HyperV’s tasks focused on developing the plasma guns and associated pulse power systems required for the 30 gun experiment at LANL. Unfortunately, funding for the entire PLX collaborative project was terminated after only two years of the four year project due to program funding realignments which necessitated recompeting the project in midstream. Despite the loss of funding, HyperV installed two Mark1 guns and pulsed power systems on PLX, and jet characterization and merging experiments were subsequently successfully performed at LANL by the PLX Team. In parallel with those PLX experiments, HyperV continued its efforts to develop a plasma gun capable of meeting the PLX goal of 8 mg of argon at 50 km/s. HyperV was ultimately successful in this effort, demonstrating 10.8 mg at 52.8 km/s and 7.5 mg at 62.4 km/s with the Mark2 MiniRailgun.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2020

TurboRVB: A many-body toolkit for ab initio electronic simulations by quantum Monte Carlo

TurboRVB is a computational package for ab initio Quantum Monte Carlo (QMC) simulations of both molecular and bulk electronic systems. The code implements two types of well established QMC algorithms: Variational Monte Carlo (VMC) and diffusion Monte Carlo in its robust and efficient lattice regularized variant. A key feature of the code is the possibility of using strongly correlated many-body wave functions (WFs), capable of describing several materials with very high accuracy, even when standard mean-field approaches [e.g., density functional theory (DFT)] fail. The electronic WF is obtained by applying a Jastrow factor, which takes into account dynamical correlations, to the most general mean-field ground state, written either as an antisymmetrized geminal power with spin-singlet pairing or as a Pfaffian, including both singlet and triplet correlations. This WF can be viewed as an efficient implementation of the so-called resonating valence bond (RVB) Ansatz, first proposed by Pauling and Anderson in quantum chemistry [L. Pauling, The Nature of the Chemical Bond (Cornell University Press, 1960)] and condensed matter physics [P.W. Anderson, Mat. Res. Bull 8, 153 (1973)], respectively. The RVB Ansatz implemented in TurboRVB has a large variational freedom, including the Jastrow correlated Slater determinant as its simplest, but nontrivial case. Moreover, it has the remarkable advantage of remaining with an affordable computational cost, proportional to the one spent for the evaluation of a single Slater determinant. Therefore, its application to large systems is computationally feasible. The WF is expanded in a localized basis set. Several basis set functions are implemented, such as Gaussian, Slater, and mixed types, with no restriction on the choice of their contraction. The code implements the adjoint algorithmic differentiation that enables a very efficient evaluation of energy derivatives, comprising the ionic forces. Thus, one can perform structural optimizations and molecular dynamics in the canonical NVT ensemble at the VMC level. For the electronic part, a full WF optimization (Jastrow and antisymmetric parts together) is made possible, thanks to state-of-the-art stochastic algorithms for energy minimization. In the optimization procedure, the first guess can be obtained at the mean-field level by a built-in DFT driver. The code was efficiently parallelized by using a hybrid MPI-OpenMP protocol, which is also an ideal environment for exploiting the computational power of modern Graphics Processing Unit accelerators.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2016

SCALE Code System

The SCALE Code System is a widely-used modeling and simulation suite for nuclear safety analysis and design that is developed, maintained, tested, and managed by the Reactor and Nuclear Systems Division (RNSD) of Oak Ridge National Laboratory (ORNL). SCALE provides a comprehensive, verified and validated, user-friendly tool set for criticality safety, reactor and lattice physics, radiation shielding, spent fuel and radioactive source term characterization, and sensitivity and uncertainty analysis. Since 1980, regulators, licensees, and research institutions around the world have used SCALE for safety analysis and design. SCALE provides an integrated framework with dozens of computational modules including three deterministic and three Monte Carlo radiation transport solvers that are selected based on the desired solution strategy. SCALE includes current nuclear data libraries and problem-dependent processing tools for continuous-energy (CE) and multigroup (MG) neutronics and coupled neutron-gamma calculations, as well as activation, depletion, and decay calculations. SCALE includes unique capabilities for automated variance reduction for shielding calculations, as well as sensitivity and uncertainty analysis. SCALE’s graphical user interfaces assist with accurate system modeling, visualization of nuclear data, and convenient access to desired results.SCALE 6.2 provides many new capabilities and significant improvements of existing features.New capabilities include:• ENDF/B-VII.1 nuclear data libraries CE and MG with enhanced group structures,• Neutron covariance data based on ENDF/B-VII.1 and supplemented with ORNL data,• Covariance data for fission product yields and decay constants,• Stochastic uncertainty and correlation quantification for any SCALE sequence with Sampler,• Parallel calculations with KENO,• Problem-dependent temperature corrections for CE calculations,• CE shielding and criticality accident alarm system analysis with MAVRIC,• CE depletion with TRITON (T5-DEPL/T6-DEPL),• CE sensitivity/uncertainty analysis with TSUNAMI-3D,• Simplified and efficient LWR lattice physics with Polaris,• Large scale detailed spent fuel characterization with ORIGAMI and ORIGAMI Automator,• Advanced fission source convergence acceleration capabilities with Sourcerer,• Nuclear data library generation with AMPX, and• Integrated user interface with Fulcrum.Enhanced capabilities include:• Accurate and efficient CE Monte Carlo methods for eigenvalue and fixed source calculations,• Improved MG resonance self-shielding methodologies and data,• Resonance self-shielding with modernized and efficient XSProc integrated into most sequences,• Accelerated calculations with TRITON/NEWT (generally 4x faster than SCALE 6.1),• Spent fuel characterization with 1470 new reactor-specific libraries for ORIGEN,• Modernization of ORIGEN (Chebyshev Rational Approximation Method [CRAM] solver, API for high-performance depletion, new keyword input format)• Extension of the maximum mixture number to values well beyond the previous limit of 2147 to ~2 billion,• Nuclear data formats enabling the use of more than 999 energy groups,• Updated standard composition library to provide more accurate use of natural abundances, andvi• Numerous other enhancements for improved usability and stability.

98 NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL P↗
DOE OSTI2017

An overlapping Domain Decomposition preconditioning method for monolithic solution of shear bands

Metals subjected to high strain rate impact often exhibit a sudden and profound drop in the material’s load bearing capability, a ductile failure phenomenon known as shear banding. Shear bands, characterized as material instabilities, are driven by shear heating and described as narrow regions that have sustained intense plastic deformation and high temperature rise. This coupled thermo-mechanical localization problem can be formulated as a nonlinear system with two balance equations, momentum and energy, and two constitutive laws for elasticity and plasticity. Here in our formulation, mixed finite elements are used to discretize the equations in space and an implicit finite difference scheme is used to advance the system in time, where at every time step, a Newton type method is used to solve the system monolithically. To that end, a block Jacobian matrix is formed analytically using Gâteaux derivatives. The resulting Jacobian is sparse, nonsymmetric and its sparsity pattern and eigenvalue content vary with the different stages of shear bands formation, consequently posing a significant challenge to iterative solvers. To address this issue, an overlapping Domain Decomposition preconditioner that takes into account the physics of shear bands and the domain in which it forms, is proposed. The key idea is to resolve the shear band domain accurately while only solving the remaining domain approximately, with selective updates when necessary. The preconditioner is formed on the basis of an additive Schwartz method that is applied to a Schur complement partition of the system. The proposed preconditioner is implemented in parallel and tested on three different benchmark problems as compared against off-the-shelf solvers. We study the effect of h- and k-refinement that are obtained from Isogeometric discretizations of the system, the overlapping strategy and its parallel performance. Excellent performance is demonstrated in serial and parallel on all benchmark examples.

42 ENGINEERING↗
DOE OSTI2015

Accelerated acquisition of tagged MRI for cardiac motion correction in simultaneous PET-MR: Phantom and patient studies

Purpose: Degradation of image quality caused by cardiac and respiratory motions hampers the diagnostic quality of cardiac PET. It has been shown that improved diagnostic accuracy of myocardial defect can be achieved by tagged MR (tMR) based PET motion correction using simultaneous PET-MR. However, one major hurdle for the adoption of tMR-based PET motion correction in the PET-MR routine is the long acquisition time needed for the collection of fully sampled tMR data. In this work, the authors propose an accelerated tMR acquisition strategy using parallel imaging and/or compressed sensing and assess the impact on the tMR-based motion corrected PET using phantom and patient data. Methods: Fully sampled tMR data were acquired simultaneously with PET list-mode data on two simultaneous PET-MR scanners for a cardiac phantom and a patient. Parallel imaging and compressed sensing were retrospectively performed by GRAPPA and kt-FOCUSS algorithms with various acceleration factors. Motion fields were estimated using nonrigid B-spline image registration from both the accelerated and fully sampled tMR images. The motion fields were incorporated into a motion corrected ordered subset expectation maximization reconstruction algorithm with motion-dependent attenuation correction. Results: Although tMR acceleration introduced image artifacts into the tMR images for both phantom and patient data, motion corrected PET images yielded similar image quality as those obtained using the fully sampled tMR images for low to moderate acceleration factors (<4). Quantitative analysis of myocardial defect contrast over ten independent noise realizations showed similar results. It was further observed that although the image quality of the motion corrected PET images deteriorates for high acceleration factors, the images were still superior to the images reconstructed without motion correction. Conclusions: Accelerated tMR images obtained with more than 4 times acceleration can still provide relatively accurate motion fields and yield tMR-based motion corrected PET images with similar image quality as those reconstructed using fully sampled tMR data. The reduction of tMR acquisition time makes it more compatible with routine clinical cardiac PET-MR studies.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2022

Direction-optimizing Label Propagation Framework for Structure Detection in Graphs: Design, Implementation, and Experimental Analysis

Label Propagation is not only a well-known machine learning algorithm for classification but also an effective method for discovering communities and connected components in networks. We propose a new Direction-optimizing Label Propagation Algorithm (DOLPA) framework that enhances the performance of the standard Label Propagation Algorithm (LPA), increases its scalability, and extends its versatility and application scope. As a central feature, the DOLPA framework relies on the use of frontiers and alternates between label push and label pull operations to attain high performance. It is formulated in such a way that the same basic algorithm can be used for finding communities or connected components in graphs by only changing the objective function used. Additionally, DOLPA has parameters for tuning the processing order of vertices in a graph to reduce the number of edges visited and improve the quality of solution obtained. We present the design and implementation of the enhanced algorithm as well as our shared-memory parallelization of it using OpenMP. We also present an extensive experimental evaluation of our implementations using the LFR benchmark and real-world networks drawn from various domains. Compared with an implementation of LPA for community detection available in a widely used network analysis software, we achieve at most five times the F-Score while maintaining similar runtime for graphs with overlapping communities. We also compare DOLPA against an implementation of the Louvain method for community detection using the same LFR-graphs and show that DOLPA achieves about three times the F-Score at just 10% of the runtime. For connected component decomposition, our algorithm achieves orders of magnitude speedups over the basic LP-based algorithm on large-diameter graphs, up to 13.2× speedup over the Shiloach-Vishkin algorithm, and up to 1.6× speedup over Afforest on an Intel Xeon processor using 40 threads.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2016

Dose-Dependent Cortical Thinning After Partial Brain Irradiation in High-Grade Glioma

Purpose: Radiation-induced cognitive deficits may be mediated by tissue damage to cortical regions. Volumetric changes in cortex can be reliably measured using high-resolution magnetic resonance imaging (MRI). We used these methods to study the association between radiation therapy (RT) dose and change in cortical thickness in high-grade glioma (HGG) patients. Methods and Materials: We performed a voxel-wise analysis of MRI from 15 HGG patients who underwent fractionated partial brain RT. Three-dimensional MRI was acquired pre- and 1 year post RT. Cortex was parceled with well-validated segmentation software. Surgical cavities were censored. Each cortical voxel was assigned a change in cortical thickness between time points, RT dose value, and neuroanatomic label by lobe. Effects of dose, neuroanatomic location, age, and chemotherapy on cortical thickness were tested using linear mixed effects (LME) modeling. Results: Cortical atrophy was seen after 1 year post RT with greater effects at higher doses. Estimates from LME modeling showed that cortical thickness decreased by −0.0033 mm (P<.001) for every 1-Gy increase in RT dose. Temporal and limbic cortex exhibited the largest changes in cortical thickness per Gy compared to that in other regions (P<.001). Age and chemotherapy were not significantly associated with change in cortical thickness. Conclusions: We found dose-dependent thinning of the cerebral cortex, with varying neuroanatomical regional sensitivity, 1 year after fractionated partial brain RT. The magnitude of thinning parallels 1-year atrophy rates seen in neurodegenerative diseases and may contribute to cognitive decline following high-dose RT.

62 RADIOLOGY AND NUCLEAR MEDICINE↗
DOE OSTI2023

Distributed out-of-memory NMF on CPU/GPU architectures

We propose an efficient distributed out-of-memory implementation of the non-negative matrix factorization (NMF) algorithm for heterogeneous high-performance-computing systems. The proposed implementation is based on prior work on NMFk, which can perform automatic model selection and extract latent variables and patterns from data. In this work, we extend NMFk by adding support for dense and sparse matrix operation on multi-node, multi-GPU systems. The resulting algorithm is optimized for out-of-memory problems where the memory required to factorize a given matrix is greater than the available GPU memory. Memory complexity is reduced by batching/tiling strategies, and sparse and dense matrix operations are significantly accelerated with GPU cores (or tensor cores when available). Input/output latency associated with batch copies between host and device is hidden using CUDA streams to overlap data transfers and compute asynchronously, and latency associated with collective communications (both intra-node and inter-node) is reduced using optimized NVIDIA Collective Communication Library (NCCL) based communicators. Benchmark results show significant improvement, from 32X to 76x speedup, with the new implementation using GPUs over the CPU-based NMFk. Good weak scaling was demonstrated on up to 4096 multi-GPU cluster nodes with approximately 25,000 GPUs when decomposing a dense 340 Terabyte-size matrix and an 11 Exabyte-size sparse matrix of density 10 -6 .

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2015

Virtual machine-based simulation platform for mobile ad-hoc network-based cyber infrastructure

In modeling and simulating complex systems such as mobile ad-hoc networks (MANETs) in de-fense communications, it is a major challenge to reconcile multiple important considerations: the rapidity of unavoidable changes to the software (network layers and applications), the difficulty of modeling the critical, implementation-dependent behavioral effects, the need to sustain larger scale scenarios, and the desire for faster simulations. Here we present our approach in success-fully reconciling them using a virtual time-synchronized virtual machine(VM)-based parallel ex-ecution framework that accurately lifts both the devices as well as the network communications to a virtual time plane while retaining full fidelity. At the core of our framework is a scheduling engine that operates at the level of a hypervisor scheduler, offering a unique ability to execute multi-core guest nodes over multi-core host nodes in an accurate, virtual time-synchronized manner. In contrast to other related approaches that suffer from either speed or accuracy issues, our framework provides MANET node-wise scalability, high fidelity of software behaviors, and time-ordering accuracy. The design and development of this framework is presented, and an ac-tual implementation based on the widely used Xen hypervisor system is described. Benchmarks with synthetic and actual applications are used to identify the benefits of our approach. The time inaccuracy of traditional emulation methods is demonstrated, in comparison with the accurate execution of our framework verified by theoretically correct results expected from analytical models of the same scenarios. In the largest high fidelity tests, we are able to perform virtual time-synchronized simulation of 64-node VM-based full-stack, actual software behaviors of MANETs containing a mix of static and mobile (unmanned airborne vehicle) nodes, hosted on a 32-core host, with full fidelity of unmodified ad-hoc routing protocols, unmodified application executables, and user-controllable physical layer effects including inter-device wireless signal strength, reachability, and connectivity.

97 MATHEMATICS AND COMPUTING↗