Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “physically distributed memory”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 289 records · Page 16

Addressing GPU memory limitations for Graph Neural Networks in High-Energy Physics applications

Introduction Reconstructing low-level particle tracks in neutrino physics can address some of the most fundamental questions about the universe. However, processing petabytes of raw data using deep learning techniques poses a challenging problem in the field of High Energy Physics (HEP). In the Exa.TrkX Project, an illustrative HEP application, preprocessed simulation data is fed into a state-of-art Graph Neural Network (GNN) model, accelerated by GPUs. However, limited GPU memory often leads to Out-of-Memory (OOM) exceptions during training, due to the large size of models and datasets. This problem is exacerbated when deploying models on High-Performance Computing (HPC) systems designed for large-scale applications. Methods We observe a high workload imbalance issue during GNN model training caused by the irregular sizes of input graph samples in HEP datasets, contributing to OOM exceptions. We aim to scale GNNs on HPC systems, by prioritizing workload balance in graph inputs while maintaining model accuracy. Our paper introduces diverse balancing strategies aimed at decreasing the maximum GPU memory footprint and avoiding the OOM exception, across various datasets. Results Our experiments showcase memory reduction of up to 32.14% compared to the baseline. We also demonstrate the proposed strategies can avoid OOM in application. Additionally, we create a distributed multi-GPU implementation using these samplers to demonstrate the scalability of these techniques on the HEP dataset. Discussion By assessing the performance of these strategies as data loading samplers across multiple datasets, we can gauge their effectiveness in both single-GPU and distributed environments. Our experiments, conducted on datasets of varying sizes and across multiple GPUs, broaden the applicability of our work to various GNN applications that handle input datasets with irregular graph sizes.

Lee, Claire Songhyun↗

Spectral quadrature for the first principles study of crystal defects: Application to magnesium

In this work, we present an accurate and efficient finite-difference formulation and parallel implementation of Kohn-Sham Density (Operator) Functional Theory (DFT) for non periodic systems embedded in a bulk environment. Specifically, employing non-local pseudopotentials, local reformulation of electrostatics, and truncation of the spatial Kohn-Sham Hamiltonian, and the Linear Scaling Spectral Quadrature method to solve for the pointwise electronic fields in real-space and the non-local component of the atomic force, we develop a parallel finite difference framework suitable for distributed memory computing architectures to simulate non-periodic systems embedded in a bulk environment. Choosing examples from magnesium-aluminum alloys, we first demonstrate the convergence of energies and forces with respect to spectral quadrature polynomial order, and the width of the spatially truncated Hamiltonian. Next, we demonstrate the parallel scaling of our framework, and show that the computation time and memory scale linearly with respect to the number of atoms. Next, we use the developed framework to simulate isolated point defects and their interactions in magnesium-aluminum alloys. Our findings conclude that the binding energies of divacancies, Al solute-vacancy and two Al solute atoms are anisotropic and are dependent on cell size. Furthermore, the binding is favorable in all three cases.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

National Security Science and Technology Initiative: Air Cargo Screening

The non-intrusive inspection (NII) of consolidated air cargo carried on commercial passenger aircraft continues to be a technically challenging, high-priority requirement of the Department of Homeland Security's Science and Technology Directorate (DHS S&T), the Transportation Security Agency and the Federal Aviation Administration. The goal of deploying a screening system that can reliably and cost-effectively detect explosive threats in consolidated cargo without adversely affecting the flow of commerce will require significant technical advances that will take years to develop. To address this critical National Security need, the Battelle Memorial Institute (Battelle), under a Cooperative Research and Development Agreement (CRADA) with four of its associated US Department of Energy (DOE) National Laboratories (Oak Ridge, Pacific Northwest, Idaho, and Brookhaven), conducted a research and development initiative focused on identifying, evaluating, and integrating technologies for screening consolidated air cargo for the presence of explosive threats. Battelle invested $8.5M of internal research and development funds during fiscal years 2007 through 2009. The primary results of this effort are described in this document and can be summarized as follows: (1) Completed a gap analysis that identified threat signatures and observables, candidate technologies for detection, their current state of development, and provided recommendations for improvements to meet air cargo screening requirements. (2) Defined a Commodity/Threat/Detection matrix that focuses modeling and experimental efforts, identifies technology gaps and game-changing opportunities, and provides a means of summarizing current and emerging capabilities. (3) Defined key properties (e.g., elemental composition, average density, effective atomic weight) for basic commodity and explosive benchmarks, developed virtual models of the physical distributions (pallets) of three commodity types and three explosive benchmarks combinations, and conducted modeling and simulation studies to begin populating the matrix of commodities, threats, and detection technologies. (4) Designed and fabricated basic (homogeneous) commodity test pallets and fabricated inert stimulants to support experiments and to validate modeling/simulation results. (5) Developed/expanded the team's capabilities to conduct full-scale imaging (neutron and x-ray) experiments of air cargo commodities and explosive benchmarks. (6) Conducted experiments to improve the collection of trace particles of explosives from a variety of surfaces representative of air cargo materials by means of mechanical (air/vibration/pressure), thermal, and electrostatic methods. Air cargo screening is a difficult challenge that will require significant investment in both research and development to find a suitable solution to ensure the safety of passengers without significantly hindering the flow of commodities. The initiative funded by Battelle has positioned this group to make major contributions in meeting the air cargo challenge by developing collaborations, developing laboratory test systems, improving knowledge of the challenges (both technical and business) for air cargo screening, and increasing the understanding of the capabilities for current inspection methods (x-ray radiography, x-ray backscatter, etc.) and potential future inspection methods (neutron radiography, fusion of detector modalities, advanced trace detection, etc.). Lastly, air cargo screening is still an issue that will benefit from collaboration between Department of Energy Laboratories and Battelle. On January 7, 2010, DHS Secretary Napolitano joined White House Press Secretary Robert Gibbs and Assistant to the President for Counterterrorism and Homeland Security John Brennan to announce several recommendations DHS has made to the President for improving the technology and procedures used to protect air travel from acts of terrorism. (This announcement followed the 25 Dec'09 Delta/Northwest Airlines Flight 253 terror attack.) Secretary Napolitano outlined five recommendations DHS will pursue to enhance the safety of the traveling public. One of the five recommendations, read as follows: 'Establish a partnership on aviation security between DHS and the Department of Energy and its National Laboratories in order to develop new and more effective technologies to deter and disrupt known threats and proactively anticipate and protect against new ways by which terrorists could seek to board an aircraft.' In conclusion, it appears very timely that Battelle and its DOE lab partners initiated a serious collaboration on the air cargo topic, and that we should continue to work toward future collaboration in response to the government's needs.

45 MILITARY TECHNOLOGY, WEAPONRY, AND NATIONAL DEF↗

Singular value decomposition of adjoint flux distributions for Monte Carlo variance reduction

Monte Carlo (MC) shielding calculations often use weight windows (WWs) and biased sources formed from a deterministic estimate of the adjoint flux to improve the convergence rate of tallies. This requires a significant amount of computer memory, which can limit the memory available for high-resolution tally output. A new method is proposed for reducing these memory requirements by using singular value decomposition (SVD) in linear or logarithmic space to approximate the adjoint flux. This method’s performance is evaluated using the Shift and Denovo codes for streaming and diffusion base case problems, followed by problems using the Westinghouse AP1000 and the Joint European Torus. The log SVD reduced WW memory requirements by an order of magnitude in all cases without a significant performance penalty. Additionally, the linear SVD reduced biased source memory requirements by an order of magnitude, but further investigation is needed to account for observed limitations.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Rate analysis for a hybrid quantum repeater

We present a detailed rate analysis for a hybrid quantum repeater assuming perfect memories and using optimal probabilistic entanglement generation and deterministic swapping routines. The hybrid quantum repeater protocol is based on atomic qubit-entanglement distribution through optical coherent-state communication. An exact, analytical formula for the rates of entanglement generation in quantum repeaters is derived, including a study on the impacts of entanglement purification and multiplexing strategies. More specifically, we consider scenarios with as little purification as possible and we show that for sufficiently low local losses, such purifications are still more powerful than multiplexing. In a possible experimental scenario, our hybrid system can create near-maximally entangled (F=0.98) pairs over a distance of 1280 km at rates of the order of 100 Hz.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

A parallel strategy for density functional theory computations on accelerated nodes

Using the Löwdin orthonormalization of tall-skinny matrices as a proxy-app for wavefunction-based Density Functional Theory solvers, we investigate a distributed memory parallel strategy focusing on Graphics Processing Unit (GPU)-accelerated nodes as available on some of the top ranked supercomputers at the present time. Here we present numerical results in the strong limit regime, as it is particularly relevant for First-Principles Molecular Dynamics. We also examine how matrix product-based iterative solvers provide a competitive alternative to dense eigensolvers on GPUs, allowing to push the strong scaling limit of these computations to a larger number of distributed tasks. Our strategy, which relies on replicated Gram matrices and efficient collective communications using the NCCL library, leads to a time-to-solution under 0.5 s for the Löwdin orthonormalization of a tall-skinny matrix of 3000 columns on Summit at Oak Ridge Leadership Facility (OLCF). Given the similarity in computational operations between one iteration of a DFT solver and this proxy-app, this shows the possibility of solving accurately the DFT equations well under a minute for 3000 electronic wave functions, and thus perform First-Principles molecular dynamics of physical systems much larger than traditionally solved on CPU systems.

97 MATHEMATICS AND COMPUTING↗

A cell-centered AMR-ALE framework for 3D multi-material hydrodynamics. Part I: Lagrangian and indirect Euler AMR algorithms

Many applications of physics and engineering involve wide ranges of time and spatial scales. The numerical simulation of localized small scales such as shock waves and material interfaces requires a large number of computational cells in these regions. For these applications, Lagrangian and Arbitrary-Lagrangian-Eulerian (ALE) related methods are engaging since the moving mesh feature naturally brings mesh cells on shock discontinuities and material interfaces are carefully captured. In addition, Adaptive-Mesh-Refinement (AMR) strategies aim to optimize computational resources by concentrating finer mesh cells only in areas of interest while using coarser cells elsewhere. A key but challenging AMR requirement consists in efficiently distributing the computational effort to achieve high accuracy without the prohibitive computational costs associated with uniformly fine grids. Here, in this document, the coupling of the p4est AMR library with a cell-centered Lagrangian scheme is presented with the goal to perform reliable 3D Lagrangian-AMR and indirect Euler-AMR multi-material simulations. In particular, it is shown that starting from a 3D indirect ALE code, the memory management and load balancing requirements can be delegated to an external library (here the p4est library) to unlock ALE-AMR capabilities. First, we present a strategy to transcribe the octant-based connectivity of the 3D AMR framework with that of an unstructured mesh of polygonal cells used in Lagrangian hydrodynamics. Then, we show how refinement and coarsening operations must be adapted to the particular Lagrangian framework to ensure the conservation of volume during those steps. Finally, several numerical test cases are presented that demonstrate the capabilities of the Lagrangian-AMR and indirect Euler-AMR algorithms.

3D cell-centered Lagrangian numerical scheme↗

Actuation method and apparatus, micropump, and PCR enhancement method

An actuation apparatus includes at least one magnetic shape memory (MSM) element containing a material configured to expand and/or contract in response to exposure to a magnetic field. Among other things, the MSM element may be configured to pump fluid through a micropump by expanding and/or contracting in response to the magnetic field. The magnetic field may rotate about an axis of rotation and exhibit a distribution having a component substantially perpendicular to the axis of rotation. Further, the magnetic field distribution may include at least two components substantially orthogonal to one another lying in one or more planes perpendicular to the axis of rotation. The at least one MSM element may contain nickel, manganese, and gallium. A polymerase chain reaction (PCR) may be enhanced by contacting a PCR reagent and DNA material with the MSM element.

36 MATERIALS SCIENCE↗

Sub-band transport mechanism and switching properties for resistive switching nonvolatile memories with structure of silver/aluminum oxide/p-type silicon

In this paper, we discuss a model of sub-band in resistive switching nonvolatile memories with a structure of silver/aluminum oxide/p-type silicon (Ag/Al{sub x}O{sub y}/p-Si), in which the sub-band is formed by overlapping of wave functions of electron-occupied oxygen vacancies in Al{sub x}O{sub y} layer deposited by atomic layer deposition technology. The switching processes exhibit the characteristics of the bipolarity, discreteness, and no need of forming process, all of which are discussed deeply based on the model of sub-band. The relationships between the SET voltages and distribution of trap levels are analyzed qualitatively. The semiconductor-like behaviors of ON-state resistance affirm the sub-band transport mechanism instead of the metal filament mechanism.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

Three-dimensional photodissociation in strong laser fields: Memory-kernel effective-mode expansion

We introduce a method for the efficient computation of non-Markovian quantum dynamics for strong (and time-dependent) system-bath interactions. The past history of the system dynamics is incorporated by expanding the memory kernel in exponential functions thereby transforming in an exact fashion the non-Markovian integrodifferential equations into a (larger) set of ''effective modes'' differential equations (EMDE). We have devised a method which easily diagonalizes the EMDE, thereby allowing for the efficient construction of an adiabatic basis and the fast propagation of the EMDE in time. We have applied this method to three-dimensional photodissociation of the H{sub 2}{sup +} molecule by strong laser fields. Our calculations properly include resonance-Raman scattering via the continuum, resulting in extensive rotational and vibrational excitations. The calculated final kinetic and angular distribution of the photofragments are in overall excellent agreement with experiments, both when transform-limited pulses and when chirped pulses are used.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Development of a phonon-based sampling method for thermal neutron scattering data

Simulations of reactor systems require access to accurate nuclear data. For many systems, thermal neutron scattering data can have large effects on the eigenvalue and neutron flux distributions. Inelastic thermal neutron scattering can excite or de-excite vibrational, rotational, and translational modes in a material, so thermal scattering evaluations are often obtained by summing over the number of phonons created/destroyed by a scattering event. In recent years, the thermal scattering cross sections and angular distributions have greatly improved in accuracy, but the format in which this data is delivered to simulation codes has remained virtually unchanged. Thermal scattering data is typically either compiled into large tables and sorted by incoming neutron energy, outgoing neutron energy, scattering angle, and material temperature, or represented as cumulative distribution functions of momentum exchange or energy exchange. Either method can be quite memory intensive when fine bins are used. In an effort to decrease the amount of space that processed thermal scattering data requires, an alternate format is proposed. The phonon-based sampling method introduced here can sample the number of phonons excited for each collision, the change in neutron energy, and the scattering angle while avoiding pre-computed angular bins and limiting the amount of data that is dependent on incoming energy. Through this method, the generation and storage of large interpolation tables is avoided, which could have benefits in both memory storage and accuracy. While the initial implementation of this method is slower than current alternatives, it is significantly more resistant to grid coarseness errors and has good potential for improvement. (authors)

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Embedding Neural Thermal Scattering (NeTS) Modules in SERPENT for Higher Fidelity Advanced Reactor Analysis

When a neutron born in fission thermalizes to the order of $k$ $B$ $T$, it’s de-Broglie wavelength and energy approach the order of inter-atomic spacing and elementary lattice oscillations, respectively. $S$($a,β,t$) or the scattering law, uuantify these temperature-dependent crystallographic contributions to total cross section (or reaction rate). In a Monte Carlo analysis, cumulative distribution functions (CDFs) of $S$($a,β,t$) are loaded to memory from “A Compact ENDF” (ACE) files for stochastically selecting thermal scattered neutron trajectories. In this work, novel neural thermal scattering (NeTS) modules for $S$($a,β,t$) CDFs are designed, trained, serialized and embedded within SERPENT using Python’s limited C-API for on-the-fly deployment of crystalline graphite $S$($a,β,t$) sampling. Torchscript tracing and Numba just-in-time (JIT) compilation streamline neural inference on NVIDIA GPUs with CUDA libraries. Demonstrations of bare sphere thermalization of fast and thermal sources show excellent agreement between embedded NeTS in SERPENT and MCNP. With an explicit model of the reactor, NeTS can predict on-the-fly changes in TREAT neutron spectra as a function of local temperature, which can serve to improve transient and accident predictions in a multiphysics analysis framework. This framework can be further extended to account on-the-fly for changes in local graphitic microstructure to scattering cross sections, and outlines a novel coupling of modern machine learning with state-of-the-art reactor physics methods.

97 MATHEMATICS AND COMPUTING↗

Progress on Optimizing Wind Farms and Rotor Designs Using Adjoints

Modern wind plants are increasingly tasked with multiple performance objectives. In addition to designing plants that maximize power output and minimize the levelized cost of energy (LCOE), the design and operation of wind plants is increasingly influenced by challenges regarding grid integration of variable generation renewables. This places a growing emphasis on making wind plants more controllable and predictable. WindSE is a Reynolds-averaged Navier-Stokes (RANS) model designed around analytical gradient and adjoint methods, with the ability to capture terrain-induced effects, as shown in Figure 1. The recent addition of an unsteady solver with an actuator line method (ALM) and ongoing work to enable massively parallel optimizations gives it a unique niche to explore coupled plant-level controls and design problems. This code is an open source python package built on the FEniCS framework that utilizes fast, parallel PETSc solvers to model fluid flow throughout wind-farm scale domains. Two recent studies performed using WindSE demonstrate the capability to optimize under a wide variety of flow conditions and objective functions. In the first, we present an optimization focused on modifying the layout of a wind farm with a fixed number of turbines for maximum total power output [1]. This study highlights the ability to quickly perform simulations using the steady Navier-Stokes solver combined with rotors represented as actuator disks while also stressing the importance of capturing terrain-induced effects. Gradient-based optimization using the RANS equations is viable due to the inclusion of efficiently computed adjoint derivatives. We interpret the physical results of the optimal layout and also discuss the computational cost of scaling to larger problems. In the second study, we present the capabilities of the unsteady Navier-Stokes solver, where rotor-blade profiles represented by actuator lines are optimized to enhance wake steering effects and overall power production [2]. We quantify the wind plant performance gains obtained from this type of simultaneous control co-design optimization as compared to optimizing the blade design and yaw independently. Figure 2 shows the differences between a baseline two-turbine system and an optimized system where we fine-tune the blade chord profile. Results and challenges from each study are quickly summarized and used to motivate the current development efforts within WindSE. Current and future work is focused on enabling higher-resolution studies with more degrees of freedom through parallelization of both the simulation and optimization algorithms. We present benchmarking results to show that WindSE performs well in both weak- and strong-scaling tests and further demonstrate that the optimizer obtains the same convergence rates in both shared- and distributed-memory environments. Using larger wind farms, we can study deep-array effects within an optimization context, allowing the use of objective functions that have been previously unstudied. As an example, we present ongoing work on a blockage metric which characterizes the loss of available kinetic energy due to wake effects from multiple upstream turbines.

adjoint optimization↗

Origin of “memory glass” effect in pressure-amorphized rare-earth molybdate single crystals

The memory glass effect (MGE) describes the ability of some materials to recover the initial structure and crystallographic orientation after pressure-induced amorphization (PIA). In spite of numerous studies the nature and underlying mechanisms of this phenomenon are still not clear. Here we report investigations of MGE in β′-Eu{sub 2}(MoO{sub 4}){sub 3} single crystal samples subjected to high pressure amorphization. Using the XRD and TEM techniques we carried out detailed analysis of the structural state of high pressure treated single crystal samples as well as structural transformations due to subsequent annealing at atmospheric pressure. The structure of the sample has been found to be complex, mainly amorphous, however, the amorphous medium contains evenly distributed nanosize inclusions of a paracrystalline phase. The inclusions are highly correlated in orientation and act as “memory units” in the MGE. - Graphical abstract: Schematic representation of pressure-induced amorphization and “memory glass” effect in rare-earth molybdate single crystals. The XRD and TEM measurements have revealed the presence of the residual identically oriented paracrystalline nanodomains in the pressure-amorphized state. These domains preserve the information about initial structure and orientation of the sample. They act as memory units and crystalline seeds during transformation of the amorphous phase back to the starting single crystalline one. - Highlights: • Pressure-amorphized Eu{sub 2}(MoO4){sub 3} single crystals were studied ex-situ by XRD and TEM. • Tiny residual crystalline inclusions were found in amorphous matrix of sample. • The inclusions keep in memory the parent crystal structure and orientation. • The inclusions account for “memory glass” effect in rare-earth molibdates.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Multigrid treatment of implicit continuum diffusion

Implicit treatment of diffusive terms of various differential orders common in continuum mechanics modeling, such as computational fluid dynamics, is investigated with spectral and multigrid algorithms in non-periodic 2D domains. In doubly periodic time dependent problems these terms are handled efficiently by spectral methods, but in non-periodic systems solved with distributed memory parallel computing and 2D domain decomposition, this efficiency is lost for a large number of processors. Here, we built and present here a multigrid algorithm for these types of problems that outperforms a spectral solution employing the highly optimized FFTW library. This solver is suitable for high performance computing and may be able to efficiently treat implicit diffusion of arbitrary order by introducing auxiliary equations of lower order. We test these solvers for fourth and sixth order diffusion with harmonic test functions as well as turbulent 2D magnetohydrodynamic simulations. It is also shown that an anisotropic operator without mixed-derivative terms improves model accuracy and speed, and we examine the impact that the various diffusion operators have on the energy, the enstrophy, and the qualitative aspect of a simulation.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Disabling CNT Electronic Devices by Use of Electron Beams

Bombardment with tightly focused electron beams has been suggested as a means of electrically disabling selected individual carbon-nanotubes (CNTs) in electronic devices. Evidence in support of the suggestion was obtained in an experiment in which a CNT field-effect transistor was disabled (see figure) by focusing a 1-keV electron beam on a CNT that served as the active channel of a field-effect transistor (FET). Such bombardment could be useful in the manufacture of nonvolatile-memory circuits containing CNT FETs. Ultimately, in order to obtain the best electronic performances in CNT FETs and other electronic devices, it will be necessary to fabricate the devices such that each one contains only a single CNT as an active element. At present, this is difficult because there is no way to grow a single CNT at a specific location and with a specific orientation. Instead, the common practice is to build CNTs into electronic devices by relying on spatial distribution to bridge contacts. This practice results in some devices containing no CNTs and some devices containing more than one CNT. Thus, CNT FETs have statistically distributed electronic characteristics (including switching voltages, gains, and mixtures of metallic and semiconducting CNTs). According to the suggestion, by using a 1-keV electron beam (e.g., a beam from a scanning electron microscope), a particular nanotube could be rendered electrically dysfunctional. This procedure could be repeated as many times as necessary on different CNTs in a device until all of the excess CNTs in the device had been disabled, leaving only one CNT as an active element (e.g., as FET channel). The physical mechanism through which a CNT becomes electrically disabled is not yet understood. On one hand, data in the literature show that electron kinetic energy >86 keV is needed to cause displacement damage in a CNT. On the other hand, inasmuch as a 1-keV beam focused on a small spot (typically a few tens of nanometers wide) deposits a significant amount of energy in a small volume, the energy density may suffice to thermally induce structural and/or electronic changes that disable the CNT. Research may be warranted to investigate this effect in detail.

Petkov, Mihail↗

Long-distance quantum key distribution with imperfect devices

Quantum key distribution over probabilistic quantum repeaters is addressed. We compare, under practical assumptions, two such schemes in terms of their secure key generation rate per memory, R{sub QKD}. The two schemes under investigation are the one proposed by Duan et al. in [Nat. 414, 413 (2001)] and that of Sangouard et al. proposed in [Phys. Rev. A 76, 050301 (2007)]. We consider various sources of imperfections in the latter protocol, such as a nonzero double-photon probability for the source, dark count per pulse, channel loss and inefficiencies in photodetectors and memories, to find the rate for different nesting levels. We determine the maximum value of the double-photon probability beyond which it is not possible to share a secret key anymore. We find the crossover distance for up to three nesting levels. We finally compare the two protocols.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗