SEARCH · Engineering Papers

Results for “parallel simulation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 667 records · Page 37

All sources NASA NTRS DOE OSTI

DOE OSTI2021

Nyx: A Massively Parallel AMR Code for Computational Cosmology

Nyx is a highly parallel, adaptive mesh, finite-volume N-body compressible hydrodynamics solver for cosmological simulations. It has been used to simulate different cosmological scenarios with a recent focus on the intergalactic medium and Lyman alpha forest. Together, Nyx, the compressible astrophysical simulation code, Castro, and the low Mach number code MAESTROeX, make up the AMReX-Astrophysics Suite of open-source, adaptive mesh, performance-portable astrophysical simulation codes. Other examples of cosmological simulation research codes include Enzo, Enzo-P/Cello, RAMSES, ART, FLASH, Cholla, as well as Gadget, Gasoline, Arepo, Gizmo, and SWIFT.

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2021

Cybersecurity for Grid Connected eXtreme Fast Charging (XFC) Station (CyberX) (Final Scientific/Technical Report)

This report summarizes the activities conducted under the DOE VTO funded project DE- EE0008451, where ABB Inc. (ABB), in collaboration with Idaho National Laboratory (INL), APS Global (APS), and XOS Trucks (XOS) pursued the development of a cyber-resilient extreme fast charging (XFC) management system. This project entitled Cybersecurity for Grid Connected eXtreme Fast Charging (XFC) Station (CyberX) focuses on a resilient architecture for smart charging EV Supply Equipment (EVSE) device control and Coordinated Anomaly Detection System (CADS) features that can be added at the charging site depot level to increase cybersecurity. The project was split into two budget periods focused first on developing the threat model and resilient control concepts and second on testing, improving, and validating those developed resilient control algorithms and features with a focus on key vulnerabilities identified during the threat assessment portion of the project. During the first budget period of the CyberX project, the ABB led team focused on activities to identify, model, and quantitatively prioritize high-impact attack scenarios with potential cyber-physical effects while also modeling and developing concepts for a resilient control system that could securely address integration of DERs and other resources with EV charging. Development of the security focused XFC management system (XMS) was accomplished first by offline simulation using a developed XFC station or depot with 480V input level and simulating measurement inputs to monitoring and control systems in concept development. A representative distribution grid model was developed supporting an EV charging site model with BESS and 6 general EV charging models. These EV charging models allowed multiple configurations of charging level, multiple connected protection and measurement devices, and simulation function to show general compromise of EV, BESS, and protection features based on parallel threat analysis. During the second budget period, the EV site and supporting systems model was developed in more detail and converted from offline model to real-time to real-time with EV charging hardware in the loop (HIL). The resilient control architecture developed as concept in the first part of the project was further tested and validated for integration of local energy resources and XFC charging station site equipment while maintaining cybersecure operating principles. The proposed resilient architecture for smart charging and cybersecurity features consists of two main concepts developed and tested within the project. The first concept is an XFC management system (XMS) consisting of a hardware gateway, software platform, and Supervisory Control and Data Acquisition (SCADA) or Distribution Management System integration components. The second concept is a Coordinated Anomaly Detection System (CADS) which forms a primarily software-related subsystem of the total CyberX solution focused on monitoring system measurements, estimation of measurement states, and predicting current at the utility point of interaction based on machine learning for anomaly detection.

33 ADVANCED PROPULSION SYSTEMS↗

DOE OSTI2022

Global Kinetic Modeling of the Intrabinary Shock in Spider Pulsars

Abstract Spider pulsars are compact binary systems composed of a millisecond pulsar and a low-mass companion. The relativistic magnetically dominated pulsar wind impacts onto the companion, ablating it and slowly consuming its atmosphere. The interaction forms an intrabinary shock, a proposed site of particle acceleration. We perform global fully kinetic particle-in-cell simulations of the intrabinary shock, assuming that the pulsar wind consists of plane-parallel stripes of alternating polarity and that the shock wraps around the companion. We find that particles are efficiently accelerated via shock-driven reconnection. We extract first-principles synchrotron spectra and light curves, which are in good agreement with X-ray observations: (1) the synchrotron spectrum is nearly flat, F ν ∝ const; (2) when the pulsar spin axis is nearly aligned with the orbital angular momentum, the light curve displays two peaks, just before and after the pulsar eclipse (pulsar superior conjunction), separated in phase by ∼0.8 rad; (3) the peak flux exceeds the one at the inferior conjunction by a factor of 10.

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2021

ParFlow

ParFlow is an open-source, modular, parallel watershed flow model. It includes fully-integrated overland flow, the ability to simulate complex topography, geology and heterogeneity and coupled land-surface processes including the land-energy budget, biogeochemistry and snow (via CLM). It is multi-platform and runs with a common I/O structure from laptop to supercomputer. ParFlow is the result of a long, multi-institutional development history and is now a collaborative effort between CSM, LLNL, UniBonn and UCB. ParFlow has been coupled to the mesoscale, meteorological code ARPS and the NCAR code WRF.

Smith, Steven↗

DOE OSTI2021

ParFlow

Smith, Steven↗

DOE OSTI2020

Exploring the Acceleration Mechanisms for Particle Injection and Power-law Formation during Transrelativistic Magnetic Reconnection

Magnetic reconnection in the relativistic and transrelativistic regimes is able to accelerate particles to hard power-law energy spectra f ∝ γ -p (approaching p = 1). The underlying acceleration mechanism that determines the spectral shape is currently a topic of intense investigation. By means of fully kinetic plasma simulations, we carry out a study of particle acceleration during magnetic reconnection in the transrelativistic regime of a proton–electron plasma. While earlier work in this parameter regime has focused on the effects of electric field parallel to the local magnetic field on the particle injection (from thermal energy to the lower-energy bound of the power-law spectrum), here we examine the roles of both parallel and perpendicular electric fields to gain a more complete understanding on the injection process and further development of a power-law spectrum. We show that the parallel electric field does contribute significantly to particle injection, and is more important in the initial phase of magnetic reconnection. However, as the simulation proceeds, the acceleration by the perpendicular electric field becomes more important for particle injection and completely dominates the acceleration responsible for the high-energy power-law spectrum. This holds robustly, in particular for longer reconnection times and larger systems, i.e., in simulations that are more indicative of the processes in astrophysical sources.

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2025

STORM: Scrape-off layer turbulence in tokamak fusion reactors

The scrape-off layer of a tokamak fusion reactor carries the plasma exhaust from the hot core plasma to the material surfaces of the reactor vessel. The heat loads imposed by the exhaust are a critical limit on the performance of fusion power plants. Turbulent transport of the plasma regulates the width of the scrape-off layer plasma and must be modelled to understand the intensity of these heat loads. STORM is a plasma turbulence code capable of simulating three dimensional turbulence across the full scrape-off layer of a tokamak fusion reactor, using a drift reduced, collisional fluid model. STORM uses mostly finite difference schemes, with a staggered grid in the direction parallel to the magnetic field. We describe the model, geometry and initialisation options used by STORM, as well as the numerical methods, which are implemented using the BOUT++ plasma simulation framework. BOUT++ has been enhanced alongside the development of STORM, providing better support for staggered grid methods. We summarise these enhancements, including a detailed explanation of the parallel derivative methods, which underwent a major update for version 4 of BOUT++.

BOUT++↗

DOE OSTI2024

Simulating ‘Two Ribbon’ Type Solar Flares [Slides]

The underlying mechanism in solar flares in a process known as magnetic reconnection. This astrophysical process is when magnetic field lines, running anti-parallel, break and reconnect. The change in configuration of field lines results in an explosive release of energy as the leftover magnetic energy is converted to kinetic and thermal energies. In this project, we are using an astrophysical magnetohydrodynamic (MHD) simulation code known as Athena++ with a reconnection problem generator file created by Li et al. 2018. Our current research is to successfully implement a radiative cooling term into the MHD equations as it could have important physical effects on the plasma. We use Klimchuk et al. 2008 and SPEX_DM as our chosen cooling functions. The results from the SPEX_DM function show there is a condensation forming that we had predicted but had not seen before with our simulation code. We will continue to analyze the SPEX_DM function and potentially implement thermal conduction.

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2025

A GPU ‐Accelerated 3D Unstructured Mesh Based Particle Tracking Code for Multi‐Species Impurity Transport Simulation in Fusion Tokamaks

ABSTRACT This paper presents the multi‐species global impurity transport capability developed in a GPU‐accelerated fully 3D unstructured mesh‐based code, GITRm, to simultaneously track multiple impurity species and handle interactions of these impurities with mixed‐material surfaces. Different computational approaches to model particle‐surface interaction or surface response have been developed and compared. Sheath electric field is taken into account by employing a fast distance‐to‐boundary calculation, which is carried out in parallel on distributed or partitioned meshes on multiple GPUs without the need for any inter‐process communication during the simulation. Several example cases, including two for the DIII‐D tokamak, that is, one with the SAS‐V divertor and the other with the collector probes, are used to demonstrate the utility of the current multi‐species capability. For the DIII‐D probe case, the capability of GITRm to resolve the spatial distribution of particles in localized regions, such as diagnostic probes, within non‐axisymmetric tokamak geometries is demonstrated. These simulations involve up to 320 million particles and utilize up to 48 GPUs.

Nath, Dhyanjyoti D. [Scientific Computation Resear↗

DOE OSTI2025

Implementation of a Mesh refinement algorithm into the quasi-static PIC code QuickPIC

Plasma-based acceleration (PBA) has emerged as a promising candidate for the accelerator technology used to build a future linear collider and/or an advanced light source. In PBA, a trailing or witness particle beam is accelerated in the plasma wave wakefield (WF) created by a laser or particle beam driver. The WF is often nonlinear and involves the crossing of plasma particle trajectories in real space and thus particle-in-cell methods are used. The distance over which the drive beam evolves is several orders of magnitude larger than the wake wavelength. This large disparity in length scales is amenable to the quasi-static approach. Three-dimensional (3D), quasi-static (QS), particle-in-cell (PIC) codes, e.g., QuickPIC, have been shown to provide high fidelity simulation capability with 2-4 orders of magnitude speedup over 3D fully explicit PIC codes. In PBA, the witness beam needs to be matched to the focusing forces of the WF to reduce the emittance growth. In some linear collider designs, the matched spot size of the witness beam can be 2 to 3 orders of magnitude smaller than the spot size (and wavelength) of the wakefield. Such an additional disparity in length scales is ideal for mesh refinement where the WF within the witness beam is described on a finer mesh than the rest of the WF. A mesh refinement scheme is described that has been implemented into the 3D QS PIC code, QuickPIC. Very fine (high) resolution is used in a small spatial region that includes the witness beam and progressively coarser resolutions in the rest of the simulation domain. A fast multigrid Poisson solver has been implemented for the field solve on the refined meshes and a Fast Fourier Transform (FFT) based Poisson solver is used for the coarse mesh. The code has been parallelized with both MPI and OpenMP, and the parallel scalability has also been improved by using pipelining. A preliminary adaptive mesh refinement technique is described to optimize the computational time for simulations with an evolving witness beam size. Several test problems are used to verify that the mesh refinement algorithm provides accurate results. Additionally, the results are benchmarked against highly resolved simulations exhibiting near-azimuthal symmetry, performed using QPAD—a novel hybrid QS PIC code that uses a PIC description in the coordinates (r, ct – z) and a gridless description in the azimuthal angle, Φ.

Linear collider↗

DOE OSTI2020

Accelerating massively parallel hemodynamic models of coarctation of the aorta using neural networks

Comorbidities such as anemia or hypertension and physiological factors related to exertion can influence a patient’s hemodynamics and increase the severity of many cardiovascular diseases. Observing and quantifying associations between these factors and hemodynamics can be difficult due to the multitude of co-existing conditions and blood flow parameters in real patient data. Machine learning-driven, physics-based simulations provide a means to understand how potentially correlated conditions may affect a particular patient. Here, we use a combination of machine learning and massively parallel computing to predict the effects of physiological factors on hemodynamics in patients with coarctation of the aorta. We first validated blood flow simulations against in vitro measurements in 3D-printed phantoms representing the patient’s vasculature. We then investigated the effects of varying the degree of stenosis, blood flow rate, and viscosity on two diagnostic metrics – pressure gradient across the stenosis (ΔP) and wall shear stress (WSS) - by performing the largest simulation study to date of coarctation of the aorta (over 70 million compute hours). Using machine learning models trained on data from the simulations and validated on two independent datasets, we developed a framework to identify the minimal training set required to build a predictive model on a per-patient basis. We then used this model to accurately predict ΔP (mean absolute error within 1.18 mmHg) and WSS (mean absolute error within 0.99 Pa) for patients with this disease.

59 BASIC BIOLOGICAL SCIENCES↗

DOE OSTI2023

Exploring model complexity in machine learned potentials for simulated properties

Abstract Machine learning (ML) enables the development of interatomic potentials with the accuracy of first principles methods while retaining the speed and parallel efficiency of empirical potentials. While ML potentials traditionally use atom-centered descriptors as inputs, different models such as linear regression and neural networks map descriptors to atomic energies and forces. This begs the question: what is the improvement in accuracy due to model complexity irrespective of descriptors? We curate three datasets to investigate this question in terms of ab initio energy and force errors: (1) solid and liquid silicon, (2) gallium nitride, and (3) the superionic conductor Li $$_{10}$$ 10 Ge(PS $$_{6}$$ 6 ) $$_{2}$$ 2 (LGPS). We further investigate how these errors affect simulated properties and verify if the improvement in fitting errors corresponds to measurable improvement in property prediction. By assessing different models, we observe correlations between fitting quantity (e.g. atomic force) error and simulated property error with respect to ab initio values. Graphical abstract

Rohskopf, A. (ORCID:0000000227128296)↗

DOE OSTI2020

Fluid Dynamics Effects on Microstructure Prediction in Single-Laser Tracks for Additive Manufacturing of IN625

Single-track laser fusion were simulated using a heat-transfer-solidification-only (HTS) model and its extension with fluid dynamics (HTS_FD) model using a parallel open-source code, which included laminar fluid dynamics, flat-free surface of the molten alloy, heat transfer, phase-change, evaporation, and surface tension phenomena. The results illustrate that the fluid dynamics affects the solidification and ensuing microstructure. For the HTS_FD simulations, thermal gradient, G was found to exhibit a maximum at the extremity of the solidified pool ( i.e. , at the free surface), while for HTS simulations, G exhibited a maximum around the entire edge of the solidified pool. HTS_FD simulations predicted a wider range of cooling rates than the HTS simulations, exhibited an increased spread in the solidification speed, V variation within the melt-pool with respect to the HTS model results. Primary dendrite arm spacing (PDAS) were evaluated based on power law correlations and marginal stability theory models using the ( G , V ) from HTS and HTS_FD simulations to quantify the effect of the fluid dynamics on the microstructure. At low-laser powers and low-scan speeds, the PDAS obtained with the fluid dynamics model (HTS_FD) was larger by more than 30 pct with respect to the PDAS calculated with the simple HTS model. A new PDAS correlation, i.e. , $ \lambda_{1} \left[ {\mu {\text{m}}} \right] = 832\;G\left[ {\text{K/m}} \right]^{ - 0.5} V\left[ {\text{m/s}} \right]^{ - 0.25} $ , which uses the ( G , V ) results from the HTS_FD model was developed and validated against experimental results.

36 MATERIALS SCIENCE↗

DOE OSTI2021

Simulation of lithium transport using the BOUT++ framework

A numerical model that calculates the collisional interactions between the lithium atoms from a lithium pellet and the background plasmas has been upgraded. The ion density ($N_t$), electron temperature ($T_e$), ion temperature ($T_i$) and parallel ion velocity ($V_{∥, i}$) are used to characterize the background plasmas. The lithium atom density ($N^{a}_{Li}$) and parallel velocity ($V_{∥,a}$) of lithium atoms evolve with time. For each lithium ion, the density ($N_{Li^{n+}}$), temperature ($T_{Li^{n+}}$) and parallel velocity ($V_{∥, Li^{n+}}$) are self-consistently calculated. A C-mod lower single null equilibrium is used to generate the grid for the BOUT++ simulation. The lithium atoms can be fully ionized to $Li^{3+}$ in ~2 μs. The rapid radial and poloidal expansion of the lithium ions are found in the simulation. After the collision interaction process, the electron temperature rapidly decreases at the pellet location; then, it rapidly poloidally expands, and the temperature at the pellet location starts to recover. The electron pressure increases at the pellet location despite the decrease in electron temperature because of the extra electrons from the lithium ionization. The ion pressure profile decreases in the pellet location due to the decrease in ion temperature.

74 ATOMIC AND MOLECULAR PHYSICS↗

DOE OSTI2020

Performance and Accuracy Implications of Parallel Split Physics-Dynamics Coupling in the Energy Exascale Earth System Atmosphere Model

Simultaneous calculation of atmospheric processes is faster than calculating processes one at a time. This type of parallelism is beneficial or perhaps even necessary to provide good performance on modern supercomputers, which achieve faster performance through increased processor count rather than improved clock speed. The scalability of the Energy Exascale Earth System Model (E3SM) Atmosphere Model (EAM) is limited by the fluid dynamics which scales up to the number of mesh cells in the global mesh. In contrast, the suite of physics parameterizations in EAM is scalable up to the total number of physics columns, which is an order of magnitude greater than the number of mesh cells. A proposed solution to unlocking the greater potential performance from the physics suite is to solve the physics and dynamics in parallel. This work represents a first attempt at parallel splitting of the grid-scale fluid dynamics model and the subgrid-scale physics parameterizations in a global atmosphere model. We will demonstrate that switching to parallel physics-dynamics coupling extends the scalability of the EAM to up to 3 times the previous peak scalability limit and is up to 20% faster than the sequentially split coupling at the highest core counts and the same time step. Decadal simulations of both coupling approaches show very little impact to the model climate. This improved performance does not come without drawbacks, however. Parallel splitting requires a shorter time step and other modifications which largely offset performance gains. A mass fixer is required for conservation. Techniques for mitigating these issues are also discussed.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2023

A parallel p ‐adaptive discontinuous Galerkin method for the Euler equations with dynamic load‐balancing on tetrahedral grids

Abstract A novel p ‐adaptive discontinuous Galerkin (DG) method has been developed to solve the Euler equations on three‐dimensional tetrahedral grids. Hierarchical orthogonal basis functions are adopted for the DG spatial discretization while a third order TVD Runge‐Kutta method is used for the time integration. A vertex‐based limiter is applied to the numerical solution in order to eliminate oscillations in the high order method. An error indicator constructed from the solution of order and is used to adapt degrees of freedom in each computational element, which remarkably reduces the computational cost while still maintaining an accurate solution. The developed method is implemented with under the Charm++ parallel computing framework. Charm++ is a parallel computing framework that includes various load‐balancing strategies. Implementing the numerical solver under Charm++ system provides us with access to a suite of dynamic load balancing strategies. This can be efficiently used to alleviate the load imbalances created by p ‐adaptation. A number of numerical experiments are performed to demonstrate both the numerical accuracy and parallel performance of the developed p ‐adaptive DG method. It is observed that the unbalanced load distribution caused by the parallel p ‐adaptive DG method can be alleviated by the dynamic load balancing from Charm++ system. Due to this, high performance gain can be achieved. For the testcases studied in the current work, the parallel performance gain ranged from 1.5× to 3.7×. Therefore, the developed p ‐adaptive DG method can significantly reduce the total simulation time in comparison to the standard DG method without p ‐adaptation.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2024

Performance Analysis of an Optimization Algorithm for Metamaterial Design on the Integrated High-Performance Computing and Quantum Systems

Optimizing metamaterials with complex geometries is a big challenge. Although an active learning algorithm, combining machine learning (ML), quantum computing, and optical simulation, has emerged as an efficient optimization tool, it still faces difficulties in optimizing complex structures that have potentially high performance. In this work, we comprehensively analyze the performance of an optimization algorithm for metamaterial design on the integrated HPC and quantum systems. We demonstrate significant time advantages through message-passing interface (MPI) parallelization on the high-performance computing (HPC) system showing approximately 54% faster ML tasks and 67 times faster optical simulation against serial workloads. Furthermore, we analyze the performance of a quantum algorithm designed for optimization, which runs with various quantum simulators on a local computer or HPC-quantum system. Results showcase ~24 times speedup when executing the optimization algorithm on the HPC-quantum hybrid system. This study paves a way to optimize complex metamaterials using the integrated HPC-quantum system.

Kim, Seongmin↗

DOE OSTI2025

AMR-Wind: A Performance-Portable, High-Fidelity Flow Solver for Wind Farm Simulations

We present AMR-Wind, a verified and validated high-fidelity computational-fluid-dynamics code for wind farm flows. AMR-Wind is a block-structured, adaptive-mesh, incompressible-flow solver that enables predictive simulations of the atmospheric boundary layer and wind plants. It is a highly scalable code designed for parallel high-performance computing with a specific focus on performance portability for current and future computing architectures, including graphical processing units (GPUs). In this paper, we detail the governing equations, the numerical methods, and the turbine models. Establishing a foundation for the correctness of the code, we present the results of formal verification and validation. The verification studies, which include a novel actuator line test case, indicate that AMR-Wind is spatially and temporally second-order accurate. The validation studies demonstrate that the key physics capabilities implemented in the code, including actuator disk models, actuator line models, turbulence models, and large eddy simulation (LES) models for atmospheric boundary layers, perform well in comparison to reference data from established computational tools and theory. We conclude with a demonstration simulation of a 12-turbine wind farm operating in a turbulent atmospheric boundary layer, detailing computational performance and realistic wake interactions.

17 WIND ENERGY↗