Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Parallel Fluid Dynamic Solver”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

Investigating the Interaction Between Circulating Tumor Cells and Local Hydrodynamics via Experiment and Simulations

Introduction: The biological and mechanical properties of circulating tumor cells (CTCs) in combination with the hemodynamics affect the preference of metastatic sites in the vasculature. Despite the extensive literature on the effects of biological properties on cell adhesion, the effects of hydrodynamic forces on primary attachment remains an active area of research. Here, using simulations in conjunction with experimentation, we provide new insight into the interplay of CTCs dynamics and local hydrodynamics. Methods: A flow experiment of CTC attachment was performed within a bioprinted, double branching endothelialized vessel. Simulations of fluid flow and CTC transport in the reconstructed and idealized bifurcated vessel were respectively performed by HARVEY, our in-house massively parallel computational fluid dynamics solver. HARVEY is based on the lattice Boltzmann and finite element methods to model the fluid and cells dynamics. The immersed boundary method is employed for resolving the fluid–structure interaction. Results: CTC attachment was quantified experimentally at all regions of the complex vessel. The results demonstrate a clear preference for CTCs to attach at the branch points. To elucidate the effect of the vessel topology on the location of attachment, a fluid-only simulation was performed assessing the differences in the hydrodynamics along the vessel. CTC transport in idealized bifurcated vessels was subsequently studied to examine the effects of cell deformability on the local hydrodynamics patterns and, thus, the preference of attachment sites. Conclusions: The current work provides evidence on the correlation of the hydrodynamics forces arising from the vessel topology and CTC properties on the attachment regions.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2016

Recommended direct simulation Monte Carlo collision model parameters for modeling ionized air transport processes

A systematic approach for calibrating the direct simulation Monte Carlo (DSMC) collision model parameters to achieve consistency in the transport processes is presented. The DSMC collision cross section model parameters are calibrated for high temperature atmospheric conditions by matching the collision integrals from DSMC against ab initio based collision integrals that are currently employed in the Langley Aerothermodynamic Upwind Relaxation Algorithm (LAURA) and Data Parallel Line Relaxation (DPLR) high temperature computational fluid dynamics solvers. The DSMC parameter values are computed for the widely used Variable Hard Sphere (VHS) and the Variable Soft Sphere (VSS) models using the collision-specific pairing approach. The recommended best-fit VHS/VSS parameter values are provided over a temperature range of 1000-20 000 K for a thirteen-species ionized air mixture. Use of the VSS model is necessary to achieve consistency in transport processes of ionized gases. The agreement of the VSS model transport properties with the transport properties as determined by the ab initio collision integral fits was found to be within 6% in the entire temperature range, regardless of the composition of the mixture. The recommended model parameter values can be readily applied to any gas mixture involving binary collisional interactions between the chemical species presented for the specified temperature range.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2023

Coupling hydrodynamics to a rigid-body motion solver for fluid-structure interaction [Slides]

Quinoa is a massively parallel computational fluid dynamics with multi-material and programmed burn capabilities developed from Programmatic and LDRD funds. Overset is a well-established method of using an “inset” mesh that communicates with a background mesh. Solutions transfer freely from one to another and operate as boundary conditions on the opposite mesh. Either mesh can move at specified velocity! This project couples the existing Mesh-to-mesh transfer with Quinoa and demonstrates use of resulting Overset solver in computational simulation of blast effects on a re-entry body for survivability assessments.

42 ENGINEERING↗
DOE OSTI2025

A Comparison of GPU-Accelerated Multiphase CFD Solvers on the Polaris Supercomputer: Part 1

This report is in support of the Innovative and Novel Computational Impact on Theory and Experiment (INCITE) program sponsored by the U.S. Department of Energy (USDOE). With INCITE-level resources, one project, titled BubblyFlow, was granted computational resources for the 2025 calendar year on the Polaris supercomputer at the Argonne Leadership Computing Facility (ALCF). The project aims to conduct simulations to understand the fundamental characteristics of turbulent bubbly flow phenomena in nature. Staff at the ALCF and Argonne’s Computational Science division, along with collaborators at the City College of New York and University of Illinois at Chicago, helped a summer student to assess the accuracy and performance of two high performance computing (HPC) codes. Both codes, ImExLBM and FluTAS, are fundamentally different in their mathematical and numerical modeling. However, both may be used to solve the same physical problem. The collaboration sought to better understand the differences between both codes in terms of accuracy and efficiency. This would ultimately help the BubblyFlow project better utilize resources and establish a knowledge-base of code capabilities in future simulation campaigns. We compare ImExLBM and FluTAS, two high-performance multiphase computational fluid dynamics (CFD) solvers, in terms of physical fidelity, time-to-solution, and parallel efficiency. We validate ImExLBM (Implicit-Explicit Lattice Boltzmann Method) against a canonical benchmark and assess it’s performance relative to FluTAS (Fluid Transport Accelerated Solver), a well-established open-source CFD code.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

kynema-sgf [SWR-20-85]

Kynema-SGF (formerly AMR-Wind), wherein SGF stands for structured-grid fluid dynamics, is a massively parallel, block-structured adaptive-mesh, incompressible flow solver. The codebase was initiated in 2019 from incflo. The solver is built on top of the AMReX library. AMReX library provides the mesh data structures, mesh adaptivity, as well as the linear solvers used for solving the governing equations. Kynema-SGF is actively developed and maintained by a dedicated multi-institutional team from Lawrence Berkeley National Laboratory, National Laboratory of the Rockies, and Sandia National Laboratories. The primary applications for Kynema-SGF are: performing large-eddy simulations (LES) of atmospheric boundary layer (ABL) flows, simulating wind farm turbine-wake interactions using actuator disk or actuator line models for turbines, and as a background solver when coupled with a near-body solver (e.g., Kynema-UGF) with overset methodology to perform blade-resolved simulations of multiple wind turbines within a wind farm. For offshore applications, the ability to model the air-sea interaction effects and its impact on the ABL characteristics is another focus for the code development effort. As with other codes in the Kynema ecosystem, Kynema-SGF shares the following objectives: *an open, well-documented implementation of the state-of-the-art computational models for modeling wind farm flow physics at various fidelities that are backed by a comprehensive verification and validation (V&V) process; *be capable of performing the highest-fidelity simulations of flow fields within wind farms; and *be able to leverage the high-performance leadership class computing facilities available at DOE national laboratories.

Ananthan, Shreyas↗
DOE OSTI2019

Reverse-Mode Algorithmic Differentiation of an OpenMP-Parallel Compressible Flow Solver

y Reverse-mode algorithmic differentiation (AD) is an established method for obtaining adjoint derivatives of computer simulation applications. In computational fluid dynamics (CFD), adjoint derivatives of a cost function output such as drag or lift with respect to design parameters such as surface coordinates or geometry control points are a key ingredient for shape optimization, uncertainty quantification and flow control. The computational cost of CFD applications and their derivatives makes it essential to use high-performance computing hardware efficiently, including multi- and many-core architectures. Nevertheless, OpenMP is not supported in most AD tools, and previously shown methods achieve poor scalability of the derivative code. We present the AD of an OpenMP-parallelized finite volume compressible flow solver for unstructured meshes. Our approach enables us to reuse the parallelization of the original code in the computation of adjoint derivatives. The method works by identifying code segments that can be differentiated in reverse-mode without changing their memory access pattern. The OpenMP parallelization is integrated into the derivative code during the build process in a way that is robust to modifications of the original code and independent of the OpenMP support of the differentiation tool. We show the scalability of our adjoint CFD solver on test cases ranging from thousands to millions of finite volume mesh cells on CPUs with up to 16 threads as well as on an Intel XeonPhi card with 236 threads. We demonstrate that our approach is more practical to implement for production-sized CFD codes and produces more efficient adjoint derivative code than previously shown AD methods.

Huckelheim, Jan↗
DOE OSTI2019

Asynchronous Navier-Stokes Solver on 3D Unstructured Grids for the Exascale Era

This project has developed multiple fluid dynamics solvers for complex 3D flows using fully asynchronous distributed-memory task-parallel algorithms on top of the Charm++ runtime system. The algorithms solve the Euler or Navier-Stokes equations of compressible flows using unstructured tetrahedron meshes with optional solution-adaptive mesh-, and polynomial-order refinement. We have demonstrated excellent strong scaling up 50K compute cores and the benefits of Charm++’s automatic load balancing.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2016

On the Strong Scaling of the Spectral Element Solver Nek5000 on Petascale Systems. In: EASC '16 Proceedings of the Exascale Applications and Software Conference 2016, Article No. 5

The present work is targeted at performing a strong scaling study of the high-order spectral element fluid dynamics solver Nek5000. Prior studies such as [5] indicated a recommendable metric for strong scalability from a theoretical viewpoint, which we test here extensively on three parallel machines with different performance characteristics and interconnect networks, namely Mira (IBM Blue Gene/Q), Beskow (Cray XC40) and Titan (Cray XK7). The test cases considered for the simulations correspond to a turbulent flow in a straight pipe at four different friction Reynolds numbers Reτ = 180, 360, 550 and 1000. Considering the linear model for parallel communication we quantify the machine characteristics in order to better assess the scaling behaviors of the code. Subsequently sampling and profiling tools are used to measure the computation and communication times over a large range of compute cores. We also study the effect of the two coarse grid solvers XXT and AMG on the computational time. Super-linear scaling due to a reduction in cache misses is observed on each computer. The strong scaling limit is attained for roughly 5000-10,000 degrees of freedom per core on Mira, 30,000 - 50,0000 on Beskow, with only a small impact of the problem size for both machines, and ranges between 10,000 and 220,000 depending on the problem size on Titan. This work aims at being a reference for Nek5000 users and also serves as a basis for potential issues to address as the community heads towards exascale supercomputers.

Offermans, Nicolas↗
DOE OSTI2014

Massively parallel solvers for elliptic partial differential equations in numerical weather and climate prediction

The demand for substantial increases in the spatial resolution of global weather and climate prediction models makes it essential to use numerically efficient and highly scalable algorithms to solve the equations of large-scale atmospheric fluid dynamics. For stability and efficiency reasons, several of the operational forecasting centres, in particular the Met Office and the European Centre for Medium-Range Weather Forecasts (ECMWF) in the UK, use semi-implicit semi-Lagrangian time-stepping in the dynamical core of the model. The additional burden with this method is that a three-dimensional elliptic partial differential equation (PDE) for the pressure correction has to be solved at every model time step and this often constitutes a significant proportion of the time spent in the dynamical core. In global models, this PDE must be solved in a thin spherical shell. To run within tight operational time-scales, the solver has to be parallelized and there seems to be a (perceived) misconception that elliptic solvers do not scale to large processor counts and hence implicit time-stepping cannot be used in very high-resolution global models. After reviewing several methods for solving the elliptic PDE for the pressure correction and their application in atmospheric models, we demonstrate the performance and very good scalability of Krylov subspace solvers and multigrid algorithms for a representative model equation with more than 1010 unknowns on 65 536 cores on the High-End Computing Terascale Resource (HECToR), the UK's national supercomputer. Here, we tested and optimized solvers from two existing numerical libraries (the Distributed and Unified Numerics Environment (DUNE) and Parallel High Performance Preconditioners (hypre)) and implemented both a conjugate gradient solver and a geometric multigrid algorithm based on a tensor-product approach, which exploits the strong vertical anisotropy of the discretized equation. We study both weak and strong scalability and compare the absolute solution times for all methods; in contrast to one-level methods, the multigrid solver is robust with respect to parameter variations.

58 GEOSCIENCES↗
DOE OSTI2023

A numerical solver for investigating the space charge effect on the electric field in liquid argon time projection chambers

This paper reports the development of a numerical solver aimed to simulate the interaction between the space charge (i.e. ions) distribution and the electric field in liquid argon time projection chamber (LArTPC) detectors. The ion transport equation is solved by a time-accurate, cell-centered finite volume method and the electric potential equation by a continuous finite element method. The electric potential equation updates the electric field which provides the drift velocity to the ion transport equation. The ion transport equation updates the space charge density distribution which appears as the source term in the electric potential equation. The interaction between the space charge distribution and the electric field is numerically simulated within each physical time step. The convective velocity in the ion transport equation can include the background flow velocity in addition to the electric drift velocity. The numerical solver has been parallelized using the Message Passing Interface (MPI) library. Numerical tests show and verify the capability and accuracy of the current numerical solver. It is planned that the developed numerical solver, together with a Computational Fluid Dynamics (CFD) package which provides the flow velocity field, can be used to investigate the space charge effect on the electric field in large-scale particle detectors.

42 ENGINEERING↗
DOE OSTI2017

Temporal parallelization of edge plasma simulations using the parareal algorithm and the SOLPS code

We show that numerical modelling of edge plasma physics may be successfully parallelized in time. The parareal algorithm has been employed for this purpose and the SOLPS code package coupling the B2.5 finite-volume fluid plasma solver with the kinetic Monte-Carlo neutral code Eirene has been used as a test bed. The complex dynamics of the plasma and neutrals in the scrape-off layer (SOL) region makes this a unique application. It is demonstrated that a significant computational gain (more than an order of magnitude) may be obtained with this technique. The use of the IPS framework for event-based parareal implementation optimizes resource utilization and has been shown to significantly contribute to the computational gain.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2015

Performance evaluation of OpenFOAM on many-core architectures

In this article application of Open Source Field Operation and Manipulation (OpenFOAM) C++ libraries for solving engineering problems on many-core architectures is presented. Objective of this article is to present scalability of OpenFOAM on parallel platforms solving real engineering problems of fluid dynamics. Scalability test of OpenFOAM is performed using various hardware and different implementation of standard PCG and PBiCG Krylov iterative methods. Speed up of various implementations of linear solvers using GPU and MIC accelerators are presented in this paper. Numerical experiments of 3D lid-driven cavity flow for several cases with various number of cells are presented.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2017

Final Report for ALCC Allocation: Predictive Simulation of Complex Flow in Wind Farms

This report documents work performed using ALCC computing resources granted under a proposal submitted in February 2016, with the resource allocation period spanning the period July 2016 through June 2017. The award allocation was 10.7 million processor-hours at the National Energy Research Scientific Computing Center. The simulations performed were in support of two projects: the Atmosphere to Electrons (A2e) project, supported by the DOE EERE office; and the Exascale Computing Project (ECP), supported by the DOE Office of Science. The project team for both efforts consists of staff scientists and postdocs from Sandia National Laboratories and the National Renewable Energy Laboratory. At the heart of these projects is the open-source computational-fluid-dynamics (CFD) code, Nalu. Nalu solves the low-Mach-number Navier-Stokes equations using an unstructured- grid discretization. Nalu leverages the open-source Trilinos solver library and the Sierra Toolkit (STK) for parallelization and I/O. This report documents baseline computational performance of the Nalu code on problems of direct relevance to the wind plant physics application - namely, Large Eddy Simulation (LES) of an atmospheric boundary layer (ABL) flow and wall-modeled LES of a flow past a static wind turbine rotor blade. Parallel performance of Nalu and its constituent solver routines residing in the Trilinos library has been assessed previously under various campaigns. However, both Nalu and Trilinos have been, and remain, in active development and resources have not been available previously to rigorously track code performance over time. With the initiation of the ECP, it is important to establish and document baseline code performance on the problems of interest. This will allow the project team to identify and target any deficiencies in performance, as well as highlight any performance bottlenecks as we exercise the code on a greater variety of platforms and at larger scales. The current study is rather modest in scale, examining performance on problem sizes of O(100 million) elements and core counts up to 8k cores. This will be expanded as more computational resources become available to the projects.

17 WIND ENERGY↗
DOE OSTI2014

Hybrid Optimization Software Suite

The hybrid optimization software suite (HOSS) is a general purpose fully 2D/3D parallel combined finite discrete element (FDEM) code that can be used to simulate problems involving fracture and fragmentation processes, large deformation and large rotations, discrete particle systems, solid-fluid interaction in hydro fracture problems, etc. HOSS uses a hybrid approach that combines finite-element and discrete-element methods with a novel computational fluid dynamics solver. The finite-element method is often used to analyze a material or object and how it responds to stress. The discrete-element method analyzes stresses and displacements in a volume containing a large number of particles, such as grains of sand. Fluid dynamics analyzes the fluid flow inside, around, or through solid domains. With these processes combined, HOSS represents a paradigm shift when it comes to generating accurate simulations of material deformations and failure. HOSS can perform the following: Resolves problems that consist of millions of deforming fracturing, and interacting solid particles; Requires no coupling—it naturally integrates with all regimes of fluid flow; Resolves all regimes of fluid flow, such as Stokes, low/high Reynolds, subsonic/transonic/supersonic/hypersonic, compressible/incompressible, viscid/inviscid, Newtonian/Non-Newtonian and turbulent/laminar flow; Combines all flow regimes in the same solver; Numerically stable for all flow regimes; Naturally combines different flow regimes in the same problem; Possesses material library for fluid behavior; Naturally matches fluid solver to solid solver; Includes non-inertial Eulerian formulation for fluids; Uses highly efficient parallel computing based on the Virtual Parallel Machine, thus enabling easy porting between different parallel architectures, including possible future architectures; Designed specifically for multi-physics problems in research and industry in virtual experimentation format, thereby complementing both theoretical-computational and experimental tools; Can be extended to address millions of deforming solid particles immersed in fluid flowing at all regimes; Can be extended to address biological applications, such as modeling bone fracture or red blood cells in blood plasma

Knight, Earl E.↗
DOE OSTI2017

Drekar v.2.0

Drekar is an application code that solves partial differential equations for fluids that can be optionally coupled to electromagnetics. Drekar solves low-mach compressible and incompressible computational fluid dynamics (CFD), compressible and incompressible resistive magnetohydrodynamics (MHD), and multiple species plasmas interacting with electromagnetic fields. Drekar discretization technology includes continuous and discontinuous finite element formulations, stabilized finite element formulations, mixed integration finite element bases (nodal, edge, face, volume) and an initial arbitrary Lagrangian Eulerian (ALE) capability. Drekar contains the implementation of the discretized physics and leverages the open source Trilinos project for both parallel solver capabilities and general finite element discretization tools. The code will be released open source under a BSD license. The code is used for fundamental research for simulation of fluids and plasmas on high performance computing environments.

Seefeldt, Ben↗
DOE OSTI2018

In‐memory integration of existing software components for parallel adaptive unstructured mesh workflows

Summary Reliable mesh‐based simulations are needed to solve complex engineering problems. Mesh adaptivity can increase reliability by reducing discretization errors but requires multiple software components to exchange information. Often, components exchange information by reading and writing a common file format. This file‐based approach becomes a problem on massively parallel computers where filesystem bandwidth is a critical performance bottleneck. Our approach using data streams and component interfaces avoids the filesystem bottleneck. In this paper, we present these techniques and their use for coupling mesh adaptivity to the PHASTA computational fluid dynamics solver, the Albany multi‐physics framework, and the Omega3P linear accelerator frequency analysis applications. Performance results are reported on up to 16,384 cores of an Intel Knights Landing‐based system.

Smith, Cameron W.↗
DOE OSTI2025

AMR-Wind: A Performance-Portable, High-Fidelity Flow Solver for Wind Farm Simulations

We present AMR-Wind, a verified and validated high-fidelity computational-fluid-dynamics code for wind farm flows. AMR-Wind is a block-structured, adaptive-mesh, incompressible-flow solver that enables predictive simulations of the atmospheric boundary layer and wind plants. It is a highly scalable code designed for parallel high-performance computing with a specific focus on performance portability for current and future computing architectures, including graphical processing units (GPUs). In this paper, we detail the governing equations, the numerical methods, and the turbine models. Establishing a foundation for the correctness of the code, we present the results of formal verification and validation. The verification studies, which include a novel actuator line test case, indicate that AMR-Wind is spatially and temporally second-order accurate. The validation studies demonstrate that the key physics capabilities implemented in the code, including actuator disk models, actuator line models, turbulence models, and large eddy simulation (LES) models for atmospheric boundary layers, perform well in comparison to reference data from established computational tools and theory. We conclude with a demonstration simulation of a 12-turbine wind farm operating in a turbulent atmospheric boundary layer, detailing computational performance and realistic wake interactions.

17 WIND ENERGY↗