DOE OSTI2019
The goal of the ExaWind project is to enable predictive simulations of wind farms comprised of many megawatt-scale turbines situated in complex terrain. Predictive simulations will require computational fluid dynamics (CFD) simulations for which the mesh resolves the geometry of the turbines and captures the rotation and large deflections of blades. Whereas such simulations for a single turbine are arguably petascale class, multi-turbine wind farm simulations will require exascale-class resources. The primary physics codes in the ExaWind project are Nalu-Wind, which is an unstructured-grid solver for the acoustically incompressible Navier-Stokes equations, and OpenFAST, which is a whole-turbine simulation code. The Nalu-Wind model consists of the mass-continuity Poisson-type equation for pressure and a momentum equation for the velocity. For such modeling approaches, simulation times are dominated by linear-system setup and solution for the continuity and momentum systems. For the ExaWind challenge problem, the moving meshes greatly affect overall solver costs as reinitialization of matrices and recomputation of preconditioners is required at every time step. This milestone represents an effort to increase the fidelity of Nalu-Wind at a fixed resolution through the implementation of a tensor-product based, matrix-free high order scheme. High order finite element methods have increased local work per datum communicated and have the potential to provide significantly more accurate solutions at a fixed number of degrees of freedom. Previous to this milestone, Nalu-Wind had an arbitrary order Control Volume Finite Element Method discretization as a solver option, but it required too much memory and was too slow to be of practical use. The work in this milestone addresses these issues by first implementing an implicit, high order solver that only partially assembles the global system. This reduces the memory footprint of the high-order scheme by orders of magnitude for higher polynomial orders. Second, a faster, tensor-product based method for evaluating the action of the left-hand side was implemented. This reduces the amount of computational work required by the scheme and dramatically enhanced the time-to-solution on example problems. Finally, this milestone is an evaluation of the value of high order methods in the wind application space. With the enhancements to memory and computational cost, accuracy vs. time-to-solution was evaluated for several resolutions on an under-resolved Taylor Green vortex test case. Results show that the high order scheme is cost-competitive with the production low-order schemes in Nalu-Wind, being moderately more expensive than the production edge-based vertex centered finite volume scheme. The evaluation of accuracy on the test case shows a potential benefit to high order at the highest resolution while not deteriorating accuracy on the lowest tested resolution. More work is needed to show value in the wind application, but positive strides have been made.