SEARCH · Engineering Papers
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Explore the source record for details and available documents.
Explore the source record for details and available documents.
Explore the source record for details and available documents.
Fourier pseudo-spectral methods for nonlinear partial differential equations are of wide interest in many areas of advanced computational science, including direct numerical simulation of three-dimensional (3-D) turbulence governed by the Navier-Stokes equations in fluid dynamics. This paper presents a new capability for simulating turbulence at a new record resolution up to 35 trillion grid points, on the world's first exascale computer, Frontier, comprising AMD MI250x GPUs with HPE's Slingshot interconnect and operated by the US Department of Energy's Oak Ridge Leadership Computing Facility (OLCF). Key programming strategies designed to take maximum advantage of the machine architecture involve performing almost all computations on the GPU which has the same memory capacity as the CPU, performing all-to-all communication among sets of parallel processes directly on the GPU, and targeting GPUs efficiently using OpenMP offloading for intensive number-crunching including 1-D Fast Fourier Transforms (FFT) performed using AMD ROCm library calls. With 99% of computing power on Frontier being on the GPU, leaving the CPU idle leads to a net performance gain via avoiding the overhead of data movement between host and device except when needed for some I/O purposes. Memory footprint including the size of communication buffers for MPI_ALLTOALL is managed carefully to maximize the largest problem size possible for a given node count. Detailed performance data including separate contributions from different categories of operations to the elapsed wall time per step are reported for five grid resolutions, from 2048 3 on a single node to 32768 3 on 4096 or 8192 nodes out of 9408 on the system. Both 1D and 2D domain decompositions which divide a 3D periodic domain into slabs and pencils respectively are implemented. The present code suite (labeled by the acronym GESTS, GPUs for Extreme Scale Turbulence Simulations) achieves a figure of merit (in grid points per second) exceeding goals set in the Center for Accelerated Application Readiness (CAAR) program for Frontier. The performance attained is highly favorable in both weak scaling and strong scaling, with notable departures only for 2048 3 where communication is entirely intra-node, and for 32768 3 , where a challenge due to small message sizes does arise. Communication performance is addressed further using a lightweight test code that performs all-to-all communication in a manner matching the full turbulence simulation code. Performance at large problem sizes is affected by both small message size due to high node counts as well as dragonfly network topology features on the machine, but is consistent with official expectations of sustained performance on Frontier. Overall, although not perfect, the scalability achieved at the extreme problem size of 32768 3 (and up to 8192 nodes — which corresponds to hardware rated at just under 1 exaflop/sec of theoretical peak computational performance) is arguably better than the scalability observed using prior state-of-the-art algorithms on Frontier's predecessor machine (Summit) at OLCF. New science results for the study of intermittency in turbulence enabled by this code and its extensions are to be reported separately in the near future.
A high-order hybrid continuous-Galerkin numerical method, designed for the simulation of non-linear, non -hydrostatic internal waves and turbulence in long computational domains with complex bathymetry, is presented. The spatial discretization in the non-periodic wave-propagating directions, utilizes the nodal spectral element method. Such a high-order element-based discretization allows the highly accurate representation of complex domain geometry along with the flexibility of concentrating resolution in areas of interest. Under the assumption of the normal-to-isobath propagation of non-linear internal waves, a third periodic direction is incorporated via a Fourier-Galerkin discretization. The distinct non-hydrostatic nature of non-linear internal waves and, any instabilities and turbulence therein, necessitates the numerically challenging solution of the pressure Poisson problem. A defining feature of this work is the application of a domain decomposition approach, combined with block-Jacobi/deflation-based preconditioning to the pressure Poisson problem. Such a combined approach is particularly suitable for the long high aspect-ratio complex domains of interest and enables the efficient high-accuracy reproduction of the non-hydrostatic dynamics of non-linear internal waves. Implementation details are also described in the context of the stability of the solver and its parallelization strategy. A series of benchmarks of increasing complexity demonstrate the robustness of the flow solver. The benchmarks culminate with the three-dimensional simulation of a convectively breaking mode-one non-linear internal wave over a realistic South-China-Sea bathymetric transect and background current/stratification profiles.
Here, we present an efficient physics-informed neural networks (PINNs) framework, termed Adaptive Interface-PINNs (AdaI-PINNs), to improve the modeling of interface problems with discontinuous coefficients and/or interfacial jumps. This framework is an enhanced version of its predecessor, Interface PINNs or I-PINNs (Sarma et al.; https://doi.org/10.1016/j.cma.2024.117135), which involves domain decomposition and assignment of different predefined activation functions to the neural networks in each subdomain across a sharp interface, while keeping all other parameters of the neural networks identical. In AdaI-PINNs, the activation functions vary solely in their slopes, which are trained along with the other parameters of the neural networks. This makes the AdaI-PINNs framework fully automated without requiring preset activation functions. Comparative studies on one-dimensional, two-dimensional, and three-dimensional benchmark elliptic interface problems reveal that AdaI-PINNs outperform I-PINNs, reducing computational costs by 2-6 times while producing similar or better accuracy.
Here, we present a novel physics-informed neural networks (PINNs) framework for modeling interface problems, termed Interface PINNs (I-PINNs). I-PINNs uses different neural networks for any two subdomains separated by a sharp interface such that the neural networks differ only through their activation functions while the other parameters remain identical. The performance of I-PINNs, conventional PINNs, and other existing domain-decomposition PINNs methods such as extended PINNs (XPINNs) and multi-domain PINN (M-PINN) is compared through several one-dimensional, two-dimensional, and three-dimensional benchmark elliptic interface problems. The results demonstrate that I-PINNs provides a root-mean-square-error accuracy, at least two orders of magnitude better than conventional PINNs and XPINNs at approximately one-tenth of the computational cost of conventional PINNs and half the cost of XPINNs. Additionally, while I-PINNs and M-PINN provide comparable accuracies, M-PINN is found to be approximately 50% more expensive.
Scaling up new scientific technologies from laboratory to industry often involves demonstrating performance on a larger scale. Computer simulations can accelerate design and predictions in the deployment process, though traditional numerical methods are computationally intractable even for intermediate pilot plant scales. Recently, the component reduced order modeling method has been developed to tackle this challenge by combining projection reduced order modeling and discontinuous Galerkin domain decomposition. However, while many scientific or engineering applications involve nonlinear physics, this method has only been demonstrated for various linear systems. In this work, the component reduced order modeling method is extended to steady Navier–Stokes flow, with application to general nonlinear physics in view. The large‐scale, global domain is decomposed into a combination of small‐scale unit component. Linear subspaces for flow velocity and pressure are identified via proper orthogonal decomposition over sample snapshots collected from each small‐scale unit component. Velocity bases are augmented with a pressure supremizer to satisfy the inf–sup condition for stable pressure prediction. Two different nonlinear reduced order modeling methods are employed and compared for efficient evaluation of nonlinear advection: A third‐order tensor projection operator and the empirical quadrature procedure. The proposed method is demonstrated on the flow over arrays of five different unit objects, achieving a 23‐fold speedup with less than 4% relative error in domains up to 256 times larger than the unit components. Furthermore, a numerical experiment with the pressure supremizer strongly indicates the need for a supremizer for stable pressure prediction. A comparison between the tensorial approach and the empirical quadrature procedure revealed a slight advantage of the empirical quadrature procedure. The framework is compared with an alternating Schwarz‐based reduced‐order approach, demonstrating improved efficiency and robustness for the DG‐based global solver while retaining flexibility for sub‐scale iterative solvers. The method is further extended to a coupled advection–diffusion and Navier–Stokes system, illustrating its applicability to multi‐physics problems and its potential for more general, inter‐coupled nonlinear systems.
Partial differential equation (PDE)-governed inverse problems are fundamental across various scientific and engineering applications; yet they face significant challenges due to nonlinearity, ill-posedness, and sensitivity to noise. Here, we introduce a computational framework, regularization by denoising using diffusion models for partial differential equations (RED-DiffEq), by integrating physics-driven inversion and data-driven learning. RED-DiffEq leverages pretrained diffusion models as a regularization mechanism for PDE-governed inverse problems. We apply RED-DiffEq to solve the full waveform inversion problem in geophysics, a challenging seismic imaging technique that seeks to reconstruct high-resolution subsurface velocity models from seismic measurement data. Our method shows enhanced accuracy and robustness compared to benchmark methods. Additionally, it exhibits strong generalization and domain decomposition capacity, enabling the inversion of more complex velocity models with larger domains than those used in training the diffusion model. Our framework can also be directly applied to diverse PDE-governed inverse problems.
Contact phenomena are crucial for understanding the behavior of mechanical systems. However, existing computational approaches for simulating mechanical contact often face numerical challenges, such as inaccurate physical predictions, energy conservation errors, and unwanted oscillations. Here, we introduce an alternative technique for simulating dynamic contact based on the non‐overlapping Schwarz alternating method, originally developed for domain decomposition. In multibody contact scenarios, this method treats each body as a separate, non‐overlapping domain and prevents interpenetration using an alternating Dirichlet–Neumann iterative process. This approach has a strong theoretical foundation, eliminates the need for contact constraints, and offers flexibility, making it ideal for multiscale and multiphysics applications. We conducted a numerical comparison between the Schwarz method and traditional methods, such as the Lagrange multiplier and penalty methods, focusing on a benchmark impact problem. Our results indicate that the Schwarz alternating method outperforms traditional methods in several key areas: it provides more accurate predictions for various measurable quantities and demonstrates exceptional energy conservation capabilities. To address unwanted oscillations in contact velocities and forces, we explored various algorithms and stabilization techniques, ultimately opting for the naïve‐stabilized Newmark scheme for its simplicity and effectiveness. Additionally, we validated the efficiency of the Schwarz method in a three‐dimensional impact problem, highlighting its inherent capacity to accommodate different mesh topologies, time‐integration schemes, and time steps for each interacting body.
Here, a major challenge in simulating the thermal behavior in additive manufacturing processes is the disparate length and time scales between transport phenomena occurring in the melt pool and the component. A common simulation approach relies on spatial decomposition for parallel computing, but due to the nature of heat transfer in AM, where most of the computational expenditure is localized near the melt pool, the computational speedup from spatial parallelization saturates quickly. Therefore, additional parallelism by means of time-domain decomposition is needed to fully take advantage of high-performance computing (HPC) resources. This work introduces a time-parallel method to improve the computational scalability of additive manufacturing simulations on HPC systems, while maintaining high temporal resolution of heat transfer near the melt pool. The method, inspired by the nonlinear paraexp formalism, performs an iterative superposition of nonlinear solutions to the initial value problem, integrating the heat equation across overlapping time-parallel intervals. For a single layer of the NIST AMB2018–01 L7 benchmark problem, the method achieves a 38.51x speedup in wall-clock time with a maximum error in the global temperature solution of 0.99%. This reduces the total solution time from 196.72 min to 5.11 min on 128 nodes of the ORNL Frontier supercomputer. The tradeoff between accuracy and total wall-clock time is investigated and recommendations for time-parallel deployment for AM problems are made.
SUMMARY We present a time-domain distributional finite-difference scheme based on the Lebedev staggered grid for the numerical simulation of wave propagation in acoustic and elastic media. The central aspect of the proposed method is the representation of the stresses and displacements with different sets of B-splines functions organized according to the staggered grid. The distributional finite-difference approach allows domain-decomposition, heterogeneity of the medium, curvilinear mesh, anisotropy, non-conformal interfaces, discontinuous grid and fluid–solid interfaces. Numerical examples show that the proposed scheme is suitable to model wave propagation through the Earth, where sharp interfaces separate large, relatively homogeneous layers. A few domains or elements are sufficient to represent the Earth’s internal structure without relying on advanced meshing techniques. We compare seismograms obtained with the proposed scheme and the spectral element method, and we show that our approach offers superior accuracy, reduced memory usage, and comparable efficiency.
Coupling low- and high-fidelity codes is a useful way to model complex engineering systems. Thanks to the high-fidelity code, complex phenomena can be resolved in areas of the system where this is required, yet the efficiency of the lower-fidelity code is still retained in modeling the rest of the system. This document details the theory and implementation of coupling two different thermal-hydraulic codes: the system thermal-hydraulics (STH) code System Analysis Module (SAM) and the coarse-mesh computational fluid dynamics (CFD) code Pronghorn. Both applications are included in the Comprehensive Reactor Analysis Bundle (BlueCRAB) code suite and are based on the Multiphysics Object-Oriented Simulation Environment (MOOSE) framework. The domain overlapping (DO) coupling approach was adopted, as it offers proven advantages over more conventional domain decomposition methods. In the DO coupling, SAM provides Pronghorn with boundary conditions that depend on the system-level simulation of the entire plant. In return, the overlapping coupled SAM components–termed “surrogate components”–receive friction factors and source terms computed online based on the Pronghorn simulation. The framework is designed to be generic and enable coupling regardless of geometry and the number of inlet/outlet boundaries in the DO coupled domain. The developed method leads to consistent pressure drops, enthalpies, and scalar concentrations when comparing the coupled SAM and Pronghorn simulations. This document presents the DO coupling approach, along with two verification and two demonstration cases. The proposed problems explore different physical aspects relevant to nuclear reactor analysis, including buoyancy-driven flows, complex flow patterns, and multiple inlets and outlets. Periodically, new versions of this “BlueCRAB Domain Overlapping Coupling: Theory & Verification” report will be issued to reflect future developments and verification tests.
Physics-Informed Neural Networks (PINNs) have emerged as a key tool in Scientific Machine Learning since their introduction in 2017, enabling the efficient solution of ordinary and partial differential equations using sparse measurements. Over the past few years, significant advancements have been made in the training and optimization of PINNs, covering aspects such as network architectures, adaptive refinement, domain decomposition, and the use of adaptive weights and activation functions. A notable recent development is the Physics-Informed Kolmogorov-Arnold Networks (PIKANS), which leverage a representation model originally proposed by Kolmogorov in 1957, offering a promising alternative to traditional PINNs. In this review, we provide a comprehensive overview of the latest advancements in PINNs, focusing on improvements in network design, feature expansion, optimization techniques, uncertainty quantification, and theoretical insights. We also survey key applications across a range of fields, including biomedicine, fluid and solid mechanics, geophysics, dynamical systems, heat transfer, chemical engineering, and beyond. Lastly, we review computational frameworks and software tools developed by both academia and industry to support PINN research and applications.
Fourier neural operators (FNOs) are a recently introduced neural network architecture for learning solution operators of partial differential equations (PDEs), which have been shown to perform significantly better than comparable deep learning approaches. Once trained, FNOs can achieve speed-ups of multiple orders of magnitude over conventional numerical PDE solvers. However, due to the high dimensionality of their input data and network weights, FNOs have so far only been applied to two-dimensional or small three-dimensional problems. To remove this limited problem-size barrier, we propose a model-parallel version of FNOs based on domain-decomposition of both the input data and network weights. Here, we demonstrate that our model-parallel FNO is able to predict time-varying PDE solutions of over 2.6 billion variables on Perlmutter using up to 512 A100 GPUs and show an example of training a distributed FNO on the Azure cloud for simulating multiphase CO 2 dynamics in the Earth’s subsurface.
Modeling plasma accelerators is a computationally challenging task and the quasi-static particle-in-cell algorithm is a method of choice in a wide range of situations. In this work, we present the first performance-portable, quasi-static, three-dimensional particle-in-cell code HiPACE++. By decomposing all the computation of a 3D domain in successive 2D transverse operations and choosing appropriate memory management, HiPACE++ demonstrates orders-of-magnitude speedups on modern scientific GPUs over CPU-only implementations. The 2D transverse operations are performed on a single GPU, avoiding time-consuming communications. The longitudinal parallelization is done through temporal domain decomposition, enabling near-optimal strong scaling from 1 to 512 GPUs. HiPACE++ is a modular, open-source code enabling efficient modeling of plasma accelerators from laptops to state-of-the-art supercomputers.
Modern HPC systems are increasingly relying on greater core counts and wider vector registers. Thus, applications need to be adapted to fully utilize these hardware capabilities. One class of applications that can benefit from this increase in parallelism are molecular dynamics simulations. In this paper, we describe our efforts at modernizing the ESPResSo++ simulation package for molecular dynamics by restructuring its particle data layout for efficient memory accesses and applying vectorization techniques to benefit the calculation of short-range non-bonded forces, which results in an overall three times speedup and serves as a baseline for further optimizations. We also implement fine-grained parallelism for multi-core CPUs through HPX, a C++ runtime system which uses lightweight threads and an asynchronous many-task approach to maximize concurrency. Our goal is to evaluate the performance of an HPX-based approach compared to the bulk-synchronous MPI-based implementation. This requires the introduction of an additional layer to the domain decomposition scheme that defines the task granularity. On spatially inhomogeneous systems, which impose a corresponding load-imbalance in traditional MPI-based approaches, we demonstrate that by choosing an optimal task size, the efficient work-stealing mechanisms of HPX can overcome the overhead of communication resulting in an overall 1.4 times speedup compared to the baseline MPI version.
The current state of the EUTERPE code is described with emphasis on the implemented models and their numerical implementation. The code solves the multi-species electromagnetic gyrokinetic equations in the full volume of a three-dimensional domain. Noise reduction of the particle-in-cell method is achieved by using a δf-method and Fourier filters. The field equations are discretized with B-splines and the resulting system of equations is solved iteratively. For linear simulations a phase-factor transformation is applied in order to strongly reduce the necessary grid resolution. Apart from the full gyrokinetic model, other numerically less expensive hybrid models are also implemented. They are mainly tailored for comparison with fluid theory and for studying the interaction of the bulk plasma with fast particles. The code is parallelized for CPUs by particle and domain decomposition. Good scalability up to several thousand nodes is demonstrated.