Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “parallel computing methodologies”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3

Analyzing the Performance Trade-Off in Implementing User-Level Threads

User-level threads have been widely adopted as a means of achieving lightweight concurrent execution without the costs of OS-level threads. Nevertheless, the costs of managing user-level threads represent a performance barrier that dictates how fine grained the concurrency exposed by an application can be without incurring significant overheads; this in turn may translate into insufficient parallelism to exploit highly parallel systems. This article is a deep dive into the fundamental costs in implementing user-level threads. We first identify that one of the highest sources of fork-join overheads stems from deviations, events that incur context switching during the execution of a thread and disrupt a run-to-completion execution. We then conduct an in-depth investigation of a wide spectrum of methods with respect to how they handle deviations while covering both parent- and child-first scheduling policies. Our methodology involves a comprehensive instruction- and cache-level analysis of all methods on several modern CPU architectures. Finally, the primary finding of our evaluation is that dynamic promotion methods that assume the absence of deviation and dynamically provide context-switching support offer the best trade-off between performance and capability when the likelihood of deviation is low.

97 MATHEMATICS AND COMPUTING↗

Water–Hydrocarbon Interactions in Anionic Pyrene Monohydrate

Interactions between water and polycyclic aromatic hydrocarbons are essential in many aspects of chemistry, from interstellar and atmospheric processes to interfacial hydrophobicity and wetting phenomena. Despite their growing importance, the intermolecular potentials of the water-hydrocarbon interactions are underdeveloped compared to water-water potentials, and there are similarly few experimental probes that are sensitive to the details of the water-hydrocarbon potential. We present a combined experimental and computational study of anionic pyrene monohydrate, one of the simplest water/hydrocarbon clusters. The action spectrum in the OH region of the mass-selected cluster ion provides a rigorous benchmark for intermolecular potentials and computational methodologies. We identify missing intermolecular interactions and shortcomings in conventional dynamics calculations by comparing experimental data to density functional theory and classical molecular dynamics calculations. Kinetic trapping is prevalent, even for one water molecule and one pyrene molecule, leading to slow equilibration in conventional molecular dynamics calculations, even on nanosecond timescales and at low temperatures (50 K). At constant energy, temperature fluctuations for the pair of molecules are substantial. Immersing the system in a bath of soft spheres and employing parallel tempering alleviates kinetic trapping and dampens temperature fluctuations, bringing the system closer to the thermodynamic limit. With such augmented sampling, a simple, flexible water model reproduces the linewidth and the asymmetric broadening of the symmetric OH stretching mode, which we assign to spectral diffusion. In the OH stretching region, dynamics calculations predict a more intense antisymmetric peak than experiments observe but do not predict the bimodal split symmetric peak that the experiments show. Furthermore, our work suggests that electronic polarization, missing in the empirical force field, is responsible for the first discrepancy and that quantum nuclear effects, captured neither in density functional theory nor in classical dynamics, may be responsible for the second.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

High-throughput determination of Hubbard $U$ and Hund $J$ values for transition metal oxides via the linear response formalism

DFT+U provides a convenient, cost-effective correction for the self-interaction error (SIE) that arises when describing correlated electronic states using conventional approximate density functional theory (DFT). The success of a DFT+U(+J) calculation hinges on the accurate determination of its Hubbard U and Hund J parameters, and the linear response (LR) methodology has proven to be computationally effective and accurate for calculating these parameters. This study provides a high-throughput computational analysis of the U and J values for transition metal d-electron states in a representative set of over 1000 magnetic transition metal oxides (TMOs), providing a frame of reference for researchers who use DFT+U to study transition metal oxides. In order to perform this high-throughput study, an ATOMATE workflow is developed for calculating U and J values automatically on massively parallel supercomputing architectures. Here, to demonstrate an application of this workflow, the spin-canting magnetic structure and unit cell parameters of the multiferroic olivine LiNiPO4 are calculated using the computed Hubbard U and Hund J values for Ni-d and O-p states, and are compared with experiment. Both the Ni-d U and J corrections have a strong effect on the Ni-moment canting angle. Additionally, including a O-pU value results in a significantly improved agreement between the computed lattice parameters and experiment

36 MATERIALS SCIENCE↗

Computational Analysis of Hydraulic Efficiency of Michigan DOT Cover C

Drainage structures are used to capture stormwater runoff in streets and highways in urban environments. These drainage structures, which typically consist of catch basins with grates, inlets, or combination grates/inlets, collect stormwater runoff and discharge through buried conveyance systems. They are strategically placed for public safety in curb and gutter systems to provide efficient drainage of water from roadways and thus reduce the risk of hydroplaning. The performance of drainage structures is measured in terms of hydraulic efficiency, which is defined as the percentage of flow captured by the basin as compared to the total flow drainage to the structure. Understanding of the performance of these drainage structures helps designers properly space inlets to promote an economic design that ensures the safety of the traveling public. The current design methodology used to determine drainage structure follows guidelines established in the current Michigan Department of Transportation (MDOT) Drainage Manual (2006). The guidelines in the MDOT Drainage Manual were modeled after the Federal Highway Administration’s (FHWA) Hydraulic Engineering Circular 22 “Urban Drainage Design” (HEC-22). HEC-22 includes empirically derived equations to calculate the interception capacity of drainage structures for several commonly used grate configurations, such as the parallel bar, curved vane and tilt bar grates, which are based on a research study performed by Burgi et al. in the 1970s. MDOT uses several drainage structures to capture runoff that are detailed as Standard Plans. Many of these drainage structures utilize sinusoidal type grates that are not described in HEC-22. Physical modeling of these structures has been limited, posing the need to have them analyzed to verify their capture efficiency. Current MDOT practice is to assume a similar sized reticuline grate, as described in HEC-22, for capture efficiencies. Until recently, evaluating the hydraulic performance of drainage structures was limited to physical modeling in a hydraulics laboratory. With advances in engineering software and computing power, computational fluid dynamics (CFD) modeling has become a more cost-effective alternative. The Federal Highway Administration (FHWA) provides states the option to evaluate their drainage structures using CFD through the Transportation Pooled Fund Program. This study, “Computational Analysis of Hydraulic Efficiency of Michigan DOT Cover C,” was carried out using the pooled fund. MDOT’s Cover C was chosen as the first test candidate, given its similar sinusoidal pattern to other MDOT grates, but it is typically used for high-volume, higher speed applications. A similar version, Cover CX, is used on interstate highways but does not have traverse bars for bicycle safety. Additional grates may be considered for evaluation in the future.

42 ENGINEERING↗

Predicting nonequilibrium Green’s function dynamics and photoemission spectra via nonlinear integral operator learning

Understanding the dynamics of nonequilibrium quantum many-body systems is an important research topic in a wide range of fields across condensed matter physics, quantum optics, and high-energy physics. However, numerical studies of large-scale nonequilibrium phenomena in realistic materials face serious challenges due to intrinsic high-dimensionality of quantum many-body problems and the absence of time-invariance. The nonequilibrium properties of many-body systems can be described by the dynamics of the correlator, or the Green's function of the system, whose time evolution is given by a high-dimensional system of integro-differential equations, known as the Kadanoff–Baym equations (KBEs). The time-convolution term in KBEs, which needs to be recalculated at each time step, makes it difficult to perform long-time numerical simulation. In this paper, we develop an operator-learning framework based on recurrent neural networks (RNNs) to address this challenge. We utilize RNNs to learn the nonlinear mapping between Green's functions and convolution integrals in KBEs. By using the learned operators as a surrogate model in the KBE solver, we obtain a general machine-learning scheme for predicting the dynamics of nonequilibrium Green's functions. Besides significant savings per each time step, the new methodology reduces the temporal computational complexity from $O(N_t^3)$ to $O(N_t)$ where N t is the number of steps taken in a simulation, thereby making it possible to study large many-body problems which are currently infeasible with conventional KBE solvers. Through various numerical examples, we demonstrate the effectiveness of the operator-learning based approach in providing accurate predictions of physical observables such as the reduced density matrix and time-resolved photoemission spectra. Moreover, our framework exhibits clear numerical convergence and can be easily parallelized, thereby facilitating many possible further developments and applications.

97 MATHEMATICS AND COMPUTING↗

Contextual subspace variational quantum eigensolver calculation of the dissociation curve of molecular nitrogen on a superconducting quantum computer

Abstract We present an experimental demonstration of the Contextual Subspace Variational Quantum Eigensolver on superconducting hardware. Calculating the potential energy curve of molecular nitrogen proves challenging for many conventional quantum chemistry techniques, since static correlation dominates in the dissociation limit. Our quantum simulations retain good agreement with the Full Configuration Interaction energy, outperforming all benchmarked single-reference wavefunction techniques in capturing the bond-breaking appropriately. Moreover, our methodology is competitive with multiconfigurational approaches but at a saving of quantum resource, meaning larger active spaces can be treated for a fixed qubit allowance. To achieve this result, we deploy an error mitigation/suppression strategy comprised of Dynamical Decoupling, Measurement-Error Mitigation and Zero-Noise Extrapolation. Circuit parallelization also provides passive noise-averaging and improves the effective shot yield to reduce the measurement overhead. Furthermore, we introduce a modified adaptive ansatz construction algorithm that incorporates hardware awareness into our variational circuits, minimizing the transpilation cost for the target qubit topology.

Physics↗

Mesoscopic Modeling and Rapid Simulation of Incremental Changes in Epidemic Scenarios on GPUs

In simulation-based studies and analyses of epidemics, a major challenge lies in resolving the conflict between fidelity of models and the speed of their simulation. Another related challenge arises in dealing with the large number of what–if scenarios that need to be explored. Here, we describe new computational methods that together provide an approach to dealing with both challenges. A mesoscopic modeling approach is described that strikes a middle ground between macroscopic models based on coupled differential equations and microscopic models built on fine-grained behaviors at the individual entity level. The mesoscopic approach offers the ability to incorporate complex compositions of multiple layers of dynamics even while retaining the potential for aggregate behaviors at varying levels. It also is an excellent match to the accelerator-based architectures of modern computing platforms in which graphical processing units (GPUs) can be exploited for fast simulation via the parallel execution mode of single instruction multiple thread (SIMT). The challenge of simulating a large number of scenarios is addressed via a method of sharing model state and computation across a tree of what–if scenarios that are localized, incremental changes to a large base simulation. A combination of the mesoscopic modeling approach and the incremental what–if scenario tree evaluation has been implemented in the software on modern GPUs. Synthetic simulation scenarios are presented to demonstrate the computational characteristics of our approach. Results from the experiments with large population data, including USA, UK, and India, illustrate the modeling methodology and computational performance on thousands of synthetically generated what–if scenarios. Execution of our implementation scaled to 8192 GPUs of supercomputing platforms demonstrates the ability to rapidly evaluate what–if scenarios several orders of magnitude faster than the conventional methods.

97 MATHEMATICS AND COMPUTING↗

Randomized Preconditioned Solvers for Strong Constraint 4D-Var Data Assimilation

The Strong Constraint 4D Variational (SC-4DVAR) data assimilation method is widely used in climate and weather applications. SC-4DVAR involves solving a minimization problem to compute the maximum a posteriori estimate, which we tackle using the Gauss-Newton method. The computation of the descent direction is expensive since it involves the solution of a large-scale and potentially ill-conditioned linear system, solved using the preconditioned conjugate gradient (PCG) method. Here, to address this cost, we efficiently construct scalable preconditioners using three different randomization techniques, which all rely on a certain low-rank structure involving the Gauss-Newton Hessian. The proposed techniques come with theoretical guarantees on the condition number, and at the same time, are amenable to parallelization. We also develop an adaptive approach to estimate the sketch size and choose between the reuse or recomputation of the preconditioner. We demonstrate the performance and effectiveness of our methodology on two representative model problems—the Burgers and barotropic vorticity equation—showing a drastic reduction in both the number of PCG iterations and the number of Gauss-Newton Hessian products after including the preconditioner construction cost.

Gauss-Newton↗

In situ multi-tier auto-ignition detection applied to dual-fuel combustion simulations

Here we use an anomaly detection methodology that is centered on analyzing fourth-order joint moments (co-kurtosis), particularly focusing on its application in auto-ignition of combustion problems with large numbers of species. Unsupervised anomaly detection is challenging to generalize across problem types and domains. A recent technique, centered on analyzing information in the fourth-order joint moment co-kurtosis, has shown promise, especially for high-dimensional scientific data. In this work we present developments to the co-kurtosis based anomaly detection method needed to make it effective and scalable for large-scale distributed scientific data, such as those generated by massively parallel simulations. An in situ co-kurtosis algorithm is employed as the anomaly detection method for identifying ignition kernels in simulations of turbulent combustion. Here, we extend an existing methodology which identifies regions of the domain where anomalies are present, and add another tier of anomaly detection where the individual samples contributing to the anomaly are identified. We apply this algorithm on-the-fly to a variety of turbulent reacting flow problems and compare it to the widely used (but significantly more expensive) chemical explosive mode analysis (CEMA). We demonstrate the ability of the method to detect and identify the onset of low and high temperature ignition which can be used for computational steering, as chemical and combustion anomalies occur intermittently at spatio-temporal locations unknown a priori. Finally, we apply our lightweight in situ algorithm to an exascale high-fidelity simulation with a total of 2.4 Trillion degrees of freedom, performed using an adaptive mesh refinement solver. Furthermore, through a scalability analysis, we show that the relative computational cost of this in-situ anomaly detection algorithm compared to an iteration of the reacting flow solver is negligible.

97 MATHEMATICS AND COMPUTING↗

Drive-pressure optimization in ramp-wave compression experiments through differential evolution

Ramp-wave dynamic-compression experiments are used to examine quasi-isentropic loading paths in materials. The gradual and continuous increase in pressure created by ramp waves make these types of experiments ideal for studying nonequilibrium material behavior, such as solidification kinetics. In ramp-wave compression experiments, the input drive pressure to the experimental setup may be exerted through one of a number of different mechanisms (e.g., magnetic fields, gas-gun-driven impactors, or high-energy lasers) and is generally required for simulating such experiments. Yet, regardless of the specific mechanism, this drive pressure cannot be measured directly (measurements are generally taken at a location near the back of the experimental setup through a transparent window), leading to an inverse problem where one must determine the drive pressure at the front of the experimental setup (i.e., the input) that corresponds to the particle velocity (the output) measured near the back of the experimental setup. Furthermore, we solve this inverse problem using a heuristic optimization algorithm, known as differential evolution, coupled with a multiphysics, hydrodynamics code that simulates the compression of the experimental setup. By running many rounds of forward simulations of the experimental setup, our optimization process iteratively searches for a drive pressure that is optimized to closely reproduce the experimentally measured particle velocity near the back of the experimental setup. While our optimization methodology requires a significant number of hydrodynamics simulations to be conducted, many of these can be performed in parallel, which greatly reduces the time cost of our methodology. One novel aspect of our method for determining the drive pressure is that it does not require physical modeling of the drive mechanism and can thus be broadly applied to many types of ramp-compression experiments, regardless of the drive mechanism.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

Towards Superior Software Portability with SHAD and HPX C++ Libraries

As hardware architectures and software stacks complexity grows, development productivity, performance and software portability, quickly evolve from desirable features to actual needs. SHAD, the Scalable High-performance Algorithms and Data-structures C++ library is designed to mitigate these issues: it provides general purpose building blocks as well as high-level custom utilities, and offers a shared-memory programming abstraction which facilitates the programming of complex systems, scaling up to High Performance Computing clusters. SHAD’s portability is achieved through an abstract runtime interface, which decouples the upper layers of the library and hides the low level details of the underlying architecture. This layer enables SHAD to interface with different runtime/threading systems, e.g. Intel TBB and Global Memory and Threading (GMT). However, current backends targeting distributed systems, rely on a centralized controller which may possibly limit scalability up to hundreds of nodes and creates a network hot spot due to all to one communication for synchronization, and possibly resulting in degraded performance at high process counts. In this research, we explore HPX, the C++ standard library for parallelism and concurrency, as an additional backend in support of the SHAD library, and present the methodologies in support of local and remote task executions in SHAD with respect to HPX. Finally, we evaluate the proposed system by comparing against existing backends of SHAD and analyzing their performance on C++ Standard Template Library algorithms.

Wu, Nanmiao↗

Parallel sorting algorithm classification: is manual instrumentation necessary?

Understanding parallel algorithms is crucial for accelerating scientific simulations on complex, distributed memory, high-performance computers. Modern algorithm classification approaches learn semantics directly from source code to differentiate between algorithms, however, accessing source code is not always possible. We can learn about parallel algorithms from observing their performance, as programs running the same algorithms and using the same hardware should exhibit similar performance characteristics. We present an approach to learn algorithm classes from parallel performance data directly in order to classify algorithms without access to the source code. We extend previous work to enable classifying parallel sorting algorithms using automatic instrumentation instead of requiring manual region annotations in the source code. In this work, we design and demonstrate a study for classification of parallel sorting algorithms using parallel performance data collected from automatic instrumentation, and evaluate the performance of our new methodology on classification. We leverage Caliper to collect the performance data, Thicket for our exploratory data analysis (EDA), and PyTorch and Scikit-learn to evaluate the effectiveness of random forests, support vector machines (SVMs), decision trees, neural networks, and logistic regressions on parallel performance data. Additionally, we study noise in parallel performance data, whether the removal of noise and pre-processing of the data is necessary to accurately classify parallel sorting algorithms, and determine the effectiveness of features created from performance data. In conclusion, we demonstrate classification accuracy for these five different models of up to 97.7% across four different parallel algorithm classes.

Algorithm Classification↗

AMR-Wind [SWR-20-85]

AMR-Wind is a massively parallel, block-structured adaptive-mesh, incompressible flow solver for wind turbine and wind farm simulations. The solver is built on top of the AMReX library. AMReX is developed at LBNL , NREL , and ANL as part of the Block-Structured AMR Co-Design Center in DOE's Exascale Computing Project. AMReX library provides the mesh data structures, mesh adaptivity, as well as the linear solvers used for solving the governing equations. The primary applications for AMR-Wind are: performing large-eddy simulations (LES) of atmospheric boundary layer (ABL) flows, simulating wind farm turbine-wake interactions using actuator disk or actuator line models for turbines, and as a background solver when coupled with a near-body solver with overset methodology to perform blade-resolved simulations of multiple wind turbines within a wind farm.

Ananthan, Shreyas↗

Characterizing Output Bottlenecks of a Production Supercomputer: Analysis and Implications

This article studies the I/O write behaviors of the Titan supercomputer and its Lustre parallel file stores under production load. The results can inform the design, deployment, and configuration of file systems along with the design of I/O software in the application, operating system, and adaptive I/O libraries.We propose a statistical benchmarking methodology to measure write performance across I/O configurations, hardware settings, and system conditions. Moreover, we introduce two relative measures to quantify the write-performance behaviors of hardware components under production load. In addition to designing experiments and benchmarking on Titan, we verify the experimental results on one real application and one real application I/O kernel, XGC and HACC IO, respectively. These two are representative and widely used to address the typical I/O behaviors of applications.In summary, we find that Titan’s I/O system is variable across the machine at fine time scales. This variability has two major implications. First, stragglers lessen the benefit of coupled I/O parallelism (striping). Peak median output bandwidths are obtained with parallel writes to many independent files, with no striping or write sharing of files across clients (compute nodes). I/O parallelism is most effective when the application—or its I/O libraries—distributes the I/O load so that each target stores files for multiple clients and each client writes files on multiple targets in a balanced way with minimal contention. Second, our results suggest that the potential benefit of dynamic adaptation is limited. In particular, it is not fruitful to attempt to identify “good locations” in the machine or in the file system: component performance is driven by transient load conditions and past performance is not a useful predictor of future performance. For example, we do not observe diurnal load patterns that are predictable.

97 MATHEMATICS AND COMPUTING↗

Parallel Battery: The Framework and Process for an Intelligent and Ecological Battery System and Related Services

The concept, framework, process methodology and applications of parallel battery were proposed from both virtual and real aspects.The parallel battery was an application of ACP-based parallel intelligence in battery and related energy system areas.The real battery system was running with its equivalent, and the artificial battery system was in a virtual space, in a parallel and interactive manner.The artificial battery system contained the descriptive, predictive, and prescriptive functions on the real battery and related energy systems.There was a closed-loop workflow between the real battery system and the artificial battery system, which iteratively optimizes the battery and related energy systems, leading to a new paradigm of intelligent and ecological parallel battery system management.

ACP approach↗

Uncovering I/O demands on HPC platforms: Peeking under the hood of Santos Dumont

High-Performance Computing (HPC) platforms are required to solve the most diverse large-scale scientific problems in various research areas, such as biology, chemistry, physics, and health sciences. Researchers use a multitude of scientific softwares, which have different requirements. These include input and output operations, which directly impact performance due to the existing difference in processing and data access speeds. Thus, supercomputers must efficiently handle mixed workload when storing data from the applications. Understanding the set of applications and their performance running in a supercomputer is paramount to understanding the storage system's usage, pinpointing possible bottlenecks, and guiding optimization techniques. This research proposes a methodology and visualization tool to evaluate a supercomputer's data storage infrastructure's performance, taking into account the diverse workload and demands of the system over a long period of operation. As a study case, we focus on the Santos Dumont supercomputer, identifying inefficient usage, problematic performance factors, and providing guidelines on how to tackle those issues.

97 MATHEMATICS AND COMPUTING↗

General field evaluation in high-order meshes on GPUs

Robust and scalable function evaluation at any arbitrary point in the finite/spectral element mesh is required for querying the partial differential equation solution at points of interest, comparison of solution between different meshes, and Lagrangian particle tracking. This is a challenging problem, particularly for high-order unstructured meshes partitioned in parallel with MPI, as it requires identifying the element that overlaps a given point and computing the corresponding reference space coordinates. Here, we present a robust and efficient technique for general field evaluation in large-scale high-order meshes with quadrilaterals and hexahedra. In the proposed method, a combination of globally partitioned and processor-local maps are used to first determine a list of candidate MPI ranks, and then locally candidate elements that could contain a given point. Next, element-wise bounding boxes further reduce the list of candidate elements. Finally, Newton’s method with trust region is used to determine the overlapping element and corresponding reference space coordinates. Since GPU-based architectures have become popular for accelerating computational analyses using meshes with tensor-product elements, specialized kernels have been developed to utilize the proposed methodology on GPUs. The method is also extended to enable general field evaluation on surface meshes. The paper concludes by demonstrating the use of the proposed method in various applications ranging from mesh-to-mesh transfer during r-adaptivity to Lagrangian particle tracking.

97 MATHEMATICS AND COMPUTING↗

Integrating ytopt and libEnsemble to autotune OpenMC

Ytopt is a Python machine-learning-based autotuning software package developed within the ECP PROTEAS-TUNE project. The ytopt software adopts an asynchronous search framework that consists of sampling a small number of input parameter configurations and progressively fitting a surrogate model over the input-output space until exhausting the user-defined maximum number of evaluations or the wall-clock time. libEnsemble is a Python toolkit for coordinating workflows of asynchronous and dynamic ensembles of calculations across massively parallel resources developed within the ECP PETSc/TAO project. libEnsemble helps users take advantage of massively parallel resources to solve design, decision, and inference problems and expands the class of problems that can benefit from increased parallelism. In this paper we present our methodology and framework to integrate ytopt and libEnsemble to take advantage of massively parallel resources to accelerate the autotuning process. Specifically, we focus on using the proposed framework to autotune the ECP ExaSMR application OpenMC, an open source Monte Carlo particle transport code. OpenMC has seven tunable parameters some of which have large ranges such as the number of particles in-flight, which is in the range of 100,000 to 8 million, with its default setting of 1 million. Setting the proper combination of these parameter values to achieve the best performance is extremely time-consuming. Therefore, we apply the proposed framework to autotune the MPI/OpenMP offload version of OpenMC based on a user-defined metric such as the figure of merit (FoM) (particles/s) or energy efficiency energy-delay product (EDP) on Crusher at Oak Ridge Leadership Computing Facility. In conclusion, the experimental results show that we achieve the improvement up to 29.49% in FoM and up to 30.44% in EDP.

Autotuning↗