Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Parallel application frameworks”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 487 records · Page 27

Aspen-based performance and energy modeling frameworks

Now with the anticipation of exascale architectures, energy consumption is becoming one of the critical design parameters, especially in light of the energy budget of 20-30 Megawatts set by the U.S. Department of Energy. Understanding an application’s execution pattern and its energy footprint is critical to improving application operation on a diverse heterogeneous architecture. Applying application-specific performance optimization can consequently improve energy consumption. However, this approach is only applicable to current systems. As we enter a new era of exascale architectures that is projected to contain more complex memory hierarchies, increased levels of parallelism, heterogeneity in hardware, and complex programming models and techniques, energy and performance management is getting more cumbersome. We therefore propose techniques that predict the energy consumption beforehand or at runtime to enable proactive tuning. Such energy prediction approaches must be generic and adapt themselves at runtime to changing application and hardware configurations. Most existing energy estimation and prediction approaches are empirical in nature and thus tied to current systems. To overcome this limitation, we propose two energy estimation techniques: ACEE (Algorithmic and Categorical Energy Estimation), which uses a combination of analytical and empirical modeling techniques; and AEEM (Aspen’s Embedded Energy Estimation), a system-level analytical energy estimation technique. Both of these models incorporate the Aspen domain specific language for performance modeling. We present the methodologies of these two models and test their accuracy using five proxy applications. We also describe three use cases.

79 ASTRONOMY AND ASTROPHYSICS↗

Accelerating science: The usage of commercial clouds in ATLAS Distributed Computing

The ATLAS experiment at CERN is one of the largest scientific machines built to date and will have ever growing computing needs as the Large Hadron Collider collects an increasingly larger volume of data over the next 20 years. ATLAS is conducting R&D projects on Amazon Web Services and Google Cloud as complementary resources for distributed computing, focusing on some of the key features of commercial clouds: lightweight operation, elasticity and availability of multiple chip architectures. The proof of concept phases have concluded with the cloud-native, vendoragnostic integration with the experiment’s data and workload management frameworks. Google Cloud has been used to evaluate elastic batch computing, ramping up ephemeral clusters of up to O(100k) cores to process tasks requiring quick turnaround. Amazon Web Services has been exploited for the successful physics validation of the Athena simulation software on ARM processors. We have also set up an interactive facility for physics analysis allowing endusers to spin up private, on-demand clusters for parallel computing with up to 4 000 cores, or run GPU enabled notebooks and jobs for machine learning applications. The success of the proof of concept phases has led to the extension of the Google Cloud project, where ATLAS will study the total cost of ownership of a production cloud site during 15 months with 10k cores on average, fully integrated with distributed grid computing resources and continue the R&D projects.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Intuitive Visualizations Through Multi-Domain Projections for Performance Analysis at Scale

Performance analysis of parallel scientific codes is becoming increasingly difficult due to the rapidly growing complexity of applications and architectures. Existing tools fall short in providing intuitive views to reveal the root causes of performance problems. We have developed a new paradigm of projecting and visualizing performance data obtained from one domain onto other domains for faster, more intuitive analysis of applications. We gather performance data in three domains: hardware, application, and communication. For each domain, we define projections that allow the data to be viewed in the other domains. This framework is called the HAC model. The model not only allows us to directly compare the data across domains, but also allows us to use data visualization and analysis tools available in the other domains. Using the methodologies and visualization techniques mentioned above, we have demonstrated the careful unscrambling of otherwise tangled measurements caused by adaptive systems. By attributing performance measurements directly to their causes, we are able to visualize performance measurements in the domains most intuitive to the user, which are not necessarily those in which measurements are collected.

97 MATHEMATICS AND COMPUTING↗

Porting Fragmentation Methods to Graphical Processing Units Using an OpenMP Application Programming Interface: Offloading the Fock Build for Low Angular Momentum Functions

Here, a framework to offload four-index two-electron repulsion integrals to graphical processing units (GPUs) using OpenMP is discussed. The method has been applied to the Fock build for low angular momentum s and p functions in both the restricted Hartree–Fock (RHF) and in the effective fragment molecular orbital (EFMO) framework. Benchmark calculations for the GPU code for the pure RHF method show an increasing speedup relative to the existing OpenMP CPU code in GAMESS from 1.04 to 52× for clusters of 70–569 water molecules. The parallel efficiency on 24 NVIDIA V100 GPU boards also increases when increasing the system size: from 75 to 94% for water clusters that contain 303–1120 molecules. In the EFMO framework, the GPU Fock build shows a high linear scalability up to 4608 V100s with a parallel efficiency of 96% for calculations on a solvated mesoporous silica nanoparticle system with ~67,000 basis functions.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

TABULATED EQUIVALENT SDR FLAMELET (TESF) MODEFL

The code consists of an implementation of a novel tabulated combustion model for non-premixed flames in CFD solvers. This novel technique/model is used to implement an unsteady flamelet tabulation without using progress variables for non-premixed flames. It also has the capability to include history effects which is unique within tabulated flamelet models. The flamelet table generation code can be run in parallel to generate tables with large chemistry mechanisms in relatively short wall clock times. The combustion model/code reads these tables. This framework can be coupled with any CFD solver with RANS as well as LES turbulence models. This framework enables CFD solvers to run large chemistry mechanisms with large number of grids at relatively lower computational costs. Currently it has been coupled with the Converge CFD code and validated against available experimental data. This model can be used to simulate non-premixed combustion in a variety of applications like reciprocating engines, gas turbines and industrial burners operating over a wide range of fuels.

KUNDU, PRITHWISH↗

TABULATED EQUIVALENT SDR FLAMELET (TESF)

The code consists of an implementation of a novel tabulated combustion model for non-premixed flames in CFD solvers. This novel technique/model is used to implement an unsteady flamelet tabulation without using progress variables for non-premixed flames. It also has the capability to include history effects which is unique within tabulated flamelet models. The flamelet table generation code can be run in parallel to generate tables with large chemistry mechanisms in relatively short wall clock times. The combustion model/code reads these tables. This framework can be coupled with any CFD solver with RANS as well as LES turbulence models. This framework enables CFD solvers to run large chemistry mechanisms with large number of grids at relatively lower computational costs. Currently it has been coupled with the Converge CFD code and validated against available experimental data. This model can be used to simulate non-premixed combustion in a variety of applications like reciprocating engines, gas turbines and industrial burners operating over a wide range of fuels.

Kundu, Prithwish↗

Mesoflow: An Open-Source Reacting Flow Solver for Catalysis at Mesoscale

We present the capabilities and software performance metrics of our open-source continuum solver for catalysis, Mesoflow, developed specifically for modeling transport and chemistry at the mesoscale. Our solver utilizes Cartesian block-structured adaptive mesh refinement to resolve complex catalyst surface morphologies directly obtained from X-ray tomography data. An immersed boundary based formulation enables rapid representation of complex geometries prevalent in most mesoporous catalyst interfaces. The solver is developed on top of open-source performance portable library, AMReX, providing parallel execution capabilities on current and upcoming high-performance-computing (HPC) architectures. Our flexible software framework enables integration of complex chemical mechanisms at heterogenous interfaces and time-split algorithms for circumventing highly disparate reaction and flow time-scales. Our current studies indicate a ten-fold performance gain by using graphics-processing-units (GPUs) compared to a single processor for representative problem sizes (2 million cell mesh). We will also present a brief introduction on how to build and use this software for application problems pertaining to catalytic upgrading and gas transport within porous catalyst particles.

adaptive meshing↗

On Design of a Coupling Component for Parallel Multimodeling

A component-based simulation framework is a favorable choice for many multi-scale simulations, which requires dedicated coupling to support flexible data communication/exchange between components, computational parallelism and on-demand dynamic load balancing. In this paper, authors first present the considerations in the course of design and implementation of a coupling component to support integrated parallel multimodeling. Then, authors use an integrated ecological modeling case, as a part of the Across Trophic Level System Simulation (ATLSS) for the ecological restoration projects in the Everglades of South Florida, to demonstrate the practical application of such component in real-world simulations. In the case study, we illustrated the internal data flow between the ecological models and the performance enhancement of the integrated simulation via the coupling component.

99 GENERAL AND MISCELLANEOUS↗

Final Technical Report

Most prokaryotes of interest to DOE are poorly understood. Even when full genomic sequences are available, the function of only a small number of gene products are clear. The critical question is how to best infer the most probable network architectures in cells that are poorly characterized. The project goal is to create a computational hypothesis testing (CHT) framework that combines large-scale dynamical simulation, a database of bioinformatics-derived probable interactions, and numerical parallel architecture data-fitting routines to explore many “what if ?” hypotheses about the functions of genes and proteins within pathways and their downstream effects on molecular concentration profiles and corresponding phenotypes. From this framework we expect to infer signal transduction pathways and gene expression networks in prokaryotes. Detailed mechanistic models of E. Coli have been developed that directly incorporate DNA sequence information. The CHT framework is implemented in the NIEngine network inference software. NIEngine has been applied to recover gene regulatory networks in E. coli to assess performance. Application to Shewanel la oneidensi and other organism of interest DOE will be conducted in partnership with Jim Collin's Lab at Boston University and other academic partners. The CHT framework has also found broad application in the automated learning of biology for purposes of improving human health.

59 BASIC BIOLOGICAL SCIENCES↗

INTEGRATION OF FLEX EQUIPMENT AND OPERATOR ACTIONS IN PLANT FORCE-ON-FORCE MODELS WITH DYNAMIC RISK ASSESSMENT

The overall operation and maintenance cost to protect nuclear power plants accounts for approximately 7% of the total cost of power generation, with labor accounting for half of this cost. In the current research, from interaction with utilities and other stakeholders, it was determined that physical security forces account for nearly 20% of the entire workforce at several nuclear power plants. Labor costs continue to rise in the U.S., so any measures to reduce the cost of operating a nuclear power plant will need to include a reduction in labor. The physical security pathway within the DOE’s Light Water Reactor Sustainability program aims to lower the cost of physical security through directed research into modeling and simulation, application of advanced sensors or deployment of advanced weapons. This report presents a modeling and simulation framework for integrating Diverse and Flexible Mitigation Capability (FLEX) portable equipment performance with Force on Force models of a plant’s physical security posture. The generic framework is described in detail, followed by a case study of modeling an adversarial attack aimed at causing a radiological release by sabotaging the plant’s power supply and its ultimate heat sink capabilities at a hypothetical nuclear power plant. Two different FLEX deployment strategies, series and parallel, are modeled with distinct timelines. The results of the adversarial attack modeled in a commercial Force on Force tool are integrated with the FLEX deployment model in INL’s dynamic modeling tool EMRALD. Monte Carlo simulation is used to model the distribution of the timeline in FLEX deployment strategies. The results demonstrate that, even in the extreme case of a successful adversarial attack, deployment of FLEX equipment can result in a significantly high likelihood of preventing radiological release. The modeling and simulation framework integrating FLEX equipment with Force on Force models enables the nuclear power plants to credit FLEX portable equipment in the plant security posture, resulting in an efficient and optimized physical security.

97 MATHEMATICS AND COMPUTING↗

Scalable Cloning on Large-Scale GPU Platforms with Application to Time-Stepped Simulations on Grids

Cloning is a technique to efficiently simulate a tree of multiple what-if scenarios that are unraveled during the course of a base simulation. However, cloned execution is highly challenging to realize on large, distributed memory computing platforms, due to the dynamic nature of the computational load across clones, and due to the complex dependencies spanning the clone tree. In this paper, we present the conceptual simulation framework, algorithmic foundations, and runtime interface of CloneX, a new system we designed for scalable simulation cloning. It efficiently and dynamically creates whole logical copies of a dynamic tree of simulations across a large parallel system without full physical duplication of computation and memory. The performance of a prototype implementation executed on up to 1,024 graphical processing units of a supercomputing system has been evaluated with three benchmarks—heat diffusion, forest fire, and disease propagation models—delivering a speed up of over two orders of magnitude compared to replicated runs. Finally, the results demonstrate a significantly faster and scalable way to execute many what-if scenario ensembles of large simulations via cloning using the CloneX interface.

97 MATHEMATICS AND COMPUTING↗

A Scaling Study for Incompressible Multispecies Solver in Vertex-CFD

Multispecies incompressible flows occur widely in engineering and environmental applications, such as chemical reactors, fuel cells, ocean mixing, and biomedical systems. However, accurately resolving the complex transport and mixing phenomena associated with multiple interacting species remains computationally challenging, especially for large-scale problems. In this study, we present a robust, high-performance computing--enabled multispecies incompressible Navier–Stokes solver integrated within the Vertex-CFD framework. Our solver employs a fully coupled, implicit, finite element--based formulation that accurately captures the advection, diffusion, and interaction of multiple species in incompressible flows by leveraging the Kokkos library for parallel computing to achieve high computational efficiency. For pressure coupling, the entropically damped artificial compressibility method is utilized. We validated the solver against canonical test cases, including multispecies advection, diffusion, and Bateman systems; the results demonstrate second- and third-order spatial accuracy and consistent convergence. Additionally, we demonstrated the strong and weak scaling study results obtained on the leadership-class high-performance computing system, Frontier at Oak Ridge National Laboratory.

Oz, Furkan [ORNL] (ORCID:0000000265831724)↗

Unexpected Diffusion Anisotropy of Carbon Dioxide in the Metal–Organic Framework Zn 2 (dobpdc)

Metal–organic frameworks are promising materials for energy-efficient gas separations, but little is known about the diffusion of adsorbates in materials featuring one-dimensional porosity at the nanoscale. An understanding of the interplay between framework structure and gas diffusion is crucial for the practical application of these materials as adsorbents or in mixed-matrix membranes, since the rate of gas diffusion within the adsorbent pores impacts the required size (and therefore cost) of the adsorbent column or membrane. In this paper, we investigate the diffusion of CO 2 within the pores of Zn 2 (dobpdc) (dobpdc 4– = 4,4'-dioxidobiphenyl-3,3'-dicarboxylate) using pulsed field gradient (PFG) nuclear magnetic resonance (NMR) spectroscopy and molecular dynamics (MD) simulations. The residual chemical shift anisotropy for pore-confined CO 2 allows PFG NMR measurements of self-diffusion in different crystallographic directions, and our analysis of the entire NMR line shape as a function of the applied field gradient provides a precise determination of the self-diffusion coefficients. In addition to observing CO 2 diffusion through the channels parallel to the crystallographic c axis (self-diffusion coefficient D ∥ = (5.8 ± 0.1) × 10 –9 m 2 s –1 at a pressure of 625 mbar CO 2 ), we unexpectedly find that CO 2 is also able to diffuse between the hexagonal channels in the crystallographic ab plane (D ⊥ = (1.9 ± 0.2) × 10 –10 m 2 s –1 ), despite the walls of these channels appearing impermeable by single-crystal X-ray crystallography and flexible lattice MD simulations. Finally, observation of such unexpected diffusion in the ab plane suggests the presence of defects that enable effective multidimensional CO 2 transport in a metal–organic framework with nominally one-dimensional porosity.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Heterogeneous scalable framework for multiphase flows

Two categories of challenges confront the developer of computational spray models: those related to the computation and those related to the physics. Regarding the computation, the trend towards heterogeneous, multi- and many-core platforms will require considerable re-engineering of codes written for the current supercomputing platforms. Regarding the physics, accurate methods for transferring mass, momentum and energy from the dispersed phase onto the carrier fluid grid have so far eluded modelers. Significant challenges also lie at the intersection between these two categories. To be competitive, any physics model must be expressible in a parallel algorithm that performs well on evolving computer platforms. This work created an application based on a software architecture where the physics and software concerns are separated in a way that adds flexibility to both. The develop spray-tracking package includes an application programming interface (API) that abstracts away the platform-dependent parallelization concerns, enabling the scientific programmer to write serial code that the API resolves into parallel processes and threads of execution. The project also developed the infrastructure required to provide similar API s to other application. The API allow object-oriented Fortran applications direct interaction with Trilinos to support memory management of distributed objects in central processing units (CPU) and graphic processing units (GPU) nodes for applications using C++.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Linear and Nonlinear Solvers for Simulating Multiphase Flow within Large-Scale Engineered Subsurface Systems

Simulation of multiphase flow in the subsurface is well-known to be computationally challenging. While there have been many studies that have explored approaches to overcoming these challenges, they often utilize relatively simple case studies. In this paper, we focus on the unique numerical challenges posed by modeling large-scale engineered subsurface systems, characterized by discrete features embedded in a heterogeneous natural subsurface setting. The man-made features such as shafts, tunnels, and barriers often cause multiple challenges in modeling the domain for multiphase porous media flow. This flow scenario can have a wide range of applications such as nuclear waste repositories, enhanced recovery of a petroleum reservoir, geothermal engineering, and carbon sequestration. An example of these severe numerical challenges is the case of performance assessment (PA) for Waste Isolation Pilot Plant (WIPP), the only operating deep geological repository in the US, which simulates extreme material properties of bedded salt rock formation and extreme contrast due to open excavation next to the formation. The models have extremes not only of permeability and porosity but also of the constitutive models needed for multiphase flow; additionally, they have process models like salt creep closure reducing porosity over time, fracturing in clay and anhydrite interbeds of the bedded salt, gas generation from the waste materials, and unintentional human borehole intrusions in some scenarios. Numerical simulations require the solution of coupled systems of nonlinear PDEs; in our work, we use the open-source simulator PFLOTRAN which is based on Finite Volume discretization. The solution of the nonlinear equations requires use of the Newton-Raphson iteration at each time step, which entails the solution of the linearized Jacobian system at each iteration. The effects of all the processes (i.e., large number of unknowns, highly nonlinear constitutive relations, large contrasts in material properties in short distances) lead to an ill-conditioned Jacobian matrix that severely challenges traditional linear solver, i.e., stabilized biconjugate gradient with block Jacobi incomplete LU preconditioner (BCGS-ILU) leading to non-convergence for traditional Newton-Raphson nonlinear solver causing unacceptably long computation time for each model. This paper presents linear solvers such as constrained pressure residual (CPR) two-stage preconditioner with alternate-block-factorization (ABF) and quasi- implicit pressure and explicit saturation (QIMPES) decouplers and flexible generalized residual solver (FGMRES). The new general-purpose nonlinear solver, Newton trust-region dogleg Cauchy (NTRDC), is also introduced to resolve extreme nonlinearities in the models. We demonstrate the effectiveness of each method relative to the default BCGS-Newton solver. The two best cases had nearly 50 times speed-up and achieved completion of a simulation in 14 hours that never completed due to non-convergence with the default solver. We also investigate the strong scalability of each method and discuss some of the deficiencies found for Block Jacobi preconditioner using parallel domain decomposition, and node packing effects of modern processor architecture.

Preconditioner, Nonlinear, Porous media, Multiphas↗

Task Parallel Incomplete Cholesky Factorization using 2D Partitioned-Block Layout

We introduce a task-parallel algorithm for sparse incomplete Cholesky factorization that utilizes a 2D sparse partitioned-block layout of a matrix. Our factorization algorithm follows the idea of algorithms-by-blocks by using the block layout. The algorithm-byblocks approach induces a task graph for the factorization. These tasks are inter-related to each other through their data dependences in the factorization algorithm. To process the tasks on various manycore architectures in a portable manner, we also present a portable tasking API that incorporates different tasking backends and device-specific features using an open-source framework for manycore platforms i.e., Kokkos. A performance evaluation is presented on both Intel Sandybridge and Xeon Phi platforms for matrices from the University of Florida sparse matrix collection to illustrate merits of the proposed task-based factorization. Experimental results demonstrate that our task-parallel implementation delivers about 26.6x speedup (geometric mean) over single-threaded incomplete Choleskyby- blocks and 19.2x speedup over serial Cholesky performance which does not carry tasking overhead using 56 threads on the Intel Xeon Phi processor for sparse matrices arising from various application problems.

2D layout↗

Compiled MPI: Cost-Effective Exascale Applications Development

The complexity of petascale and exascale machines makes it increasingly difficult to develop applications that can take advantage of them. Future systems are expected to feature billion-way parallelism, complex heterogeneous compute nodes and poor availability of memory (Peter Kogge, 2008). This new challenge for application development is motivating a significant amount of research and development on new programming models and runtime systems designed to simplify large-scale application development. Unfortunately, DoE has significant multi-decadal investment in a large family of mission-critical scientific applications. Scaling these applications to exascale machines will require a significant investment that will dwarf the costs of hardware procurement. A key reason for the difficulty in transitioning today's applications to exascale hardware is their reliance on explicit programming techniques, such as the Message Passing Interface (MPI) programming model to enable parallelism. MPI provides a portable and high performance message-passing system that enables scalable performance on a wide variety of platforms. However, it also forces developers to lock the details of parallelization together with application logic, making it very difficult to adapt the application to significant changes in the underlying system. Further, MPI's explicit interface makes it difficult to separate the application's synchronization and communication structure, reducing the amount of support that can be provided by compiler and run-time tools. This is in contrast to the recent research on more implicit parallel programming models such as Chapel, OpenMP and OpenCL, which promise to provide significantly more flexibility at the cost of reimplementing significant portions of the application. We are developing CoMPI, a novel compiler-driven approach to enable existing MPI applications to scale to exascale systems with minimal modifications that can be made incrementally over the application's lifetime. It includes: (1) New set of source code annotations, inserted either manually or automatically, that will clarify the application's use of MPI to the compiler infrastructure, enabling greater accuracy where needed; (2) A compiler transformation framework that leverages these annotations to transform the original MPI source code to improve its performance and scalability; (3) Novel MPI runtime implementation techniques that will provide a rich set of functionality extensions to be used by applications that have been transformed by our compiler; and (4) A novel compiler analysis that leverages simple user annotations to automatically extract the application's communication structure and synthesize most complex code annotations.

97 MATHEMATICS AND COMPUTING↗

Investigation of the applicability of a functional programming model to fault-tolerant parallel processing for knowledge-based systems

In a fault-tolerant parallel computer, a functional programming model can facilitate distributed checkpointing, error recovery, load balancing, and graceful degradation. Such a model has been implemented on the Draper Fault-Tolerant Parallel Processor (FTPP). When used in conjunction with the FTPP's fault detection and masking capabilities, this implementation results in a graceful degradation of system performance after faults. Three graceful degradation algorithms have been implemented and are presented. A user interface has been implemented which requires minimal cognitive overhead by the application programmer, masking such complexities as the system's redundancy, distributed nature, variable complement of processing resources, load balancing, fault occurrence and recovery. This user interface is described and its use demonstrated. The applicability of the functional programming style to the Activation Framework, a paradigm for intelligent systems, is then briefly described.

Harper, Richard↗