Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Parallel application frameworks”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 631 records · Page 35

DataStager: scalable data staging services for petascale applications

Known challenges for petascale machines are that (1) the costs of I/O for high performance applications can be substantial, especially for output tasks like checkpointing, and (2) noise from I/O actions can inject undesirable delays into the runtimes of such codes on individual compute nodes. This paper introduces the flexible 'DataStager' framework for data staging and alternative services within that jointly address (1) and (2). Data staging services moving output data from compute nodes to staging or I/O nodes prior to storage are used to reduce I/O overheads on applications total processing times, and explicit management of data staging offers reduced perturbation when extracting output data from a petascale machine's compute partition. Experimental evaluations of DataStager on the Cray XT machine at Oak Ridge National Laboratory establish both the necessity of intelligent data staging and the high performance of our approach, using the GTC fusion modeling code and benchmarks running on 1000+ processors.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Bubble Generation in a Flowing Liquid Medium and Resulting Two-Phase Flow in Microgravity

The present investigation reports a study of bubble generation under reduced gravity conditions, using both a co-flow and a cross-flow configuration. This study may be used in the conceptual design of a space-based thermal management system. Ensuing two-phase flow void fraction can be accurately monitored using a single nozzle gas injection system within a continuous liquid flow conduit, as utilized in the present investigation. Accurate monitoring of void fraction leads to precise control of heat and mass transfer coefficients related to a thermal management system; hence providing an efficient and highly effective means of removing heat aboard spacecraft or space stations. Our experiments are performed in parabolic flight aboard the modified DC-9 Reduced Gravity Research Aircraft at NASA Lewis Research Center, using an air-water system. For the purpose of bubble dispersion in a flowing liquid, we use both a co-flow and a cross-flow configuration. In the co-flow geometry, air is introduced through a nozzle in the same direction with the liquid flow. On the other hand, in the cross-flow configuration, air is injected perpendicular to the direction of water flow, via a nozzle protruding inside the two-phase flow conduit. Three different flow conduit (pipe) diameters are used, namely, 1.27 cm, 1.9 cm and 2.54 cm. Two different ratios of nozzle to pipe diameter (D(sub N))sup * are considered, namely (D(sub N))sup * = 0.1 and 0.2, while superficial liquid velocities are varied from 8 to 70 cm/s depending on flow conduit diameter. It is experimentally observed that by holding all other flow conditions and geometry constant, generated bubbles decrease in size with increase in superficial liquid velocity. Detached bubble diameter is shown to increase with air injection nozzle diameter. Likewise, generated bubbles grow in size with increasing pipe diameter. Along the same lines, it is shown that bubble frequency of formation increases and hence the time to detachment of a forming bubble decreases, as the superficial liquid velocity is in-creased. Furthermore, it is shown that the void fraction of the resulting two-phase flow increases with volumetric gas flow rate Q(sub d), pipe diameter and gas injection nozzle diameter, while they decrease with surrounding liquid flow. The important role played by flowing liquid in detaching bubbles in a reduced gravity environment is thus emphasized. We observe that the void fraction can be accurately controlled by using single nozzle gas injection, rather than by employing multiple port injection, since the later system gives rise to unpredictable coalescence of adjacent bubbles. It is of interest to note that empirical bubble size and corresponding void fraction are somewhat smaller for the co-flow geometry than the cross-flow configuration at similar flow conditions with similar pipe and nozzle diameters. In order to supplement the empirical data, a theoretical model is employed to study single bubble generation in the dynamic (Q(sub d) = 1 - 1000 cu cm/s) and bubbly flow regime within the framework of the co-flow configuration. This theoretical model is based on an overall force balance acting on the bubble during the two stages of generation, namely the expansion and the detachment stage. Two sets of forces, one aiding and the other inhibiting bubble detachment are identified. Under conditions of reduced gravity, gas momentum flux enhances, while the surface tension force at the air injection nozzle tip inhibits bubble detachment. In parallel, liquid drag and inertia can act as both attaching and detaching forces, depending on the relative velocity of the bubble with respect to the surrounding liquid. Predictions of the theoretical model compare well with our experimental results. However, at higher superficial liquid velocities, as the bubble loses its spherical form, empirical bubble size no longer matches the theoretical predictions. In summary, we have developed a combined experimental and theoretical work, which describes the complex process of bubble generation and resulting two-phase flow in a microgravity environment. Results of the present study can be used in a wide range of space-based applications, such as thermal energy and power generation, propulsion, cryogenic storage and long duration life support systems, necessary for programs such as NASA's Human Exploration for the Development of Space (HEDS).

Pais, S. C.↗

Complik: A versatile tool for automatizing the representation of material data in MCNP models - 14412

At the Paul Scherrer Institut, the CASMO/SIMULATE code system is used as reference scheme for core analyses of the Swiss reactors. Besides, Monte Carlo (MC) based neutron transport codes are in parallel being established both for out-of-core applications and full core analyses, in order to provide higher fidelity solutions to problems beyond the modelling capabilities of the standard core modelling methods. In this perspective, a necessity is to be able to conduct the MC calculations at any operating conditions, requiring thus proper initialisation of the thermal-hydraulic feedback as well as at any burnup, prompting the need for a detailed modelling of all material zones in the irradiated fuel assemblies. To tackle the later challenge, the development of the COMPLINK tool, aimed at initialising large-scale MC models of burned core configurations, using as basis upstream 2-D assembly/ 3-D core analyses results, was initiated and is presented in this paper. A main feature is the capability to extend in an automatic manner, a user given MC model in order to represent material data with a higher resolution, keeping track of the location of such materials also within nested repeated structures. This capability prevents the user from the burden of developing specific computational framework for a target application. Since the same functionalities are required to analyse out-of-core spent fuel managements systems, a first verification of COMPLINK is illustrated on the basis of a criticality safety evaluation (CSE) with MCNP6.1 of a Swiss PWR wet storage pool loaded with fresh and burned fuel assemblies. Starting from CASMO-5 depletion calculations at each axial layer of the depleted fuel assemblies and reconstructed based on realistic irradiation histories, COMPLINK is then applied to load the isotopic compositions into the material zones of the corresponding MCNP6.1 assembly models. On this basis, the CSE analyses are conducted with focus on investigating axial and radial modelling effects. All in all, the obtained results are found to be well in line with the impact on CSE expected at the considered burnup levels, providing thus confidence in the correct implementation of the COMPLINK functionalities and confirming its effectiveness as well as efficiency for the initialization of all material zones in large-scale MC models of complex and strongly heterogeneous burned configurations. (authors)

42 ENGINEERING↗

A Generalized Framework for Auto-tuning Stencil Computations

This work introduces a generalized framework for automatically tuning stencil computations to achieve superior performance on a broad range of multicore architectures. Stencil (nearest-neighbor) based kernels constitute the core of many important scientific applications involving block-structured grids. Auto-tuning systems search over optimization strategies to find the combination of tunable parameters that maximizes computational efficiency for a given algorithmic kernel. Although the auto-tuning strategy has been successfully applied to libraries, generalized stencil kernels are not amenable to packaging as libraries. Studied kernels in this work include both memory-bound kernels as well as a computation-bound bilateral filtering kernel. We introduce a generalized stencil auto-tuning framework that takes a straightforward Fortran expression of a stencil kernel and automatically generates tuned implementations of the kernel in C or Fortran to achieve performance portability across diverse computer architectures.

97 MATHEMATICS AND COMPUTING↗

Enabling Large-Scale Condensed-Phase Hybrid Density Functional Theory Based Ab Initio Molecular Dynamics. 1. Theory, Algorithm, and Performance

By including a fraction of exact exchange (EXX), hybrid functionals reduce the self-interaction error in semilocal density functional theory (DFT) and thereby furnish a more accurate and reliable description of the underlying electronic structure in systems throughout biology, chemistry, physics, and materials science. However, the high computational cost associated with the evaluation of all required EXX quantities has limited the applicability of hybrid DFT in the treatment of large molecules and complex condensed-phase materials. To overcome this limitation, we describe a linear-scaling approach that utilizes a local representation of the occupied orbitals (e.g., maximally localized Wannier functions (MLWFs)) to exploit the sparsity in the real-space evaluation of the quantum mechanical exchange interaction in finite-gap systems. In this work, we present a detailed description of the theoretical and algorithmic advances required to perform MLWF-based ab initio molecular dynamics (AIMD) simulations of large-scale condensed-phase systems of interest at the hybrid DFT level. We focus our theoretical discussion on the integration of this approach into the framework of Car–Parrinello AIMD, and highlight the central role played by the MLWF-product potential (i.e., the solution of Poisson’s equation for each corresponding MLWF-product density) in the evaluation of the EXX energy and wave function forces. We then provide a comprehensive description of the exx algorithm implemented in the open-source Quantum ESPRESSO program, which employs a hybrid MPI/OpenMP parallelization scheme to efficiently utilize the high-performance computing (HPC) resources available on current- and next-generation supercomputer architectures. Furthermore, this is followed by a critical assessment of the accuracy and parallel performance (e.g., strong and weak scaling) of this approach when AIMD simulations of liquid water are performed in the canonical (NVT) ensemble. With access to HPC resources, we demonstrate that exx enables hybrid DFT-based AIMD simulations of condensed-phase systems containing 500–1000 atoms (e.g., (H₂O)₂₅₆) with a wall time cost that is comparable to that of semilocal DFT. In doing so, exx takes us one step closer to routinely performing AIMD simulations of complex and large-scale condensed-phase systems for sufficiently long time scales at the hybrid DFT level of theory.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

NASA SpaceCube Edge TPU SmallSat Card for Autonomous Operations and Onboard Science-Data Analysis

Using state-of-the-art artificial intelligence (AI)frameworks onboard spacecraft is challenging because common spacecraft processors cannot provide comparable performance to datacenters with server-grade CPUs and GPUs available for terrestrial applications and advanced deep-learning networks. This limitation makes small, lo w-p o we r AI microchip architectures, such as the Google Coral Edge Tensor Processing Unit (TPU), attractive for space missions where the application-specific design enables both high-performance and power-efficient computing for AI applications. To address these challenging considerations for space deployment, this research introduces the design and capabilities of a CubeSat-sized Edge TPU-based co-processor card, known as the SpaceCube Low-power Ed g e Artificial Intelligence Resilient Node (SC-LEARN). This design conforms to NASA’s CubeSat Card Specification (CS2) for integration into next-generation SmallSat and CubeSat systems. This paper describes the overarching architecture and design of the SC-LEARN, as well as, the supporting test card designed for rapid prototyping and evaluation. The SC-LEARN was developed with three operational modes: (1) a high-performance parallel-processing mode,(2)a fault-tolerant mode for onboard resilience, and (3) a power-saving mode with cold spares. Importantly, this research also elaborates on both training and quantization of Tensor Flow models for the SC-LEARN for use onboard with representative, open-source datasets. Lastly, we describe future research plans, including radiation-beam testing and flight demonstration.

Advanced avionics↗

A Generalized Framework for Auto-tuning Stencil Computations

This work introduces a generalized framework for automatically tuning stencil computations to achieve superior performance on a broad range of multicore architectures. Stencil (nearest-neighbor) based kernels constitute the core of many important scientific applications involving block-structured grids. Auto-tuning systems search over optimization strategies to find the combination of tunable parameters that maximizes computational efficiency for a given algorithmic kernel. Although the auto-tuning strategy has been successfully applied to libraries, generalized stencil kernels are not amenable to packaging as libraries. Studied kernels in this work include both memory-bound kernels as well as a computation-bound bilateral filtering kernel. We introduce a generalized stencil auto-tuning framework that takes a straightforward Fortran expression of a stencil kernel and automatically generates tuned implementations of the kernel in C or Fortran to achieve performance portability across diverse computer architectures.

97 MATHEMATICS AND COMPUTING↗

Enabling machine learning-ready HPC ensembles with Merlin

With the growing complexity of computational and experimental facilities, many scientific researchers are turning to machine learning (ML) techniques to analyze large scale ensemble data. With complexities such as multi-component workflows, heterogeneous machine architectures, parallel file systems, and batch scheduling, care must be taken to facilitate this analysis in a high performance computing (HPC) environment. Here, we present Merlin, a workflow framework to enable large ML-friendly ensembles of scientific HPC simulations. By augmenting traditional HPC with distributed compute technologies, Merlin aims to lower the barrier for scientific subject matter experts to incorporate ML into their analysis. As a producer–consumer workflow model, Merlin enables multi-machine, cross-batch job, dynamically allocated yet persistent workflows capable of utilizing surge-compute resources. Key features of Merlin are a flexible HPC-centric interface, low per-task overhead, multi-tiered fault recovery, and a hierarchical sampling algorithm that allows for $\mathscr{O}$(N) task execution and $\mathscr{O}$(N ln N) task queuing to ensembles of millions of tasks. In addition to Merlin’s design, we test the algorithm’s performance in an HPC center and demonstrate the ability to enqueue 40 million simulations in 100 s, with a 30 millisecond per-task overhead that is independent of ensemble size. Finally, we describe some example applications that Merlin has enabled on leadership-class HPC resources, such as the ML-augmented optimization of nuclear fusion experiments and the calibration of infectious disease models to study the progression of and possible mitigation strategies for COVID-19.

97 MATHEMATICS AND COMPUTING↗

Advancement of hybrid fluid-kinetic modeling for HEDP and ICF science

We report on the development progress of a hybrid fluid-kinetic code for simulating fluids and plasmas in a wide range of environments, such as laser–matter interactions, inertial confinement fusion, magnetic confinement fusion, and pulsed power. The suite of numerical tools under development utilizes heterogeneous computer architectures and leverages the benefits of particle–based simulation techniques. By working to combine the kinetic particle-in-cell (PIC) model with a particle-based fluid simulation technique, such as smoothed particle hydrodynamics, we are developing a flexible framework capable of accurately modeling complex flows within and between kinetic and fluid regimes. The TriForce code is under development as a C++ framework for parallel, 3D, particle-based, hybrid fluid-kinetic plasma simulations. The fluid half of TriForce will be based upon the meshless smoothed-particle-hydrodynamics (SPH) approach, well-suited for shear, mixing, and turbulence, whereas the kinetic half resembles a traditional particle-in-cell (PIC) code; other particle-based approaches to fluid modeling that do use a mesh are also possible to use and are under investigation. Maxwell’s electromagnetic field equations are solved either via explicit or implicit algorithms or approximated via resistive magnetohydrodynamics (MHD) using an Ohm’s law and resulting induction equation (extended MHD is under development). A primary goal of enabling direct comparisons, from the same code, between results from the variants of MHD and implicit electromagnetic solutions is to improve our fundamental understanding of systems with magnetic fields. The code is under development to recover results from both radiation-MHD and fully kinetic codes in those limits, and is continuing to be developed from other follow-on grants to operate in between where both descriptions may co-exist and interact. For certain applications, it is desired for a simulation to contain fluid ions and electrons as well as kinetic ions and electrons. Typically, it is too computationally intensive to model a full-scale ICF or HEDP experiment fully kinetically since many cycles are expended with very small time steps on modeling the fluid part of a material that is well treated by the fluid approximation. In this case, many traditional PIC particles can be replaced with a single fluid particle representing the thermal part of the distribution function, and there are fewer needed kinetic particles, which describe the non-thermal part and can be sub-cycled relative to the fluid particle advance. Furthermore, a pure fluid code may, depending on the problem, simply lack many physically important details that are beyond the scope of its reduced approximations and assumptions. In this report, we summarize the objectives achieved in the development of the collisional and kinetic half of the code, and the physics problems to which the code has been applied in the areas of advanced and innovative fusion concepts, pulsed power, and magneto-inertial fusion.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

A Risk Assessment Model for Reduced Aircraft Separation: A Quantitative Method to Evaluate the Safety of Free Flight

As new technologies and procedures are introduced into the National Airspace System, whether they are intended to improve efficiency, capacity, or safety level, the quantification of potential changes in safety levels is of vital concern. Applications of technology can improve safety levels and allow the reduction of separation standards. An excellent example is the Precision Runway Monitor (PRM). By taking advantage of the surveillance and display advances of PRM, airports can run instrument parallel approaches to runways separated by 3400 feet with the same level of safety as parallel approaches to runways separated by 4300 feet using the standard technology. Despite a wealth of information from flight operations and testing programs, there is no readily quantifiable relationship between numerical safety levels and the separation standards that apply to aircraft on final approach. This paper presents a modeling approach to quantify the risk associated with reducing separation on final approach. Reducing aircraft separation, both laterally and longitudinally, has been the goal of several aviation R&D programs over the past several years. Many of these programs have focused on technological solutions to improve navigation accuracy, surveillance accuracy, aircraft situational awareness, controller situational awareness, and other technical and operational factors that are vital to maintaining flight safety. The risk assessment model relates different types of potential aircraft accidents and incidents and their contribution to overall accident risk. The framework links accident risks to a hierarchy of failsafe mechanisms characterized by procedures and interventions. The model will be used to assess the overall level of safety associated with reducing separation standards and the introduction of new technology and procedures, as envisaged under the Free Flight concept. The model framework can be applied to various aircraft scenarios, including parallel and in-trail approaches. This research was performed under contract to NASA and in cooperation with the FAA's Safety Division (ASY).

Cassell, Rick↗

Frog: Asynchronous Graph Processing on GPU with Hybrid Coloring Model

GPUs have been increasingly used to accelerate graph processing for complicated computational problems regarding graph theory. Many parallel graph algorithms adopt the asynchronous computing model to accelerate the iterative convergence. Unfortunately, the consistent asynchronous computing requires locking or atomic operations, leading to significant penalties/overheads when implemented on GPUs. As such, coloring algorithm is adopted to separate the vertices with potential updating conflicts, guaranteeing the consistency/correctness of the parallel processing. Common coloring algorithms, however, may suffer from low parallelism because of a large number of colors generally required for processing a large-scale graph with billions of vertices. We propose a light-weight asynchronous processing framework called Frog with a preprocessing/hybrid coloring model. The fundamental idea is based on Pareto principle (or 80-20 rule) about coloring algorithms as we observed through masses of realworld graph coloring cases. We find that a majority of vertices (about 80%) are colored with only a few colors, such that they can be read and updated in a very high degree of parallelism without violating the sequential consistency. Accordingly, our solution separates the processing of the vertices based on the distribution of colors. In this work, we mainly answer three questions: (1) how to partition the vertices in a sparse graph with maximized parallelism, (2) how to process large-scale graphs that cannot fit into GPU memory, and (3) how to reduce the overhead of data transfers on PCIe while processing each partition. Here, we conduct experiments on real-world data (Amazon, DBLP, YouTube, RoadNet-CA, WikiTalk and Twitter) to evaluate our approach and make comparisons with well-known non-preprocessed (such as Totem, Medusa, MapGraph and Gunrock) and preprocessed (Cusha) approaches, by testing four classical algorithms (BFS, PageRank, SSSP and CC). On all the tested applications and datasets, Frog is able to significantly outperform existing GPU-based graph processing systems except Gunrock and MapGraph. MapGraph gets better performance than Frog when running BFS on RoadNet-CA. The comparison between Gunrock and Frog is inconclusive. Frog can outperform Gunrock more than 1.04X when running PageRank and SSSP, while the advantage of Frog is not obvious when running BFS and CC on some datasets especially for RoadNet-CA.

97 MATHEMATICS AND COMPUTING↗

Project Title: Adaptive Vertical Grid Enhancement for E3SM (Final Technical Report)

During the first phase of the project (2018/05-2021/05), we proved the hypothesis that, without tuning, high vertical resolution improves the representation of unresolved, parameterized low altitude clouds such as stratocumulus clouds in global climate models. We evaluated the effectiveness of the newly developed computational framework, Framework for Improvement by Vertical Enhancement (FIVE) by implementing it into E3SM. FIVE aims to improve representation of the target cloud, low level cloud for this project, with use of a locally high vertical resolution for computation of selected processes. FIVE was assessed via direct comparison among E3SM, E3SM-FIVE, and satellite data as well as a newly developed ensemble artificial neural network methodology. We demonstrated that E3SM-FIVE remarkably improves marine stratocumulus representation. To advance the FIVE methodology, we took a first step toward the adaptive vertical grid in the FIVE method by modifying the E3SM single column model. For future application of FIVE to the global storm resolving model with a few km horizontal grid resolution, we developed the vertical-meridional two-dimensional (2D) Hadley circulation model, which allows one to run simulations with much higher vertical resolution and significantly longer duration than current three-dimensional global storm resolving model. During the second phase of the project (2021/05-2023/11; period for this technical report), we sought a way to appropriately transform FIVE to a regionalized version of FIVE, which allows one to select locations with/without FIVE. In parallel, we demonstrated that E3SM-FIVE with a 25-km mesh significantly reduced the bias associated with coastal stratocumulus. Furthermore, we showed that the same improvement was able to be obtained with the use of both the Regional Refined Model (RRM) targeted to one of the stratocumulus regions with 25-km resolution and FIVE. We explored a new implementation strategy of FIVE with our 2D Hadley circulation model. In parallel we applied the 2D Hadley circulation model to study the double intertropical convergence zone bias with storm resolving model resolution and aerosol-cloud interactions within the Hadley circulation with near large eddy simulation resolution. Last, we have been investigating Arctic mixed-phase stratocumulus clouds with large eddy simulations in order to improve their representation in global models. During the course of the entire project, 6 journal articles were published, and 3 manuscripts are currently in preparation. We gave a total of 30 presentations at various domestic and international conferences. In addition we gave 6 seminar talks at universities and a national laboratory. E3SM- FIVE is available as a branch of E3SM.

54 ENVIRONMENTAL SCIENCES↗

nbodykit: An Open-source, Massively Parallel Toolkit for Large-scale Structure

We present nbodykit, an open-source, massively parallel Python toolkit for analyzing large-scale structure (LSS) data. Using Python bindings of the Message Passing Interface, we provide parallel implementations of many commonly used algorithms in LSS. nbodykit is both an interactive and scalable piece of scientific software, performing well in a supercomputing environment while still taking advantage of the interactive tools provided by the Python ecosystem. Existing functionality includes estimators of the power spectrum, two- and three-point correlation functions, a friends-of-friends grouping algorithm, mock catalog creation via the halo occupation distribution technique, and approximate N-body simulations via the FastPM scheme. The package also provides a set of distributed data containers, insulated from the algorithms themselves, that enables nbodykit to provide a unified treatment of both simulation and observational data sets. nbodykit can be easily deployed in a high-performance computing environment, overcoming some of the traditional difficulties of using Python on supercomputers. We provide performance benchmarks illustrating the scalability of the software. The modular, component-based approach of nbodykit allows researchers to easily build complex applications using its tools. The package is extensively documented at http://nbodykit.readthedocs.io, which also includes an interactive set of example recipes for new users to explore. As open-source software, we hope nbodykit provides a common framework for the community to use and develop in confronting the analysis challenges of future LSS surveys.

79 ASTRONOMY AND ASTROPHYSICS↗

Striped Data Analysis Framework

A columnar data representation is known to be an efficient way for data storage, specifically in cases when the analysis is often done based only on a small fragment of the available data structures. A data representation like Apache Parquet is a step forward from a columnar representation, which splits data horizontally to allow for easy parallelization of data analysis. Based on the general idea of columnar data storage, working on the [LDRD Project], we have developed a striped data representation, which, we believe, is better suited to the needs of High Energy Physics data analysis. A traditional columnar approach allows for efficient data analysis of complex structures. While keeping all the benefits of columnar data representations, the striped mechanism goes further by enabling easy parallelization of computations without requiring special hardware. We will present an implementation and some performance characteristics of such a data representation mechanism using a distributed no-SQL database or a local file system, unified under the same API and data representation model. The representation is efficient and at the same time simple so that it allows for a common data model and APIs for wide range of underlying storage mechanisms such as distributed no-SQL databases and local file systems. Striped storage adopts Numpy arrays as its basic data representation format, which makes it easy and efficient to use in Python applications. The Striped Data Server is a web service, which allows to hide the server implementation details from the end user, easily exposes data to WAN users, and allows to utilize well known and developed data caching solutions to further increase data access efficiency. We are considering the Striped Data Server as the core of an enterprise scale data analysis platform for High Energy Physics and similar areas of data processing. We have been testing this architecture with a 2TB dataset from a CMS dark matter search and plan to expand it to multiple 100 TB or even PB scale. We will present the striped format, Striped Data Server architecture and performance test results.

Gutsche, Oliver↗

Tuning Shinkarev’s Bicycle: Separating the Parallel Cycles of Photosystem II Using Empirical Wavelet Transform

The oxygen-evolving complex (OEC) of Photosystem II (PSII) catalyzes light-driven water oxidation, a process necessary to sustain Earth’s atmospheric oxygen. Oxygen yields measured during single-turnover flash sequences exhibit period-four oscillations, which form the basis of the Joliot–Kok (S-state) model. However, when the oscillations of other processes contribute to the measured oxygen yield, fitting methods can conflate these signals and distort estimates of inefficiencies and initial S-state populations. To address this, we applied the empirical wavelet transform (EWT) as a model-independent method to separate overlapping oscillators and capture damping dynamics that are not well represented in Fourier analysis. We tested this framework on polarographic flash-oxygen traces from both our Synechocystis sp. PCC 6803 thylakoid membrane preparations and archival datasets on Chlorella and isolated chloroplasts. EWT consistently resolves the expected period-four component alongside a distinct binary oscillation. Simulations suggest that fitting this isolated period-four signal recovers VZAD parameters more accurately than analysis of raw traces, yielding different estimates for S-state distributions and transition probabilities. Notably, this binary oscillation aligns closely with semiquinone dynamics predicted solely from period-four fit parameters. These findings indicate that EWT can effectively distinguish complex signals in oxygen evolution, offering a framework potentially applicable to other spectroscopic probes of the S-state cycle.

Ferrari, Nicholas [Louisiana State Univ., Baton Ro↗

Parallel physics-informed neural networks via domain decomposition

Here we develop a distributed framework for the physics-informed neural networks (PINNs) based on two recent extensions, namely conservative PINNs (cPINNs) and extended PINNs (XPINNs), which employ domain decomposition in space and in time-space, respectively. This domain decomposition endows cPINNs and XPINNs with several advantages over the vanilla PINNs, such as parallelization capacity, large representation capacity, efficient hyperparameter tuning, and is particularly effective for multi-scale and multi-physics problems. Here, we present a parallel algorithm for cPINNs and XPINNs constructed with a hybrid programming model described by MPI + X, where X ∈ {CPUs, GPUs}. The main advantage of cPINN and XPINN over the more classical data and model parallel approaches is the flexibility of optimizing all hyperparameters of each neural network separately in each subdomain. We compare the performance of distributed cPINNs and XPINNs for various forward problems, using both weak and strong scalings. Our results indicate that for space domain decomposition, cPINNs are more efficient in terms of communication cost but XPINNs provide greater flexibility as they can also handle time-domain decomposition for any differential equations, and can deal with any arbitrarily shaped complex subdomains. To this end, we also present an application of the parallel XPINN method for solving an inverse diffusion problem with variable conductivity on the United States map, using ten regions as subdomains.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

A High Performance Sparse Tensor Algebra Compiler in MLIR

Sparse tensor algebra is widely used in many applications, including scientific computing, machine learning, and data analytics. The performance of sparse tensor algebra kernels strongly depends on the intrinsic characteristics of the input tensors, hence many storage formats are designed for tensors to achieve optimal performance for particular applications/architectures, which makes it challenging to implement and optimize every tensor operation of interest on a given architecture. We propose a tensor algebra domain-specific language (DSL) and compiler framework to automatically generate kernels for mixed sparse-dense tensor algebra operations. The proposed DSL provides high-level programming abstractions that resemble the familiar Einstein notation to represent tensor algebra operations. The compiler introduces a new Sparse Tensor Algebra dialect built on top of LLVM's extensible MLIR compiler infrastructure for efficient code generation while covering a wide range of tensor storage formats. Our compiler also leverages input-dependent code optimization to enhance data locality for better performance. Our results show that the performance of automatically generated kernels outperforms the state-of-the-art sparse tensor algebra compiler, with up to 20.92x, 6.39x, and 13.9x performance improvement over state-of-the-art tensor algebra compilers, for parallel SpMV, SpMM, and TTM, respectively.

Tian, Ruiqin↗

Accelerating Collective Communication in Data Parallel Training across Deep Learning Frameworks

This work develops new techniques within Horovod, a generic communication library supporting data parallel training across deep learning frameworks. In particular, we improve the Horovod control plane by implementing a new coordination scheme that takes advantage of the characteristics of the typical data parallel training paradigm, namely the repeated execution of collectives on the gradients of a fixed set of tensors. Using a caching strategy, we execute Horovod’s existing coordinator-worker logic only once during a typical training run, replacing it with a more efficient decentralized orchestration strategy using the cached data and a global intersection of a bitvector for the remaining training duration. Next, we introduce a feature for end users to explicitly group collective operations, enabling finer grained control over the communication buffer sizes. To evaluate our proposed strategies, we conduct experiments on a world-class supercomputer — Summit. We compare our proposals to Horovod’s original design and observe 2x performance improvement at a scale of 6000 GPUs; we also compare them against tf.distribute and torch.DDP and achieve 12% better and comparable performance, respectively, using up to 1536 GPUs; we compare our solution against BytePS in typical HPC settings and achieve about 20% better performance on a scale of 768 GPUs. Finally, we test our strategies on a scientific application (STEMDL) using up to 27,600 GPUs (the entire Summit) and show that we achieve a near-linear scaling of 0.93 with a sustained performance of 1.54 exaflops (with standard error +- 0.02) in FP16 precision.

Romero, Joshua↗