Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “GPU Graphical Processing Units”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

448 records · Page 25

Algorithmic Patterns for $$\mathcal {H}$$-Matrices on Many-Core Processors

In this work, we consider the reformulation of hierarchical ( \(\mathcal {H}\) ) matrix algorithms for many-core processors with a model implementation on graphics processing units (GPUs). \(\mathcal {H}\) matrices approximate specific dense matrices, e.g., from discretized integral equations or kernel ridge regression, leading to log-linear time complexity in dense matrix–vector products. The parallelization of \(\mathcal {H}\) matrix operations on many-core processors is difficult due to the complex nature of the underlying algorithms. While previous algorithmic advances for many-core hardware focused on accelerating existing \(\mathcal {H}\) matrix CPU implementations by many-core processors, we here aim at totally relying on that processor type. As main contribution, we introduce the necessary parallel algorithmic patterns allowing to map the full \(\mathcal {H}\) matrix construction and the fast matrix–vector product to many-core hardware. In this work, crucial ingredients are space filling curves, parallel tree traversal and batching of linear algebra operations. The resulting model GPU implementation hmglib is the, to the best of the authors knowledge, first entirely GPU-based Open Source \(\mathcal {H}\) matrix library of this kind. We investigate application examples as present in kernel ridge regression, Gaussian Process Regression and kernel-based interpolation. In this context, an in-depth performance analysis and a comparative performance study against a standard multi-core CPU \(\mathcal {H}\) matrix library highlights profound speedups of our many-core parallel approach.

97 MATHEMATICS AND COMPUTING↗

Portable C++ Code that can Look and Feel Like Fortran Code with Yet Another Kernel Launcher (YAKL)

This paper introduces the Yet Another Kernel Launcher (YAKL) C++ portability library, which strives to enable user-level code with the look and feel of Fortran code. The intended audience includes both C++ developers and Fortran developers unfamiliar with C++. The C++ portability approach is briefly explained, YAKL’s main features are described, and code examples are given that demonstrate YAKL’s usage. YAKL fills a niche capability important particularly to scientific applications seeking to port Fortran code quickly to a portable C++ library. YAKL places heavy emphasis on simplicity, readability, and productivity with performance mainly emphasizing Graphics Processing Units (GPUs). Central to YAKL’s ability to allow Fortran-like user-level code are three features: (1) a multi-dimensional Array class that allows Fortran behavior; (2) a limited library of Fortran intrinsic functions; and (3) an efficient pool allocator that transparently enables cheap frequent allocations and deallocations of YAKL Arrays. While YAKL allows Fortran-style code, it also allows Arrays that exhibit C-like behavior as well, including row-major index ordering and lower bounds of “0”. YAKL currently supports CPUs, CPU threading, and Nvidia, AMD, and Intel GPUs.

97 MATHEMATICS AND COMPUTING↗

Lattice Wess-Zumino model with Ginsparg-Wilson fermions: One-loop results and GPU benchmarks

We numerically evaluate the one-loop counterterms for the four-dimensional Wess-Zumino model formulated on the lattice using Ginsparg-Wilson fermions of the overlap (Neuberger) variety, together with an auxiliary fermion (plus superpartners), such that a lattice version of U(1){sub R} symmetry is exactly preserved in the limit of vanishing bare mass. We confirm previous findings by other authors that at one loop there is no renormalization of the superpotential in the lattice theory, but that there is a mismatch in the wave-function renormalization of the auxiliary field. We study the range of the Dirac operator that results when the auxiliary fermion is integrated out, and show that localization does occur, but that it is less pronounced than the exponential localization of the overlap operator. We also present preliminary simulation results for this model, and outline a strategy for nonperturbative improvement of the lattice supercurrent through measurements of supersymmetry Ward identities. Related to this, some benchmarks for our graphics processing unit code are provided. Our simulation results find a nearly vanishing vacuum expectation value for the auxiliary field, consistent with approximate supersymmetry at weak coupling.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

ERF: Energy Research and Forecasting Model

High performance computing (HPC) architectures have undergone rapid development in recent years. As a result, established software suites face an ever increasing challenge to remain performant on and portable across modern systems. Many of the widely adopted atmospheric modeling codes cannot fully (or in some cases, at all) leverage the acceleration provided by General-Purpose Graphics Processing Units, leaving users of those codes constrained to increasingly limited HPC resources. Energy Research and Forecasting (ERF) is a regional atmospheric modeling code that leverages the latest HPC architectures, whether composed of only Central Processing Units (CPUs) or incorporating GPUs. ERF contains many of the standard discretizations and basic features needed to model general atmospheric dynamics. The modular design of ERF provides a flexible platform for exploring different physics parameterizations and numerical strategies. ERF is built on a state-of-the-art, well-supported, software framework (AMReX) that provides a performance portable interface and ensures ERF's long-term sustainability on next generation computing systems. This paper details the numerical methodology of ERF, presents results for a series of verification/validation cases, and documents ERF's performance on current HPC systems. The roughly 5× speed up of ERF (using GPUs) over Weather Research and Forecasting (CPUs only) for a 3D squall line test case highlights the significance of leveraging GPU acceleration.

17 WIND ENERGY↗

Parallel Monte Carlo Methods for Heterogeneous Hardware Computer Systems Using GPUs and Coprocessors: Recent Development of ARCHER Code - Paper 54

Traditional methods of accelerating Monte Carlo (MC) calculations typically use CPUs as the computing unit. The parallelism is based upon multithreading on a multi-core processor, and message passing interface (MPI) across many processors and nodes. In recent years, the hardware accelerators have been introduced by the high performance computing industry as an alternative approach to achieve massive parallelism. Hardware accelerators have the advantage of high Floating-point Operations Per Second (FLOPS) and high energy efficiency. Examples include graphics processing units (GPUs) and coprocessors. Compute-intensive programs can be off-loaded to them to be concurrently executed by hundreds or thousands of active threads. Existing production MC codes, however, cannot be directly compiled or executed on the new platforms. It is therefore necessary to rewrite or redevelop the code using the hardware accelerator specific programming models and APIs. Several studies have been conducted on using GPUs to accelerate MC calculations in reactor analysis and medical physics. In this paper, we present the recent progress at Rensselaer Polytechnic Institute (RPI) in developing a new parallel MC code called ARCHER (Accelerated Radiation-transport Computations in Heterogeneous Environments) for coupled photon-electron transport simulations. The code is designed as a test bed to evaluate the efficacy of heterogeneous computing systems for low-cost MC-based radiation dosimetric calculations. The code has three variants corresponding to three parallel hardware platforms: ARCHER-CPU (multi-core CPU), ARCHER-GPU (Nvidia GPU) and ARCHER-COP (Intel Xeon Phi coprocessor). The code has been used for several applications, including CT imaging dosimetry, radiotherapy dosimetry and radiation shielding design. (authors)

61 RADIATION PROTECTION AND DOSIMETRY↗

HELIOS-K: An ultrafast, open-source opacity calculator for radiative transfer

We present an ultrafast opacity calculator that we name HELIOS-K. It takes a line list as an input, computes the shape of each spectral line, and provides an option for grouping an enormous number of lines into a manageable number of bins. We implement a combination of Algorithm 916 and Gauss–Hermite quadrature to compute the Voigt profile, write the code in CUDA, and optimize the computation for graphics processing units (GPUs). We restate the theory of the k-distribution method and use it to reduce ∼10{sup 5}–10{sup 8} lines to ∼10–10{sup 4} wavenumber bins, which may then be used for radiative transfer, atmospheric retrieval and general circulation models. The choice of line-wing cutoff for the Voigt profile is a significant source of error and affects the value of the computed flux by ∼10%. This is an outstanding physical (rather than computational) problem, due to our incomplete knowledge of pressure broadening of spectral lines in the far line wings. We emphasize that this problem remains regardless of whether one performs line-by-line calculations or uses the k-distribution method and affects all calculations of exoplanetary atmospheres requiring the use of wavelength-dependent opacities. We elucidate the correlated-k approximation and demonstrate that it applies equally to inhomogeneous atmospheres with a single atomic/molecular species or homogeneous atmospheres with multiple species. Using a NVIDIA K20 GPU, HELIOS-K is capable of computing an opacity function with ∼10{sup 5} spectral lines in ∼1 s and is publicly available as part of the Exoclimes Simulation Platform (www.exoclime.org).

79 ASTRONOMY AND ASTROPHYSICS↗

Parallel accelerated vector similarity calculations for genomics applications

The surge in availability of genomic data holds promise for enabling determination of genetic causes of observed individual traits, with applications to problems such as discovery of the genetic roots of phenotypes, be they molecular phenotypes such as gene expression or metabolite concentrations, or complex phenotypes such as diseases. However, the growing sizes of these datasets and the quadratic, cubic or higher scaling characteristics of the relevant algorithms pose a serious computational challenge necessitating use of leadership scale computing. In this study we describe a new approach to performing vector similarity metrics calculations, suitable for parallel systems equipped with graphics processing units (GPUs) or Intel Xeon Phi processors. Our primary focus is the Proportional Similarity metric applied to Genome Wide Association Studies (GWAS) and Phenome Wide Association Studies (PheWAS). We describe the implementation of the algorithms on accelerated processors, methods used for eliminating redundant calculations due to symmetries, and techniques for efficient mapping of the calculations to many-node parallel systems. Finally, results are presented demonstrating high per-node performance and parallel scalability with rates of more than five quadrillion (5 × 10 15 ) elementwise comparisons achieved per second on the ORNL Titan system. In a companion paper we describe corresponding techniques applied to calculations of the Custom Correlation Coefficient for comparative genomics applications.

59 BASIC BIOLOGICAL SCIENCES↗

Data Parallel Bin-Based Indexing for Answering Queries on Multi-Core Architectures

The multi-core trend in CPUs and general purpose graphics processing units (GPUs) offers new opportunities for the database community. The increase of cores at exponential rates is likely to affect virtually every server and client in the coming decade, and presents database management systems with a huge, compelling disruption that will radically change how processing is done. This paper presents a new parallel indexing data structure for answering queries that takes full advantage of the increasing thread-level parallelism emerging in multi-core architectures. In our approach, our Data Parallel Bin-based Index Strategy (DP-BIS) first bins the base data, and then partitions and stores the values in each bin as a separate, bin-based data cluster. In answering a query, the procedures for examining the bin numbers and the bin-based data clusters offer the maximum possible level of concurrency; each record is evaluated by a single thread and all threads are processed simultaneously in parallel. We implement and demonstrate the effectiveness of DP-BIS on two multi-core architectures: a multi-core CPU and a GPU. The concurrency afforded by DP-BIS allows us to fully utilize the thread-level parallelism provided by each architecture--for example, our GPU-based DP-BIS implementation simultaneously evaluates over 12,000 records with an equivalent number of concurrently executing threads. In comparing DP-BIS's performance across these architectures, we show that the GPU-based DP-BIS implementation requires significantly less computation time to answer a query than the CPU-based implementation. We also demonstrate in our analysis that DP-BIS provides better overall performance than the commonly utilized CPU and GPU-based projection index. Finally, due to data encoding, we show that DP-BIS accesses significantly smaller amounts of data than index strategies that operate solely on a column's base data; this smaller data footprint is critical for parallel processors that possess limited memory resources (e.g., GPUs).

97 MATHEMATICS AND COMPUTING↗

HyKKT: a hybrid direct-iterative method for solving KKT linear systems

Here, we propose a solution strategy for the large indefinite linear systems arising in interior methods for nonlinear optimization. The method is suitable for implementation on hardware accelerators such as graphical processing units (GPUs). The current gold standard for sparse indefinite systems is the LBLT factorization where L is a lower triangular matrix and B is 1×1 or 2×2 block diagonal. However, this requires pivoting, which substantially increases communication cost and degrades performance on GPUs. Our approach solves a large indefinite system by solving multiple smaller positive definite systems, using an iterative solver on the Schur complement and an inner direct solve (via Cholesky factorization) within each iteration. Cholesky is stable without pivoting, thereby reducing communication and allowing reuse of the symbolic factorization. We demonstrate the practicality of our approach on large optimal power flow problems and show that it can efficiently utilize GPUs and outperform LBL T factorization of the full system.

97 MATHEMATICS AND COMPUTING↗

Parallel accelerated Custom Correlation Coefficient calculations for genomics applications

The massive quantities of genomic data being made available through gene sequencing techniques are enabling breakthroughs in genomic science in many areas such as medical advances in the diagnosis and treatment of diseases. Analyzing this data, however, is a computational challenge insofar as the computational costs of the relevant algorithms can grow with quadratic, cubic or higher complexity—leading to the need for leadership scale computing. In this paper we describe a new approach to calculations of the Custom Correlation Coefficient (CCC) between Single Nucleotide Polymorphisms (SNPs) across a population, suitable for parallel systems equipped with graphics processing units (GPUs) or Intel Xeon Phi processors. We describe the mapping of the algorithms to accelerated processors, techniques used for eliminating redundant calculations due to symmetries, and strategies for efficient mapping of the calculations to many-node parallel systems. Results are presented demonstrating high per-node performance and near-ideal parallel scalability with rates of more than nine quadrillion (9 × 10 15 ) elementwise comparisons achieved per second with the latest optimized code on the ORNL Titan system, this being orders of magnitude faster than rates achieved using other codes and platforms as reported in the literature. Also it is estimated that as many as 90 quadrillion (90 × 10 15 ) comparisons per second may be achievable on the upcoming ORNL Summit system, an additional 10X performance increase. Finally, in a companion paper we describe corresponding techniques applied to calculations of the Proportional Similarity metric for comparative genomics applications.

97 MATHEMATICS AND COMPUTING↗

Lightfall v0.0.1

Lightfall is a desktop application for synchrotron beamline instrument control, data acquisition, and live analysis at the Advanced Light Source (ALS). Built on Python and Qt, it provides a native graphical interface for operating beamline hardware, configuring and executing experimental scans, and visualizing results in real time. Key features include direct integration with EPICS control systems, a built-in electronic logbook, remote beamline access over secure tunnels, and an interprocess communication (IPC) architecture that coordinates with external analysis applications via ZMQ and EPICS process variables. This IPC approach allows Lightfall to orchestrate specialized analysis tools—including GPU-accelerated streaming correlators—without embedding them, avoiding the dependency conflicts common in monolithic scientific software platforms. Compared to prior approaches such as Xi-CAM's plugin-based architecture, Lightfall's design cleanly separates instrument control from domain-specific analysis, enabling feedback-driven acquisition where live analysis results can adjust scan parameters during an experiment. Its native Qt interface provides responsive performance for real-time data visualization that web-based alternatives struggle to match. Lightfall is designed for use by beamline scientists and staff operating synchrotron instruments at national user facilities.

Pandolfi, Ronald [Lawrence Berkeley National Labor↗

Performance-portable Binary Neutron Star Mergers with AthenaK

We introduce an extension to the AthenaK code for general-relativistic magnetohydrodynamics (GRMHD) in dynamical spacetimes using a 3+1 conservative Eulerian formulation. Like the fixed-spacetime GRMHD solver, we use standard finite-volume methods to evolve the fluid and a constrained-transport scheme to preserve the divergence-free constraint for the magnetic field. We also utilize a first-order flux correction (FOFC) scheme to reduce the need for an artificial atmosphere and optionally enforce a maximum principle to improve robustness. We demonstrate the accuracy of AthenaK using a set of standard tests in flat and curved spacetimes. Using a SANE accretion disk around a Kerr black hole, we compare the new solver to the existing solver for stationary spacetimes using the so-called "HARM-like" formulation. We find that both formulations converge to similar results. We also include the first published binary neutron star (BNS) mergers performed on graphical processing units (GPUs). Thanks to the FOFC scheme, our BNS mergers maintain a relative error of $\mathcal{O}$(10 –11 ) or better in baryon mass conservation up to collapse. Finally, we perform scaling tests of AthenaK on OLCF Frontier, where we show excellent weak scaling of ≥80% efficiency up to 32,768 GPUs and 74% up to 65,536 GPUs for a GRMHD problem in dynamical spacetimes with six levels of mesh refinement. AthenaK achieves an order-of-magnitude speedup using GPUs compared to CPUs, demonstrating that it is suitable for performing numerical relativity problems on modern exascale resources.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Fast DRR generation for 2D to 3D registration on GPUs

Purpose: The generation of digitally reconstructed radiographs (DRRs) is the most time consuming step on the CPU in intensity based two-dimensional x-ray to three-dimensional (CT or 3D rotational x-ray) medical image registration, which has application in several image guided interventions. This work presents optimized DRR rendering on graphical processor units (GPUs) and compares performance achievable on four commercially available devices. Methods: A ray-cast based DRR rendering was implemented for a 512 Multiplication-Sign 512 Multiplication-Sign 72 CT volume. The block size parameter was optimized for four different GPUs for a region of interest (ROI) of 400 Multiplication-Sign 225 pixels with different sampling ratios (1.1%-9.1% and 100%). Performance was statistically evaluated and compared for the four GPUs. The method and the block size dependence were validated on the latest GPU for several parameter settings with a public gold standard dataset (512 Multiplication-Sign 512 Multiplication-Sign 825 CT) for registration purposes. Results: Depending on the GPU, the full ROI is rendered in 2.7-5.2 ms. If sampling ratio of 1.1%-9.1% is applied, execution time is in the range of 0.3-7.3 ms. On all GPUs, the mean of the execution time increased linearly with respect to the number of pixels if sampling was used. Conclusions: The presented results outperform other results from the literature. This indicates that automatic 2D to 3D registration, which typically requires a couple of hundred DRR renderings to converge, can be performed quasi on-line, in less than a second or depending on the application and hardware in less than a couple of seconds. Accordingly, a whole new field of applications is opened for image guided interventions, where the registration is continuously performed to match the real-time x-ray.

07 ISOTOPE AND RADIATION SOURCES↗

Acceleration of the IMplicit–EXplicit nonhydrostatic unified model of the atmosphere on manycore processors

We present the acceleration of an IMplicit-EXplicit (IMEX) nonhydrostatic atmospheric model on manycore processors such as graphic processing units (GPUs) and Intel's Many Integrated Core (MIC) architecture. IMEX time integration methods sidestep the constraint imposed by the Courant-Friedrichs-Lewy condition on explicit methods through corrective implicit solves within each time step. In this work, we implement and evaluate the performance of IMEX on manycore processors relative to explicit methods. Using 3D-IMEX at Courant number C = 15, we obtained a speedup of about 4x relative to an explicit time stepping method run with the maximum allowable C = 1. Moreover, the unconditional stability of IMEX with respect to the fast waves means the speedup can increase significantly with the Courant number as long as the accuracy of the resulting solution is acceptable. We show a speedup of 100x at C = 150 using 1D-IMEX to demonstrate this point. Several improvements on the IMEX procedure were necessary in order to outperform our results with explicit methods: (a) reducing the number of degrees of freedom of the IMEX formulation by forming the Schur complement, (b) formulating a horizontally explicit vertically implicit 1D-IMEX scheme that has a lower workload and better scalability than 3D-IMEX, (c) using high-order polynomial preconditioners to reduce the condition number of the resulting system, and (d) using a direct solver for the 1D-IMEX method by performing and storing LU factorizations once to obtain a constant cost for any Courant number. Without all of these improvements, explicit time integration methods turned out to be difficult to beat. We discuss in detail the IMEX infrastructure required for formulating and implementing efficient methods on manycore processors. Several parametric studies are conducted to demonstrate the gain from each of the abovementioned improvements. Lastly, we validate our results with standard benchmark problems in numerical weather prediction and evaluate the performance and scalability of the IMEX method using up to 4192 GPUs and 16 Knights Landing processors.

97 MATHEMATICS AND COMPUTING↗

A High-Throughput Solver for Marginalized Graph Kernels on GPU

Here, we present the design and optimization of a solver for efficient and high-throughput computation of the marginalized graph kernel on General Purpose GPUs. The graph kernel is computed using the conjugate gradient method to solve a generalized Laplacian of the tensor product between a pair of graphs. To cope with the large gap between the instruction throughput and the memory bandwidth of the GPUs, our solver forms the graph tensor product on-the-fly without storing it in memory. This is achieved by using threads in a warp cooperatively to stream the adjacency and edge label matrices of individual graphs by small square matrix blocks called tiles, which are then staged in registers and the shared memory for later reuse. Warps across a thread block can further share tiles via the shared memory to increase data reuse. We exploit the sparsity of the graphs hierarchically by storing only non-empty tiles using a coordinate format and nonzero elements within each tile using bitmaps. We propose a new partition-based reordering algorithm for aggregating nonzero elements of the graphs into fewer but denser tiles to further exploit sparsity. We carry out extensive theoretical analyses on the graph tensor product primitives for tiles of various density and evaluate their performance on synthetic and real-world datasets. Our solver delivers three to four orders of magnitude speedup over existing CPU-based solvers such as GraKeL and GraphKernels. The capability of the solver enables kernel-based learning tasks at unprecedented scales.

97 MATHEMATICS AND COMPUTING↗

Quantum Monte Carlo Endstation for Petascale Computing

The major achievements enabled by QMC Endstation grant include * Performance improvement on clusters of x86 multi-core systems, especially on Cray XT systems * New and improved methods for the wavefunction optimizations * New forms of trial wavefunctions * Implementation of the full application on NVIDIA GPUs using CUDA The scaling studies of QMCPACK on large-scale systems show excellent parallel efficiency up to 216K cores on Jaguarpf (Cray XT5). The GPU implementation shows speedups of 10-15x over the CPU implementation on older generation of x86. We have implemented hybrid OpenMP/MPI scheme in QMC to take advantage of multi-core shared memory processors of petascale systems. Our hybrid scheme has several advantages over the standard MPI-only scheme. * Memory optimized: large read-only data to store one-body orbitals and other shared properties to represent the trial wave function and many-body Hamiltonian can be shared among threads, which reduces the memory footprint of a large-scale problem. * Cache optimized: the data associated with an active Walker are in cache during the compute-intensive drift-diffusion process and the operations on an Walker are optimized for cache reuse. Thread-local objects are used to ensure the data affinity to a thread. * Load balanced: Walkers in an ensemble are evenly distributed among threads and MPI tasks. The two-level parallelism reduces the population imbalance among MPI tasks and reduces the number of point-to-point communications of large messages (serialized objects) for the Walker exchange. * Communication optimized: the communication overhead, especially for the collective operations necessary to determine ET and measure the properties of an ensemble, is significantly lowered by using less MPI tasks. The multiple forms of parallelism afforded by QMC algorithms make them ideal candidates for acceleration in the many-core paradigm. We presented the results of our effort to port the QMCPACK simulation code to the NVIDIA CUDA GPU platform. We restructured the CPU algorithms to express additional parallelism, minimize GPU-CPU communication, and efficiently utilize the GPU memory hierarchy. Using mixed precision on GT200 GPUs and MPI for intercommunication and load balancing, we observe typical full-application speedups of approximately 10x to 15x relative to quad-core Xeon CPUs alone, while reproducing the double-precision CPU results within statistical error. We developed an all-electron quantum Monte Carlo (QMC) method for solids that does not rely on pseudopotentials, and used it to construct a primary ultra-high-pressure calibration based on the equation of state of cubic boron nitride. We computed the static contribution to the free energy with the QMC method and obtained the phonon contribution from density functional theory, yielding a high-accuracy calibration up to 900 GPa usable directly in experiment. We computed the anharmonic Raman frequency shift with QMC simulations as a function of pressure and temperature, allowing optical pressure calibration. In contrast to present experimental approaches, small systematic errors in the theoretical EOS do not increase with pressure, and no extrapolation is needed. This all-electron method is applicable to first-row solids, providing a new reference for ab initio calculations of solids and benchmarks for pseudopotential accuracy. We compared experimental and theoretical results on the momentum distribution and the quasiparticle renormalization factor in sodium. From an x-ray Compton-profile measurement of the valence-electron momentum density, we derived its discontinuity at the Fermi wavevector finding an accurate measure of the renormalization factor that we compared with quantum-Monte-Carlo and G0W0 calculations performed both on crystalline sodium and on the homogeneous electron gas. Our calculated results are in good agreement with the experiment. We have been studying the heat of formation for various Kubas complexes of molecular hydrogen on Ti(1,2)ethylene-nH2 using Diffusion Monte Carlo. This work has been started and is ongoing. We are studying systems involving 1 and 2 Ti bonding sites with up to 10 hydrogen molecules in numerous configurations. This work will establish a benchmark that will test the accuracy of density functional calculations and establish the feasibility of our methods for similar systems.

97 MATHEMATICS AND COMPUTING↗