Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “scalable computing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 757 records · Page 42

Aggregate data‐driven dynamic modeling of active distribution networks with DERs for voltage stability studies

Abstract Electric distribution networks increasingly host distributed energy resources based on power electronic converter (PEC) toward active distribution networks (ADN). Despite advances in computational capabilities, electromagnetic transient models are limited in scalability because of their reliance on exact data about the distribution system and each of its components. Similarly, the use of the DER_A model, which is intended to examine the combined dynamic behavior of many DERs, is limited by the difficulty in parameterization. There is a need for improved dynamic models of DERs for use in large power system simulations for stability analysis. This paper proposes an aggregate model‐free, data‐driven approach for deriving a dynamic partitioned model (DPM) of ADNs. Detailed residential distribution feeders were first developed, including PEC‐based DERs and composite load models (CMLDs), from which the aggregated DPM was derived. The performance was evaluated through various case studies and validated against the detailed ADN model and state‐of‐the‐art DER_A model with CMLD. The data‐driven DPM achieved a of over 90%, accurately representing the aggregated dynamic behavior of ADNs. Furthermore, the DPM significantly accelerated the simulation process with a computational speedup of 68 times compared to the detailed ADN and a 3.5 times speedup compared to the DER_A CMLD model.

42 ENGINEERING↗

Collective Framework and Performance Optimizations to Open MPI for Cray XT Platforms

The performance and scalability of collective operations plays a key role in the performance and scalability of many scientific applications. Within the Open MPI code base we have developed a general purpose hierarchical collective operations framework called Cheetah, and applied it at large scale on the Oak Ridge Leadership Computing Facility's Jaguar (OLCF) platform, obtaining better performance and scalability than the native MPI implementation. This paper discuss Cheetah's design and implementation, and optimizations to the framework for Cray XT 5 platforms. Our results show that the Cheetah's Broadcast and Barrier perform better than the native MPI implementation. For medium data, the Cheetah's Broadcast outperforms the native MPI implementation by 93% for 49,152 processes problem size. For small and large data, it out performs the native MPI implementation by 10% and 9%, respectively, at 24,576 processes problem size. The Cheetah's Barrier performs 10% better than the native MPI implementation for 12,288 processes problem size.

99 GENERAL AND MISCELLANEOUS↗

ASSESSMENT OF MULTIPHASE CFD WITH ZERO CLOSURE MODEL FOR BOILING WATER REACTOR FUEL ASSEMBLIES

Extension of existing boiling closures to the high void fraction conditions of BWRs is one of challenges for M-CFD applications. This paper presents the assessment of an Eulerian M-CFD model with Zero closure model that has been specifically designed for two-phase flow and heat transfer applications in BWR fuel assemblies. The baseline Zero closure aims at including the simplest and most robust representation of the key physical mechanisms, rather than aiming at absolute accuracy of the predictions. The capabilities of M-CFD with Zero were evaluated against the international OECD/NRC BFBT benchmark data through the comparative analyses of local void fraction, sub-channel void fraction error and exit quality. INL high performance computing system has been employed to demonstrate the scalability of the M-CFD application to nuclear reactor fuel assemblies. The M-CFD simulation results show a good agreement with the experimental data for both quantitative and qualitative results, with the highest uncertainty being observed in the prediction of the local void fraction profile at low void fraction condition. This paper presents a recent view of the best practices for M-CFD of BWR fuel assemblies to provide a useful reference to identify and overcome common challenges in the applications of M-CFD tools. We provide recommendations for geometric modeling, mesh generation and solver configurations to optimize the methodology application.

97 MATHEMATICS AND COMPUTING↗

Design and Implementation of a Scalable Membership Service for Supercomputer Resiliency-Aware Runtime

As HPC systems and applications get bigger and more complex, we are approaching an era in which resiliency and run-time elasticity concerns be- come paramount.We offer a building block for an alternative resiliency approach in which computations will be able to make progress while components fail, in addition to enabling a dynamic set of nodes throughout a computation lifetime. The core of our solution is a hierarchical scalable membership service provid- ing eventual consistency semantics. An attribute replication service is used for hierarchy organization, and is exposed to external applications. Our solution is based on P2P technologies and provides resiliency and elastic runtime support at ultra large scales. Resulting middleware is general purpose while exploiting HPC platform unique features and architecture. We have implemented and tested this system on BlueGene/P with Linux, and using worst-case analysis, evaluated the service scalability as effective for up to 1M nodes.

Tock, Yoav↗

Modeling Distributed Computing Infrastructures for HEP Applications

Predicting the performance of various infrastructure design options in complex federated infrastructures with computing sites distributed over a wide area network that support a plethora of users and workflows, such as the Worldwide LHC Computing Grid (WLCG), is not trivial. Due to the complexity and size of these infrastructures, it is not feasible to deploy experimental test-beds at large scales merely for the purpose of comparing and evaluating alternate designs. An alternative is to study the behaviours of these systems using simulation. This approach has been used successfully in the past to identify efficient and practical infrastructure designs for High Energy Physics (HEP). A prominent example is the Monarc simulation framework, which was used to study the initial structure of the WLCG. New simulation capabilities are needed to simulate large-scale heterogeneous computing systems with complex networks, data access and caching patterns. A modern tool to simulate HEP workloads that execute on distributed computing infrastructures based on the SimGrid and WRENCH simulation frameworks is outlined. Studies of its accuracy and scalability are presented using HEP as a case-study. Hypothetical adjustments to prevailing computing architectures in HEP are studied providing insights into the dynamics of a part of the WLCG and candidates for improvements.

Horzela, Maximilian↗

Joint Genome Institute Analysis Workflow Service (JAWS) v2.7

The U.S. Department of Energy Joint Genome Institute (JGI) has developed the JGI Analysis Workflow Service (JAWS) as a distributed framework to run computational workflows across diverse high-performance computing (HPC) and cloud environments. JAWS enhances the reusability, scalability, and robustness of scientific workflows by orchestrating data movement, code execution, and results retrieval across multiple DOE facilities. At its core, JAWS integrates the Cromwell workflow engine to run workflows expressed in the Workflow Description Language (WDL), ensuring portability and interoperability. To provide consistent runtime environments, JAWS employs container technologies such as Shifter, Apptainer, and Docker. Workflow tasks are managed via HTCondor on HPC backends, while Globus ensures secure and efficient data transfer between sites. JAWS is deployed as a multi-site workflow manager across national laboratory computing facilities, with dedicated instances supporting community projects such as the National Microbiome Data Collaborative (NMDC) and KBase. This distributed, service-oriented architecture enables users to "write once, run anywhere," providing scalable, production-quality workflow execution.

Kirton, Edward↗

A massively parallel time-domain coupled electrodynamics–micromagnetics solver

We present a high-performance coupled electrodynamics–micromagnetics solver for full physical modeling of signals in microelectronic circuitry. The overall strategy couples a finite-difference time-domain approach for Maxwell’s equations to a magnetization model described by the Landau–Lifshitz–Gilbert equation. The algorithm is implemented in the Exascale Computing Project software framework, AMReX, which provides effective scalability on manycore and GPU-based supercomputing architectures. Furthermore, the code leverages ongoing developments of the Exascale Application Code, WarpX, which is primarily being developed for plasma wakefield accelerator modeling. Our temporal coupling scheme provides second-order accuracy in space and time by combining the integration steps for the magnetic field and magnetization into an iterative sub-step that includes a trapezoidal temporal discretization for the magnetization. The performance of the algorithm is demonstrated by the excellent scaling results on NERSC multicore and GPU systems, with a significant (59×) speedup on the GPU using a node-by-node comparison. We demonstrate the utility of our code by performing simulations of an electromagnetic waveguide and a magnetically tunable filter.

97 MATHEMATICS AND COMPUTING↗

A Methodology for the Rapid Development of Scalable HPC Data Services

There is growing evidence in the scientifc computing community that parallel file systems are not sufficient for all HPC storage workloads. This realization has motivated extensive research in new storage system designs. The question of which alternative we should turn towards implies that there could be a single answer satisfying a wide range of very diverse applications. We argue that such a generic solution does not exist. Instead, custom data services should be designed and tailored to the needs of specific applications on specific hardware. Furthermore, custom data services should be designed in close collaboration with users. In this paper, we present a methodology for the rapid development of such data services. This methodology promotes the design of reusable building blocks that can be composed together efficiently through a runtime based on high-performance threading, tasking, and remote procedure calls. We illustrate the success of our methodology by showcasing three examples of data services designed from the same building blocks, yet targeting entirely different applications.

Dorier, Mattieu↗

pcircle - A Suite of Scalable Parallel File System Tools

Most of the software related to file system are written for conventional local file system, they are serialized and can't take advantage of the benefit of a large scale parallel file system. "pcircle" software builds on top of ubiquitous MPI in cluster computing environment and "work-stealing" pattern to provide a scalable, high-performance suite of file system tools. In particular - it implemented parallel data copy and parallel data checksumming, with advanced features such as async progress report, checkpoint and restart, as well as integrity checking.

WANG, FEIYI↗

pcircle - A Suite of Scalable Parallel File System Tools

Most of the software related to file system are written for conventional local file system, they are serialized and can't take advantage of the benefit of a large scale parallel file system. "pcircle" software builds on top of ubiquitous MPI in cluster computing environment and "work-stealing" pattern to provide a scalable, high-performance suite of file system tools. In particular - it implemented parallel data copy and parallel data checksumming, with advanced features such as async progress report, checkpoint and restart, as well as integrity checking.

WANG, FEIYI↗

A Petascale Non-Hydrostatic Atmospheric Dynamical Core in the HOMME Framework

The High-Order Method Modeling Environment (HOMME) is a framework for building scalable, conserva- tive atmospheric models for climate simulation and general atmospheric-modeling applications. Its spatial discretizations are based on Spectral-Element (SE) and Discontinuous Galerkin (DG) methods. These are local methods employing high-order accurate spectral basis-functions that have been shown to perform well on massively parallel supercomputers at any resolution and scale particularly well at high resolutions. HOMME provides the framework upon which the CAM-SE community atmosphere model dynamical-core is constructed. In its current incarnation, CAM-SE employs the hydrostatic primitive-equations (PE) of motion, which limits its resolution to simulations coarser than 0.1 per grid cell. The primary objective of this project is to remove this resolution limitation by providing HOMME with the capabilities needed to build nonhydrostatic models that solve the compressible Euler/Navier-Stokes equations.

58 GEOSCIENCES↗

Coherent electrical control of a single high-spin nucleus in silicon

Nuclear spins are highly coherent quantum objects. In large ensembles, their control and detection via magnetic resonance is widely exploited, for example, in chemistry, medicine, materials science and mining. Nuclear spins also featured in early proposals for solid-state quantum computers and demonstrations of quantum search and factoring algorithms. Scaling up such concepts requires controlling individual nuclei, which can be detected when coupled to an electron. However, the need to address the nuclei via oscillating magnetic fields complicates their integration in multi-spin nanoscale devices, because the field cannot be localized or screened. Control via electric fields would resolve this problem, but previous methods relied on transducing electric signals into magnetic fields via the electron–nuclear hyperfine interaction, which severely affects nuclear coherence. In this work, we demonstrate the coherent quantum control of a single 123 Sb (spin-7/2) nucleus using localized electric fields produced within a silicon nanoelectronic device. The method exploits an idea proposed in 1961 but not previously realized experimentally with a single nucleus. Our results are quantitatively supported by a microscopic theoretical model that reveals how the purely electrical modulation of the nuclear electric quadrupole interaction results in coherent nuclear spin transitions that are uniquely addressable owing to lattice strain. The spin dephasing time, 0.1 seconds, is orders of magnitude longer than those obtained by methods that require a coupled electron spin to achieve electrical driving. These results show that high-spin quadrupolar nuclei could be deployed as chaotic models, strain sensors and hybrid spin-mechanical quantum systems using all-electrical controls. Integrating electrically controllable nuclei with quantum dots could pave the way to scalable, nuclear- and electron-spin-based quantum computers in silicon that operate without the need for oscillating magnetic fields.

42 ENGINEERING↗

Surrogate Constructed Scalable Circuits ADAPT-VQE in the Schwinger model

Inspired by recent advancements of simulating periodic systems on quantum computers, we develop a new approach, (SC)$^2$-ADAPT-VQE, to further advance the simulation of these systems. Our approach extends the scalable circuits ADAPT-VQE framework, which builds an ansatz from a pool of coordinate-invariant operators defined for arbitrarily large, though not arbitrarily small, volumes. Our method uses a classically tractable ``Surrogate Constructed'' method to remove irrelevant operators from the pool, reducing the minimum size for which the scalable circuits are defined. Bringing together the scalable circuits and the surrogate constructed approaches forms the core of the (SC)$^2$ methodology. Our approach allows for a wider set of classical computations, on small volumes, which can be used for a more robust extrapolation protocol. While developed in the context of lattice models, the surrogate construction portion is applicable to a wide variety of problems where information about the relative importance of operators in the pool is available. As an example, we use it to compute properties of the Schwinger model - quantum electrodynamics for a single, massive fermion in $1+1$ dimensions - and show that our method can be used to accurately extrapolate to the continuum limit.

Gustafson, Erik [RIACS, Mtn. View] (ORCID:00000001↗

Scaling Deep Learning on GPU and Knights Landing clusters

The speed of deep neural networks training has become a big bottleneck of deep learning research and development. For example, training GoogleNet by ImageNet dataset on one Nvidia K20 GPU needs 21 days. To speed up the training process, the current deep learning systems heavily rely on the hardware accelerators. However, these accelerators have limited on-chip memory compared with CPUs. To handle large datasets, they need to fetch data from either CPU memory or remote processors. We use both self-hosted Intel Knights Landing (KNL) clusters and multi-GPU clusters as our target platforms. From an algorithm aspect, current distributed machine learning systems are mainly designed for cloud systems. These methods are asynchronous because of the slow network and high fault-tolerance requirement on cloud systems. We focus on Elastic Averaging SGD (EASGD) to design algorithms for HPC clusters. Original EASGD used round-robin method for communication and updating. The communication is ordered by the machine rank ID, which is inefficient on HPC clusters. First, we redesign four efficient algorithms for HPC systems to improve EASGD's poor scaling on clusters. Async EASGD, Async MEASGD, and Hogwild EASGD are faster \textcolor{black}{than} their existing counterparts (Async SGD, Async MSGD, and Hogwild SGD, resp.) in all the comparisons. Finally, we design Sync EASGD, which ties for the best performance among all the methods while being deterministic. In addition to the algorithmic improvements, we use some system-algorithm codesign techniques to scale up the algorithms. By reducing the percentage of communication from 87% to 14%, our Sync EASGD achieves 5.3x speedup over original EASGD on the same platform. We get 91.5% weak scaling efficiency on 4253 KNL cores, which is higher than the state-of-the-art implementation.

60 APPLIED LIFE SCIENCES↗

Open Source Platform for Live Processing of High Data Rate Electron Microscopy (SBIR Phase I Award Topic 15, Subtopic B, Final Scientific / Technical Report)

4D scanning transmission electron microscopy (STEM) is experiencing a revolution both in terms of data size, and data veracity, as are many other areas of experimental data acquisition. This presents enormous opportunities to make new scientific discoveries, but is also hamstrung by aging software that is no longer fit for purpose. Experimental data acquisition platforms must make the same investments in software infrastructure that the simulation community made decades ago to take advantage of high-performance computing (HPC) if they hope to realize the full potential of the latest advances made in detector technology. This requires rethinking the design and deployment of hardware, moving away from small, fixed commodity computing capabilities with closed interfaces to open, modular, scalable systems designed for parallelism and throughput. The technical approach will leverage best-in-class open source technologies, and develop new approaches strategically to build an open source platform for experimental data acquisition targeting high data rate electron microscopy. The project will develop a software platform that uses the power and performance of modern HPC resources to ingest the large volume of data produced by electron microscopes using direct electron detectors. A set of configurable pipelines will be used to enable the processing to be tailored to the problem being investigated with builtin visualization of data at interactive rates. This project will leverage investments made by the DOE and other agencies to provide a powerful software platform for the experimental data community working with most powerful electron microscopes, such as 4D STEM detectors. It will create a platform that serves as a reference for the community. Permissive open source licensing will make the platform available to all, with a simple deployment strategy offering the ability to keep data close to where it is generated. This advanced software platform will require customization for each deployment, creating opportunities for software services that are typically offered to extend open source software platforms.

97 MATHEMATICS AND COMPUTING↗

Petascale Computing Enabling Technologies Project Final Report

The Petascale Computing Enabling Technologies (PCET) project addressed challenges arising from current trends in computer architecture that will lead to large-scale systems with many more nodes, each of which uses multicore chips. These factors will soon lead to systems that have over one million processors. Also, the use of multicore chips will lead to less memory and less memory bandwidth per core. We need fundamentally new algorithmic approaches to cope with these memory constraints and the huge number of processors. Further, correct, efficient code development is difficult even with the number of processors in current systems; more processors will only make it harder. The goal of PCET was to overcome these challenges by developing the computer science and mathematical underpinnings needed to realize the full potential of our future large-scale systems. Our research results will significantly increase the scientific output obtained from LLNL large-scale computing resources by improving application scientist productivity and system utilization. Our successes include scalable mathematical algorithms that adapt to these emerging architecture trends and through code correctness and performance methodologies that automate critical aspects of application development as well as the foundations for application-level fault tolerance techniques. PCET's scope encompassed several research thrusts in computer science and mathematics: code correctness and performance methodologies, scalable mathematics algorithms appropriate for multicore systems, and application-level fault tolerance techniques. Due to funding limitations, we focused primarily on the first three thrusts although our work also lays the foundation for the needed advances in fault tolerance. In the area of scalable mathematics algorithms, our preliminary work established that OpenMP performance of the AMG linear solver benchmark and important individual kernels on Atlas did not match the predictions of our simple initial model. Our investigations demonstrated that a poor default memory allocation mechanism degraded performance. We developed a prototype NUMA library to provide generic mechanisms to overcome these issues, resulting in significantly improved OpenMP performance. After additional testing, we will make this library available to all users, providing a simple means to improve threading on LLNL's production Linux platforms. We also made progress on developing new scalable algorithms that target multicore nodes. We designed and implemented a new AMG interpolation operator with improved convergence properties for very low complexity coarsening schemes. This implementation will also soon be available to LLNL's application teams as part of the hypre library. We presented results for both topics in an invited plenary talk entitled 'Efficient Sparse Linear Solvers for Multi-Core Architectures' at the 2009 HPCMP Institutes Annual Meeting/CREATE Annual All-Hands Meeting. The interpolation work was summarized in a talk entitled 'Improving Interpolation for Aggressive Coarsening' at the 14th Copper Mountain Conference on Multigrid Methods and in a research paper that will appear in Numerical Linear Algebra with Applications. In the area of code correctness, we significantly extended our behavior equivalence class identification mechanism. Specifically, we not only demonstrated it works well at very large scales but we also added the ability to classify MPI tasks not only by function call traces, but also by specific call sites (source code line numbers) being executed by tasks. More importantly, we developed a new technique to determine relative logical execution progress of tasks in the equivalence classes by combining static analysis with our original dynamic approach. We applied this technique to a correctness issue that arose at 4096 tasks during the development of the new AMG interpolation operator discussed above. This scale isat the limit of effectiveness of production tools, but our technique quickly located the erroneous source code, demonstrating the power of understanding relationships between behavioral equivalence classes. This work is the subject of a paper recently accepted to SC09, as well as a presentation entitled 'Providing Order to Extreme Scale Debugging Chaos' given at the ParaDyn Week annual conference in College Park, MD. In addition to this theoretical extension, we have made significant progress in developing a front end for this tool set, and the front-end is now available on several of LLNL's largescale computing resources. In addition, we explored mechanisms to identify exact locations of erroneous MPI usage in application source code. In this work, we developed a new model that led to a highly efficient algorithm for detecting deadlock during dynamic software testing. This work was the subject of a well-received paper at ICS 2009 [4].

99 GENERAL AND MISCELLANEOUS↗

Investigating applications portability with the Uintah DAG-based runtime system on PetaScale supercomputers

Present trends in high performance computing present formidable challenges for applications code using multicore nodes possibly with accelerators and/or co-processors and reduced memory while still attaining scalability. Software frameworks that execute machine-independent applications code using a runtime system that shields users from architectural complexities offer a possible solution. The Uintah framework for example, solves a broad class of large-scale problems on structured adaptive grids using fluid-flow solvers coupled with particle-based solids methods. Uintah executes directed acyclic graphs of computational tasks with a scalable asynchronous and dynamic runtime system for CPU cores and/or accelerators/co-processors on a node. Uintah's clear separation between application and runtime code has led to scalability increases of 1000x without significant changes to application code. This methodology is tested on three leading Top500 machines; OLCF Titan, TACC Stampede and ALCF Mira using three diverse and challenging applications problems. This investigation of scalability with regard to the different processors and communications performance leads to the overall conclusion that the adaptive DAG-based approach provides a very powerful abstraction for solving challenging multi-scale multi-physics engineering problems on some of the largest and most powerful computers available today.

Meng, Qingyu↗

Computing the ankle-brachial index with parallel computational fluid dynamics

The ankle-brachial index (ABI), a ratio of arterial blood pressure in the ankles and upper arms, is used to diagnose and monitor circulatory conditions such as coarctation of the aorta and peripheral artery disease. Computational simulations of the ABI can potentially determine the parameters that produce an ABI indicative of ischemia or other abnormalities in blood flow. However, 0- and 1-D computational methods are limited in describing a 3-D patient-derived geometry. Thus, we present a massively parallel framework for computational fluid dynamics (CFD) simulations in the full arterial system. Additionally, using the lattice Boltzmann method to solve the Navier–Stokes equations, we employ highly parallelized and scalable methods to generate the simulation domain and efficiently distribute the computational load among processors. For the first time, we compute an ABI with 3-D CFD. In this proof-of-concept study, we investigate the dependence of ABI on the presence of stenoses, or narrowed regions of the arteries, by directly modifying the arterial geometry. As a result, our framework enables the computation a hemodynamic factor characterizing flow at the scale of the full arterial system, in a manner that is extensible to patient-specific imaging data and holds potential for treatment planning.

59 BASIC BIOLOGICAL SCIENCES↗