SEARCH · Engineering Papers
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
GOTCHA is a runtime API intercepting function calls between shared libraries. It is intended to be used by HPC Tools (i.e., performance analysis tools like Open/SpeedShop, HPCToolkit, TAU, etc.). 2:18 PMThese other tools can use Gotch to intercept interesting functions, such as MPI functions, and collect performance metrics about those functions. We intend for this to be open-source software that gets adopted by other open-s0urse tools that are used at LLNL.
Terry steam turbines are widely used in various industries because of their robust design. Within the nuclear power generation industry, they are used in the Reactor Core Isolation Cooling System to remove decay heat during reactor isolation events. During the Fukushima Daiichi nuclear power station disaster in Japan in 2011, the Reactor Core Isolation Cooling System and associated Terry turbine operated for over 70 hours in Unit 2; this runtime is well beyond the expected operating duration. Theories suggest the turbine was subjected to a two-phase inlet flow, which could degrade the turbine performance. In this work, an experimental test rig was constructed to test a full-scale Terry model GS-2 steam turbine under two-phase air/water flows. Steady-state efficiency and torque performance maps of the turbine were developed over a range of turbine inlet pressures (1.38–4.83 bar or 20–70 psia), air mass fractions (0.05–1.0) and rotational speeds up to 4000 RPM. Furthermore, turbine performance followed expected trends with torque varying linearly and efficiency varying quadratically with rotational speed. In addition, high-speed images of the two-phase flow entering the turbine were also analyzed to understand how changes in inlet pressure and air mass fraction affect the flow regime and homogenization. The present tests with air–water two-phase mixtures are an important step towards providing an understanding of the full-scale Terry turbine’s behavior and performance curves under two-phase conditions. The results of this work will be combined with air/water and steam/water data gathered using a small-scale Terry ZS-1 steam turbine in order to understand the scaling relationship between large and small size Terry turbines and fluid pairs. The combined data set will enable further development of analytical models over a wide range of conditions and may be used to provide technical justification for expanded use of the Terry turbines in nuclear power plant safety systems and other systems.
Data parallel techniques have been widely adopted both in academia and industry as a tool to enable scalable training of deep learning models. At scale, DL training jobs can fail due to software or hardware bugs, may need to be preempted or terminated due to unexpected events, or may perform suboptimally because they were misconfigured. Under such circumstances, there is a need to recover and/or reconfigure data-parallel DL training jobs on-the-fly, while minimizing the impact on the accuracy of the DNN model and the runtime overhead. In this regard, state-of-art techniques adopted by the HPC community mostly rely on checkpoint-restart, which inevitably leads to loss of progress, thus increasing the runtime overhead. In this paper we explore alternative techniques that exploit the properties of modern deep learning frameworks (overlapping of gradient averaging and weight updates with local gradient computations through pipeline parallelism) to reduce the overhead of resilience/elasticity. To this end we introduce a failure simulation framework and two resilience strategies (immediate mini-batch rollback and lossy forward recovery), which we study compared with checkpoint-restart approaches in a variety of settings in order to understand the trade-offs between the accuracy loss of the DNN model and the runtime overhead.
The methodology and standardization layer provided by the Performance Application Programming Interface (PAPI) has played a vital role in application profiling for almost two decades. It has enabled sophisticated performance analysis tool designers and performance-conscious scientists to gain insights into their applications by simply instrumenting their code using a handful of PAPI functions that “just work” across different hardware components. In the past, PAPI development had focused primarily on hardware-specific performance metrics. However, the rapidly increasing complexity of software infrastructure poses new measurement and analysis challenges for the developers of large-scale applications. In particular, acquiring information regarding the behavior of libraries and runtimes—used by scientific applications—requires low-level binary instrumentation, or APIs specific to each library and runtime. No uniform API for monitoring events that originate from inside the software stack has emerged. In this article, we present our efforts to extend PAPI’s role so that it becomes the de facto standard for exposing performance-critical events, which we refer to as software-defined events (SDEs), from different software layers. Upgrading PAPI with SDEs enables monitoring of both types of performance events—hardware- and software-related events—in a uniform way, through the same consistent PAPI. The goal of this article is threefold. First, we motivate the need for SDEs and describe our design decisions regarding the functionality we offer through PAPI’s new SDE interface. Second, we illustrate how SDEs can be utilized by different software packages, specifically, by showcasing their use in the numerical linear algebra library MAGMA-Sparse, the tensor algebra library TAMM that is part of the NWChem suite, and the compiler-based performance analysis tool Byfl. Third, we provide a performance analysis of the overhead that results from monitoring SDEs and discuss the trade-offs between overhead and functionality.
Applications running on future high performance computing (HPC) systems are more likely to experience transient faults due to technology scaling trends with respect to higher circuit density, smaller transistor size and near-threshold voltage (NTV) operations. A transient fault could corrupt application state without warning, possibly leading to incorrect application output. Such errors are called silent data corruptions (SDCs). In this paper, we present LADR, a low-cost application-level SDC detector for scientific applications. LADR protects scientific applications from SDCs by watching for data anomalies in their state variables (those of scientific interest). It employs compile-time data-flow analysis to minimize the number of monitored variables, thereby reducing runtime and memory overheads while maintaining a high level of fault coverage with low false positive rates. We evaluated LADR with 4 scientific workloads and results show that LADR achieved < 80% fault coverage with only ∼ 3% runtime overheads and ∼ 1% memory overheads. As compared to prior state-of-the-art anomaly-based detection methods, SDC achieved comparable or improved fault coverage, but reduced runtime overheads by 21% ∼ 75%, and memory overheads by 35% ∼ 55% for the evaluated workloads. We believe that such an approach with low memory and runtime overheads coupled with attractive detection precision makes LADR a viable approach for assuring the correct output from large-scale high performance simulations.
This report is a follow on to a previous study performed by Sandia National Labs and Alaska Center for Energy and Power which investigated the use of an energy storage system (ESS) providing spinning reserve within the Cordova Electric Cooperative (CEC) grid. The study provided the savings using the ESS as spinning reserve through reduced fossil fuel consumption and runtime on the diesel generators. In this report, the saving values are used from the previous study to determine the benefit-to-cost ratio for various ratings of ESS performing spinning reserve and quantifying other applications that are applicable to CEC.
Concern is growing in the High-Performance Computing community regarding the reliability of proposed exascale systems. Current research has shown that the expected reliability of these machines will greatly reduce their scalability. In constrast to current fault tolerance methods whose reliability focus is only the application, this project investigates the benefits integrating reliability mechcanisms in the operating system and runtime, as well as the appli- cation. More specifically, this project has three broad contributions in the field: First, using failure logs from current leadership-class high-performance computing systems, we outline the failures common on these large-scale systems. Second, we describe a novel memory pro- tection mechcanism capable of protecting common observed failures that uses the similarity inherrant in many OS and applications state, thereby reducing overheads. Finally, using an analogy with OS jitter, we develop a highly effecient simulator capable predicting the performance of resilience methods at the scales expected for future extreme-scale systems.
The conductor–like polarizable continuum model (C–PCM) with switching/Gaussian smooth discretization is a widely used implicit solvation model in quantum chemistry. We have previously implemented C–PCM solvation for Hartree–Fock (HF) and density functional theory (DFT) on graphical processing units (GPUs), enabling the quantum mechanical treatment of large solvated biomolecules. Here, we first propose a GPU–based algorithm for the PCM conjugate gradient linear solver that greatly improves the performance for very large molecules. The overhead for PCM–related evaluations now consumes less than 15% of the total runtime for DFT calculations on large molecules. Second, we demonstrate that our algorithms tailored for ground state C–PCM are transferable to excited state properties. Using a single GPU, our method evaluates the analytic gradient of the linear response PCM time–dependent density functional theory energy up to 80× faster than a conventional central processing unit (CPU)–based implementation. In addition, our C–PCM algorithms are transferable to other methods that require electrostatic potential (ESP) evaluations. For example, we achieve speed–ups of up to 130× for restricted ESP–based atomic charge evaluations, when compared to CPU–based codes. As a result, we also summarize and compare the different PCM cavity discretization schemes used in some popular quantum chemistry packages as a reference for both users and developers.
The ever increasing amount of data generated by scientific simulations coupled with system I/O constraints are fueling a need for in-situ analysis techniques. Of particular interest are approaches that produce reduced data representations while maintaining the ability to redefine, extract, and study features in a post-process to obtain scientific insights. This paper presents two variants of in-situ feature extraction techniques using segmented merge trees, which encode a wide range of threshold based features. The first approach is a fast, low communication cost technique that generates an exact solution but has limited scalability. The second is a scalable, local approximation that nevertheless is guaranteed to correctly extract all features up to a predefined size. We demonstrate both variants using some of the largest combustion simulations available on leadership class supercomputers. Our approach allows state-of-the-art, feature-based analysis to be performed in-situ at significantly higher frequency than currently possible and with negligible impact on the overall simulation runtime.
Extravasation of circulating tumor cells (CTCs) occurs primarily in the microvasculature, where flow and cell interactions significantly affect the blood rheology. Capturing cell trajectory at this scale requires the coupling of several interaction models, leading to increased computational cost that scales as more cells are added or the domain size is increased. In this work, we focus on micro-scale vessels and study the influence of certain hemorheological factors, including the presence of red blood cell aggregation, hematocrit level, microvessel size, and shear rate, on the trajectory of a circulating tumor cell. We determine which of the aforementioned factors significantly affect CTC motion and identify those which can potentially be disregarded, thus reducing simulation time. We measure the effect of these elements by studying the radial CTC movement and runtime at various combinations of these hemorheological parameters. To accurately capture blood flow dynamics and single cell movement, we perform high-fidelity hemodynamic simulations at a sub-micron resolution using our in-house fluid dynamics solver, HARVEY. We find that increasing hematocrit increases the likelihood of tumor cell margination, which is exacerbated by the presence of red blood cell aggregation. As microvessel diameter increases, there is no major CTC movement towards the wall; however, including aggregation causes the CTC to marginate quicker as the vessel size increases. Finally, as the shear rate is increased, the presence of aggregation has a diminished effect on tumor cell margination.
The simulation of the physical movement of multi-body systems at an atomistic level, with forces calculated from a quantum mechanical description of the electrons, motivates a graph partitioning problem studied in this article. Several advanced algorithms relying on evaluations of matrix polynomials have been published in the literature for such simulations. We aim to use a special type of graph partitioning to efficiently parallelize these computations. For this, we create a graph representing the zero–nonzero structure of a thresholded density matrix, and partition that graph into several components. Each separate submatrix (corresponding to each subgraph) is then substituted into the matrix polynomial, and the result for the full matrix polynomial is reassembled at the end from the individual polynomials. This paper starts by introducing a rigorous definition as well as a mathematical justification of this partitioning problem. We assess the performance of several methods to compute graph partitions with respect to both the quality of the partitioning and their runtime.
The Coarse-Mesh Finite Difference (CMFD) method has been used for over a decade to accelerate the convergence of the Method of Characteristics (MOC) solution to the two- dimensional particle transport equation in CASMO5. Numerical testing, along with widespread use in production-level calculations, have shown that the current CMFD implementation provides stability and robustness for a wide range of realistic reactor physics problems. However, the recent development of linear prolongation has attracted attention from the community as a way to further improve the performance and stability of CMFD. Two interpolation methods for linear prolongation are presented in this work and implemented into a test version of CASMO5. The performance of the proposed interpolations, referred to as the bilinear and linear directional schemes, is evaluated in terms of runtime relative to the default constant or uniform scaling update. Numerical results indicate that the use of linear prolongation can reduce the transport solver runtime on average by approximately 10% when tested with two hundred randomly selected cases. The new directional linear interpolation, combined with default constant boundary updates, is found to provide the highest reduction in runtime for the cases analyzed. (authors)
The ubiquitous in-node heterogeneity of HPC and cloud computing platforms makes software portability and performance optimization extremely challenging. Described here, the MatRIS multilevel math library abstraction framework employs tasking to alleviate these difficulties. MatRIS includes the IRIS task-based runtime on the bottom level and exposes different layers of abstraction to render algorithms architecturally agnostic. MatRIS ensures the decomposition and creation of tasks that represent the necessary encapsulation of the optimized kernels from both vendor and open-source math libraries. Once built, MatRIS can select different combinations of accelerators at runtime, making it portable even on diverse heterogeneous architectures. By leveraging the IRIS runtime’s features for managing heterogeneity, MatRIS deploys algorithms that remove the need to specify orchestration and data transfer. This study describes how the serial task abstraction of a tiled Cholesky factorization is made portable and scalable in the case of multi-device and multi-vendor heterogeneity on a node with NVIDIA and AMD GPUs by using MatRIS. First, we demonstrate that Cholesky in MatRIS provides multi-GPU scalability that offers competitive performance versus cuSolverMG. Then, we present the challenges and opportunities for heterogeneous execution.
HipMCL is a high-performance distributed memory implementation of the popular Markov Cluster Algorithm (MCL) and can cluster large-scale networks within hours using a few thousand CPU-equipped nodes. It relies on sparse matrix computations and heavily makes use of the sparse matrix-sparse matrix multiplication kernel (SpGEMM). The existing parallel algorithms in HipMCL are not scalable to Exascale architectures, both due to their communication costs dominating the runtime at large concurrencies and also due to their inability to take advantage of accelerators that are increasingly popular. In this work, we systematically remove scalability and performance bottlenecks of HipMCL. We enable GPUs by performing the expensive expansion phase of the MCL algorithm on GPU. Additionally, we propose a CPU-GPU joint distributed SpGEMM algorithm called pipelined Sparse SUMMA and integrate a probabilistic memory requirement estimator that is fast and accurate. Furthermore, we develop a new merging algorithm for the incremental processing of partial results produced by the GPUs, which improves the overlap efficiency and the peak memory usage. We also integrate a recent and faster algorithm for performing SpGEMM on CPUs. We validate our new algorithms and optimizations with extensive evaluations. With the enabling of the GPUs and integration of new algorithms, HipMCL is up to 12.4x faster, being able to cluster a network with 70 million proteins and 68 billion connections just under 15 minutes using 1024 nodes of ORNL's Summit supercomputer.
"9IBACOS monitored 37 homes during the late summer and early fall of 2014 in a hot and humid climate to better understand indoor comfort conditions. These homes were constructed in the last several years by four home builders that offered a comfort and performance guarantee for the homes. The homes were located in one of four cities: Tampa, Florida; Orlando, Florida; Houston, Texas; and San Antonio, Texas. Temperature and humidity data were collected from the thermostat and each room of the house using small, battery-powered data loggers. To understand system runtime and its impact on comfort, supply air temperature also was measured on a 1-minute interval. Overall, the group of homes only exceeded a room-to-room temperature difference of 6 degrees F for 5% of the time. For 80% of the time, the rooms in each house were within 4 degrees F of each other. Additionally, the impact of system runtime on comfort is discussed. Finally, measurements made at the thermostat were used to better understand the occupant operation of each cooling system's thermostat setpoint. Builders were questioned on their perceived impact of offering a comfort and performance guarantee. Their feedback, which generally indicates a positive perception, has been summarized in the report.
Hiding communication behind useful computation is an important performance programming technique but remains an inscrutable programming exercise even for the expert. We present Bamboo, a code transformation framework that can realize communication overlap in applications written in MPI without the need to intrusively modify the source code. Bamboo reformulates MPI source into the form of a task dependency graph that expresses a partial ordering among tasks, enabling the program to execute in a data-driven fashion under the control of an external runtime system. Experimental results demonstrate that Bamboo significantly reduces communication delays while requiring only modest amounts of programmer annotation for a variety of applications and platforms, including those employing co-processors and accelerators. Moreover, Bamboo's performance meets or exceeds that of labor-intensive hand coding. The translator is more than a means of hiding communication costs automatically; it demonstrates the utility of semantic level optimization against a wellknown library.
Power is becoming an increasingly scarce resource on the next generation of supercomputers, and should be used wisely to improve overall performance. One strategy for improving power usage is hardware overprovisioning, i.e., systems with more nodes than can be run at full power simultaneously without exceeding the system-wide power limit. With this study, we compare two strategies for allocating power throughout an overprovisioned system – adaptation and prediction – in the context of visualization workloads. While adaptation has been suitable for workloads with more regular execution behaviors, it may not be as suitable on visualization workloads, since they can have variable execution behaviors. This study considers a total of 104 experiments, which vary the rendering workload, power budget, allocation strategy, and node concurrency, including tests processing data sets up to 1 billion cells and using up to 18,432 cores across 512 nodes. Overall, we find that prediction is a superior strategy for this use case, improving performance up to 27% compared to an adaptive strategy.
Physics-of-Failure (PoF) based reliability assessment for photovoltaic (PV) inverters requires long-duration electrical and electrothermal stress histories, yet generating such stress histories with high-fidelity switching models over year long mission profiles is computationally prohibitive. Conventional methods either sacrifice modeling fidelity for speed or require runtimes that are impractical for design iteration and uncertainty studies. To address this bottleneck, this paper presents a High-Performance Computing (HPC) based simulation frame work for faster-than-real-time reliability-oriented simulation. The proposed framework integrates the Average-to-Switching (A2S) method with parallel computing techniques to accelerate switching-level waveform reconstruction. We further introduce optimization strategies, including cluster merging and sensitivity based mission profile screening, to reduce the computational burden. Evaluated using real-world mission profile inputs and a MATLAB/Simulink switching-model reference, the framework reduces the simulation time for a one-year mission from an intractable multi-year duration to approximately 7.3 minutes while maintaining low waveform error. This acceleration provides a practical reliability-oriented simulation engine that can be coupled with component-specific aging models for subsequent PV inverter PoF assessment.