Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Proxy Applications”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

ECP Report: Update on Proxy Applications and Vendor Interactions

The ExaLearn miniGAN team (Ellis and Rajamanickam) have released miniGAN, a generative adversarial network(GAN) proxy application, through the ECP proxy application suite. miniGAN is the first machine learning proxy application in the suite (note: the ECP CANDLE project did previously release some benchmarks) and models the performance for training generator and discriminator networks. The GAN's generator and discriminator generate plausible 2D/3D maps and identify fake maps, respectively. miniGAN aims to be a proxy application for related applications in cosmology (CosmoFlow, ExaGAN) and wind energy (ExaWind). miniGAN has been developed so that optimized mathematical kernels (e.g., kernels provided by Kokkos Kernels) can be plugged into to the proxy application to explore potential performance improvements. miniGAN has been released as open source software and is available through the ECP proxy application website (https://proxyapps.exascaleproject.ordecp-proxy-appssuite/) and on GitHub (https://github.com/SandiaMLMiniApps/miniGAN). As part of this release, a generator is provided to generate a data set (series of images) that are inputs to the proxy application.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Proxy Applications for Converged Workloads: DMC LDRD Initiative

Modern scientific applications are complicated and require coordination of several components. Proxy application driven software-hardware co-design plays a vital role in driving innovation among the developments of applications, software infrastructure and hardware architecture. Proxy applications are self-contained and simplified codes that are intended to model the performance-critical computations within applications. Applications executing on modern High Performance Computing (HPC) systems are susceptible to network congestion, insufficient memory bandwidth within and across compute nodes, and inadvertent loss of performance due to bugs and unoptimized programming models. Modern numerical simulations and machine learning models play a critical role in studying physical phenomenon under myriad uncertainties. Such applications often exhibit irregular computation and memory accesses at specific regions of the application code, which can contribute to various performance bottlenecks at scale. To mitigate such issues and prepare the next generation hardware for a variety of computation and data movement contingencies, a well-known practice is to consider "proxy" applications as representative motifs for various classes of scientific applications. While there is disagreement in the HPC community on the mechanisms of construction of the proxy applications, there is a strong consensus on their positive impact in co-design. Proxy Applications for Converged Workloads (PACER) is about facilitating software-hardware co-design through proxy applications with the goal of improving the performance of converged science workflows on heterogeneous systems.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

miniGAN: A Generative Adversarial Network proxy application WBS 2.2.6.08 ECP-2.1.3 (Q3 FY2020 Milestone Report) (V.1.0)

In order to support the machine learning co-design needs of ECP applications in current and future DOE HPC hardware, we have developed a generative adversarial network (GAN) proxy application, miniGAN, that has been released through the ECP proxy application suite. The proxy application is representative of the needs of ExaLearn's target applications, specifically the Cosmoflow and ExaGAN cosmology applications and the ExaWind energy application. The proxy application also demonstrates the first use of performance portable kernels within widely-used machine learning frameworks: PyTorch (Facebook) and Horovod (Uber). We provide performance scaling results for similar workloads to ExaGAN and a profile of individual GAN training components.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

miniGAN: a proxy application for generative adversarial networks

miniGAN is a python-based machine learning proxy application for generative adversarial networks, developed through the Exascale Computing Project's (ECP) ExaLearn project. It will be included in the main ECP proxy application and the machine learning proxy application suite. It is a proxy for ECP cosmological(CosmoFlow, ExaGAN) and wind energy(ExaWind) applications. miniGAN will be distributed to ECP hardware vendors as part of hardware codesign. miniGAN uses the Numpy/PyTorch/TensorFlow/Keras/Horovod frameworks and libraries. It also relies on the Kokkos and Kokkos-Kernels packages developed here at Sandia Labs. SAND2020-2038 M Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy's National Nuclear Security Administration under contract DE-NA0003525.

Ellis, John↗
DOE OSTI2020

Understanding the use of message passing interface in exascale proxy applications

The Exascale Computing Project (ECP) focuses on the development of future exascale-capable applications. Most ECP applications use the message passing interface (MPI) as their parallel programming model with mini-apps serving as proxies. This paper explores the explicit usage of MPI in such ECP proxy applications. We empirically analyze 14 proxy applications from the ECP Proxy Apps Suite. We use the MPI profiling interface (PMPI) to collect MPI usage patterns in ECP proxy apps. Our analysis shows that a small subset of features from MPI is commonly used in the proxies of exascale-capable applications, even when they reference third-party libraries. Overall, this study is intended to provide a better understanding of the use of MPI in current exascale applications. The findings can help focus software investments made for exascale systems in the MPI middleware including optimization, fault-tolerance, tuning, and hardware-offload.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Modeling pre-Exascale AMR Parallel I/O Workloads via Proxy Applications

The present work investigates the modeling of preexascale input/output (I/O) workloads of Adaptive Mesh Refinement (AMR) simulations through a simple proxy application. We collect data from the AMReX Castro framework running on the Summit supercomputer for a wide range of scales and mesh partitions for the hydrodynamic Sedov case as a baseline to provide sufficient coverage to the formulated proxy model. The non-linear analysis data production rates are quantified as a function of a set of input parameters such as output frequency, grid size, number of levels, and the Courant-Friedrichs-Lewy (CFL) condition number for each rank, mesh level and simulation time step. Linear regression is then applied to formulate a simple analytical model which allows to translate AMReX inputs into MACSio proxy I/O application parameters, resulting in a simple “kernel” approximation for data production at each time step. Results show that MACSio can simulate actual AMReX nonlinear “static” I/O workloads to a certain degree of confidence on the Summit supercomputer using the present methodology. The goal is to provide an initial level of understanding of AMR I/O workloads via lightweight proxy applications models to facilitate autotune data management strategies in anticipation of exascale systems.

Godoy, William↗
DOE OSTI2023

Optimizing Performance on Trinity Utilizing Machine Learning, Proxy Applications and Scheduling Priorities

The sheer number of nodes continues to increase in today’s supercomputers, the first half of Trinity alone contains more than 9400 compute nodes. Since the speed of today’s clusters are limited by the slowest nodes, it more important than ever to identify slow nodes, improve their performance if it can be done, and assure minimal usage of slower nodes during performance critical runs. This is an ongoing maintenance task that occurs on a regular basis and, therefore, it is important to minimize the impact upon its users by assessing and addressing slow performing nodes and mitigating their consequences while minimizing down time. These issues can be solved, in large part, through a systematic application of fast running hardware assessment tests, the application of Machine Learning, and making use of performance data to increase efficiency of large clusters. Proxy applications utilizing both MPI and OpenMP were developed to produce data as a substitute for long runtime applications to evaluate node performance. Machine learning is applied to identify underperforming nodes, and policies are being discussed to both minimize the impact of underperforming nodes and increase the efficiency of the system. In this paper, I will describe the process used to produce quickly performing proxy tests, consider various methods to isolate the outliers, and produce ordered lists for use in scheduling to accomplish this task.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Stellar mass as a galaxy cluster mass proxy: application to the Dark Energy Survey redMaPPer clusters

Abstract We introduce a galaxy cluster mass observable, μ⋆, based on the stellar masses of cluster members, and we present results for the Dark Energy Survey (DES) Year 1 (Y1) observations. Stellar masses are computed using a Bayesian model averaging method, and are validated for DES data using simulations and COSMOS data. We show that μ⋆ works as a promising mass proxy by comparing our predictions to X-ray measurements. We measure the X-ray temperature–μ⋆ relation for a total of 129 clusters matched between the wide-field DES Y1 redMaPPer catalogue and Chandra and XMM archival observations, spanning the redshift range 0.1 < $z$ < 0.7. For a scaling relation that is linear in logarithmic space, we find a slope of α = 0.488 ± 0.043 and a scatter in the X-ray temperature at fixed μ⋆ of $\sigma _{{\rm ln} T_\mathrm{ X}|\mu _\star }= 0.266^{+0.019}_{-0.020}$ for the joint sample. By using the halo mass scaling relations of the X-ray temperature from the Weighing the Giants program, we further derive the μ⋆-conditioned scatter in mass, finding $\sigma _{{\rm ln} M|\mu _\star }= 0.26^{+ 0.15}_{- 0.10}$. These results are competitive with well-established cluster mass proxies used for cosmological analyses, showing that μ⋆ can be used as a reliable and physically motivated mass proxy to derive cosmological constraints.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2021

Quantitative Performance Assessment of Proxy Apps and Parents (Report for ECP Proxy App Project Milestone ADCD-504-11)

The ECP Proxy Application Project has an annual milestone to assess the state of ECP proxy applications. Our FY21 milestone (ADCD-504-11) proposed to: "Assess the performance and fidelity of proxy applications, including those in the ECP Proxy App Suite, relative to the ECP Application workload on heterogeneous platforms. Use proxy applications and selected ECP applications to assess the utility of critical elements of the Exascale toolchain, especially tools used to collect performance data. Identify gaps in coverage and/or common situations in which proxies may fail to adequately represent ECP applications." To satisfy this milestone, the following specific tasks were completed: Study ability of MACSio to represent I/O workloads of adaptive mesh codes; Prepare report on assessed performance and debugging tools; Perform cosine similarity study on Intel and P9 and submit paper; Complete LDMS/GPU subcontract, complete and receive report; Complete validation of LDMS data collection for GPUs and submit paper; and Perform cosine similarity study on GPUs.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Quantitative Performance Assessment of Proxy Apps and Parents

The ECP Proxy Application Project has an annual milestone to assess the state of ECP proxy applications. Our FY21 milestone (ADCD-504-11) proposed to: Assess the performance and fidelity of proxy applications, including those in the ECP Proxy App Suite, relative to the ECP Application workload on heterogeneous platforms. Use proxy applications and selected ECP applications to assess the utility of critical elements of the Exascale toolchain, especially tools used to collect performance data. Identify gaps in coverage and/or common situations in which proxies may fail to adequately represent ECP applications.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Quantitative Performance Assessment of Proxy Apps and Parents (Report for ECP Proxy App Project Milestone ADCD-504-28)

The ECP Proxy Application Project has an annual milestone to assess the state of ECP proxy applications and their role in the overall ECP ecosystem. Our FY22 March/April milestone (ADCD- 504-28) proposed to: Assess the fidelity of proxy applications compared to their respective parents in terms of kernel and I/O behavior, and predictability. Similarity techniques will be applied for quantitative comparison of proxy/parent kernel behavior. MACSio evaluation will continue and support for OpenPMD backends will be explored. The execution time predictability of proxy apps with respect to their parents will be explored through a carefully designed scaling study and code comparisons. Note that in this FY, we also have quantitative assessment milestones that are due in September and are, therefore, not included in the description above or in this report. Another report on these deliverables will be generated and submitted upon completion of these milestones. To satisfy this milestone, the following specific tasks were completed: Study the ability of MACSio to represent I/O workloads of adaptive mesh codes. Re-define the performance counter groups for contemporary Intel and IBM platforms to better match specific hardware components and to better align across platforms (make cross-platform comparison more accurate). Perform cosine similarity study based on the new performance counter groups on the Intel and IBM P9 platforms. Perform detailed analysis of performance counter data to accurately average and align the data to maintain phases across all executions and develop methods to reduce the set of collected performance counters used in cosine similarity analysis. Apply a quantitative similarity comparison between proxy and parent CPU kernels. Perform scaling studies to understand the accuracy of predictability of the parent performance using its respective proxy application. This report presents highlights of these efforts.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

FY21 Proxy App Suite Release: Report for ECP Proxy App Project Milestone ADCD504-12

The FY21 Proxy App Suite Release milestone includes the following activities: Curate a collection of proxy applications that represents the breadth of ECP applications, including application domains, programming models, supporting libraries, numerical methods, etc. Identify gaps in coverage and work with application teams to commission or develop proxies to cover gaps. From within this collection, designate the "ECP Proxy Application Suite" of 12-15 proxies that balance breadth of coverage with ease of use and quality of implementation. Also designate approximately 8-10 proxies to form the \ECP Machine Learning Proxy Suite". The ML suite will represent algorithms, use cases, and programming methods typically used by ECP science workloads to incorporate machine learning into their workflows.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

FY22 Proxy App Suite Release

The FY22 Proxy App Suite Release milestone includes the following activities: Curate a collection of proxy applications that represents the breadth of ECP applications, including application domains, programming models, supporting libraries, numerical methods, etc. Identify gaps in coverage and work with application teams to commission or develop proxies to cover gaps. From within this collection, designate the ”ECP Proxy Application Suite” of 10–15 proxies that balance breadth of coverage with ease of use and quality of implementation. Also designate approximately 6–10 proxies to form the “ECP Machine Learning Proxy Suite”. The ML suite will represent algorithms, use cases, and programming methods typically used by ECP science workloads to incorporate machine learning into their workflows.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Smart Proxy Modeling Application of Artificial Intelligence & Machine Learning in Computational Fluid Dynamics

Smart proxy technology leverages the art of artificial intelligence and machine learning in order to build accurate and very fast proxy models for highly complex numerical simulation models. In this project, smart proxy technology is used to replicate the results of a highly complex single phase CFD simulation with reasonable degree of accuracy while reducing the computational cost associated with such CFD simulations. The CFD model under study simulates the combustion of natural gas under various conditions such as varying natural gas composition and flow rate, inlet air flow rate and temperature, and outlet pressure in a High-Pressure Combustion Facility (B6 Combustor) with more than 4 million simulation cells. Only eight CFD simulation runs were used to create a smart proxy model that replicates the detail distribution of Pressure, Temperature, Nitrogen, Oxygen and Carbon-dioxide concentration in the CFD simulation model in seconds with less than 10% error.

03 NATURAL GAS↗
DOE OSTI2026

Data-driven analysis to understand GPU hardware resource usage of optimizations

With heterogeneous systems, the number of GPUs per chip increases to provide computational capabilities for solving science at a nanoscopic scale. However, low utilization for single GPUs defies the need to invest more money in expensive accelerators. Although related work develops optimizations to improve application performance, none studies how these optimizations impact hardware resource usage or average GPU utilization. Here, this paper takes a data-driven analysis approach in addressing this gap by (1) characterizing how hardware resource usage affects device utilization, execution time, or both, (2) presenting a multiobjective metric to identify important application-device interactions that can be optimized to improve device utilization and application performance jointly, (3) studying hardware resource usage behaviors of several optimizations for a benchmark application, and finally (4) identifying optimization opportunities for several scientific proxy applications based on their hardware resource usage behaviors. Furthermore, we demonstrate the applicability of our methodology by applying the identified optimizations to a proxy application, which improves the execution time, device utilization, and power consumption by up to 29.6%, 5.3% and 26.5% respectively.

Computer science↗