E2EWatch: End-to-end Anomaly Diagnosis Framework for Production HPC Systems.
Abstract not provided.
Engineering topics
Publications and source records attributed to Ates, Emre.
Abstract not provided.
Abstract not provided.
CoMTE is a novel explainability technique that provides counterfactual explanations for supervised machine learning frameworks on multivariate time series data. CoMTE outperforms state-of-the-art explainability methods on several different machine learning frameworks and data sets in comprehensibility and robustness. CoMTE can be used to debug machine learning frameworks and gain a better understanding of the underlying multivariate time series data. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525. SAND2021-1686 O
Abstract not provided.
Abstract not provided.
Abstract not provided.
Abstract not provided.
As the size and complexity of HPC systems grow in line with advancements in hardware and software technology, HPC systems increasingly suffer from performance variation due to shared resource contention as well as software- and hardware-related problems. Such performance variations can lead to failures and inefficiencies, and are among the main challenges in system resiliency. To minimize the impact of performance variation, one must quickly and accurately detect and diagnose the anomalies that cause the variation and take mitigating actions. However, it is difficult to identify anomalies based on the voluminous, high-dimensional, and noisy data collected by system monitoring infrastructures. This paper presents a novel machine learning based framework to automatically diagnose performance anomalies at runtime. Our framework leverages historical resource usage data to extract signatures of previously-observed anomalies. We first convert the collected time series data into easy-to-compute statistical features. We then identify the features that are required to detect anomalies, and extract the signatures of these anomalies. At runtime, we use these signatures to diagnose anomalies with negligible overhead. Here, we evaluate our framework using experiments on a real-world HPC supercomputer and demonstrate that our approach successfully identifies 98% of injected anomalies and consistently outperforms existing anomaly diagnosis techniques.
Abstract not provided.
Abstract not provided.