SEARCH · Engineering Papers

Results for “runtime monitoring”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2

All sources NASA NTRS DOE OSTI

DOE OSTI2019

PAPI software-defined events for in-depth performance analysis

The methodology and standardization layer provided by the Performance Application Programming Interface (PAPI) has played a vital role in application profiling for almost two decades. It has enabled sophisticated performance analysis tool designers and performance-conscious scientists to gain insights into their applications by simply instrumenting their code using a handful of PAPI functions that “just work” across different hardware components. In the past, PAPI development had focused primarily on hardware-specific performance metrics. However, the rapidly increasing complexity of software infrastructure poses new measurement and analysis challenges for the developers of large-scale applications. In particular, acquiring information regarding the behavior of libraries and runtimes—used by scientific applications—requires low-level binary instrumentation, or APIs specific to each library and runtime. No uniform API for monitoring events that originate from inside the software stack has emerged. In this article, we present our efforts to extend PAPI’s role so that it becomes the de facto standard for exposing performance-critical events, which we refer to as software-defined events (SDEs), from different software layers. Upgrading PAPI with SDEs enables monitoring of both types of performance events—hardware- and software-related events—in a uniform way, through the same consistent PAPI. The goal of this article is threefold. First, we motivate the need for SDEs and describe our design decisions regarding the functionality we offer through PAPI’s new SDE interface. Second, we illustrate how SDEs can be utilized by different software packages, specifically, by showcasing their use in the numerical linear algebra library MAGMA-Sparse, the tensor algebra library TAMM that is part of the NWChem suite, and the compiler-based performance analysis tool Byfl. Third, we provide a performance analysis of the overhead that results from monitoring SDEs and discuss the trade-offs between overhead and functionality.

Jagode, Heike↗

DOE OSTI2012

Dual compile strategy for parallel heterogeneous execution

The purpose of the Dual Compile Strategy is to increase our trust in the Compute Engine during its execution of instructions. This is accomplished by introducing a heterogeneous Monitor Engine that checks the execution of the Compute Engine. This leads to the production of a second and custom set of instructions designed for monitoring the execution of the Compute Engine at runtime. This use of multiple engines differs from redundancy in that one engine is working on the application while the other engine is monitoring and checking in parallel instead of both applications (and engines) performing the same work at the same time.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2014

Cross-Layer Self-Adaptive/Self-Aware System Software for Exascale Systems

The extreme level of parallelism coupled with the limited available power budget expected in the exascale era brings unprecedented challenges that demand optimization of performance, power and resiliency in unison. Scalability on such systems is of paramount importance, while power and reliability issues may change the execution environment in which a parallel application runs. To solve these challenges exascale systems will require an introspective system software that combines system and application observations across all system stack layers with online feedback and adaptation mechanisms. In this paper we propose the design of a novel self-aware, self-adaptive system software in which a kernel-level Monitor, which continuously inspects the evolution of the target system through observation of Sensors, is combined with a user-level Controller, which reacts to changes in the execution environment, explores opportunities to increase performance, save power and adapts applications to new execution scenarios. We show that the monitoring system accurately monitors the evolution of parallel applications with a runtime overhead below 1-2%. As a test case, we design and implement a user-runtime system that aims at optimizing application’s performance and system power consumption on complex hierarchical architectures. Our results show that our adaptive system reaches 98% of performance efficiency of manually-tuned applications.

High Performance computing, Self-aware/self-adapti↗

DOE OSTI2019

Simulation Planning Using Component Based Cost Model

Successful simulations for scientific discovery on high-performance computing platforms require careful planning, including verification of specific application configuration and runtime parameters, estimation of resource requirements, and steering and monitoring of the simulation. However, simulation planning is an aspect of scientific computing that is extremely sparse in available literature or training. In this paper we focus on the resource management aspect of such planning through formulation of a component-based cost model. We illustrate the methodology and formulation through FLASH, a highly configurable simulation code used in multiple scientific domains.

Dubey, Anshu↗

DOE OSTI2022

Elastic Resource Management for Deep Learning Applications in a Container Cluster

The increasing demand for learning from massive datasets is restructuring our economy. Effective learning, however, involves nontrivial computing resources. Most businesses utilize commercial infrastructure providers (e.g., AWS) to host their computing clusters in the cloud, where various jobs compete for available resources. While cloud resource management is a fruitful research field that has made many advances in production, such as Kubernetes and YARN, few efforts have been invested to further optimize the system performance, especially for deep learning (DL) training jobs in a container cluster. This work introduces FlowCon, a system that is able to monitor the individual evaluation functions of DL jobs at runtime, and thus to make placement decisions on resource allocations elastically. Here, we present a detailed design and implementation of FlowCon and conduct intensive experiments over various DL models. The results demonstrate that FlowCon significantly improves DL job completion time and resource utilization efficiency, compared to default systems. According to the results, FlowCon is able to improve the completion time by up to 68.8% and meanwhile, reduce the makespan by 18.0%, in the presence of various DL job workloads.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2021

Achieving Runtime State Verification Assurance in Critical Cyber-Physical Infrastructures

Industrial Cyber-Physical Systems (ICPS) are an essential backbone of national critical infrastructures. They help monitor and control crucial cyber-enabled services such as energy generation. Commonly ICPS monitors the physical process through Supervisory Control and Data Acquisition (SCADA) systems. The SCADA ecosystem takes critical real-time and future system operational decisions based on the runtime state behavior of field sensors. Traditional SCADA systems use legacy and insecure communication protocols such as the Modbus protocol that lack adequate security mechanisms to provide robust runtime state behavior assurance of constrained field sensors. Therefore, constrained field sensors are commonly vulnerable to standard semantic attacks that gradually change the behavior state of infected devices. This paper discusses process integrity assurance techniques necessary to enhance the security of behavior-based protocols such as the Modbus protocol. The Runtime State Verification (RSV) protocol proposed in this paper aims to address semantic attacks in the SCADA ecosystem by integrating behavior-based Mandatory Results Automata (MRA) and a Hyperledger Fabric (HLF) network. The RSV protocol provides high process integrity assurance through enhanced behavior-based MRA suitable for the constrained field devices. A proof of concept of the RSV protocol has been evaluated in an emulated water-tube boiler. Preliminary evaluations of the RSV protocol aimed to measure the efficiency of the proposed protocol by monitoring an Combustion Efficiency (CE) process necessary to preserve optimal combustion, thus minimizing costs and future maintenance of water-tube boilers. We analyze the overall network overhead and latency of the proposed RSV protocol by evaluating the HLF network performance and comparing the proposed RSV protocol with the state-ofart BloSPAI protocol. Through the preliminary evaluations of the proposed RSV protocol, this paper demonstrates that the proposed RSV protocol overcomes the shortcomings and network overhead of the BloSPAI protocol by integrating behavior-based authentication through novel MRAs and HLF networks.

Rivera, Abel Gomez↗

DOE OSTI2022

Barriers to Broader Utilization of Fault Detection Technologies for Improving Residential HVAC Equipment Efficiency

Faults in residential heating, ventilating, and air conditioning (HVAC) equipment may occur due to poor installation practices or develop over time, and these faults can negatively impact system efficiency, thermal comfort, and equipment lifespan. Automated fault detection and diagnostic (AFDD) technologies identify energy wasting HVAC faults, such as low indoor airflow and improper refrigerant charge, and guide technicians in improving system efficiency. For residential HVAC, AFDD consists of a range of fault detecting and diagnostic capabilities, sensor configurations, and target applications. AFDD technology can either be permanently installed by the original equipment manufacturer (OEM) using embedded sensors or as an add-on product either during or after installation. Additionally, several advanced installation tools and refrigerant gauge sets include AFDD features for temporary use during equipment installation and tune-ups. Some technologies can detect a fault but have limited diagnostic capabilities. For example, a single-point measurement from the home's thermostat or energy monitor can provide certain fault detection capability by analyzing the equipment runtime or energy consumption. These technologies, though limited at determining the cause of a given fault, may have significant energy savings potential due to their low cost and prevalence in the residential HVAC market. Despite the potential benefits, fault detection technologies face many technical and market barriers preventing broad adoption. Beyond the cost barriers due to the added sensor requirements and technology development, fault detection technologies face many implementation and adoption barriers such as installer training, customer awareness, standardized communication protocols, and methods of test for evaluating accuracy. The purpose of this whitepaper is to characterize market and technical barriers impeding broader utilization of fault detection technology for residential HVAC energy efficiency applications.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

DOE OSTI2013

Award ER25750: Coordinated Infrastructure for Fault Tolerance Systems Indiana University Final Report

The main purpose of the Coordinated Infrastructure for Fault Tolerance in Systems initiative has been to conduct research with a goal of providing end-to-end fault tolerance on a systemwide basis for applications and other system software. While fault tolerance has been an integral part of most high-performance computing (HPC) system software developed over the past decade, it has been treated mostly as a collection of isolated stovepipes. Visibility and response to faults has typically been limited to the particular hardware and software subsystems in which they are initially observed. Little fault information is shared across subsystems, allowing little flexibility or control on a system-wide basis, making it practically impossible to provide cohesive end-to-end fault tolerance in support of scientific applications. As an example, consider faults such as communication link failures that can be seen by a network library but are not directly visible to the job scheduler, or consider faults related to node failures that can be detected by system monitoring software but are not inherently visible to the resource manager. If information about such faults could be shared by the network libraries or monitoring software, then other system software, such as a resource manager or job scheduler, could ensure that failed nodes or failed network links were excluded from further job allocations and that further diagnosis could be performed. As a founding member and one of the lead developers of the Open MPI project, our efforts over the course of this project have been focused on making Open MPI more robust to failures by supporting various fault tolerance techniques, and using fault information exchange and coordination between MPI and the HPC system software stack from the application, numeric libraries, and programming language runtime to other common system components such as jobs schedulers, resource managers, and monitoring tools.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2017

Traleika Glacier: A hardware-software co-designed approach to exascale computing

The move from current petascale machines to future exascale machines will need both hardware improvements and software changes. Hardware will need to evolve to focus primarily on features that lower energy consumption: near-threshold voltage operation, fine-grained power and clock management and heterogeneity. Software will also need to evolve and be able to express more parallelism, become more dynamic and adaptable in order to be able to operate on a much more variable hardware. Here, we present Traleika Glacier, an effort that seeks to evaluate radical design changes to meet the constraints, both in terms of power and cost, of exascale computing. The salient features of the hardware design presented in the work include a) a use of heterogeneous cores, b) a redesign of the memory system that centers around hierarchical scratchpads and a global address space, c) the hardware acceleration of certain memory and network operations through specialized engines and, d) very fine-grained control and monitoring capabilities. On the software side, we describe a task-based runtime system, the Open Community Runtime (OCR) which aims to express a wide range of higher-level programming models with a very limited set of core concepts: event-driven tasks for computation, events for synchronization and relocatable data-blocks for data management.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2016

Reducing Waste in Extreme Scale Systems through Introspective Analysis

Resilience is an important challenge for extreme- scale supercomputers. Today, failures in supercomputers are assumed to be uniformly distributed in time. However, recent studies show that failures in high-performance computing systems are partially correlated in time, generating periods of higher failure density. Our study of the failure logs of multiple supercomputers show that periods of higher failure density occur with up to three times more than the average. We design a monitoring system that listens to hardware events and forwards important events to the runtime to detect those regime changes. We implement a runtime capable of receiving notifications and adapt dynamically. In addition, we build an analytical model to predict the gains that such dynamic approach could achieve. We demonstrate that in some systems, our approach can reduce the wasted time by over 30%.

Bautista-Gomez, Leonardo↗

DOE OSTI2012

Simplifying virtual machine introspection using LibVMI

Ensuring the security of a computer system requires the careful integration of many components. Key among these is security monitoring. Recent research trends show an increasing acceptance of external host-based monitoring techniques such as virtual machine introspection (VMI), a technique for viewing the runtime state of a virtual machine (VM). VMI’s primary drawbacks include performance and the semantic gap problem (i.e., understanding the low-level information available through VMI). This report describes work performed under an Early Career Laboratory Directed Research and Development (LDRD) project that aimed to address these two key challenges. Our results are promising, with significant performance improvements and a much more usable VMI programming environment. This work resulted in the creation and release of LibVMI, an open source software project based on the author’s previous work with the XenAccess library.

Payne, Bryan D.↗

DOE OSTI2013

Coordinated Fault Tolerance for High-Performance Computing

Our work to meet our goal of end-to-end fault tolerance has focused on two areas: (1) improving fault tolerance in various software currently available and widely used throughout the HEC domain and (2) using fault information exchange and coordination to achieve holistic, systemwide fault tolerance and understanding how to design and implement interfaces for integrating fault tolerance features for multiple layers of the software stack—from the application, math libraries, and programming language runtime to other common system software such as jobs schedulers, resource managers, and monitoring tools.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2018

Lightweight Detection of Cache Conflicts

In memory hierarchies, caches perform an important role in reducing average memory access latency. Minimizing cache misses can yield significant performance gains. As set-associative caches are widely used in modern architectures, capacity and conflict cache misses co-exist. These two types of cache misses require different optimization strategies. While cache misses are commonly studied using cache simulators, state-of-the-art simulators usually incur hundreds to thousands of times a program’s execution runtime. Moreover, a simulator has difficulty in simulating complex real hardware. To overcome these limitations, measurement methods are proposed to directly monitor program execution on real hardware via performance monitoring units. However, existing measurement-based tools either focus on capacity cache misses or do not distinguish capacity and conflict cache misses. In this paper, we design and implement CCProf, a lightweight measurement-based profiler that identifies conflict cache misses and associates them with program source code and data structures. CCProf incurs moderate runtime overhead that is at least an order of magnitude lower than simulators. With the evaluation on a number of representative programs, CCProf is able to guide optimizations on cache conflict misses and obtain nontrivial speedups.

Roy, Probir↗

DOE OSTI2018

Online Diagnosis of Performance Variation in HPC Systems Using Machine Learning

As the size and complexity of HPC systems grow in line with advancements in hardware and software technology, HPC systems increasingly suffer from performance variation due to shared resource contention as well as software- and hardware-related problems. Such performance variations can lead to failures and inefficiencies, and are among the main challenges in system resiliency. To minimize the impact of performance variation, one must quickly and accurately detect and diagnose the anomalies that cause the variation and take mitigating actions. However, it is difficult to identify anomalies based on the voluminous, high-dimensional, and noisy data collected by system monitoring infrastructures. This paper presents a novel machine learning based framework to automatically diagnose performance anomalies at runtime. Our framework leverages historical resource usage data to extract signatures of previously-observed anomalies. We first convert the collected time series data into easy-to-compute statistical features. We then identify the features that are required to detect anomalies, and extract the signatures of these anomalies. At runtime, we use these signatures to diagnose anomalies with negligible overhead. Here, we evaluate our framework using experiments on a real-world HPC supercomputer and demonstrate that our approach successfully identifies 98% of injected anomalies and consistently outperforms existing anomaly diagnosis techniques.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2025

Benchmark Tracking System for Performance Monitoring

Benchmarking is essential for high-performance software development, particularly for monitoring performance across code iterations. This project focused on enhancing the benchmarking process for Lamellar, an asynchronous runtime for High-Performance Computing (HPC) systems developed at Pacific Northwest National Laboratory. Prior to this work, benchmark results were difficult to track and compare across code versions, presenting significant challenges in identifying performance regressions and long-term trends. The primary objective was to establish a systematic, reproducible approach for measuring performance and detecting regressions following code commits. Our methodology involved three key components: standardizing benchmark outputs, implementing data versioning, and developing analysis tools. We standardized the benchmark output format to JSON Line records containing specific fields (execution time, hardware specifications, and environmental variables). To address data management challenges, we evaluated several options and eventually chose a git repository dedicated to benchmark data. We developed a suite of Python tools that processed benchmark results, enriched them with metadata, and facilitated search in the repository. The resulting system enables more efficient filtering and comparison of performance metrics across commit histories, hardware configurations, and benchmark variants through a unified query interface. Our implementation reduces computational overhead by first checking for existing results through configuration matching before initiating new benchmark runs, thereby conserving resources. The system has been validated by Lamellar developers. It organizes results by benchmark type and build configurations for efficient retrieval. Future developments include a planned Large Language Model interface for predicting benchmark performance, incorporating the criterion package for statistical analysis, which will enable automated detection of statistically significant performance changes, and integration with continuous integration pipelines. Despite these enhancements being reserved for future work, this project has successfully provided the Lamellar development team with a framework for maintaining consistent performance standards and identifying optimization opportunities across workloads and hardware environments.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2019

LibPowerMon: A vendor-neutral system monitoring library

As power becomes an important resource to provision for modern HPC systems it becomes crucial to obtain deeper insights into application's power and thermal characteristics and to correlate application context with important system-level measurements. Existing profiling tools to monitor power and thermal measurements either operate at a granularity that is not fine enough to correlate with application-level events that describe application context or are not equipped to sample application-level events transparently. We introduce LibPowerMon, a lightweight, vendor-neutral user-level profiling framework to simultaneously sample user-specified application events and system-level metrics at up to 1 kHz sampling intervals. At the application level, LibPowerMon provides a source-level phase markup interface to capture application context. At runtime level, LibPowerMon records MPI and OpenMP events and can be extended to include other runtimes. At the system level, LibPowerMon samples power, thermal and relevant metrics, and provides an extensible interface to system's control capabilities. In this release, LibPowerMon supports monitoring and control functionalities on the Intel, IBM Power9 and Nvidia Volta architectures.

MARATHE, ANIRUDDHAP↗

DOE OSTI2020

Greggd

greg(g)d - Global runtime for eBPF-enabled gathering (w/ gumption) daemon Recently the linux kernel has added support for low-level kernel monitoring and profiling through a in-kernel virtual machine. The tooling around these new features (the extended Berkley Packet Filter or eBPF for short) is not mature and is difficult to use. Benefits from eBPF are especially hard to realize while trying to do large scale deployments and integrate with existing metric analysis stacks. A tool was needed to enable loading and collecting data from eBPF programs on large scale HPC systems. Given the problems above it was obvious we needed some wrapper program to compile, load, and collect data from eBPF programs running in the kernel. This tool needed to be lightweight without a heavy set of dependencies, relatively stable between different kernel versions, and integrate nicely with existing widely used metric collection tools. We wrote a program that wraps the eBPF tooling and sends data to our metric gathering tool. eBPF programs are either compiled using the host compiler stack, or loaded in the kernel directly from an object file. These programs are then attached to the system calls that we want to profile. Whenever these system calls are run, the eBPF program collects information of interest and writes that to memory. Our wrapper program polls these memory locations, reads and formats the data, then sends the information to a local unix socket. Our other monitoring tools are configured to read from that socket and send it to the rest of our metric monitoring stack for analysis.

Voss, Joseph [Oak Ridge National Lab. (ORNL), Oak ↗

DOE OSTI2016

Advancing Replicable Solutions for High-Performance Homes in the Southeast - JMC - Patrick Square

The work presented in this report advances the goals of the U.S. Department of Energy Building America program by improving the energy peformance of affordable and market-rate housing. Southface Energy Institute (Southface), part of the U.S. Department of Energy Building America research team Partnership for Home Innovation, worked with owners and builders with various market constraints and ultimate goals for three projects in three climate zones (CZs): Savannah Gardens in Savannah, Georgia (CZ 2); JMC Patrick Square in Clemson, South Carolina (CZ 3); and LaFayette in LaFayette, Georgia (CZ 4). This report documents the design process, computational energy modeling, construction, envelope performance metrics, long-term monitoring results, and successes and failures of the design and execution of these highperformance homes. The JMC Patrick Square project is a single floor with 1,828 ft2 of conditioned living space, three bedrooms, two bathrooms, and an attached two-car garage. This small-scale production builder wanted to increase its level of energy efficiency beyond its current green building practices, including bringing ducts into conditioned space. The team met this goal through a combination of upgrade measures and achieved many Zero Energy Ready Home program requirements. Monitoring the four ducted HPWHs in LaFayette and one in Savannah revealed that HPWH exhaust air impacts attic air during HPWH runtime only, and attic conditions return to previous levels shortly after the HPWH turns off. The HPWH did not appear to impact the loads on the heating and cooling systems, which were also placed in the attic. HPWHs should not be considered dehumidifiers if one is needed in an attic or basement/crawlspace. Ducting the HPWHs did not negatively impact performance compared to other published data of field performance. Changing duct configurations also did not alter the coefficient of performance. HPWHs in efficiency mode (heat pump only) could satisfy hot water demand for most residents. This mode maximizes energy efficiency. Adding .5 in. of insulated sheathing using the Huber ZIP System R Sheathing reduced peak summer temperatures and increased minimum winter temperatures inside the wall assemblies compared to the neighboring home. The neighboring home experienced significantly more risk of condensation and failed the American Society of Heating, Refrigerating and Air-Conditioning Engineers, Inc., Standard 160-2009: Criteria for Moisture-Control Design Analysis in Buildings. Despite the fact that energy modeling predicted only a 2% annual savings from the insulated sheathing, preliminary data indicate that reduced heating, ventilating, and air-conditioning runtimes and energy consumption attributed to this measure provide significantly greater savings. Additional research is necessary.

Array↗