Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “runtime monitoring”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
NASA NTRS2022

Integrating FRET with Copilot: Automated Translation of Natural Language Requirements to Runtime Monitors

Runtime verification (RV) enables monitoring systems at runtime, to detect property violations early and limit their potential consequences. To provide the level of assurance required for ultra-critical systems, monitor specifications must faithfully reflect the original mission requirements, which are often written in ambiguous natural language. This paper presents an end-to-end framework to capture requirements in structured natural language and generate monitors that capture their semantics faithfully. We leverage NASA’s Formal Requirement Elicitation Tool (FRET), and the RV system Copilot. We extend FRET with mechanisms to capture additional information needed to generate monitors, and introduce OGMA, a new tool to bridge the gap between FRET and Copilot. With this framework, users can write requirements in an intuitive format and obtain real-time C monitors suitable for use in embedded systems. Our tool chain is available as open source.

FRET↗
NASA NTRS2012

Quantified Event Automata: Towards Expressive and Efficient Runtime Monitors

Runtime verification is the process of checking a property on a trace of events produced by the execution of a computational system. Runtime verification techniques have recently focused on parametric specifications where events take data values as parameters. These techniques exist on a spectrum inhabited by both efficient and expressive techniques. These characteristics are usually shown to be conflicting - in state-of-the-art solutions, efficiency is obtained at the cost of loss of expressiveness and vice-versa. To seek a solution to this conflict we explore a new point on the spectrum by defining an alternative runtime verification approach.We introduce a new formalism for concisely capturing expressive specifications with parameters. Our technique is more expressive than the currently most efficient techniques while at the same time allowing for optimizations.

state machines↗
NASA NTRSDate not supplied

Runtime Monitoring for Unmanned Aerospace Systems with Neural Network Components

AI components (e.g., Deep Neural Networks) are increasingly used in unmanned Aerospace systems for safety-relevant applications. Rigorous Verification and Validation methods for such components are still in their infancy and thus, monitoring of the AI's behavior during runtime is essential. In this paper, we will present a runtime-monitoring architecture, which combines the advanced statistical analysis framework SYSAI (System Analysis using Statistical AI) with temporal and probabilistic runtime monitoring carried out by R2U2 (Realizable, Responsive, and Unobtrusive Unit). Learned statistical models of complex systems with AI components are produced by the SYSAI framework and provide detailed information to enable the R2U2 runtime monitor to efficiently perform advanced safety and performance checks in nominal and off-nominal conditions. We will present initial results of our tool set and architecture on a case study, a DNN-based autonomous centerline tracking system (ACT).

Yuning He↗
NASA NTRSDate not supplied

Runtime Monitoring with R2U2 for Aircraft Systems with Neural Networks

R2U2 (Realizable, Responsive, Unobtrusive Unit) is a hardware-supported tool and framework for real-time system monitoring and software health management of cyber-physical systems. During system operation, R2U2 continuously monitors properties about safety, performance, and security of the vehicle and its vital components and can perform diagnostic reasoning. Efficient observers for past-time and future-time Metric Temporal Logic, fast reasoners for Bayesian Networks, and model-based prognostics algorithms are key components of R2U2 and designed for minimal computational footprint. R2U2 has been implemented in software supporting ROS, NASA's cFS/cFE, and Simulink and as an FPGA configuration. The synergistic combination of monitors and observers in R2U2 makes it possible to design powerful models for system runtime monitoring, diagnostics, software health management, prognostics, and security monitoring. In this presentation, I will give a detailed overview of the R2U2 architecture and its features and will discuss the application of R2U2 for safety-monitoring of a neural-network based autonomous centerline tracking system (ACT) for autonomous aircraft.

Runtime Monitoring↗
NASA NTRSDate not supplied

Seeing is Believing: Monitoring Future Time Temporal Logic

Runtime monitors for future-time unbounded temporal logics like RVLTL, LTL 3 and FLTL, have double-exponential (2^2^n) worst-case space complexity bounds in size of the input formula. The semantics of these logics require monitors to perform general satisfiability solving for LTL expressions, a well-studied problem whose computational complexity is NP-hard and PSPACE-complete. This paper introduces an unbounded future-time linear temporal logic defined over a lattice. We call our logic an incremental temporal logic as it can be viewed as incrementally constructing proofs about the trace. On this account, we view online runtime monitoring as a decision procedure for proofs systems about incrementally growing traces. We demonstrate that our incremental temporal logic allows monitor construction to void satisfiability solving while still soundly detecting when the property is violated in an online fashion. This enables asymptotic improvements in space complexity. As proof, we provide a procedure to construct monitors that utilize linear space and time in the size of the input formula, while remaining constant in the size of the input stream and suitable for online monitoring. We further demonstrate, through several examples, that our incremental temporal logic is straightforward to adopt and practical for runtime verification.

temporal logic↗
NASA NTRS2018

Refining the Safety - Liveness Classification of Temporal Properties According to Monitorability

Runtime verification is the topic of analyzing execution traces using formal techniques. It includes monitoring the execution of a system against temporal properties, commonly to detect violations. Not every temporal property is fully monitorable however: in some cases, the correctness of the execution does not depend on any finite prefix. We study the connection between monitorability and Lamport’s classification of properties to safety and liveness and their dual classes.We refine the definition of monitorability and provide algorithms to check which verdicts can be expected, a priori and during runtime verification.

Peled, Doron↗
NASA NTRS2010

Monitoring Distributed Real-Time Systems: A Survey and Future Directions

Runtime monitors have been proposed as a means to increase the reliability of safety-critical systems. In particular, this report addresses runtime monitors for distributed hard real-time systems. This class of systems has had little attention from the monitoring community. The need for monitors is shown by discussing examples of avionic systems failure. We survey related work in the field of runtime monitoring. Several potential monitoring architectures for distributed real-time systems are presented along with a discussion of how they might be used to monitor properties of interest.

Goodloe, Alwyn E.↗
NASA NTRSDate not supplied

Monitoring ROS2: From Requirements to Autonomous Robots

Context/Goals - Capture mission requirements in high-level language. - Monitor requirements for robots. At a Glance - Transform requirements into runtime monitors for ROS2: - Requirements elicitation.* - Transform requirements into Temporal Logic formulas.* - Transform Temporal Logic formulas into runtime monitors.* - Generate hard real-time code for monitors.* - Generate ROS2 application directly from requirements.+ *Steps done in prior work. +Steps extended from prior work.

Runtime verification↗
NASA NTRSDate not supplied

From Requirements to Autonomous Flight: An Overview of the Monitoring ICAROUS Project

The Independent Configurable Architecture for Reliable Operations of Unmanned Systems(ICAROUS) is a software architecture incorporating a set of algorithms to enable autonomous operations of unmanned aircraft applications. This paper provides an overview of Monitoring ICAROUS, a project whose objective is to provide a formal approach to generating runtime monitors for autonomous systems from requirements written in a structured natural language. This approach integrates FRET, a formal requirement elicitation and authoring tool, and Copilot, a runtime verification framework. FRET is used to specify formal requirements in structured natural language. These requirements are translated into temporal logic formulae. Copilot is then used to generate executable runtime monitors from these temporal logic specifications. The generated monitors are directly integrated into ICAROUS to perform runtime verification during flight.

Formal Methods↗
NASA NTRSDate not supplied

Monitoring ICAROUS: From Requirements to Autonomous Flight

The Independent Configurable Architecture for Reliable Operations of Unmanned Systems (ICAROUS) is a software architecture incorporating a set of algorithms to enable autonomous operations of unmanned aircraft applications. This paper provides an overview of Monitoring ICAROUS, a project whose objective is to provide a formal approach to generating runtime monitors for autonomous systems from requirements written in a structured natural language. This approach integrates FRET, a formal requirement elicitation and authoring tool, and Copilot, a runtime verification framework. FRET is used to specify formal requirements in structured natural language. These requirements are translated into temporal logic formulae. Copilot is then used to generate executable runtime monitors from these temporal logic specifications. The generated monitors are directly integrated into ICAROUS to perform runtime verification during flight.

Formal Methods↗
NASA NTRS2015

R2U2: Monitoring and Diagnosis of Security Threats for Unmanned Aerial Systems

We present R2U2, a novel framework for runtime monitoring of security properties and diagnosing of security threats on-board Unmanned Aerial Systems (UAS). R2U2, implemented in FPGA hardware, is a real-time, REALIZABLE, RESPONSIVE, UNOBTRUSIVE Unit for security threat detection. R2U2 is designed to continuously monitor inputs from the GPS and the ground control station, sensor readings, actuator outputs, and flight software status. By simultaneously monitoring and performing statistical reasoning, attack patterns and post-attack discrepancies in the UAS behavior can be detected. R2U2 uses runtime observer pairs for linear and metric temporal logics for property monitoring and Bayesian networks for diagnosis of security threats. We discuss the design and implementation that now enables R2U2 to handle security threats and present simulation results of several attack scenarios on the NASA DragonEye UAS.

Formal Methods↗
NASA NTRS2012

Optimized Temporal Monitors for SystemC

SystemC is a modeling language built as an extension of C++. Its growing popularity and the increasing complexity of designs have motivated research efforts aimed at the verification of SystemC models using assertion-based verification (ABV), where the designer asserts properties that capture the design intent in a formal language such as PSL or SVA. The model then can be verified against the properties using runtime or formal verification techniques. In this paper we focus on automated generation of runtime monitors from temporal properties. Our focus is on minimizing runtime overhead, rather than monitor size or monitor-generation time. We identify four issues in monitor generation: state minimization, alphabet representation, alphabet minimization, and monitor encoding. We conduct extensive experimentation and identify a combination of settings that offers the best performance in terms of runtime overhead.

Tabakov, Deian↗
NASA NTRS2019

Software and System Health Management with R2U2

R2U2 (Realizable, Responsive, Unobtrusive Unit) is a hardware-supported tool and framework for the real-time system and software health management of cyber-physical systems. R2U2 continuously monitors properties about safety, performance, and security of the vehicle and can perform diagnostic reasoning. Efficient observers for past-time and future-time Metric Temporal Logic, reasoners for Bayesian Networks, and model-based prognostics algorithms are major components of R2U2. Their combination makes it possible to design powerful models for system runtime monitoring, diagnostics, software health management, prognostics, and security monitoring. The R2U2 monitoring engine is designed for minimal runtime overhead and is available as Simulink block or as a software component for integration into the flight software stack, and enables R2U2 to monitor complex cyber-physical systems without any instrumentation of the flight software. In this presentation, we give an overview of R2U2 architecture and reasoning algorithms, present its features, and give a life demo of the tool.

Schumann, Johann↗
NASA NTRS2017

Traveler Phase 1A Joint Review

The briefing contains the preliminary findings and suggestions for improvement of methods used in development and evaluation of a multi monitor runtime assurance architecture for autonomous flight vehicles. Initial system design, implementation, verification, and flight testing has been conducted. As of yet detailed data review is incomplete, and flight testing has been limited to initial monitor force fights. Detailed monitor flight evaluations have yet to be performed.

autonomy↗
NASA NTRSDate not supplied

Trustworthy Autonomy for Gateway Vehicle System Manager

The Vehicle System Manager (VSM) is the highest-level software control system in the Gateway hierarchical Autonomous System Management Architecture. The VSM provides four function categories: Mission Management and Timeline Execution, Resource Management, Fault Management, Vehicle Control and Operation. VSM provides various levels of automation ranging from fully autonomous operations with no flight crew and minimal ground monitoring to advisory automation when Gateway is crewed and has full ground monitoring. Trustworthiness is achieved via verified specification, comprehensive development verification, and real-time verification using assume-guarantee contracts. Development verification includes semantic verification of the data model via peer review and testing and assume-guarantee contracts implemented using the PlusCal/TLA+ environment. VSM also uses runtime assume-guarantee contracts, implemented in R2U2 via a runtime monitor that feeds the necessary telemetry data to R2U2 and which receives and responds to the R2U2 verdict stream. The full lifecycle verification approach and use of assume-guarantee contracts provides increased trustworthiness to VSM. Preliminary results provide encouragement that VSM can be both autonomous and trustworthy.

Assume-guarantee contracts↗
NASA NTRS2013

Towards Real-time, On-board, Hardware-Supported Sensor and Software Health Management for Unmanned Aerial Systems

Unmanned aerial systems (UASs) can only be deployed if they can effectively complete their missions and respond to failures and uncertain environmental conditions while maintaining safety with respect to other aircraft as well as humans and property on the ground. In this paper, we design a real-time, on-board system health management (SHM) capability to continuously monitor sensors, software, and hardware components for detection and diagnosis of failures and violations of safety or performance rules during the flight of a UAS. Our approach to SHM is three-pronged, providing: (1) real-time monitoring of sensor and/or software signals; (2) signal analysis, preprocessing, and advanced on the- fly temporal and Bayesian probabilistic fault diagnosis; (3) an unobtrusive, lightweight, read-only, low-power realization using Field Programmable Gate Arrays (FPGAs) that avoids overburdening limited computing resources or costly re-certification of flight software due to instrumentation. Our implementation provides a novel approach of combining modular building blocks, integrating responsive runtime monitoring of temporal logic system safety requirements with model-based diagnosis and Bayesian network-based probabilistic analysis. We demonstrate this approach using actual data from the NASA Swift UAS, an experimental all-electric aircraft.

System & Software Health Management↗
NASA NTRS1990

Validation environment for AIPS/ALS: Implementation and results

The work is presented which was performed in porting the Fault Injection-based Automated Testing (FIAT) and Programming and Instrumentation Environments (PIE) validation tools, to the Advanced Information Processing System (AIPS) in the context of the Ada Language System (ALS) application, as well as an initial fault free validation of the available AIPS system. The PIE components implemented on AIPS provide the monitoring mechanisms required for validation. These mechanisms represent a substantial portion of the FIAT system. Moreover, these are required for the implementation of the FIAT environment on AIPS. Using these components, an initial fault free validation of the AIPS system was performed. The implementation is described of the FIAT/PIE system, configured for fault free validation of the AIPS fault tolerant computer system. The PIE components were modified to support the Ada language. A special purpose AIPS/Ada runtime monitoring and data collection was implemented. A number of initial Ada programs running on the PIE/AIPS system were implemented. The instrumentation of the Ada programs was accomplished automatically inside the PIE programming environment. PIE's on-line graphical views show vividly and accurately the performance characteristics of Ada programs, AIPS kernel and the application's interaction with the AIPS kernel. The data collection mechanisms were written in a high level language, Ada, and provide a high degree of flexibility for implementation under various system conditions.

Segall, Zary↗
NASA NTRS2015

Towards Real-Time, On-Board, Hardware-Supported Sensor and Software Health Management for Unmanned Aerial Systems

For unmanned aerial systems (UAS) to be successfully deployed and integrated within the national airspace, it is imperative that they possess the capability to effectively complete their missions without compromising the safety of other aircraft, as well as persons and property on the ground. This necessity creates a natural requirement for UAS that can respond to uncertain environmental conditions and emergent failures in real-time, with robustness and resilience close enough to those of manned systems. We introduce a system that meets this requirement with the design of a real-time onboard system health management (SHM) capability to continuously monitor sensors, software, and hardware components. This system can detect and diagnose failures and violations of safety or performance rules during the flight of a UAS. Our approach to SHM is three-pronged, providing: (1) real-time monitoring of sensor and software signals; (2) signal analysis, preprocessing, and advanced on-the-fly temporal and Bayesian probabilistic fault diagnosis; and (3) an unobtrusive, lightweight, read-only, low-power realization using Field Programmable Gate Arrays (FPGAs) that avoids overburdening limited computing resources or costly re-certification of flight software. We call this approach rt-R2U2, a name derived from its requirements. Our implementation provides a novel approach of combining modular building blocks, integrating responsive runtime monitoring of temporal logic system safety requirements with model-based diagnosis and Bayesian network-based probabilistic analysis. We demonstrate this approach using actual flight data from the NASA Swift UAS.

Unmanned Aerial System↗