Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “runtime assurance”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

32 records · Page 2

Copilot 3

Ultra-critical systems require high-level assurance, which cannot always be guaranteed in compile time. The use of runtime verification (RV) enables monitoring these systems in runtime, to detect property violations early and limit their potential consequences. The introduction of monitors in ultra-critical systems poses a challenge, as failures and delays in the RV subsystem could affect other subsystems and threaten the mission as a whole. This paper presents Copilot 3, a runtime verification framework for real-time embedded systems. Copilot monitors are written in a compositional, stream-based language with support for a variety of Temporal Logics (TL), which results in robust, high-level specifications that are easier to understand than their traditional counterparts. The framework translates monitor specifications into C code with static memory requirements, which can be compiled to run on embedded hardware. This paper presents version 3 of the Copilot language, demonstrates its suitability with a number of examples, and discusses its use in larger applications. Additionally, it describes the framework?s architecture, its implementation as a Domain Specific Language (DSL) embedded in Haskell, and the progress of the project over the years.

Ivan Perez↗

Integrating FRET with Copilot: Automated Translation of Natural Language Requirements to Runtime Monitors

Runtime verification (RV) enables monitoring systems at runtime, to detect property violations early and limit their potential consequences. To provide the level of assurance required for ultra-critical systems, monitor specifications must faithfully reflect the original mission requirements, which are often written in ambiguous natural language. This paper presents an end-to-end framework to capture requirements in structured natural language and generate monitors that capture their semantics faithfully. We leverage NASA’s Formal Requirement Elicitation Tool (FRET), and the RV system Copilot. We extend FRET with mechanisms to capture additional information needed to generate monitors, and introduce OGMA, a new tool to bridge the gap between FRET and Copilot. With this framework, users can write requirements in an intuitive format and obtain real-time C monitors suitable for use in embedded systems. Our tool chain is available as open source.

FRET↗

R2U2: Tool Overview

R2U2 (Realizable, Responsive, Unobtrusive Unit) is an extensible framework for runtime System HealthManagement (SHM) of cyber-physical systems. R2U2 can be run in hardware (e.g., FPGAs), or software; can monitorhardware, software, or a combination of the two; and can analyze a range of different types of system requirementsduring runtime. An R2U2 requirement is specified utilizing a hierarchical combination of building blocks: temporal formula runtime observers (in LTL or MTL), Bayesian networks, sensor filters, and Boolean testers. Importantly, the framework is extensible; it is designed to enable definitions of new building blocks in combination with the core structure. Originally deployed on Unmanned Aerial Systems (UAS), R2U2 is designed to run on a wide range of embedded platforms, from autonomous systems like rovers, satellites, and robots, to human-assistive ground systems and cockpits. R2U2 is named after the requirements it satisfies; while the exact requirements vary by platform and mission, the ability to formally reason about realizability, responsiveness, and unobtrusiveness is necessary for flight certifiability, safety-critical system assurance, and achievement of technology readiness levels for target systems. Realizability ensures that R2U2 is suficiently expressive to encapsulate meaningful runtime requirements while maintaining adaptability to run on different platforms, transition between different mission stages, and update quickly between missions. Responsiveness entails continuously monitoring the system under test, real-time reasoning, reporting intermediate status, and as-early-as-possible requirements evaluations. Unobtrusiveness ensures compliance with the crucial properties of the target architecture: functionality, certifiability, timing, tolerances, cost, or other constraints.

Rozier, Kristin Y.↗

Assuring and Securing Machine Learning

A short presentation highlighting using machine learning and topological data analysis to address the challenges of assuring and securing machine learning enabled systems.

Machine Learning↗

Quantum Software Engineering (Dagstuhl Seminar 24512)

The Dagstuhl Seminar 24512 on "Quantum Software Engineering" was held from December 15 to 20, 2024. It brought together 26 participants from industry and academia from 13 different countries, including senior and junior researchers as well as practitioners in the field of Quantum Software Engineering. The aim of the seminar was to advance software engineering methods and tools for the engineering of hybrid quantum systems by promoting personal interaction and open discussion among researchers who are already working in this emerging area of knowledge. The first day of the seminar was devoted to the topic "When software engineering meets quantum mechanics", while the second day focused on "Quantum software engineering and its challenges." During both days, 16 invited presentations were given. The rest of the seminar was organized into three working groups to address the topics "Quantum Software Design, Modelling and Architecturing", "Adaptive Hybrid Quantum Systems", and "Quantum Software Quality Assurance". The seminar was a very fruitful experience for all participants both in terms of scientific outcomes and in terms of the personal relationships that were generated to jointly address future experiences.

97 MATHEMATICS AND COMPUTING↗

Certification Concepts for AI/ML Systems

This presentation goes over some of the tools developed at NASA Ames in the Robust Software Engineering group for the assurance and certification of autonomous systems. The research themes presented include improving safety and risk assessment as early as possible in the lifecycle, elicitation and formalization of requirements to facilitate traceability throughout the lifecycle, especially when formal methods are used, algorithms, tools and techniques for the V&V of ML-enabled systems, advanced testing, use of runtime monitoring to ease use of untrusted components, and contribution to draft regulatory standards and assistance in producing and presenting certification evidences.

Autonomy↗

Experience Report: A Do-It-Yourself High-Assurance Compiler

Embedded domain-specific languages (EDSLs) are an approach for quickly building new languages while maintaining the advantages of a rich metalanguage. We argue in this experience report that the "EDSL approach" can surprisingly ease the task of building a high-assurance compiler.We do not strive to build a fully formally-verified tool-chain, but take a "do-it-yourself" approach to increase our confidence in compiler-correctness without too much effort. Copilot is an EDSL developed by Galois, Inc. and the National Institute of Aerospace under contract to NASA for the purpose of runtime monitoring of flight-critical avionics. We report our experience in using type-checking, QuickCheck, and model-checking "off-the-shelf" to quickly increase confidence in our EDSL tool-chain.

Pike, Lee↗

Semantic embedding for quantum algorithms

The study of classical algorithms is supported by an immense understructure, founded in logic, type, and category theory, that allows an algorithmist to reason about the sequential manipulation of data irrespective of a computation’s realizing dynamics. As quantum computing matures, a similar need has developed for an assurance of the correctness of high-level quantum algorithmic reasoning. Parallel to this need, many quantum algorithms have been unified and improved using quantum signal processing (QSP) and quantum singular value transformation (QSVT), which characterize the ability, by alternating circuit ansätze, to transform the singular values of sub-blocks of unitary matrices by polynomial functions. However, while the algebraic manipulation of polynomials is simple (e.g., compositions and products), the QSP/QSVT circuits realizing analogous manipulations of their embedded polynomials are non-obvious. This work constructs and characterizes the runtime and expressivity of QSP/QSVT protocols where circuit manipulation maps naturally to the algebraic manipulation of functional transforms (termed semantic embedding). In this way, QSP/QSVT can be treated and combined modularly, purely in terms of the functional transforms they embed, with key guarantees on the computability and modularity of the realizing circuits. We also identify existing quantum algorithms whose use of semantic embedding is implicit, spanning from distributed search to proofs of soundness in quantum cryptography. The methods used, based in category theory, establish a theory of semantically embeddable quantum algorithms, and provide a new role for QSP/QSVT in reducing sophisticated algorithmic problems to simpler algebraic ones.

Physics↗

CLAS12 remote data-stream processing using ERSAP framework

Implementing a physics data processing application is relatively straightforward with the use of current containerization technologies and container image runtime services, which are prevalent in most high-performance computing (HPC) environments. However, the process is complicated by the challenges associated with data provisioning and migration, impacting the ease of workflow migration and deployment. Transitioning from traditional file-based batch processing to data-stream processing workflows is suggested as a method to streamline these workflows. This transition not only simplifies file provisioning and migration but also significantly reduces the necessity for extensive disk space. Data-stream processing is particularly effective for real-time processing during data acquisition, thereby enhancing data quality assurance. This paper introduces the integration of the JLAB CLAS12 event reconstruction application within the ERSAP data-stream processing framework that facilitates the execution of streaming event reconstruction at a remote data center and enables the return streaming of reconstructed events to JLAB while circumventing the need for temporary data storage throughout the process.

Gyurjyan, Vardan↗

Optimizing Performance on Trinity Utilizing Machine Learning, Proxy Applications and Scheduling Priorities

The sheer number of nodes continues to increase in today’s supercomputers, the first half of Trinity alone contains more than 9400 compute nodes. Since the speed of today’s clusters are limited by the slowest nodes, it more important than ever to identify slow nodes, improve their performance if it can be done, and assure minimal usage of slower nodes during performance critical runs. This is an ongoing maintenance task that occurs on a regular basis and, therefore, it is important to minimize the impact upon its users by assessing and addressing slow performing nodes and mitigating their consequences while minimizing down time. These issues can be solved, in large part, through a systematic application of fast running hardware assessment tests, the application of Machine Learning, and making use of performance data to increase efficiency of large clusters. Proxy applications utilizing both MPI and OpenMP were developed to produce data as a substitute for long runtime applications to evaluate node performance. Machine learning is applied to identify underperforming nodes, and policies are being discussed to both minimize the impact of underperforming nodes and increase the efficiency of the system. In this paper, I will describe the process used to produce quickly performing proxy tests, consider various methods to isolate the outliers, and produce ordered lists for use in scheduling to accomplish this task.

97 MATHEMATICS AND COMPUTING↗

Efficient and assured reinforcement learning-based building HVAC control with heterogeneous expert-guided training

Abstract Building heating, ventilation, and air conditioning (HVAC) systems account for nearly half of building energy consumption and $$20\%$$ of total energy consumption in the US. Their operation is also crucial for ensuring the physical and mental health of building occupants. Compared with traditional model-based HVAC control methods, the recent model-free deep reinforcement learning (DRL) based methods have shown good performance while do not require the development of detailed and costly physical models. However, these model-free DRL approaches often suffer from long training time to reach a good performance, which is a major obstacle for their practical deployment. In this work, we present a systematic approach to accelerate online reinforcement learning for HVAC control by taking full advantage of the knowledge from domain experts in various forms . Specifically, the algorithm stages include learning expert functions from existing abstract physical models and from historical data via offline reinforcement learning, integrating the expert functions with rule-based guidelines, conducting training guided by the integrated expert function and performing policy initialization from distilled expert function. Moreover, to ensure that the learned DRL-based HVAC controller can effectively keep room temperature within the comfortable range for occupants, we design a runtime shielding framework to reduce the temperature violation rate and incorporate the learned controller into it. Experimental results demonstrate up to 8.8 X speedup in DRL training from our approach over previous methods, with low temperature violation rate.

Xu, Shichao↗

Cloud Computing Option for Modeling the Debris Environment

NASA’s Digital Transformation Initiative aims to promote the agency’s adoption of current and evolving digital technologies. Through agency-wide collaboration with other NASA teams, the Office of Safety and Mission Assurance (OSMA) has directed the Orbital Debris Program Office and the Meteoroid Environment Office to integrate cloud computing technologies in their publicly released software models: the Orbital Debris Engineering Model (ORDEM) and the Meteoroid Engineering Model (MEM). Decoupling the user interface from the backend processor was key for the software packages to run on a cloud computing framework. Benefits to this design include horizontal scaling of computing resources, user authentication and authorization, and automated deployment. Both models are hosted on a cloud computing platform supported by the NASA authorized IT security and compliance framework. This paper focuses on the new ORDEM web application, which includes the current features of the publicly released ORDEM software with an upgraded frontend design. The underlying ORDEM processor is run on a cloud container, allowing the user to run multiple spacecraft and telescope/radar mode simulations. Featuresexclusive to the ORDEM web application, such as importing multiple TLEs, auto-generated plotting, and the ability to check runtime progress are discussed. Comparisons between the current ORDEM software and the web application are summarized.

Andrew Vavrin↗

Cloud Computing Option for Modeling the Debris Environment

NASA’s Digital Transformation Initiative aims to promote the agency’s adoption of current and evolving digital technologies. Through agency-wide collaboration with other NASA teams, the Office of Safety and Mission Assurance (OSMA) has directed the Orbital Debris Program Office and the Meteoroid Environment Office to integrate cloud computing technologies into their publicly released software models: the Orbital Debris Engineering Model (ORDEM) and the Meteoroid Engineering Model (MEM). Decoupling the user interface from the backend processor was key for the software packages to run on a cloud computing framework. Benefits to this design include horizontal scaling of computing resources, user authentication and authorization, and automated deployment. Both models are hosted on a cloud computing platform supported by the NASA authorized IT security and compliance framework. This paper focuses on the new ORDEM web application, which includes the current features of the publicly released ORDEM software with an upgraded frontend design, although parallels between ORDEM and MEM are also discussed. The underlying ORDEM processor is run on a cloud container, allowing the user to run multiple spacecraft and telescope/radar mode simulations. Features exclusive to the ORDEM web application, such as importing multiple TLEs, auto-generated plotting, and the ability to check runtime progress are discussed. Comparisons between the current ORDEM software and the web application are summarized.

Andrew Vavrin↗

Assessment of the Griffin Reactor Multiphysics Application Using the Empire Micro Reactor Design Concept

In late 2019, INL and ANL agreed to jointly develop the reactor physics code named Griffin based on the integration of the two code suites, MAMMOTH/Rattlesnake (INL) and MC2 - 3/PROTEUS (ANL). Griffin is being developed based on the MOOSE framework and MOOSE quality assurance procedures. This decision was made to be able to allow DOE-NE to efficiently invest funding to this area and to provide effective and timely support for existing and potential users; the latter includes industry and government organizations who are developing various types of advanced reactors in the near and long term. Since MAMMOTH/Rattlesnake has been developed based on the MOOSE framework, the INL/ANL Griffin development team agreed to build Griffin beginning with a merger of MAMMOTH and Rattlesnake into a single code and moving forward by implementing capabilities from the PROTEUS suite into Griffin. Moving forward, both ANL and INL efforts are equally invested in the Griffin project, with management support, to provide an advanced reactor multiphysics tool to assist in reactor design, optimization, and safety analysis. Much work remains in moving Griffin forward to migrate PROTEUS capabilities and to optimize performance to meet user needs. The main objective of this work is to assess the current status of Griffin capabilities in terms of performance and accuracy, to determine priorities for PROTEUS migration, and to identify capabilities and features to improve for supporting the code integration effort. For this assessment, the Empire micro reactor problem that was developed in the ARPA-E MEITNER program was selected as an advance reactor concept of interest to the technical community. The Empire reactor problem was expanded from its original incomplete specification to be a small heat-pipe-cooled micro reactor core with ~113 cm radius and 70 cm in height, composed of 18 fuel assemblies, 12 control drums, and beryllium radial and axial reflectors. In the current model, using 5 cm axial reflectors specified in the original Empire assembly model, more than 10% of neutrons leak axially and through the empty center safety hole, as well as through heat pipe channels in fuel assembly elements that extend through the top reflector region. Several calculation models of the core were defined for systematic assessment, including 2-D and 3-D fuel assemblies and whole cores with cylindrical boundaries. Cross sections were generated using Serpent 2, and meshes were produced using the Argonne mesh tool or the INL neutronics meshing tools combined with CUBIT. Cross sections and meshes were converted to the ISOXML and Exodus formats, respectively, so that Griffin and PROTEUS could use consistent data for solving the reactor problems. With the prepared cross sections and meshes, PROTEUS was run first to ensure that all input data were correctly generated and input options in terms of angle, mesh, and energy group were accurately determined. Comparisons against Serpent 2 solutions were made in terms of eigenvalue and pin power. The same calculations and comparisons were then conducted using Griffin. For the fuel assembly and whole core problems, the PROTEUS eigenvalues agreed well with reference Serpent 2 solutions within 100 and 30 pcm, respectively, and pin power differences relative to Serpent 2 were overall less than 2.2% and RMS 0.8% for the whole core models. This indicated that all input data were properly prepared. Using the same data, Griffin was run selecting the SAAF-CFEM SN solver with Legendre-Gaussian quadrature and NDA and DSA for acceleration. It was found that the SAAF-CFEM solver of Griffin required finer meshes to achieve eigenvalue and pin power solutions in good agreement with Serpent 2, consequently requiring more memory requirement and longer computation time. On the other hand, the SPH-Diffusion 2-D core calculations performed using Griffin were able to recover the exact eigenvalue from the reference Serpent 2 solutions, resulting in a pin-power distribution with an RMS of 0.6% and maximum absolute difference of less than 1.4%. The runtimes for SPH-Diffusion for the 2-D core were less than 3 minutes on 40 cores. During this evolution of this evaluation, many updates were made in Griffin by the Griffin development team of INL (focusing on software updates) and ANL (reviewing and supporting software updates) to complete this assessment. Observations from the code assessment are presented in the conclusion section of this report, followed by a discussion of recommendations for future work.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗