Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Formal verification”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 487 records · Page 27

Aviation Safety: Modeling and Analyzing Complex Interactions between Humans and Automated Systems

The on-going transformation from the current US Air Traffic System (ATS) to the Next Generation Air Traffic System (NextGen) will force the introduction of new automated systems and most likely will cause automation to migrate from ground to air. This will yield new function allocations between humans and automation and therefore change the roles and responsibilities in the ATS. Yet, safety in NextGen is required to be at least as good as in the current system. We therefore need techniques to evaluate the safety of the interactions between humans and automation. We think that current human factor studies and simulation-based techniques will fall short in front of the ATS complexity, and that we need to add more automated techniques to simulations, such as model checking, which offers exhaustive coverage of the non-deterministic behaviors in nominal and off-nominal scenarios. In this work, we present a verification approach based both on simulations and on model checking for evaluating the roles and responsibilities of humans and automation. Models are created using Brahms (a multi-agent framework) and we show that the traditional Brahms simulations can be integrated with automated exploration techniques based on model checking, thus offering a complete exploration of the behavioral space of the scenario. Our formal analysis supports the notion of beliefs and probabilities to reason about human behavior. We demonstrate the technique with the Ueberligen accident since it exemplifies authority problems when receiving conflicting advices from human and automated systems.

Muti-agent System↗

An approach to verification and validation of a reliable multicasting protocol: Extended Abstract

This paper describes the process of implementing a complex communications protocol that provides reliable delivery of data in multicast-capable, packet-switching telecommunication networks. The protocol, called the Reliable Multicasting Protocol (RMP), was developed incrementally using a combination of formal and informal techniques in an attempt to ensure the correctness of its implementation. Our development process involved three concurrent activities: (1) the initial construction and incremental enhancement of a formal state model of the protocol machine; (2) the initial coding and incremental enhancement of the implementation; and (3) model-based testing of iterative implementations of the protocol. These activities were carried out by two separate teams: a design team and a V&V team. The design team built the first version of RMP with limited functionality to handle only nominal requirements of data delivery. This initial version did not handle off-nominal cases such as network partitions or site failures. Meanwhile, the V&V team concurrently developed a formal model of the requirements using a variant of SCR-based state tables. Based on these requirements tables, the V&V team developed test cases to exercise the implementation. In a series of iterative steps, the design team added new functionality to the implementation while the V&V team kept the state model in fidelity with the implementation. This was done by generating test cases based on suspected errant or off-nominal behaviors predicted by the current model. If the execution of a test in the model and implementation agreed, then the test either found a potential problem or verified a required behavior. However, if the execution of a test was different in the model and implementation, then the differences helped identify inconsistencies between the model and implementation. In either case, the dialogue between both teams drove the co-evolution of the model and implementation. We have found that this interactive, iterative approach to development allows software designers to focus on delivery of nominal functionality while the V&V team can focus on analysis of off nominal cases. Testing serves as the vehicle for keeping the model and implementation in fidelity with each other. This paper describes (1) our experiences in developing our process model; and (2) three example problems found during the development of RMP. Although RMP has provided our research effort with a rich set of test cases, it also has practical applications within NASA. For example, RMP is being considered for use in the NASA EOSDIS project due to its significant performance benefits in applications that need to replicate large amounts of data to many network sites.

Callahan, John R.↗

Automatic Review of Abstract State Machines by Meta Property Verification

A model review is a validation technique aimed at determining if a model is of sufficient quality and allows defects to be identified early in the system development, reducing the cost of fixing them. In this paper we propose a technique to perform automatic review of Abstract State Machine (ASM) formal specifications. We first detect a family of typical vulnerabilities and defects a developer can introduce during the modeling activity using the ASMs and we express such faults as the violation of meta-properties that guarantee certain quality attributes of the specification. These meta-properties are then mapped to temporal logic formulas and model checked for their violation. As a proof of concept, we also report the result of applying this ASM review process to several specifications.

Arcaini, Paolo↗

Calibration of the Gamma Knife Perfexion using TG-21 and the solid water Leksell dosimetry phantom

Purpose: To calibrate a Gamma Knife (GK) Perfexion using TG-21 with updated chamber-dependent values for modern microionization chambers in the new solid water Leksell dosimetry phantom. This work illustrates a calibration method using commercially available equipment, instruments, and an established dosimetry protocol that may be adopted at any GK center, thus reducing the interinstitutional variation in GK calibration. The calibration was verified by three third-party dosimetry checks. In addition, measurements of the relative output factors are presented and compared to available data and the new manufacturer-provided relative output factors yet to be released. Methods: An absolute dose calibration based on the TG-21 formalism, utilizing recently reported phantom material and chamber-dependent factors, was performed using a microionization chamber in a spherical solid water phantom. The result was compared to other calibration protocols based on TG-51. Independent verification of the machine output was conducted through M.D. Anderson Dosimetry Services (MDADS), using thermoluminescent dosimeters (TLDs) in an anthropomorphic head phantom; the Radiological Physics Center (RPC), using TLDs in the standard Elekta ABS plastic calibration phantom (gray phantom), included with the GK; and through a collaborative international calibration survey by the University of Pittsburgh Medical Center (UPMC) using alanine dosimeters, also in the gray phantom. The alanine dosimeters were read by the National Institute of Standards and Technology. Finally, Gafchromic EBT film was used to measure relative output factors and these factors were compared to values reported in the literature as well as new values announced for release by Elekta. The films were exposed in the solid water phantom using an included film insert accessory. Results: Compared to the TG-21 protocol in the solid water phantom, the modified and unmodified TG-51 calibrations resulted in dose rates which were 1.8% and 1.3% lower, respectively. Ratios of the doses measured by third parties to the dose reported showed excellent agreement. MDADS returned ratios of 1.00 and 0.98 for the two TLDs irradiated. The RPC returned a mean ratio of 0.98 of the dose reported and the UPMC alanine study returned a mean ratio of 1.008. Relative output factors were found to be 0.817{+-}0.009 and 0.897{+-}0.008 for the 4 and 8 mm collimators, respectively, which are in excellent agreement with revised Monte Carlo-derived relative output factors Elekta is expected to recommend with the next version of the GK treatment planning software (GAMMAPLAN version 10). Conclusions: The TG-21 dosimetry protocol, performed in a solid water phantom in conjunction with modern dosimeters and phantom material and chamber-dependent factors, can yield an accurate dose measurement in the unique GK treatment geometry. The technique described here can be easily adopted by institutions worldwide since all equipment and instruments used are commercially available, thus reducing the existing interinstitutional variation in GK calibration techniques. Relative output factor measurements made in this same solid water phantom were used to verify the relative output factors provided by Elekta and agreed excellently with output factors expected to be released in conjunction with GAMMAPLAN version 10.

60 APPLIED LIFE SCIENCES↗

V&V Framework

A Verification and Validation (V&V) framework is presented for the development and execution of coordinated modeling and experimental program s to assess the predictive capability of computational models of complex systems through focused, well structured, and formal processes. The elements of the framework are based on established V&V methodology developed by various organizations including the Department of Energy, National Aeronautics and Space Administration, the American Institute of Aeronautics and Astronautics, and the American Society of Mechanical Engineers. Four main topics are addressed: 1) Program planning based on expert elicitation of the modeling physics requirements, 2) experimental design for model assessment, 3) uncertainty quantification for experimental observations and computational model simulations, and 4) assessment of the model predictive capability. The audience for this document includes program planners, modelers, experimentalist, V &V specialist, and customers of the modeling results.

97 MATHEMATICS AND COMPUTING↗

Traceable Black-Box Watermarks For Federated Learning

Due to the distributed nature of Federated Learning (FL) systems, each local client has access to the global model, which poses a critical risk of model leakage. Existing works have explored injecting watermarks into local models to enable intellectual property protection. However, these methods either focus on non-traceable watermarks or traceable but white-box watermarks. We identify a gap in the literature regarding the formal definition of traceable black-box watermarking and the formulation of the problem of injecting such watermarks into FL systems. In this work, we first formalize the problem of injecting traceable black-box watermarks into FL. Based on the problem, we propose a novel server-side watermarking method, TraMark, which creates a traceable watermarked model for each client, enabling verification of model leakage in black-box settings. To achieve this, TraMark partitions the model parameter space into two distinct regions: the main task region and the watermarking region. Subsequently, a personalized global model is constructed for each client by aggregating only the main task region while preserving the watermarking region. Each model then learns a unique watermark exclusively within the watermarking region using a distinct watermark dataset before being sent back to the local client. Extensive results across various FL systems demonstrate that TraMark ensures the traceability of all watermarked models while preserving their main task performance.

Xu, Jiahao [University of Nevada, Reno]↗

Nuclear Energy Knowledge and Validation Center (NEKVaC) Needs Workshop Summary Report

The Department of Energy (DOE) has made significant progress developing simulation tools to predict the behavior of nuclear systems with greater accuracy and of increasing our capability to predict the behavior of these systems outside of the standard range of applications. These analytical tools require a more complex array of validation tests to accurately simulate the physics and multiple length and time scales. Results from modern simulations will allow experiment designers to narrow the range of conditions needed to bound system behavior and to optimize the deployment of instrumentation to limit the breadth and cost of the campaign. Modern validation, verification and uncertainty quantification (VVUQ) techniques enable analysts to extract information from experiments in a systematic manner and provide the users with a quantified uncertainty estimate. Unfortunately, the capability to perform experiments that would enable taking full advantage of the formalisms of these modern codes has progressed relatively little (with some notable exceptions in fuels and thermal-hydraulics); the majority of the experimental data available today is the "historic" data accumulated over the last decades of nuclear systems R&D. A validated code-model is a tool for users. An unvalidated code-model is useful for code developers to gain understanding, publish research results, attract funding, etc. As nuclear analysis codes have become more sophisticated, so have the measurement and validation methods and the challenges that confront them. A successful yet cost-effective validation effort requires expertise possessed only by a few, resources possessed only by the well-capitalized (or a willing collective), and a clear, well-defined objective (validating a code that is developed to satisfy the need(s) of an actual user). To that end, the Idaho National Laboratory established the Nuclear Energy Knowledge and Validation Center to address the challenges of modern code validation and to manage the knowledge from past, current, and future experimental campaigns. By pulling together the best minds involved in code development, experiment design, and validation to establish and disseminate best practices and new techniques, the Nuclear Energy Knowledge and Validation Center (NEKVaC or the ‘Center’) will be a resource for industry, DOE Programs, and academia validation efforts.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

Specification and verification of gate-level VHDL models of synchronous and asynchronous circuits

We present a mathematical definition of hardware description language (HDL) that admits a semantics-preserving translation to a subset of VHDL. Our HDL includes the basic VHDL propagation delay mechanisms and gate-level circuit descriptions. We also develop formal procedures for deriving and verifying concise behavioral specifications of combinational and sequential devices. The HDL and the specification procedures have been formally encoded in the computational logic of Boyer and Moore, which provides a LISP implementation as well as a facility for mechanical proof-checking. As an application, we design, specify, and verify a circuit that achieves asynchronous communication by means of the biphase mark protocol.

Russinoff, David M.↗

Using Life-Cycle Human Factors Engineering to Avoid $2.4 Million in Costs: Lessons Learned from NASA's Requirements Verification Process for Space Payloads

The Human Factors Implementation Team (HFIT) process has been used to verify human factors requirements for NASA International Space Station (ISS) payloads since 2003, resulting in $2.4 million in avoided costs. This cost benefit has been realized by greatly reducing the need to process time-consuming formal waivers (exceptions) for individual requirements violations. The HFIT team, which includes astronauts and their technical staff, acts as the single source for human factors requirements integration of payloads. HFIT has the authority to provide inputs during early design phases, thus eliminating many potential requirements violations in a cost-effective manner. In those instances where it is not economically or technically feasible to meet the precise metric of a given requirement, HFIT can work with the payload engineers to develop common sense solutions and formally document that the resulting payload design does not materially affect the astronaut s ability to operate and interact with the payload. The HFIT process is fully ISO 9000 compliant and works concurrently with NASA s formal systems engineering work flow. Due to its success with payloads, the HFIT process is being adapted and extended to ISS systems hardware. Key aspects of this process are also being considered for NASA's Space Shuttle replacement, the Crew Exploration Vehicle.

Carr, Daniel↗

Automated Verification of Design Patterns with LePUS3

Specification and [visual] modelling languages are expected to combine strong abstraction mechanisms with rigour, scalability, and parsimony. LePUS3 is a visual, object-oriented design description language axiomatized in a decidable subset of the first-order predicate logic. We demonstrate how LePUS3 is used to formally specify a structural design pattern and prove ( verify ) whether any JavaTM 1.4 program satisfies that specification. We also show how LePUS3 specifications (charts) are composed and how they are verified fully automatically in the Two-Tier Programming Toolkit.

Nicholson, Jonathan↗

DETECTION OF LENSING SUBSTRUCTURE USING ALMA OBSERVATIONS OF THE DUSTY GALAXY SDP.81

We study the abundance of substructure in the matter density near galaxies using ALMA Science Verification observations of the strong lensing system SDP.81. We present a method to measure the abundance of subhalos around galaxies using interferometric observations of gravitational lenses. Using simulated ALMA observations we explore the effects of various systematics, including antenna phase errors and source priors, and show how such errors may be measured or marginalized. We apply our formalism to ALMA observations of SDP.81. We find evidence for the presence of a M = 10{sup 8.96±0.12} M {sub ⊙} subhalo near one of the images, with a significance of 6.9 σ in a joint fit to data from bands 6 and 7; the effect of the subhalo is also detected in both bands individually. We also derive constraints on the abundance of dark matter (DM) subhalos down to M ∼ 2 × 10{sup 7} M {sub ⊙}, pushing down to the mass regime of the smallest detected satellites in the Local Group, where there are significant discrepancies between the observed population of luminous galaxies and predicted DM subhalos. We find hints of additional substructure, warranting further study using the full SDP.81 data set (including, for example, the spectroscopic imaging of the lensed carbon monoxide emission). We compare the results of this search to the predictions of ΛCDM halos, and find that given current uncertainties in the host halo properties of SDP.81, our measurements of substructure are consistent with theoretical expectations. Observations of larger samples of gravitational lenses with ALMA should be able to improve the constraints on the abundance of galactic substructure.

79 ASTRONOMY AND ASTROPHYSICS↗

A Multi-Encoding Approach for LTL Symbolic Satisfiability Checking

Formal behavioral specifications written early in the system-design process and communicated across all design phases have been shown to increase the efficiency, consistency, and quality of the system under development. To prevent introducing design or verification errors, it is crucial to test specifications for satisfiability. Our focus here is on specifications expressed in linear temporal logic (LTL). We introduce a novel encoding of symbolic transition-based Buchi automata and a novel, "sloppy," transition encoding, both of which result in improved scalability. We also define novel BDD variable orders based on tree decomposition of formula parse trees. We describe and extensively test a new multi-encoding approach utilizing these novel encoding techniques to create 30 encoding variations. We show that our novel encodings translate to significant, sometimes exponential, improvement over the current standard encoding for symbolic LTL satisfiability checking.

Rozier, Kristin Y.↗

Kodiak: An Implementation Framework for Branch and Bound Algorithms

Recursive branch and bound algorithms are often used to refine and isolate solutions to several classes of global optimization problems. A rigorous computation framework for the solution of systems of equations and inequalities involving nonlinear real arithmetic over hyper-rectangular variable and parameter domains is presented. It is derived from a generic branch and bound algorithm that has been formally verified, and utilizes self-validating enclosure methods, namely interval arithmetic and, for polynomials and rational functions, Bernstein expansion. Since bounds computed by these enclosure methods are sound, this approach may be used reliably in software verification tools. Advantage is taken of the partial derivatives of the constraint functions involved in the system, firstly to reduce the branching factor by the use of bisection heuristics and secondly to permit the computation of bifurcation sets for systems of ordinary differential equations. The associated software development, Kodiak, is presented, along with examples of three different branch and bound problem types it implements.

Smith, Andrew P.↗

Formal Requirements Elicitation with FRET

FRET is a tool for writing, understanding, formalizing and analyzing requirements. Users write requirements in an intuitive, restricted natural language, called FRETISH, with precise, unambiguous meaning. For a FRETISH requirement, FRET: 1) produces natural language and diagrammatic explanations of its exact meaning, 2) formalizes the requirement in logics, and 3) supports interactive simulation of produced logic formulas to ensure that they capture user intentions. FRET connects to analysis tools by facilitating the mapping between requirements and models/code, and by generating verification code. FRET is available open source at https://github.com/NASA-SW-VnV/fret; a video can be accessed at : https://tinyurl.com/fretForREFSQ.

Giannakopoulou, Dimitra↗

Second order gyrokinetic theory for particle-in-cell codes

The main idea of the gyrokinetic dynamical reduction consists in a systematical removal of the fast scale motion (the gyromotion) from the dynamics of the plasma, resulting in a considerable simplification and a significant gain of computational time. The gyrokinetic Maxwell–Vlasov equations are nowadays implemented in for modeling (both laboratory and astrophysical) strongly magnetized plasmas. Different versions of the reduced set of equations exist, depending on the construction of the gyrokinetic reduction procedure and the approximations performed in the derivation. The purpose of this article is to explicitly show the connection between the general second order gyrokinetic Maxwell–Vlasov system issued from the modern gyrokinetic theory and the model currently implemented in the global electromagnetic Particle-in-Cell code ORB5. Necessary information about the modern gyrokinetic formalism is given together with the consistent derivation of the gyrokinetic Maxwell–Vlasov equations from first principles. The variational formulation of the dynamics is used to obtain the corresponding energy conservation law, which in turn is used for the verification of energy conservation diagnostics currently implemented in ORB5. This work fits within the context of the code verification project VeriGyro currently run at IPP Max-Planck Institut in collaboration with others European institutions.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Generating Code Review Documentation for Auto-Generated Mission-Critical Software

Model-based design and automated code generation are increasingly used at NASA to produce actual flight code, particularly in the Guidance, Navigation, and Control domain. However, since code generators are typically not qualified, there is no guarantee that their output is correct, and consequently auto-generated code still needs to be fully tested and certified. We have thus developed AUTOCERT, a generator-independent plug-in that supports the certification of auto-generated code. AUTOCERT takes a set of mission safety requirements, and formally verifies that the autogenerated code satisfies these requirements. It generates a natural language report that explains why and how the code complies with the specified requirements. The report is hyper-linked to both the program and the verification conditions and thus provides a high-level structured argument containing tracing information for use in code reviews.

Denney, Ewen↗

Verification and validation of a reliable multicast protocol

This paper describes the methods used to specify and implement a complex communications protocol that provides reliable delivery of data in multicast-capable, packet-switching telecommunication networks. The protocol, called the Reliable Multicasting Protocol (RMP), was developed incrementally by two complementary teams using a combination of formal and informal techniques in an attempt to ensure the correctness of the protocol implementation. The first team, called the Design team, initially specified protocol requirements using a variant of SCR requirements tables and implemented a prototype solution. The second team, called the V&V team, developed a state model based on the requirements tables and derived test cases from these tables to exercise the implementation. In a series of iterative steps, the Design team added new functionality to the implementation while the V&V team kept the state model in fidelity with the implementation through testing. Test cases derived from state transition paths in the formal model formed the dialogue between teams during development and served as the vehicles for keeping the model and implementation in fidelity with each other. This paper describes our experiences in developing our process model, details of our approach, and some example problems found during the development of RMP.

Callahan, John R.↗

Probability of Loss of Assured Safety in Systems with Multiple Time-Dependent Failure Modes: Incorporation of Delayed Link Failure in the Presence of Aleatory Uncertainty

Probability of loss of assured safety (PLOAS) is modeled for weak link (WL)/strong link (SL) systems in which one or more WLs or SLs could potentially degrade into a precursor condition to link failure that will be followed by an actual link failure after some amount of elapsed time. The descriptor loss of assured safety (LOAS) is used because failure of the WL system places the entire system in an inoperable configuration while failure of the SL system before failure of the WL system, although undesirable, does not necessarily result in an unintended operation of the entire system. Thus, safety is “assured” by failure of the WL system before failure of the SL system. Here, the following topics are considered: (i) Definition of precursor occurrence time cumulative distribution functions (CDFs) for individual WLs and SLs, (ii) Formal representation, approximation and illustration of PLOAS with (a) constant delay times, (b) aleatory uncertainty in delay times, and (c) delay times defined by functions of link properties at occurrence times for link failure precursors, and (iii) Procedures for the verification of PLOAS calculations for the three indicated definitions of delayed link failure.

58 GEOSCIENCES↗