SEARCH · Engineering Papers

Results for “verification”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records

All sources NASA NTRS DOE OSTI

NASA NTRS2003

Design for Verification: Enabling Verification of High Dependability Software-Intensive Systems

Strategies to achieve confidence that high-dependability applications are correctly implemented include testing and automated verification. Testing deals mainly with a limited number of expected execution paths. Verification usually attempts to deal with a larger number of possible execution paths. While the impact of architecture design on testing is well known, its impact on most verification methods is not as well understood. The Design for Verification approach considers verification from the application development perspective, in which system architecture is designed explicitly according to the application's key properties. The D4V-hypothesis is that the same general architecture and design principles that lead to good modularity, extensibility and complexity/functionality ratio can be adapted to overcome some of the constraints on verification tools, such as the production of hand-crafted models and the limits on dynamic and static analysis caused by state space explosion.

Mehlitz, Peter C.↗

NASA NTRS2015

Verification of Functional Fault Models and the Use of Resource Efficient Verification Tools

Functional fault models (FFMs) are a directed graph representation of the failure effect propagation paths within a system's physical architecture and are used to support development and real-time diagnostics of complex systems. Verification of these models is required to confirm that the FFMs are correctly built and accurately represent the underlying physical system. However, a manual, comprehensive verification process applied to the FFMs was found to be error prone due to the intensive and customized process necessary to verify each individual component model and to require a burdensome level of resources. To address this problem, automated verification tools have been developed and utilized to mitigate these key pitfalls. This paper discusses the verification of the FFMs and presents the tools that were developed to make the verification process more efficient and effective.

reliability↗

NASA NTRS2015

Verification of a Viscous Computational Aeroacoustics Code Using External Verification Analysis

The External Verification Analysis approach to code verification is extended to solve the three-dimensional Navier-Stokes equations with constant properties, and is used to verify a high-order computational aeroacoustics (CAA) code. After a brief review of the relevant literature, the details of the EVA approach are presented and compared to the similar Method of Manufactured Solutions (MMS). Pseudocode representations of EVA's algorithms are included, along with the recurrence relations needed to construct the EVA solution. The code verification results show that EVA was able to convincingly verify a high-order, viscous CAA code without the addition of MMS-style source terms, or any other modifications to the code.

Computational↗

NASA NTRS2015

Verification of a Viscous Computational Aeroacoustics Code using External Verification Analysis

Computational↗

NASA NTRS1991

Verification of the FtCayuga fault-tolerant microprocessor system. Volume 1: A case study in theorem prover-based verification

The design and formal verification of a hardware system for a task that is an important component of a fault tolerant computer architecture for flight control systems is presented. The hardware system implements an algorithm for obtaining interactive consistancy (byzantine agreement) among four microprocessors as a special instruction on the processors. The property verified insures that an execution of the special instruction by the processors correctly accomplishes interactive consistency, provided certain preconditions hold. An assumption is made that the processors execute synchronously. For verification, the authors used a computer aided design hardware design verification tool, Spectool, and the theorem prover, Clio. A major contribution of the work is the demonstration of a significant fault tolerant hardware design that is mechanically verified by a theorem prover.

Srivas, Mandayam↗

NASA NTRS2011

Ares I-X Range Safety Simulation Verification and Analysis Independent Validation and Verification

NASA s Ares I-X vehicle launched on a suborbital test flight from the Eastern Range in Florida on October 28, 2009. To obtain approval for launch, a range safety final flight data package was generated to meet the data requirements defined in the Air Force Space Command Manual 91-710 Volume 2. The delivery included products such as a nominal trajectory, trajectory envelopes, stage disposal data and footprints, and a malfunction turn analysis. The Air Force s 45th Space Wing uses these products to ensure public and launch area safety. Due to the criticality of these data, an independent validation and verification effort was undertaken to ensure data quality and adherence to requirements. As a result, the product package was delivered with the confidence that independent organizations using separate simulation software generated data to meet the range requirements and yielded consistent results. This document captures Ares I-X final flight data package verification and validation analysis, including the methodology used to validate and verify simulation inputs, execution, and results and presents lessons learned during the process

Merry, Carl M.↗

NASA NTRS1982

Verification and transfer of thermal pollution model. Volume 3: Verification of 3-dimensional rigid-lid model

The six-volume report: describes the theory of a three dimensional (3-D) mathematical thermal discharge model and a related one dimensional (1-D) model, includes model verification at two sites, and provides a separate user's manual for each model. The 3-D model has two forms: free surface and rigid lid. The former, verified at Anclote Anchorage (FL), allows a free air/water interface and is suited for significant surface wave heights compared to mean water depth; e.g., estuaries and coastal regions. The latter, verified at Lake Keowee (SC), is suited for small surface wave heights compared to depth (e.g., natural or man-made inland lakes) because surface elevation has been removed as a parameter. These models allow computation of time-dependent velocity and temperature fields for given initial conditions and time-varying boundary conditions. The free-surface model also provides surface height variations with time.

Lee, S. S.↗

NASA NTRS1982

Verification and transfer of thermal pollution model. Volume 5: Verification of 2-dimensional numerical model

The six-volume report: describes the theory of a three dimensional (3-D) mathematical thermal discharge model and a related one dimensional (1-D) model, includes model verification at two sites, and provides a separate user's manual for each model. The 3-D model has two forms: free surface and rigid lid. The former, verified at Anclote Anchorate (FL), allows a free air/water interface and is suited for significant surface wave heights compared to mean water depth; e.g., estuaries and coastal regions. The latter, verified at Lake Keowee (SC), is suited for small surface wave heights compared to depth (e.g., natural or man-made inland lakes) because surface elevation has been removed as a parameter. These models allow computation of time dependent velocity and temperature fields for given initial conditions and time-varying boundary conditions.

Lee, S. S.↗

NASA NTRS2012

Probabilistic Requirements (Partial) Verification Methods Best Practices Improvement. Variables Acceptance Sampling Calculators: Derivations and Verification of Plans

The NASA Engineering and Safety Center was requested to improve on the Best Practices document produced for the NESC assessment, Verification of Probabilistic Requirements for the Constellation Program, by giving a recommended procedure for using acceptance sampling by variables techniques. This recommended procedure would be used as an alternative to the potentially resource-intensive acceptance sampling by attributes method given in the document. This document contains the outcome of the assessment.

Johnson, Kenneth L.↗

NASA NTRS2000

Computer Simulations to Study Diffraction Effects of Stacking Faults in Beta-SiC: II. Experimental Verification: Experimental Verification - 2

Earlier results from computer simulation studies suggest a correlation between the spatial distribution of stacking errors in the Beta-SiC structure and features observed in X-ray diffraction patterns of the material. Reported here are experimental results obtained from two types of nominally Beta-SiC specimens, which yield distinct XRD data. These samples were analyzed using high resolution transmission electron microscopy (HRTEM) and the stacking error distribution was directly determined. The HRTEM results compare well to those deduced by matching the XRD data with simulated spectra, confirming the hypothesis that the XRD data is indicative not only of the presence and density of stacking errors, but also that it can yield information regarding their distribution. In addition, the stacking error population in both specimens is related to their synthesis conditions and it appears that it is similar to the relation developed by others to explain the formation of the corresponding polytypes.

Pujar, Vijay V.↗

NASA NTRSDate not supplied

An Evaluation of Extended Reality Technologies for Use in Verification Testing at NASA 2024 HRP IWS Abstract

BACKGROUND At NASA, verification testing is the formal process of ensuring that a product conforms to requirements set by a project or program. Some verification methods, such as Demonstrations and Test, require either the end product or a mockup of the product with sufficient fidelity to stand-in for the product during the test. Traditionally, these mockups have been physical (e.g., foam-core and wood) but there is growing interest in exploring new methods for testing with these mockups. These methods include virtual reality (VR), mixed reality, and augmented reality which are collectively referred to as eXtended Reality (XR) technologies. VR has already been adopted and used by many in the aerospace industry as a tool for use in early design phases (e.g., developmental testing) and may have the most potential for use in verification tests. Benefits of using VR mockups offer cost effectiveness, ease of iteration, simulation of hazardous conditions (e.g., an egress through a hatch with smoke obscuring vision), and the ability to simulate microgravity conditions, which are challenging to do with physical mockups. However, the validity of test results obtained from VR mockup demonstrations or testing, compared to the current gold standard of physical mockups, remains uncertain. It is unlikely that there is one clean answer as there are many different types of verification outcomes and each XR technology must be evaluated on its own merits. This is not an issue during developmental testing as the design is still in flux and the total success of the design is not dependent upon the results of a developmental test. Verification tests, however, only happen once, assuming no change to the design, and the results are used to certify the product. Therefore, establishing the validity of XR mockup-based verification outcomes is essential before considering them for any use in verification tests. OBJECTIVE AND METHOD To address this concern, the Human Research Program has funded a project to explore and qualify how XR technologies might be used in verification demonstration and testing at NASA. Currently, we are conducting a review of the literature on the utilization of XR mockups for design activities, prototyping, and user testing. We are employing the Strengths, Weaknesses, Opportunities, and Threats (SWOT) analysis method to identify the pros, cons, and barriers to adoption of XR technologies for verification testing at NASA. Additionally, we are developing a framework to guide the deployment of XR mockups for verification tests. Building upon available evidence from the literature and subject-matter expert feedback, our goal for the framework is to provide guidelines for which forms of XR mockups are suitable for a given verification test, when only physical mockups should be employed and to highlight areas for which more evidence is needed. To further refine our framework and to contribute to the body of evidence, we are planning a lab-based experiment comparing a VR mockup to a physical twin for a set of select verification outcomes. ANTICIPATED RESULTS In this presentation, we will present the work we conducted to evaluate XR technologies for use in verification tests at NASA. We will summarize and report our findings from the SWOT analysis and our lab-based study, and we will present the current state of the XR Technologies for Verification Testing framework. We will conclude by summarizing remaining work and future directions for the project. Technologies for Verification Testing framework. We will conclude by summarizing remaining work and future directions for the project.

Extended Reality↗

NASA NTRS2013

Regression Verification Using Impact Summaries

Regression verification techniques are used to prove equivalence of syntactically similar programs. Checking equivalence of large programs, however, can be computationally expensive. Existing regression verification techniques rely on abstraction and decomposition techniques to reduce the computational effort of checking equivalence of the entire program. These techniques are sound but not complete. In this work, we propose a novel approach to improve scalability of regression verification by classifying the program behaviors generated during symbolic execution as either impacted or unimpacted. Our technique uses a combination of static analysis and symbolic execution to generate summaries of impacted program behaviors. The impact summaries are then checked for equivalence using an o-the-shelf decision procedure. We prove that our approach is both sound and complete for sequential programs, with respect to the depth bound of symbolic execution. Our evaluation on a set of sequential C artifacts shows that reducing the size of the summaries can help reduce the cost of software equivalence checking. Various reduction, abstraction, and compositional techniques have been developed to help scale software verification techniques to industrial-sized systems. Although such techniques have greatly increased the size and complexity of systems that can be checked, analysis of large software systems remains costly. Regression analysis techniques, e.g., regression testing [16], regression model checking [22], and regression verification [19], restrict the scope of the analysis by leveraging the differences between program versions. These techniques are based on the idea that if code is checked early in development, then subsequent versions can be checked against a prior (checked) version, leveraging the results of the previous analysis to reduce analysis cost of the current version. Regression verification addresses the problem of proving equivalence of closely related program versions [19]. These techniques compare two programs with a large degree of syntactic similarity to prove that portions of one program version are equivalent to the other. Regression verification can be used for guaranteeing backward compatibility, and for showing behavioral equivalence in programs with syntactic differences, e.g., when a program is refactored to improve its performance, maintainability, or readability. Existing regression verification techniques leverage similarities between program versions by using abstraction and decomposition techniques to improve scalability of the analysis [10, 12, 19]. The abstractions and decomposition in the these techniques, e.g., summaries of unchanged code [12] or semantically equivalent methods [19], compute an over-approximation of the program behaviors. The equivalence checking results of these techniques are sound but not complete-they may characterize programs as not functionally equivalent when, in fact, they are equivalent. In this work we describe a novel approach that leverages the impact of the differences between two programs for scaling regression verification. We partition program behaviors of each version into (a) behaviors impacted by the changes and (b) behaviors not impacted (unimpacted) by the changes. Only the impacted program behaviors are used during equivalence checking. We then prove that checking equivalence of the impacted program behaviors is equivalent to checking equivalence of all program behaviors for a given depth bound. In this work we use symbolic execution to generate the program behaviors and leverage control- and data-dependence information to facilitate the partitioning of program behaviors. The impacted program behaviors are termed as impact summaries. The dependence analyses that facilitate the generation of the impact summaries, we believe, could be used in conjunction with other abstraction and decomposition based approaches, [10, 12], as a complementary reduction technique. An evaluation of our regression verification technique shows that our approach is capable of leveraging similarities between program versions to reduce the size of the queries and the time required to check for logical equivalence. The main contributions of this work are: - A regression verification technique to generate impact summaries that can be checked for functional equivalence using an off-the-shelf decision procedure. - A proof that our approach is sound and complete with respect to the depth bound of symbolic execution. - An implementation of our technique using the LLVMcompiler infrastructure, the klee Symbolic Virtual Machine [4], and a variety of Satisfiability Modulo Theory (SMT) solvers, e.g., STP [7] and Z3 [6]. - An empirical evaluation on a set of C artifacts which shows that the use of impact summaries can reduce the cost of regression verification.

Backes, John↗

NASA NTRSDate not supplied

Guiding Integration of Formal Verification in Assurance Cases

Assurance cases are being increasingly acknowledged as away to build trust in complex systems with autonomous capabilities. An assurance case is a comprehensive, defensible, and valid justification that a system will function as intended for a specific mission and operating environment. Formal verification is often reserved for the most critical components of such systems. However, formal verification tools are often complex, and their usage is subject to many constraints and contextual dependencies. This can raise challenges both for performing the verification as well as reflecting the verification results appropriately in the assurance case, especially for non-expert users of the verification tool. To address these challenges, we present a tool-supported methodology for integrating formal verification results in an assurance case by capturing key verification method information in a rigorously constructed assurance case. In particular, we capture the tool specification in terms of its inputs, outputs, and assurance constraints as assumptions over inputs and guarantees provided over its outputs. The tool specification is parametrized over the inputs and outputs to both guide the intended application of the tool, as well as to check that the tool has been applied following the stated assumptions and that the guarantees hold. We define a generic tool assurance argument pattern that enables integration of the verification results in the assurance case by allowing custom refinement and automated instantiation for each tool use. We demonstrate our methodology on two formal verification tools and their applications to the verification of neural network properties for the aircraft domain.

Assurance Cases↗

NASA NTRS2005

Structural Verification of the Redesigned Space Shuttle Bipod Foam Closeout

This document outlines the structural verification approach for the Space Shuttle External Tank Forward Bipod Foam Closeout. Due to the Space Shuttle Columbia accident, debris has become a major concern. The intent of the structural verification is to ensure that any debris shed from the bipod is within acceptable limits. Since cohesive failure due to internal defects was identified as the most likely cause of the STS-107 bipod ramp foam failure, verification for this failure mode receives particular emphasis. However, all failure modes for TPS are considered and appropriate verification rationale is developed for each failure mode. Figure 1 depicts the structural verification of a production design where analysis and test are the primary methods of verification. It can be seen that the successful completion of structural verification is dependent on three main areas: 1. Production process control and quality assurance must ensure that test articles and/or analytical models are representative of (or conservatively envelope) production hardware in terms of geometry, materials and processing. Variability and defects must be considered. 2. Flight environments must be sufficiently characterized to bound driving environments for all failure modes. Applied environments, either test or analytical, must be representative of flight environments and have a load factor that satisfies design requirements. 3. Structural verification must include all failure modes. A comprehensive list of failure modes and the underlying failure mechanisms has been generated based on flight and test experience. Verification tests and / or analyses must address each failure mode. ET TPS Verification is accomplished by a combination of analysis, test, and similarity.

Poole, Eric L.↗

NASA NTRS2015

The SeaHorn Verification Framework

In this paper, we present SeaHorn, a software verification framework. The key distinguishing feature of SeaHorn is its modular design that separates the concerns of the syntax of the programming language, its operational semantics, and the verification semantics. SeaHorn encompasses several novelties: it (a) encodes verification conditions using an efficient yet precise inter-procedural technique, (b) provides flexibility in the verification semantics to allow different levels of precision, (c) leverages the state-of-the-art in software model checking and abstract interpretation for verification, and (d) uses Horn-clauses as an intermediate language to represent verification conditions which simplifies interfacing with multiple verification tools based on Horn-clauses. SeaHorn provides users with a powerful verification tool and researchers with an extensible and customizable framework for experimenting with new software verification techniques. The effectiveness and scalability of SeaHorn are demonstrated by an extensive experimental evaluation using benchmarks from SV-COMP 2015 and real avionics code.

Model Checking↗

NASA NTRSDate not supplied

An Analysis of Extended Reality Mockups for Use in Verification: Phase 2

NASA commercial providers are increasing their use of Virtual Reality (VR) and Hybrid Reality (HR) technologies in their system development process. The use of these technologies, collectively referred to as Extended Reality (XR) technologies, has been limited to the development phase, but there have been requests to integrate VR and HR mockups into verifications. Verifications require, at minimum, a high-fidelity mockup for activities involving human participation (tests, demonstrations, inspections). HR technology shares many of the strengths but may not share some of the critical shortcomings of VR. Specifically, HR allows for integrating physical objects, such as a suit, which may be critical to evaluating a system. Adopting HR technology still has many of the same challenges as VR. Namely, there is little to no data about the validity of evaluations conducted with HR mockups, an established process, or criteria for evaluating the appropriateness of HR mockups. Because verifications are final and only need to happen once, HR mockups must be adequately vetted before being approved for use in verifications. Further hindering the adoption of these technologies is the lack of consensus on what constitutes a high-fidelity XR mockup. Even for physical mockups, there are guidelines and common criteria, but nothing has been formalized. Discussion about when an XR mockup can be used would be greatly helped by clearly defining what a high-fidelity XR mockup is and how it might be measured. The team will engage with relevant stakeholders to learn more about the benefits and challenges of adopting hybrid reality mockups for verification. The team will build upon work from Phase 1 by maturing a framework to guide decisions for evaluating XR mockups for use in verifications. The team will also conduct experimental studies to evaluate advantages and tradeoffs of hybrid/mixed reality relative to physical and virtual reality mockups. Finally, data from Phase 1 and Phase 2 will be synthesized into a set of best practices for developing and validating XR mockups for verification. We will highlight the work conducted to date in Phase 2. We will present the status of the XR for Verification Framework, a summary of best practices identified thus far, and give a synopsis of future work.

verification testing↗

NASA NTRS1992

HDL to verification logic translator

The increasingly higher number of transistors possible in VLSI circuits compounds the difficulty in insuring correct designs. As the number of possible test cases required to exhaustively simulate a circuit design explodes, a better method is required to confirm the absence of design faults. Formal verification methods provide a way to prove, using logic, that a circuit structure correctly implements its specification. Before verification is accepted by VLSI design engineers, the stand alone verification tools that are in use in the research community must be integrated with the CAD tools used by the designers. One problem facing the acceptance of formal verification into circuit design methodology is that the structural circuit descriptions used by the designers are not appropriate for verification work and those required for verification lack some of the features needed for design. We offer a solution to this dilemma: an automatic translation from the designers' HDL models into definitions for the higher-ordered logic (HOL) verification system. The translated definitions become the low level basis of circuit verification which in turn increases the designer's confidence in the correctness of higher level behavioral models.

Gambles, J. W.↗

NASA NTRS2011

Requirement Assurance: A Verification Process

Requirement Assurance is an act of requirement verification which assures the stakeholder or customer that a product requirement has produced its "as realized product" and has been verified with conclusive evidence. Product requirement verification answers the question, "did the product meet the stated specification, performance, or design documentation?". In order to ensure the system was built correctly, the practicing system engineer must verify each product requirement using verification methods of inspection, analysis, demonstration, or test. The products of these methods are the "verification artifacts" or "closure artifacts" which are the objective evidence needed to prove the product requirements meet the verification success criteria. Institutional direction is given to the System Engineer in NPR 7123.1A NASA Systems Engineering Processes and Requirements with regards to the requirement verification process. In response, the verification methodology offered in this report meets both the institutional process and requirement verification best practices.

Alexander, Michael G.↗