Verification and Validation Testing of the FLEXO Extended-MHD Code
Explore the source record for details and available documents.
Bond, Stephen David↗
SEARCH · Engineering Papers
Results for “Verification and validation”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
STAT7 v2.0 Verification and Validation Report
This report documents the software testing which has been performed for STAT7 Version 2.0 (hereafter STAT7 v2.0), a code for the statistical propagation of uncertainties in steady-state thermal hydraulics (TH) analysis of plate-fueled research and test reactors. The key capabilities addressed in this testing were selected based on the use of the software for analysis work supporting the conversion to low-enriched uranium fuel of U.S. High-Performance Research Reactors (USHPRR) such as the Massachusetts Institute of Technology Research Reactor (MITR-II). This testing scope includes comparison of the code’s statistical processing capabilities using statistical tests, comparison of coolant property evaluations to NIST data tables, and systematic comparison of key TH solver capabilities to the results of hand calculations and to predictions from PLTEMP/ANL Version 4.4. The
22 GENERAL STUDIES OF NUCLEAR REACTORS↗
Verification and Validation of Performance with Dissemination of Best Practices in District Energy and CHP for Enhanced Resiliency, Energy Efficiency, and Cybersecurity
This report contains the results of the International District Energy Association’s work to analyze, validate, and verify performance data of existing district energy systems and identify industy best practices for the purpose of improving system reliability, resiliency, and efficiency, and to accellerate decarbonization. In addition to a technical evaluation of the surveyed systems and identification of a series of technical performance metrics, the report illustrates the accompanying operations and financial best practices employed by surveyed systems to fully serve their customer base. Additionally, the third chapter of the report describes the current landscape of cybersecurity threats and counteracting measures, and recommends a series of steps for effectively guarding highly networked district energy systems against cybersecurity attacks.
96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
Verification and Validation of Flow Confinement Modeling using OpenFAST Source Panel Method
Blockage or flow confinement effects in scale wind and water turbine testing in experimental facilities, including wind and water tunnels and tow tanks, can significantly alter incident aerodynamic and hydrodynamic loads, turbine performance (thrust, torque, power output), and wake dynamics compared to unconfined conditions. As scale model tests are key sources of OpenFAST model validation measurements, directly modeling flow confinement and its effects in OpenFAST enables improved model validat
Kim, Dongyoung [Sandia National Laboratories (SNL-↗
Verification and validation of the NASA Terminal Configured Vehicle's /TCV/ Wind Analysis program using real-time digital simulation
NASA's Terminal Configured Vehicle (TCV) program has been established to develop and evaluate aircraft and flight management technology that will benefit conventional takeoff and landing operations in the terminal area. To this end, a process whereby estimates of longitudinal, lateral, and vertical winds are obtained using the present complement of sensors onboard the TCV aircraft (B737-100) is developed. The results of the simulation experiment to verify and validate the TCV B737 Data Merging and Wind Analysis program have shown that the wind analysis algorithms are correct and that very satisfactory estimates of all three wind components are obtained even in the presence of adverse atmospheric conditions and realistic measurement noise. In realistic worst case conditions, wind component errors of the order of + or - 2.0 knots longitudinally, + or - 0.5 knot laterally, and + or - 1.0 knot vertically are to be expected.
Dieudonne, J. E.↗
A plan for validation and verification of Apollo flight computer software
Explore the source record for details and available documents.
Engel, J. S.↗
Validation and verification of expert systems using evidence flow graphs
This paper describes an ongoing investigation into the use of evidence flow graph techniques for performing V&V of expert systems. This method involves translating a rule-base into an evidence flow graph, a representation originally developed for real-time intelligent systems in distributed environments, and then running simulations of the evidence flow graph. Certain errors can be found during the translation process. The simulations can detect output sensitivity to rule firing order, to order of presentation of inputs, and to small changes in input values.
Becker, Lee A.↗
Expert system verification and validation study. Delivery 1: Survey and interview questions
The NASA funded questionnaire is presented to help define the state-of-the-practice in the formal evaluation of Expert Systems on current NASA and industry applications. The answers to this questionnaire, together with follow-up interviews, will provide realistic answers to the following questions: (1) How much evaluation is being performed; (2) What evaluation techniques are in use; and (3) What, if any, are the unique issues in evaluating Expert Systems.
Source record↗
Expert system verification and validation guidelines/workshop task. Deliverable no. 1: ES V/V guidelines
The goals are to show that verifying and validating a software system is a required part of software development and has a direct impact on the software's design and structure. Workshop tasks are given in the areas of statistics, integration/system test, unit and architectural testing, and a traffic controller problem.
French, Scott W.↗
Expert system verification and validation study: Workshop and presentation material
Workshop and presentation material are included. Following an introduction, the basic concepts, techniques, and guidelines are discussed. Handouts and worksheets are included.
French, Scott W.↗
Software risk management through independent verification and validation
Software project managers need tools to estimate and track project goals in a continuous fashion before, during, and after development of a system. In addition, they need an ability to compare the current project status with past project profiles to validate management intuition, identify problems, and then direct appropriate resources to the sources of problems. This paper describes a measurement-based approach to calculating the risk inherent in meeting project goals that leverages past project metrics and existing estimation and tracking models. We introduce the IV&V Goal/Questions/Metrics model, explain its use in the software development life cycle, and describe our attempts to validate the model through the reverse engineering of existing projects.
Callahan, John R.↗
An approach to verification and validation of a reliable multicasting protocol: Extended Abstract
This paper describes the process of implementing a complex communications protocol that provides reliable delivery of data in multicast-capable, packet-switching telecommunication networks. The protocol, called the Reliable Multicasting Protocol (RMP), was developed incrementally using a combination of formal and informal techniques in an attempt to ensure the correctness of its implementation. Our development process involved three concurrent activities: (1) the initial construction and incremental enhancement of a formal state model of the protocol machine; (2) the initial coding and incremental enhancement of the implementation; and (3) model-based testing of iterative implementations of the protocol. These activities were carried out by two separate teams: a design team and a V&V team. The design team built the first version of RMP with limited functionality to handle only nominal requirements of data delivery. This initial version did not handle off-nominal cases such as network partitions or site failures. Meanwhile, the V&V team concurrently developed a formal model of the requirements using a variant of SCR-based state tables. Based on these requirements tables, the V&V team developed test cases to exercise the implementation. In a series of iterative steps, the design team added new functionality to the implementation while the V&V team kept the state model in fidelity with the implementation. This was done by generating test cases based on suspected errant or off-nominal behaviors predicted by the current model. If the execution of a test in the model and implementation agreed, then the test either found a potential problem or verified a required behavior. However, if the execution of a test was different in the model and implementation, then the differences helped identify inconsistencies between the model and implementation. In either case, the dialogue between both teams drove the co-evolution of the model and implementation. We have found that this interactive, iterative approach to development allows software designers to focus on delivery of nominal functionality while the V&V team can focus on analysis of off nominal cases. Testing serves as the vehicle for keeping the model and implementation in fidelity with each other. This paper describes (1) our experiences in developing our process model; and (2) three example problems found during the development of RMP. Although RMP has provided our research effort with a rich set of test cases, it also has practical applications within NASA. For example, RMP is being considered for use in the NASA EOSDIS project due to its significant performance benefits in applications that need to replicate large amounts of data to many network sites.
Callahan, John R.↗
Verification and validation of a reliable multicast protocol
This paper describes the methods used to specify and implement a complex communications protocol that provides reliable delivery of data in multicast-capable, packet-switching telecommunication networks. The protocol, called the Reliable Multicasting Protocol (RMP), was developed incrementally by two complementary teams using a combination of formal and informal techniques in an attempt to ensure the correctness of the protocol implementation. The first team, called the Design team, initially specified protocol requirements using a variant of SCR requirements tables and implemented a prototype solution. The second team, called the V&V team, developed a state model based on the requirements tables and derived test cases from these tables to exercise the implementation. In a series of iterative steps, the Design team added new functionality to the implementation while the V&V team kept the state model in fidelity with the implementation through testing. Test cases derived from state transition paths in the formal model formed the dialogue between teams during development and served as the vehicles for keeping the model and implementation in fidelity with each other. This paper describes our experiences in developing our process model, details of our approach, and some example problems found during the development of RMP.
Callahan, John R.↗
An approach to verification and validation of a reliable multicasting protocol
This paper describes the process of implementing a complex communications protocol that provides reliable delivery of data in multicast-capable, packet-switching telecommunication networks. The protocol, called the Reliable Multicasting Protocol (RMP), was developed incrementally using a combination of formal and informal techniques in an attempt to ensure the correctness of its implementation. Our development process involved three concurrent activities: (1) the initial construction and incremental enhancement of a formal state model of the protocol machine; (2) the initial coding and incremental enhancement of the implementation; and (3) model-based testing of iterative implementations of the protocol. These activities were carried out by two separate teams: a design team and a V&V team. The design team built the first version of RMP with limited functionality to handle only nominal requirements of data delivery. In a series of iterative steps, the design team added new functionality to the implementation while the V&V team kept the state model in fidelity with the implementation. This was done by generating test cases based on suspected errant or off-nominal behaviors predicted by the current model. If the execution of a test was different between the model and implementation, then the differences helped identify inconsistencies between the model and implementation. The dialogue between both teams drove the co-evolution of the model and implementation. Testing served as the vehicle for keeping the model and implementation in fidelity with each other. This paper describes (1) our experiences in developing our process model; and (2) three example problems found during the development of RMP.
Callahan, John R.↗
An Approach to Verification and Validation of a Reliable Multicasting Protocol
This paper describes the process of implementing a complex communications protocol that provides reliable delivery of data in multicast-capable, packet-switching telecommunication networks. The protocol, called the Reliable Multicasting Protocol (RMP), was developed incrementally using a combination of formal and informal techniques in an attempt to ensure the correctness of its implementation. Our development process involved three concurrent activities: (1) the initial construction and incremental enhancement of a formal state model of the protocol machine; (2) the initial coding and incremental enhancement of the implementation; and (3) model-based testing of iterative implementations of the protocol. These activities were carried out by two separate teams: a design team and a V&V team. The design team built the first version of RMP with limited functionality to handle only nominal requirements of data delivery. In a series of iterative steps, the design team added new functionality to the implementation while the V&V team kept the state model in fidelity with the implementation. This was done by generating test cases based on suspected errant or offnominal behaviors predicted by the current model. If the execution of a test was different between the model and implementation, then the differences helped identify inconsistencies between the model and implementation. The dialogue between both teams drove the co-evolution of the model and implementation. Testing served as the vehicle for keeping the model and implementation in fidelity with each other. This paper describes (1) our experiences in developing our process model; and (2) three example problems found during the development of RMP.
Callahan, John R.↗
Quantitative Measures for Software Independent Verification and Validation
As software is maintained or reused, it undergoes an evolution which tends to increase the overall complexity of the code. To understand the effects of this, we brought in statistics experts and leading researchers in software complexity, reliability, and their interrelationships. These experts' project has resulted in our ability to statistically correlate specific code complexity attributes, in orthogonal domains, to errors found over time in the HAL/S flight software which flies in the Space Shuttle. Although only a prototype-tools experiment, the result of this research appears to be extendable to all other NASA software, given appropriate data similar to that logged for the Shuttle onboard software. Our research has demonstrated that a more complete domain coverage can be mathematically demonstrated with the approach we have applied, thereby ensuring full insight into the cause-and-effects relationship between the complexity of a software system and the fault density of that system. By applying the operational profile we can characterize the dynamic effects of software path complexity under this same approach We now have the ability to measure specific attributes which have been statistically demonstrated to correlate to increased error probability, and to know which actions to take, for each complexity domain. Shuttle software verifiers can now monitor the changes in the software complexity, assess the added or decreased risk of software faults in modified code, and determine necessary corrections. The reports, tool documentation, user's guides, and new approach that have resulted from this research effort represent advances in the state of the art of software quality and reliability assurance. Details describing how to apply this technique to other NASA code are contained in this document.
Lee, Alice↗
Knowledge-Based Aircraft Automation: Managers Guide on the use of Artificial Intelligence for Aircraft Automation and Verification and Validation Approach for a Neural-Based Flight Controller
The ultimate goal of this report was to integrate the powerful tools of artificial intelligence into the traditional process of software development. To maintain the US aerospace competitive advantage, traditional aerospace and software engineers need to more easily incorporate the technology of artificial intelligence into the advanced aerospace systems being designed today. The future goal was to transition artificial intelligence from an emerging technology to a standard technology that is considered early in the life cycle process to develop state-of-the-art aircraft automation systems. This report addressed the future goal in two ways. First, it provided a matrix that identified typical aircraft automation applications conducive to various artificial intelligence methods. The purpose of this matrix was to provide top-level guidance to managers contemplating the possible use of artificial intelligence in the development of aircraft automation. Second, the report provided a methodology to formally evaluate neural networks as part of the traditional process of software development. The matrix was developed by organizing the discipline of artificial intelligence into the following six methods: logical, object representation-based, distributed, uncertainty management, temporal and neurocomputing. Next, a study of existing aircraft automation applications that have been conducive to artificial intelligence implementation resulted in the following five categories: pilot-vehicle interface, system status and diagnosis, situation assessment, automatic flight planning, and aircraft flight control. The resulting matrix provided management guidance to understand artificial intelligence as it applied to aircraft automation. The approach taken to develop a methodology to formally evaluate neural networks as part of the software engineering life cycle was to start with the existing software quality assurance standards and to change these standards to include neural network development. The changes were to include evaluation tools that can be applied to neural networks at each phase of the software engineering life cycle. The result was a formal evaluation approach to increase the product quality of systems that use neural networks for their implementation.
Broderick, Ron↗
Verification and Validation in a Rapid Software Development Process
The high cost of software production is driving development organizations to adopt more automated design and analysis methods such as rapid prototyping, computer-aided software engineering (CASE) tools, and high-level code generators. Even developers of safety-critical software system have adopted many of these new methods while striving to achieve high levels Of quality and reliability. While these new methods may enhance productivity and quality in many cases, we examine some of the risks involved in the use of new methods in safety-critical contexts. We examine a case study involving the use of a CASE tool that automatically generates code from high-level system designs. We show that while high-level testing on the system structure is highly desirable, significant risks exist in the automatically generated code and in re-validating releases of the generated code after subsequent design changes. We identify these risks and suggest process improvements that retain the advantages of rapid, automated development methods within the quality and reliability contexts of safety-critical projects.
Callahan, John R.↗