Engineering PapersSearch

SEARCH · Engineering Papers

Results for “software validation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records

Software Validation via Model Animation

This paper explores a new approach to validating software implementations that have been produced from formally-verified algorithms. Although visual inspection gives some confidence that the implementations faithfully reflect the formal models, it does not provide complete assurance that the software is correct. The proposed approach, which is based on animation of formal specifications, compares the outputs computed by the software implementations on a given suite of input values to the outputs computed by the formal models on the same inputs, and determines if they are equal up to a given tolerance. The approach is illustrated on a prototype air traffic management system that computes simple kinematic trajectories for aircraft. Proofs for the mathematical models of the system's algorithms are carried out in the Prototype Verification System (PVS). The animation tool PVSio is used to evaluate the formal models on a set of randomly generated test cases. Output values computed by PVSio are compared against output values computed by the actual software. This comparison improves the assurance that the translation from formal models to code is faithful and that, for example, floating point errors do not greatly affect correctness and safety properties.

Dutle, Aaron M.

PIV Data Validation Software Package

A PIV data validation and post-processing software package was developed to provide semi-automated data validation and data reduction capabilities for Particle Image Velocimetry data sets. The software provides three primary capabilities including (1) removal of spurious vector data, (2) filtering, smoothing, and interpolating of PIV data, and (3) calculations of out-of-plane vorticity, ensemble statistics, and turbulence statistics information. The software runs on an IBM PC/AT host computer working either under Microsoft Windows 3.1 or Windows 95 operating systems.

Blackshire, James L.

A methodology for validating software reliability

A significant problem associated with fault tolerant computer system design is how to insure that there are no embedded software errors, so that an avionics computer system meets the required reliability level. To accomplish this, it is necessary to associate a 'probability of failure' with the operational flight program. It would be more correct to say that the probability of excitation of existing latent design errors within the program is required. In this sense, latent software errors are like latent hardware faults, and techniques that were previously used to measure the probability of failure of hardware due to fault latency can be used to measure the probability of failure of the software. A methodology was developed and applied to a flight control program that was known to operate in a well defined environment. The results indicated that the technique could be used to provide a final validation of the software to a specified reliability level and to evaluate the role of flight test in software validation.

Swern, Frederic L.

Sensor Validation Software

Under a Small Business Innovation Research contract from Lewis Research Center, Expert Microsystems, Inc. developed SureSense, real-time sensor data validation software. This ultra-reliable control and sensing system product was produced through a partnership in 1994 between Expert Microsystems and Intelligent Software Associates, Inc. SureSense was created in response to a NASA need for verifying the reliability of sensor input that operated advanced automation and control systems. The immediate applications included improving the safety and reliability of Space Shuttle Main Engine operations. The company has structured the software to enable application to virtually any process control environment, such as computer integrated manufacturing, power plants, and hazardous gas sensing and control systems.

Source record

RotCFD Software Validation - Computational and Experimental Data Comparison

RotCFD is a software intended to ease the design of NextGen rotorcraft. Since RotCFD is a new software still in the development process, the results need to be validated to determine the software's accuracy. The purpose of the present document is to explain one of the approaches to accomplish that goal.

Experimental Data Comparison

Designing a configuration control mechanism for a flight software validation facility

Embedded computer systems and supporting operating systems have recently enabled the automation of many time-critical flight functions. This automation requires the integration of computer hardware and software, aircraft hardware, and the flight application. To properly integrate and control these critical functions requires a real-time environment that is provided by the onboard flight computer, a real-time operating system, and a real-time flight application that supplies the input commands to the aircraft and responds to the resulting actions. The increased complexity of developing and testing such real-time applications can best be supported by a separate test facility that provides the processing visibility and control which is virtually impossible in the actual real-time operating environment. Such a flight software validation facility has been developed at the NASA Langley Research Center to support applications using the Shuttle real-time language HAL/S. The development and control of this facility are described. Some of the management techniques used were project schedules, status monitoring, code walk-throughs, configuration control, phased releases, validation suites, and modern programming techniques.

Smith-Taylor, R.

Software Validation Work With The ZPPR-15 Data

The analysis activities for fast reactors involve using many different pieces of software that are relied upon for their predictive capabilities. For this software to be considered reliable, documented proof that the predictions of the software are accurate is required. In this manuscript, the validation work that covers some of the Argonne software used in fast reactor design activities is discussed and displayed. This validation work includes neutron and gamma flux distributions, reaction rate distributions, and reactivity worth. In an ideal world, a reactor development program would have access to a comprehensive set of experimental facilities to help inform the design aspects of the reactor itself. While thermal-hydraulics experiments, and to a limited degree mechanical experiments, can be carried out today for validation needs, neutronics related experimental facilities are rather impractical because of the lack of experimental facilities. Given the desired time table for construction of new reactors, the reconstitution or creation of new neutronic experimental facilities is untenable and thus those reactor development programs must rely upon any available experimental measurements that are qualitatively similar to the design. While a methodology has been proposed to assess the similarity between the past experimental measurements and the reactor itself, that aspect is beyond the scope of this manuscript. In this manuscript, the focus is entirely placed on the analysis results for a series of experiments carried out at the ZPPR facility in Idaho in the mid-1980s. In this regard, this manuscript only shows the validation of the stated neutronics software for specific loadings of the ZPPR reactor. Because of the fuel form, its proposed enrichment, and the material content of the reactor core, the ZPPR-15 experiments were identified as potential validation data for the reactor. The ZPPR-15 experiments were intended as mockups of a 330 MWe Integral Fast Reactor program which was a follow on program to the Clinch River Breeder Reactor. In the ZPPR-15 series of experiments, measurements of the neutron spectrum, control rod worth, sodium void worth, foil reaction rate distributions, Doppler worth of heated samples, gamma dose, and axial expansion worth were all carried out and published. In many cases, these reactivity coefficients are good candidates to validate the reactivity coefficient calculation scheme used by the analysis software and included in the safety analysis activities of fast reactor development projects today. This manuscript discusses the modeling methodology and accuracy of the calculated experimental results using the LANL software MCNP and the ANL software package ARC (Argonne Reactor Codes). As will be shown, for many of the experimental measurements, the two software packages are found to be good predictive analysis tools for those experiments. In other cases, problems with the analysis methodology or underlying cross section data are exposed which indicates where predictive analysis is not as reliable. Finally, in some of the measurements the conclusion is reached that the experimental measurement cannot be reproduced with the analysis software as it is simply too difficult.

22 GENERAL STUDIES OF NUCLEAR REACTORS

Software Validation Work With The ZPPR-15 Data

The analysis activities for fast reactors involve using many different pieces of software that are relied upon for their predictive capabilities. For this software to be considered reliable, documented proof that the predictions of the software are accurate is required. In this manuscript, the validation work that covers some of the Argonne software used in fast reactor design activities is discussed and displayed. This validation work includes neutron and gamma flux distributions, reaction rate distributions, and reactivity worth. In an ideal world, a reactor development program would have access to a comprehensive set of experimental facilities to help inform the design aspects of the reactor itself. While thermal-hydraulics experiments, and to a limited degree mechanical experiments, can be carried out today for validation needs, neutronics related experimental facilities are rather impractical because of the lack of experimental facilities. Given the desired time table for construction of new reactors, the reconstitution or creation of new neutronic experimental facilities is untenable and thus those reactor development programs must rely upon any available experimental measurements that are qualitatively similar to the design. While a methodology has been proposed to assess the similarity between the past experimental measurements and the reactor itself, that aspect is beyond the scope of this manuscript. In this manuscript, the focus is entirely placed on the analysis results for a series of experiments carried out at the ZPPR facility in Idaho in the mid-1980s. In this regard, this manuscript only shows the validation of the stated neutronics software for specific loadings of the ZPPR reactor. Because of the fuel form, its proposed enrichment, and the material content of the reactor core, the ZPPR-15 experiments were identified as potential validation data for the reactor. The ZPPR-15 experiments were intended as mockups of a 330 MWe Integral Fast Reactor program which was a follow on program to the Clinch River Breeder Reactor. In the ZPPR-15 series of experiments, measurements of the neutron spectrum, control rod worth, sodium void worth, foil reaction rate distributions, Doppler worth of heated samples, gamma dose, and axial expansion worth were all carried out and published. In many cases, these reactivity coefficients are good candidates to validate the reactivity coefficient calculation scheme used by the analysis software and included in the safety analysis activities of fast reactor development projects today. This manuscript discusses the modeling methodology and accuracy of the calculated experimental results using the LANL software MCNP and the ANL software package ARC (Argonne Reactor Codes). As will be shown, for many of the experimental measurements, the two software packages are found to be good predictive analysis tools for those experiments. In other cases, problems with the analysis methodology or underlying cross section data are exposed which indicates where predictive analysis is not as reliable. Finally, in some of the measurements the conclusion is reached that the experimental measurement cannot be reproduced with the analysis software as it is simply too difficult.

Aliberti, Gerardo

A methodology for collecting valid software engineering data

An effective data collection method for evaluating software development methodologies and for studying the software development process is described. The method uses goal-directed data collection to evaluate methodologies with respect to the claims made for them. Such claims are used as a basis for defining the goals of the data collection, establishing a list of questions of interest to be answered by data analysis, defining a set of data categorization schemes, and designing a data collection form. The data to be collected are based on the changes made to the software during development, and are obtained when the changes are made. To insure accuracy of the data, validation is performed concurrently with software development and data collection. Validation is based on interviews with those people supplying the data. Results from using the methodology show that data validation is a necessary part of change data collection. Without it, as much as 50% of the data may be erroneous. Feasibility of the data collection methodology was demonstrated by applying it to five different projects in two different environments. The application showed that the methodology was both feasible and useful.

Basili, Victor R.

Production of Reliable Flight Crucial Software: Validation Methods Research for Fault Tolerant Avionics and Control Systems Sub-Working Group Meeting

The state of the art in the production of crucial software for flight control applications was addressed. The association between reliability metrics and software is considered. Thirteen software development projects are discussed. A short term need for research in the areas of tool development and software fault tolerance was indicated. For the long term, research in format verification or proof methods was recommended. Formal specification and software reliability modeling, were recommended as topics for both short and long term research.

Dunham, J. R.

System Level Analysis Software Validation with Argonne's THETA Experimental Facility (Final CRADA Report)

Under this CRADA, the Contractor will be generating and collecting sodium-based experimental data with modern instrumentation. The Contractor will work symbiotically with the Participant to generate SAS4A/SASSYS-1 and SAM computational models of the THETA experimental facility for validation of the system-level analysis codes utilized by Oklo, Inc. This effort will be specifically focused on the design basis event space scoped by Chapter 15 of the NRC’s Standard Review Plan (NUREG-0800) with an emphasis on natural circulation and thermal stratification phenomena in a prototypic sodium-cooled fast reactor (SFR).

22 GENERAL STUDIES OF NUCLEAR REACTORS

Oklo Sponsored Testing using PELICAN: System, Subchannel, and High-Fidelity Software Validation (Final CRADA Report)

Argonne National Laboratory (the Contractor), located in Lemont IL, and Oklo, Inc., (the Participant), headquartered in Santa Clara, CA, propose to enter into a Cooperative Research and Development Agreement (CRADA) to perform a gap analysis of thermal hydraulic data, perform prototypical fuel assembly pressure drop and cavitation model validation, generate the experimental data as well as the corresponding uncertainties for this matrix and, finally, develop the validation models with the Argonne system level code SAS4A/SASSYS-1, the Argonne subchannel analysis code DASSH, the Argonne high fidelity code Nek5000, and/or the Idaho National Laboratory code Pronghorn Subchannel. The work outlined below will significantly improve the experimental and validation database currently available for liquid metal fast reactors, thus making it a viable part of a comprehensive reactor design and licensing suite to be used by the participant.

22 GENERAL STUDIES OF NUCLEAR REACTORS

Precision orbit determination software validation experiment

This paper presents the results of an experiment which was designed to ascertain the level of agreement between GEODYN and UTOPIA, two completely independent computer programs used for precision orbit determination, and to identify the sources which limit the agreement. For a limited set of models and a seven-day data set arc length, the altitude components of the ephemeris obtained by the two programs agree at the sub-centimeter level throughout the arc.

Schutz, B. E.

A methodology for collecting valid software engineering data

An effective data collection method for evaluating software development methodologies and for studying the software development process is described. The method uses goal-directed data collection to evaluate methodologies with respect to the claims made for them. Such claims are used as a basis for defining the goals of the data collection, establishing a list of questions of interest to be answered by data analysis, defining a set of data categorization schemes, and designing a data collection form. Feasibility of the data collection methodology was demonstrated by applying it to five different projects in two different environments (other NRL Reports). The application showed that the methodology was both feasible and useful.

Basili, V. R.

Composing, Analyzing and Validating Software Models

This research has been conducted at the Computational Sciences Division of the Information Sciences Directorate at Ames Research Center (Automated Software Engineering Grp). The principle work this summer has been to review and refine the agenda that were carried forward from last summer. Formal specifications provide good support for designing a functionally correct system, however they are weak at incorporating non-functional performance requirements (like reliability). Techniques which utilize stochastic Petri nets (SPNs) are good for evaluating the performance and reliability for a system, but they may be too abstract and cumbersome from the stand point of specifying and evaluating functional behavior. Therefore, one major objective of this research is to provide an integrated approach to assist the user in specifying both functionality (qualitative: mutual exclusion and synchronization) and performance requirements (quantitative: reliability and execution deadlines). In this way, the merits of a powerful modeling technique for performability analysis (using SPNs) can be combined with a well-defined formal specification language. In doing so, we can come closer to providing a formal approach to designing a functionally correct system that meets reliability and performance goals.

Sheldon, Frederick T.

TCV software test and validation tools and technique

The paper describes techniques for testing and validating software for the TCV (Terminal Configured Vehicle) program which is intended to solve problems associated with operating a commercial transport aircraft in the terminal area. The TCV research test bed is a Boeing 737 specially configured with digital computer systems to carry out automatic navigation, guidance, flight controls, and electronic displays research. The techniques developed for time and cost reduction include automatic documentation aids, an automatic software configuration, and an all software generation and validation system.

Straeter, T. A.