Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “software verification and validation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5
All sourcesNASA NTRSDOE OSTI
NASA NTRS2003

Optimizing IV and V for Mature Organizations

NASA is intending for its future software development agencies to have at least a Level 3 rating in the Carnegie Mellon University Capability Maturity Model (CMM). The CMM has built-in Verification and Validation (V&V) processes that support higher software quality. Independent Verification and Validation (IV&V) of software developed by mature agencies can be therefore more effective than for software developed by less mature organizations. How is Independent V&V different with respect to the maturity of an organization? Knowing a priori the maturity of an organization's processes, how can IV&V planners better identify areas of need choose IV&V activities, etc? The objective of this research is to provide a complementary set of guidelines and criteria to assist the planning of IV&V activities on a project using a priori knowledge of the measurable levels of maturity of the organization developing the software.

Fuhman, Christopher↗
NASA NTRS1990

Software safety - A user's practical perspective

Software safety assurance philosophy and practices at the NASA Ames are discussed. It is shown that, to be safe, software must be error-free. Software developments on two digital flight control systems and two ground facility systems are examined, including the overall system and software organization and function, the software-safety issues, and their resolution. The effectiveness of safety assurance methods is discussed, including conventional life-cycle practices, verification and validation testing, software safety analysis, and formal design methods. It is concluded (1) that a practical software safety technology does not yet exist, (2) that it is unlikely that a set of general-purpose analytical techniques can be developed for proving that software is safe, and (3) that successful software safety-assurance practices will have to take into account the detailed design processes employed and show that the software will execute correctly under all possible conditions.

Dunn, William R.↗
NASA NTRSDate not supplied

Psyche Hardware Abstraction Simulation Environment (PHASE): Development of High-Fidelity Downlink Model for Flight Software Performance Testing

NASA’s Katherine Johnson Independent Verification and Validation (IV&V) Independent Test Capability (ITC) Team develops high fidelity software-only simulations (or “digital-twins”) of NASA’s most critical missions that are then used for software verification and validation (V&V). One such digital-twin simulator nearing maturity is the Psyche Hardware Abstraction Simulation Environment (PHASE). PHASE utilizes an instruction set simulator along with ITC-developed avionics models. PHASE includes a newly-developed model of the telecom hardware with a high-fidelity downlink interface, capable of supporting reconfigurable downlink data rates and transmission modes. The different downlink transmission modes allow more processing to be ceded from software onto the telecom hardware. Thus, we are able to monitor the performance of the downlink software module in the time-partitioned OS and perform more realistic dynamic testing based on software’s resource utilization. This presentation will cover PHASE, avionics modeling, the downlink model, and its usage for flight software performance testing.

modeling↗
NASA NTRS2005

Architecture for Verifiable Software

Verifiable MDS Architecture (VMA) is a software architecture that facilitates the construction of highly verifiable flight software for NASA s Mission Data System (MDS), especially for smaller missions subject to cost constraints. More specifically, the purpose served by VMA is to facilitate aggressive verification and validation of flight software while imposing a minimum of constraints on overall functionality. VMA exploits the state-based architecture of the MDS and partitions verification issues into elements susceptible to independent verification and validation, in such a manner that scaling issues are minimized, so that relatively large software systems can be aggressively verified in a cost-effective manner.

Reinholtz, William↗
NASA NTRS2020

Risk considerations for autonomy software

This paper summarizes key findings related to methods for the risk considerations of autonomy software. Existing methods for Verification and Validation (V&V) of autonomy software are summarized and a method for the assurance of autonomy software is suggested and demonstrated on a command execution use case. Risk and reliability are defined in the context of autonomy and an approach for risk assessment of autonomy is presented using an example use case. Key insights regarding areas of uncertainty for autonomy are provided, along with a suggested architecture for the systematic consideration of reliability within the context of a given autonomous planner.

Meshkat, Leila↗
NASA NTRS1989

Initial Ada components evaluation

The SAIC has the responsibility for independent test and validation of the SSE. They have been using a mathematical functions library package implemented in Ada to test the SSE IV and V process. The library package consists of elementary mathematical functions and is both machine and accuracy independent. The SSE Ada components evaluation includes code complexity metrics based on Halstead's software science metrics and McCabe's measure of cyclomatic complexity. Halstead's metrics are based on the number of operators and operands on a logical unit of code and are compiled from the number of distinct operators, distinct operands, and total number of occurrences of operators and operands. These metrics give an indication of the physical size of a program in terms of operators and operands and are used diagnostically to point to potential problems. McCabe's Cyclomatic Complexity Metrics (CCM) are compiled from flow charts transformed to equivalent directed graphs. The CCM is a measure of the total number of linearly independent paths through the code's control structure. These metrics were computed for the Ada mathematical functions library using Software Automated Verification and Validation (SAVVAS), the SSE IV and V tool. A table with selected results was shown, indicating that most of these routines are of good quality. Thresholds for the Halstead measures indicate poor quality if the length metric exceeds 260 or difficulty is greater than 190. The McCabe CCM indicated a high quality of software products.

Moebes, Travis↗
NASA NTRS1983

Adaptation of a software development methodology to the implementation of a large-scale data acquisition and control system

A progress report is presented on a program to upgrade the existing NASA Deep Space Network in terms of a redesigned computer-controlled data acquisition system for channelling tracking, telemetry, and command data between a California-based control center and three signal processing centers in Australia, California, and Spain. The methodology for the improvements is oriented towards single subsystem development with consideration for a multi-system and multi-subsystem network of operational software. Details of the existing hardware configurations and data transmission links are provided. The program methodology includes data flow design, interface design and coordination, incremental capability availability, increased inter-subsystem developmental synthesis and testing, system and network level synthesis and testing, and system verification and validation. The software has been implemented thus far to a 65 percent completion level, and the methodology being used to effect the changes, which will permit enhanced tracking and communication with spacecraft, has been concluded to feature effective techniques.

Madrid, G. A.↗
NASA NTRS1988

NASA Shuttle Training Aircraft flight simulation overview

The Shuttle Training Aircraft (STA) is a variable stability, variable control law flying simulator used by NASA/JSC to train astronauts in the final landing phase of a Space Shuttle Orbiter. A general outline is given for the STA flight simulation system. An overview is given of the software generation and verification process through the Advanced Validation System (AVAS). The flight test techniques for software verification will be reviewed and the process for releasing the software for flight training will be covered. The astronaut STA training syllabus is examined. Parameter matching with the Orbiter in the final approach phase of de-orbit and landing is briefly examined. Simulation performance will be assessed against flight data, performance measurement, and cue synchronization.

Justiz, Charles R.↗
NASA NTRS2022

Flight Software Dictionary Development for the Mars2020 Rover

The Mars2020 project, developed and operated by the Jet Propulsion Laboratory (JPL), successfully landed the Perseverance rover and its flying companion Ingenuity on the surface of Mars on February 18th 2021. Perseverance combines heritage and cutting-edge flight software and hardware to accomplish crucial mission requirements related to Martian surface sampling. The design, development, and operation of NASA’s large strategic science missions require the ability to communicate spacecraft capabilities to hundreds of engineers across multiple disciplines. The interaction between flight and ground software development, Verification and Validation (V&V), Assembly, Test, and Launch Operations (ATLO), and management each demand quick understanding of unique slices of information for each discipline. This information includes the current capabilities of the flight system as well as future capabilities and their status as they are developed and tested. Despite the fundamental and critical nature of this information, the flight software dictionaries used to track it are a stumbling block for many projects. These dictionaries provide the cornerstone for the interpretation of data sent from the spacecraft, allowing for quick comprehension by engineers on the ground. During both spacecraft development and operations, flight software dictionary management includes significant challenges due to the large number of interfacing systems and the subtle yet distinct needs of each.The engineering of flight software dictionaries for Mars2020 had numerous challenges, most-notably: parallel dictionary development to support simultaneous separate flight software build campaigns for each mission phase (cruise and surface), managing requests for operations-enabling information without perturbing the heritage interface with the rover, and the introduction of new tools by the dictionary stakeholders that forced the dictionary team to innovate and redesign the heritage tool chain. These challenges generated guiding principles for the dictionary development effort: emphasize coding best practices and unit testing in the dictionary code development tool chain, use institutionally provided COTS (commercial-off-the-shelf) tools whenever possible, and maintain the heritage flight-ground interface all while advancing operations-enabling information via a loosely coupled interface.Throughout development and operations, the Mars2020 dictionary toolchain included IBM DOORS Next Generation, GitHub, Microsoft Excel, Docker, Jenkins, and a significant custom-built Python codebase. Significant interfaces included JPL’s command and control software, heritage flight software team tools and processes, and the many cloud-based ground tools developed for the mission.This paper will discuss the requirements for the Mars2020 dictionary development, the development team’s response to those requirements, lessons learned throughout the process, steps taken towards automated deliveries and continuous integration of stakeholder inputs, potential toolchain improvements for Mars2020, and key takeaways that could be applied to future missions.

Pyrzak, Guy↗
NASA NTRS2015

NASA Operational Simulator for Small Satellites (NOS3)

The Simulation-to-Flight 1 (STF-1) CubeSat mission aims to demonstrate how legacy simulation technologies may be adapted for flexible and effective use on missions using the CubeSat platform. These technologies, named NASA Operational Simulator (NOS), have demonstrated significant value on several missions such as James Webb Space Telescope, Global Precipitation Measurement, Juno, and Deep Space Climate Observatory in the areas of software development, mission operationstraining, verification and validation (VV), test procedure development and software systems check-out. STF-1 will demonstrate a highly portable simulation and test platform that allows seamless transition of mission development artifacts to flight products. This environment will decrease development time of future CubeSat missions by lessening the dependency on hardware resources. In addition, through a partnership between NASA GSFC, the West Virginia Space Grant Consortium and West Virginia University, the STF-1 CubeSat will hosts payloads for three secondary objectives that aim to advance engineering and physical-science research in the areas of navigation systems of small satellites, provide useful data for understanding magnetosphere-ionosphere coupling and space weather, and verify the performance and durability of III-V Nitride-based materials.

modeling simulation↗
NASA NTRS2008

Assessing Requirements Quality through Requirements Coverage

In model-based development, the development effort is centered around a formal description of the proposed software system the model. This model is derived from some high-level requirements describing the expected behavior of the software. For validation and verification purposes, this model can then be subjected to various types of analysis, for example, completeness and consistency analysis [6], model checking [3], theorem proving [1], and test-case generation [4, 7]. This development paradigm is making rapid inroads in certain industries, e.g., automotive, avionics, space applications, and medical technology. This shift towards model-based development naturally leads to changes in the verification and validation (V&V) process. The model validation problem determining that the model accurately captures the customer's high-level requirements has received little attention and the sufficiency of the validation activities has been largely determined through ad-hoc methods. Since the model serves as the central artifact, its correctness with respect to the users needs is absolutely crucial. In our investigation, we attempt to answer the following two questions with respect to validation (1) Are the requirements sufficiently defined for the system? and (2) How well does the model implement the behaviors specified by the requirements? The second question can be addressed using formal verification. Nevertheless, the size and complexity of many industrial systems make formal verification infeasible even if we have a formal model and formalized requirements. Thus, presently, there is no objective way of answering these two questions. To this end, we propose an approach based on testing that, when given a set of formal requirements, explores the relationship between requirements-based structural test-adequacy coverage and model-based structural test-adequacy coverage. The proposed technique uses requirements coverage metrics defined in [9] on formal high-level software requirements and existing model coverage metrics such as the Modified Condition and Decision Coverage (MC/DC) used when testing highly critical software in the avionics industry [8]. Our work is related to Chockler et al. [2], but we base our work on traditional testing techniques as opposed to verification techniques.

Rajan, Ajitha↗
NASA NTRS1986

Software engineering environment tool set integration

Space Transportation System Division (STSD) Engineering has a program to promote excellence within the engineering function. This program resulted in a capital funded facility based on a VAX cluster called the Rockwell Operational Engineering System (ROSES). The second phase of a three phase plan to establish an integrated software engineering environment for ROSES is examined. It discusses briefly phase one which establishes the basic capability for a modern software development environment to include a tool set, training and standards. Phase two is a tool set integration. The tool set is primarily off-the-shelf tools acquired through vendors or government agencies (public domain). These tools were placed into categories of software development. These categories are: requirements, design, and construction support; verification and validation support; and software management support. The integration of the tool set is being performed through concept prototyping and development of tools specifically designed to support the life cycle and provide transition from one phase to the next.

Selfridge, William P.↗
NASA NTRS2004

From Bridges and Rockets, Lessons for Software Systems

Although differences exist between building software systems and building physical structures such as bridges and rockets, enough similarities exist that software engineers can learn lessons from failures in traditional engineering disciplines. This paper draws lessons from two well-known failures the collapse of the Tacoma Narrows Bridge in 1940 and the destruction of the space shuttle Challenger in 1986 and applies these lessons to software system development. The following specific applications are made: (1) the verification and validation of a software system should not be based on a single method, or a single style of methods; (2) the tendency to embrace the latest fad should be overcome; and (3) the introduction of software control into safety-critical systems should be done cautiously.

Holloway, C. Michael↗
NASA NTRS2009

Implementation of an Adaptive Controller System from Concept to Flight Test

The National Aeronautics and Space Administration Dryden Flight Research Center (Edwards, California) is conducting ongoing flight research using adaptive controller algorithms. A highly modified McDonnell-Douglas NF-15B airplane called the F-15 Intelligent Flight Control System (IFCS) was used for these algorithms. This airplane has been modified by the addition of canards and by changing the flight control systems to interface a single-string research controller processor for neural network algorithms. Research goals included demonstration of revolutionary control approaches that can efficiently optimize aircraft performance for both normal and failure conditions, and to advance neural-network-based flight control technology for new aerospace systems designs. Before the NF-15B IFCS airplane was certified for flight test, however, certain processes needed to be completed. This paper presents an overview of these processes, including a description of the initial adaptive controller concepts followed by a discussion of modeling formulation and performance testing. Upon design finalization, the next steps are: integration with the system interfaces, verification of the software, validation of the hardware to the requirements, design of failure detection, development of safety limiters to minimize the effect of erroneous neural network commands, and creation of flight test control room displays to maximize human situational awareness.

Larson, Richard R.↗
NASA NTRS2009

Implementation of an Adaptive Controller System from Concept to Flight Test

The National Aeronautics and Space Administration Dryden Flight Research Center (Edwards, California) is conducting ongoing flight research using adaptive controller algorithms. A highly modified McDonnell-Douglas NF-15B airplane called the F-15 Intelligent Flight Control System (IFCS) is used to test and develop these algorithms. Modifications to this airplane include adding canards and changing the flight control systems to interface a single-string research controller processor for neural network algorithms. Research goals include demonstration of revolutionary control approaches that can efficiently optimize aircraft performance in both normal and failure conditions and advancement of neural-network-based flight control technology for new aerospace system designs. This report presents an overview of the processes utilized to develop adaptive controller algorithms during a flight-test program, including a description of initial adaptive controller concepts and a discussion of modeling formulation and performance testing. Design finalization led to integration with the system interfaces, verification of the software, validation of the hardware to the requirements, design of failure detection, development of safety limiters to minimize the effect of erroneous neural network commands, and creation of flight test control room displays to maximize human situational awareness; these are also discussed.

Larson, Richard R.↗
NASA NTRS2018

Software and Human-Machine Interface Development for Environmental Controls Subsystem Support

The Space Launch System (SLS) is the next premier launch vehicle for NASA. It is the next stage of manned space exploration from American soil, and will be the platform in which we push further beyond Earth orbit. In preparation of the SLS maiden voyage on Exploration Mission 1 (EM-1), the existing ground support architecture at Kennedy Space Center required significant overhaul and updating. A comprehensive upgrade of controls systems was necessary, including programmable logic controller software, as well as Launch Control Center (LCC) firing room and local launch pad displays for technician use. Environmental control acts as an integral component in these systems, being the foremost system for conditioning the pad and extremely sensitive launch vehicle until T-0. The Environmental Controls Subsystem (ECS) required testing and modification to meet the requirements of the designed system, as well as the human factors requirements of NASA software for Validation and Verification (V&V). This term saw significant strides in the progress and functionality of the human-machine interfaces used at the launch pad, and improved integration with the controller code.

Dobson, Matthew↗
NASA NTRS2024

Out of Plane Stresses Using a Beam-Based Elastic Foundation Bonded Joint Model

Efficient, semi-analytical models of bonded joints are utilized for initial joint design and sizing due to their convenience and speed. Many of these models are based on beam and plate theories, which are most applicable when in-plane stresses/strains dominate. While this can be a sensible assumption for obtaining the deformations of the joint, out of plane (transverse) stresses are often of critical importance for failure in bonded joints composed of fiber reinforced composite materials. Since through-thickness strains are neglected in these semi-analytical theories, the present study shows a method to back-calculate the out of plane stresses based on stress balance equations, which use the in-plane stresses and displacements that have already been obtained. The method was implemented within the Joint Element Designer (JED) software, and verification and validation examples have been shown. As is typical for these types of models, corner stress singularities are not captured by the JED model, but the general trends of the stress and deformation fields agreed with a higher fidelity finite element model. As such, this method for obtaining out of plane stresses, when coupled with the JED code, is very useful for the complete stress analysis of bonded composite joints.

composites↗
NASA NTRS1988

Verification and validation of rulebased systems for Hubble Space Telescope ground support

As rulebase systems become more widely used in operational environments, the focus is on the problems and concerns of maintaining expert systems. In the conventional software model, the verification and validation of a system have two separate and distinct meanings. To validate a system means to demonstrate that the system does what is advertised. The verification process refers to investigating the actual code to identify inconsistencies and redundancies within the logic path. In current literature regarding maintaining rulebased systems, little distinction is made between these two terms. In fact, often the two terms are used interchangeably. Verification and validation of rulebased systems are discussed as separate but equally important aspects of the maintenance phase. Also described are some of the tools and methods that were developed at the Space Telescope Science Institute to aid in the maintenance of the rulebased system.

Vick, Shon↗