SEARCH · Engineering Papers

Results for “software verification and validation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 199 records · Page 11

All sources NASA NTRS DOE OSTI

NASA NTRSDate not supplied

A Discussion of Time Management Concepts and Time Constraint Equations for Multi-Rate Federation Executions

The High Level Architecture (HLA) is a simulation interoperability standard developed by the Simulation Interoperability Standards Organization (SISO) and published as the international standard IEEE 1516-2010 by the Institute for Electrical and Electronics Engineers (IEEE). HLA is a widely used standard for the development and execution of collaborative distributed simulations. HLA provides a number of Management Services to simulation developers: Federation, Declaration, Object, Ownership, Data Distribution, and Time. Of those services, Time Management Services is probably one of the least understood and least used. However, Time Management Services are critical to technical simulations like those created for space systems using the Space Reference Federation Object Model (SpaceFOM). Time Management can be used to insure data coherence and execution repeatability in distributed simulations. When combined with real time execution policies, Time Management is being used to support real time execution of mixed software and hardware in the loop integration, verification, and validation simulations for active space systems development. This paper starts by providing an overview of the HLA Time Management Services. This provides the background to discuss the challenges associated with Time Management and its use, starting with simple common rate frame scheduled simulations, then simple multi-rate simulations, and ending with complex mixed rate simulations. The authors then formulate the significant time constraint relationships between identified frame scheduling parameters. The intent of the paper is to provide a concise discussion of how to use Time Management in both simple cases and in more complex mixed frame rate federation executions.

Simulation Interoperability↗

NASA NTRSDate not supplied

A Discussion of Time Management Concepts and Time Constraint Equations for Multi-Rate Federation Executions

Simulation Interoperability↗

NASA NTRSDate not supplied

Verification and Validation of Safety-Critical Aircraft Systems Operating under Off-Nominal, Contingency, and Emergency Conditions

Verification and validation (V&V) of safety-critical technologies developed for loss of control (LOC) prevention and recovery and other aviation safety concerns pose significant challenges. Aircraft LOC can result from a wide spectrum of hazards, often occurring in combination, which cannot be fully replicated during evaluation. Technologies developed for LOC prevention and recovery must therefore be effective under a wide variety of hazardous and uncertain conditions, and the verification and validation of these technologies must provide some measure of assurance that the new vehicle safety technologies do no harm (i.e., that they themselves do not introduce new safety risks). V&V technologies must also enable the identification of system limitations and constraints, as well as enable the identification of safe and unsafe operating conditions (and their boundaries). Additionally, the V&V of complex, increasingly autonomous systems is a fundamental concern. Scalable, reproducible and cost-effective techniques for the assurance of safety critical systems during their design and operation is a key barrier to fielding new systems or updating current systems. Moreover, these techniques need to provide artifacts that enable a comprehensive evidence-based approach to certification. This briefing summarizes research performed under NASA’s Aviation Safety Program and follow-on research for the V&V of safety-critical aircraft system technologies developed for LOC prevention and recovery and increasingly autonomous systems, and for a broad assurance capability in both current and emerging aviation applications. Note that, in this briefing, the term “validation” refers to a confirmation that the system implementation (e.g., algorithms etc.) is performing the intended function(s), as well as an affirmation of effectiveness in these functions. “Verification” refers to a confirmation that the system implementation in the software and hardware meets its (hopefully validated) specifications (e.g., correctly executes algorithms as designed).

Validation↗

NASA NTRS1985

Some key considerations in evolving a computer system and software engineering support environment for the space station program

The space station data management system involves networks of computing resources that must work cooperatively and reliably over an indefinite life span. This program requires a long schedule of modular growth and an even longer period of maintenance and operation. The development and operation of space station computing resources will involve a spectrum of systems and software life cycle activities distributed across a variety of hosts, an integration, verification, and validation host with test bed, and distributed targets. The requirement for the early establishment and use of an apporopriate Computer Systems and Software Engineering Support Environment is identified. This environment will support the Research and Development Productivity challenges presented by the space station computing system.

Mckay, C. W.↗

NASA NTRS2009

Proceedings of the Third International Workshop on Proof-Carrying Code and Software Certification

This NASA conference publication contains the proceedings of the Third International Workshop on Proof-Carrying Code and Software Certification, held as part of LICS in Los Angeles, CA, USA, on August 15, 2009. Software certification demonstrates the reliability, safety, or security of software systems in such a way that it can be checked by an independent authority with minimal trust in the techniques and tools used in the certification process itself. It can build on existing validation and verification (V&V) techniques but introduces the notion of explicit software certificates, Vvilich contain all the information necessary for an independent assessment of the demonstrated properties. One such example is proof-carrying code (PCC) which is an important and distinctive approach to enhancing trust in programs. It provides a practical framework for independent assurance of program behavior; especially where source code is not available, or the code author and user are unknown to each other. The workshop wiII address theoretical foundations of logic-based software certification as well as practical examples and work on alternative application domains. Here "certificate" is construed broadly, to include not just mathematical derivations and proofs but also safety and assurance cases, or any fonnal evidence that supports the semantic analysis of programs: that is, evidence about an intrinsic property of code and its behaviour that can be independently checked by any user, intermediary, or third party. These guarantees mean that software certificates raise trust in the code itself, distinct from and complementary to any existing trust in the creator of the code, the process used to produce it, or its distributor. In addition to the contributed talks, the workshop featured two invited talks, by Kelly Hayhurst and Andrew Appel. The PCC 2009 website can be found at http://ti.arc.nasa.gov /event/pcc 091.

Ewen, Denney, W.↗

NASA NTRS1992

Software assurance standard

This standard specifies the software assurance program for the provider of software. It also delineates the assurance activities for the provider and the assurance data that are to be furnished by the provider to the acquirer. In any software development effort, the provider is the entity or individual that actually designs, develops, and implements the software product, while the acquirer is the entity or individual who specifies the requirements and accepts the resulting products. This standard specifies at a high level an overall software assurance program for software developed for and by NASA. Assurance includes the disciplines of quality assurance, quality engineering, verification and validation, nonconformance reporting and corrective action, safety assurance, and security assurance. The application of these disciplines during a software development life cycle is called software assurance. Subsequent lower-level standards will specify the specific processes within these disciplines.

Source record↗

NASA NTRS2014

Evolution of Software-Only-Simulation at NASA IV and V

Software-Only-Simulations have been an emerging but quickly developing field of study throughout NASA. The NASA Independent Verification Validation (IVV) Independent Test Capability (ITC) team has been rapidly building a collection of simulators for a wide range of NASA missions. ITC specializes in full end-to-end simulations that enable developers, VV personnel, and operators to test-as-you-fly. In four years, the team has delivered a wide variety of spacecraft simulations that have ranged from low complexity science missions such as the Global Precipitation Management (GPM) satellite and the Deep Space Climate Observatory (DSCOVR), to the extremely complex missions such as the James Webb Space Telescope (JWST) and Space Launch System (SLS).This paper describes the evolution of ITCs technologies and processes that have been utilized to design, implement, and deploy end-to-end simulation environments for various NASA missions. A comparison of mission simulators are discussed with focus on technology and lessons learned in complexity, hardware modeling, and continuous integration. The paper also describes the methods for executing the missions unmodified flight software binaries (not cross-compiled) for verification and validation activities.

Embedded↗

NASA NTRS1989

A verification library for multibody simulation software

A multibody dynamics verification library, that maintains and manages test and validation data is proposed, based on RRC Robot arm and CASE backhoe validation and a comparitive study of DADS, DISCOS, and CONTOPS that are existing public domain and commercial multibody dynamic simulation programs. Using simple representative problems, simulation results from each program are cross checked, and the validation results are presented. Functionalities of the verification library are defined, in order to automate validation procedure.

Kim, Sung-Soo↗

NASA NTRS2003

Development of Methodologies for IV and V of Neural Networks

Non-deterministic systems often rely upon neural network (NN) technology to "lean" to manage flight systems under controlled conditions using carefully chosen training sets. How can these adaptive systems be certified to ensure that they will become increasingly efficient and behave appropriately in real-time situations? The bulk of Independent Verification and Validation (IV&V) research of non-deterministic software control systems such as Adaptive Flight Controllers (AFC's) addresses NNs in well-behaved and constrained environments such as simulations and strict process control. However, neither substantive research, nor effective IV&V techniques have been found to address AFC's learning in real-time and adapting to live flight conditions. Adaptive flight control systems offer good extensibility into commercial aviation as well as military aviation and transportation. Consequently, this area of IV&V represents an area of growing interest and urgency. ISR proposes to further the current body of knowledge to meet two objectives: Research the current IV&V methods and assess where these methods may be applied toward a methodology for the V&V of Neural Network; and identify effective methods for IV&V of NNs that learn in real-time, including developing a prototype test bed for IV&V of AFC's. Currently. no practical method exists. lSR will meet these objectives through the tasks identified and described below. First, ISR will conduct a literature review of current IV&V technology. TO do this, ISR will collect the existing body of research on IV&V of non-deterministic systems and neural network. ISR will also develop the framework for disseminating this information through specialized training. This effort will focus on developing NASA's capability to conduct IV&V of neural network systems and to provide training to meet the increasing need for IV&V expertise in such systems.

Taylor, Brian↗

NASA NTRSDate not supplied

Design and Testing of an Approach to Automated In-Flight Safety Risk Management for sUAS Operations

An onboard risk management automation design is presented based on run-time assurance principles, as well as the concept for In-Time Aviation Safety Management Systems (IASMS) as described by the National Academies. The automation is designed to operate independently of the autopilot and perform real-time risk assessment spanning multiple classes of hazards, predict constraint violations, and track autopilot states. In the event of elevated risk conditions or predicted constraint violations, the automation will select from a set of available contingencies and trigger autopilot mode changes if necessary to mitigate risk exposure. The onboard automation also informs the remote operator/pilot of what the independent monitor is observing and any contingency decisions or actions that may arise during flight. Details of an implementation of this design and results of verification and validation activities, as required to meet stringent NASA software and system assurance standards, are also presented. This includes simulation and flight testing using small unmanned aircraft systems.

Ersin Ancel↗

NASA NTRS1978

Tools for Embedded Computing Systems Software

A workshop was held to assess the state of tools for embedded systems software and to determine directions for tool development. A synopsis of the talk and the key figures of each workshop presentation, together with chairmen summaries, are presented. The presentations covered four major areas: (1) tools and the software environment (development and testing); (2) tools and software requirements, design, and specification; (3) tools and language processors; and (4) tools and verification and validation (analysis and testing). The utility and contribution of existing tools and research results for the development and testing of embedded computing systems software are described and assessed.

Source record↗

NASA NTRS2007

Full Life-Cycle Defect Management Assessment: Initial Inspection Data Collection Results and Research Questions for Further Study

It is often the case in software projects that when schedule and budget resources are limited, the Verification and Validation (V&V) activities suffer. Fewer V&V activities can be afforded and moreover, short-term challenges can result in V&V activities being scaled back or dropped altogether. As a result, too often the default solution is to save activities for improving software quality until too late in the life-cycle, relying on late-term code inspections followed by thorough testing activities to reduce defect counts to acceptable levels. As many project managers realize, however, this is a resource-intensive way of achieving the required quality for software. The Full Life-cycle Defect Management Assessment Initiative, funded by NASA s Office of Safety and Mission Assurance under the Software Assurance Research Program, aims to address these problems by: Improving the effectiveness of early life-cycle V&V activities to make their benefits more attractive to team leads. Specifically, we focus on software inspection, a proven method that can be applied to any software work product, long before executable code has been developed; Better communicating this effectiveness to software development teams, along with suggestions for parameters to improve in the future to increase effectiveness; Analyzing the impact of early life-cycle V&V on the effectiveness and cost required for late life-cycle V&V activities, such as testing, in order to make the tradeoffs more apparent. This white paper reports on an initial milestone in this work, the development of a preliminary model of inspection effectiveness across multiple NASA Centers. This model contributes toward reaching our project goals by: Allowing an examination of inspection parameters, across different types of projects and different work products, for an analysis of factors that impact defect detection effectiveness. Allowing a comparison of this NASA-specific model to existing recommendations in the literature regarding how to plan effective inspections. Forming a baseline model which can be extended to incorporate factors describing: the numbers and types of defects that are missed by inspections; how such defects flow downstream through software development phases; how effectively they can be caught by testing activities in the late stages of development. The model has been implemented in a prototype web-enabled decision-support tool which allows developers to enter their inspection data and receive feedback based on a comparison against the model. The tool also allows users to access reusable materials (such as checklists) from projects included in the baseline. Both the tool itself and the model underlying it will continue to be extended throughout the remainder of this initiative. As results of analyzing inspection effectiveness for defect containment are determined, they can be shared via the tool and also via updates to existing training courses on metrics and software inspections. Moreover, the tool will help satisfy key CMMI requirements for the NASA Centers, as it will enable NASA to take a global view across peer review results for various types of projects to identify systemic problems. This analysis can result in continuous improvements to the approach to verification.

Shull, Forrest↗

NASA NTRS1981

Management of the Galileo attitude and articulation control flight software development

Management concepts are presented for software development for a new technology area, i.e., real-time autonomous, computer-based spacecraft control. Flight computer selection and sizing are done initially to maximize performance within constraints of size, power, and cost. A higher order language is chosen to enhance productivity. Because the computer is embedded in the control systems hardware and is tied to the iterative design process of the spacecraft, the management and configuration control of the software is different from more typical applications. The development process must permit early coding but accept late changes. Margin management must be a continuing process in the development. Validation and verification is a special problem because it is not feasible to test the software in the actual operating environment prior to launch.

Pace, G. D.↗

NASA NTRS1982

Experiences and perspectives with SRI's tools for software design and validation

Development of tools that include the STP theorem poer and its associated Design Verification Systems; PHIL, a meta-programmable context sensitive structured editor; Pegasus, a system for support of graphical programming; and OBJ, an ultra high level programming language based on rewrite rules and abstract data type is reported.

Goguen, J.↗

NASA NTRS1983

Simulations used in the development and flight test of the HiMAT vehicle

Real-time simulations have been essential in the flight-test program of the highly maneuverable aircraft technology (HiMAT) remotely piloted research vehicle at the Dryden Flight Research Facility of NASA Ames Research Center. The HiMAT project makes extensive use of simulations in design, development, and qualification for flight, pilot training, and flight planning. Four distinct simulations, each with varying amounts of hardware in the loop, were developed for the HiMAT project. The use of simulations has been the key to flight qualification of the HiMAT vehicle. Specifically, they are useful in detecting anomalous behavior of the flight software and hardware at the various stages of development, verification, and validation.

Evans, M. B.↗

NASA NTRS1984

The role of simulation in the development and flight test of the HiMAT vehicle

Real time simulations have been essential in the flight test program of the highly maneuverable aircraft technology (HiMAT) remotely piloted research vehicle at NASA Ames Research Center's Dryden Flight Research Facility. The HiMAT project makes extensive use of simulations in design, development, and qualification for flight, pilot training, and flight planning. Four distinct simulations, each with varying amounts of hardware in the loop, were developed for the HiMAT project. The use of simulations in detecting anomalous behavior of the flight software and hardware at the various stages of development, verification, and validation has been the key to flight qualification of the HiMAT vehicle.

Evans, M. B.↗

NASA NTRS2022

Actuator and Motor Control End-to-End V&V on the Mars 2020 Rover

The Mars 2020 Perseverance rover is the most advanced robotic exploration system ever sent to another planet. To support the complex scientific and mobility needs of the mission, the rover utilizes 33 actuators, three multi-degree-of-freedom force-torque sensors, fifteen single or dual-speed resolvers, two solenoid valves, and twelve contact switches. The control for these actuators and sensors is achieved by several levels of flight software, coordinated between two computers with varying bandwidth control loops. Furthermore, the actuators and sensors were integrated into multiple larger robotic mechanisms that were delivered by different organizations at various points in the Integration and Test (I&T) timeline. All of this created a very complex Verification and Validation (V&V) scenario involving multiple subsystems and teams, several hardware and software testbeds with varying levels of fidelity, and significant systems engineering to ensure the overall I&T schedule could be maintained while ensuring system hardware safety.This paper details the integrated V&V effort across multiple teams and venues to provide full coverage of all necessary functionality, performance, and fault protection. First, it provides an overview of how the V&V campaign was subdivided among teams and venues and provides descriptions of the various hardware configurations used to support the testing. The Mars 2020 implementation of the plan incorporates many of the lessons learned from Mars Science Laboratory’s test campaign, and these value-added modifications are discussed here. Also included in this section is the system-level environmental testing approach used for mechanisms. Second, the paper describes the phased approach used by the teams to support new hardware and software deliveries to testbed and Systems I&T. In this approach the test campaign was built upon higher-level mechanism needs for performance, functionality, and safety at specific times in the campaign. Finally, the paper discusses lessons learned from the V&V campaign that should be applied to future large-scale motion control testing efforts.

Borne, Davis↗

NASA NTRS1975

Simulation verification techniques study

Results are summarized of the simulation verification techniques study which consisted of two tasks: to develop techniques for simulator hardware checkout and to develop techniques for simulation performance verification (validation). The hardware verification task involved definition of simulation hardware (hardware units and integrated simulator configurations), survey of current hardware self-test techniques, and definition of hardware and software techniques for checkout of simulator subsystems. The performance verification task included definition of simulation performance parameters (and critical performance parameters), definition of methods for establishing standards of performance (sources of reference data or validation), and definition of methods for validating performance. Both major tasks included definition of verification software and assessment of verification data base impact. An annotated bibliography of all documents generated during this study is provided.

Schoonmaker, P. B.↗