SEARCH · Engineering Papers

Results for “software verification and validation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 253 records · Page 14

All sources NASA NTRS DOE OSTI

NASA NTRS1988

Knowledge-based assistance in costing the space station DMS

The Software Cost Engineering (SCE) methodology developed over the last two decades at IBM Systems Integration Division (SID) in Houston is utilized to cost the NASA Space Station Data Management System (DMS). An ongoing project to capture this methodology, which is built on a foundation of experiences and lessons learned, has resulted in the development of an internal-use-only, PC-based prototype that integrates algorithmic tools with knowledge-based decision support assistants. This prototype Software Cost Engineering Automation Tool (SCEAT) is being employed to assist in the DMS costing exercises. At the same time, DMS costing serves as a forcing function and provides a platform for the continuing, iterative development, calibration, and validation and verification of SCEAT. The data that forms the cost engineering database is derived from more than 15 years of development of NASA Space Shuttle software, ranging from low criticality, low complexity support tools to highly complex and highly critical onboard software.

Henson, Troy↗

NASA NTRS1986

X-29A technology demonstrator flight test program overview

An overview of the X-29A functional flight program and concept evaluation program is presented, including some of the unique and different preparations for the first flight. Included are a discussion of the many organizational responsibilities and a description of the program management structure for the test team. Also discussed are preflight ground, flight functional envelope expansion, and flight research test objectives and qualitive results to date for both a limited-envelope flight control system and an expanded-envelope system. The aircraft, including the instrumentation system and measurements, is described. In addition, a discussion is included regarding the use of major support facilities, such as ground and flight simulators, the NASA Western Aeronautical Test Range and mission control center, and the Grumman automated telemetry station. An overview of the associated real-time and postflight batch data processing software approaches is presented. The use of hardware-in-the-loop simulation for independent verification and validation and mission planning and practice is discussed. Also included is a description of the flight-readiness review, the airworthiness and flight safety review, work scheduling, technical briefings, and preflight and postflight crew briefings. The configuration control process used on the X-29A program is described, and its relationship to both simulation and aircraft operations is discussed. An X-29A schedule overview is presented with an outline of a proposed follow-on program.

Sefic, W. J.↗

NASA NTRS1992

Knowledge-based system V and V in the Space Station Freedom program

Knowledge Based Systems (KBS's) are expected to be heavily used in the Space Station Freedom Program (SSFP). Although SSFP Verification and Validation (V&V) requirements are based on the latest state-of-the-practice in software engineering technology, they may be insufficient for Knowledge Based Systems (KBS's); it is widely stated that there are differences in both approach and execution between KBS V&V and conventional software V&V. In order to better understand this issue, we have surveyed and/or interviewed developers from sixty expert system projects in order to understand the differences and difficulties in KBS V&V. We have used this survey results to analyze the SSFP V&V requirements for conventional software in order to determine which specific requirements are inappropriate for KBS V&V and why they are inappropriate. Further work will result in a set of recommendations that can be used either as guidelines for applying conventional software V&V requirements to KBS's or as modifications to extend the existing SSFP conventional software V&V requirements to include KBS requirements. The results of this work are significant to many projects, in addition to SSFP, which will involve KBS's.

Kelley, Keith↗

NASA NTRS1999

Abstract for 1999 Rational Software User Conference

We develop spacecraft fault-protection software at NASA/JPL. Challenges exemplified by our task: 1) high-quality systems - need for extensive validation & verification; 2) multi-disciplinary context - involves experts from diverse areas; 3) embedded systems - must adapt to external practices, notations, etc.; and 4) development pressures - NASA's mandate of "better, faster, cheaper".

Dunphy, Julia↗

NASA NTRS2000

Independent Verification and Validation (IV and V) Criteria

The purpose of this appendix is to establish quantifiable criteria for determining whether IV&V should be applied to a given software development. Since IV&V should begin in the Formulation Subprocess of a project, the process here described is based on metrics which are available before project approval.

McGill, Kenneth↗

NASA NTRS1990

Cost and quality planning for large NASA programs

The Software Cost and Quality Engineering methodology developed over the last two decades at IBM Federal Sector Div. is used to plan the NASA Space Station Data Management System (DMS). An ongoing project to capture this methodology, which is built on a foundation of experiences and lessons learned, has resulted in the development of a PC-based tool that integrates cost and quality forecasting methodologies and data in a consistent manner. This tool, Software Cost and Quality Engineering Starter Set (SCQESS), is being used to assist in the DMS costing exercises. At the same time, DMS planning serves as a forcing function and provides a platform for the continuing, iterative development, calibration, and validation and verification of SCQESS. The data that forms the cost and quality engineering data base is derived from more than 17 years of development of NASA Space Shuttle software, ranging from low criticality, low complexity support tools to highly complex and highly critical onboard software.

Rone, Kyle Y.↗

NASA NTRS2019

Agile Approach to Assuring the Safety-Critical Embedded Software for NASA's Orion Spacecraft

Human-rated missions like those in NASA's Orion Program continue to grow in complexity. The role of software in achieving ambitious mission objectives has expanded dramatically in the last few decades. Assuring the safety and performance of the embedded flight software is quickly growing beyond the reach of traditional methods and resource levels. The methods used to build these software-dominant systems evolve in an on-going attempt to keep pace with the scope of our ambitions. Agile software development is now commonplace. The long timelines and large batches of work associated with traditional methods are being replaced by rapid delivery of small increments _ as system capabilities are realized in waves. Assurance of these critical software capabilities must therefore conquer an ever-expanding frontier of challenges, and do so with an approach matched to the evolving development methods. This paper recounts the journey of the Orion Independent Verification and Validation (IV&V) team as we addressed this dynamic environment. Widening our aperture to encompass a dramatically larger mission scope, while adjusting our cadence to synchronize with the rapid pace of agile software development, a new approach to IV&V is emerging. This approach is characterized by a sharper focus on mission capabilities, matched with a method to dynamically _follow the risk' as the IV&V team delivers more compelling assurance data in waves. Traditional methods prevalent in IV&V tend to scope the work using artifacts of the development process as they evolve from preliminary to final versions, and the pace of delivery was synchronized with the development timelines prevalent in the waterfall lifecycle. That more static approach is out of phase with the demands of the new environment. Scoping work according to the critical capabilities of the system (rather than artifacts of development) and synchronizing with the rapid pace of agile development, we are moving toward more effective parity with the demands of the environment. We explain the concrete steps we took, the principles that motivated our choices, and the results we have achieved to date.

Capability based assurance↗

NASA NTRS2015

Independent Verification and Validation Program

Presentation to be given to European Space Agency counterparts to give an overview of NASA's IVV Program and the layout and structure of the Software Testing and Research laboratory maintained at IVV. Seeking STI-ITAR review due to the international audience. Most of the information has been presented to public audiences in the past, with some variations on data, or is in the public domain.

Research↗

NASA NTRS1986

Investigation of an advanced fault tolerant integrated avionics system

Presented is an advanced, fault-tolerant multiprocessor avionics architecture as could be employed in an advanced rotorcraft such as LHX. The processor structure is designed to interface with existing digital avionics systems and concepts including the Army Digital Avionics System (ADAS) cockpit/display system, navaid and communications suites, integrated sensing suite, and the Advanced Digital Optical Control System (ADOCS). The report defines mission, maintenance and safety-of-flight reliability goals as might be expected for an operational LHX aircraft. Based on use of a modular, compact (16-bit) microprocessor card family, results of a preliminary study examining simplex, dual and standby-sparing architectures is presented. Given the stated constraints, it is shown that the dual architecture is best suited to meet reliability goals with minimum hardware and software overhead. The report presents hardware and software design considerations for realizing the architecture including redundancy management requirements and techniques as well as verification and validation needs and methods.

Dunn, W. R.↗

NASA NTRS2002

The New NASA Orbital Debris Engineering Model ORDEM2000

The NASA Orbital Debris Program Office at Johnson Space Center has developed a new computer-based orbital debris engineering model, ORDEM2000, which describes the orbital debris environment in the low Earth orbit region between 200 and 2000 km altitude. The model is appropriate for those engineering solutions requiring knowledge and estimates of the orbital debris environment (debris spatial density, flux, etc.). ORDEM2000 can also be used as a benchmark for ground-based debris measurements and observations. We incorporated a large set of observational data, covering the object size range from 10 mm to 10 m, into the ORDEM2000 debris database, utilizing a maximum likelihood estimator to convert observations into debris population probability distribution functions. These functions then form the basis of debris populations. We developed a finite element model to process the debris populations to form the debris environment. A more capable input and output structure and a user-friendly graphical user interface are also implemented in the model. ORDEM2000 has been subjected to a significant verification and validation effort. This document describes ORDEM2000, which supersedes the previous model, ORDEM96. The availability of new sensor and in situ data, as well as new analytical techniques, has enabled the construction of this new model. Section 1 describes the general requirements and scope of an engineering model. Data analyses and the theoretical formulation of the model are described in Sections 2 and 3. Section 4 describes the verification and validation effort and the sensitivity and uncertainty analyses. Finally, Section 5 describes the graphical user interface, software installation, and test cases for the user.

Liou, Jer-Chyi↗

NASA NTRS2015

Dynamic Emulation of NASA Missions for IVandV: A Case Study of JWST and SLS

Software-Only-Simulations are an emerging but quickly developing field of study throughout NASA. The NASA Independent Verification Validation (IVV) Independent Test Capability (ITC) team has been rapidly building a collection of simulators for a wide range of NASA missions. ITC specializes in full end-to-end simulations that enable developers, VV personnel, and operators to test-as-you-fly. In four years, the team has delivered a wide variety of spacecraft simulations ranging from low complexity science missions such as the Global Precipitation Management (GPM) satellite and the Deep Space Climate Observatory (DSCOVR), to the extremely complex missions such as the James Webb Space Telescope (JWST) and Space Launch System (SLS).

Emulation↗

NASA NTRS2013

State Machine Modeling of the Space Launch System Solid Rocket Boosters

The Space Launch System is a Shuttle-derived heavy-lift vehicle currently in development to serve as NASA's premiere launch vehicle for space exploration. The Space Launch System is a multistage rocket with two Solid Rocket Boosters and multiple payloads, including the Multi-Purpose Crew Vehicle. Planned Space Launch System destinations include near-Earth asteroids, the Moon, Mars, and Lagrange points. The Space Launch System is a complex system with many subsystems, requiring considerable systems engineering and integration. To this end, state machine analysis offers a method to support engineering and operational e orts, identify and avert undesirable or potentially hazardous system states, and evaluate system requirements. Finite State Machines model a system as a finite number of states, with transitions between states controlled by state-based and event-based logic. State machines are a useful tool for understanding complex system behaviors and evaluating "what-if" scenarios. This work contributes to a state machine model of the Space Launch System developed at NASA Ames Research Center. The Space Launch System Solid Rocket Booster avionics and ignition subsystems are modeled using MATLAB/Stateflow software. This model is integrated into a larger model of Space Launch System avionics used for verification and validation of Space Launch System operating procedures and design requirements. This includes testing both nominal and o -nominal system states and command sequences.

model-based systems engineering↗

NASA NTRS2022

Robotics Verification and Validation Strategies for Perseverance Rover Sampling and Caching

The Mars 2020 Sampling and Caching Subsystem(SCS) is the most complex robotic system ever fielded on a MarsRover. It includes a 5 degree-of-freedom Robotic Arm, coringdrill, gas Dust Removal Tool, interfaces for two turret-mountedinstruments, and an Adaptive Caching Assembly (ACA). TheACA is itself a complex robotic system, containing hardware tosupport docking and bit exchange, a 3 degree-of-freedomSample Handling Assembly for manipulating sample tubes,storage for several drill bits and sample tubes, and mechanismsto support observing and sealing samples collected by the drill.To successfully verify and validate the SCS hardware andsoftware and its integration with the Mars 2020 flight systemseveral key strategies were employed.The SCS Verification and Validation (V&V) program utilizedmultiple test venues with tiered levels of fidelity. These includedsimulation and visualization software environments, low fidelitydevelopment testbeds, testbeds with high fidelity SCS hardwareand commercial off-the shelf avionics, integrated systemtestbeds with flight-like avionics, and environmental testbedscapable of simulating Martian surface temperature andpressure. Multiple units of each SCS hardware componentmoved fluidly between test venues to accomplish myriadstandalone and coordinated test objectives. Test preparationand executions were performed by a diverse team of engineerswith training and technical ownership tailored for individualexperience and role. Despite significant differences between testvenues, the SCS V&V team established efficient and consistentprocesses and tools for procedure development, test execution,and data review that enabled personnel, as well as technicalproducts such as sequences and parameter configurations, toflow between venues effectively. A series of benchmark testsprovided evidence of performance consistency as elements weretransferred between venues and as system capability evolved.This paper provides an overview of the SCS V&V program andexplores several overarching strategies that enabled successfuloperation in the face of unprecedented complexity. Keyoutcomes of the SCS validation effort are summarized, alongwith lessons learned and beneficial integrations of validationtool and process innovations into Mars surface operations.

Brooks, Sawyer↗

NASA NTRS2022

Making or Breaking a Rover: System Engineering Parameters On-Board the Mars 2020 Perseverance Rover

On February 18, 2021, Perseverance, NASA’s Jet Propulsion Laboratory’s (JPL’s) Mars 2020 Rover, successfully landed on Mars with all systems nominal, despite the risk surrounding the over 200,000 internal flight parameters that had to be properly configured. The Perseverance team defines these parameters as software variables that are configurable, commandable and retrievable from Earth. In 2015, the Mars 2020 project leaders focused on improving systems engineering of parameters based on their experiences from parameter management on previous Mars rovers (Curiosity, Opportunity, Spirit, and Pathfinder) and parameter failures of past missions, such as the mission-ending parameter of the Mars Climate Orbiter. The new rigorous development process allowed for efficient certification and effective implementation of the parameters, allowing the rover to approach and land on the red planet (the most challenging phase of the mission) with zero parameter issues. Although successful, the Perseverance team learned many lessons for how to better manage parameters for the continued surface operations of the Mars 2020 mission and future missions. This paper will discuss eight parameter-management topics for the Perseverance Mission. The first is parameter definition: how we define parameters on our mission, where they are physically located on the vehicle, and why we have so many of them. The second topic is the updated parameter flight software module from Curiosity, including details on the 99% reduction in parameter commands, new bulk configuration capabilities, and improved parameter traceability. The third topic is parameter selection for different mission phases; this includes improving and tweaking our preferred parameter settings until they become certification candidates and managing parameter configurations based on test venue throughout the mission life cycle. The fourth topic is our flight certification process; this includes certification of flight values for four different epochs in the mission: Launch, Entry Decent and Landing (EDL) - 6days, Landing + 5 Sols (Martian Days, still on Cruise Flight Software), and once are on Surface Flight Software (FSW). The fifth topic covers in-flight command implementation, along with details on testing, validation, and verification of those commands. In the sixth section, we will explain our use of open-source management tools, including how we used GitHub for version control and management approvals. The seventh topic will describe the ground tools used in operations, including capabilities of the in-house built tool called Parasol. The eighth and final topic will dig into lessons learned for improving parameter management in the future of this mission and others.

Roth, Brian↗

NASA NTRS2022

Making or Breaking a Rover- Systems Engineering Parameters On-Board the Mars 2020 Perseverance Rover

Roth, Brian↗

NASA NTRS1992

Expert system verification and validation study: ES V/V Workshop

The primary purpose of this document is to build a foundation for applying principles of verification and validation (V&V) of expert systems. To achieve this, some V&V as applied to conventionally implemented software is required. Part one will discuss the background of V&V from the perspective of (1) what is V&V of software and (2) V&V's role in developing software. Part one will also overview some common analysis techniques that are applied when performing V&V of software. All of these materials will be presented based on the assumption that the reader has little or no background in V&V or in developing procedural software. The primary purpose of part two is to explain the major techniques that have been developed for V&V of expert systems.

French, Scott↗

NASA NTRS1997

Knowledge-Based Aircraft Automation: Managers Guide on the use of Artificial Intelligence for Aircraft Automation and Verification and Validation Approach for a Neural-Based Flight Controller

The ultimate goal of this report was to integrate the powerful tools of artificial intelligence into the traditional process of software development. To maintain the US aerospace competitive advantage, traditional aerospace and software engineers need to more easily incorporate the technology of artificial intelligence into the advanced aerospace systems being designed today. The future goal was to transition artificial intelligence from an emerging technology to a standard technology that is considered early in the life cycle process to develop state-of-the-art aircraft automation systems. This report addressed the future goal in two ways. First, it provided a matrix that identified typical aircraft automation applications conducive to various artificial intelligence methods. The purpose of this matrix was to provide top-level guidance to managers contemplating the possible use of artificial intelligence in the development of aircraft automation. Second, the report provided a methodology to formally evaluate neural networks as part of the traditional process of software development. The matrix was developed by organizing the discipline of artificial intelligence into the following six methods: logical, object representation-based, distributed, uncertainty management, temporal and neurocomputing. Next, a study of existing aircraft automation applications that have been conducive to artificial intelligence implementation resulted in the following five categories: pilot-vehicle interface, system status and diagnosis, situation assessment, automatic flight planning, and aircraft flight control. The resulting matrix provided management guidance to understand artificial intelligence as it applied to aircraft automation. The approach taken to develop a methodology to formally evaluate neural networks as part of the software engineering life cycle was to start with the existing software quality assurance standards and to change these standards to include neural network development. The changes were to include evaluation tools that can be applied to neural networks at each phase of the software engineering life cycle. The result was a formal evaluation approach to increase the product quality of systems that use neural networks for their implementation.

Broderick, Ron↗

NASA NTRS2012

Upgrades at the NASA Langley Research Center National Transonic Facility

Several projects have been completed or are nearing completion at the NASA Langley Research Center (LaRC) National Transonic Facility (NTF). The addition of a Model Flow-Control/Propulsion Simulation test capability to the NTF provides a unique, transonic, high-Reynolds number test capability that is well suited for research in propulsion airframe integration studies, circulation control high-lift concepts, powered lift, and cruise separation flow control. A 1992 vintage Facility Automation System (FAS) that performs the control functions for tunnel pressure, temperature, Mach number, model position, safety interlock and supervisory controls was replaced using current, commercially available components. This FAS upgrade also involved a design study for the replacement of the facility Mach measurement system and the development of a software-based simulation model of NTF processes and control systems. The FAS upgrades were validated by a post upgrade verification wind tunnel test. The data acquisition system (DAS) upgrade project involves the design, purchase, build, integration, installation and verification of a new DAS by replacing several early 1990's vintage computer systems with state of the art hardware/software. This paper provides an update on the progress made in these efforts. See reference 1.

Paryz, Roman W.↗