Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “software verification and validation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 127 records · Page 7
All sourcesNASA NTRSDOE OSTI
NASA NTRS2018

NOS3: NASA Operational Simulator for Small Satellites

The NASA Operational Simulator for Small Satellites (NOS3) is a suite of open-source software tools to aid in areas such as software development, integration & test (I&T), mission operations/training, verification and validation (V&V), and software systems check-out. NOS3 provides a software development environment, a multi-target build system, operational interface/ground software, dynamics and environment simulations, and software-based hardware models. NOS3 has just recently been open-sourced by NASA and is available for immediate use. It enables the development of flight software (FSW) early in the project life cycle when hardware availability is limited. Small satellite development suffers from extensive lead times on many of the commercial-off-the-shelf (COTS) components as well as limited funding for engineering test units (ETUs). To alleviate the need to provide a hardware test-bed for each developer/tester, NOS3 hardware models are based upon characteristic data or manufacturer's data sheets for each individual component. The NOS3 hardware models' fidelity is such that FSW executes unaware that physical hardware is not present. This allows FSW binaries to be compiled for both the simulation environment and the flight computer without changing the FSW source code. For hardware models that provide data which is dependent upon the environment and spacecraft dynamics, such as a GPS receiver or magnetometer, an open-source tool from NASA GSFC (42 Spacecraft Simulator) is used to provide the necessary data. The underlying infrastructure used to transfer messages between FSW and the hardware models can also be used to monitor, intercept, and inject messages, which has proven to be beneficial for V&V of larger missions such as James Webb Space Telescope (JWST). As hardware is selected and becomes available, drivers can be added to the NOS3 environment to enable hardware-in-the-loop (HWIL) testing. When strict time synchronization is not vital, any number of combinations of hardware components and software-based models can be tested. NOS3 was actively used for FSW development and component testing of the Simulation-to-Flight 1 (STF-1) CubeSat and the Lunar IceCube CubeSat. As NOS3 matures, hardware models have been added for common small satellite components such as GPS receivers, electrical power systems and batteries, and antenna systems.

Suder, Mark↗
NASA NTRS2011

Integrated Software Health Management for Aircraft GN and C

Modern aircraft rely heavily on dependable operation of many safety-critical software components. Despite careful design, verification and validation (V&V), on-board software can fail with disastrous consequences if it encounters problematic software/hardware interaction or must operate in an unexpected environment. We are using a Bayesian approach to monitor the software and its behavior during operation and provide up-to-date information about the health of the software and its components. The powerful reasoning mechanism provided by our model-based Bayesian approach makes reliable diagnosis of the root causes possible and minimizes the number of false alarms. Compilation of the Bayesian model into compact arithmetic circuits makes SWHM feasible even on platforms with limited CPU power. We show initial results of SWHM on a small simulator of an embedded aircraft software system, where software and sensor faults can be injected.

Schumann, Johann↗
NASA NTRS2016

Assurance of Fault Management: Risk-Significant Adverse Condition Awareness

Fault Management (FM) systems are ranked high in risk-based assessment of criticality within flight software, emphasizing the importance of establishing highly competent domain expertise to provide assurance for NASA projects, especially as spaceflight systems continue to increase in complexity. Insight into specific characteristics of FM architectures seen embedded within safety- and mission-critical software systems analyzed by the NASA Independent Verification Validation (IVV) Program has been enhanced with an FM Technical Reference (TR) suite. Benefits are aimed beyond the IVV community to those that seek ways to efficiently and effectively provide software assurance to reduce the FM risk posture of NASA and other space missions. The identification of particular FM architectures, visibility, and associated IVV techniques provides a TR suite that enables greater assurance that critical software systems will adequately protect against faults and respond to adverse conditions. The role FM has with regard to overall asset protection of flight software systems is being addressed with the development of an adverse condition (AC) database encompassing flight software vulnerabilities.Identification of potential off-nominal conditions and analysis to determine how a system responds to these conditions are important aspects of hazard analysis and fault management. Understanding what ACs the mission may face, and ensuring they are prevented or addressed is the responsibility of the assurance team, which necessarily should have insight into ACs beyond those defined by the project itself. Research efforts sponsored by NASAs Office of Safety and Mission Assurance defined terminology, categorized data fields, and designed a baseline repository that centralizes and compiles a comprehensive listing of ACs and correlated data relevant across many NASA missions. This prototype tool helps projects improve analysis by tracking ACs, and allowing queries based on project, mission type, domain component, causal fault, and other key characteristics. The repository has a firm structure, initial collection of data, and an interface established for informational queries, with plans for integration within the Enterprise Architecture at NASA IVV, enabling support and accessibility across the Agency. The development of an improved workflow process for adaptive, risk-informed FM assurance is currently underway.

Software Verification & Validation↗
NASA NTRS2008

Towards Certification of a Space System Application of Fault Detection and Isolation

Advanced fault detection, isolation and recovery (FDIR) software is being investigated at NASA as a means to the improve reliability and availability of its space systems. Certification is a critical step in the acceptance of such software. Its attainment hinges on performing the necessary verification and validation to show that the software will fulfill its requirements in the intended setting. Presented herein is our ongoing work to plan for the certification of a pilot application of advanced FDIR software in a NASA setting. We describe the application, and the key challenges and opportunities it offers for certification.

Software verification↗
NASA NTRS2015

PHM for Ground Support Systems Case Study: From Requirements to Integration

This session will detail the experience of members of the NASA Ames Prognostic Center of Excellence (PCoE) producing PHM tools for NASA Advanced Ground Support Systems, including the challenges in applying their research in a production environment. Specifically, we will 1) go over the systems engineering and review process used; 2) Discuss the challenges and pitfalls in this process; 3) discuss software architecting, documentation, verification and validation activities and 4) discuss challenges in communicating the benefits and limitations of PHM Technologies.

Prognostics↗
DOE OSTI2020

Verification and Validation of PARET/ANL (V.7.6)

This report documents the software testing which has been performed for the PARET/ANL version 7.6 software. The capabilities addressed in this testing were selected based on the use of the software for analysis work under the Research and Test Reactors Department quality assurance program. The verification and validation have been performed and documented to address the steady-state capabilities of the software, as described in Chapter 2, and the transient capabilities, as described in Chapter 3. Testing based on the comparison between PARET/ANL calculations and analytical solutions, hand calculations, or another code calculations of the test cases confirms that all the identified capabilities of the software were implemented correctly.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1987

Using Colored Stochastic Petri Net (CS-PN) software for protocol specification, validation, and evaluation

The specification, verification, validation, and evaluation, which make up the different steps of the CS-PN software are outlined. The colored stochastic Petri net software is applied to a Wound/Wait protocol decomposable into two principal modules: request or couple (transaction, granule) treatment module and wound treatment module. Each module is specified, verified, validated, and then evaluated separately, to deduce a verification, validation and evaluation of the complete protocol. The colored stochastic Petri nets tool is shown to be a natural extension of the stochastic tool, adapted to distributed systems and protocols, because the color conveniently takes into account the numerous sites, transactions, granules and messages.

Zenie, Alexandre↗
NASA NTRS2014

Turbulence Modeling Verification and Validation

Computational fluid dynamics (CFD) software that solves the Reynolds-averaged Navier-Stokes (RANS) equations has been in routine use for more than a quarter of a century. It is currently employed not only for basic research in fluid dynamics, but also for the analysis and design processes in many industries worldwide, including aerospace, automotive, power generation, chemical manufacturing, polymer processing, and petroleum exploration. A key feature of RANS CFD is the turbulence model. Because the RANS equations are unclosed, a model is necessary to describe the effects of the turbulence on the mean flow, through the Reynolds stress terms. The turbulence model is one of the largest sources of uncertainty in RANS CFD, and most models are known to be flawed in one way or another. Alternative methods such as direct numerical simulations (DNS) and large eddy simulations (LES) rely less on modeling and hence include more physics than RANS. In DNS all turbulent scales are resolved, and in LES the large scales are resolved and the effects of the smallest turbulence scales are modeled. However, both DNS and LES are too expensive for most routine industrial usage on today's computers. Hybrid RANS-LES, which blends RANS near walls with LES away from walls, helps to moderate the cost while still retaining some of the scale-resolving capability of LES, but for some applications it can still be too expensive. Even considering its associated uncertainties, RANS turbulence modeling has proved to be very useful for a wide variety of applications. For example, in the aerospace field, many RANS models are considered to be reliable for computing attached flows. However, existing turbulence models are known to be inaccurate for many flows involving separation. Research has been ongoing for decades in an attempt to improve turbulence models for separated and other nonequilibrium flows. When developing or improving turbulence models, both verification and validation are important steps in the process. Verification insures that the CFD code is solving the equations as intended (no errors in the implementation). This is typically done either through the method of manufactured solutions (MMS) or through careful step-by-step comparisons with other verified codes. After the verification step is concluded, validation is performed to document the ability of the turbulence model to represent different types of flow physics. Validation can involve a large number of test case comparisons with experiments, theory, or DNS. Organized workshops have proved to be valuable resources for the turbulence modeling community in its pursuit of turbulence modeling verification and validation. Workshop contributors using different CFD codes run the same cases, often according to strict guidelines, and compare results. Through these comparisons, it is often possible to (1) identify codes that have likely implementation errors, and (2) gain insight into the capabilities and shortcomings of different turbulence models to predict the flow physics associated with particular types of flows. These are valuable lessons because they help bring consistency to CFD codes by encouraging the correction of faulty programming and facilitating the adoption of better models. They also sometimes point to specific areas needed for improvement in the models. In this paper, several recent workshops are summarized primarily from the point of view of turbulence modeling verification and validation. Furthermore, the NASA Langley Turbulence Modeling Resource website is described. The purpose of this site is to provide a central location where RANS turbulence models are documented, and test cases, grids, and data are provided. The goal of this paper is to provide an abbreviated survey of turbulence modeling verification and validation efforts, summarize some of the outcomes, and give some ideas for future endeavors in this area.

Rumsey, Christopher L.↗
NASA NTRS2016

R2U2 in Space: System and Software Health Management for Small Satellites

In order for small but complex systems like rovers, SmallSats, or Unmanned Aircraft (UAS) to operate autonomously, they must have a real-time solution for assessing their own system health. System and Software Health Management (SHM) enables better detection of faulty sensors and software problems, and enables better fault management including mitigation of unpredicted fault scenarios in the absence of a human on-board. In recent work, we have developed a Responsive, Realizable, Unobtrusive Unit (R2U2) for on-board SHM of autonomous UAS and demonstrated its ability to detect faults during flight time. These faults, from sensor failures, to software problems, to malicious security attacks, can present as transient temporal faults that even humans are challenged to find. An R2U2 congfiuration is a modular combination of multiple types of temporal logic runtime observers with fault-specic Bayesian Nets and sensor filters. R2U2 reasons about both on-board hardware and software components; R2U2 itself can be instantiated as an independent FPGA (Field-Programmable Gate Array)-based conguration or as a software component running independently from other software on-board. Small satellites, such as CubeSats, also require on-board SHM and failure mitigation, as limited telemetry bandwidth does not allow the transmission of the entire system state for ground-based health management. However, the autonomous operation of satellites brings a set of challenges different from UAS, including the effects of radiation on non-rad-hard, low-cost components, and the harsher environment of space. We surmise that a new extension of R2U2 could be adapted to help better detect, for example, radiation errors in cheaper COTS (Commercial Off the Shelf) (not rad-hard) components often used in small space systems. Since small satellites often operate in coordination, we will also examine new ways of distributed monitoring of their communication and cooperation and real-time detection of off-nominal situations utilizing multiple satellites. This talk will discuss preliminary work and ideas for building on terrestrial success of system and software health management for the harsher, and differently challenging, environment of space.

Runtime Verification & Validation↗
DOE OSTI2022

Qualification of SAS4A/SASSYS-1 for Sodium-Cooled Fast Reactor Authorization and Licensing (Final CRADA Report)

This project focuses on assessment and qualification of the SAS4A/SASSYS-1 advanced reactor safety analysis code and its capacity to fulfill requirements associated with license/authorization of a sodium-cooled fast reactor (SFR). Specific objectives of this proposal include: development of a formal documentation requirements matrix that complies with a vendor’s NQA-1 program to identify needs for qualification documentation; completion of software methods qualification gap analysis to identify and close gaps in the verification and validation basis of the software; generation of an Acceptance Test Report to confirm compliance with the software requirements specification; and closure of gaps in Software Design Description (SDD) to assist with the eventual generation of a Modeling Licensing Topic Report (LTR).

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
NASA NTRS1991

Safety

Software requirements, design, implementation, verification and validation, and especially management are affected by the need to produce safe software. This paper discusses the changes in the software life cycle that are necessary to ensure that software will execute without resulting in unacceptable risk. Software is being used increasingly to monitor and control safety-critical processes in which a run-time failure or error could result in unacceptable losses such as death, injury, loss of property, or environmental harm. Examples of such processes maybe found in transportation, energy, aerospace, basic industry, medicine, and defense systems.

Leveson, Nancy G.↗
NASA NTRS2000

Software IV and V Research Priorities and Applied Program Accomplishments Within NASA

The mission of this research is to be world-class creators and facilitators of innovative, intelligent, high performance, reliable information technologies that enable NASA missions to (1) increase software safety and quality through error avoidance, early detection and resolution of errors, by utilizing and applying empirically based software engineering best practices; (2) ensure customer software risks are identified and/or that requirements are met and/or exceeded; (3) research, develop, apply, verify, and publish software technologies for competitive advantage and the advancement of science; and (4) facilitate the transfer of science and engineering data, methods, and practices to NASA, educational institutions, state agencies, and commercial organizations. The goals are to become a national Center Of Excellence (COE) in software and system independent verification and validation, and to become an international leading force in the field of software engineering for improving the safety, quality, reliability, and cost performance of software systems. This project addresses the following problems: Ensure safety of NASA missions, ensure requirements are met, minimize programmatic and technological risks of software development and operations, improve software quality, reduce costs and time to delivery, and improve the science of software engineering

Blazy, Louis J.↗
NASA NTRS1997

IV and V Issues in Achieving High Reliability and Safety in Critical Control System Software

Risk analysis and integrated verification and validation are two important elements in a plan for ensuring the safety of critical software systems. We describe an approach we are currently developing for integrating risk analysis, and metrics analysis, and propose a fault predictor that would integrate the results of these activities. Practical difficulties associated with our approach are also discussed, as are limitations of the proposed predictor. We conclude with a discussion of what as been learned to date, and with suggestions for future work.

validation software reliability software risk asse↗
NASA NTRS2019

Agile Approach to Assuring the Safety-Critical Embedded Software for NASA's Orion Spacecraft

Human-rated missions like NASA's Exploration Mission - 1 (EM-1) and the Orion Multi-Purpose Crew Vehicle are becoming exceedingly complex in terms of software's contribution to achieving mission objectives. The increasing complexity and inherent safety critical nature of the embedded flight software imposes a unique resource challenge to assurance providers responsible for affirming that the mission is going to fly safely. Another challenge NASA and other Government agencies are facing is that more and more software is being developed using an agile development methodology, which is divergent from the typical waterfall, iterative, and incremental development methodologies assurance providers generally observe in the development of safety-critical embedded software. Orion Independent Verification and Validation (IV&V) has addressed these challenges by providing focused assurance results of critical mission capabilities prioritized by a dynamic assessment of risk level. Prior to this approach, Orion IV&V evaluated areas of risk in much broader, and more static, terms. Due to the agile software development life cycle that Orion follows, IV&V findings were often reported months out of phase with the developer thereby imposing increased rework costs. As a result of evolving the approach to adding assurance on Orion, IV&V is able to incrementally deliver high-priority assurance conclusions and more impactful issues more in phase with the developer activities, thereby increasing the value of the findings to the project. The agile IV&V approach employed by the Orion IV&V team strives to achieve a cadence of delivery that matches the pace of development. This agile approach provides increased flexibility for the assurance provider to become more efficient in reporting assurance conclusions and issues. This paper and presentation will discuss the principles which drive the design of our approach, results to date, and stimulate thinking for groups looking to add assurance to software being developed using an agile methodology.

Justin Smith↗
NASA NTRS2018

Customer Avionics Interface Development and Analysis (CAIDA): Software Developer for Avionics Systems

The Customer Avionics Interface Development and Analysis (CAIDA) supports the testing of the Launch Control System (LCS), NASA's command and control system for the Space Launch System (SLS), Orion Multi-Purpose Crew Vehicle (MPCV), and ground support equipment. The objective of the semester-long internship was to support day-to-day operations of CAIDA and help prepare for verification and validation of CAIDA software.

Mitchell, Sherry L.↗
NASA NTRS2002

Infusing and selecting V&V activities

The evolving nature of software development poses a continuing series of challenges for V&V. In response, the V&V community selectively adapts the use of existing V&V activities, and introduces new and improved ones.

verification validation risk software quality assu↗
NASA NTRS1975

Automatic documentation system extension to multi-manufacturers' computers and to measure, improve, and predict software reliability

The DOMONIC system has been modified to run on the Univac 1108 and the CDC 6600 as well as the IBM 370 computer system. The DOMONIC monitor system has been implemented to gather data which can be used to optimize the DOMONIC system and to predict the reliability of software developed using DOMONIC. The areas of quality metrics, error characterization, program complexity, program testing, validation and verification are analyzed. A software reliability model for estimating program completion levels and one on which to base system acceptance have been developed. The DAVE system which performs flow analysis and error detection has been converted from the University of Colorado CDC 6400/6600 computer to the IBM 360/370 computer system for use with the DOMONIC system.

Simmons, D. B.↗
NASA NTRS2015

Assuring NASA's Safety and Mission Critical Software

What is IV&V? Independent Verification and Validation (IV&V) is an objective examination of safety and mission critical software processes and products. Independence: 3 Key parameters: Technical Independence; Managerial Independence; Financial Independence. NASA IV&V perspectives: Will the system's software: Do what it is supposed to do?; Not do what it is not supposed to do?; Respond as expected under adverse conditions?. Systems Engineering: Determines if the right system has been built and that it has been built correctly. IV&V Technical Approaches: Aligned with IEEE 1012; Captured in a Catalog of Methods; Spans the full project lifecycle. IV&V Assurance Strategy: The IV&V Project's strategy for providing mission assurance; Assurance Strategy is driven by the specific needs of an individual project; Implemented via an Assurance Design; Communicated via Assurance Statements.

Validation↗