Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “software verification and validation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 235 records · Page 13
All sourcesNASA NTRSDOE OSTI
NASA NTRSDate not supplied

A Discussion of Time Management Concepts and Time Constraint Equations for Multi-Rate Federation Executions

The High Level Architecture (HLA) is a simulation interoperability standard developed by the Simulation Interoperability Standards Organization (SISO) and published as the international standard IEEE 1516-2010 by the Institute for Electrical and Electronics Engineers (IEEE). HLA is a widely used standard for the development and execution of collaborative distributed simulations. HLA provides a number of Management Services to simulation developers: Federation, Declaration, Object, Ownership, Data Distribution, and Time. Of those services, Time Management Services is probably one of the least understood and least used. However, Time Management Services are critical to technical simulations like those created for space systems using the Space Reference Federation Object Model (SpaceFOM). Time Management can be used to insure data coherence and execution repeatability in distributed simulations. When combined with real time execution policies, Time Management is being used to support real time execution of mixed software and hardware in the loop integration, verification, and validation simulations for active space systems development. This paper starts by providing an overview of the HLA Time Management Services. This provides the background to discuss the challenges associated with Time Management and its use, starting with simple common rate frame scheduled simulations, then simple multi-rate simulations, and ending with complex mixed rate simulations. The authors then formulate the significant time constraint relationships between identified frame scheduling parameters. The intent of the paper is to provide a concise discussion of how to use Time Management in both simple cases and in more complex mixed frame rate federation executions.

Simulation Interoperability↗
NASA NTRSDate not supplied

Verification and Validation of Safety-Critical Aircraft Systems Operating under Off-Nominal, Contingency, and Emergency Conditions

Verification and validation (V&V) of safety-critical technologies developed for loss of control (LOC) prevention and recovery and other aviation safety concerns pose significant challenges. Aircraft LOC can result from a wide spectrum of hazards, often occurring in combination, which cannot be fully replicated during evaluation. Technologies developed for LOC prevention and recovery must therefore be effective under a wide variety of hazardous and uncertain conditions, and the verification and validation of these technologies must provide some measure of assurance that the new vehicle safety technologies do no harm (i.e., that they themselves do not introduce new safety risks). V&V technologies must also enable the identification of system limitations and constraints, as well as enable the identification of safe and unsafe operating conditions (and their boundaries). Additionally, the V&V of complex, increasingly autonomous systems is a fundamental concern. Scalable, reproducible and cost-effective techniques for the assurance of safety critical systems during their design and operation is a key barrier to fielding new systems or updating current systems. Moreover, these techniques need to provide artifacts that enable a comprehensive evidence-based approach to certification. This briefing summarizes research performed under NASA’s Aviation Safety Program and follow-on research for the V&V of safety-critical aircraft system technologies developed for LOC prevention and recovery and increasingly autonomous systems, and for a broad assurance capability in both current and emerging aviation applications. Note that, in this briefing, the term “validation” refers to a confirmation that the system implementation (e.g., algorithms etc.) is performing the intended function(s), as well as an affirmation of effectiveness in these functions. “Verification” refers to a confirmation that the system implementation in the software and hardware meets its (hopefully validated) specifications (e.g., correctly executes algorithms as designed).

Validation↗
NASA NTRS1985

Some key considerations in evolving a computer system and software engineering support environment for the space station program

The space station data management system involves networks of computing resources that must work cooperatively and reliably over an indefinite life span. This program requires a long schedule of modular growth and an even longer period of maintenance and operation. The development and operation of space station computing resources will involve a spectrum of systems and software life cycle activities distributed across a variety of hosts, an integration, verification, and validation host with test bed, and distributed targets. The requirement for the early establishment and use of an apporopriate Computer Systems and Software Engineering Support Environment is identified. This environment will support the Research and Development Productivity challenges presented by the space station computing system.

Mckay, C. W.↗
NASA NTRS2009

Proceedings of the Third International Workshop on Proof-Carrying Code and Software Certification

This NASA conference publication contains the proceedings of the Third International Workshop on Proof-Carrying Code and Software Certification, held as part of LICS in Los Angeles, CA, USA, on August 15, 2009. Software certification demonstrates the reliability, safety, or security of software systems in such a way that it can be checked by an independent authority with minimal trust in the techniques and tools used in the certification process itself. It can build on existing validation and verification (V&V) techniques but introduces the notion of explicit software certificates, Vvilich contain all the information necessary for an independent assessment of the demonstrated properties. One such example is proof-carrying code (PCC) which is an important and distinctive approach to enhancing trust in programs. It provides a practical framework for independent assurance of program behavior; especially where source code is not available, or the code author and user are unknown to each other. The workshop wiII address theoretical foundations of logic-based software certification as well as practical examples and work on alternative application domains. Here "certificate" is construed broadly, to include not just mathematical derivations and proofs but also safety and assurance cases, or any fonnal evidence that supports the semantic analysis of programs: that is, evidence about an intrinsic property of code and its behaviour that can be independently checked by any user, intermediary, or third party. These guarantees mean that software certificates raise trust in the code itself, distinct from and complementary to any existing trust in the creator of the code, the process used to produce it, or its distributor. In addition to the contributed talks, the workshop featured two invited talks, by Kelly Hayhurst and Andrew Appel. The PCC 2009 website can be found at http://ti.arc.nasa.gov /event/pcc 091.

Ewen, Denney, W.↗
DOE OSTI2022

Verification and validation testing and tools: comparison between MCNP code versions and nuclear data libraries [Slides]

This presentation discusses the primary goal of software testing which is to test the code for correctness. It also discusses the results for individual suites and the role of validation and verification also referred to in the presentation as V&V. In summation, the V&V framework enables easy comparison between calculations performed with different code versions and/or nuclear data libraries. This entire framework will be distributed with the upcoming MCNP6.3 release. V&V test suites shown and several that were not (Criticality, LAQGSM, Lockwood) will be distributed in the new framework.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2021

Network Architecture Verification & Validation Tool

The NAVV Tool is an automation of Linux commands run Zeek IDS software on a packet capture to create a Microsoft Excel spreadsheet table breaking down network traffic observed. The tool automates the Zeek software analysis, the collation of logs, and then the dissection of the Conn.log and DNS.logs to create a summary table within a Excel. This spreadsheet can then be updated with network segments using CIDR formatting and labels along with inventory information including name and IP address. Using the tool again will integrate these label and color coding into the existing analysis table to aid in conducting an evaluation of the network traffic.

Nichols, DonovanW↗
DOE OSTI2021

SAM Co-Development to Support Fluoride-salt-cooled High-temperature Reactor Design and Licensing (CRADA 2018-18177 Final Report)

This project aims to raise the SAM code's technical and commercial maturity level to enable the Kairos Power ("Kairos") to use SAM to support its KP-FHR design analysis and licensing application. Argonne National Laboratory ("Argonne") has actively developed and maintained SAM, a modern system-level analysis tool for advanced nonlight water reactors safety analysis. Kairos is currently actively pursuing development of an FHR design and associated technology, and consequently requires a pedigreed safety analysis tool. The purpose of this project is to increase the maturity of the SAM code for the modeling and simulation of the KP-FHR design, thus enabling its use in safety analyses that support licensing application. Argonne will primarily be responsible for software design, development, and testing; while Kairos will be responsible for software requirements, assessment of software capabilities, needs, gaps, and priorities, and development of proprietary models. The research and development activities for the joint SAM development project include: a series of identification and prioritization studies on design characteristics, event sequences, relevant phenomena, and software capabilities; SAM capability enhancements for specific KP-FHR systems and components; performing code verification and validations; integrating uncertainty quantification (UQ), model calibration, and sensitivity analysis (SA) techniques in safety analyses; and raising the software quality rigor level for commercial-grade applications.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
NASA NTRS1992

Software assurance standard

This standard specifies the software assurance program for the provider of software. It also delineates the assurance activities for the provider and the assurance data that are to be furnished by the provider to the acquirer. In any software development effort, the provider is the entity or individual that actually designs, develops, and implements the software product, while the acquirer is the entity or individual who specifies the requirements and accepts the resulting products. This standard specifies at a high level an overall software assurance program for software developed for and by NASA. Assurance includes the disciplines of quality assurance, quality engineering, verification and validation, nonconformance reporting and corrective action, safety assurance, and security assurance. The application of these disciplines during a software development life cycle is called software assurance. Subsequent lower-level standards will specify the specific processes within these disciplines.

Source record↗
NASA NTRS2014

Evolution of Software-Only-Simulation at NASA IV and V

Software-Only-Simulations have been an emerging but quickly developing field of study throughout NASA. The NASA Independent Verification Validation (IVV) Independent Test Capability (ITC) team has been rapidly building a collection of simulators for a wide range of NASA missions. ITC specializes in full end-to-end simulations that enable developers, VV personnel, and operators to test-as-you-fly. In four years, the team has delivered a wide variety of spacecraft simulations that have ranged from low complexity science missions such as the Global Precipitation Management (GPM) satellite and the Deep Space Climate Observatory (DSCOVR), to the extremely complex missions such as the James Webb Space Telescope (JWST) and Space Launch System (SLS).This paper describes the evolution of ITCs technologies and processes that have been utilized to design, implement, and deploy end-to-end simulation environments for various NASA missions. A comparison of mission simulators are discussed with focus on technology and lessons learned in complexity, hardware modeling, and continuous integration. The paper also describes the methods for executing the missions unmodified flight software binaries (not cross-compiled) for verification and validation activities.

Embedded↗
NASA NTRS1989

A verification library for multibody simulation software

A multibody dynamics verification library, that maintains and manages test and validation data is proposed, based on RRC Robot arm and CASE backhoe validation and a comparitive study of DADS, DISCOS, and CONTOPS that are existing public domain and commercial multibody dynamic simulation programs. Using simple representative problems, simulation results from each program are cross checked, and the validation results are presented. Functionalities of the verification library are defined, in order to automate validation procedure.

Kim, Sung-Soo↗
DOE OSTI2022

Code Verification and Solution Verification framework in pin-resolved neutron transport code MPACT

Program verification in scientific computing encompasses the application of formal and mathematical techniques to a scientific computing code for its credibility, accuracy, and validity. Code Verification identifies bugs and performance issues in the software development stage. Solution Verification assesses the applicability of the code and the accuracy of the solution to problems of interest. Both activities utilize application cases and quantify the error against prescribed acceptance criteria. However, simply executing more application cases does not guarantee stronger or more comprehensive credibility. Here, we establish a verification framework that involves Code Verification and Solution Verification, both of which work together such that the overarching goal of “converge to the correct answer for the intended application” can be reasonably inferred. The application of such a verification framework is demonstrated using the pin-resolved neutron transport code MPACT, where standard unit tests and regression tests are covered, and where the Method of Exact Solutions and the Method of Manufactured Solutions are successfully used. Additionally, the applicability of Method of Manufactured Solutions is extended to the OECD/NEA C5G7 benchmark problems of practical material and geometric configurations. Solution Verification activities are demonstrated on a practical hierarchy of application models of increasing complexity ranging from 2D pin cell problems to 3D assembly problems. The convergence behavior and rate of convergence with respect to each individual variable are studied and provided. This framework can be adapted broadly to other fields involving scientific computing codes.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Suitability of Altair FEKO's Computational Electromagnetic Techniques for Tree Model Scattering and Penetration at Ku-band

Foliage penetration (FOPEN) radar at lower frequencies (VHF, UHF) is a well-studied area with many contributions. However, there is growing interest in using higher Ku-band frequencies (12-18 GHz) for FOPEN. Specifically, the reduced wavelength sizes provide some key saliencies for developing more optimized detection solutions. The disadvantage is that exploiting Ku-band for FOPEN is complicated because higher frequencies have pronounced scattering effects due to their smaller wavelengths. A methodology h as been developed to model and simulate FOPEN problems that characterize the phenomenology of Ku-band electromagnetic ( EM ) wave transmissions through moderate foliage. The details of this research (i.e. the realistic tree models, simulation setup and results) are documented in multiple reports. The main focus of this report is to describe the preliminary validation and verification of Altair FEKO, the computational EM (CEM) software used for this research, as well as present a simplified symmetrical tree model and an introductory CAD tree model.

42 ENGINEERING↗
NASA NTRS2003

Development of Methodologies for IV and V of Neural Networks

Non-deterministic systems often rely upon neural network (NN) technology to "lean" to manage flight systems under controlled conditions using carefully chosen training sets. How can these adaptive systems be certified to ensure that they will become increasingly efficient and behave appropriately in real-time situations? The bulk of Independent Verification and Validation (IV&V) research of non-deterministic software control systems such as Adaptive Flight Controllers (AFC's) addresses NNs in well-behaved and constrained environments such as simulations and strict process control. However, neither substantive research, nor effective IV&V techniques have been found to address AFC's learning in real-time and adapting to live flight conditions. Adaptive flight control systems offer good extensibility into commercial aviation as well as military aviation and transportation. Consequently, this area of IV&V represents an area of growing interest and urgency. ISR proposes to further the current body of knowledge to meet two objectives: Research the current IV&V methods and assess where these methods may be applied toward a methodology for the V&V of Neural Network; and identify effective methods for IV&V of NNs that learn in real-time, including developing a prototype test bed for IV&V of AFC's. Currently. no practical method exists. lSR will meet these objectives through the tasks identified and described below. First, ISR will conduct a literature review of current IV&V technology. TO do this, ISR will collect the existing body of research on IV&V of non-deterministic systems and neural network. ISR will also develop the framework for disseminating this information through specialized training. This effort will focus on developing NASA's capability to conduct IV&V of neural network systems and to provide training to meet the increasing need for IV&V expertise in such systems.

Taylor, Brian↗
NASA NTRSDate not supplied

Design and Testing of an Approach to Automated In-Flight Safety Risk Management for sUAS Operations

An onboard risk management automation design is presented based on run-time assurance principles, as well as the concept for In-Time Aviation Safety Management Systems (IASMS) as described by the National Academies. The automation is designed to operate independently of the autopilot and perform real-time risk assessment spanning multiple classes of hazards, predict constraint violations, and track autopilot states. In the event of elevated risk conditions or predicted constraint violations, the automation will select from a set of available contingencies and trigger autopilot mode changes if necessary to mitigate risk exposure. The onboard automation also informs the remote operator/pilot of what the independent monitor is observing and any contingency decisions or actions that may arise during flight. Details of an implementation of this design and results of verification and validation activities, as required to meet stringent NASA software and system assurance standards, are also presented. This includes simulation and flight testing using small unmanned aircraft systems.

Ersin Ancel↗
DOE OSTI2020

Improvements to the New CTH Code Verification & Validation Test Suite (FY2020)

The CTH multiphysics hydrocode, which is used for a wide range of important calculations, has undertaken in recent years to overhaul its software quality and testing processes. A key part of this effort entailed building a new, robust V&V test suite made up of traditional hydrocode verification problems, such as those listed in the ASC Tri-Lab Test Suite and the Enhanced Tri-Lab Test Suite, as well as validation problems for some of CTHs most frequently used equations of state, materials models, and other key capabilities. Substantial progress towards this goal was made in FY19. In FY20, this test suite has been expanded to include verification and validation tests of the Sesame and JWL equation of state models as well as the Mader verification problem from the Tri-Lab Test Suite and the Blake verification problem - a linear elastic analog to the Hunter problem from the Enhanced Tri-Lab Test Suite. This report documents CTH performance on the new test suite problems. Verification test results are compared to analytic solutions and, for most tests, convergence results are presented. Validation test results are compared to experimental data and mesh refinement studies are included. CTH performs well overall on the new test problems. Convergence rates for the Blake and Mader problems are comparable to those for similar ASC codes. The JWL and Sesame verification tests show good agreement with analytic solutions. Likewise, CTH simulation results show good agreement with experimental validation data for the Sesame and JWL equations of state for the materials tested. Future V&V work will focus on adding tests for other key capabilities like fracture and high explosive models.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2025

MOOSE-based Tritium Migration Analysis Program, Version 8 (TMAP8) for advanced open-source tritium transport and fuel cycle modeling

Tritium management is critical for the safety, sustainability, and economics of fusion energy systems, and advanced and reliable modeling tools help accelerate the development of tritium technologies. This paper presents the Tritium Migration Analysis Program, Version 8 (TMAP8), an open-source, MOOSE-based application developed to provide state-of-the-art tritium transport and fuel cycle modeling capabilities. TMAP8 aims to expand the capabilities of previous versions (i.e., TMAP4 and TMAP7) by leveraging modern computational techniques, ensuring high software quality assurance standards (key to building trust), and enabling multispecies, multiscale, and multiphysics simulations for integrated tritium transport modeling in complex geometries. This paper outlines TMAP8’s scope and rigorous development practices, emphasizing its transparency, accessibility, modularity, and reliability. We present the current suite of verification and validation cases based on those from TMAP4, demonstrating TMAP8’s accuracy and reliability against analytical solutions and experimental data. Additionally, the paper showcases TMAP8’s integrated fuel cycle modeling capabilities, highlighting its applicability at various scales and levels. The TMAP8 code and documentation are openly available, promoting collaborative development and widespread adoption within the fusion community. Future work will soon expand TMAP8’s verification and validation suite to include those from TMAP7 and other recent experimental studies for validation.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
NASA NTRS1978

Tools for Embedded Computing Systems Software

A workshop was held to assess the state of tools for embedded systems software and to determine directions for tool development. A synopsis of the talk and the key figures of each workshop presentation, together with chairmen summaries, are presented. The presentations covered four major areas: (1) tools and the software environment (development and testing); (2) tools and software requirements, design, and specification; (3) tools and language processors; and (4) tools and verification and validation (analysis and testing). The utility and contribution of existing tools and research results for the development and testing of embedded computing systems software are described and assessed.

Source record↗
NASA NTRS2007

Full Life-Cycle Defect Management Assessment: Initial Inspection Data Collection Results and Research Questions for Further Study

It is often the case in software projects that when schedule and budget resources are limited, the Verification and Validation (V&V) activities suffer. Fewer V&V activities can be afforded and moreover, short-term challenges can result in V&V activities being scaled back or dropped altogether. As a result, too often the default solution is to save activities for improving software quality until too late in the life-cycle, relying on late-term code inspections followed by thorough testing activities to reduce defect counts to acceptable levels. As many project managers realize, however, this is a resource-intensive way of achieving the required quality for software. The Full Life-cycle Defect Management Assessment Initiative, funded by NASA s Office of Safety and Mission Assurance under the Software Assurance Research Program, aims to address these problems by: Improving the effectiveness of early life-cycle V&V activities to make their benefits more attractive to team leads. Specifically, we focus on software inspection, a proven method that can be applied to any software work product, long before executable code has been developed; Better communicating this effectiveness to software development teams, along with suggestions for parameters to improve in the future to increase effectiveness; Analyzing the impact of early life-cycle V&V on the effectiveness and cost required for late life-cycle V&V activities, such as testing, in order to make the tradeoffs more apparent. This white paper reports on an initial milestone in this work, the development of a preliminary model of inspection effectiveness across multiple NASA Centers. This model contributes toward reaching our project goals by: Allowing an examination of inspection parameters, across different types of projects and different work products, for an analysis of factors that impact defect detection effectiveness. Allowing a comparison of this NASA-specific model to existing recommendations in the literature regarding how to plan effective inspections. Forming a baseline model which can be extended to incorporate factors describing: the numbers and types of defects that are missed by inspections; how such defects flow downstream through software development phases; how effectively they can be caught by testing activities in the late stages of development. The model has been implemented in a prototype web-enabled decision-support tool which allows developers to enter their inspection data and receive feedback based on a comparison against the model. The tool also allows users to access reusable materials (such as checklists) from projects included in the baseline. Both the tool itself and the model underlying it will continue to be extended throughout the remainder of this initiative. As results of analyzing inspection effectiveness for defect containment are determined, they can be shared via the tool and also via updates to existing training courses on metrics and software inspections. Moreover, the tool will help satisfy key CMMI requirements for the NASA Centers, as it will enable NASA to take a global view across peer review results for various types of projects to identify systemic problems. This analysis can result in continuous improvements to the approach to verification.

Shull, Forrest↗