Results for “verification and validation” — Page 29

NASA NTRS1990

Third Annual Workshop on Space Operations Automation and Robotics (SOAR 1989)

Papers presented at the Third Annual Workshop on Space Operations Automation and Robotics (SOAR '89), hosted by the NASA Lyndon 8. Johnson Space Center at Houston, Texas, on July 25-27, 1989, are documented herein. During the three days, approximately 100 technical papers wer3 presented by experts from NASA, the USAF, universities, and technical companies. Also held were panel discussions on Air Force/NASA AI Overview and Expert System Verification and Validation. Tutorial sessions included Neural Networks; Theory and Application of Back Propagation; Verification and Validation of Expert Systems/ Evaluation of Expert System Tools; and Technical Environment for Modular Architectures for Robotics in Space; and are not documented herein. Technical topics addressed included intelligent systems, robotics, human factors, and environment.

Knowledge representation↗

DOE OSTI2023

CYVET: A Cyber-Physical Assurance Framework Based on a Semi-Supervised Vetting Approach

The project is designed to directly address the needs described in the Topic Area 4 (“Cybersecurity Verification and Validation”) of the CEDS’19 Research Call. The proposed CYVET system directly addresses the need to elevate the current industry capabilities to verify and validate OT cybersecurity and associated control system infrastructure. Currently, there is a significant gap in the energy sector’s capability in that regard during infrastructure improvement, equipment procurement, and compliance certification. CYVET provides that needed capability. CYVET is device and architecture agnostic and thus broadly applicable across the energy sector. The goal of this project is to develop and deliver a cybersecurity verification and validation framework testing capability to verify and validate OT equipment, software, and the underlying control system architecture. The primary project objectives are (i) Verification: the synthesis and reconciliation of standards and vendor supplied features, (ii) Validation: the generation, execution, and presentation of testing scripts of verified security features, and (3) Demonstration: apply the developed technology capabilities for verification and validation at a relevant end-user facility in the energy sector.

Lopez Jr, Juan↗

NASA NTRS2021

Operational Modal Analysis of the Space Launch System Mobile Launcher on the Crawler Transporter ISVV-010 Rollout

NASA is developing an expendable heavy lift launch vehicle capability, the Space Launch System, to support lunar and deep space exploration. To support this capability, an updated ground infrastructure is required including modifying an existing Mobile Launcher system. The Mobile Launcher is a very large heavy beam/truss steel structure designed to support the Space Launch System during its buildup and integration in the Vehicle Assembly Building, transportation between the Vehicle Assembly Building and launch pad 39B by the Crawler Transporter, and provides the launch platform at the launch pad. As part of the Verification and Validation of the Mobile Launcher and Crawler Transporter, two rollouts of the Mobile Launcher transported by the Crawler Transporter, Integrated System Verification and Validation (ISVV) 005 and 010, have been performed to demonstrate the Crawler Transporter’s ability to transport the Mobile Launcher. ISVV-005 occurred in September 2018 and ISVV-010 occurred in late June 2019. ISVV-005 and ISVV-010 also provided the opportunity to gather data that can be used identify the Mobile Launcher on the Crawler Transporter rollout modal characteristics and refine the estimates of the Artemis I integrated launch vehicle rollout forcing functions. While the rollout environment has historically produced relatively small launch vehicle structural loads for the Saturn/Apollo and Space Shuttle programs in comparison to launch and ascent loads, these relatively small structural loads are inputs to structural fatigue analyses. The same holds true for the Space Launch System. Because the rollout forces acting on the Mobile Launcher and the Crawler Transporter are not directly measurable, Operational Modal Analysis techniques, instead of traditional Experimental Modal Analysis techniques, provide an empirical means to identify the Mobile Launcher on the Crawler Transporter rollout modal characteristics. The ISVV-010 rollout modal characteristics provide important supplemental modal information, which along with the Mobile Launcher modal test that was performed in June 2019 immediately prior to ISVV-010 rollout, combine to reduce uncertainty in the test correlated Mobile Launcher on the Crawler Transporter finite element model. A well test correlated finite element model will play a key role in the Building Block approach the Space Launch System program has implemented as part of its certification process for the Artemis I flight and in refining the Artemis I rollout forcing functions. At the time of the ISVV-005 rollout in September 2018, the Mobile Launcher was still undergoing construction, and therefore its modal characteristics are not directly comparable to those of the Mobile Launcher during the June 2019 Mobile Launcher modal test and subsequent ISVV-010 rollout. Hence the ISVV-005 rollout modal characteristics will not be looked at in this paper. This paper will briefly describe the Mobile Launcher and Crawler Transporter physical characteristics, ISVV-010 rollout data collection, the challenges in implementing Operational Modal Analysis techniques due in part to the Crawler Transporter harmonics, and how these challenges were overcome to obtain the ISVV-010 Mobile Launcher on the Crawler Transporter rollout modal characteristics.

Apollo↗

NASA NTRSDate not supplied

Operational Modal Analysis of the Space Launch System Mobile Launcher on the Crawler Transporter ISVV-010 Rollout

NASA is developing an expendable heavy lift launch vehicle capability, the Space Launch System, to support lunar and deep space exploration. To support this capability, an updated ground infrastructure is required including modifying an existing Mobile Launcher system. The Mobile Launcher is a very large heavy beam/truss steel structure designed to support the Space Launch System during its buildup and integration in the Vehicle Assembly Building, transportation between the Vehicle Assembly Building and launch pad 39B by the Crawler Transporter, and provides the launch platform at the launch pad. As part of the Verification and Validation of the Mobile Launcher and Crawler Transporter, two rollouts of the Mobile Launcher transported by the Crawler Transporter, Integrated System Verification and Validation (ISVV) 005 and 010, have been performed to demonstrate the Crawler Transporter’s ability to transport the Mobile Launcher. ISVV-005 occurred in September 2018 and ISVV-010 occurred in late June 2019. ISVV-005 and ISVV-010 also provided the opportunity to gather data that can be used identify the Mobile Launcher on the Crawler Transporter rollout modal characteristics and refine the estimates of the Artemis I integrated launch vehicle rollout forcing functions. While the rollout environment has historically produced relatively small launch vehicle structural loads for the Saturn/Apollo and Space Shuttle programs in comparison to launch and ascent loads, these relatively small structural loads are inputs to structural fatigue analyses. The same holds true for the Space Launch System. Because the rollout forces acting on the Mobile Launcher and the Crawler Transporter are not directly measurable, Operational Modal Analysis techniques, instead of traditional Experimental Modal Analysis techniques, provide an empirical means to identify the Mobile Launcher on the Crawler Transporter rollout modal characteristics. The ISVV-010 rollout modal characteristics provide important supplemental modal information, which along with the Mobile Launcher modal test that was performed in June 2019 immediately prior to ISVV-010 rollout, combine to reduce uncertainty in the test correlated Mobile Launcher on the Crawler Transporter finite element model. A well test correlated finite element model will play a key role in the Building Block approach the Space Launch System program has implemented as part of its certification process for the Artemis I flight and in refining the Artemis I rollout forcing functions. At the time of the ISVV-005 rollout in September 2018, the Mobile Launcher was still undergoing construction, and therefore its modal characteristics are not directly comparable to those of the Mobile Launcher during the June 2019 Mobile Launcher modal test and subsequent ISVV-010 rollout. Hence the ISVV-005 rollout modal characteristics will not be looked at in this paper. This paper will briefly describe the Mobile Launcher and Crawler Transporter physical characteristics, ISVV-010 rollout data collection, the challenges in implementing Operational Modal Analysis techniques due in part to the Crawler Transporter harmonics, and how these challenges were overcome to obtain the ISVV-010 Mobile Launcher on the Crawler Transporter rollout modal characteristics.

Apollo↗

DOE OSTI2022

Analysis of Validating and Verifying OpenACC Compilers 3.0 and Above

OpenACC is a high-level directive-based parallel programming model that can manage the sophistication of heterogeneity in architectures and abstract it from the users. The portability of the model across CPUs and accelerators has gained the model a wide variety of users. This means it is also crucial to analyze the reliability of the compilers’ implementations. To address this challenge, the OpenACC Validation and Verification team has proposed a validation testsuite to verify the OpenACC implementations across various compilers with an infrastructure for a more streamlined execution. This paper will cover the following aspects: (a) the new developments since the last publication on the testsuite, (b) outline the use of the infrastructure, (c) discuss tests that highlight our workflow process, (d) analyze the results from executing the testsuite on various systems, and (e) outline future developments.

Jarmusch, Aaron↗

DOE OSTI2022

Evaluation of 3D Simulation Capabilities within xRAGE using a Suite of Test Problems

This presentation describes a 3D verification and validation suite for xRAGE. Three verification test problems for hydrodynamics are presented: the Kidder ball problem, the Verney shell problem, and a 5-material compression problem. There is excellent agreement between 2D and 3D XRAGE simulation results for these problems, and between the xRAGE results and the benchmark solutions. In addition, two 3D ICF test problems are presented, based on an OMEGA direct drive capsule experiment, and on a NIF indirect drive capsule experiment. These two ICF test problems are used to evaluate two hydrodynamic methods within xRAGE. In addition to hydrodynamics the simulations also include three temperature plasma physics, thermal conduction and radiation diffusion. When simulating either capsule in 3D the newer unsplit hydrodynamic method in xRAGE produces more vorticity relative to the older default method. For the indirect drive capsule the 3D simulations are in reasonable agreement with the experimental values of ion temperature and neutron production. Unstable 3D vortex rings are present near the DT ice/carbon interface which enhance the confinement of the DT ice.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

DOE OSTI2025

Adopting Code Verification Methodology Based on Model Form

Code verification is an essential part of credibility analysis for computational models. It assesses whether the mathematical model is implemented correctly into the code and whether the numerical methods behave consistently, and is done before solution verification and validation. Robust guidance for code verification exists in the literature. However, there is no known, concise guide for selecting the approach based on the model form that also presents an overview of the common elements. This document was written to address this gap as an accessible reference for beginning a code-verification effort.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2025

Adapting Code Verification Methodology to Model Form

Code verification is an essential part of credibility analysis for computational models. It assesses whether the mathematical model is implemented correctly into the code and whether the numerical methods behave consistently, and is done before solution verification and validation. Robust guidance for code verification exists in the literature. However, there is no known, concise guide for selecting the approach based on the model form that also presents an overview of the common elements. This document was written to address this gap as an accessible reference for beginning a code-verification effort.

97 MATHEMATICS AND COMPUTING↗

NASA NTRS2018

SLS Navigation Model-Based Design Approach

The SLS Program chose to implement a Model-based Design and Model-based Requirements approach for managing component design information and system requirements. This approach differs from previous large-scale design efforts at Marshall Space Flight Center where design documentation alone conveyed information required for vehicle design and analysis and where extensive requirements sets were used to scope and constrain the design. The SLS Navigation Team has been responsible for the Program-controlled Design Math Models (DMMs) which describe and represent the performance of the Inertial Navigation System (INS) and the Rate Gyro Assemblies (RGAs) used by Guidance, Navigation, and Controls (GN&C). The SLS Navigation Team is also responsible for the navigation algorithms. The navigation algorithms are delivered for implementation on the flight hardware as a DMM. For the SLS Block 1-B design, the additional GPS Receiver hardware is managed as a DMM at the vehicle design level. This paper provides a discussion of the processes and methods used to engineer, design, and coordinate engineering trades and performance assessments using SLS practices as applied to the GN&C system, with a particular focus on the Navigation components. These include composing system requirements, requirements verification, model development, model verification and validation, and modeling and analysis approaches. The Model-based Design and Requirements approach does not reduce the effort associated with the design process versus previous processes used at Marshall Space Flight Center. Instead, the approach takes advantage of overlap between the requirements development and management process, and the design and analysis process by efficiently combining the control (i.e. the requirement) and the design mechanisms. The design mechanism is the representation of the component behavior and performance in design and analysis tools. The focus in the early design process shifts from the development and management of design requirements to the development of usable models, model requirements, and model verification and validation efforts. The models themselves are represented in C/C++ code and accompanying data files. Under the idealized process, potential ambiguity in specification is reduced because the model must be implementable versus a requirement which is not necessarily subject to this constraint. Further, the models are shown to emulate the hardware during validation. For models developed by the Navigation Team, a common interface/standalone environment was developed. The common environment allows for easy implementation in design and analysis tools. Mechanisms such as unit test cases ensure implementation as the developer intended. The model verification and validation process provides a very high level of component design insight. The origin and implementation of the SLS variant of Model-based Design is described from the perspective of the SLS Navigation Team. The format of the models and the requirements are described. The Model-based Design approach has many benefits but is not without potential complications. Key lessons learned associated with the implementation of the Model Based Design approach and process from infancy to verification and certification are discussed

Oliver, T. Emerson↗

NASA NTRS1987

Approaches to the verification of rule-based expert systems

Expert systems are a highly useful spinoff of artificial intelligence research. One major stumbling block to extended use of expert systems is the lack of well-defined verification and validation (V and V) methodologies. Since expert systems are computer programs, the definitions of verification and validation from conventional software are applicable. The primary difficulty with expert systems is the use of development methodologies which do not support effective V and V. If proper techniques are used to document requirements, V and V of rule-based expert systems is possible, and may be easier than with conventional code. For NASA applications, the flight technique panels used in previous programs should provide an excellent way to verify the rules used in expert systems. There are, however, some inherent differences in expert systems that will affect V and V considerations.

Culbert, Chris↗

NASA NTRS1994

A Tool for Automatic Verification of Real-Time Expert Systems

The creation of an automated, user-driven tool for expert system development, validation, and verification is curretly onoging at NASA's Jet Propulsion Laboratory. In the new age of faster, better, cheaper missions, there is an increased willingness to utilize embedded expert systems for encapsulating and preserving mission expertise in systems which combine conventional algorithmic processing and artifical intelligence. The once-questioned role of automation in spacecraft monitoring is now becoming one of increasing importance.

real-time expert systems automated utilities knowl↗

NASA NTRSDate not supplied

Tradeoffs When Considering Deep Reinforcement Learning for Contingency Management in Advanced Air Mobility

Air transportation is undergoing a rapid evolution globally with the introduction of Advanced Air Mobility (AAM) and with it comes novel challenges and opportunities for transforming aviation. As AAM operations introduce increasing heterogeneity in vehicle capabilities and density, increased levels of automation are likely necessary to achieve operational safety and efficiency goals. This paper focuses on one example where increased automation has been suggested. Autonomous operations will need contingency management systems that can monitor evolving risk across a span of interrelated (or interdependent) hazards and, if necessary, execute appropriate control interventions via supervised or automated decision making. Accommodating this complex environment may require automated functions (autonomy) that apply artificial intelligence (AI) techniques that can adapt and respond to a quickly changing environment. This paper explores the use of Deep Reinforcement Learning (DRL) which has shown promising performance in complex and high-dimensional environments where the objective can be constructed as a sequential decision-making problem. An extension of a prior formulation of the contingency management problem as a Markov Decision Process (MDP) is presented and uses a DRL framework to train agents that mitigate hazards present in the simulation environment. A comparison of these learning-based agents and classical techniques is presented in terms of their performance, verification difficulties, and development process.

machine learningautonomous systems; flight simulat↗

DOE OSTI2024

Sierra/SD – Verification Test Manual – 5.22

Verification and validation (V&V) of scientific computing programs are important at Sandia National Labs due to the expanding role of computational simulation in managing the United States nuclear stockpile. The complexities of structural response calculations used to analyze physical problems, the varieties of codes applied to the calculations, and the importance of accurate predictions when assessing field conditions demand confidence in the consistency and accuracy of computer codes. Confidence in the accuracy of the predictions arising from computer simulations must ultimately be gained through verification and validation. The Sierra salinas structural dynamics analysis code, Sierra/SD, is used at the DOE Laboratories, and in several DOD projects. The roles of Sierra/SD in the qualification of weapon systems and components for normal and hostile environments throughout the Stockpile-to-Target Sequence include to, • Redesign weapon components. • Certify weapon components and systems for target environments such as hypersonic vehicles. • Certify that components will survive the thermal mechanical shock loads associated with hostile environments. • Evaluate current stockpile issues, including issues associated with uncertainty quantification. • Address many other problems that are encountered in stockpile management. The Sierra/SD verification plan is described, and an evolving set of key verification tests are described in detail. The verification tests ensure the correctness of the mathematics and numerical algorithms associated with functionality describing engineering phenomena. Development is in accordance with a set of tailored Software Quality Engineering (SQE) practices. SQE practices guide the overall verification and validation effort.

97 MATHEMATICS AND COMPUTING↗

NASA NTRS1985

Software verification and testing

General procedures for software verification and validation are provided as a guide for managers, programmers, and analysts involved in software development. The verification and validation procedures described are based primarily on testing techniques. Testing refers to the execution of all or part of a software system for the purpose of detecting errors. Planning, execution, and analysis of tests are outlined in this document. Code reading and static analysis techniques for software verification are also described.

Source record↗

NASA NTRS2005

Mars Exploration Rover Entry, Descent, and Landing: A Thermal Perspective

Perhaps the most challenging mission phase for the Mars Exploration Rovers was the Entry, Descent, and Landing (EDL). During this phase, the entry vehicle attached to its cruise stage was transformed into a stowed tetrahedral Lander that was surrounded by inflated airbags through a series of complex events. There was only one opportunity to successfully execute an automated command sequence without any possible ground intervention. The success of EDL was reliant upon the system thermal design: 1) to thermally condition EDL hardware from cruise storage temperatures to operating temperature ranges; 2) to maintain the Rover electronics within operating temperature ranges without the benefit of the cruise single phase cooling loop, which had been evacuated in preparation for EDL; and 3) to maintain the cruise stage propulsion components for the critical turn to entry attitude. Since the EDL architecture was inherited from Mars Pathfinder (MPF), the initial EDL thermal design would be inherited from MPF. However, hardware and implementation differences from MPF ultimately changed the MPF inheritance approach for the EDL thermal design. With the lack of full inheritance, the verification and validation of the EDL thermal design took on increased significance. This paper will summarize the verification and validation approach for the EDL thermal design along with applicable system level thermal testing results as well as appropriate thermal analyses. In addition, the lessons learned during the system-level testing will be discussed. Finally, the in-flight EDL experiences of both MER-A and -B missions (Spirit and Opportunity, respectively) will be presented, demonstrated how lessons learned from Spirit were applied to Opportunity.

thermal↗

DOE OSTI2024

Use of Grid-Forming Medium-Voltage Power Electronics Hub in a Microgrid Setting

This paper presents the application of a new design of a multiport, modular, medium-voltage power electronics hub (M3PE-HUB) in a microgrid setting. The M3PE-HUB system was modeled in a digital real-time simulator (DRTS) and integrated into the Banshee microgrid test system. This paper presents the preliminary DRTS simulation-based results of the M3PE-HUB system connected in a test microgrid system. Verification and validation of the M3PE-HUB architecture and controls in the test microgrid setting are the primary contributions of this work. The results of the operation during the islanding and resynchronization process indicate the feasibility of the proposed architecture in a microgrid setting. This paper also presents results for a system reconfiguration use case where the M3PE-HUB was used to reconfigure the system under a fault condition.

grid forming↗

NASA NTRS1998

Hyper-X Engine Design and Ground Test Program

The Hyper-X Program, NASA's focused hypersonic technology program jointly run by NASA Langley and Dryden, is designed to move hypersonic, air-breathing vehicle technology from the laboratory environment to the flight environment, the last stage preceding prototype development. The Hyper-X research vehicle will provide the first ever opportunity to obtain data on an airframe integrated supersonic combustion ramjet propulsion system in flight, providing the first flight validation of wind tunnel, numerical and analytical methods used for design of these vehicles. A substantial portion of the integrated vehicle/engine flowpath development, engine systems verification and validation and flight test risk reduction efforts are experimentally based, including vehicle aeropropulsive force and moment database generation for flight control law development, and integrated vehicle/engine performance validation. The Mach 7 engine flowpath development tests have been completed, and effort is now shifting to engine controls, systems and performance verification and validation tests, as well as, additional flight test risk reduction tests. The engine wind tunnel tests required for these efforts range from tests of partial width engines in both small and large scramjet test facilities, to tests of the full flight engine on a vehicle simulator and tests of a complete flight vehicle in the Langley 8-Ft. High Temperature Tunnel. These tests will begin in the summer of 1998 and continue through 1999. The first flight test is planned for early 2000.

Voland, R. T.↗

NASA NTRS2017

Fuzzy Logic Controller Stability Analysis Using a Satisfiability Modulo Theories Approach

While many widely accepted methods and techniques exist for validation and verification of traditional controllers, at this time no solutions have been accepted for Fuzzy Logic Controllers (FLCs). Due to the highly nonlinear nature of such systems, and the fact that developing a valid FLC does not require a mathematical model of the system, it is quite difficult to use conventional techniques to prove controller stability. Since safety-critical systems must be tested and verified to work as expected for all possible circumstances, the fact that FLC controllers cannot be tested to achieve such requirements poses limitations on the applications for such technology. Therefore, alternative methods for verification and validation of FLCs needs to be explored. In this study, a novel approach using formal verification methods to ensure the stability of a FLC is proposed. Main research challenges include specification of requirements for a complex system, conversion of a traditional FLC to a piecewise polynomial representation, and using a formal verification tool in a nonlinear solution space. Using the proposed architecture, the Fuzzy Logic Controller was found to always generate negative feedback, but inconclusive for Lyapunov stability.

Fuzzy Logic Controller↗