Results for “verification and validation” — Page 19

NASA NTRS2011

CFD Validation with Experiment and Verification with Physics of a Propellant Damping Device

This paper will document our effort in validating a coupled fluid-structure interaction CFD tool in predicting a damping device performance in the laboratory condition. Consistently good comparisons of "blind" CFD predictions against experimental data under various operation conditions, design parameters, and cryogenic environment will be presented. The power of the coupled CFD-structures interaction code in explaining some unexpected phenomena of the device observed during the technology development will be illustrated. The evolution of the damper device design inside the LOX tank will be used to demonstrate the contribution of the tool in understanding, optimization and implementation of LOX damper in Ares I vehicle. It is due to the present validation effort, the LOX damper technology has matured to TRL 5. The present effort has also contributed to the transition of the technology from an early conceptual observation to the baseline design of thrust oscillation mitigation for the Ares I within a 10 month period.

Yang, H. Q.↗

NASA NTRS2016

Development and Use of Engineering Standards for Computational Fluid Dynamics for Complex Aerospace Systems

Computational fluid dynamics (CFD) and other advanced modeling and simulation (M&S) methods are increasingly relied on for predictive performance, reliability and safety of engineering systems. Analysts, designers, decision makers, and project managers, who must depend on simulation, need practical techniques and methods for assessing simulation credibility. The AIAA Guide for Verification and Validation of Computational Fluid Dynamics Simulations (AIAA G-077-1998 (2002)), originally published in 1998, was the first engineering standards document available to the engineering community for verification and validation (V&V) of simulations. Much progress has been made in these areas since 1998. The AIAA Committee on Standards for CFD is currently updating this Guide to incorporate in it the important developments that have taken place in V&V concepts, methods, and practices, particularly with regard to the broader context of predictive capability and uncertainty quantification (UQ) methods and approaches. This paper will provide an overview of the changes and extensions currently underway to update the AIAA Guide. Specifically, a framework for predictive capability will be described for incorporating a wide range of error and uncertainty sources identified during the modeling, verification, and validation processes, with the goal of estimating the total prediction uncertainty of the simulation. The Guide's goal is to provide a foundation for understanding and addressing major issues and concepts in predictive CFD. However, this Guide will not recommend specific approaches in these areas as the field is rapidly evolving. It is hoped that the guidelines provided in this paper, and explained in more detail in the Guide, will aid in the research, development, and use of CFD in engineering decision-making.

Lee, Hyung B.↗

NASA NTRS1990

Third Annual Workshop on Space Operations Automation and Robotics (SOAR 1989)

Papers presented at the Third Annual Workshop on Space Operations Automation and Robotics (SOAR '89), hosted by the NASA Lyndon 8. Johnson Space Center at Houston, Texas, on July 25-27, 1989, are documented herein. During the three days, approximately 100 technical papers wer3 presented by experts from NASA, the USAF, universities, and technical companies. Also held were panel discussions on Air Force/NASA AI Overview and Expert System Verification and Validation. Tutorial sessions included Neural Networks; Theory and Application of Back Propagation; Verification and Validation of Expert Systems/ Evaluation of Expert System Tools; and Technical Environment for Modular Architectures for Robotics in Space; and are not documented herein. Technical topics addressed included intelligent systems, robotics, human factors, and environment.

Knowledge representation↗

NASA NTRS2021

Operational Modal Analysis of the Space Launch System Mobile Launcher on the Crawler Transporter ISVV-010 Rollout

NASA is developing an expendable heavy lift launch vehicle capability, the Space Launch System, to support lunar and deep space exploration. To support this capability, an updated ground infrastructure is required including modifying an existing Mobile Launcher system. The Mobile Launcher is a very large heavy beam/truss steel structure designed to support the Space Launch System during its buildup and integration in the Vehicle Assembly Building, transportation between the Vehicle Assembly Building and launch pad 39B by the Crawler Transporter, and provides the launch platform at the launch pad. As part of the Verification and Validation of the Mobile Launcher and Crawler Transporter, two rollouts of the Mobile Launcher transported by the Crawler Transporter, Integrated System Verification and Validation (ISVV) 005 and 010, have been performed to demonstrate the Crawler Transporter’s ability to transport the Mobile Launcher. ISVV-005 occurred in September 2018 and ISVV-010 occurred in late June 2019. ISVV-005 and ISVV-010 also provided the opportunity to gather data that can be used identify the Mobile Launcher on the Crawler Transporter rollout modal characteristics and refine the estimates of the Artemis I integrated launch vehicle rollout forcing functions. While the rollout environment has historically produced relatively small launch vehicle structural loads for the Saturn/Apollo and Space Shuttle programs in comparison to launch and ascent loads, these relatively small structural loads are inputs to structural fatigue analyses. The same holds true for the Space Launch System. Because the rollout forces acting on the Mobile Launcher and the Crawler Transporter are not directly measurable, Operational Modal Analysis techniques, instead of traditional Experimental Modal Analysis techniques, provide an empirical means to identify the Mobile Launcher on the Crawler Transporter rollout modal characteristics. The ISVV-010 rollout modal characteristics provide important supplemental modal information, which along with the Mobile Launcher modal test that was performed in June 2019 immediately prior to ISVV-010 rollout, combine to reduce uncertainty in the test correlated Mobile Launcher on the Crawler Transporter finite element model. A well test correlated finite element model will play a key role in the Building Block approach the Space Launch System program has implemented as part of its certification process for the Artemis I flight and in refining the Artemis I rollout forcing functions. At the time of the ISVV-005 rollout in September 2018, the Mobile Launcher was still undergoing construction, and therefore its modal characteristics are not directly comparable to those of the Mobile Launcher during the June 2019 Mobile Launcher modal test and subsequent ISVV-010 rollout. Hence the ISVV-005 rollout modal characteristics will not be looked at in this paper. This paper will briefly describe the Mobile Launcher and Crawler Transporter physical characteristics, ISVV-010 rollout data collection, the challenges in implementing Operational Modal Analysis techniques due in part to the Crawler Transporter harmonics, and how these challenges were overcome to obtain the ISVV-010 Mobile Launcher on the Crawler Transporter rollout modal characteristics.

Apollo↗

NASA NTRSDate not supplied

Operational Modal Analysis of the Space Launch System Mobile Launcher on the Crawler Transporter ISVV-010 Rollout

NASA is developing an expendable heavy lift launch vehicle capability, the Space Launch System, to support lunar and deep space exploration. To support this capability, an updated ground infrastructure is required including modifying an existing Mobile Launcher system. The Mobile Launcher is a very large heavy beam/truss steel structure designed to support the Space Launch System during its buildup and integration in the Vehicle Assembly Building, transportation between the Vehicle Assembly Building and launch pad 39B by the Crawler Transporter, and provides the launch platform at the launch pad. As part of the Verification and Validation of the Mobile Launcher and Crawler Transporter, two rollouts of the Mobile Launcher transported by the Crawler Transporter, Integrated System Verification and Validation (ISVV) 005 and 010, have been performed to demonstrate the Crawler Transporter’s ability to transport the Mobile Launcher. ISVV-005 occurred in September 2018 and ISVV-010 occurred in late June 2019. ISVV-005 and ISVV-010 also provided the opportunity to gather data that can be used identify the Mobile Launcher on the Crawler Transporter rollout modal characteristics and refine the estimates of the Artemis I integrated launch vehicle rollout forcing functions. While the rollout environment has historically produced relatively small launch vehicle structural loads for the Saturn/Apollo and Space Shuttle programs in comparison to launch and ascent loads, these relatively small structural loads are inputs to structural fatigue analyses. The same holds true for the Space Launch System. Because the rollout forces acting on the Mobile Launcher and the Crawler Transporter are not directly measurable, Operational Modal Analysis techniques, instead of traditional Experimental Modal Analysis techniques, provide an empirical means to identify the Mobile Launcher on the Crawler Transporter rollout modal characteristics. The ISVV-010 rollout modal characteristics provide important supplemental modal information, which along with the Mobile Launcher modal test that was performed in June 2019 immediately prior to ISVV-010 rollout, combine to reduce uncertainty in the test correlated Mobile Launcher on the Crawler Transporter finite element model. A well test correlated finite element model will play a key role in the Building Block approach the Space Launch System program has implemented as part of its certification process for the Artemis I flight and in refining the Artemis I rollout forcing functions. At the time of the ISVV-005 rollout in September 2018, the Mobile Launcher was still undergoing construction, and therefore its modal characteristics are not directly comparable to those of the Mobile Launcher during the June 2019 Mobile Launcher modal test and subsequent ISVV-010 rollout. Hence the ISVV-005 rollout modal characteristics will not be looked at in this paper. This paper will briefly describe the Mobile Launcher and Crawler Transporter physical characteristics, ISVV-010 rollout data collection, the challenges in implementing Operational Modal Analysis techniques due in part to the Crawler Transporter harmonics, and how these challenges were overcome to obtain the ISVV-010 Mobile Launcher on the Crawler Transporter rollout modal characteristics.

Apollo↗

NASA NTRS2018

SLS Navigation Model-Based Design Approach

The SLS Program chose to implement a Model-based Design and Model-based Requirements approach for managing component design information and system requirements. This approach differs from previous large-scale design efforts at Marshall Space Flight Center where design documentation alone conveyed information required for vehicle design and analysis and where extensive requirements sets were used to scope and constrain the design. The SLS Navigation Team has been responsible for the Program-controlled Design Math Models (DMMs) which describe and represent the performance of the Inertial Navigation System (INS) and the Rate Gyro Assemblies (RGAs) used by Guidance, Navigation, and Controls (GN&C). The SLS Navigation Team is also responsible for the navigation algorithms. The navigation algorithms are delivered for implementation on the flight hardware as a DMM. For the SLS Block 1-B design, the additional GPS Receiver hardware is managed as a DMM at the vehicle design level. This paper provides a discussion of the processes and methods used to engineer, design, and coordinate engineering trades and performance assessments using SLS practices as applied to the GN&C system, with a particular focus on the Navigation components. These include composing system requirements, requirements verification, model development, model verification and validation, and modeling and analysis approaches. The Model-based Design and Requirements approach does not reduce the effort associated with the design process versus previous processes used at Marshall Space Flight Center. Instead, the approach takes advantage of overlap between the requirements development and management process, and the design and analysis process by efficiently combining the control (i.e. the requirement) and the design mechanisms. The design mechanism is the representation of the component behavior and performance in design and analysis tools. The focus in the early design process shifts from the development and management of design requirements to the development of usable models, model requirements, and model verification and validation efforts. The models themselves are represented in C/C++ code and accompanying data files. Under the idealized process, potential ambiguity in specification is reduced because the model must be implementable versus a requirement which is not necessarily subject to this constraint. Further, the models are shown to emulate the hardware during validation. For models developed by the Navigation Team, a common interface/standalone environment was developed. The common environment allows for easy implementation in design and analysis tools. Mechanisms such as unit test cases ensure implementation as the developer intended. The model verification and validation process provides a very high level of component design insight. The origin and implementation of the SLS variant of Model-based Design is described from the perspective of the SLS Navigation Team. The format of the models and the requirements are described. The Model-based Design approach has many benefits but is not without potential complications. Key lessons learned associated with the implementation of the Model Based Design approach and process from infancy to verification and certification are discussed

Oliver, T. Emerson↗

NASA NTRS1987

Approaches to the verification of rule-based expert systems

Expert systems are a highly useful spinoff of artificial intelligence research. One major stumbling block to extended use of expert systems is the lack of well-defined verification and validation (V and V) methodologies. Since expert systems are computer programs, the definitions of verification and validation from conventional software are applicable. The primary difficulty with expert systems is the use of development methodologies which do not support effective V and V. If proper techniques are used to document requirements, V and V of rule-based expert systems is possible, and may be easier than with conventional code. For NASA applications, the flight technique panels used in previous programs should provide an excellent way to verify the rules used in expert systems. There are, however, some inherent differences in expert systems that will affect V and V considerations.

Culbert, Chris↗

NASA NTRS1994

A Tool for Automatic Verification of Real-Time Expert Systems

The creation of an automated, user-driven tool for expert system development, validation, and verification is curretly onoging at NASA's Jet Propulsion Laboratory. In the new age of faster, better, cheaper missions, there is an increased willingness to utilize embedded expert systems for encapsulating and preserving mission expertise in systems which combine conventional algorithmic processing and artifical intelligence. The once-questioned role of automation in spacecraft monitoring is now becoming one of increasing importance.

real-time expert systems automated utilities knowl↗

NASA NTRSDate not supplied

Tradeoffs When Considering Deep Reinforcement Learning for Contingency Management in Advanced Air Mobility

Air transportation is undergoing a rapid evolution globally with the introduction of Advanced Air Mobility (AAM) and with it comes novel challenges and opportunities for transforming aviation. As AAM operations introduce increasing heterogeneity in vehicle capabilities and density, increased levels of automation are likely necessary to achieve operational safety and efficiency goals. This paper focuses on one example where increased automation has been suggested. Autonomous operations will need contingency management systems that can monitor evolving risk across a span of interrelated (or interdependent) hazards and, if necessary, execute appropriate control interventions via supervised or automated decision making. Accommodating this complex environment may require automated functions (autonomy) that apply artificial intelligence (AI) techniques that can adapt and respond to a quickly changing environment. This paper explores the use of Deep Reinforcement Learning (DRL) which has shown promising performance in complex and high-dimensional environments where the objective can be constructed as a sequential decision-making problem. An extension of a prior formulation of the contingency management problem as a Markov Decision Process (MDP) is presented and uses a DRL framework to train agents that mitigate hazards present in the simulation environment. A comparison of these learning-based agents and classical techniques is presented in terms of their performance, verification difficulties, and development process.

machine learningautonomous systems; flight simulat↗

NASA NTRS1985

Software verification and testing

General procedures for software verification and validation are provided as a guide for managers, programmers, and analysts involved in software development. The verification and validation procedures described are based primarily on testing techniques. Testing refers to the execution of all or part of a software system for the purpose of detecting errors. Planning, execution, and analysis of tests are outlined in this document. Code reading and static analysis techniques for software verification are also described.

Source record↗

NASA NTRS2005

Mars Exploration Rover Entry, Descent, and Landing: A Thermal Perspective

Perhaps the most challenging mission phase for the Mars Exploration Rovers was the Entry, Descent, and Landing (EDL). During this phase, the entry vehicle attached to its cruise stage was transformed into a stowed tetrahedral Lander that was surrounded by inflated airbags through a series of complex events. There was only one opportunity to successfully execute an automated command sequence without any possible ground intervention. The success of EDL was reliant upon the system thermal design: 1) to thermally condition EDL hardware from cruise storage temperatures to operating temperature ranges; 2) to maintain the Rover electronics within operating temperature ranges without the benefit of the cruise single phase cooling loop, which had been evacuated in preparation for EDL; and 3) to maintain the cruise stage propulsion components for the critical turn to entry attitude. Since the EDL architecture was inherited from Mars Pathfinder (MPF), the initial EDL thermal design would be inherited from MPF. However, hardware and implementation differences from MPF ultimately changed the MPF inheritance approach for the EDL thermal design. With the lack of full inheritance, the verification and validation of the EDL thermal design took on increased significance. This paper will summarize the verification and validation approach for the EDL thermal design along with applicable system level thermal testing results as well as appropriate thermal analyses. In addition, the lessons learned during the system-level testing will be discussed. Finally, the in-flight EDL experiences of both MER-A and -B missions (Spirit and Opportunity, respectively) will be presented, demonstrated how lessons learned from Spirit were applied to Opportunity.

thermal↗

NASA NTRS1998

Hyper-X Engine Design and Ground Test Program

The Hyper-X Program, NASA's focused hypersonic technology program jointly run by NASA Langley and Dryden, is designed to move hypersonic, air-breathing vehicle technology from the laboratory environment to the flight environment, the last stage preceding prototype development. The Hyper-X research vehicle will provide the first ever opportunity to obtain data on an airframe integrated supersonic combustion ramjet propulsion system in flight, providing the first flight validation of wind tunnel, numerical and analytical methods used for design of these vehicles. A substantial portion of the integrated vehicle/engine flowpath development, engine systems verification and validation and flight test risk reduction efforts are experimentally based, including vehicle aeropropulsive force and moment database generation for flight control law development, and integrated vehicle/engine performance validation. The Mach 7 engine flowpath development tests have been completed, and effort is now shifting to engine controls, systems and performance verification and validation tests, as well as, additional flight test risk reduction tests. The engine wind tunnel tests required for these efforts range from tests of partial width engines in both small and large scramjet test facilities, to tests of the full flight engine on a vehicle simulator and tests of a complete flight vehicle in the Langley 8-Ft. High Temperature Tunnel. These tests will begin in the summer of 1998 and continue through 1999. The first flight test is planned for early 2000.

Voland, R. T.↗

NASA NTRS2017

Fuzzy Logic Controller Stability Analysis Using a Satisfiability Modulo Theories Approach

While many widely accepted methods and techniques exist for validation and verification of traditional controllers, at this time no solutions have been accepted for Fuzzy Logic Controllers (FLCs). Due to the highly nonlinear nature of such systems, and the fact that developing a valid FLC does not require a mathematical model of the system, it is quite difficult to use conventional techniques to prove controller stability. Since safety-critical systems must be tested and verified to work as expected for all possible circumstances, the fact that FLC controllers cannot be tested to achieve such requirements poses limitations on the applications for such technology. Therefore, alternative methods for verification and validation of FLCs needs to be explored. In this study, a novel approach using formal verification methods to ensure the stability of a FLC is proposed. Main research challenges include specification of requirements for a complex system, conversion of a traditional FLC to a piecewise polynomial representation, and using a formal verification tool in a nonlinear solution space. Using the proposed architecture, the Fuzzy Logic Controller was found to always generate negative feedback, but inconclusive for Lyapunov stability.

Fuzzy Logic Controller↗

NASA NTRS2017

Fuzzy Logic Controller Stability Analysis Using a Satisfiability Modulo Theories Approach

While many widely accepted methods and techniques exist for validation and verification of traditional controllers, at this time no solutions have been accepted for Fuzzy Logic Controllers (FLCs). Due to the highly nonlinear nature of such systems, and the fact that developing a valid FLC does not require a mathematical model of the system, it is quite difficult to use conventional techniques to prove controller stability. Since safety-critical systems must be tested and verified to work as expected for all possible circumstances, the fact that FLC controllers cannot be tested to achieve such requirements poses limitations on the applications for such technology. Therefore, alternative methods for verification and validation of FLCs needs to be explored. In this study, a novel approach using formal verification methods to ensure the stability of a FLC is proposed. Main research challenges include specification of requirements for a complex system, conversion of a traditional FLC to a piecewise polynomial representation, and using a formal verification tool in a nonlinear solution space. Using the proposed architecture, the Fuzzy Logic Controller was found to always generate negative feedback, but inconclusive for Lyapunov stability.

Fuzzy Logic Controller↗

NASA NTRS1998

Towards a Theory for Integration of Mathematical Verification and Empirical Testing

From the viewpoint of a project manager responsible for the V&V (verification and validation) of a software system, mathematical verification techniques provide a possibly useful orthogonal dimension to otherwise standard empirical testing. However, the value they add to an empirical testing regime both in terms of coverage and in fault detection has been difficult to quantify. Furthermore, potential cost savings from replacing testing with mathematical verification techniques cannot be realized until the tradeoffs and synergies can be formulated. Integration of formal verification with empirical testing is also difficult because the idealized view of mathematical verification providing a correctness proof with total coverage is unrealistic and does not reflect the limitations imposed by computational complexity of mathematical techniques. This paper first describes a framework based on software reliability and formalized fault models for a theory of software design fault detection - and hence the utility of various tools for debugging. It then describes a utility model for integrating mathematical and empirical techniques with respect to fault detection and coverage analysis. It then considers the optimal combination of black-box testing, white-box (structural) testing, and formal methods in V&V of a software system. Using case studies from NASA software systems, it then demonstrates how this utility model can be used in practice.

Lowry, Michael↗

NASA NTRS2005

Architecture for Verifiable Software

Verifiable MDS Architecture (VMA) is a software architecture that facilitates the construction of highly verifiable flight software for NASA s Mission Data System (MDS), especially for smaller missions subject to cost constraints. More specifically, the purpose served by VMA is to facilitate aggressive verification and validation of flight software while imposing a minimum of constraints on overall functionality. VMA exploits the state-based architecture of the MDS and partitions verification issues into elements susceptible to independent verification and validation, in such a manner that scaling issues are minimized, so that relatively large software systems can be aggressively verified in a cost-effective manner.

Reinholtz, William↗

NASA NTRS2010

L(sub 1) Adaptive Flight Control System: Flight Evaluation and Technology Transition

Certification of adaptive control technologies for both manned and unmanned aircraft represent a major challenge for current Verification and Validation techniques. A (missing) key step towards flight certification of adaptive flight control systems is the definition and development of analysis tools and methods to support Verification and Validation for nonlinear systems, similar to the procedures currently used for linear systems. In this paper, we describe and demonstrate the advantages of L(sub l) adaptive control architectures for closing some of the gaps in certification of adaptive flight control systems, which may facilitate the transition of adaptive control into military and commercial aerospace applications. As illustrative examples, we present the results of a piloted simulation evaluation on the NASA AirSTAR flight test vehicle, and results of an extensive flight test program conducted by the Naval Postgraduate School to demonstrate the advantages of L(sub l) adaptive control as a verifiable robust adaptive flight control system.

Xargay, Enric↗

NASA NTRSDate not supplied

Psyche Hardware Abstraction Simulation Environment (PHASE): Development of High-Fidelity Downlink Model for Flight Software Performance Testing

NASA’s Katherine Johnson Independent Verification and Validation (IV&V) Independent Test Capability (ITC) Team develops high fidelity software-only simulations (or “digital-twins”) of NASA’s most critical missions that are then used for software verification and validation (V&V). One such digital-twin simulator nearing maturity is the Psyche Hardware Abstraction Simulation Environment (PHASE). PHASE utilizes an instruction set simulator along with ITC-developed avionics models. PHASE includes a newly-developed model of the telecom hardware with a high-fidelity downlink interface, capable of supporting reconfigurable downlink data rates and transmission modes. The different downlink transmission modes allow more processing to be ceded from software onto the telecom hardware. Thus, we are able to monitor the performance of the downlink software module in the time-partitioned OS and perform more realistic dynamic testing based on software’s resource utilization. This presentation will cover PHASE, avionics modeling, the downlink model, and its usage for flight software performance testing.

modeling↗