SEARCH · Engineering Papers

Results for “Verification and validation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 163 records · Page 9

All sources NASA NTRS DOE OSTI

NASA NTRS1994

Experiences in improving the state of the practice in verification and validation of knowledge-based systems

Knowledge-based systems (KBS's) are in general use in a wide variety of domains, both commercial and government. As reliance on these types of systems grows, the need to assess their quality and validity reaches critical importance. As with any software, the reliability of a KBS can be directly attributed to the application of disciplined programming and testing practices throughout the development life-cycle. However, there are some essential differences between conventional software and KBSs, both in construction and use. The identification of these differences affect the verification and validation (V&V) process and the development of techniques to handle them. The recognition of these differences is the basis of considerable on-going research in this field. For the past three years IBM (Federal Systems Company - Houston) and the Software Technology Branch (STB) of NASA/Johnson Space Center have been working to improve the 'state of the practice' in V&V of Knowledge-based systems. This work was motivated by the need to maintain NASA's ability to produce high quality software while taking advantage of new KBS technology. To date, the primary accomplishment has been the development and teaching of a four-day workshop on KBS V&V. With the hope of improving the impact of these workshops, we also worked directly with NASA KBS projects to employ concepts taught in the workshop. This paper describes two projects that were part of this effort. In addition to describing each project, this paper describes problems encountered and solutions proposed in each case, with particular emphasis on implications for transferring KBS V&V technology beyond the NASA domain.

Culbert, Chris↗

NASA NTRS2016

Development, Verification and Validation of Enclosure Radiation Capabilities in the CHarring Ablator Response (CHAR) Code

With the recent development of multi-dimensional thermal protection system (TPS) material response codes including the capabilities to account for radiative heating is a requirement. This paper presents the recent efforts to implement such capabilities in the CHarring Ablator Response (CHAR) code developed at NASA's Johnson Space Center. This work also describes the different numerical methods implemented in the code to compute view factors for radiation problems involving multiple surfaces. Furthermore, verification and validation of the code's radiation capabilities are demonstrated by comparing solutions to analytical results, to other codes, and to radiant test data.

Salazar, Giovanni↗

NASA NTRSDate not supplied

Analysis of the Value Added When Deploying a Model-Based Approach for the Validation and Verification of the Medical Database Software

The Medical Database (MD) is a virtual repository consisting of two software components: Medical Item Database (MedID) and the Evidence Library (EL). MedID consists of engineering data and associated information for specific medical resource items (e.g., pharmaceutical, medical devices, and supporting components), while the EL is a tool which provides all of the medical evidence necessary. The MD will 1) serve as the single “source of truth” for the Informing Mission Planning via Analysis of Complex Tradespaces (IMPACT) tool suite for both medical evidence and medical resource engineering data and 2) will be used in conjunction with the IMPACT tool suite to inform research prioritizations and perform systematic trade study evaluations to aid stakeholders in making informed decisions regarding simulated human spaceflight missions. The MD project used a Model-Based Systems Engineering (MBSE) approach to support all life cycles of the software development, while in parallel the human factors engineering team used modeling to support Human Centered Design (HCD) strategies in an effort to improve software usability. HCD is a frequently used approach in design frameworks that develops resolutions to complexities and challenges by involving the human perspective in all steps of the problem-solving process. By integrating the model-based approaches used for systems engineering and human factors activities, the project is able to leverage the model-based artifacts originally created for HCD activities for system level and human factors validation. In this presentation, our team highlights the value added when leveraging these model-based artifacts to support the on-going verification and validation activities.

C. Laing↗

NASA NTRS2017

Unmanned Aircraft Systems Detect and Avoid System: End-to-End Verification and Validation Simulation Study of Minimum Operations Performance Standards for Integrating Unmanned Aircraft into the National Airspace System

As Unmanned Aircraft Systems (UAS) make their way to mainstream aviation operations within the National Airspace System (NAS), research efforts are underway to develop a safe and effective environment for their integration into the NAS. Detect and Avoid (DAA) systems are required to account for the lack of "eyes in the sky" due to having no human on-board the aircraft. The technique, results, and lessons learned from a detailed End-to-End Verification and Validation (E2-V2) simulation study of a DAA system representative of RTCA SC-228's proposed Phase I DAA Minimum Operational Performance Standards (MOPS), based on specific test vectors and encounter cases, will be presented in this paper.

Ghatas, Rania W.↗

NASA NTRS2017

The Effects of Severity of Losses of Well Clear on Minimum Operations Performance Standards End-To-End Verification and Validation Simulation Study for Integrating Unmanned Aircraft Systems into the National Airspace System Using Detect and Avoid Systems

As Unmanned Aircraft Systems (UAS) make their way to mainstream aviation operations within the National Airspace System (NAS), research efforts are underway to develop a safe and effective environment for their integration into the NAS. Detect and Avoid (DAA) systems are required to account for the lack of “eyes in the sky” due to having no human on-board the aircraft. The technique, results, and lessons learned from a detailed End-to-End Verification and Validation (E2-V2) simulation study of a DAA system representative of RTCA Special Committee(SC)-228’s proposed Phase I DAA Minimum Operational Performance Standards (MOPS), based on specific test vectors and encounter cases, will be presented in this paper.

Ghatas, Rania W.↗

NASA NTRS2017

Applying Standard Independent Verification and Validation (IVV) Techniques Within an Agile Framework: Is There a Compatibility Issue?

Agile methods have gained wide acceptance over the past several years, to the point that they are now a standard management and execution approach for small-scale software development projects. While conventional Agile methods are not generally applicable to large multi-year and mission-critical systems, Agile hybrids are now being developed (such as SAFe) to exploit the productivity improvements of Agile while retaining the necessary process rigor and coordination needs of these projects. From the perspective of Independent Verification and Validation (IVV), however, the adoption of these hybrid Agile frameworks is becoming somewhat problematic. Hence, we find it prudent to question the compatibility of conventional IVV techniques with (hybrid) Agile practices.This paper documents our investigation of (a) relevant literature, (b) the modification and adoption of Agile frameworks to accommodate the development of large scale, mission critical systems, and (c) the compatibility of standard IVV techniques within hybrid Agile development frameworks. Specific to the latter, we found that the IVV methods employed within a hybrid Agile process can be divided into three groups: (1) early lifecycle IVV techniques that are fully compatible with the hybrid lifecycles, (2) IVV techniques that focus on tracing requirements, test objectives, etc. are somewhat incompatible, but can be tailored with a modest effort, and (3) IVV techniques involving an assessment requiring artifact completeness that are simply not compatible with hybrid Agile processes, e.g., those that assume complete requirement specification early in the development lifecycle.

Agile↗

NASA NTRS1996

Verification and Validation of KBS with Neural Network Components

Artificial Neural Network (ANN) play an important role in developing robust Knowledge Based Systems (KBS). The ANN based components used in these systems learn to give appropriate predictions through training with correct input-output data patterns. Unlike traditional KBS that depends on a rule database and a production engine, the ANN based system mimics the decisions of an expert without specifically formulating the if-than type of rules. In fact, the ANNs demonstrate their superiority when such if-then type of rules are hard to generate by human expert. Verification of traditional knowledge based system is based on the proof of consistency and completeness of the rule knowledge base and correctness of the production engine.These techniques, however, can not be directly applied to ANN based components.In this position paper, we propose a verification and validation procedure for KBS with ANN based components. The essence of the procedure is to obtain an accurate system specification through incremental modification of the specifications using an ANN rule extraction algorithm.

Wen, Wu↗

NASA NTRS2001

RIACS Workshop on the Verification and Validation of Autonomous and Adaptive Systems

The long-term future of space exploration at NASA is dependent on the full exploitation of autonomous and adaptive systems: careful monitoring of missions from earth, as is the norm now, will be infeasible due to the sheer number of proposed missions and the communication lag for deep-space missions. Mission managers are however worried about the reliability of these more intelligent systems. The main focus of the workshop was to address these worries and hence we invited NASA engineers working on autonomous and adaptive systems and researchers interested in the verification and validation (V&V) of software systems. The dual purpose of the meeting was to: (1) make NASA engineers aware of the V&V techniques they could be using; and (2) make the V&V community aware of the complexity of the systems NASA is developing.

Pecheur, Charles↗

NASA NTRS2006

Verification and Validation of NASA-Supported Enhancements to PECAD's Decision Support Tools

The NASA Applied Sciences Directorate (ASD), part of the Earth-Sun System Division of NASA's Science Mission Directorate, has partnered with the U.S. Department of Agriculture (USDA) to enhance decision support in the area of agricultural efficiency-an application of national importance. The ASD integrated the results of NASA Earth science research into USDA decision support tools employed by the USDA Foreign Agricultural Service (FAS) Production Estimates and Crop Assessment Division (PECAD), which supports national decision making by gathering, analyzing, and disseminating global crop intelligence. Verification and validation of the following enhancements are summarized: 1) Near-real-time Moderate Resolution Imaging Spectroradiometer (MODIS) products through PECAD's MODIS Image Gallery; 2) MODIS Normalized Difference Vegetation Index (NDVI) time series data through the USDA-FAS MODIS NDVI Database; and 3) Jason-1 and TOPEX/Poseidon lake level estimates through PECAD's Global Reservoir and Lake Monitor. Where possible, each enhanced product was characterized for accuracy, timeliness, and coverage, and the characterized performance was compared to PECAD operational requirements. The MODIS Image Gallery and the GRLM are more mature and have achieved a semi-operational status, whereas the USDA-FAS MODIS NDVI Database is still evolving and should be considered

McKellipo, Rodney↗

DOE OSTI2026

Georgetown University – SYSM 5630 Systems Integration Verification and Validation : Todd Noste

At Lawrence Livermore National Laboratory in the National Ignition Facility Optics Group, we use the systems engineering approach for project management and as a design tool. Systems engineering is used in a graded approach to design and project management that is based on risk, informing how much rigor to apply. The tools and techniques from systems engineering offer a framework to organize projects with everyone speaking the same language to provide consistent and repeatable project success that satisfies the stakeholders’ needs and meets the mission. The classes have provided a framework with tools for communicating system design, requirements, verification and validation, and an operational context.

42 ENGINEERING↗

NASA NTRS2009

Verification and Validation of Adaptive and Intelligent Systems with Flight Test Results

F-15 IFCS project goals are: a) Demonstrate Control Approaches that can Efficiently Optimize Aircraft Performance in both Normal and Failure Conditions [A] & [B] failures. b) Advance Neural Network-Based Flight Control Technology for New Aerospace Systems Designs with a Pilot in the Loop. Gen II objectives include; a) Implement and Fly a Direct Adaptive Neural Network Based Flight Controller; b) Demonstrate the Ability of the System to Adapt to Simulated System Failures: 1) Suppress Transients Associated with Failure; 2) Re-Establish Sufficient Control and Handling of Vehicle for Safe Recovery. c) Provide Flight Experience for Development of Verification and Validation Processes for Flight Critical Neural Network Software.

Burken, John J.↗

NASA NTRS2010

Validation and Verification of Future Integrated Safety-Critical Systems Operating under Off-Nominal Conditions

Loss of control remains one of the largest contributors to aircraft fatal accidents worldwide. Aircraft loss-of-control accidents are highly complex in that they can result from numerous causal and contributing factors acting alone or (more often) in combination. Hence, there is no single intervention strategy to prevent these accidents and reducing them will require a holistic integrated intervention capability. Future onboard integrated system technologies developed for preventing loss of vehicle control accidents must be able to assure safe operation under the associated off-nominal conditions. The transition of these technologies into the commercial fleet will require their extensive validation and verification (V and V) and ultimate certification. The V and V of complex integrated systems poses major nontrivial technical challenges particularly for safety-critical operation under highly off-nominal conditions associated with aircraft loss-of-control events. This paper summarizes the V and V problem and presents a proposed process that could be applied to complex integrated safety-critical systems developed for preventing aircraft loss-of-control accidents. A summary of recent research accomplishments in this effort is also provided.

Belcastro, Christine M.↗

NASA NTRS2012

Validation and Verification (V&V) of Safety-Critical Systems Operating Under Off-Nominal Conditions

Loss of control (LOC) remains one of the largest contributors to aircraft fatal accidents worldwide. Aircraft LOC accidents are highly complex in that they can result from numerous causal and contributing factors acting alone or more often in combination. Hence, there is no single intervention strategy to prevent these accidents. Research is underway at the National Aeronautics and Space Administration (NASA) in the development of advanced onboard system technologies for preventing or recovering from loss of vehicle control and for assuring safe operation under off-nominal conditions associated with aircraft LOC accidents. The transition of these technologies into the commercial fleet will require their extensive validation and verification (V&V) and ultimate certification. The V&V of complex integrated systems poses highly significant technical challenges and is the subject of a parallel research effort at NASA. This chapter summarizes the V&V problem and presents a proposed process that could be applied to complex integrated safety-critical systems developed for preventing aircraft LOC accidents. A summary of recent research accomplishments in this effort is referenced.

Belcastro, Christine M.↗

NASA NTRS2022

Visual Odometry Thinking While Driving for the Curiosity Mars Rover's Three-Year Test Campaign: Impact of Evolving Constraints on Verification and Validation

Over the first 9 years of the Mars Science Laboratory (MSL) Curiosity rover's surface mission, more than 87\% of its driving has been performed using Visual Odometry (VO). The benefits of using VO during driving are that it minimizes rover position uncertainty and can be used to monitor wheel slip, halting a drive if excessive wheel slip is occurring. The VO implementation on board Curiosity acquires and processes VO images in between drive steps while the rover is stationary. A VO Thinking While Driving (VTWD) flight software capability has been developed which enables the processing of VO images during rover driving, increasing the distance Curiosity can drive with VO in a given time period up to as much as 1.75x total distance. Verification and Validation (V\&V) of the capability has been challenging due to impacts from the COVID-19 pandemic and unavailability of the JPL Mars Yard outdoor test site. The VTWD V\&V test procedures were modified to use a small indoor space with Mars-like terrain. This paper describes the 3 year V\&V effort under challenging conditions to approve the VTWD capability for use on the Curiosity rover.

Rankin, Arturo↗

NASA NTRS2024

Looking to the Future: A Call to Action for Advanced GNC Algorithm Verification and Validation

Future space systems will rely on autonomous Guidance, Navigation, and Control (GNC) functions to efficiently manage safe and precise self-directed operations in uncertain complex environments. Fundamentally, the GNC system plays a key role in mission performance and safety because it computes the ideal trajectory (Guidance), determines the actual trajectory (Navigation), and executes the ideal trajectory (Control) of a vehicle’s position and attitude. Our current GNC systems are highly automated and already have a high degree of complexity. As missions become more ambitious, GNC systems for launch vehicles and space platforms (e.g., spacecraft, probes, and landers) will require higher levels of performance and autonomous operation than previously encountered, for example, this includes GNC for optimizing aerodynamic and/or propulsion performance during planetary entry. This GNC Verification and Validation (V&V) paper highlights concerns with what undoubtedly will be a trend towards increased complexity as fully autonomous GNC systems are developed for future space missions. Clearly, complex GNC systems pose challenges in the prelaunch V&V phase, which is a relatively expensive part of a mission’s life cycle. Essentially the V&V phase is focused on checking that the system effectively meets all the design and operational requirements for the mission. The authors of this paper (i.e., the Inter-Agency Working Group of GNC subject matter experts) focused on this fundamental question over the past few years: Will the GNC engineering community of practice be sufficiently prepared to perform the necessary V&V on evolving GNC architectures that are driven by very demanding requirements for autonomy, resiliency, reconfigurability, adaptability, and mission cost-benefit balance? It is the viewpoint of our Inter-Agency team that the GNC V&V approaches and processes needed to address the next generation of complex GNC systems, which likely will employ various forms of modern GNC technology, are not currently established to the level the community will need in the future. While researchers and practitioners have made some progress in developing new GNC V&V methods for modern GNC systems, a good deal of work remains to be done to codify such methods in a comprehensive and systematic manner. Thus, the Inter-Agency team’s partner organizations [the National Aeronautics and Space Administration (NASA), the European Space Agency (ESA), the National Centre for Space Studies (CNES), the German Aerospace Center (DLR), the French Aerospace Lab (ONERA), and ISAE-SUPAERO] have conducted preliminary investigations into advancing GNC V&V techniques, which resulted in the identification of the need for education, new V&V tools, and benchmark problems for the GNC community. The necessary proactive steps to be taken to meet the challenges and fill the gaps in GNC V&V are summarized in this paper. The first steps include identifying advanced analysis tools, developing a GNC V&V roadmap, and expanding education and training programs for GNC practitioners. This paper is a call to action and proposes a comprehensive set of recommended actions for all our stakeholders: space agencies, researchers, and industry.

Samir Bennani↗

NASA NTRS1996

Independent Verification and Validation of Complex User Interfaces: A Human Factors Approach

The Usability Testing and Analysis Facility (UTAF) at the NASA Johnson Space Center has identified and evaluated a potential automated software interface inspection tool capable of assessing the degree to which space-related critical and high-risk software system user interfaces meet objective human factors standards across each NASA program and project. Testing consisted of two distinct phases. Phase 1 compared analysis times and similarity of results for the automated tool and for human-computer interface (HCI) experts. In Phase 2, HCI experts critiqued the prototype tool's user interface. Based on this evaluation, it appears that a more fully developed version of the tool will be a promising complement to a human factors-oriented independent verification and validation (IV&V) process.

Whitmore, Mihriban↗

NASA NTRSDate not supplied

Verification and Validation of Progressive Damage and Failure Analysis Methods for Intralaminar Failure Modes in Thermoplastic Composites

Current progressive damage and failure analysis (PDFA) methods were developed around thermoset material systems and their physical behavior. However, the applicability of these analysis methods has not been evaluated for use in modeling thermoplastic material systems. As part of the Hi-Rate Composite Aircraft Manufacturing (HiCAM) Project sponsored by NASA, intralaminar failure characterization was completed on a thermoplastic material system, AS4D/PEKK-FC, with off-axis tension (OAT) and off-axis compression (OAC) testing. The experiments were able to capture matrix non-linear shear behavior and the failure envelope of the material system. Two PDFA methods, LS-DYNA MAT299 and NASA’s CompDam-DGD, were then employed to simulate individual coupon behavior and simulation results were compared to published verification and validation guidelines from the CMH-17 crashworthiness working group. The results found that the two modeling methods provided suitable representation of material behavior of thermoplastics.

PDFA↗

NASA NTRSDate not supplied

Verification and Validation of Progressive Damage and Failure Analysis Methods for Intralaminar Failure Modes in Thermoplastic Composites

PDFA↗