SEARCH · Engineering Papers

Results for “verification and validation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 145 records · Page 8

All sources NASA NTRS DOE OSTI

NASA NTRS1996

Verification and Validation of KBS with Neural Network Components

Artificial Neural Network (ANN) play an important role in developing robust Knowledge Based Systems (KBS). The ANN based components used in these systems learn to give appropriate predictions through training with correct input-output data patterns. Unlike traditional KBS that depends on a rule database and a production engine, the ANN based system mimics the decisions of an expert without specifically formulating the if-than type of rules. In fact, the ANNs demonstrate their superiority when such if-then type of rules are hard to generate by human expert. Verification of traditional knowledge based system is based on the proof of consistency and completeness of the rule knowledge base and correctness of the production engine.These techniques, however, can not be directly applied to ANN based components.In this position paper, we propose a verification and validation procedure for KBS with ANN based components. The essence of the procedure is to obtain an accurate system specification through incremental modification of the specifications using an ANN rule extraction algorithm.

Wen, Wu↗

NASA NTRS2001

RIACS Workshop on the Verification and Validation of Autonomous and Adaptive Systems

The long-term future of space exploration at NASA is dependent on the full exploitation of autonomous and adaptive systems: careful monitoring of missions from earth, as is the norm now, will be infeasible due to the sheer number of proposed missions and the communication lag for deep-space missions. Mission managers are however worried about the reliability of these more intelligent systems. The main focus of the workshop was to address these worries and hence we invited NASA engineers working on autonomous and adaptive systems and researchers interested in the verification and validation (V&V) of software systems. The dual purpose of the meeting was to: (1) make NASA engineers aware of the V&V techniques they could be using; and (2) make the V&V community aware of the complexity of the systems NASA is developing.

Pecheur, Charles↗

NASA NTRS2006

Verification and Validation of NASA-Supported Enhancements to PECAD's Decision Support Tools

The NASA Applied Sciences Directorate (ASD), part of the Earth-Sun System Division of NASA's Science Mission Directorate, has partnered with the U.S. Department of Agriculture (USDA) to enhance decision support in the area of agricultural efficiency-an application of national importance. The ASD integrated the results of NASA Earth science research into USDA decision support tools employed by the USDA Foreign Agricultural Service (FAS) Production Estimates and Crop Assessment Division (PECAD), which supports national decision making by gathering, analyzing, and disseminating global crop intelligence. Verification and validation of the following enhancements are summarized: 1) Near-real-time Moderate Resolution Imaging Spectroradiometer (MODIS) products through PECAD's MODIS Image Gallery; 2) MODIS Normalized Difference Vegetation Index (NDVI) time series data through the USDA-FAS MODIS NDVI Database; and 3) Jason-1 and TOPEX/Poseidon lake level estimates through PECAD's Global Reservoir and Lake Monitor. Where possible, each enhanced product was characterized for accuracy, timeliness, and coverage, and the characterized performance was compared to PECAD operational requirements. The MODIS Image Gallery and the GRLM are more mature and have achieved a semi-operational status, whereas the USDA-FAS MODIS NDVI Database is still evolving and should be considered

McKellipo, Rodney↗

NASA NTRS2009

Verification and Validation of Adaptive and Intelligent Systems with Flight Test Results

F-15 IFCS project goals are: a) Demonstrate Control Approaches that can Efficiently Optimize Aircraft Performance in both Normal and Failure Conditions [A] & [B] failures. b) Advance Neural Network-Based Flight Control Technology for New Aerospace Systems Designs with a Pilot in the Loop. Gen II objectives include; a) Implement and Fly a Direct Adaptive Neural Network Based Flight Controller; b) Demonstrate the Ability of the System to Adapt to Simulated System Failures: 1) Suppress Transients Associated with Failure; 2) Re-Establish Sufficient Control and Handling of Vehicle for Safe Recovery. c) Provide Flight Experience for Development of Verification and Validation Processes for Flight Critical Neural Network Software.

Burken, John J.↗

NASA NTRS2010

Validation and Verification of Future Integrated Safety-Critical Systems Operating under Off-Nominal Conditions

Loss of control remains one of the largest contributors to aircraft fatal accidents worldwide. Aircraft loss-of-control accidents are highly complex in that they can result from numerous causal and contributing factors acting alone or (more often) in combination. Hence, there is no single intervention strategy to prevent these accidents and reducing them will require a holistic integrated intervention capability. Future onboard integrated system technologies developed for preventing loss of vehicle control accidents must be able to assure safe operation under the associated off-nominal conditions. The transition of these technologies into the commercial fleet will require their extensive validation and verification (V and V) and ultimate certification. The V and V of complex integrated systems poses major nontrivial technical challenges particularly for safety-critical operation under highly off-nominal conditions associated with aircraft loss-of-control events. This paper summarizes the V and V problem and presents a proposed process that could be applied to complex integrated safety-critical systems developed for preventing aircraft loss-of-control accidents. A summary of recent research accomplishments in this effort is also provided.

Belcastro, Christine M.↗

NASA NTRS2012

Validation and Verification (V&V) of Safety-Critical Systems Operating Under Off-Nominal Conditions

Loss of control (LOC) remains one of the largest contributors to aircraft fatal accidents worldwide. Aircraft LOC accidents are highly complex in that they can result from numerous causal and contributing factors acting alone or more often in combination. Hence, there is no single intervention strategy to prevent these accidents. Research is underway at the National Aeronautics and Space Administration (NASA) in the development of advanced onboard system technologies for preventing or recovering from loss of vehicle control and for assuring safe operation under off-nominal conditions associated with aircraft LOC accidents. The transition of these technologies into the commercial fleet will require their extensive validation and verification (V&V) and ultimate certification. The V&V of complex integrated systems poses highly significant technical challenges and is the subject of a parallel research effort at NASA. This chapter summarizes the V&V problem and presents a proposed process that could be applied to complex integrated safety-critical systems developed for preventing aircraft LOC accidents. A summary of recent research accomplishments in this effort is referenced.

Belcastro, Christine M.↗

NASA NTRS2022

Visual Odometry Thinking While Driving for the Curiosity Mars Rover's Three-Year Test Campaign: Impact of Evolving Constraints on Verification and Validation

Over the first 9 years of the Mars Science Laboratory (MSL) Curiosity rover's surface mission, more than 87\% of its driving has been performed using Visual Odometry (VO). The benefits of using VO during driving are that it minimizes rover position uncertainty and can be used to monitor wheel slip, halting a drive if excessive wheel slip is occurring. The VO implementation on board Curiosity acquires and processes VO images in between drive steps while the rover is stationary. A VO Thinking While Driving (VTWD) flight software capability has been developed which enables the processing of VO images during rover driving, increasing the distance Curiosity can drive with VO in a given time period up to as much as 1.75x total distance. Verification and Validation (V\&V) of the capability has been challenging due to impacts from the COVID-19 pandemic and unavailability of the JPL Mars Yard outdoor test site. The VTWD V\&V test procedures were modified to use a small indoor space with Mars-like terrain. This paper describes the 3 year V\&V effort under challenging conditions to approve the VTWD capability for use on the Curiosity rover.

Rankin, Arturo↗

NASA NTRS2024

Looking to the Future: A Call to Action for Advanced GNC Algorithm Verification and Validation

Future space systems will rely on autonomous Guidance, Navigation, and Control (GNC) functions to efficiently manage safe and precise self-directed operations in uncertain complex environments. Fundamentally, the GNC system plays a key role in mission performance and safety because it computes the ideal trajectory (Guidance), determines the actual trajectory (Navigation), and executes the ideal trajectory (Control) of a vehicle’s position and attitude. Our current GNC systems are highly automated and already have a high degree of complexity. As missions become more ambitious, GNC systems for launch vehicles and space platforms (e.g., spacecraft, probes, and landers) will require higher levels of performance and autonomous operation than previously encountered, for example, this includes GNC for optimizing aerodynamic and/or propulsion performance during planetary entry. This GNC Verification and Validation (V&V) paper highlights concerns with what undoubtedly will be a trend towards increased complexity as fully autonomous GNC systems are developed for future space missions. Clearly, complex GNC systems pose challenges in the prelaunch V&V phase, which is a relatively expensive part of a mission’s life cycle. Essentially the V&V phase is focused on checking that the system effectively meets all the design and operational requirements for the mission. The authors of this paper (i.e., the Inter-Agency Working Group of GNC subject matter experts) focused on this fundamental question over the past few years: Will the GNC engineering community of practice be sufficiently prepared to perform the necessary V&V on evolving GNC architectures that are driven by very demanding requirements for autonomy, resiliency, reconfigurability, adaptability, and mission cost-benefit balance? It is the viewpoint of our Inter-Agency team that the GNC V&V approaches and processes needed to address the next generation of complex GNC systems, which likely will employ various forms of modern GNC technology, are not currently established to the level the community will need in the future. While researchers and practitioners have made some progress in developing new GNC V&V methods for modern GNC systems, a good deal of work remains to be done to codify such methods in a comprehensive and systematic manner. Thus, the Inter-Agency team’s partner organizations [the National Aeronautics and Space Administration (NASA), the European Space Agency (ESA), the National Centre for Space Studies (CNES), the German Aerospace Center (DLR), the French Aerospace Lab (ONERA), and ISAE-SUPAERO] have conducted preliminary investigations into advancing GNC V&V techniques, which resulted in the identification of the need for education, new V&V tools, and benchmark problems for the GNC community. The necessary proactive steps to be taken to meet the challenges and fill the gaps in GNC V&V are summarized in this paper. The first steps include identifying advanced analysis tools, developing a GNC V&V roadmap, and expanding education and training programs for GNC practitioners. This paper is a call to action and proposes a comprehensive set of recommended actions for all our stakeholders: space agencies, researchers, and industry.

Samir Bennani↗

NASA NTRS1996

Independent Verification and Validation of Complex User Interfaces: A Human Factors Approach

The Usability Testing and Analysis Facility (UTAF) at the NASA Johnson Space Center has identified and evaluated a potential automated software interface inspection tool capable of assessing the degree to which space-related critical and high-risk software system user interfaces meet objective human factors standards across each NASA program and project. Testing consisted of two distinct phases. Phase 1 compared analysis times and similarity of results for the automated tool and for human-computer interface (HCI) experts. In Phase 2, HCI experts critiqued the prototype tool's user interface. Based on this evaluation, it appears that a more fully developed version of the tool will be a promising complement to a human factors-oriented independent verification and validation (IV&V) process.

Whitmore, Mihriban↗

NASA NTRSDate not supplied

Verification and Validation of Progressive Damage and Failure Analysis Methods for Intralaminar Failure Modes in Thermoplastic Composites

Current progressive damage and failure analysis (PDFA) methods were developed around thermoset material systems and their physical behavior. However, the applicability of these analysis methods has not been evaluated for use in modeling thermoplastic material systems. As part of the Hi-Rate Composite Aircraft Manufacturing (HiCAM) Project sponsored by NASA, intralaminar failure characterization was completed on a thermoplastic material system, AS4D/PEKK-FC, with off-axis tension (OAT) and off-axis compression (OAC) testing. The experiments were able to capture matrix non-linear shear behavior and the failure envelope of the material system. Two PDFA methods, LS-DYNA MAT299 and NASA’s CompDam-DGD, were then employed to simulate individual coupon behavior and simulation results were compared to published verification and validation guidelines from the CMH-17 crashworthiness working group. The results found that the two modeling methods provided suitable representation of material behavior of thermoplastics.

PDFA↗

NASA NTRSDate not supplied

Verification and Validation of Progressive Damage and Failure Analysis Methods for Intralaminar Failure Modes in Thermoplastic Composites

PDFA↗

NASA NTRSDate not supplied

Verification and Validation of a Conceptual Model of the Auto-Rigging Payload Handling and Off-Loading System Using LEGO Technic System and Three-Dimensional Printed Parts

A conceptual model (CM) can be used to validate a concept in modeling and simulation life cycles. During the 2020-2022 Coronavirus disease 2019 (COVID-19) pandemic, for employee safety NASA implemented center closures and mandatory telework for the entire workforce. During this challenging time, engineers and researchers at NASA Langley Research Center (LaRC) looked for safe and innovative approaches and methods to continue the development of CMs for various projects. Engineers and researchers at LaRC researched Auto-Rigging Payload Handling and Off-Loading System (ARPHOLS) for payload handing and off-loading a system on an inclined lunar lander deck. In this paper, the development of a CM and the verification and validation of a conceptual idea for ARPHOLS using a LEGO Technic system and three-dimensional printed parts is presented.

Payload offloading↗

NASA NTRS2017

Unmanned Aircraft Systems Minimum Operations Performance Standards End-to-End Verification and Validation (E2-V2) Simulation

As Unmanned Aircraft Systems (UAS) make their way to mainstream aviation operations within the National Airspace System (NAS), research efforts are underway to develop a safe and effective environment for their integration into the NAS. Detect and Avoid (DAA) systems are required to account for the lack of "eyes in the sky" due to having no human on-board the aircraft. The current NAS relies on pilot's vigilance and judgement to remain Well Clear (CFR 14 91.113) of other aircraft. RTCA SC-228 has defined DAA Well Clear (DAAWC) to provide a quantified Well Clear volume to allow systems to be designed and measured against. Extended research efforts have been conducted to understand and quantify system requirements needed to support a UAS pilot's ability to remain well clear of other aircraft. The efforts have included developing and testing sensor, algorithm, alerting, and display requirements. More recently, sensor uncertainty and uncertainty mitigation strategies have been evaluated. This paper discusses results and lessons learned from an End-to-End Verification and Validation (E2-V2) simulation study of a DAA system representative of RTCA SC-228's proposed Phase I DAA Minimum Operational Performance Standards (MOPS). NASA Langley Research Center (LaRC) was called upon to develop a system that evaluates a specific set of encounters, in a variety of geometries, with end-to-end DAA functionality including the use of sensor and tracker models, a sensor uncertainty mitigation model, DAA algorithmic guidance in both vertical and horizontal maneuvering, and a pilot model which maneuvers the ownship aircraft to remain well clear from intruder aircraft, having received collective input from the previous modules of the system. LaRC developed a functioning batch simulation and added a sensor/tracker model from the Federal Aviation Administration (FAA) William J. Hughes Technical Center, an in-house developed sensor uncertainty mitigation strategy, and implemented a pilot model similar to one from the Massachusetts Institute of Technology's Lincoln Laboratory (MIT/LL). The resulting simulation provides the following key parameters, among others, to evaluate the effectiveness of the MOPS DAA system: severity of loss of well clear (SLoWC), alert scoring, and number of increasing alerts (alert jitter). The technique, results, and lessons learned from a detailed examination of DAA system performance over specific test vectors and encounter cases during the simulation experiment will be presented in this paper.

Ghatas, Rania W.↗

NASA NTRS2015

Verification and Validation Testing of the Parachute Decelerator System Prior to the First Supersonic Flight Dynamics Test for the Low Density Supersonic Decelerator Program

The Parachute Decelerator System (PDS) is comprised of all components associated with the supersonic parachute and its associated deployment. During the Supersonic Flight Dynamics Test (SFDT), for the Low Density Supersonic Decelerators Program, the PDS was required to deploy the supersonic parachute in a defined fashion. The PDS hardware includes three major subsystems that must function together. The first subsystem is the Parachute Deployment Device (PDD), which acts as a modified pilot deployment system. It is comprised of a pyrotechnic mortar, a Kevlar ballute, a lanyard actuated pyrotechnic inflation aid, and rigging with its associated thermal protection material (TPS). The second subsystem is the supersonic parachute deployment hardware. This includes all of the parachute specific rigging that includes the parachute stowage can and the rigging including TPS and bridle stiffeners for bridle management during deployment. The third subsystem is the Supersonic Parachute itself, which includes the main parachute and deployment bags. This paper summarizes the verification and validation of the deployment process, from the initialization of the PDS system through parachute bag strip that was done prior to the first SFDT.

Gallon, John C.↗

NASA NTRS2018

Verification and Validation Challenges for Adaptive Flight Control of Complex Autonomous Systems

Autonomy of aerospace systems requires the ability for flight control systems to be able to adapt to complex uncertain dynamic environment. In spite of the five decades of research in adaptive control, the fact still remains that currently no adaptive control system has ever been deployed on any safety-critical or human-rated production systems such as passenger transport aircraft. The problem lies in the difficulty with the certification of adaptive control systems since existing certification methods cannot readily be used for nonlinear adaptive control systems. Research to address the notion of metrics for adaptive control began to appear in the recent years. These metrics, if accepted, could pave a path towards certification that would potentially lead to the adoption of adaptive control as a future control technology for safety-critical and human-rated production systems. Development of certifiable adaptive control systems represents a major challenge to overcome. Adaptive control systems with learning algorithms will never become part of the future unless it can be proven that they are highly safe and reliable. Rigorous methods for adaptive control software verification and validation must therefore be developed to ensure that adaptive control system software failures will not occur, to verify that the adaptive control system functions as required, to eliminate unintended functionality, and to demonstrate that certification requirements imposed by regulatory bodies such as the Federal Aviation Administration (FAA) can be satisfied. This presentation will discuss some of the technical issues with adaptive flight control and related V&V challenges.

Nguyen, Nhan T.↗

NASA NTRS2005

Real-Time Simulation for Verification and Validation of Diagnostic and Prognostic Algorithms

To verify that a health management system (HMS) performs as expected, a virtual system simulation capability, including interaction with the associated platform or vehicle, very likely will need to be developed. The rationale for developing this capability is discussed and includes the limited capability to seed faults into the actual target system due to the risk of potential damage to high value hardware. The capability envisioned would accurately reproduce the propagation of a fault or failure as observed by sensors located at strategic locations on and around the target system and would also accurately reproduce the control system and vehicle response. In this way, HMS operation can be exercised over a broad range of conditions to verify that it meets requirements for accurate, timely response to actual faults with adequate margin against false and missed detections. An overview is also presented of a real-time rocket propulsion health management system laboratory which is available for future rocket engine programs. The health management elements and approaches of this lab are directly applicable for future space systems. In this paper the various components are discussed and the general fault detection, diagnosis, isolation and the response (FDIR) concept is presented. Additionally, the complexities of V&V (Verification and Validation) for advanced algorithms and the simulation capabilities required to meet the changing state-of-the-art in HMS are discussed.

Aguilar, Robet↗

NASA NTRS2019

The NASA Orbital Debris Engineering Model 3.1: Development, Verification, and Validation

The NASA Orbital Debris Program Office has developed the Orbital Debris Engineering Model (ORDEM) primarily as a tool for spacecraft designers and other users to understand the long-term risk of collisions with orbital debris. The newest version, ORDEM 3.1, incorporates the latest and highest fidelity datasets available to build and validate representative orbital debris populations encompassing low Earth orbit (LEO) to geosynchronous orbit (GEO) altitudes for the years 2016-2050. ORDEM 3.1 models fluxes for object sizes > 10 μm within or transiting LEO and > 10 cm in GEO. The deterministic portion of the populations in ORDEM 3.1 is based on the U.S. Space Surveillance Network (SSN) catalog, which provides coverage down to approximately 10 cm in LEO and 1 m in GEO. Observational datasets from radar, in situ, and optical sources provide a foundation from which the model populations are statistically extrapolated to smaller sizes and orbit regions that are not well-covered by the SSN catalog, yet may pose the greatest threat to operational spacecraft. Objects in LEO ranging from approximately 5 mm to 10 cm are modeled using observational data from ground-based radar, namely the Haystack Ultrawideband Satellite Imaging Radar (HUSIR – formerly known as Haystack). The LEO population smaller than approximately 3 mm in size is characterized based on a reanalysis of in situ data from impacts to the windows and radiators of the U.S. Space Transportation System orbiter vehicle, i.e., the Space Shuttle. Data from impacts on the Hubble Space Telescope are also used to validate the sub-millimeter model populations in LEO. Debris in GEO with sizes ranging from 10 cm to 1 m is modeled using optical measurement data from the Michigan Orbital DEbris Survey Telescope (MODEST). Specific, major debris-producing events, including the Fengyun-1C, Iridium 33, and Cosmos 2251 debris clouds, and unique populations, such as sodium-potassium droplets, have been re-examined and are modeled and added to the ORDEM environment separately. The debris environment greater than 1 mm is forecast using NASA’s LEO-to- GEO ENvironment Debris model (LEGEND). Future explosions of intact objects and collisions involving objects greater than 10 cm are assessed statistically, and the NASA Standard Satellite Breakup Model is used to generate fragments from these events. Fragments smaller than 10 cm are further differentiated based on material density categories, i.e., high-, medium-, and low-density, to better characterize the potential debris risk posed to spacecraft. The future projection of the sub-millimeter environment is computed using a special small-particle degradation model where small particles are created from intact spacecraft and rocket bodies. This work discusses the development, features, and capabilities of the ORDEM 3.1 model; the ne new data analyses used to build the model populations; and sample verification and validation results.

Matney, M.↗

NASA NTRS1975

Simulation verification techniques study

Results are summarized of the simulation verification techniques study which consisted of two tasks: to develop techniques for simulator hardware checkout and to develop techniques for simulation performance verification (validation). The hardware verification task involved definition of simulation hardware (hardware units and integrated simulator configurations), survey of current hardware self-test techniques, and definition of hardware and software techniques for checkout of simulator subsystems. The performance verification task included definition of simulation performance parameters (and critical performance parameters), definition of methods for establishing standards of performance (sources of reference data or validation), and definition of methods for validating performance. Both major tasks included definition of verification software and assessment of verification data base impact. An annotated bibliography of all documents generated during this study is provided.

Schoonmaker, P. B.↗