Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “verification and validation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 811 records · Page 45
All sourcesNASA NTRSDOE OSTI
NASA NTRS2020

IV&V Assurance Case Design for Artemis II

As human-rated missions like those in NASA’s Artemis program continue to grow in both size and complexity, and the role of software in achieving mission objectives expands dramatically, NASA’s Independent Verification and Validation (IV&V) Teams face evolving challenges in assuring the safety and performance of the safety- and mission-critical embedded software that is essential to landing astronauts on the surface of the Moon by 2024. Key among these challenges is IV&V’s desire to present a cohesive, integrated assurance statement to its stakeholders that encapsulates and summarizes our assurance positions across the integrated Artemis systems and their combined role in support of a safe and successful flight. In order to meet this challenge, the IV&V Teams have begun a transition to using formal assurance case concepts and documentation in the Goal Structuring Notation (GSN) to build an argument in support of software assurance. IV&V recognizes significant benefits to the logical argumentation structure provided by assurance cases and GSN over our current practices for documenting and managing assurance claims. In order to reap these benefits, IV&V is integrating the use of assurance case concepts with our paradigm of follow-the-risk capability based assurance. Because of this, assurance cases created and used by IV&V are distinct from the sort of assurance case created by a development project or embedded software assurance organization. IV&V’s assurance cases depend much less upon standards and regulations, and more on evidence captured by IV&V regarding the environment, requirements, design, and implementation. IV&V constructs an independent network of claims based on an independent decomposition of arguments. Based upon the risk posture of these claims and their associated software and software artifacts, IV&V then develops and executes engineering analyses and testing, which provide evidence to either support or refute the claim. This emerging risk-informed assurance case methodology is being put into practice as IV&V plans for support of the Artemis II mission, the first flight of the Orion capsule and Space Launch System with astronauts on board.

Gerek Whitman↗
NASA NTRS2020

IV&V Assurance Case Design for Artemis II

As human-rated missions like those in NASA's Artemis program continue to grow in both size and complexity, and the role of software in achieving mission objectives expands dramatically, NASA's Independent Verification and Validation (IV&V) Teams face evolving challenges in assuring the safety and performance of the safety- and mission-critical embedded software that is essential to landing astronauts on the surface of the Moon by 2024. Key among these challenges is IV&V's desire to present a cohesive, integrated assurance statement to its stakeholders that encapsulates and summarizes our assurance positions across the integrated Artemis systems and their combined role in support of a safe and successful flight. In order to meet this challenge, the IV&V Teams have begun a transition to using formal assurance case concepts and documentation in the Goal Structuring Notation (GSN) to build an argument in support of software assurance. IV&V recognizes significant benefits to the logical argumentation structure provided by assurance cases and GSN over our current practices for documenting and managing assurance claims. In order to reap these benefits, IV&V is integrating the use of assurance case concepts with our paradigm of follow-the-risk capability based assurance. Because of this, assurance cases created and used by IV&V are distinct from the sort of assurance case created by a development project or embedded software assurance organization. IV&V's assurance cases depend much less upon standards and regulations, and more on evidence captured by IV&V regarding the environment, requirements, design, and implementation. IV&V constructs an independent network of claims based on an independent decomposition of arguments. Based upon the risk posture of these claims and their associated software and software artifacts, IV&V then develops and executes engineering analyses and testing, which provide evidence to either support or refute the claim. This emerging risk-informed assurance case methodology is being put into practice as IV&V plans for support of the Artemis II mission, the first flight of the Orion capsule and Space Launch System with astronauts on board.

Whitman, Gerek↗
NASA NTRS2019

Medical System Concept of Operations for Mars Exploration Mission-11: Exploration Medical Capability (ExMC) Element - Human Research Program

NASA’s exploration missions to Mars will have durations of 2-3 years and will take humans farther away from Earth than ever before. This will result in a paradigm shift for mission planning, spacecraft design, human systems integration, and in-flight medical care. Constraints on real-time communication, resupply, and medical evacuation are major architectural drivers. These constraints require medical system development to be tightly integrated with mission and vehicle design to provide crew autonomy and enable mission success. This concept of operations provides a common vision of medical care for developing a medical system for Mars exploration missions. It documents an overview of the stakeholder needs and goals of a medical system and provides examples of the types of activities the system will be used for during the mission. Development of the concept of operations considers mission variables such as distance from Earth, duration of mission, time to definitive medical care, communication protocols between crewmembers and ground support, personnel capabilities and skill sets, medical hardware and software, and medical data management. The information provided in this document informs the ExMC Systems Engineering effort to define the functions to be provided by the medical system. In addition, this concept of operations will inform the subsequent systems engineering process of developing technical requirements, system architectures, interfaces, and verification and validation approaches for the medical system. This document supports the closure of ExMC Gap Med01: We do not have a concept of operations for medical care during exploration missions, corresponding to the ExMC-managed human system risk: Risk of Adverse Health Outcomes & Decrements in Performance due to Inflight Medical Conditions. This document is applicable to the ExMC Element Systems Engineering process and may be used for collaboration within the Human Research Program.

Urbina, Michelle↗
NASA NTRS2019

A Planning Pipeline for Large Multi-Agent Missions

In complex multi-agent applications, human operators are often tasked with planning and managing large heterogeneous teams of humans and autonomous vehicles. Although the use of these autonomous vehicles broadens the scope of meaningful applications, many of their systems remain unintuitive and difficult to master for human operators whose expertise lies in the application domain and not at the platform level. Current research focuses on the development of individual capabilities necessary to plan multi-agent missions of this scope, placing little emphasis on the integration of these components in to a full pipeline. The work presented in this paper presents a complete and user-agnostic planning pipeline for large multiagent missions known as the HOLII GRAILLE. The system takes a holistic approach to mission planning by integrating capabilities in human machine interaction, flight path generation, and validation and verification. Components – modules – of the pipeline are explored on an individual level, as well as their integration into a whole system. Lastly, implications for future mission planning are discussed.

Chandarana, Meghan↗
NASA NTRS2019

Advancing Aircraft Operations in a Net-Centric Environment with the Incorporation of Increasingly Autonomous Systems and Human Teaming

NextGen has begun the modernization of the nation’s air transportation system, with goals to improve system safety, increase operation efficiency and capacity, provide enhanced predictability, resilience and robustness. With these improvements, NextGen is poised to handle significant increases in air traffic operations, more than twice the number recorded in 2016, by 2025.1 NextGen is evolving toward collaborative decision-making across many agents, including automation, by use of a Net-Centric architecture, which in itself creates a very complex environment in which the navigation and operation of aircraft are to take place. An intricate environment such as this, coupled with the expected upsurge of air traffic operations generates concern respecting the ability of the human-agent to both fly and manage aircraft within. Therefore, it is both necessary and practical to begin the process of increasingly autonomous systems within the cockpit that will act independently to assist the human-agent achieve the overall goal of NextGen. However, the straightforward technological development and implementation of intelligent machines into the cockpit is only part of what is necessary to maintain, at minimum, or improve human-agent functionality, as desired, while operating in NextGen. The full integration of Increasingly Autonomous Systems (IAS) within the cockpit can only be accomplished when the IAS works in concert with the human, formulating trust between the two, thereby establishing a team atmosphere. Imperative to cockpit implementation is ensuring the proper performance of the IAS by the development team and the human-agent with which it will be paired when given a specific piloting, navigation, or observational task. Described in this paper are the steps taken, at NASA Langley Research Center, during the second and third phases of the development of an IAS, the Traffic Data Manager (TDM), its verification and validation by human-agents, and the foundational development of Human Autonomy Teaming (HAT) between the two.

Houston, Vincent E.↗
NASA NTRSDate not supplied

TPSAS-NF1676L-16422-DND

The Nowcast of Atmospheric Ionizing Radiation for Aviation Safety (NAIRAS) is a prototype operational model for predicting aircraft radiation exposure from galactic and solar cosmic rays. NAIRAS predictions are currently streaming live from the project's public website, and the exposure rate nowcast is also available on the SpaceWx smartphone app for iPhone, IPad, and Android. Cosmic rays are the primary source of human exposure to high linear energy transfer radiation at aircraft altitudes, which increases the risk of cancer and other adverse health effects. Results from recent NAIRAS model updates and new data products are presented. In addition, current and ongoing radiation measurement campaigns and verification and validation efforts are discussed.

Christopher J Mertens↗
NASA NTRSDate not supplied

The System Modeling and Analysis of Resiliency in STEReO (SMARt-STEReO)

Wildfire emergency response has remained rooted in relatively low-tech solutions for coordination between ground and aerial assets. These low-tech solutions are robust for the remote environments in which wildfires are usually fought, but limit strategic cross-organizational support and the ability to deploy and effectively utilize aerial assets. As aircraft become more advanced and new technology, including drones, become available to firefighters, a new, more modern method of asset coordination is needed. NASA is working on a project called ‘Scalable Traffic Management for Emergency Response Operations’ (STEReO) to integrate unmanned aerial systems (UAS)and UAS traffic management (UTM)into wildfire response. STEReO’s goals include simplifying the coordination of aerial assets, improving the existing UAS framework, and increasing the role of additional autonomous systems to reduce human risk and to increase system resilience. This paper describes the development of the ‘System Modeling and Analysis of Resiliency in STEReO’ (SMARt-STEReO) project, which aims to model wildfire response and to quantify the additional system resilience that STEReO technology provides firefighters. This paper verifies SMARt-STEReO and defines its scope; it includes experimental and statistical analysis of the impact that the addition of UAS has on both performance metrics and also on performance resiliency response to a given fault. SMARt-STEReO is a grid-based model of fire propagation that incorporates varying crew responses. Through the use of a Python package called ‘fmdtools’, the model easily allows for the addition of faults to the system. These faults allow analysts to investigate various response parameters. Factors including terrain, fuel type and wind speed can be modified to affect the fire propagation; additionally, the number of ground crews, engines, fixed wing aircraft, helicopters, and UAS can be changed to affect the crew response. The communication lines between actors mimic those used in real life situations. This paper explains the development of SMARt-STEReO including background research, verification and validation, and preliminary experimental analysis of system resilience to both a minor and major fault in systems with and without UAS.

Resiliency↗
NASA NTRSDate not supplied

Artificial Neural Networks and AI in high Assurance Applications: Gaps and Techniques

In recent years, capabilties of Deep Neural Networks (DNN) and Artificial Intelligence (AI) systems have grown tremendously. They are now applied in many areas ranging from game playing, social media, science, to robotics, automotive, and aerospace applications.Based upon requirements for safety of DNN and AI in high assurance automotive and aerospace applications, I will discuss the necessity to ensure that AI technqiues for the analysis of Earth observation data and reasoning are working correctly and reliably.In this talk I will present modern techniques for the verification and validation (V&V) of DNN and other AI components as well as approaches for interpretable AI. I will discuss how these techniques can help to ensure quality of the AI results, improve confidence in their application, and facilitate human-AI interaction and collaboration.

Johann Schumann↗
NASA NTRSDate not supplied

Credible Practice of Models and Simulations for Healthcare

As computational models penetrate clinical care and drive healthcare policy decisions, there is increasing need to communicate that model creation and testing follows reasonable credible practice. Although verification and validation have been generally accepted as significant components of a model’s credibility, they cannot be assumed to equate to a holistic credible practice, which includes activities that can impact comprehension and in-depth examination inherent in the development and reuse of the models. This presentation introduces the Ten Rules for credible practice of modeling and simulation in healthcare developed from a comparative multidisciplinary analysis and practitioner survey. These rules establish a unified conceptual framework for modeling and simulation design, implementation, evaluation, dissemination and usage across the modeling and simulation life-cycle. Although some of these are common sense guidelines, many are often missed or misconstrued, even by seasoned practitioners.

credibility↗
NASA NTRSDate not supplied

Structured Overset and Unstructured Grid Simulations for the Third AIAA Sonic Boom Prediction Workshop

Predictions using the Launch Ascent and Vehicle Aerodynamics (LAVA) Unstructured and Structured Curvilinear solvers for the Biconvex and C608 geometries are compared in support of the 3rd AIAA Sonic Boom Prediction workshop. Description of the mesh generation techniques and numerical methods are provided along with a comparison of nearfield and far-field predictions with available experimental data as well as other workshop sub-missions. Strong consistency was observed between the Biconvex and C608 nearfield predictions across the two solvers used for this study, as well as validation and verification data. Additionally, near-field and farfield ground signature results using a combination of a truncated computational fluid dynamics (CFD) domain coupled to a mid-field space marching method and propagated to the ground using sBOOM will be discussed. This approach is shown to provide an efficient way to propagate near-field pressure signatures to the ground with the same accuracy as the near-field CFD coupled to sBOOM, but at less computational cost.

CST↗
NASA NTRSDate not supplied

A survey of thermocouple anomalies: mechanisms, interpretation, and mitigation

Thermocouples (TCs) are a critical diagnostic sensor for high enthalpy environments including thermal protection systems (TPS) for atmospheric re-entry, hypersonic wind tunnels, and arc jet test facilities. They are used for both evaluating facility environmental conditions (enthalpy, heat flux) as well as for measuring in-situ material thermal response under test conditions (thermal conductivity, heat capacity, B’, etc.). Consequently, accurate TC measurements are fundamental for both material and facility verification and validation (V&V). However, a number of TC anomalies have been observed in both ground tests and flight instrumentation. These anomalies are caused via various coupling mechanisms: electrostatic gradients, temperature-dependent electrical shorts, radio pickup, magnetic induction, and pressure-dependent leaks. All of these mechanisms can, if unaddressed, result in obviously compromised results or worse, non-physical measurements which appear reasonable. This work will survey known anomaly mechanisms and discuss ongoing efforts to quantify and mitigate anomaly mechanisms including anomaly modeling, diagnostics, and new TC designs/development.

thermocouple anomalies↗
NASA NTRS1998

Assessing the Effectiveness of Environmental Test Programs

NASA's Code QE Test Effectiveness Program is funding a series of applied research activities focused on utilizing the principles of physics and engineering of failure along with those of engineering economics to assess and improve the value added by various validation and verification activities, with the primary emphasis being on testing.

test↗
NASA NTRS2013

The unparalleled systems engineering of MSL’s backup entry, descent, and landing system : second chance

Second Chance (SECC) was a bare bones version of Mars Science Laboratory’s (MSL) Entry Descent & Landing (EDL) flight software that ran on Curiosity’s backup computer, which could have taken over swiftly in the event of a reset of Curiosity’s prime computer, in order to land her safely on Mars. Without SECC, a reset of Curiosity’s prime computer would have lead to catastrophic mission failure. Even though a reset of the prime computer never occurred, SECC had the important responsibility as EDL’s guardian angel, and this responsibility would not have seen such success without unparalleled systems engineering. This paper will focus on the systems engineering behind SECC: Covering a brief overview of SECC’s design, the intense schedule to use SECC as a backup system, the verification and validation of the system’s “Do No Harm” mandate, the system’s overall functional performance, and finally, its use on the fateful day of August 5th, 2012.

Reeves, Glenn↗
NASA NTRS2019

Europa Clipper Mission: preliminary design report

This paper will describe the progress of the Europa Clipper Mission since January 2018, including maturation of the spacecraft, subsystem and instrument preliminary designs, issues and trades, and planning for the Verification and Validation phase.

Laslo, Nori↗
NASA NTRS2019

Near earth asteroid scout CubeSat science data retrieval optimization using onboard data analysis

We present a collection of techniques designed for mitigating limited pointing stability for target acquisition, onboard image calibration and decision making, in a low bandwidth environment. These technologies will fly on the Near Earth Asteroid Scout (NEA Scout) interplanetary CubeSat mission in 2019. To support verification and validation, the flight software implementations of these algorithms have been run on New Horizons, Rosetta and terrestrially acquired data sets. These experiments validate the capability to detect the target, through noise and target location uncertainty.

Basset, Christophe↗
NASA NTRSDate not supplied

Electromagnetic Thermocouple (TC) Anomaly Mechanisms During Atmospheric Entry

Thermocouples (TCs) are a critical diagnostic sensor for high enthalpy environments including Thermal Protection Systems (TPS) for atmospheric entry, hypersonic wind tunnels, and arc jet test facilities. They are used for both evaluating facility environmental conditions (enthalpy, and heat flux) as well as for measuring in-situ material thermal response under test conditions (thermal conductivity, heat capacity, etc.). Consequently, accurate TC measurements are fundamental for both material and facility Verification and Validation (V&V). However, a number of electromagnetic TC anomalies have been observed in both ground tests and flight instrumentations. These anomalies are caused via various coupling mechanisms: surface voltage coupling, Hall effect currents, and RF rectification. All of these mechanisms can, if unaddressed, result in obviously compromised results or worse, non-physical measurements which appear reasonable. This talk will provide an overview of each of these anomaly mechanisms and discuss ongoing efforts to quantify and mitigate them.

Magnus Haw↗
NASA NTRS2021

Chapter 6: Fundamental Satellite Precipitation Data Records

The observation and measurement of precipitation across a range of temporal and spatial scales is of great importance for many user requirements and applications. Precipitation is a key component of the global energy and water cycle moving water, and its’ associated energy, through the Earth system at relatively fine spatial and temporal scales. At the societal level, precipitation provides fresh water and moderates our climate, but can also be destructive through storms, droughts and spreading diseases. Monitoring and measuring the distribution and quantities of precipitation, across the Earth’s surface is therefore crucial to physical and social applications, together with changes in precipitation over time. While conventional surface instruments provide a reasonably long record, these tend to be confined to well populated land surface areas: most ocean regions and sparsely populated regions have few measurements. Satellite-based precipitation products can provide global measurements, although the timespan of such products is limited and the data record is comprised of different instruments that operate over different periods of record. The challenge of any climate record is to provide a consistent product over the full period of record. Many satellite precipitation products incorporate ancillary data, such as surface gauge data or model data, which may limit their utility in the validation or verification of other climate data records. This chapter provides a review of the importance of precipitation to the climate system, the basis of satellite precipitation retrievals, climate data records of satellite data for precipitation retrievals and the development of fundamental precipitation data records for climate applications.

Climate data record↗