Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “verification and validation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 307 records · Page 17
All sourcesNASA NTRSDOE OSTI
NASA NTRS2015

Can We Trust Computational Modeling for Medical Applications?

Operations in extreme environments such as spaceflight pose human health risks that are currently not well understood and potentially unanticipated. In addition, there are limited clinical and research data to inform development and implementation of therapeutics for these unique health risks. In this light, NASA's Human Research Program (HRP) is leveraging biomedical computational models and simulations (M&S) to help inform, predict, assess and mitigate spaceflight health and performance risks, and enhance countermeasure development. To ensure that these M&S can be applied with confidence to the space environment, it is imperative to incorporate a rigorous verification, validation and credibility assessment (VV&C) processes to ensure that the computational tools are sufficiently reliable to answer questions within their intended use domain. In this presentation, we will discuss how NASA's Integrated Medical Model (IMM) and Digital Astronaut Project (DAP) have successfully adapted NASA's Standard for Models and Simulations, NASA-STD-7009 (7009) to achieve this goal. These VV&C methods are also being leveraged by organization such as the Food and Drug Administration (FDA), National Institute of Health (NIH) and the American Society of Mechanical Engineers (ASME) to establish new M&S VV&C standards and guidelines for healthcare applications. Similarly, we hope to provide some insight to the greater aerospace medicine community on how to develop and implement M&S with sufficient confidence to augment medical research and operations.

Mulugeta, Lealem↗
NASA NTRS2015

Credibility Assessment of Deterministic Computational Models and Simulations for Space Biomedical Research and Operations

Human missions beyond low earth orbit to destinations, such as to Mars and asteroids will expose astronauts to novel operational conditions that may pose health risks that are currently not well understood and perhaps unanticipated. In addition, there are limited clinical and research data to inform development and implementation of health risk countermeasures for these missions. Consequently, NASA's Digital Astronaut Project (DAP) is working to develop and implement computational models and simulations (M&S) to help predict and assess spaceflight health and performance risks, and enhance countermeasure development. In order to effectively accomplish these goals, the DAP evaluates its models and simulations via a rigorous verification, validation and credibility assessment process to ensure that the computational tools are sufficiently reliable to both inform research intended to mitigate potential risk as well as guide countermeasure development. In doing so, DAP works closely with end-users, such as space life science researchers, to establish appropriate M&S credibility thresholds. We will present and demonstrate the process the DAP uses to vet computational M&S for space biomedical analysis using real M&S examples. We will also provide recommendations on how the larger space biomedical community can employ these concepts to enhance the credibility of their M&S codes.

Mulugeta, Lealem↗
NASA NTRS2014

NASA-STD-7009 Guidance Document for Human Health and Performance Models and Simulations

Rigorous verification, validation, and credibility (VVC) processes are imperative to ensure that models and simulations (MS) are sufficiently reliable to address issues within their intended scope. The NASA standard for MS, NASA-STD-7009 (7009) [1] was a resultant outcome of the Columbia Accident Investigation Board (CAIB) to ensure MS are developed, applied, and interpreted appropriately for making decisions that may impact crew or mission safety. Because the 7009 focus is engineering systems, a NASA-STD-7009 Guidance Document is being developed to augment the 7009 and provide information, tools, and techniques applicable to the probabilistic and deterministic biological MS more prevalent in human health and performance (HHP) and space biomedical research and operations.

numerical analysis↗
NASA NTRS2015

Internal NASA Study: NASAs Protoflight Research Initiative

The NASA Protoflight Research Initiative is an internal NASA study conducted within the Office of the Chief Engineer to better understand the use of Protoflight within NASA. Extensive literature reviews and interviews with key NASA members with experience in both robotic and human spaceflight missions has resulted in three main conclusions and two observations. The first conclusion is that NASA's Protoflight method is not considered to be "prescriptive." The current policies and guidance allows each Program/Project to tailor the Protoflight approach to better meet their needs, goals and objectives. Second, Risk Management plays a key role in implementation of the Protoflight approach. Any deviations from full qualification will be based on the level of acceptable risk with guidance found in NPR 8705.4. Finally, over the past decade (2004 - 2014) only 6% of NASA's Protoflight missions and 6% of NASA's Full qualification missions experienced a publicly disclosed mission failure. In other words, the data indicates that the Protoflight approach, in and of it itself, does not increase the mission risk of in-flight failure. The first observation is that it would be beneficial to document the decision making process on the implementation and use of Protoflight. The second observation is that If a Project/Program chooses to use the Protoflight approach with relevant heritage, it is extremely important that the Program/Project Manager ensures that the current project's requirements falls within the heritage design, component, instrument and/or subsystem's requirements for both the planned and operational use, and that the documentation of the relevant heritage is comprehensive, sufficient and the decision well documented. To further benefit/inform this study, a recommendation to perform a deep dive into 30 missions with accessible data on their testing/verification methodology and decision process to research the differences between Protoflight and Full Qualification missions' Design Requirements and Verification & Validation (V&V) (without any impact or special request directly to the project).

Protoflight↗
NASA NTRS2015

Dynamic Emulation of NASA Missions for IVandV: A Case Study of JWST and SLS

Software-Only-Simulations are an emerging but quickly developing field of study throughout NASA. The NASA Independent Verification Validation (IVV) Independent Test Capability (ITC) team has been rapidly building a collection of simulators for a wide range of NASA missions. ITC specializes in full end-to-end simulations that enable developers, VV personnel, and operators to test-as-you-fly. In four years, the team has delivered a wide variety of spacecraft simulations ranging from low complexity science missions such as the Global Precipitation Management (GPM) satellite and the Deep Space Climate Observatory (DSCOVR), to the extremely complex missions such as the James Webb Space Telescope (JWST) and Space Launch System (SLS).

Emulation↗
NASA NTRS2015

Can Simulation Credibility Be Improved Using Sensitivity Analysis to Understand Input Data Effects on Model Outcome?

Model and simulation (MS) credibility is defined as, the quality to elicit belief or trust in MS results. NASA-STD-7009 [1] delineates eight components (Verification, Validation, Input Pedigree, Results Uncertainty, Results Robustness, Use History, MS Management, People Qualifications) that address quantifying model credibility, and provides guidance to the model developers, analysts, and end users for assessing the MS credibility. Of the eight characteristics, input pedigree, or the quality of the data used to develop model input parameters, governing functions, or initial conditions, can vary significantly. These data quality differences have varying consequences across the range of MS application. NASA-STD-7009 requires that the lowest input data quality be used to represent the entire set of input data when scoring the input pedigree credibility of the model. This requirement provides a conservative assessment of model inputs, and maximizes the communication of the potential level of risk of using model outputs. Unfortunately, in practice, this may result in overly pessimistic communication of the MS output, undermining the credibility of simulation predictions to decision makers. This presentation proposes an alternative assessment mechanism, utilizing results parameter robustness, also known as model input sensitivity, to improve the credibility scoring process for specific simulations.

Credibility↗
NASA NTRS2015

Ongoing Fixed Wing Research within the NASA Langley Aeroelasticity Branch

The NASA Langley Aeroelasticity Branch is involved in a number of research programs related to fixed wing aeroelasticity and aeroservoelasticity. These ongoing efforts are summarized here, and include aeroelastic tailoring of subsonic transport wing structures, experimental and numerical assessment of truss-braced wing flutter and limit cycle oscillations, and numerical modeling of high speed civil transport configurations. Efforts devoted to verification, validation, and uncertainty quantification of aeroelastic physics in a workshop setting are also discussed. The feasibility of certain future civil transport configurations will depend on the ability to understand and control complex aeroelastic phenomena, a goal that the Aeroelasticity Branch is well-positioned to contribute through these programs.

Bartels, Robert↗
NASA NTRS2015

NASA Space Radiation Risk Project: Overview and Recent Results

The NASA Space Radiation Risk project is responsible for integrating new experimental and computational results into models to predict risk of cancer and acute radiation syndrome (ARS) for use in mission planning and systems design, as well as current space operations. The project has several parallel efforts focused on proving NASA's radiation risk projection capability in both the near and long term. This presentation will give an overview, with select results from these efforts including the following topics: verification, validation, and streamlining the transition of models to use in decision making; relative biological effectiveness and dose rate effect estimation using a combination of stochastic track structure simulations, DNA damage model calculations and experimental data; ARS model improvements; pathway analysis from gene expression data sets; solar particle event probabilistic exposure calculation including correlated uncertainties for use in design optimization.

Blattnig, Steve R.↗
NASA NTRS2014

Addressing the Hard Factors for Command File Errors by Probabilistic Reasoning

Command File Errors (CFE) are managed using standard risk management approaches at the Jet Propulsion Laboratory. Over the last few years, more emphasis has been made on the collection, organization, and analysis of these errors for the purpose of reducing the CFE rates. More recently, probabilistic modeling techniques have been used for more in depth analysis of the perceived error rates of the DAWN mission and for managing the soft factors in the upcoming phases of the mission. We broadly classify the factors that can lead to CFE's as soft factors, which relate to the cognition of the operators and hard factors which relate to the Mission System which is composed of the hardware, software and procedures used for the generation, verification & validation and execution of commands. The focus of this paper is to use probabilistic models that represent multiple missions at JPL to determine the root cause and sensitivities of the various components of the mission system and develop recommendations and techniques for addressing them. The customization of these multi-mission models to a sample interplanetary spacecraft is done for this purpose.

Meshkat, Leila↗
NASA NTRS2016

Zero Boil-Off Tank (ZBOT) Experiment

The Zero-Boil-Off Tank (ZBOT) experiment has been developed as a small scale ISS experiment aimed at delineating important fluid flow, heat and mass transport, and phase change phenomena that affect cryogenic storage tank pressurization and pressure control in microgravity. The experiments use a simulant transparent low boiling point fluid (PnP) in a sealed transparent Dewar to study and quantify: (a) fluid flow and thermal stratification during pressurization; (b) mixing, thermal destratification, depressurization, and jet-ullage penetration during pressure control by jet mixing. The experiment will provide valuable microgravity empirical two-phase data associated with the above-mentioned physical phenomena through highly accurate local wall and fluid temperature and pressure measurements, full-field phase-distribution and flow visualization. Moreover, the experiments are performed under tightly controlled and definable heat transfer boundary conditions to provide reliable high-fidelity data and precise input as required for validation verification of state-of-the-art two-phase CFD models developed as part of this research and by other groups in the international scientific and cryogenic fluid management communities.

ISS↗
NASA NTRS2017

Risk-Significant Adverse Condition Awareness Strengthens Assurance of Fault Management Systems

As spaceflight systems increase in complexity, Fault Management (FM) systems are ranked high in risk-based assessment of software criticality, emphasizing the importance of establishing highly competent domain expertise to provide assurance. Adverse conditions (ACs) and specific vulnerabilities encountered by safety- and mission-critical software systems have been identified through efforts to reduce the risk posture of software-intensive NASA missions. Acknowledgement of potential off-nominal conditions and analysis to determine software system resiliency are important aspects of hazard analysis and FM. A key component of assuring FM is an assessment of how well software addresses susceptibility to failure through consideration of ACs. Focus on significant risk predicted through experienced analysis conducted at the NASA Independent Verification & Validation (IV&V) Program enables the scoping of effective assurance strategies with regard to overall asset protection of complex spaceflight as well as ground systems. Research efforts sponsored by NASAs Office of Safety and Mission Assurance (OSMA) defined terminology, categorized data fields, and designed a baseline repository that centralizes and compiles a comprehensive listing of ACs and correlated data relevant across many NASA missions. This prototype tool helps projects improve analysis by tracking ACs and allowing queries based on project, mission type, domain/component, causal fault, and other key characteristics. Vulnerability in off-nominal situations, architectural design weaknesses, and unexpected or undesirable system behaviors in reaction to faults are curtailed with the awareness of ACs and risk-significant scenarios modeled for analysts through this database. Integration within the Enterprise Architecture at NASA IV&V enables interfacing with other tools and datasets, technical support, and accessibility across the Agency. This paper discusses the development of an improved workflow process utilizing this database for adaptive, risk-informed FM assurance that critical software systems will safely and securely protect against faults and respond to ACs in order to achieve successful missions.

IV&V↗
NASA NTRS2017

Risk-Significant Adverse Condition Awareness Strengthens Assurance of Fault Management Systems

As spaceflight systems increase in complexity, Fault Management (FM) systems are ranked high in risk-based assessment of software criticality, emphasizing the importance of establishing highly competent domain expertise to provide assurance. Adverse conditions (ACs) and specific vulnerabilities encountered by safety- and mission-critical software systems have been identified through efforts to reduce the risk posture of software-intensive NASA missions. Acknowledgement of potential off-nominal conditions and analysis to determine software system resiliency are important aspects of hazard analysis and FM. A key component of assuring FM is an assessment of how well software addresses susceptibility to failure through consideration of ACs. Focus on significant risk predicted through experienced analysis conducted at the NASA Independent Verification Validation (IVV) Program enables the scoping of effective assurance strategies with regard to overall asset protection of complex spaceflight as well as ground systems. Research efforts sponsored by NASA's Office of Safety and Mission Assurance defined terminology, categorized data fields, and designed a baseline repository that centralizes and compiles a comprehensive listing of ACs and correlated data relevant across many NASA missions. This prototype tool helps projects improve analysis by tracking ACs and allowing queries based on project, mission type, domaincomponent, causal fault, and other key characteristics. Vulnerability in off-nominal situations, architectural design weaknesses, and unexpected or undesirable system behaviors in reaction to faults are curtailed with the awareness of ACs and risk-significant scenarios modeled for analysts through this database. Integration within the Enterprise Architecture at NASA IVV enables interfacing with other tools and datasets, technical support, and accessibility across the Agency. This paper discusses the development of an improved workflow process utilizing this database for adaptive, risk-informed FM assurance that critical software systems will safely and securely protect against faults and respond to ACs in order to achieve successful missions.

Fault management↗
NASA NTRS2017

The Integrated Medical Model: Outcomes from Independent Review

In 2016, the Integrated Medical Model (IMM) v4.0 underwent an extensive external review in preparation for transition to an operational status. In order to insure impartiality of the review process, the Exploration Medical Capabilities Element of NASA's Human Research Program convened the review through the Systems Review Office at NASA Goddard Space Flight Center (GSFC). The review board convened by GSFC consisted of persons from both NASA and academia with expertise in the fields of statistics, epidemiology, modeling, software development, aerospace medicine, and project management (see Figure 1). The board reviewed software and code standards, as well as evidence pedigree associated with both the input and outcomes information. The board also assesses the models verification, validation, sensitivity to parameters and ability to answer operational questions. This talk will discuss the processes for designing the review, how the review progressed and the findings from the board, as well as summarize the IMM project responses to those findings. Overall, the board found that the IMM is scientifically sound, represents a necessary, comprehensive approach to identifying medical and environmental risks facing astronauts in long duration missions and is an excellent tool for communication between engineers and physicians. The board also found IMM and its customer(s) should convene an additional review of the IMM data sources and to develop a sustainable approach to augment, peer review, and maintain the information utilized in the IMM. The board found this is critically important because medical knowledge continues to evolve. Delivery of IMM v4.0 to the Crew Health and Safety (CHS) Program will occur in the 2017. Once delivered for operational decision support, IMM v4.0 will provide CHS with additional quantitative capability in to assess astronaut medical risks and required medical capabilities to help drive down overall mission risks.

medical equipment↗
NASA NTRS2019

Uncertainty Quantification for JPL Retrievals

We define the notions of verification, validation, and uncertainty quantification (VVUQ), and place them in the context of retrievals of Earth science data. Uncertainty quantification (UQ) is the process of characterizing the uncertainty of a measurement system relative to the true, physical quantity it measures. UQ is needed when measurements are used for scientific inference, assimilated into models with other data, and used for decision-making and risk analysis. Many measurements come with measures of standard error and bias, but these uncertainty measures are often inaccurate, due to the operational need to assume crisp values for parameters which are in fact uncertain, as well as the failure of technical assumptions needed by operational error-estimation approaches. UQ provides a way to study which assumptions fail to hold, and a tool to improve the measurement. We describe a particular, broadly-applicable UQ setup that has been successful for JPL retrievals including OCO-2, and provide recommendations for furthering the UQ state of the practice at JPL.

Turmon, Michael↗
NASA NTRS2020

Defining Handling Qualities of Unmanned Aerial Systems: Phase II Final Report

Unmanned Air Systems (UAS) are no longer coming, they are here, and operators from first responders to Google and Amazon are demanding access to the National Airspace System (NAS) for a wide variety of missions. This includes a proliferation of small UAS or sUAS that will operate beyond line of sight at altitudes of 500 ft and below. A myriad of issues continues to slow the development of verification, validation, and certification methods that will enable the safe introduction of UAS to the NAS. These issues include the lack of both a consensus in UAS categorization process and quantitative certification requirements, including the definition of handling qualities. Because of the wide variety of UAS types (fixed wing, rotary wing from traditional helicopters to multirotor configurations, ducted fans, airships, etc.) and vehicle size from micro vehicles to the Global Hawk with a wing span similar to that of a Boeing 737, there cannot be a one-size-fits-all set of requirements. To address these issues, Systems Technology Inc. (STI) has developed the UAS Handling Qualities Assessment (UAS-HQ) process and corresponding draft specification that will guide UAS stakeholders through a systematic evaluation process. The work described herein builds on the existing, highly successful, military rotorcraft handling qualities specifications that features a mission-oriented approach, a concept that originated at STI. The vehicle is first identified by a simple weight-based classification and then the associated vehicle mission task elements are considered. These missions have specific tasks inclusive to them that then dictate the criteria and demonstration maneuvers necessary to evaluate the UAS handling qualities. An assessment of both modeled responses and flight test data can then be conducted to examine the predicted versus actual handling qualities and, if required, design modifications can then be made. Mr. David Klyde, Vice President and Technical Director, Engineering Services, served as Principal Investigator, while Dr. Natalia Alexandrov served as the NASA LaRC technical representative. In the Phase II program, STI was joined by David Mitchell of Mitchell Aerospace Research and the University of Minnesota. Mr. Mitchell led the draft specification development effort, while the University of Minnesota UAV Lab conducted sUAS flight tests under the direction of Dr. Peter Seiler.

Klyde, David H.↗
NASA NTRS2020

Europa Clipper Mission: Onward to Implementation!

Europa, the fourth largest moon of Jupiter, is believed to be one of the best places in the solar system to look for extant life beyond Earth, and the Europa Clipper Mission is preparing to investigate. Now entering the fifth year of life as a formal project, the Europa Clipper Mission has completed its major design phase and is preparing for implementation. The mission envisions sending a flight system, consisting of a spacecraft equipped with a payload of NASA-selected scientific instruments, to execute numerous flybys of Europa while in Jupiter orbit. The mission is being formulated and implemented by a joint Jet Propulsion Laboratory (JPL) and Applied Physics Laboratory (APL) Project team. At the Project Preliminary Design Review (PDR) in August 2018, it was determined that the unique integration of the REASON instrument’s antennas with the spacecraft’s solar arrays required additional time for maturity and review. The required maturity was demonstrated, and in June 2019 the Clipper Mission passed its Delta Preliminary Design Review, and formal confirmation for entry into Phase C was announced by NASA in August 2019. The earliest launch opportunity is now in July 2023, one year later than originally planned. The Project is now working toward its Critical Design Review in May 2020. This paper will describe the progress of the Europa Clipper Mission since January 2019, including maturation of the spacecraft, subsystem and instrument designs, the beginning of fabrication of engineering models and the first flight models, planning for the verification & validation phase, and planning for operations.

Jackson, Maddalena↗
NASA NTRS2020

Establishing Launch Readiness of NASA ISS Payload OCO-3

The Orbiting Carbon Observatory – 3 (OCO-3) is a NASA Earth-monitoring payload, designed and built by the Jet Propulsion Laboratory (JPL) to perform space-based observation of variations of global carbon dioxide (CO2) by sampling its emission sources and gradients from the unique vantage point of the International Space Station (ISS). This paper describes the development and results of the end-to-end verification, validation and calibration activities that were performed to meet OCO-3's science measurement, functional and performance requirements prior to launch. The pre-launch activities encompassed a suite of tests and analyses that were performed after payload integration and during launch vehicle integration that helped the OCO-3 project demonstrate its launch and mission readiness in terms of ISS compatibility, ground-based instrument calibrations, flight dynamics and environmental integrity and pointing knowledge and accuracy.

Srivastava, Priyanka↗
NASA NTRSDate not supplied

Applications of Advanced Perception and Distributed Sensing Technology towards Autonomous Advanced Air Mobility Applications

Emerging concepts for Advanced Air Mobility (AAM) envisions responsive air transportation capabilities that can safely move people and cargo between places - including local, regional, intraregional, and urban - previously not served or underserved by aviation. Expanding traditional aviation services to these environments, particularly when autonomous operations are involved, face a number of challenges that will require advances beyond the state-of-the-art techniques for airborne sensing and perception, which goes beyond the limits of scalability and applicability of the current air transportation system infrastructure. Additionally, the emerging field of distributed sensing and ‘smart spaces’– where sensing, processing, communication, and actuation are embedded in the environment in which agents are acting and can be exploited by the agents through real-time wireless communication – may provide realistic near-time solutions to limitations imposed by traditional aviation techniques. This paper outlines the needs, challenges, and opportunities for advanced perception and distributed sensing techniques to meet the emerging needs for advanced AAM operations in the national airspace. A general roadmap for research, development, and maturation of perception and distributed sensing (P&DS) technologies is proposed to guide future development through verification, validation, certification into airborne systems. Through this analysis of challenges and the proposed roadmap for technology maturation, we hope to accelerate transition of advanced research techniques from other disciplines into this domain.

Autonomy↗