Independent Verification and Validation of Artemis I Separation Events
Explore the source record for details and available documents.
Paul V Tartabini↗
SEARCH · Engineering Papers
Results for “verification and validation”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
An assessment of space shuttle flight software development processes
In early 1991, the National Aeronautics and Space Administration's (NASA's) Office of Space Flight commissioned the Aeronautics and Space Engineering Board (ASEB) of the National Research Council (NRC) to investigate the adequacy of the current process by which NASA develops and verifies changes and updates to the Space Shuttle flight software. The Committee for Review of Oversight Mechanisms for Space Shuttle Flight Software Processes was convened in Jan. 1992 to accomplish the following tasks: (1) review the entire flight software development process from the initial requirements definition phase to final implementation, including object code build and final machine loading; (2) review and critique NASA's independent verification and validation process and mechanisms, including NASA's established software development and testing standards; (3) determine the acceptability and adequacy of the complete flight software development process, including the embedded validation and verification processes through comparison with (1) generally accepted industry practices, and (2) generally accepted Department of Defense and/or other government practices (comparing NASA's program with organizations and projects having similar volumes of software development, software maturity, complexity, criticality, lines of code, and national standards); (4) consider whether independent verification and validation should continue. An overview of the study, independent verification and validation of critical software, and the Space Shuttle flight software development process are addressed. Findings and recommendations are presented.
Source record↗
Validation (not just verification) of Deep Space Missions
ion & Validation (V&V) is a widely recognized and critical systems engineering function. However, the often used definition 'Verification proves the design is right; validation proves it is the right design' is rather vague. And while Verification is a reasonably well standardized systems engineering process, Validation is a far more abstract concept and the rigor and scope applied to it varies widely between organizations and individuals. This is reflected in the findings in recent Mishap Reports for several NASA missions, in which shortfalls in Validation (not just Verification) were cited as root- or contributing-factors in catastrophic mission loss. Furthermore, although there is strong agreement in the community that Test is the preferred method for V&V, many people equate 'V&V' with 'Test', such that Analysis and Modeling aren't given comparable attention. Another strong motivator is a realization that the rapid growth in complexity of deep-space missions (particularly Planetary Landers and Space Observatories given their inherent unknowns) is placing greater demands on systems engineers to 'get it right' with Validation.
verification↗
Robotic and Crewed Mars Missions Increasing the Demand for Planetary Protection Technology Needs
Planetary protection (PP) policy seeks to avoid harmful contamination by limiting biological and relevant organic contamination from spacecraft as well as preventing adverse changes to Earth’s biosphere when extraterrestial samples are brought back to Earth. The PP policy at NASA was updated in 2021 (NPR 8715.24) and 2022 (NASA-STD-8719.27) to enable missions by expanding the decades old prescriptive requirements to allow for an option of adopting performance-based requirements that are objectivesdriven, risk-informed and case-assured. In parallel, the final PP knowledge gap workshop was completed representing the international consensus on the key areas to be considered in developing crew PP policy. These knowledge gaps focused on key technology development areas in 1) microbial and human health monitoring, 2) technical and operations needed for contamination control and 3) natural transport of contamination on Mars. As robotic missions start to implement performance-based approaches and research and technology efforts commence to inform crew policy the demand for data quality driven verification and validation in relevant space environments. Examples of the types of testing that is envisioned includes test as you fly validation and verification of decontamination systems in a relevant on-orbit and Mars environment, developing lethality curves of terrestrial organisms to further our understanding of the biocidal impacts of Mars and the space environment, and particle transport model validation and verification. Thus, the PP discipline has identified the need for groundbased space environments to perform preliminary testing as validation and verification of flight systems and to advance the technology readiness level prior to further testing on-orbit or lunar environments to prepare for Mars.
J. Nick Benardini↗
Robotic and Crewed Mars Missions Increasing the Demand for Planetary Protection Technology Needs
Planetary protection (PP) policy seeks to avoid harmful contamination by limiting biological and relevant organic contamination from spacecraft as well as preventing adverse changes to Earth’s biosphere when extraterrestrial samples are brought back to Earth. The PP policy at NASA was updated in 2021 (NPR 8715.24) and 2022 (NASA-STD-8719.27) to enable missions by expanding the decades old prescriptive requirements to allow for an option of adopting performance-based requirements that are objectives-driven, risk-informed and case-assured. In parallel, the final PP knowledge gap workshop was completed representing the international consensus on the key areas to be considered in developing crew PP policy. These knowledge gaps focused on key technology development areas in 1) microbial and human health monitoring, 2) technical and operations needed for contamination control and 3) natural transport of contamination on Mars. As robotic missions start to implement performance-based approaches and research and technology efforts commence to inform crew policy the demand for data quality driven verification and validation in relevant space environments. Examples of the types of testing that is envisioned includes test as you fly validation and verification of decontamination systems in a relevant on-orbit and Mars environment, developing lethality curves of terrestrial organisms to further our understanding of the biocidal impacts of Mars and the space environment, and particle transport model validation and verification. Thus, the PP discipline has identified the need for ground-based space environments to perform preliminary testing as validation and verification of flight systems and to advance the technology readiness level prior to further testing on-orbit or lunar environments to prepare for Mars.
J Nick Benardini↗
Fusion of Test and Analysis: Artemis I Booster to Mobile Launcher Interface Validation
NASA is in the midst of bold and exciting next steps in human exploration and spaceflight. The designs of the new Space Launch System (SLS), the Orion spacecraft and the Exploration Ground Systems (EGS) for vehicle processing and launch are essentially complete and there has been significant progress in manufacturing and assembly of specific hardware for the Artemis I and Artemis II missions. Equally as important, the program level and integrated system level testing and analyses are also well underway to support integrated verification, validation, and Certificate of Flight Readiness (CoFR) for Artemis I. Testing and analysis are key to addressing technical challenges faced by the Artemis missions. Building block approaches are required that provide the right balance between component, element, and/or system level testing that satisfies verification and validation objectives where uncertainties are quantified and minimized. Artemis I is a system of systems that requires a fusion of test and analysis that adeptly characterizes critical interfaces between major program elements. An example of this fusion involves characterizing the interface between the SLS booster and the Mobile Launcher (ML) Vertical Support Post (VSP) interfaces. Proper characterization of this interface represents a number of challenges beginning with the fact that it is a mating of ground support structure in the form of a civil structure to flight hardware. Both sides of the interface are built to different construction standards, but are governed by interface requirements to ensure compatibility when mated. From past program experience, the flexibility at the booster to ML interface is critical in developing accurate prelaunch stacking and cryogenic preloads, squat loads, and pad separation release of preloads and squat loads. This same premise holds for Artemis I. To characterize the asymmetric characteristics at this interface, careful consideration of static forces due to gravity loading with the commensurate effects due to leveling during booster stacking (i.e., spacing and shimming) and nonlinear geometric forces are necessary for inclusion in pre-test assessments. This paper will look at these issues for the upcoming Booster Pull Test in which two boosters will be installed on the ML and one of these boosters will undergo static lateral loading followed afterwards with dynamic excitation into resonance and free-decay. This paper evaluates the booster to ML interface characteristics by characterizing the interface flexibility between the booster aft skirt and the ML VSP interfaces. Furthermore, this paper methodically evaluates the effect of the following on the test outcome: gravitational effects on the booster and ML, the effects of VSP leveling, spacing, and shimming under gravitational loading during booster stacking, the effect of geometric nonlinear follower force due to cg offset as booster is laterally displaced, and the system coupling between the booster under test, ML, and the second booster. Simulated results for a static load pull and dynamic excitation provide insight into the differences in measurement responses when boundary conditions and geometric conditions are included and not included.
Joel W Sills Jr.↗
Multidisciplinary Dynamic Testing Challenges in Validating the NASA Artemis Architecture
NASA is in the midst of bold and exciting next steps in human exploration and spaceflight. The designs of the new Space Launch System (SLS), the Orion spacecraft and the Exploration Ground Systems (EGS) for vehicle processing and launch are essentially complete and there has been significant progress in manufacturing and assembly of specific hardware for the Artemis I and Artemis II missions. Equally as important, the program level and integrated system level testing and analyses are also well underway to support integrated verification, validation, and certificate of flight readiness (CoFR) for the first Artemis mission. Testing and analysis are key to addressing technical challenges that the Artemis missions offer. Building block approaches are required that provide the right balance between component, system, and/or element level testing that satisfies verification and validation objectives and where, uncertainties are quantified and minimized. Artemis I is a system of systems that requires a fusion of test and analysis that adeptly characterizes critical interfaces between major program elements. NASA is implementing new in-situ testing that fuse traditional aerospace structures with civil structures, such as the Integrated Modal Test for the Artemis I vehicle where the Mobile Launcher and Crawler Transporter serve as a support structure whose dynamics couple with that of the Artemis I vehicle. This new paradigm requires a closer inspection of structural behavior of the Crawler Transporter and the Mobile Launcher as they now serve multiple purposes. This requires a paradigm shift to look beyond experimental modal techniques and incorporates operational modal analysis techniques to validate dynamic models from data collected during rollout to the launch pad. A further complicating factor is the Crawler Transporter generated ground forces have numerous harmonics making extracting dynamic responses of the Artemis I, Mobile Launcher, and Crawler Transporter coupled system challenging. This discussion explores all these challenges with and attempts to understand how we best build confidence in systems and system-of-systems performance capabilities and margins and understand uncertainties.
Joel W Sills↗
Autonomy Software: V&V Challenges and Characteristics
The successful operation of unmanned air vehicles requires software with a high degree of autonomy. Only if high level functions can be carried out without human control and intervention, complex missions in a changing and potentially unknown environment can be carried out successfully. Autonomy software is highly mission and safety critical: failures, caused by flaws in the software cannot only jeopardize the mission, but could also endanger human life (e.g., a crash of an UAV in a densely populated area). Due to its large size, high complexity, and use of specialized algorithms (planner, constraint-solver, etc.), autonomy software poses specific challenges for its verification, validation, and certification. -- - we have carried out a survey among researchers aid scientists at NASA to study these issues. In this paper, we will present major results of this study, discussing the broad spectrum. of notions and characteristics of autonomy software and its challenges for design and development. A main focus of this survey was to evaluate verification and validation (V&V) issues and challenges, compared to the development of "traditional" safety-critical software. We will discuss important issues in V&V of autonomous software and advanced V&V tools which can help to mitigate software risks. Results of this survey will help to identify and understand safety concerns in autonomy software and will lead to improved strategies for mitigation of these risks.
Schumann, Johann↗
Systems engingeering for the Kepler Mission : a search for terrestrial planets
The Kepler mission will launch in 2007 and determine the distribution of earth-size planets (0.5 to 10 earth masses) in the habitable zones (HZs) of solar-like stars. The mission will monitor > 100,000 dwarf stars simultaneously for at least 4 years. Precision differential photometry will be used to detect the periodic signals of transiting planets. Kepler will also support asteroseismology by measuring the pressure-mode (p-mode) oscillations of selected stars. Key mission elements include a spacecraft bus and 0.95 meter, wide-field, CCD-based photometer injected into an earth-trailing heliocentric orbit by a 3-stage Delta II launch vehicle as well as a distributed Ground Segment and Follow-up Observing Program. The project is currently preparing for Preliminary Design Review (October 2004) and is proceeding with detailed design and procurement of long-lead components. In order to meet the unprecedented photometric precision requirement and to ensure a statistically significant result, the Kepler mission involves technical challenges in the areas of photometric noise and systematic error reduction, stability, and false-positive rejection. Programmatic and logistical challenges include the collaborative design, modeling, integration, test, and operation of a geographically and functionally distributed project. A very rigorous systems engineering program has evolved to address these challenges. This paper provides an overview of the Kepler systems engineering program, including some examples of our processes and techniques in areas such as requirements synthesis, validation & verification, system robustness design, and end-to-end performance modeling.
systems engineering↗
Evolution of Software-Only-Simulation at NASA IV and V
Software-Only-Simulations have been an emerging but quickly developing field of study throughout NASA. The NASA Independent Verification Validation (IVV) Independent Test Capability (ITC) team has been rapidly building a collection of simulators for a wide range of NASA missions. ITC specializes in full end-to-end simulations that enable developers, VV personnel, and operators to test-as-you-fly. In four years, the team has delivered a wide variety of spacecraft simulations that have ranged from low complexity science missions such as the Global Precipitation Management (GPM) satellite and the Deep Space Climate Observatory (DSCOVR), to the extremely complex missions such as the James Webb Space Telescope (JWST) and Space Launch System (SLS).This paper describes the evolution of ITCs technologies and processes that have been utilized to design, implement, and deploy end-to-end simulation environments for various NASA missions. A comparison of mission simulators are discussed with focus on technology and lessons learned in complexity, hardware modeling, and continuous integration. The paper also describes the methods for executing the missions unmodified flight software binaries (not cross-compiled) for verification and validation activities.
Embedded↗
Development of NASA's Models and Simulations Standard
From the Space Shuttle Columbia Accident Investigation, there were several NASA-wide actions that were initiated. One of these actions was to develop a standard for development, documentation, and operation of Models and Simulations. Over the course of two-and-a-half years, a team of NASA engineers, representing nine of the ten NASA Centers developed a Models and Simulation Standard to address this action. The standard consists of two parts. The first is the traditional requirements section addressing programmatics, development, documentation, verification, validation, and the reporting of results from both the M&S analysis and the examination of compliance with this standard. The second part is a scale for evaluating the credibility of model and simulation results using levels of merit associated with 8 key factors. This paper provides an historical account of the challenges faced by and the processes used in this committee-based development effort. This account provides insights into how other agencies might approach similar developments. Furthermore, we discuss some specific applications of models and simulations used to assess the impact of this standard on future model and simulation activities.
simulation↗
HRP's Healthcare Spin-Offs Through Computational Modeling and Simulation Practice Methodologies
Spaceflight missions expose astronauts to novel operational and environmental conditions that pose health risks that are currently not well understood, and perhaps unanticipated. Furthermore, given the limited number of humans that have flown in long duration missions and beyond low Earth-orbit, the amount of research and clinical data necessary to predict and mitigate these health and performance risks are limited. Consequently, NASA's Human Research Program (HRP) conducts research and develops advanced methods and tools to predict, assess, and mitigate potential hazards to the health of astronauts. In this light, NASA has explored the possibility of leveraging computational modeling since the 1970s as a means to elucidate the physiologic risks of spaceflight and develop countermeasures. Since that time, substantial progress has been realized in this arena through a number of HRP funded activates such as the Digital Astronaut Project (DAP) and the Integrated Medical Model (IMM). Much of this success can be attributed to HRP's endeavor to establish rigorous verification, validation, and credibility (VV&C) processes that ensure computational models and simulations (M&S) are sufficiently credible to address issues within their intended scope. This presentation summarizes HRP's activities in credibility of modeling and simulation, in particular through its outreach to the community of modeling and simulation practitioners. METHODS: The HRP requires all M&S that can have moderate to high impact on crew health or mission success must be vetted in accordance to NASA Standard for Models and Simulations, NASA-STD-7009 (7009) [5]. As this standard mostly focuses on engineering systems, the IMM and DAP have invested substantial efforts to adapt the processes established in this standard for their application to biological M&S, which is more prevalent in human health and performance (HHP) and space biomedical research and operations [6,7]. These methods have also generated substantial interest by the broader medical community though institutions like the National Institutes of Health (NIH) and the Food and Drug Administration (FDA) to develop similar standards and guidelines applicable to the larger medical operations and research community. DISCUSSION: Similar to NASA, many leading government agencies, health institutions and medical product developers around the world are recognizing the potential of computational M&S to support clinical research and decision making. In this light, substantial investments are being made in computational medicine and notable discoveries are being realized [8]. However, there is a lack of broadly applicable practice guidance for the development and implementation of M&S in clinical care and research in a manner that instills confidence among medical practitioners and biological researchers [9,10]. In this presentation, we will give an overview on how HRP is working with the NIH's Interagency Modeling and Analysis Group (IMAG), the FDA and the American Society of Mechanical Engineers (ASME) to leverage NASA's biomedical VV&C processes to establish a new regulatory standard for Verification and Validation in Computational Modeling of Medical Devices, and Guidelines for Credible Practice of Computational Modeling and Simulation in Healthcare.
Mulugeta, Lealem↗
Assurance of Fault Management: Risk-Significant Adverse Condition Awareness
Fault Management (FM) systems are ranked high in risk-based assessment of criticality within flight software, emphasizing the importance of establishing highly competent domain expertise to provide assurance for NASA projects, especially as spaceflight systems continue to increase in complexity. Insight into specific characteristics of FM architectures seen embedded within safety- and mission-critical software systems analyzed by the NASA Independent Verification Validation (IVV) Program has been enhanced with an FM Technical Reference (TR) suite. Benefits are aimed beyond the IVV community to those that seek ways to efficiently and effectively provide software assurance to reduce the FM risk posture of NASA and other space missions. The identification of particular FM architectures, visibility, and associated IVV techniques provides a TR suite that enables greater assurance that critical software systems will adequately protect against faults and respond to adverse conditions. The role FM has with regard to overall asset protection of flight software systems is being addressed with the development of an adverse condition (AC) database encompassing flight software vulnerabilities.Identification of potential off-nominal conditions and analysis to determine how a system responds to these conditions are important aspects of hazard analysis and fault management. Understanding what ACs the mission may face, and ensuring they are prevented or addressed is the responsibility of the assurance team, which necessarily should have insight into ACs beyond those defined by the project itself. Research efforts sponsored by NASAs Office of Safety and Mission Assurance defined terminology, categorized data fields, and designed a baseline repository that centralizes and compiles a comprehensive listing of ACs and correlated data relevant across many NASA missions. This prototype tool helps projects improve analysis by tracking ACs, and allowing queries based on project, mission type, domain component, causal fault, and other key characteristics. The repository has a firm structure, initial collection of data, and an interface established for informational queries, with plans for integration within the Enterprise Architecture at NASA IVV, enabling support and accessibility across the Agency. The development of an improved workflow process for adaptive, risk-informed FM assurance is currently underway.
Software Verification & Validation↗
Commercial Systems Validation Overview
The Commercial Systems Validation Laboratory (CSVL) is composed of the 1) Virtual Product Laboratory (VPL), 2) Commercial Instrument Validation Laboratory (CIVL), and 3) In-Flight Verification & Validation Network (IVVN). All three laboratories are described in this viewgraph presentation, which covers the characterization of CCD cameras, hyperspectral sensors, ground reference equipment, and the Multispectral Thermal Imager (MTI). The presentation also includes sections on thermal modeling, measurements taken from a float, and the Scientific Data Purchase (SDP) Program. The section of the SDP profiles four data providers: 1) EarthSat; 2) Intermap; 3) Positive Systems; 4) Space Imaging.
Ryan, Robert↗
Microthruster Propulsion for the Space Technology 7 (ST7) Technology Demonstration Mission
For future applications to precision formation flying missions, NASA's New Millennium Program is scheduled to test colloid micro-Newton thrusters (CMNTs) on the ST7 technology demonstration mission. These CMNTs are part of a disturbance reduction system (DRS) on the ESA SMART-2 Spacecraft or LISA Pathfinder. The goal of the ST7 DRS is to demonstrate technologies necessary to meet the nanometer precision positioning control requirements of the LISA mission. In order to achieve these goals, the CMNTs are required to demonstrate a thrust resolution of less than 0.1 micro-N and a thrust noise of less than 0.1 micro-N/[square root]Hz for thrust levels between 5 and 30 micro-N. Developed by Busek Co. with support from JPL in testing an design, the CMNT has been developed over the last four years into a flight-ready microthrust system. The development, validation testing, and flight unit production of the CMNTs are described. Development tests and analysis include preliminary wear tests, propellant loading process verification, flow testing, and performance verification. Validation and flight unit verification includes thermal and structural analysis, life testing, thermal and dynamic load testing, and performance verification. Final delivery of the units is planned in 2007 with and planned launch and flight demonstration 2009.
microthruster↗
Thermal System Verification and Model Validation for NASA's Cryogenic Passively Cooled James Webb Space Telescope
A thorough and unique thermal verification and model validation plan has been developed for NASA s James Webb Space Telescope. The JWST observatory consists of a large deployed aperture optical telescope passively cooled to below 50 Kelvin along with a suite of several instruments passively and actively cooled to below 37 Kelvin and 7 Kelvin, respectively. Passive cooling to these extremely low temperatures is made feasible by the use of a large deployed high efficiency sunshield and an orbit location at the L2 Lagrange point. Another enabling feature is the scale or size of the observatory that allows for large radiator sizes that are compatible with the expected power dissipation of the instruments and large format Mercury Cadmium Telluride (HgCdTe) detector arrays. This passive cooling concept is simple, reliable, and mission enabling when compared to the alternatives of mechanical coolers and stored cryogens. However, these same large scale observatory features, which make passive cooling viable, also prevent the typical flight configuration fully-deployed thermal balance test that is the keystone to most space missions thermal verification plan. JWST is simply too large in its deployed configuration to be properly thermal balance tested in the facilities that currently exist. This reality, when combined with a mission thermal concept with little to no flight heritage, has necessitated the need for a unique and alternative approach to thermal system verification and model validation. This paper describes the thermal verification and model validation plan that has been developed for JWST. The plan relies on judicious use of cryogenic and thermal design margin, a completely independent thermal modeling cross check utilizing different analysis teams and software packages, and finally, a comprehensive set of thermal tests that occur at different levels of JWST assembly. After a brief description of the JWST mission and thermal architecture, a detailed description of the three aspects of the thermal verification and model validation plan is presented.
Cleveland, Paul E.↗
Thermal System Verification and Model Validation for NASA's Cryogenic Passively Cooled James Webb Space Telescope (JWST)
A thorough and unique thermal verification and model validation plan has been developed for NASA s James Webb Space Telescope. The JWST observatory consists of a large deployed aperture optical telescope passively cooled to below 50 Kelvin along with a suite of several instruments passively and actively cooled to below 37 Kelvin and 7 Kelvin, respectively. Passive cooling to these extremely low temperatures is made feasible by the use of a large deployed high efficiency sunshield and an orbit location a! !he L2 Lagrange p~in!. Another enabling feature is the scale or size of the observatory that allows for large radiator sizes that are compatible with the expected power dissipation of the instruments and large format Mercury Cadmium Telluride (HgCdTe) detector arrays. This passive cooling concept is simple, reliable, and mission enabling when compared to the alternatives of mechanical coolers and stored cryogens. However, these same large scale observatory features, which make passive cooling viable, also prevent the typical flight configuration fully-deployed thermal balance test that is the keystone to most space missions thermal verification plan. JWST is simply too large in its deployed configuration to be properly thermal balance tested in the facilities that currently exist. This reality, when combined with a mission thermal concept with little to no flight heritage, has necessitated the need for a unique and alternative approach to thermal system verification and model validation. This paper describes the thermal verification and model validation plan that has been developed for JWST.
Parrish, Keith↗
Emerging technologies for V&V of ISHM software for space exploration
Systems1,2 required to exhibit high operational reliability often rely on some form of fault protection to recognize and respond to faults, preventing faults' escalation to catastrophic failures. Integrated System Health Management (ISHM) extends the functionality of fault protection to both scale to more complex systems (and systems of systems), and to maintain capability rather than just avert catastrophe. Forms of ISHM have been utilized to good effect in the maintenance phase of systems' total lifecycles (often referred to as 'condition-based mainte-nance'), but less so in a 'fault protection' role during actual operations. One of the impediments to such use lies in the challenges of verification, validation and certification of ISHM systems themselves. This paper makes the case that state-of-the-practice V&V and certification techniques will not suffice for emerging forms of ISHM systems; however, a number of maturing software engineering assurance technologies show particular promise for addressing these ISHM V&V challenges.
fault detection, isolation, and recovery↗