Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “software verification and validation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 433 records · Page 24
All sourcesNASA NTRSDOE OSTI
NASA NTRS2016

GPS and Galileo Developments on Board the International Space Station With the Space Communications and Navigation (SCaN) Testbed

The Space Communications and Navigation (SCaN) is a facility developed by NASA and hosted on board the International Space Station (ISS) on an external truss since 2013.It has the objective of testing navigation and communication experimentations with a Software Defined Radio (SDR) approach, which permits software updates for testing new experimentations.NASA has developed the Space Telecommunications Radio System (STRS) architecture standard for SDRs used in space and ground-based platforms to provide commonality among radio developments to provide enhanced capability. The hardware is equipped with both L band front-end radios and the NASA space network communicates with it using S-band, Ku-band and Ka-band links.In May 2016 Qascom started GARISS (GPS and Galileo Receiver for the ISS), an activity of experimentation in collaboration with ESA and NASA that has the objective to develop and validate the acquisition and processing of combined GPS and Galileo signals on board the ISS SCaN testbed. This paper has the objective to present the mission, and provide preliminary details about the challenges in the design, development and verification of the waveform that will be installed on equipment with limited resources. GARISS is also the first attempt to develop a waveform for the ISS as part of an international collaboration between US and Europe. Although the final mission objective is to target dual frequency processing, initial operations will foresee a single frequency processing. Initial results and trade-off between the two options, as well as the final decision will be presented and discussed. The limited resources on board the SCaN with respect to the challenging requirements to acquire and track contemporaneously two satellite navigation systems, with different modulations and data structure, led to the need to assess the possibility of aiding from ground through the S-band. This option would allow assistance to the space receiver in order to provide knowledge of GNSS orbits and reduce the processing on board. Trade off and various options for telemetry and uplink data are presented and discussed. Finally, integration and validation of the waveform are one of the major challenges of GARISS: The Experiment Development System (EDS) and the the Ground Integration Unit (GIU) for VV will be used prior to conducting the experiment on the ISS. The EDS can be used in lab environment and allows prototyping and verification activities with the simulator, but does not include all hardware components. The GIU on the other side is the flight model which replicates the flying equipment, but has limited flexibility for testing.As conclusion, the project is now approaching the Preliminary Design Review (PDR) and indeed only preliminary results are available. This paper is an opportunity to present the GARISS mission as part of an International cooperation between ESA, NASA and Qascom. The preliminary results include GPS and Galileo processing from space signals, the challenges and trade off decisions, the high level STRS architecture and foreseen experimentation campaign. Detailed results from the test campaigns are expected in 2017.

space navigation↗
NASA NTRS2022

Mars 2020 Entry, Descent, and Landing Software Implementation

On February 18th, 2021, the Mars 2020 project's Perseverance Rover successfully touched down on the Martian surface after nearly eight years of development. The Mars 2020 Entry, Descent, and Landing (EDL) System largely leveraged heritage from the Mars Science Laboratory (MSL) EDL System while employing targeted technological advancements. The landing process is autonomously directed by a software behavior implemented in the rover's primary flight computer called the EDL Timeline that assumes control of the vehicle six days before atmospheric entry. In addition to performing the critical function of landing the rover on the Martian surface, the EDL timeline behavior must co-exist in a non-partitioned software and system environment with other high-level functions that accomplish the goals for the rest of the mission. Due to the criticality of EDL, the potential for loss of mission, and a need for complete system autonomy, the standard for how the EDL Timeline interacts with other functions in the system is highly constrained. This paper first walks through the basics of the EDL Timeline mechanics and how the behavior is designed to account for internal system variations and environmental unknowns. It then summarizes the interactions between the EDL timeline and other high-level system behaviors like spacecraft mode transitions and system fault protection, focusing on the complications that arise when passing spacecraft control between executive functions. Although the MSL-inherited EDL System is reliable and capable, targeted updates and a thorough verification and validation program were required for Mars 2020. This paper discusses changes made to close vulnerabilities discovered during both MSL and Mars 2020 development cycles, landing system capability enhancements that were enabling for Mars 2020's mission, and how these updates were integrated with the heritage system. It then describes how both analysis and testing campaigns were utilized to verify and validate all aspects of EDL and system behaviors that run during the six days before landing, as well as the operational workarounds that were needed to address problems found during the development and commissioning process. Finally, this paper imparts lessons learned from Mars 2020 EDL development, implementation, and operations, emphasizing how systems designed to conduct time-critical mission events with low margin of error can be improved in the future.

Stehura, Aaron↗
NASA NTRS2000

Comprehensive Design Reliability Activities for Aerospace Propulsion Systems

This technical publication describes the methodology, model, software tool, input data, and analysis result that support aerospace design reliability studies. The focus of these activities is on propulsion systems mechanical design reliability. The goal of these activities is to support design from a reliability perspective. Paralleling performance analyses in schedule and method, this requires the proper use of metrics in a validated reliability model useful for design, sensitivity, and trade studies. Design reliability analysis in this view is one of several critical design functions. A design reliability method is detailed and two example analyses are provided-one qualitative and the other quantitative. The use of aerospace and commercial data sources for quantification is discussed and sources listed. A tool that was developed to support both types of analyses is presented. Finally, special topics discussed include the development of design criteria, issues of reliability quantification, quality control, and reliability verification.

Christenson, R. L.↗
NASA NTRS2018

Development and Use of the SPACE Computer Code for Analyzing the Space Station Electrical Power System

This special publication tells the story of the dedicated efforts of very talented individuals to create a preeminent space electrical power system modeling and simulation tool called SPACE, short for System Power Analysis for Capability Evaluation. This computer model has evolved for 30 years, withstanding the test of time and obsolescence, and garnering international recognition for its ability to simulate complex space electrical power systems. The analytical results from this model have saved millions of dollars in hardware redesign, testing, and verification for NASA's International Space Station (ISS) and its European and Russian partners. SPACE has played a pivotal role in the station's design and development and continues to support its ongoing operation. It has also extended its reach beyond the ISS to other key NASA programs, where it guides the design and planned operation of NASA's Multi-Purpose Crew Vehicle Orion and simulates electric power system operation in a dusty atmosphere on Mars' surface. The SPACE lineage was created by a core civil servant staff, supplemented by a cadre of interns and other temporary helpers. They created a tightly integrated tool that includes all phenomena that impact a solar array and battery space power system performance. SPACE is self-contained, requiring no other software modules and associated license fees. SPACE "rings true" in that is has been extensively validated with ISS on-orbit telemetry data. This report is being released as the generation of engineers who created it are nearing retirement, passing the baton to a new generation. This next generation will carry the code into the future, no doubt further evolving it to be able to assure mission planners that newly conceived systems will successfully power NASA's next endeavors. As a previous SPACE code developer and analyst, I have worked alongside many of the people mentioned in this report. The engineers who created the code, along with those just now learning it, are among the best and brightest at NASA. It is an honor to write this foreword as the present Branch Chief under which the legacy of SPACE continues to thrive.

Electrical Power System↗
NASA NTRS2020

Enabling Assurance in the MBSE Environment

A number of specific benefits that fit within the hallmarks of effective development are realized with implementation of model-based approaches to systems and assurance. Model Based Systems Engineering (MBSE) enabled by standardized modeling languages (e.g., SysML®) is at the core. These benefits in the context of spaceflight system challenges can include [1]: • Improved management of complex development • Reduced risk in the development process • Improved cost management • Improved design decisions With appropriate modeling techniques the assurance community also can improve early oversight and insight into project development. NASA has shown the basic constructs of SysML in an MBSE environment offer several key advantages, within a Model Based Mission Assurance (MBMA) initiative [2, 3]. These include the following: • Model viewpoints that promote rapid and systematic assessment of requirements coverage, hazard tagging and risk management • Embedded safety assessments for launch vehicles • Deployment of model assisted development of reliability products - Failure Modes and Effects Analyses (FMEAs) and Fault Trees • Test Planning • Validation and Verification of complex functions • Support of Assurance Case development for complex systems In addition, while there are benefits to be harvested, there is a realization that these do not come without effort and cost. Enabling model-based approaches requires structure, not only in an organizational context, but in a modeling context as well. There can be a steep learning curve and costs associated to train skilled modelers. But, on the other hand, not all of the assurance community need to be modelers. Models themselves must conform to ontologies that enable assurance. This places constraints upon the models and modelers. Optimums have yet to be developed where resources and constraints on modeling must be traded off in the organization and modeling efforts for projects. A number of barriers need to be overcome, as well, which pose challenges to the developers of the software that supports MBSE/MBMA. Information and data must be made to flow seamlessly through the life cycle. Because there is a wide variety of tools used in the community, to avoid the problems of the past of silos, delays, and diverging interests, information should flow among these tools to support the “single source of truth” paradigm of MBSE. This will greatly facilitate MBMA and advancement of assurance functions.

Evans, John W.↗
NASA NTRS1993

Integrated design of the CSI evolutionary structure: A verification of the design methodology

One of the main objectives of the Controls-Structures Interaction (CSI) program is to develop and evaluate integrated controls-structures design methodology for flexible space structures. Thus far, integrated design methodologies for a class of flexible spacecraft, which require fine attitude pointing and vibration suppression with no payload articulation, have been extensively investigated. Various integrated design optimization approaches, such as single-objective optimization, and multi-objective optimization, have been implemented with an array of different objectives and constraints involving performance and cost measures such as total mass, actuator mass, steady-state pointing performance, transient performance, control power, and many more. These studies have been performed using an integrated design software tool (CSI-DESIGN CODE) which is under development by the CSI-ADM team at the NASA Langley Research Center. To date, all of these studies, irrespective of the type of integrated optimization posed or objectives and constraints used, have indicated that integrated controls-structures design results in an overall spacecraft design which is considerably superior to designs obtained through a conventional sequential approach. Consequently, it is believed that validation of some of these results through fabrication and testing of a structure which is designed through an integrated design approach is warranted. The objective of this paper is to present and discuss the efforts that have been taken thus far for the validation of the integrated design methodology.

Maghami, Peiman G.↗
NASA NTRS2010

Verification and Validation of Flight-Critical Systems

For the first time in many years, the NASA budget presented to congress calls for a focused effort on the verification and validation (V&V) of complex systems. This is mostly motivated by the results of the VVFCS (V&V of Flight-Critical Systems) study, which should materialize as a a concrete effort under the Aviation Safety program. This talk will present the results of the study, from requirements coming out of discussions with the FAA and the Joint Planning and Development Office (JPDO) to technical plan addressing the issue, and its proposed current and future V&V research agenda, which will be addressed by NASA Ames, Langley, and Dryden as well as external partners through NASA Research Announcements (NRA) calls. This agenda calls for pushing V&V earlier in the life cycle and take advantage of formal methods to increase safety and reduce cost of V&V. I will present the on-going research work (especially the four main technical areas: Safety Assurance, Distributed Systems, Authority and Autonomy, and Software-Intensive Systems), possible extensions, and how VVFCS plans on grounding the research in realistic examples, including an intended V&V test-bench based on an Integrated Modular Avionics (IMA) architecture and hosted by Dryden.

Brat, Guillaume↗
NASA NTRS2000

SAVANT: Solar Array Verification and Analysis Tool Demonstrated

The photovoltaics (PV) industry is now being held to strict specifications, such as end-oflife power requirements, that force them to overengineer their products to avoid contractual penalties. Such overengineering has been the only reliable way to meet such specifications. Unfortunately, it also results in a more costly process than is probably necessary. In our conversations with the PV industry, the issue of cost has been raised again and again. Consequently, the Photovoltaics and Space Environment Effects branch at the NASA Glenn Research Center at Lewis Field has been developing a software tool to address this problem. SAVANT, Glenn's tool for solar array verification and analysis is in the technology demonstration phase. Ongoing work has proven that more efficient and less costly PV designs should be possible by using SAVANT to predict the on-orbit life-cycle performance. The ultimate goal of the SAVANT project is to provide a user-friendly computer tool to predict PV on-orbit life-cycle performance. This should greatly simplify the tasks of scaling and designing the PV power component of any given flight or mission. By being able to predict how a particular PV article will perform, designers will be able to balance mission power requirements (both beginning-of-life and end-of-life) with survivability concerns such as power degradation due to radiation and/or contamination. Recent comparisons with actual flight data from the Photovoltaic Array Space Power Plus Diagnostics (PASP Plus) mission validate this approach.

Chock, Ricaurte↗
NASA NTRS1993

TARGET: Rapid Capture of Process Knowledge

TARGET (Task Analysis/Rule Generation Tool) represents a new breed of tool that blends graphical process flow modeling capabilities with the function of a top-down reporting facility. Since NASA personnel frequently perform tasks that are primarily procedural in nature, TARGET models mission or task procedures and generates hierarchical reports as part of the process capture and analysis effort. Historically, capturing knowledge has proven to be one of the greatest barriers to the development of intelligent systems. Current practice generally requires lengthy interactions between the expert whose knowledge is to be captured and the knowledge engineer whose responsibility is to acquire and represent the expert's knowledge in a useful form. Although much research has been devoted to the development of methodologies and computer software to aid in the capture and representation of some types of knowledge, procedural knowledge has received relatively little attention. In essence, TARGET is one of the first tools of its kind, commercial or institutional, that is designed to support this type of knowledge capture undertaking. This paper will describe the design and development of TARGET for the acquisition and representation of procedural knowledge. The strategies employed by TARGET to support use by knowledge engineers, subject matter experts, programmers and managers will be discussed. This discussion includes the method by which the tool employs its graphical user interface to generate a task hierarchy report. Next, the approach to generate production rules for incorporation in and development of a CLIPS based expert system will be elaborated. TARGET also permits experts to visually describe procedural tasks as a common medium for knowledge refinement by the expert community and knowledge engineer making knowledge consensus possible. The paper briefly touches on the verification and validation issues facing the CLIPS rule generation aspects of TARGET. A description of efforts to support TARGET's interoperability issues on PCs, Macintoshes and UNIX workstations concludes the paper.

Ortiz, C. J.↗
NASA NTRS2023

Autonomy Verification & Validation Roadmap and Vision 2045

Advanced capabilities planned for the next generation of autonomous and increasingly autonomous air vehicles will include non-traditional components based on artificial intelligence, machine learning, and complex optimization and planning algorithms. These complex components will be used to provide enhanced safety and high-level decision-making functions. However, there are serious barriers to the deployment of autonomous aircraft in the National Airspace System (NAS). Current civil aviation certification processes are based on the concept that the correct behavior of a system or a component must be completely specified and verified prior to operation. This report from the Autonomy Verification and Validation (V&V) Roadmap and Vision 2045 project presents the most recent effort to build a comprehensive list of verification challenges and needs for autonomous aircraft, a roadmap to meet those autonomy V&V needs, the services they can enable, and point to the certification gaps they fill. To accomplish these goals, we assembled a team of world-class researchers from the aerospace industry (Boeing, Collins Aerospace, and GeneralElectric) and academia (University of Michigan, University of Texas, and Massachusetts Institute of Technology) with deep expertise in autonomy, aerospace systems, and assurance of Artificial Intelligence/machine learning systems.

Software Assurance↗
NASA NTRS2014

Spot: A Programming Language for Verified Flight Software

The C programming language is widely used for programming space flight software and other safety-critical real time systems. C, however, is far from ideal for this purpose: as is well known, it is both low-level and unsafe. This paper describes Spot, a language derived from C for programming space flight systems. Spot aims to maintain compatibility with existing C code while improving the language and supporting verification with the SPIN model checker. The major features of Spot include actor-based concurrency, distributed state with message passing and transactional updates, and annotations for testing and verification. Spot also supports domain-specific annotations for managing spacecraft state, e.g., communicating telemetry information to the ground. We describe the motivation and design rationale for Spot, give an overview of the design, provide examples of Spot's capabilities, and discuss the current status of the implementation.

validation↗
NASA NTRS2008

Automated Generation and Assessment of Autonomous Systems Test Cases

This slide presentation reviews some of the issues concerning verification and validation testing of autonomous spacecraft routinely culminates in the exploration of anomalous or faulted mission-like scenarios using the work involved during the Dawn mission's tests as examples. Prioritizing which scenarios to develop usually comes down to focusing on the most vulnerable areas and ensuring the best return on investment of test time. Rules-of-thumb strategies often come into play, such as injecting applicable anomalies prior to, during, and after system state changes; or, creating cases that ensure good safety-net algorithm coverage. Although experience and judgment in test selection can lead to high levels of confidence about the majority of a system's autonomy, it's likely that important test cases are overlooked. One method to fill in potential test coverage gaps is to automatically generate and execute test cases using algorithms that ensure desirable properties about the coverage. For example, generate cases for all possible fault monitors, and across all state change boundaries. Of course, the scope of coverage is determined by the test environment capabilities, where a faster-than-real-time, high-fidelity, software-only simulation would allow the broadest coverage. Even real-time systems that can be replicated and run in parallel, and that have reliable set-up and operations features provide an excellent resource for automated testing. Making detailed predictions for the outcome of such tests can be difficult, and when algorithmic means are employed to produce hundreds or even thousands of cases, generating predicts individually is impractical, and generating predicts with tools requires executable models of the design and environment that themselves require a complete test program. Therefore, evaluating the results of large number of mission scenario tests poses special challenges. A good approach to address this problem is to automatically score the results based on a range of metrics. Although the specific means of scoring depends highly on the application, the use of formal scoring - metrics has high value in identifying and prioritizing anomalies, and in presenting an overall picture of the state of the test program. In this paper we present a case study based on automatic generation and assessment of faulted test runs for the Dawn mission, and discuss its role in optimizing the allocation of resources for completing the test program.

Testing challenges↗
NASA NTRS2012

Rapid Diagnostics of Onboard Sequences

Keeping track of sequences onboard a spacecraft is challenging. When reviewing Event Verification Records (EVRs) of sequence executions on the Mars Exploration Rover (MER), operators often found themselves wondering which version of a named sequence the EVR corresponded to. The lack of this information drastically impacts the operators diagnostic capabilities as well as their situational awareness with respect to the commands the spacecraft has executed, since the EVRs do not provide argument values or explanatory comments. Having this information immediately available can be instrumental in diagnosing critical events and can significantly enhance the overall safety of the spacecraft. This software provides auditing capability that can eliminate that uncertainty while diagnosing critical conditions. Furthermore, the Restful interface provides a simple way for sequencing tools to automatically retrieve binary compiled sequence SCMFs (Space Command Message Files) on demand. It also enables developers to change the underlying database, while maintaining the same interface to the existing applications. The logging capabilities are also beneficial to operators when they are trying to recall how they solved a similar problem many days ago: this software enables automatic recovery of SCMF and RML (Robot Markup Language) sequence files directly from the command EVRs, eliminating the need for people to find and validate the corresponding sequences. To address the lack of auditing capability for sequences onboard a spacecraft during earlier missions, extensive logging support was added on the Mars Science Laboratory (MSL) sequencing server. This server is responsible for generating all MSL binary SCMFs from RML input sequences. The sequencing server logs every SCMF it generates into a MySQL database, as well as the high-level RML file and dictionary name inputs used to create the SCMF. The SCMF is then indexed by a hash value that is automatically included in all command EVRs by the onboard flight software. Second, both the binary SCMF result and the RML input file can be retrieved simply by specifying the hash to a Restful web interface. This interface enables command line tools as well as large sophisticated programs to download the SCMF and RMLs on-demand from the database, enabling a vast array of tools to be built on top of it. One such command line tool can retrieve and display RML files, or annotate a list of EVRs by interleaving them with the original sequence commands. This software has been integrated with the MSL sequencing pipeline where it will serve sequences useful in diagnostics, debugging, and situational awareness throughout the mission.

Starbird, Thomas W.↗
NASA NTRSDate not supplied

Developing Methods for Exercise System Kinematics Tracking

BACKGROUND How to quantify the load and forces produced by exercise equipment and their Vibration Isolation and Stabilization (VIS) platforms in-flight is an active area of investigation. Kinematic tracking paired with system modeling can provide insights as well as verification and validation of simulations used for system design and development. Traditional motion capture methods can require significant cost in equipment procurement and crew-time, but newer lessons learned can be leveraged [1]. The VIS systems of current and future exercise hardware on the International Space Station (ISS) such as the Cycle Ergometer with Vibration Isolation System (CEVIS) and the European Enhanced Exploration Exercise Device (E4D) are not currently outfitted with IMUs or similar measurement devices. Video-based methods would enable use of multi-purpose, crew-familiar flight equipment. An initial exploration of video-based solutions was performed utilizing 2-camera video from crew cycling on Teal-CEVIS on the ISS. METHODS AND RESULTS Our group has scoped a variety of video-based object tracking methods. To date, we have primarily investigated computer vision toolkits such as openCV. Techniques explored include key-point detection, background subtraction, region-of interest tracking, color-based tracking, tag masking and tracking, and corner detection. Although object-tracking and 6D pose estimation is a rich field, space applications are a unique problem that are challenging for existing software and toolkits. The majority of the existing object-tracking applications involve vehicles/pedestrians and household objects with simple backgrounds. We have identified the following features which pose particular challenges for on-station exercise equipment tracking: Busy and visually cluttered background Low-textured tracking object with relatively small motions Occlusions and motion by human subject and loose, floating objects Limited number of video cameras with no fixed global references Limited ability to add tags, markers, or visual references to the tracking object Lack of training data for Machine Learning (ML) algorithms CONCLUSION We will summarize the efficacy of techniques tested for a ground mock-trial and the on-station exercise trial. It is likely that human-in-loop feedback or a conglomerate of methods is required. ML-based methods, like those implemented for human body tracking [2], may still be a viable option, but more training data and validation is needed.

L B Nilsson↗
NASA NTRSDate not supplied

Developing Methods for Exercise System Kinematic Tracking

BACKGROUND How to quantify the load and forces produced by exercise equipment and their Vibration Isolation and Stabilization (VIS) platforms in-flight is an active area of investigation. Kinematic tracking paired with system modeling can provide insights as well as verification and validation of simulations used for system design and development. Traditional motion capture methods can require significant cost in equipment procurement and crew-time, but newer lessons learned can be leveraged [1]. The VIS systems of current and future exercise hardware on the International Space Station (ISS) such as the Cycle Ergometer with Vibration Isolation System (CEVIS) and the European Enhanced Exploration Exercise Device (E4D) are not currently outfitted with IMUs or similar measurement devices. Video-based methods would enable use of multi-purpose, crew-familiar flight equipment. An initial exploration of video-based solutions was performed utilizing 2-camera video from crew cycling on Teal-CEVIS on the ISS. METHODS AND RESULTS Our group has scoped a variety of video-based object tracking methods. To date, we have primarily investigated computer vision toolkits such as open CV. Techniques explored include key-point detection, background subtraction, region-of interest tracking, color-based tracking, tag masking and tracking, and corner detection. Although object-tracking and 6D pose estimation is a rich field, space applications are a unique problem that are challenging for existing software and toolkits. The majority of the existing object-tracking applications involve vehicles/pedestrians and household objects with simple backgrounds. We have identified the following features which pose particular challenges for on-station exercise equipment tracking: 1. Busy and visually cluttered background 2. Low-textured tracking object with relatively small motions 3. Occlusions and motion by human subject and loose, floating objects 4. Limited number of video cameras with no fixed global references 5. Limited ability to add tags, markers, or visual references to the tracking object 6. Lack of training data for Machine Learning (ML) algorithms CONCLUSION We will summarize the efficacy of techniques tested for a ground mock-trial and the on-station exercise trial. It is likely that human-in-loop feedback or a conglomerate of methods is required. ML-based methods, like those implemented for human body tracking [2], may still be a viable option, but more training data and validation is needed.

L Nilsson↗
NASA NTRS2013

SILHIL Replication of Electric Aircraft Powertrain Dynamics and Inner-Loop Control for V&V of System Health Management Routines

Software-in-the-loop and Hardware-in-the-loop testing of failure prognostics and decision making tools for aircraft systems will facilitate more comprehensive and cost-effective testing than what is practical to conduct with flight tests. A framework is described for the offline recreation of dynamic loads on simulated or physical aircraft powertrain components based on a real-time simulation of airframe dynamics running on a flight simulator, an inner-loop flight control policy executed by either an autopilot routine or a human pilot, and a supervisory fault management control policy. The creation of an offline framework for verifying and validating supervisory failure prognostics and decision making routines is described for the example of battery charge depletion failure scenarios onboard a prototype electric unmanned aerial vehicle.

Verification and Validation↗
NASA NTRS1995

An approach to verification and validation of a reliable multicasting protocol: Extended Abstract

This paper describes the process of implementing a complex communications protocol that provides reliable delivery of data in multicast-capable, packet-switching telecommunication networks. The protocol, called the Reliable Multicasting Protocol (RMP), was developed incrementally using a combination of formal and informal techniques in an attempt to ensure the correctness of its implementation. Our development process involved three concurrent activities: (1) the initial construction and incremental enhancement of a formal state model of the protocol machine; (2) the initial coding and incremental enhancement of the implementation; and (3) model-based testing of iterative implementations of the protocol. These activities were carried out by two separate teams: a design team and a V&V team. The design team built the first version of RMP with limited functionality to handle only nominal requirements of data delivery. This initial version did not handle off-nominal cases such as network partitions or site failures. Meanwhile, the V&V team concurrently developed a formal model of the requirements using a variant of SCR-based state tables. Based on these requirements tables, the V&V team developed test cases to exercise the implementation. In a series of iterative steps, the design team added new functionality to the implementation while the V&V team kept the state model in fidelity with the implementation. This was done by generating test cases based on suspected errant or off-nominal behaviors predicted by the current model. If the execution of a test in the model and implementation agreed, then the test either found a potential problem or verified a required behavior. However, if the execution of a test was different in the model and implementation, then the differences helped identify inconsistencies between the model and implementation. In either case, the dialogue between both teams drove the co-evolution of the model and implementation. We have found that this interactive, iterative approach to development allows software designers to focus on delivery of nominal functionality while the V&V team can focus on analysis of off nominal cases. Testing serves as the vehicle for keeping the model and implementation in fidelity with each other. This paper describes (1) our experiences in developing our process model; and (2) three example problems found during the development of RMP. Although RMP has provided our research effort with a rich set of test cases, it also has practical applications within NASA. For example, RMP is being considered for use in the NASA EOSDIS project due to its significant performance benefits in applications that need to replicate large amounts of data to many network sites.

Callahan, John R.↗
NASA NTRS2013

Model Verification and Validation Concepts for a Probabilistic Fracture Assessment Model to Predict Cracking of Knife Edge Seals in the Space Shuttle Main Engine High Pressure Oxidizer

Physics-based models are routinely used to predict the performance of engineered systems to make decisions such as when to retire system components, how to extend the life of an aging system, or if a new design will be safe or available. Model verification and validation (V&V) is a process to establish credibility in model predictions. Ideally, carefully controlled validation experiments will be designed and performed to validate models or submodels. In reality, time and cost constraints limit experiments and even model development. This paper describes elements of model V&V during the development and application of a probabilistic fracture assessment model to predict cracking in space shuttle main engine high-pressure oxidizer turbopump knife-edge seals. The objective of this effort was to assess the probability of initiating and growing a crack to a specified failure length in specific flight units for different usage and inspection scenarios. The probabilistic fracture assessment model developed in this investigation combined a series of submodels describing the usage, temperature history, flutter tendencies, tooth stresses and numbers of cycles, fatigue cracking, nondestructive inspection, and finally the probability of failure. The analysis accounted for unit-to-unit variations in temperature, flutter limit state, flutter stress magnitude, and fatigue life properties. The investigation focused on the calculation of relative risk rather than absolute risk between the usage scenarios. Verification predictions were first performed for three units with known usage and cracking histories to establish credibility in the model predictions. Then, numerous predictions were performed for an assortment of operating units that had flown recently or that were projected for future flights. Calculations were performed using two NASA-developed software tools: NESSUS(Registered Trademark) for the probabilistic analysis, and NASGRO(Registered Trademark) for the fracture mechanics analysis. The goal of these predictions was to provide additional information to guide decisions on the potential of reusing existing and installed units prior to the new design certification.

Pai, Shantaram S.↗