Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “software verification”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,081 records · Page 60
All sourcesNASA NTRSDOE OSTI
NASA NTRS1991

Tethered satellite system dynamics and control review panel and related activities, phase 3

Two major tests of the Tethered Satellite System (TSS) engineering and flight units were conducted to demonstrate the functionality of the hardware and software. Deficiencies in the hardware/software integration tests (HSIT) led to a recommendation for more testing to be performed. Selected problem areas of tether dynamics were analyzed, including verification of the severity of skip rope oscillations, verification or comparison runs to explore dynamic phenomena observed in other simulations, and data generation runs to explore the performance of the time domain and frequency domain skip rope observers.

Source record↗
NASA NTRS2010

Application of Diagnostic Analysis Tools to the Ares I Thrust Vector Control System

The NASA Ares I Crew Launch Vehicle is being designed to support missions to the International Space Station (ISS), to the Moon, and beyond. The Ares I is undergoing design and development utilizing commercial-off-the-shelf tools and hardware when applicable, along with cutting edge launch technologies and state-of-the-art design and development. In support of the vehicle s design and development, the Ares Functional Fault Analysis group was tasked to develop an Ares Vehicle Diagnostic Model (AVDM) and to demonstrate the capability of that model to support failure-related analyses and design integration. One important component of the AVDM is the Upper Stage (US) Thrust Vector Control (TVC) diagnostic model-a representation of the failure space of the US TVC subsystem. This paper first presents an overview of the AVDM, its development approach, and the software used to implement the model and conduct diagnostic analysis. It then uses the US TVC diagnostic model to illustrate details of the development, implementation, analysis, and verification processes. Finally, the paper describes how the AVDM model can impact both design and ground operations, and how some of these impacts are being realized during discussions of US TVC diagnostic analyses with US TVC designers.

Maul, William A.↗
NASA NTRS2012

CoNNeCT Antenna Positioning System Dynamic Simulator Modal Model Correlation

The National Aeronautics and Space Administration (NASA) developed an on-orbit, adaptable, Software Defined Radios (SDR)/Space Telecommunications Radio System (STRS)-based testbed facility to conduct a suite of experiments to advance technologies, reduce risk, and enable future mission capabilities on the International Space Station (ISS). The Communications, Navigation, and Networking reConfigurable Testbed (CoNNeCT) Project will provide NASA, industry, other Government agencies, and academic partners the opportunity to develop and field communications, navigation, and networking technologies in both the laboratory and space environment based on reconfigurable, software-defined radio platforms and the STRS Architecture. The CoNNeCT Payload Operations Nomenclature is SCAN Testbed, and this nomenclature will be used in all ISS integration, safety, verification, and operations documentation. The SCAN Testbed (payload) is a Flight Releasable Attachment Mechanism (FRAM) based payload that will launch aboard the Japanese H-II Transfer Vehicle (HTV) Multipurpose Exposed Pallet (EP-MP) to the International Space Station (ISS), and will be transferred to the Express Logistics Carrier 3 (ELC3) via Extravehicular Robotics (EVR). The SCAN Testbed will operate on-orbit for a minimum of two years.

Jones, Trevor M.↗
NASA NTRS2012

CoNNeCT Antenna Positioning System Dynamic Simulator Modal Model Correlation

The National Aeronautics and Space Administration (NASA) developed an on-orbit, adaptable, Software Defined Radios (SDR)/Space Telecommunications Radio System (STRS)-based testbed facility to conduct a suite of experiments to advance technologies, reduce risk, and enable future mission capabilities on the International Space Station (ISS). The Communications, Navigation, and Networking reConfigurable Testbed (CoNNeCT) Project will provide NASA, industry, other Government agencies, and academic partners the opportunity to develop and field communications, navigation, and networking technologies in both the laboratory and space environment based on reconfigurable, software-defined radio platforms and the STRS Architecture. The CoNNeCT Payload Operations Nomenclature is "SCAN Testbed," and this nomenclature will be used in all ISS integration, safety, verification, and operations documentation. The SCAN Testbed (payload) is a Flight Releasable Attachment Mechanism (FRAM) based payload that will launch aboard the Japanese H-II Transfer Vehicle (HTV) Multipurpose Exposed Pallet (EP-MP) to the International Space Station (ISS), and will be transferred to the Express Logistics Carrier 3 (ELC3) via Extravehicular Robotics (EVR). The SCAN Testbed will operate on-orbit for a minimum of two years.

Jones, Tevor M.↗
NASA NTRS2022

Europa Clipper Payload Verification and Validation: Avionics-Instrument Interface Test Campaign

NASA's Europa Clipper mission will investigate Jupiter's icy moon Europa using a payload suite consisting of nine instruments to address a range of scientific objectives concerning Europa's habitability. As the project proceeds past its Critical Design Review, confidence is being built in the system's ability to achieve mission objectives through the implementation of a rigorous payload verification and validation (V&V) program. As part of this payload V&V program, instrument box-level testing was performed by the payload team to verify select instrument-avionics interface requirements. This testing was performed at JPL using the avionics testbed's Bulk Data Storage Emulator (BDSEM) with visiting instrument Test Models. This paper summarizes the Data Link test campaign involving roughly four days of functional testing per instrument, including planning, testing methods, types of issues found, and the requirement closure process. Detail is also provided on the development, deployment, and validation of a standardized analysis tool used in data reviews. This testing verified requirements related to commanding rates, loss of link, packet format, clock counters, loopback test capability, and SpaceWire jitter and skew margins. Additional risk reduction testing of basic commanding, counter behavior, science data collection and transfer, and interface swapping was also performed. Because the BDSEM venue was not originally designed to be a run for record venue, the process of characterizing venue fidelity and establishing suitability for requirement closure using data collected in this venue will also be addressed.In order to close requirements, an extensible tool was developed to post-process instrument command and telemetry data from their original binary to a human-readable format and give visibility to errors detected within the data, such as packets with Cyclic Redundancy Check errors. This tool, called payload-packet-parser, is a Python 3.9 command line tool built using a variety of open-source Python libraries. Payload-packet-parser was designed to support parsing command and telemetry packets for all Europa Clipper instruments and additional analysis tools were developed for verification of specific information interface requirements. This test campaign, including post-processing using a single parsing and verification toolset, allowed for early interface testing, alleviating testing burdens on instrument teams and buying down risk on the instrument-avionics interface by finding hardware and software issues and idiosyncrasies prior to integration with system test venues. Over twenty issues were discovered across the payload, resulting in software updates and instrument rework well in advance of any system impacts. This paper concludes with an assessment of benefits and costs of this type of testing and lessons learned.

Montanez, Leticia↗
NASA NTRSDate not supplied

An Integrated Software Architecture for Solar Cruiser Mission Design and Navigation

Solar Cruiser is a solar sailing mission, riding as a secondary payload to the Interstellar Mapping and Acceleration Probe (IMAP) mission, slated to launch in February of 2025. The Solar Cruiser vehicle will generate thrust via a complex, low-thrust solar sail. The extreme low-thrust nature of the solar sail will leave Solar Cruiser highly sensitive to external environmental effects (such as solar radiation pressure and high-order gravitational perturbations) throughout the entirety of flight. Because of this, preliminary operational optimization routines must be intricately tied to high-order predictive propagation models to ensure the greatest possible confidence in mission success. The Solar Cruiser Mission Design and Navigation (MDNav) team has designed a software tool suite, employing the latest in software containerization technology, to accomplish this task; allowing for seamless development across several users and operating systems. Combining JPL’s Monte toolkit with high-performance optimizers written by the University of Alabama, the proposed architecture allows for instant verification of optimized trajectories within the same development environment that the optimization takes place, removing the need for mission designers and navigators to switch between tools. The MDNav suite image deployment process is separated from the development and operational scripts to be used in flight, which allows for maintaining a low-footprint version control profile without unnecessary file bloating. This paper discusses the historical differences between previous iterations of the Solar Cruiser MDNav tool suite and the current iteration, planned operational interfaces of the tool with other software and subsystems, and the planned path forward in maintaining containerization services for the software throughout the lifetime of Solar Cruiser.

optimization↗
NASA NTRSDate not supplied

Preliminary Application of Formal Verification to An Autonomy Architecture for Unmanned Aircraft

There is a desire to design autonomous systems in such a way that capabilities can be easily added or re-combined to produce new behaviors while preserving their safety properties. ICAROUS, a prototype software architecture for building safety-centric autonomous unmanned aircraft applications, is designed to support this type of extensibility and re-configurability. In ICAROUS, core capabilities are implemented as individual soft- ware services, so that enabling access to new capabilities simply requires adding new services. To make use of these capabilities, ICAROUS includes a specialized service that provides a general framework for config- uring the relative priorities, conditions, and rules that govern how different modules should be engaged and disengaged during flight. The inherent complexity of coordinating multiple modules under changing conditions makes it difficult to determine whether a particular configuration could have erroneous behaviors in certain circumstances. A robust set of integration tests can help discover errors, but testing can only realistically cover a relatively small proportion of total system behaviors. Developing good tests and interpreting the results to pinpoint the cause of errors when they arise can also be very time-consuming. To supplement testing, formal methods can be used to model and analyze complex systems, achieving better coverage and simplifying the process of finding, understanding, and fixing errors. To demonstrate these benefits, this paper explores the ap- plication of formal methods to ICAROUS. In particular, the Spin model checker is used to specify requirements for and model portions of the system, then verify whether the model satisfies the requirements and find and fix errors when it does not.

Formal Methods↗
NASA NTRS2015

Rapid State Space Modeling Tool for Rectangular Wing Aeroservoelastic Studies

This paper introduces a modeling and simulation tool for aeroservoelastic analysis of rectangular wings with trailing-edge control surfaces. The inputs to the code are planform design parameters such as wing span, aspect ratio, and number of control surfaces. Using this information, the generalized forces are computed using the doublet-lattice method. Using Roger's approximation, a rational function approximation is computed. The output, computed in a few seconds, is a state space aeroservoelastic model which can be used for analysis and control design. The tool is fully parameterized with default information so there is little required interaction with the model developer. All parameters can be easily modified if desired. The focus of this paper is on tool presentation, verification, and validation. These processes are carried out in stages throughout the paper. The rational function approximation is verified against computed generalized forces for a plate model. A model composed of finite element plates is compared to a modal analysis from commercial software and an independently conducted experimental ground vibration test analysis. Aeroservoelastic analysis is the ultimate goal of this tool, therefore, the flutter speed and frequency for a clamped plate are computed using damping-versus-velocity and frequency-versus-velocity analysis. The computational results are compared to a previously published computational analysis and wind-tunnel results for the same structure. A case study of a generic wing model with a single control surface is presented. Verification of the state space model is presented in comparison to damping-versus-velocity and frequency-versus-velocity analysis, including the analysis of the model in response to a 1-cos gust.

Modeling and Simulation↗
NASA NTRS2015

Rapid State Space Modeling Tool for Rectangular Wing Aeroservoelastic Studies

This report introduces a modeling and simulation tool for aeroservoelastic analysis of rectangular wings with trailing-edge control surfaces. The inputs to the code are planform design parameters such as wing span, aspect ratio, and number of control surfaces. Using this information, the generalized forces are computed using the doublet-lattice method. Using Roger's approximation, a rational function approximation is computed. The output, computed in a few seconds, is a state space aeroservoelastic model which can be used for analysis and control design. The tool is fully parameterized with default information so there is little required interaction with the model developer. All parameters can be easily modified if desired. The focus of this report is on tool presentation, verification, and validation. These processes are carried out in stages throughout the report. The rational function approximation is verified against computed generalized forces for a plate model. A model composed of finite element plates is compared to a modal analysis from commercial software and an independently conducted experimental ground vibration test analysis. Aeroservoelastic analysis is the ultimate goal of this tool, therefore, the flutter speed and frequency for a clamped plate are computed using damping-versus-velocity and frequency-versus-velocity analysis. The computational results are compared to a previously published computational analysis and wind-tunnel results for the same structure. A case study of a generic wing model with a single control surface is presented. Verification of the state space model is presented in comparison to damping-versus-velocity and frequency-versus-velocity analysis, including the analysis of the model in response to a 1-cos gust.

Verification and Validation↗
DOE OSTI2018

Connectivity and functional verification for the LSST science raft towers

The LSST Camera focal plane will be constructed with 21 144-Mpixel modules (\Raft Tower Module", RTM). An extensive operational test is performed to confirm the integrity of all connections and to verify the basic functionality. Each RTM undergoes at least four connectivity tests. A python script communicates with Java- based control software and performs the test. A python script parses the test data and generates a PDF report. The report includes a summary PASS/FAIL table, several hundred current, voltage and temperature parameters, and images taken with the CCD array at room temperature.

79 ASTRONOMY AND ASTROPHYSICS↗
NASA NTRS1991

Onboard navigation rendezvous expert system

The Onboard Navigation rendezvous expert system is designed to aid the ground flight controller in monitoring the shuttle onboard navigation system. The system is designed to keep track of the navigation sensors and relative state vectors. In addition, the system also keeps an event log and fills out forms usually handled by the flight controller. This expert system is one of the few rendezvous specific systems being developed for the Mission Control Center. The expert system has been in development for six years. Through these years the system has seen hardware, software, and personnel changes. Initial development was done by the Information Systems Directorate (ISD) and Mission Operations Directorate (MOD) at Johnson Space Center. As of October 1, 1991 the system has been turned over to MOD. The system is completely developed except for some minor adjustments to the user interface. The rule base is in the verification stage with total certification of the system due to be completed by May 1992. Test cases for verification are obtained by saving data used for flight controller integrated simulations. The actual data comes from both the shuttle mission simulator and the Mission Control Center Computer. So far no actual flight data has been available. This paper covers all aspects of the system from the development history to the current hardware, software, and use of the system.

Kocen, Michelle↗
NASA NTRS2020

Integrating the Science Opportunity Analyzer with a Reusable Opportunity Search Framework

In our interactions with the Science Opportunity Analyzer (SOA) software, we recognized how its ability to search for geometric events in space is a need for robotic space missions in general. To satisfy this need, we propose the Tychonis framework, which is built upon the principles of: (1) separations of concerns, (2) extensibility, (3) reusability, and (4) independent verification and validation. Tychonis’ separation of concerns results in the availability of different constructs to model geometric events and search for them. These constructs can be extended by users as needed and reused across missions without changes. Given the low coupling between concerns, and the fact Tychonis can be augmented in isolation, its constructs can be validated independently from other pieces of software. This paper elaborates on these topics and presents an integration case study between Tychonis and SOA that relies on the concept of dynamic integration. Dynamic integration entails that augmentations of the framework are reflected automatically in the host application without any changes to the host application’s code. The SOA-Tychonis integration case study can be extrapolated to other tools that need to search for geometric events as the pattern repeats across implementations: (i) the manipulation of a user interface to model opportunities, (ii) the execution of algorithms to search for opportunities, and (iii) the presentation of search results to users. Overall, Tychonis’ is a story about how the application of proven software principles and good design choices can reduce risk and cost to space missions

Soria, Manel↗
DOE OSTI2021

Simple, Secure, Internet Delivery of MOOSE-based Applications

Application packaging and distribution are the final steps for delivering software to end-users; both are frequently neglected when creating scientific software. Commercial businesses rely on electronic distribution systems that have rendered disk drives obsolete. Still, national laboratories continue to rely heavily on removable media to distribute and limit access to controlled applications. With increasing concerns of unauthorized copying of sensitive applications, a modern distribution system that utilizes cryptographically secure communication and authentication protocols has been developed. This new distribution system will secure the chain of custody for nuclear software while simultaneously simplifying access to these tools. This report summarizes four primary advancements made toward the secure distribution of Nuclear Energy Advanced Modeling and Simulation (NEAMS)-developed, Multiphysics Object Oriented Simulation Environment (MOOSE)-based applications: application installation, package distribution, automated package building, and distribution of documentation. NEAMS is currently developing more than ten separate applications based on the open-source MOOSE Framework. Distribution of these applications has primarily been accomplished by distributing source code, with end-users compiling the applications themselves. This work created a mechanism where MOOSE applications can be installed in a similar way to any other software. This allows both administrators and end-users simplified access to runnable executables. With this new installation capability, it was then possible to rethink distribution. A new, secure capability for delivering MOOSE-based applications over the internet has been created. This system requires unique cryptographic tokens for authentication, greatly securing the custody chain for software. Once granted access, installation of any NEAMS code can be accomplished with these terminal commands: "conda install ncrc" "ncrc install ncrc-bison." After these two commands (and authenticating) the BISON application will be securely down- loaded from Idaho National Laboratory (INL)’s servers, installed, and ready to use. To enable this new distribution capability to be successful, the open-source Continuous Integration, Verification, Enhancement, and Testing (CIVET) Continuous Integration (CI) capability was augmented to add Continuous Delivery (CD). CD enables the automated building and packaging of MOOSE-based applications as they are modified by development teams, ensuring that our customers can obtain up-to-date versions of the software at any time. The need for instruction on how to use these applications was addressed through modifications to the MOOSE documentation system. The MooseDocs capability, which enables robust documentation of MOOSE-based applications, has been extended to allow both for the installation of documentation and the packaging of documentation with installed applications. Together, these enhancements form the core of a new, secure distribution mechanism for nuclear simulation tools. In concert with the Nuclear Computational Resource Center (NCRC), NEAMS- developed applications will now be straightforward to obtain securely.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2015

WE-AB-201-00: Treatment Planning System Commissioning and QA

Treatment planning systems (TPS) are a cornerstone of modern radiation therapy. Errors in their commissioning or use can have a devastating impact on many patients. To support safe and high quality care, medical physicists must conduct efficient and proper commissioning, good clinical integration, and ongoing quality assurance (QA) of the TPS. AAPM Task Group 53 and related publications have served as seminal benchmarks for TPS commissioning and QA over the past two decades. Over the same time, continuing innovations have made the TPS even more complex and more central to the clinical process. Medical goals are now expressed in terms of the dose and margins around organs and tissues that are delineated from multiple imaging modalities (CT, MR and PET); and even temporally resolved (i.e., 4D) imaging. This information is passed on to optimization algorithms to establish accelerator movements that are programmed directly for IMRT, VMAT and stereotactic treatments. These advances have made commissioning and QA of the TPS much more challenging. This education session reviews up-to-date experience and guidance on this subject; including the recently published AAPM Medical Physics Practice Guideline (MPPG) #5 “Commissioning and QA of Treatment Planning Dose Calculations: Megavoltage Photon and Electron Beams”. Treatment Planning System Commissioning and QA: Challenges and Opportunities (Greg Salomons) This session will provide some key background and review publications describing prominent incidents relating to TPS commissioning and QA. Traditional approaches have been hardware and feature oriented. They aim to establish a functional configuration and establish specifications for regular testing of features (like dose calculation) to assure stable operation and detect failures. With the advent of more complex systems, more patient-specific testing has also been adopted. A number of actual TPS defects will be presented along with heuristics for identifying similar defects in the future. Finally, the Gamma test has become a popular metric for reporting TPS Commissioning and QA results. It simplifies complex testing into a numerical index, but noisy data and casual application can make it misleading. A brief review of the issues around the use of the Gamma test will be presented. TPS commissioning and QA: A process orientation and application of control charts (Michael Sharpe) A framework for commissioning a treatment planning system will be presented, focusing on preparations, practical aspects of configuration, priorities, specifications, and establishing performance. The complexity of the modern TPS make modular testing of features inadequate, and modern QA tools can provide “too much information” about the performance of techniques like IMRT and VMAT. We have adopted a process orientation and quality tools, like control charts, for ongoing TPS QA and assessment of patient-specific tests. The trending nature of these tools reveals the overall performance of the TPS system, and quantifies the variations that arise from individual plans, discrete calculations, and experimentation based on discrete measurements. Examples demonstrating application of these tools to TPS QA will be presented. TPS commissioning and QA: Incorporating the entire planning process (Sasa Mutic) The TPS and its features do not perform in isolation. Instead, the features and modules are key components in a complex process that begins with CT Simulation and extends to treatment delivery, along with image guidance and verification. Most importantly, the TPS is used by people working in a multi-disciplinary environment. It is very difficult to predict the outcomes of human interactions with software. Therefore, an interdisciplinary approach to training, commissioning and QA will be presented, along with an approach to the physics chart check and end-to-end testing as a tool for TPS QA. The role of standardization and automation in QA will also be discussed. The recommendations of MPPG #5 and practical implementation strategies (Jennifer Smilowitz) The recently published recommendations from Task Group No. 244, Medical Physics Practice Guideline on Commissioning and QA of Treatment Planning Dose Calculations: Megavoltage Photon and Electron Beams will be presented. The recommendations focus on the validation of commissioning data and dose calculations. Tolerance values for non-IMRT beam configurations are summarized based on established criteria and data collected by the IROC. More stringent evaluation criteria for IMRT dose calculations are suggested to test the limitations of the TPS dose algorithms for advanced delivery conditions. The MPPG encourages users to create a suite of validation tests for dose calculation for various conditions for static photon beams, heterogeneities, IMRT/VMAT and electron beams. This test suite is intended to be used for subsequent testing, including TPS software upgrades. In the past, the recommendations of some reports have not been widely implemented due to practical limitations. Implementation strategies, tools and processes developed by multiple centers for efficient and “do-able” MPPG # 5 testing will be presented, as well as a discussion on the overall validation experience. Learning Objectives: Identify some of the key documents relevant for TPS commissioning and QA Understand strategies for testing TPS software Gain a practical knowledge of the Gamma test criteria Increase familiarity with the process of commissioning a TPS Learn about the use of Control Charts for TPS QA Review the role of the TPS in the overall planning process Increase awareness of the link between TPS QA and chart checking Gain an increased appreciation for the importance of interdisciplinary communication Understand the new recommendations from MPPG #5 on TPS Dose Algorithm Commissioning and QC/QA Learn practical implementation processes and tools for MPPG #5 validation recommendations.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2015

WE-AB-201-01: Treatment Planning System Commissioning and QA: Challenges and Opportunities

Treatment planning systems (TPS) are a cornerstone of modern radiation therapy. Errors in their commissioning or use can have a devastating impact on many patients. To support safe and high quality care, medical physicists must conduct efficient and proper commissioning, good clinical integration, and ongoing quality assurance (QA) of the TPS. AAPM Task Group 53 and related publications have served as seminal benchmarks for TPS commissioning and QA over the past two decades. Over the same time, continuing innovations have made the TPS even more complex and more central to the clinical process. Medical goals are now expressed in terms of the dose and margins around organs and tissues that are delineated from multiple imaging modalities (CT, MR and PET); and even temporally resolved (i.e., 4D) imaging. This information is passed on to optimization algorithms to establish accelerator movements that are programmed directly for IMRT, VMAT and stereotactic treatments. These advances have made commissioning and QA of the TPS much more challenging. This education session reviews up-to-date experience and guidance on this subject; including the recently published AAPM Medical Physics Practice Guideline (MPPG) #5 “Commissioning and QA of Treatment Planning Dose Calculations: Megavoltage Photon and Electron Beams”. Treatment Planning System Commissioning and QA: Challenges and Opportunities (Greg Salomons) This session will provide some key background and review publications describing prominent incidents relating to TPS commissioning and QA. Traditional approaches have been hardware and feature oriented. They aim to establish a functional configuration and establish specifications for regular testing of features (like dose calculation) to assure stable operation and detect failures. With the advent of more complex systems, more patient-specific testing has also been adopted. A number of actual TPS defects will be presented along with heuristics for identifying similar defects in the future. Finally, the Gamma test has become a popular metric for reporting TPS Commissioning and QA results. It simplifies complex testing into a numerical index, but noisy data and casual application can make it misleading. A brief review of the issues around the use of the Gamma test will be presented. TPS commissioning and QA: A process orientation and application of control charts (Michael Sharpe) A framework for commissioning a treatment planning system will be presented, focusing on preparations, practical aspects of configuration, priorities, specifications, and establishing performance. The complexity of the modern TPS make modular testing of features inadequate, and modern QA tools can provide “too much information” about the performance of techniques like IMRT and VMAT. We have adopted a process orientation and quality tools, like control charts, for ongoing TPS QA and assessment of patient-specific tests. The trending nature of these tools reveals the overall performance of the TPS system, and quantifies the variations that arise from individual plans, discrete calculations, and experimentation based on discrete measurements. Examples demonstrating application of these tools to TPS QA will be presented. TPS commissioning and QA: Incorporating the entire planning process (Sasa Mutic) The TPS and its features do not perform in isolation. Instead, the features and modules are key components in a complex process that begins with CT Simulation and extends to treatment delivery, along with image guidance and verification. Most importantly, the TPS is used by people working in a multi-disciplinary environment. It is very difficult to predict the outcomes of human interactions with software. Therefore, an interdisciplinary approach to training, commissioning and QA will be presented, along with an approach to the physics chart check and end-to-end testing as a tool for TPS QA. The role of standardization and automation in QA will also be discussed. The recommendations of MPPG #5 and practical implementation strategies (Jennifer Smilowitz) The recently published recommendations from Task Group No. 244, Medical Physics Practice Guideline on Commissioning and QA of Treatment Planning Dose Calculations: Megavoltage Photon and Electron Beams will be presented. The recommendations focus on the validation of commissioning data and dose calculations. Tolerance values for non-IMRT beam configurations are summarized based on established criteria and data collected by the IROC. More stringent evaluation criteria for IMRT dose calculations are suggested to test the limitations of the TPS dose algorithms for advanced delivery conditions. The MPPG encourages users to create a suite of validation tests for dose calculation for various conditions for static photon beams, heterogeneities, IMRT/VMAT and electron beams. This test suite is intended to be used for subsequent testing, including TPS software upgrades. In the past, the recommendations of some reports have not been widely implemented due to practical limitations. Implementation strategies, tools and processes developed by multiple centers for efficient and “do-able” MPPG #5 testing will be presented, as well as a discussion on the overall validation experience. Learning Objectives: Identify some of the key documents relevant for TPS commissioning and QA Understand strategies for testing TPS software Gain a practical knowledge of the Gamma test criteria Increase familiarity with the process of commissioning a TPS Learn about the use of Control Charts for TPS QA Review the role of the TPS in the overall planning process Increase awareness of the link between TPS QA and chart checking Gain an increased appreciation for the importance of interdisciplinary communication Understand the new recommendations from MPPG #5 on TPS Dose Algorithm Commissioning and QC/QA Learn practical implementation processes and tools for MPPG #5 validation recommendations.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2015

WE-AB-201-02: TPS Commissioning and QA: A Process Orientation and Application of Control Charts

Treatment planning systems (TPS) are a cornerstone of modern radiation therapy. Errors in their commissioning or use can have a devastating impact on many patients. To support safe and high quality care, medical physicists must conduct efficient and proper commissioning, good clinical integration, and ongoing quality assurance (QA) of the TPS. AAPM Task Group 53 and related publications have served as seminal benchmarks for TPS commissioning and QA over the past two decades. Over the same time, continuing innovations have made the TPS even more complex and more central to the clinical process. Medical goals are now expressed in terms of the dose and margins around organs and tissues that are delineated from multiple imaging modalities (CT, MR and PET); and even temporally resolved (i.e., 4D) imaging. This information is passed on to optimization algorithms to establish accelerator movements that are programmed directly for IMRT, VMAT and stereotactic treatments. These advances have made commissioning and QA of the TPS much more challenging. This education session reviews up-to-date experience and guidance on this subject; including the recently published AAPM Medical Physics Practice Guideline (MPPG) #5 “Commissioning and QA of Treatment Planning Dose Calculations: Megavoltage Photon and Electron Beams”. Treatment Planning System Commissioning and QA: Challenges and Opportunities (Greg Salomons) This session will provide some key background and review publications describing prominent incidents relating to TPS commissioning and QA. Traditional approaches have been hardware and feature oriented. They aim to establish a functional configuration and establish specifications for regular testing of features (like dose calculation) to assure stable operation and detect failures. With the advent of more complex systems, more patient-specific testing has also been adopted. A number of actual TPS defects will be presented along with heuristics for identifying similar defects in the future. Finally, the Gamma test has become a popular metric for reporting TPS Commissioning and QA results. It simplifies complex testing into a numerical index, but noisy data and casual application can make it misleading. A brief review of the issues around the use of the Gamma test will be presented. TPS commissioning and QA: A process orientation and application of control charts (Michael Sharpe) A framework for commissioning a treatment planning system will be presented, focusing on preparations, practical aspects of configuration, priorities, specifications, and establishing performance. The complexity of the modern TPS make modular testing of features inadequate, and modern QA tools can provide “too much information” about the performance of techniques like IMRT and VMAT. We have adopted a process orientation and quality tools, like control charts, for ongoing TPS QA and assessment of patient-specific tests. The trending nature of these tools reveals the overall performance of the TPS system, and quantifies the variations that arise from individual plans, discrete calculations, and experimentation based on discrete measurements. Examples demonstrating application of these tools to TPS QA will be presented. TPS commissioning and QA: Incorporating the entire planning process (Sasa Mutic) The TPS and its features do not perform in isolation. Instead, the features and modules are key components in a complex process that begins with CT Simulation and extends to treatment delivery, along with image guidance and verification. Most importantly, the TPS is used by people working in a multi-disciplinary environment. It is very difficult to predict the outcomes of human interactions with software. Therefore, an interdisciplinary approach to training, commissioning and QA will be presented, along with an approach to the physics chart check and end-to-end testing as a tool for TPS QA. The role of standardization and automation in QA will also be discussed. The recommendations of MPPG #5 and practical implementation strategies (Jennifer Smilowitz) The recently published recommendations from Task Group No. 244, Medical Physics Practice Guideline on Commissioning and QA of Treatment Planning Dose Calculations: Megavoltage Photon and Electron Beams will be presented. The recommendations focus on the validation of commissioning data and dose calculations. Tolerance values for non-IMRT beam configurations are summarized based on established criteria and data collected by the IROC. More stringent evaluation criteria for IMRT dose calculations are suggested to test the limitations of the TPS dose algorithms for advanced delivery conditions. The MPPG encourages users to create a suite of validation tests for dose calculation for various conditions for static photon beams, heterogeneities, IMRT/VMAT and electron beams. This test suite is intended to be used for subsequent testing, including TPS software upgrades. In the past, the recommendations of some reports have not been widely implemented due to practical limitations. Implementation strategies, tools and processes developed by multiple centers for efficient and “do-able” MPPG #5 testing will be presented, as well as a discussion on the overall validation experience. Learning Objectives: Identify some of the key documents relevant for TPS commissioning and QA Understand strategies for testing TPS software Gain a practical knowledge of the Gamma test criteria Increase familiarity with the process of commissioning a TPS Learn about the use of Control Charts for TPS QA Review the role of the TPS in the overall planning process Increase awareness of the link between TPS QA and chart checking Gain an increased appreciation for the importance of interdisciplinary communication Understand the new recommendations from MPPG #5 on TPS Dose Algorithm Commissioning and QC/QA Learn practical implementation processes and tools for MPPG #5 validation recommendations.

60 APPLIED LIFE SCIENCES↗