SEARCH · Engineering Papers

Results for “Automated Testing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,261 records · Page 70

All sources NASA NTRS DOE OSTI

NASA NTRSDate not supplied

Automated Software for Crewed Spacecraft - Bridging the Gap from Sci Fi to Reality

With a voice command or a few taps on the console, the spacecraft pivots on a dime at high velocity and gently docks to an orbiting space platform. This is the image most people have of the complex software computations and integrated hardware performance necessary for a spacecraft to successfully perform an automated launch, rendezvous, and docking. Today’s reality is that while computer operations are advancing rapidly, science fiction over-simplifies and over-sells current capabilities. This paper discusses the integration of spacecraft computer automation into the operation of one of the United States’ new Commercial Crew vehicles - the Boeing CST-100 Starliner. Lessons learned by the Boeing Mission Operations team, a unique private-public partnership with NASA, from conceptual design through real-time operation of the first test flight will be discussed along with evolution of the system in preparation for the second uncrewed test flight. Focus will center on how operations has learned to use the automated software to their advantage while also knowing how to adjust the automation in response to spacecraft or mission anomalies.

Robert C. Dempsey↗

DOE OSTI2022

FLISR in the Presence of DERs ADMS Test Bed Use Case

Fault location, isolation, and restoration (FLISR) is one of the distribution automation applications that utilities are most interested in. It operates groups of switches on distribution feeders to improve the reliability of power delivery after localizing outages. FLISR is an essential function for enabling a self-healing grid, and it directly affects grid reliability and resilience. The presence of DERs brings both new opportunities and challenges for FLISR applications. For instance, some DERs can work as backup generation sources and help energize local networks so more switching options can be achieved with the coordination of FLISR operations; however, the existence of DERs also brings new challenges. The bidirectional power flow resulting from DERs might require dynamic changes to protective devices' fault current settings, and thus the FLISR application will need to dynamically adjust its operations in coordination with the dynamic protection settings. Also, some FLISR applications make their service restoration decisions and close tie-switches based on predefined rules; however, the presence of DERs and their impact on altering net loads on the local feeder network are not considered. For this use case, the project team is using the FLISR application of the Survalent ADMS operating on a real-time simulation of a feeder from Central Georgia EMC, a cooperative electric utility. The performance of the FLISR application is being evaluated for different locations and combinations of DERs, including grid-forming battery energy storage systems.

ADMS↗

DOE OSTI2023

A Workflow for Accelerating Multimodal Data Collection for Electrodeposited Films

Abstract Future machine learning strategies for materials process optimization will likely replace human capital-intensive artisan research with autonomous and/or accelerated approaches. Such automation enables accelerated multimodal characterization that simultaneously minimizes human errors, lowers costs, enhances statistical sampling, and allows scientists to allocate their time to critical thinking instead of repetitive manual tasks. Previous acceleration efforts to synthesize and evaluate materials have often employed elaborate robotic self-driving laboratories or used specialized strategies that are difficult to generalize. Herein we describe an implemented workflow for accelerating the multimodal characterization of a combinatorial set of 915 electroplated Ni and Ni–Fe thin films resulting in a data cube with over 160,000 individual data files. Our acceleration strategies do not require manufacturing-scale resources and are thus amenable to typical materials research facilities in academic, government, or commercial laboratories. The workflow demonstrated the acceleration of six characterization modalities: optical microscopy, laser profilometry, X-ray diffraction, X-ray fluorescence, nanoindentation, and tribological (friction and wear) testing, each with speedup factors ranging from 13–46x. In addition, automated data upload to a repository using FAIR data principles was accelerated by 64x.

36 MATERIALS SCIENCE↗

DOE OSTI2014

Demonstration of automated price response in large customers in New York City using Auto-DR and OpenADR

Demand response (DR) – allowing customers to respond to reliability requests and market prices by changing electricity use from their normal consumption pattern – continues to be seen as an attractive means of demand-side management and a fundamental smart-grid improvement that links supply and demand. From October 2011 to December 2013, the Demand Response Research Center at Lawrence Berkeley National Laboratory, the New York State Energy Research and Development Authority, and partners Honeywell and Akuacom, have conducted a demonstration project enabling Automated Demand Response (Auto-DR) in large commercial buildings located in New York City using Open Automated Demand Response (OpenADR) communication protocols. In particular, this project focuses on demonstrating how the OpenADR platform, enabled by Akuacom, can automate and simplify interactions between buildings and various stakeholders in New York State and enable the automation of customers’ price response to yield bill savings under dynamic pricing. In this paper, the cost control opportunities under day-ahead hourly pricing and Auto-DR control strategies are presented for four demonstration buildings; present the breakdown of Auto-DR enablement costs; summarize the field test results and their load impact; and show potential bill savings by enabling automated price response under Consolidated Edison’s Mandatory Hourly Pricing (MHP) tariff. For one of the sites, the potential bill savings at the site’s current retail rate are shown. Facility managers were given granular equipment-level opt-out capability to ensure full control of the sites during the Auto-DR implementation. The expected bill savings ranged from 1.1% to 8.0% of the total MHP bill. The automation and enablement costs ranged from $70 to $725 per kW shed. The results show that OpenADR can facilitate the automation of price response, deliver savings to the customers and opt-out capability of the implementation retains control of the sites by facility managers.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

NASA NTRS2003

Learning About Cockpit Automation: From Piston Trainer to Jet Transport

Two experiments explored the idea of providing cockpit automation training to airline-bound student pilots using cockpit automation equipment commonly found in small training airplanes. In a first experiment, pilots mastered a set of tasks and maneuvers using a GPS navigation computer, autopilot, and flight director system installed in a small training airplane Students were then tested on their ability to complete a similar set of tasks using the cockpit automation system found in a popular jet transport aircraft. Pilot were able to successfully complete 77% of all tasks in the jet transport on their first attempt. An analysis of a control group suggests that the pilot's success was attributable to the application of automation principles they had learned in the small airplane. A second experiment looked at two different ways of delivering small-aeroplane cockpit automation training: a self-study method, and a dual instruction method. The results showed a slight advantage for the self-study method. Overall, the results of the two studies cast a strong vote for the incorporation of cockpit automation training in curricula designed for pilot who will later transition to the jet fleet.

Casner, Stephen M.↗

DOE OSTI2012

Measurement of joint kinematics using a conventional clinical single-perspective flat-panel radiography system

Purpose: The ability to accurately measure joint kinematics is an important tool in studying both normal joint function and pathologies associated with injury and disease. The purpose of this study is to evaluate the efficacy, accuracy, precision, and clinical safety of measuring 3D joint motion using a conventional flat-panel radiography system prior to its application in an in vivo study. Methods: An automated, image-based tracking algorithm was implemented to measure the three-dimensional pose of a sparse object from a two-dimensional radiographic projection. The algorithm was tested to determine its efficiency and failure rate, defined as the number of image frames where automated tracking failed, or required user intervention. The accuracy and precision of measuring three-dimensional motion were assessed using a robotic controlled, tibiofemoral knee phantom programmed to mimic a subject with a total knee replacement performing a stair ascent activity. Accuracy was assessed by comparing the measurements of the single-plane radiographic tracking technique to those of an optical tracking system, and quantified by the measurement discrepancy between the two systems using the Bland-Altman technique. Precision was assessed through a series of repeated measurements of the tibiofemoral kinematics, and was quantified using the across-trial deviations of the repeated kinematic measurements. The safety of the imaging procedure was assessed by measuring the effective dose of ionizing radiation associated with the x-ray exposures, and analyzing its relative risk to a human subject. Results: The automated tracking algorithm displayed a failure rate of 2% and achieved an average computational throughput of 8 image frames/s. Mean differences between the radiographic and optical measurements for translations and rotations were less than 0.08 mm and 0.07 Degree-Sign in-plane, and 0.24 mm and 0.6 Degree-Sign out-of-plane. The repeatability of kinematics measurements performed using the radiographic tracking technique was better than {+-}0.09 mm and 0.12 Degree-Sign in-plane, and {+-}0.70 mm and {+-}0.07 Degree-Sign out-of-plane. The effective dose associated with the imaging protocol used was 15 {mu}Sv for 10 s of radiographic cine acquisition. Conclusions: This study demonstrates the ability to accurately measure knee-joint kinematics using a single-plane radiographic measurement technique. The measurement technique can be easily implemented at most clinical centers equipped with a modern-day radiographic x-ray system. The dose of ionizing radiation associated with the image acquisition represents a minimal risk to any subjects undergoing the examination.

60 APPLIED LIFE SCIENCES↗

DOE OSTI2015

The Electrolyte Genome project: A big data approach in battery materials discovery

We present a high-throughput infrastructure for the automated calculation of molecular properties with a focus on battery electrolytes. The infrastructure is largely open-source and handles both practical aspects (input file generation, output file parsing, and information management) as well as more complex problems (structure matching, salt complex generation, and failure recovery). Using this infrastructure, we have computed the ionization potential (IP) and electron affinities (EA) of 4830 molecules relevant to battery electrolytes (encompassing almost 55,000 quantum mechanics calculations) at the B3LYP/6-31+G(*) level. We describe automated workflows for computing redox potential, dissociation constant, and salt-molecule binding complex structure generation. We present routines for automatic recovery from calculation errors, which brings the failure rate from 9.2% to 0.8% for the QChem DFT code. Automated algorithms to check duplication between two arbitrary molecules and structures are described. We present benchmark data on basis sets and functionals on the G2-97 test set; one finding is that a IP/EA calculation method that combines PBE geometry optimization and B3LYP energy evaluation requires less computational cost and yields nearly identical results as compared to a full B3LYP calculation, and could be suitable for the calculation of large molecules. Our data indicates that among the 8 functionals tested, XYGJ-OS and B3LYP are the two best functionals to predict IP/EA with an RMSE of 0.12 and 0.27 eV, respectively. Application of our automated workflow to a large set of quinoxaline derivative molecules shows that functional group effect and substitution position effect can be separated for IP/EA of quinoxaline derivatives, and the most sensitive position is different for IP and EA. Published by Elsevier B.V

Battery↗

NASA NTRS1980

Development of Automated Power Systems Management for planetary spacecraft

Automated Power Systems Management is a technology being developed at JPL to provide the capability for onboard monitoring, computation, and control of planetary-spacecraft electrical power systems without the need for ground intervention. The technology uses microcomputers to provide the control and flexibility necessary to achieve rapid and decisive control of power system functions in the event of electrical power failures or unplanned events requiring power system responses. The current program at JPL includes the design, fabrication, test, and evaluation of a breadboard spacecraft power system, modified to incorporate automated power system functions which are implemented by means of a distributed microcomputer system.

Bridgeforth, A. O.↗

DOE OSTI2023

BigNeuron: a resource to benchmark and predict performance of algorithms for automated tracing of neurons in light microscopy datasets

BigNeuron is an open community bench-testing platform with the goal of setting open standards for accurate and fast automatic neuron tracing. We gathered a diverse set of image volumes across several species that is representative of the data obtained in many neuroscience laboratories interested in neuron tracing. Here, we report generated gold standard manual annotations for a subset of the available imaging datasets and quantified tracing quality for 35 automatic tracing algorithms. The goal of generating such a hand-curated diverse dataset is to advance the development of tracing algorithms and enable generalizable benchmarking. Together with image quality features, we pooled the data in an interactive web application that enables users and developers to perform principal component analysis, t-distributed stochastic neighbor embedding, correlation and clustering, visualization of imaging and tracing data, and benchmarking of automatic tracing algorithms in user-defined data subsets. The image quality metrics explain most of the variance in the data, followed by neuromorphological features related to neuron size. Furthermore, we observed that diverse algorithms can provide complementary information to obtain accurate results and developed a method to iteratively combine methods and generate consensus reconstructions. The consensus trees obtained provide estimates of the neuron structure ground truth that typically outperform single algorithms in noisy datasets. However, specific algorithms may outperform the consensus tree strategy in specific imaging conditions. Finally, to aid users in predicting the most accurate automatic tracing results without manual annotations for comparison, we used support vector machine regression to predict reconstruction quality given an image volume and a set of automatic tracings.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2024

Path to Automated Validation of ENDF/B-VIII.1 [Slides]

We need to expand the variety of applications to rigorously test libraries. Advanced reactors: – Decreasing reactivity for 8.1b2 compared to 8.0, some unexpected nuclides causing major differences (F-19, Cr), no clear performance difference when compared to experiment. Depletion RCA: – High impact isotopes closer to 7.1 – Small improvement on average (U-5, Pu-9, BC FPs), worse for Am and Cm. Fuel reactivity: – 8.1b2 is higher reactivity at high burnups than 8.0, but likely under predicting k eff for PWRs at high burnup.

97 MATHEMATICS AND COMPUTING↗

NASA NTRS1998

Hybrid IR-Gas Heater for Automated Tow Placement

A hybrid infrared-hot gas heat source has been developed and tested for the NASA Langley Tow Placement Facility. The IR heat source provides supplemental heat to the nip region. The additional heat is intended to reduce the need for conduction heating by the compaction roller, which causes (he roller to stick to the panel surface. Initial bench scale testing was performed to identify the most effective means of focusing IR energy into the nip region. A compact lamp and reflector that placed the lamp as close to the nip point as possible was found to deliver the highest heat flux in the nip region. A prototype heater was installed on the NASA Langley tow placement robot. Panels placed with a 400 C (sticking) compaction roller gave DCB initiation toughness numbers comparable to those reported for autoclave processed panels but were found to have unexpectedly high void contents. Placement with compaction roller temperatures that prevented roller sticking resulted in mode I fracture toughness approximately 70% that reported for autoclave processed panels. The variability in strength among specimens placed with reduced roller temperature was found to be greatly reduced, which implies that use of supplemental nip point heat may improve the robustness of the tow placement process. Use of the IR heat source permitted placement with a compaction roller temperature that would have resulted in negligible interfacial strength with the hot gas torch alone. The roller temperature reductions eliminated the need for the robot operator to attend placement operations.

Grenoble, Ray W.↗

NASA NTRS2001

High-Speed Observer: Automated Streak Detection in SSME Plumes

A high frame rate digital video camera installed on test stands at Stennis Space Center has been used to capture images of Space Shuttle main engine plumes during test. These plume images are processed in real time to detect and differentiate anomalous plume events occurring during a time interval on the order of 5 msec. Such speed yields near instantaneous availability of information concerning the state of the hardware. This information can be monitored by the test conductor or by other computer systems, such as the integrated health monitoring system processors, for possible test shutdown before occurrence of a catastrophic engine failure.

Rieckoff, T. J.↗

DOE OSTI2016

SU-F-T-423: Automating Treatment Planning for Cervical Cancer in Low- and Middle- Income Countries

Purpose: To develop and test two independent algorithms that automatically create the photon treatment fields for a four-field box beam arrangement, a common treatment technique for cervical cancer in low- and middle-income countries. Methods: Two algorithms were developed and integrated into Eclipse using its Advanced Programming Interface:3D Method: We automatically segment bony anatomy on CT using an in-house multi-atlas contouring tool and project the structures into the beam’s-eye-view. We identify anatomical landmarks on the projections to define the field apertures. 2D Method: We generate DRRs for all four beams. An atlas of DRRs for six standard patients with corresponding field apertures are deformably registered to the test patient DRRs. The set of deformed atlas apertures are fitted to an expected shape to define the final apertures. Both algorithms were tested on 39 patient CTs, and the resulting treatment fields were scored by a radiation oncologist. We also investigated the feasibility of using one algorithm as an independent check of the other algorithm. Results: 96% of the 3D-Method-generated fields and 79% of the 2D-method-generated fields were scored acceptable for treatment (“Per Protocol” or “Acceptable Variation”). The 3D Method generated more fields scored “Per Protocol” than the 2D Method (62% versus 17%). The 4% of the 3D-Method-generated fields that were scored “Unacceptable Deviation” were all due to an improper L5 vertebra contour resulting in an unacceptable superior jaw position. When these same patients were planned with the 2D method, the superior jaw was acceptable, suggesting that the 2D method can be used to independently check the 3D method. Conclusion: Our results show that our 3D Method is feasible for automatically generating cervical treatment fields. Furthermore, the 2D Method can serve as an automatic, independent check of the automatically-generated treatment fields. These algorithms will be implemented for fully automated cervical treatment planning.

60 APPLIED LIFE SCIENCES↗

DOE OSTI2021

Initial demonstration of automated fuel performance modeling with 1977 EBR-II metallic fuel pins using BISON code with FIPD and IMIS databases

Using the BISON fuel performance code, simulations were conducted using an automated process to read initial and operating conditions from the Fuels Irradiation and Physics Database (FIPD) and Integral Fast Reactor materials information system (IMIS) database, which contains metallic fuel data from the Experimental Breeder Reactor-II (EBR-II). This work demonstrates use of an integrated framework to access the vast majority of EBR-II experimental fuel pin data to support rapid development of fuel performance models for next-generation metallic fuel systems. With this capability, validation for fuel qualification can be performed rapidly. Between IMIS and FIPD, there is enough information to conduct 1977 unique EBR-II metallic fuel pin histories from 24 different experiments, at varying levels of detail between the two databases. Each of these histories includes a high-resolution power history, flux history, coolant channel flow rates, and coolant channel temperatures. Fission gas release (FGR), cumulative damage fraction (CDF), fuel axial swelling, cladding profilometry, and burnup were all simulated in BISON. The results were compared to post-irradiation examination (PIE) results for the initial demonstration of automated BISON modeling. BISON simulations conducted with IMIS and FIPD were in rough agreement with PIE measurements and calculations. Cladding profilometry, FGR, and fuel axial swelling were found to be in rough agreement with PIE measurements, depending on the physics used within the BISON input files. Here, the mechanical contact solver chosen was found to significantly impact axial fuel swelling and cladding strain predictions. CDF values were assessed to see whether pin failure may have been predicted (CDF ≥ 1). This work suggests that continued development of an automated tool for BISON should focus on inclusion of the Fast Flux Test Facility (FFTF) experimental data for a larger database for metallic fuel, improved physical models to better capture fuel performance, such as fuel-cladding interactions, and a more detailed comparison with available PIE data to further the BISON model development.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

NASA NTRS2019

Sketch-To-Solution: An Exploration of Viscous CFD with Automatic Grids

Numerical simulation of the Reynolds-averaged Navier–Stokes (RANS) equations has become a critical tool for the design of aerospace vehicles. However, the issues that affect the grid convergence of three dimensional RANS solutions are not completely understood, as documented in the AIAA Drag Prediction Workshop series. Grid adaption methods have the potential for increasing the automation and discretization error control of RANS solutions to impact the aerospace design and certification process. The realization of the CFD Vision 2030 Study includes automated management of errors and uncertainties of physics-based, predictive modeling that can set the stage for ensuring a vehicle is in compliance with a regulation or specification by using analysis without demonstration in flight test (i.e., certification or qualification by analysis). For example, the Cart3D inviscid analysis package has automated Cartesian cut-cell gridding with output-based error control. Fueled by recent advances in the fields of anisotropic grid adaptation, error estimation, and geometry modeling, a similar work flow is explored for viscous CFD simulations; where a CFD application engineer provides geometry, boundary conditions, and flow parameters, and the sketch-to-solution process yields a CFD simulation through automatic, error-based, grid adaptation.

Kleb, William L.↗

DOE OSTI2016

A thesis on the Development of an Automated SWIFT Edge Detection Algorithm

Throughout the world, scientists and engineers such as those at Los Alamos National Laboratory, perform research and testing unique only to applications aimed towards advancing technology, and understanding the nature of materials. With this testing, comes a need for advanced methods of data acquisition and most importantly, a means of analyzing and extracting the necessary information from such acquired data. In this thesis, I aim to produce an automated method implementing advanced image processing techniques and tools to analyze SWIFT image datasets for Detonator Technology at Los Alamos National Laboratory. Such an effective method for edge detection and point extraction can prove to be advantageous in analyzing such unique datasets and provide for consistency in producing results.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2015

WE-AB-201-00: Treatment Planning System Commissioning and QA

Treatment planning systems (TPS) are a cornerstone of modern radiation therapy. Errors in their commissioning or use can have a devastating impact on many patients. To support safe and high quality care, medical physicists must conduct efficient and proper commissioning, good clinical integration, and ongoing quality assurance (QA) of the TPS. AAPM Task Group 53 and related publications have served as seminal benchmarks for TPS commissioning and QA over the past two decades. Over the same time, continuing innovations have made the TPS even more complex and more central to the clinical process. Medical goals are now expressed in terms of the dose and margins around organs and tissues that are delineated from multiple imaging modalities (CT, MR and PET); and even temporally resolved (i.e., 4D) imaging. This information is passed on to optimization algorithms to establish accelerator movements that are programmed directly for IMRT, VMAT and stereotactic treatments. These advances have made commissioning and QA of the TPS much more challenging. This education session reviews up-to-date experience and guidance on this subject; including the recently published AAPM Medical Physics Practice Guideline (MPPG) #5 “Commissioning and QA of Treatment Planning Dose Calculations: Megavoltage Photon and Electron Beams”. Treatment Planning System Commissioning and QA: Challenges and Opportunities (Greg Salomons) This session will provide some key background and review publications describing prominent incidents relating to TPS commissioning and QA. Traditional approaches have been hardware and feature oriented. They aim to establish a functional configuration and establish specifications for regular testing of features (like dose calculation) to assure stable operation and detect failures. With the advent of more complex systems, more patient-specific testing has also been adopted. A number of actual TPS defects will be presented along with heuristics for identifying similar defects in the future. Finally, the Gamma test has become a popular metric for reporting TPS Commissioning and QA results. It simplifies complex testing into a numerical index, but noisy data and casual application can make it misleading. A brief review of the issues around the use of the Gamma test will be presented. TPS commissioning and QA: A process orientation and application of control charts (Michael Sharpe) A framework for commissioning a treatment planning system will be presented, focusing on preparations, practical aspects of configuration, priorities, specifications, and establishing performance. The complexity of the modern TPS make modular testing of features inadequate, and modern QA tools can provide “too much information” about the performance of techniques like IMRT and VMAT. We have adopted a process orientation and quality tools, like control charts, for ongoing TPS QA and assessment of patient-specific tests. The trending nature of these tools reveals the overall performance of the TPS system, and quantifies the variations that arise from individual plans, discrete calculations, and experimentation based on discrete measurements. Examples demonstrating application of these tools to TPS QA will be presented. TPS commissioning and QA: Incorporating the entire planning process (Sasa Mutic) The TPS and its features do not perform in isolation. Instead, the features and modules are key components in a complex process that begins with CT Simulation and extends to treatment delivery, along with image guidance and verification. Most importantly, the TPS is used by people working in a multi-disciplinary environment. It is very difficult to predict the outcomes of human interactions with software. Therefore, an interdisciplinary approach to training, commissioning and QA will be presented, along with an approach to the physics chart check and end-to-end testing as a tool for TPS QA. The role of standardization and automation in QA will also be discussed. The recommendations of MPPG #5 and practical implementation strategies (Jennifer Smilowitz) The recently published recommendations from Task Group No. 244, Medical Physics Practice Guideline on Commissioning and QA of Treatment Planning Dose Calculations: Megavoltage Photon and Electron Beams will be presented. The recommendations focus on the validation of commissioning data and dose calculations. Tolerance values for non-IMRT beam configurations are summarized based on established criteria and data collected by the IROC. More stringent evaluation criteria for IMRT dose calculations are suggested to test the limitations of the TPS dose algorithms for advanced delivery conditions. The MPPG encourages users to create a suite of validation tests for dose calculation for various conditions for static photon beams, heterogeneities, IMRT/VMAT and electron beams. This test suite is intended to be used for subsequent testing, including TPS software upgrades. In the past, the recommendations of some reports have not been widely implemented due to practical limitations. Implementation strategies, tools and processes developed by multiple centers for efficient and “do-able” MPPG # 5 testing will be presented, as well as a discussion on the overall validation experience. Learning Objectives: Identify some of the key documents relevant for TPS commissioning and QA Understand strategies for testing TPS software Gain a practical knowledge of the Gamma test criteria Increase familiarity with the process of commissioning a TPS Learn about the use of Control Charts for TPS QA Review the role of the TPS in the overall planning process Increase awareness of the link between TPS QA and chart checking Gain an increased appreciation for the importance of interdisciplinary communication Understand the new recommendations from MPPG #5 on TPS Dose Algorithm Commissioning and QC/QA Learn practical implementation processes and tools for MPPG #5 validation recommendations.

60 APPLIED LIFE SCIENCES↗

DOE OSTI2015

WE-AB-201-01: Treatment Planning System Commissioning and QA: Challenges and Opportunities

Treatment planning systems (TPS) are a cornerstone of modern radiation therapy. Errors in their commissioning or use can have a devastating impact on many patients. To support safe and high quality care, medical physicists must conduct efficient and proper commissioning, good clinical integration, and ongoing quality assurance (QA) of the TPS. AAPM Task Group 53 and related publications have served as seminal benchmarks for TPS commissioning and QA over the past two decades. Over the same time, continuing innovations have made the TPS even more complex and more central to the clinical process. Medical goals are now expressed in terms of the dose and margins around organs and tissues that are delineated from multiple imaging modalities (CT, MR and PET); and even temporally resolved (i.e., 4D) imaging. This information is passed on to optimization algorithms to establish accelerator movements that are programmed directly for IMRT, VMAT and stereotactic treatments. These advances have made commissioning and QA of the TPS much more challenging. This education session reviews up-to-date experience and guidance on this subject; including the recently published AAPM Medical Physics Practice Guideline (MPPG) #5 “Commissioning and QA of Treatment Planning Dose Calculations: Megavoltage Photon and Electron Beams”. Treatment Planning System Commissioning and QA: Challenges and Opportunities (Greg Salomons) This session will provide some key background and review publications describing prominent incidents relating to TPS commissioning and QA. Traditional approaches have been hardware and feature oriented. They aim to establish a functional configuration and establish specifications for regular testing of features (like dose calculation) to assure stable operation and detect failures. With the advent of more complex systems, more patient-specific testing has also been adopted. A number of actual TPS defects will be presented along with heuristics for identifying similar defects in the future. Finally, the Gamma test has become a popular metric for reporting TPS Commissioning and QA results. It simplifies complex testing into a numerical index, but noisy data and casual application can make it misleading. A brief review of the issues around the use of the Gamma test will be presented. TPS commissioning and QA: A process orientation and application of control charts (Michael Sharpe) A framework for commissioning a treatment planning system will be presented, focusing on preparations, practical aspects of configuration, priorities, specifications, and establishing performance. The complexity of the modern TPS make modular testing of features inadequate, and modern QA tools can provide “too much information” about the performance of techniques like IMRT and VMAT. We have adopted a process orientation and quality tools, like control charts, for ongoing TPS QA and assessment of patient-specific tests. The trending nature of these tools reveals the overall performance of the TPS system, and quantifies the variations that arise from individual plans, discrete calculations, and experimentation based on discrete measurements. Examples demonstrating application of these tools to TPS QA will be presented. TPS commissioning and QA: Incorporating the entire planning process (Sasa Mutic) The TPS and its features do not perform in isolation. Instead, the features and modules are key components in a complex process that begins with CT Simulation and extends to treatment delivery, along with image guidance and verification. Most importantly, the TPS is used by people working in a multi-disciplinary environment. It is very difficult to predict the outcomes of human interactions with software. Therefore, an interdisciplinary approach to training, commissioning and QA will be presented, along with an approach to the physics chart check and end-to-end testing as a tool for TPS QA. The role of standardization and automation in QA will also be discussed. The recommendations of MPPG #5 and practical implementation strategies (Jennifer Smilowitz) The recently published recommendations from Task Group No. 244, Medical Physics Practice Guideline on Commissioning and QA of Treatment Planning Dose Calculations: Megavoltage Photon and Electron Beams will be presented. The recommendations focus on the validation of commissioning data and dose calculations. Tolerance values for non-IMRT beam configurations are summarized based on established criteria and data collected by the IROC. More stringent evaluation criteria for IMRT dose calculations are suggested to test the limitations of the TPS dose algorithms for advanced delivery conditions. The MPPG encourages users to create a suite of validation tests for dose calculation for various conditions for static photon beams, heterogeneities, IMRT/VMAT and electron beams. This test suite is intended to be used for subsequent testing, including TPS software upgrades. In the past, the recommendations of some reports have not been widely implemented due to practical limitations. Implementation strategies, tools and processes developed by multiple centers for efficient and “do-able” MPPG #5 testing will be presented, as well as a discussion on the overall validation experience. Learning Objectives: Identify some of the key documents relevant for TPS commissioning and QA Understand strategies for testing TPS software Gain a practical knowledge of the Gamma test criteria Increase familiarity with the process of commissioning a TPS Learn about the use of Control Charts for TPS QA Review the role of the TPS in the overall planning process Increase awareness of the link between TPS QA and chart checking Gain an increased appreciation for the importance of interdisciplinary communication Understand the new recommendations from MPPG #5 on TPS Dose Algorithm Commissioning and QC/QA Learn practical implementation processes and tools for MPPG #5 validation recommendations.

60 APPLIED LIFE SCIENCES↗