Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “software verification”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,729 records · Page 96

Cantera and Cantera Electrolyte Thermodynamics Objects

Cantera is a suite of object-oriented software tools for problems involving chemical kinetics, thermodynamics, and/or transport processes. It is a multi-organizational effort to create and formulate high quality 0D and 1D constitutive modeling tools for reactive transport codes.Institutions involved with the effort include Sandia, MIT, Colorado School of Mines, U. Texas, NASA, and Oak Ridge National Labs. Specific to Sandia's contributions, the Cantera Electrolyte Thermo Objects (CETO) packages is comprised of add-on routines for Cantera that handle electrolyte thermochemistry and reactions within the overall Cantera package. Cantera is a C++ Cal Tech code that handles gas phase species transport, reaction, and thermodynamics. With this addition, Cantera can be extended to handle problems involving liquid phase reactions and transport in electrolyte systems, and phase equilibrium problemsinvolving concentrated electrolytes and gas/solid phases. A full treatment of molten salt thermodynamics and transport has also been implemented in CETO. The routines themselves consist of .cpp and .h files containing C++ objects that are derived from parent Cantera objects representing thermodynamic functions. They are linked unto the main Cantera libraries when requested by the user. As an addendum to the main thermodynamics objects, several utility applications are provided. The first is multiphase Gibbs free energy minimizer based on the vcs algorithm, called vcs_cantera. This code allows for the calculation of thermodynamic equilibrium in multiple phases at constant temperature and pressure. Note, a similar code capability exists already in Cantera. This version follows the same algorithm, but gas a different code-base starting point, and is used as a research tool for algorithm development. The second program, cttables, prints out tables of thermodynamic and kinetic information for thermodynamic and kinetic objects within Cantera. This program serves as a "Get the numbers out" utility for Cantera, and as such it is very useful as a verification tool. These add-on utilities are encapsulated into a directory structure named cantera_apps, whose installation uses autoconf and also utilizes Cantera's application environment (i.e., they utilize Cantera as a library).

John Hewson, Harry Moffat↗

Comparison of 3D dose distributions for HDR {sup 192}Ir brachytherapy sources with normoxic polymer gel dosimetry and treatment planning system

Radiation fluence changes caused by the dosimeter itself and poor spatial resolution may lead to lack of 3-dimensional (3D) information depending on the features of the dosimeter and quality assurance of dose distributions for high–dose rate (HDR) iridium-192 ({sup 192}Ir) brachytherapy sources is challenging and experimental dosimetry methods used for brachytherapy sources are limited. In this study, we investigated 3D dose distributions of {sup 192}Ir brachytherapy sources for irradiation with single and multiple dwell positions using a normoxic gel dosimeter and compared them with treatment planning system (TPS) calculations. For dose calibration purposes, 100-mL gel-containing vials were irradiated at predefined doses and then scanned in an magnetic resonance (MR) imaging unit. Gel phantoms prepared in 2 spherical glasses were irradiated with {sup 192}Ir for the calculated dwell positions, and MR scans of the phantoms were obtained. The images were analyzed with MATLAB software. Dose distributions and profiles derived with 1-mm resolution were compared with TPS calculations. Linearity was observed between the delivered dose and the reciprocal of the T2 relaxation time constant of the gel. The x-, y-, and z-axes were defined as the sagittal, coronal, and axial planes, respectively, the sagittal and axial planes were defined parallel to the long axis of the source while the coronal plane was defined horizontally to the long axis of the source. The differences between measured and calculated profile widths of 3-cm source length and point source for 70%, 50%, and 30% isodose lines were evaluated at 3 dose levels using 18 profiles of comparison. The calculations for 3-cm source length revealed a difference of > 3 mm in 1 coordinate at 50% profile width on the sagittal plane and 3 coordinates at 70% profile width and 2 coordinates at 50% and 30% profile widths on the axial plane. Calculations on the coronal plane for 3-cm source length showed > 3-mm difference in 1 coordinate at 50% and 70% and 2 coordinates at 30% profile widths. The point source measurements and calculations for 50% profile widths revealed a difference > 3 mm in 1 coordinate on the sagittal plane and 2 coordinates on the axial plane. The doses of 3 coordinates on the sagittal plane and 4 coordinates on the axial plane could not be evaluated in 30% profile width because of low doses. There was good agreement between the gel dosimetry and TPS results. Gel dosimetry provides dose distributions in all 3 planes at the same time, which enables us to define the dose distributions in any plane with high resolution. It can be used to obtain 3D dose distributions for HDR {sup 192}Ir brachytherapy sources and 3D dose verification of TPS.

61 RADIATION PROTECTION AND DOSIMETRY↗

Electronics Upgrades to the Green is Clean Phoswich Detector Systems and Programmatic Implementation at LANL - Phase II Completion - 18403

Los Alamos National Laboratory (LANL) radiological facilities produce low-density room trash that, in many cases, is not contaminated with radioactivity. It has been estimated that 50 to 90% of low-density room trash is free of radioactive contamination and eligible for inclusion in LANL's Green is Clean (GIC) program. The GIC program is a verification program for non-regulated waste from radiation controlled areas that has been actively segregated as nonradioactive through the use of the waste generator's acceptable knowledge. The GIC program has employed two large area Phoswich gamma ray detector systems (HERCULES and ZEUS) to verify that GIC candidate low-density waste contains no radioactivity above background. The large area (127 cm{sup 2}) Phoswich detectors consist of a thin NaI front crystal (3 mm thick) that is optically coupled to a thick CsI back crystal (50 mm thick). The scintillation properties of the two crystals are different enough to allow the system's electronic components to identify the origin of any pulse. The original systems utilize Nuclear Instrumentation Module (NIM) electronics to process the detector signals. The electronics currently in use by the systems are out of date and some components are no longer manufactured. In 2015, phase I upgrades to the GIC detector systems were completed. Phase I objectives included identifying more modern commercially available electronics that could replace the NIM electronics. A pulse shape discriminator (PSD) module, the Digital Gamma Finder (DGF) Pixie-4 made by XIA LLC, that fits in a compact National Instruments{sup TM} PXI chassis was identified to replace the majority of the original NIM bin electronics - a single Pixie-4 module replaces four NIM amplifiers and four NIM pulse shape analyzers (PSAs). Another phase I objective was to show that that new PSD could separate out the NaI and CsI detector signals in a manner equivalent to or better than the NIM electronics. Proof of concept was achieved in 2015 when a prototype upgrade system on ZEUS was built and tested and gating parameter settings were determined that successfully separated the NaI and CsI signals. Phase II required establishing a LabVIEW software interface with Pixie-4 electronics in lieu of the manufacturer provided Pixie-4 software called the Pixie Viewer. In addition, phase II included 1) final optimization of the gating parameters on the prototype Pixie-4 ZEUS system, 2) finalizing Requirements, Design and Test Plan documents, 3) LabVIEW software interface quality assurance (QA) testing, 4) setting up the HERCULES system with the Pixie-4 upgrade electronics, 5) check source and background calibration of both systems, 6) absolute efficiency determination on both systems and 7) procedure revisions and implementation of both systems into routine operations. In 2017 phase II was completed including the preparation of detailed Requirements, Design and Test Plan documents. The LabVIEW interface software that communicates and controls the Pixie-4 has been completed, merged with the preexisting LabVIEW operations software and fully tested. Gating parameter optimization, configuration of HERCULES with the Pixie-4 electronics, check source and background calibrations and absolute efficiency determinations have all been performed - quantitative results are presented in the full paper. For performance validation, comparison measurements of actual GIC bags on both the HERCULES system using the old NIM Bin electronics and the ZEUS system using the upgraded Pixie-4 electronics have been done and evaluated. Finally, procedure revisions and implementation of both systems into routine operations is currently in process and planned to be complete by the end of calendar year 2017. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

Nek5000 developments in support of industry and the NRC

This year, the Nuclear Energy Advanced Modeling Simulation program (NEAMS) thermal-hydraulics verification and validation (V&V) work has focused in three areas of Nek5000 V&V-driven development. First, in a close collaborative effort with the U. S. Nuclear Regulatory Commission (NRC) staff, we have continued V&V efforts for the HYMERES-2 project using the OECD/NEA sponsored testing in the PSI PANDA facility. This year’s focus of ANL-NRC collaboration involves Nek5000 setups and validation for a range of problems relevant to and including the HYMERES-2 benchmark from PSI. The primary outcome of this year efforts is a more efficient geometry and inlet modeling simplification after a careful sensitivity study of the inlet profiles and pipe geometries. The resulting modeling choice of a short recycling/fully-developed turbulent inlet is within the experimental uncertainty estimate. This finding simplifies the next step of the cross-V&V HYMERES-2 project. In addition, the ANL team continue to provide assistance to the NRC staff in the form of Nek5000 application support in general and on the use of the HPC platforms of ALCF and INL in particular. This supports the NRC’s assessment of Nek5000 for use with the NRC Blue CRAB code suite. Second, we have implemented and tested more robust model of URANS, namely the k – τ model, a variant of the k-ω model, along with other improvements to RANS Nek5000 modeling in general. Because of its demonstrated robustness and stability, the k – τ model is the only RANS model that has been implemented in the new GPU version of the Nek5000 code, nekRS. Lastly, we report the initial implementation of Jacobian-free Newton Krylov approach to the direct Newton method for steady fluid solvers aimed at acceleration of RANS modeling and at IC improvement for LES campaigns. Also leveraging the Exascale Computing Project (ECP) ANL/CEED & SMR team’s software development effort to support NEAMS problems at large scale of the advanced computing architectures, NekRS, a GPU variant of Nek5000, built on top of kernels from libParanumal using OCCA for portability, has been successfully run on the full system of Summit (4608 nodes, 27648 GPUs).

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Modal Correlation of Complex Aerospace Joints Using Automated Variable Substitution

A critical task involved with being able to predict flight loads accurately in aerospace finite element models (FEMs) is the prior verification of the FEMs by conducting modal survey testing (MST). Experience comparing dynamic response of initial FEMs to MST data tends to demonstrate that FEMs can have unacceptable accuracy even when best modeling practices are followed. One inherent source of inaccuracy in linear dynamic FEMs is the modeling of nonlinear joints with mechanisms such as spherical bearings. These joints are usually designed to freely translate or rotate under the high levels of loading experienced in flight. Engineers who create linear FEMs conventionally model these joints without any stiffness in the mechanism degrees of freedom to meet this design intent. However, inaccuracy is observed during test validation of these FEMs, which usually relies on low-level force excitation orders of magnitude below flight load levels. This low-level modal test rarely overcomes the joint friction that is present, and thus the mechanism joints are able to react loads. This divide between the test results and the FEM creates a significant challenge to the engineer who is performing the correlation, in that the engineer has no basis for what stiffness value should be used to make the FEM match the test results. A compounding challenge is that complicated built-up aerospace structures commonly have multiple joints through a load path where each joint will “stick” and “slip” at different levels of force input. Explicitly matching the dynamics of a system containing these nonlinear mechanisms would require a nonlinear FEM, which is prohibitively costly for dynamic simulations of most aerospace systems. The objective of this paper is to present a workflow that can efficiently cycle through many iterations of a FEM, allowing a Monte Carlo style examination of the design space to identify candidate stiffness values for nonlinear mechanism joints. The outlined approach is specific to MSC Nastran and utilizes MSC Nastran’s symbolic substitution capabilities, coupled with the IMAT™ and Attune™ software packages developed by ATA Engineering, Inc. (ATA). The workflow is demonstrated with a case study from the correlation effort for The Boeing Company’s Crew Space Transportation (CST)-100 Starliner FEM. Keywords: Correlation, MSC Nastran, IMAT™, Attune™, Finite Element Model, Modal Testing

Correlation↗

North Dakota Integrated Carbon Storage Complex Feasibility Study. Final report

In spring 2017, the Energy & Environmental Research Center (EERC) initiated an effort to determine the feasibility of developing a commercial-scale CO 2 geologic storage complex able to store 50+ million tonnes (Mt) of CO 2 in central North Dakota safely, permanently, and economically. The objective was to fulfill the goals of the U.S. Department of Energy (DOE) Carbon Storage Assurance Facility Enterprise (CarbonSAFE) Initiative and address technical and nontechnical challenges specific to commercial-scale deployment of a CO2 storage project. The findings clearly show that the concept of capturing CO 2 from a lignite-fired electrical generation facility in central North Dakota and safely and permanently storing the CO 2 in the deep subsurface is indeed technically, economically, and socially feasible. This project evaluated two study areas and their respective geologic storage complexes located adjacent to separate coal-fired facilities in North Dakota: the Basin Electric Power Cooperative (BEPC)-owned Great Plains Synfuels Plant (GPSP) and the Minnkota Power Cooperative (Minnkota)-owned Milton R. Young Station (MRYS). These locations, one with CO 2 capture in place and an existing CO 2 pipeline, are bolstered by progressive North Dakota pore space ownership and long-term liability laws. These elements and a motivated team created an ideal synergistic scenario for ensuring success of the CarbonSAFE Initiative and promoting North Dakota’s statewide vision for carbon management. The project included drilling two new geologic characterization wells, integrating an existing 3-D seismic survey, creating a geologic model subsequently used for injection simulation, a risk assessment, public outreach, and generating a site development plan based on results. In addition, the performance of select National Risk Assessment Partnership tools was evaluated. The geologic characterization wells were drilled ~5600 feet deep to the Broom Creek Formation; ~350 feet of core was retrieved from each well. The core included the Broom Creek (targeted injection zone) and a portion of the overlying Opeche Shale (seal). The Flemmer-1 well, west of Beulah, North Dakota, yielded 169 feet of sandstone. The BNI-1 well located south of Center, North Dakota, yielded 124 feet of sandstone. In each case, laboratory analysis of the sandstone showed permeability in the 300–1000-mD range, with porosity of 20%–30%. The Flemmer-1 well was sited within the boundaries of an existing 3-D seismic survey. Colocating the well with the seismic survey maximized the relationship between new and legacy data and developed a first-of-its-kind interpretation of the geologic fabric of the Broom Creek. Geologic characterization data were integrated into a 5544-mi 2 geocellular model that encompassed both new wells and stratigraphy from the surface to the Amsden Formation (underlying the Broom Creek). The model was later expanded vertically to include the deeper Black Island–Deadwood interval to examine its potential viability as a storage target. The geocellular model provided the foundation for dynamic simulation of CO 2 into the Broom Creek. Results of the simulation suggest that the Broom Creek could accept the DOE target rate of 2 Mt/yr of CO 2 into as few as two wells. To bracket the expected capture from MRYS, simulations were also investigated for a 4-Mt/yr rate near MRYS. Although more wells are needed (two additional), the Broom Creek still has the storage resource to accept the CO 2 at the increased rate. A risk assessment exercise was conducted to identify and assess technical and nontechnical risks that could prevent potential candidate storage complexes within the study area from serving as commercial storage sites. The assessment identified and evaluated six technical risk categories: 1) CO 2 injectivity, 2) storage capacity, 3) lateral migration of CO 2 , 4) lateral pressure propagation, 5) vertical migration of CO 2 or formation brine, and 6) induced seismicity. Following two rounds of analysis and scoring, no risks were determined to preclude continued efforts to develop carbon capture, utilization, and storage (CCUS) in central North Dakota. The risk assessment results will be used to guide future site characterization, modeling and simulation, and monitoring activities. A specific nontechnical strategic risk based on challenges that may be realized in amalgamation of pore space resulted in vertically expanding the geologic model to incorporate the potential for stacked storage in multiple saline reservoirs. By including the Black Island–Deadwood interval (the basal sedimentary reservoir in this region), the amalgamated areal extent could be reduced by as much as 45%. Working with a smaller geographic area reduces risks and costs associated with monitoring and pore space leasing. An economic evaluation incorporating capture; transport (<5 mi); Class VI wells; permitting; and monitoring, verification, and accounting suggests implementing commercial-scale CCUS is economically attractive if the federal tax benefits of 45Q are included. This is validated by Minnkota’s continued pursuit of CO 2 capture and geologic storage at MRYS through its Project Tundra initiative, indicating that there is a business case for CCUS in central North Dakota. Currently, North Dakota is the only state with underground injection control (UIC) Class VI primacy. Built into the North Dakota Century Code is a series of regulatory requirements that guide the process to obtain a Class VI CO 2 storage facility permit. As part of this project, a site development plan was compiled to assure compliance with North Dakota’s requirements to permit a commercial-scale CO 2 storage operation and includes a prospective time line encompassing a general breakdown of activities. In total, an estimated 30 months would be needed to execute the necessary steps to attain a North Dakota CO 2 storage facility permit. Outreach was an integral part of the project and encompassed any project-related activity that had contact or exposure beyond the project team. The goals of outreach were to foster an environment from which stakeholders could make informed decisions about the project and gauge community receptiveness to a CCUS project. A consistent set of messages and outreach products were developed in conjunction with an outreach advisory board that integrated project partners and team members. To gauge public acceptability of geologically storing CO 2 , 5611 households in the project area were invited to participate in an online survey. The survey results indicate that the public attitude regarding CCUS is neutral to positive, with strong sentiment that CO 2 capture and storage may be an approach to maintain the economic vitality of the region. To achieve project objectives, critical support in the form of financial backing, engineering evaluations, site access, outreach collaboration, operations data, risk assessment/evaluation, and software access was provided by BEPC, the North Dakota Industrial Commission Lignite Research Council, ALLETE Clean Energy, BNI Energy, North American Coal Corporation, Minnkota, Prairie Public Broadcasting, Computer Modelling Group Ltd., and Schlumberger.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

Web-Based Tools for Data-Informed Remedy Optimization: Software Theory and User Guide

This report documents the development and application of two web-based decision-support tools for pump-and-treat (P&T) groundwater remediation systems: PTOLEMY (Pump-and-Treat Optimized Location Evaluation to Maximize Yields) and OPTIMA (Optimization for Pump-and-Treat Implementation, Management, & Assessment). These tools enhance remedy design and management by leveraging advanced computational methods – specifically deep learning and multi-objective optimization – within a user-friendly platform. By integrating data-driven models with established hydrogeological knowledge, PTOLEMY and OPTIMA enable more efficient evaluation of well placement and operational strategies, helping site managers balance multiple remediation objectives under complex conditions. Both tools are implemented as modules within the SOCRATES (Suite Of Comprehensive Rapid Analysis Tools for Environmental Sites) web platform, which provides data access, visualization, and analytics to support remedy optimization across sites in the U.S. Department of Energy Office of Environmental Management complex. PTOLEMY is a rapid screening module designed to identify promising locations for new extraction wells. It employs a multi-channel three-dimensional convolutional neural network (MC3D-CNN) trained on high-fidelity simulation data to predict the relative performance (in terms of contaminant mass recovery) of potential well sites. Through an interactive web interface, PTOLEMY visualizes the probability of high performance across a site, highlighting areas where an extraction well is likely to yield above-threshold contaminant removal over a multi-year period. PTOLEMY’s map-based displays and exportable results support transparent communication of screening analyses. By focusing attention on the most favorable candidate locations, the tool augments traditional engineering judgment and physics-based modeling, providing a data informed basis for subsequent detailed evaluations. OPTIMA is a multi objective optimization module designed to find wellfield layouts and operating schedules that meet various cleanup goals. It quickly evaluates thousands of candidate setups – combinations of well locations, timing, and rates – and returns a small set of best trade-off options for comparison. At its core, OPTIMA uses a U-Net-based surrogate model – a deep-learning emulator of a groundwater flow and transport simulator – to dramatically accelerate scenario evaluations. Coupling this fast surrogate with the NSGA-II (Non-dominated Sorting Genetic Algorithm II) evolutionary algorithm, OPTIMA explores a wide decision space of well locations and schedules to identify Pareto-optimal solutions that trade off key objectives (e.g., minimizing cleanup time, maximizing contaminant mass removal, and minimizing plume extent). The tool outputs a family of optimal configurations and visualizes their trade-offs (Pareto frontiers of cleanup metrics and maps of optimized well placements). Site managers can use these results to understand the range of viable strategies and to select candidate designs for more detailed verification. OPTIMA is currently under active development and not yet fully released; this guide provides early documentation to support planning and gather user feedback.

54 ENVIRONMENTAL SCIENCES↗

Status of Electronics Upgrades to the LANL Green is Clean Phoswich Detector Systems - 16419

Los Alamos National Laboratory radiological facilities produce low-density room trash that, in many cases, is not contaminated with radioactivity. It has been estimated that 50 to 90% of low-density room trash is free of radioactive contamination and eligible for inclusion in LANL's Green is Clean (GIC) program. The GIC program is a verification program for non-regulated waste from radiation controlled areas that has been actively segregated as nonradioactive through the use of the waste generator's acceptable knowledge. The GIC program has employed two large area Phoswich gamma ray detector systems (HERCULES and ZEUS) to assay and verify that GIC candidate waste meets free release disposal requirements. The large area (127 cm{sup 2}) Phoswich detectors consist of a thin NaI front crystal (3 mm thick) that is optically coupled to a thick CsI back crystal (50 mm thick). The crystals are set in an oxygen-free high-conductivity copper housing with a very thin (0.025 mm) aluminum entrance window. The scintillation properties of the two crystals are different enough to allow the system's electronic components to identify the origin of any pulse. The original systems utilize Nuclear Instrumentation Module (NIM) electronics to process the detector signals. These electronic modules provide the buffering, amplification, and pulse shape analyzer needed for the proper presentation of signals to the custom Phosmux router module. The router module separates the sodium iodide and cesium iodide signals by their rise times and transfers this information into the computer-mounted multichannel analyzer, a Canberra Model S-100 system. While these NIM Bin counting electronics configurations have been effective for approximately 20 years of operation, they are bulky requiring numerous modules per detector and the important Phosmux module is no longer manufactured by LANL. Modern advancement of commercial counting electronics now offer more compact options for equivalent detector signal processing. A pulse shape discriminator module, the DGF Pixie-4, made by XIA LLC, has been identified that can replace the majority of the original NIM bin electronics in a much more compact chassis - one Pixie-4 replaces four NIM amplifiers and four NIM PSAs and fits in a small National Instruments{sup TM} chassis. Thus, LANL has initiated a system upgrade project to replace the historic NIM electronics with the XIA Pixie4 and associated configuration software. The hardware for one Phoswich systems has been procured for the project and the configuration software, customized for the LANL Phoswich detector application, has been written by the manufacturer. Initial setup and optimization of the Pixie-4 configuration has been performed. Qualitative and quantitative baseline comparisons to the NIM electronics has been performed. Good agreement of key low energy gamma lines between the systems has allowed 'Proof of Concept' to be concluded. Further optimization and performance evaluations are in progress. Modifications to the LabVIEW{sup TM} based operating software (originally written by VI Control Systems Ltd.) are being performed by a LANL LabVIEW{sup TM} programmer to accommodate the electronics upgrade. A new LabVIEW{sup TM} interface to control the Pixie-4 has already been developed. Final optimization and testing, formal software QA, and revisions to operating procedures are in progress to achieve completion of the GIC system upgrade project. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

Comparison of 3-dimensional dose reconstruction system between fluence-based system and dose measurement-guided system

COMPASS system (IBA Dosimetry, Schwarzenbruck, Germany) and ArcCHECK with 3DVH software (Sun Nuclear Corp., Melbourne, FL) are commercial quasi-3-dimensional (3D) dosimetry arrays. Cross-validation to compare them under the same conditions, such as a treatment plan, allows for clear evaluation of such measurement devices. In this study, we evaluated the accuracy of reconstructed dose distributions from the COMPASS system and ArcCHECK with 3DVH software using Monte Carlo simulation (MC) for multi-leaf collimator (MLC) test patterns and clinical VMAT plans. In a phantom study, ArcCHECK 3DVH showed clear differences from COMPASS, measurement and MC due to the detector resolution and the dose reconstruction method. Especially, ArcCHECK 3DVH showed 7% difference from MC for the heterogeneous phantom. ArcCHECK 3DVH only corrects the 3D dose distribution of treatment planning system (TPS) using ArcCHECK measurement, and therefore the accuracy of ArcCHECK 3DVH depends on TPS. In contrast, COMPASS showed good agreement with MC for all cases. However, the COMPASS system requires many complicated installation procedures such as beam modeling, and appropriate commissioning is needed. In terms of clinical cases, there were no large differences for each QA device. The accuracy of the compass and ArcCHECK 3DVH systems for phantoms and clinical cases was compared. Both systems have advantages and disadvantages for clinical use, and consideration of the operating environment is important. The QA system selection is depending on the purpose and workflow in each hospital.

61 RADIATION PROTECTION AND DOSIMETRY↗

Verification and Validation of the Stakeholder Tool for Assessing Radioactive Transportation (START) – 22323

The U.S. Department of Energy (DOE) Office of Integrated Waste Management is planning for the eventual transportation, storage, and disposal of spent nuclear fuel (SNF) and high-level radioactive waste (HLW) from nuclear power plant and DOE sites. The Stakeholder Tool for Assessing Radioactive Transportation (START) is a web-based, geospatial decision-support tool developed for evaluating routing options and other aspects of transporting SNF and HLW, covering rail, truck, barge, and intermodal infrastructure and operations in the continental United States. The verification and validation (V&V) process is intended to independently assess START to provide confidence in the ability of START to accurately provide intended results. The V&V process checks the START tool using a variety of methods, ranging from independent hand calculations to comparison of START performance and results to those of other codes. The V&V activity was conducted independently from the START development team with opportunities to provide feedback and collaborate throughout the process. The V&V analyzed attributes of transportation routes produced by START, including route distance and both population and population density captured within buffer zones around routes. Population in the buffer zone, population density in the buffer zone, and route distance were all identified as crucial outputs of the START code and were subject to V&V tasks. Some of the improvements identified through the V&V process were standardizing the underlying population data in START, changing the projection of the population raster data, and changes to the methodology used for population density to improve its applicability for expected users. This collaboration also led to suggested improvements to some of the underlying shape file segments within START. A formal series of test routes went through the V&V process from START version 3.2.1 following the implementation of the recommended improvements to START. The V&V of the buffer zone population and distance reported in START were achieved by utilizing a custom workflow developed in QGIS. This platform was chosen because it is fully independent of START. The buffer zone population and route distances in START showed excellent agreement to independent V&V test results. Over 200 route test cases were run; in all cases, the percent difference in the population within the buffer zone was less than +/- 5%, and the majority of cases were below +/- 1%. These cases were also duplicated in ArcMap, which is Environmental Systems Research Institute’s (ESRI) GIS desktop application. The ArcMap results also show good agreement; however, they cannot be considered independent of START because START also uses ESRI software. The V&V tasks are currently addressing population density and estimated radiation dose in START as well as implementing further automation of the V&V process to allow larger suites of test routes to be run more easily and quickly by less experienced GIS users. These improvements to the V&V process will make it a valuable tool for the development team and the stakeholders so that each new iteration of START can easily be checked for quality and consistency. This paper and presentation will describe the V&V process, findings and implications, and ongoing V&V activities.

START, DOE, Transporation, dose, GIS↗

Impact of MODIS High-Resolution Sea-Surface Temperatures on WRF Forecasts at NWS Miami, FL

Over the past few years,studies at the Short-term Prediction Research and Transition (SPoRT) Center have suggested that the use of Moderate Resolution Imaging Spectroradiometer (MODIS) composite sea-surface temperature (SST) products in regional weather forecast models can have a significant positive impact on short-term numerical weather prediction in coastal regions. The recent paper by LaCasse et al. (2007, Monthly Weather Review) highlights lower atmospheric differences in regional numerical simulations over the Florida offshore waters using 2-km SST composites derived from the MODIS instrument aboard the polar-orbiting Aqua and Terra Earth Observing System satellites. To help quantify the value of this impact on NWS Weather Forecast Offices (WFOs), the SPoRT Center and the NWS WFO at Miami, FL (MIA) are collaborating on a project to investigate the impact of using the high-resolution MODIS SST fields within the Weather Research and Forecasting (WRF) prediction system. The scientific hypothesis being tested is: More accurate specification of the lower-boundary forcing within WRF will result in improved land/sea fluxes and hence, more accurate evolution of coastal mesoscale circulations and the associated sensible weather elements. The NWS MIA is currently running the WRF system in real-time to support daily forecast operations, using the National Centers for Environmental Prediction Nonhydrostatic Mesoscale Model dynamical core within the NWS Science and Training Resource Center's Environmental Modeling System (EMS) software; The EMS is a standalone modeling system capable of downloading the necessary daily datasets, and initializing, running and displaying WRF forecasts in the NWS Advanced Weather Interactive Processing System (AWIPS) with little intervention required by forecasters. Twenty-seven hour forecasts are run daily with start times of 0300,0900, 1500, and 2100 UTC on a domain with 4-km grid spacing covering the southern half of Florida and the far western portions of the Bahamas, the Florida Keys, the Straights of Florida, and adjacent waters of the Gulf of Mexico and Atlantic Ocean. Each model run is initialized using the Local Analysis and Prediction System (LAPS) analyses available in AWIPS, invoking the diabatic. "hot-start" capability. In this WRF model "hot-start", the LAPS-analyzed cloud and precipitation features are converted into model microphysics fields with enhanced vertical velocity profiles, effectively reducing the model spin-up time required to predict precipitation systems. The SSTs are initialized with the NCEP Real-Time Global (RTG) analyses at l/12 degree resolution (approx. 9 km); however, the RTG product does not exhibit fine-scale details consistent with its grid resolution. SPoRT is conducting parallel WRF EMS runs identical to the operational runs at NWS MIA in every respect except for the use of MODIS SST composites in place of the RTG product as the initial and boundary conditions over water. The MODIS SST composites for initializing the SPoRT WRF runs are generated on a 2-km grid four times daily at 0400, 0700, 1600, and 1900 UTC, based on the times of the overhead passes of the Aqua and Terra satellites. The incorporation of the MODIS SST composites into the SPoRTWRF runs is staggered such that the 0400UTC composite initializes the 0900 UTC WRF, the 0700 UTC composite initializes the 1500 UTC WRF, the 1600 UTC composite initializes the 2100 UTC WRF, and the 1900 UTC composite initializes the 0300 UTC WRF. A comparison of the SPoRT and Miami forecasts is underway in 2007, and includes quantitative verification of near-surface temperature, dewpoint, and wind forecasts at surface observation locations. In addition, particular days of interest are being analyzed to determine the impact of the MODIS SST data on the development and evolution of predicted sea/land-breeze circulations, clouds, and precipitation. This paper will present verification results comparing the NWS MIA forecasts the SPoRT experimental WRF forecasts, and highlight any substantial differences noted in the predicted mesoscale phenomena.

Case, Jonathan L.↗

MO-B-BRD-02: 3D Printing in the Clinic

This session is designed so that the learning objectives are practical. The intent is that the attendee may take home an understanding of not just the technology, but also the logistical steps necessary to execute these 3D printing techniques in the clinic. Four practical 3D printing topics will be discussed: (i) Creating bolus and compensators for photon machines; (ii) tools for proton therapy; (iii) clinical applications in imaging; (iv) custom phantom design for clinic and research use. The use of 3D printers within the radiation oncology setting is proving to be a useful tool for creating patient specific bolus and compensators with the added benefit of cost savings. Creating the proper protocol is essential to ensuring that the desired effect is achieved and modeled in the treatment planning system. The critical choice of printer material (since it determines the interaction with the radiation) will be discussed. Selection of 3D printer type, design methods, verification of dose calculation, and the printing process will be detailed to give the basis for establishing your own protocol for electron and photon fields. A practical discussion of likely obstacles that may be encountered will be included. The diversity of systems and techniques in proton facilities leads to different facilities having very different requirements for beam modifying hardware and quality assurance devices. Many departments find the need to design and fabricate facility-specific equipment, making 3D printing an attractive technology. 3D printer applications in proton therapy will be discussed, including beam filters and compensators, and the design of proton therapy specific quality assurance tools. Quality control specific to 3D printing in proton therapy will be addressed. Advantages and disadvantages of different printing technology for these applications will also be discussed. 3D printing applications using high-resolution radiology-based imaging data will be presented. This data is used to 3D print individualized physical models of patient’s unique anatomy for aid in planning complex and challenging surgical procedures. Methods, techniques and imaging requirements for 3D printing anatomic models from imaging data will be discussed. Specific applications currently being used in the radiology clinic will be detailed. Standardized phantoms for radiation therapy are abundant. However, custom phantom designs can be advantageous for both clinical tasks and research. 3D printing is a useful method of custom fabrication that allows one to construct custom objects relatively quickly. Possibilities for custom radiotherapy phantoms range from 3D printing a hollow shell and filling the shell with tissue equivalent materials to fully printing the entire phantom with materials that are tissue equivalent as well as suitable for 3D printing. A range of materials available for use in radiotherapy phantoms and in the case of phantoms for dosimetric measurements, this choice is critical. The necessary steps required will be discussed including: modalities of 3D model generation, 3D model requirements for 3D printing, generation of machine instructions from the 3D model, and 3D printing techniques, choice of phantoms material, and troubleshooting techniques for each step in the process. Case examples of 3D printed phantoms will be shown. Learning Objectives: Understand the types of 3D modeling software required to design your device, the file formats required for data transfer from design software to 3D printer, and general troubleshooting techniques for each step of the process. Learn the differences between materials and design for photons vs. electrons vs. protons. Understand the importance of material choice and design geometries for your custom phantoms. Learn specific steps of quality assurance and quality control for 3D printed beam filters and compensators for proton therapy. Learn of special 3D printing applications for imaging. Cunha: Research support from Phillips Healthcare.

60 APPLIED LIFE SCIENCES↗

MO-B-BRD-03: Principles, Pitfalls and Techniques of 3D Printing for Bolus and Compensators

This session is designed so that the learning objectives are practical. The intent is that the attendee may take home an understanding of not just the technology, but also the logistical steps necessary to execute these 3D printing techniques in the clinic. Four practical 3D printing topics will be discussed: (i) Creating bolus and compensators for photon machines; (ii) tools for proton therapy; (iii) clinical applications in imaging; (iv) custom phantom design for clinic and research use. The use of 3D printers within the radiation oncology setting is proving to be a useful tool for creating patient specific bolus and compensators with the added benefit of cost savings. Creating the proper protocol is essential to ensuring that the desired effect is achieved and modeled in the treatment planning system. The critical choice of printer material (since it determines the interaction with the radiation) will be discussed. Selection of 3D printer type, design methods, verification of dose calculation, and the printing process will be detailed to give the basis for establishing your own protocol for electron and photon fields. A practical discussion of likely obstacles that may be encountered will be included. The diversity of systems and techniques in proton facilities leads to different facilities having very different requirements for beam modifying hardware and quality assurance devices. Many departments find the need to design and fabricate facility-specific equipment, making 3D printing an attractive technology. 3D printer applications in proton therapy will be discussed, including beam filters and compensators, and the design of proton therapy specific quality assurance tools. Quality control specific to 3D printing in proton therapy will be addressed. Advantages and disadvantages of different printing technology for these applications will also be discussed. 3D printing applications using high-resolution radiology-based imaging data will be presented. This data is used to 3D print individualized physical models of patient’s unique anatomy for aid in planning complex and challenging surgical procedures. Methods, techniques and imaging requirements for 3D printing anatomic models from imaging data will be discussed. Specific applications currently being used in the radiology clinic will be detailed. Standardized phantoms for radiation therapy are abundant. However, custom phantom designs can be advantageous for both clinical tasks and research. 3D printing is a useful method of custom fabrication that allows one to construct custom objects relatively quickly. Possibilities for custom radiotherapy phantoms range from 3D printing a hollow shell and filling the shell with tissue equivalent materials to fully printing the entire phantom with materials that are tissue equivalent as well as suitable for 3D printing. A range of materials available for use in radiotherapy phantoms and in the case of phantoms for dosimetric measurements, this choice is critical. The necessary steps required will be discussed including: modalities of 3D model generation, 3D model requirements for 3D printing, generation of machine instructions from the 3D model, and 3D printing techniques, choice of phantoms material, and troubleshooting techniques for each step in the process. Case examples of 3D printed phantoms will be shown. Learning Objectives: Understand the types of 3D modeling software required to design your device, the file formats required for data transfer from design software to 3D printer, and general troubleshooting techniques for each step of the process. Learn the differences between materials and design for photons vs. electrons vs. protons. Understand the importance of material choice and design geometries for your custom phantoms. Learn specific steps of quality assurance and quality control for 3D printed beam filters and compensators for proton therapy. Learn of special 3D printing applications for imaging. Cunha: Research support from Phillips Healthcare.

60 APPLIED LIFE SCIENCES↗

MO-B-BRD-04: Sterilization for 3D Printed Brachytherapy Applicators

This session is designed so that the learning objectives are practical. The intent is that the attendee may take home an understanding of not just the technology, but also the logistical steps necessary to execute these 3D printing techniques in the clinic. Four practical 3D printing topics will be discussed: (i) Creating bolus and compensators for photon machines; (ii) tools for proton therapy; (iii) clinical applications in imaging; (iv) custom phantom design for clinic and research use. The use of 3D printers within the radiation oncology setting is proving to be a useful tool for creating patient specific bolus and compensators with the added benefit of cost savings. Creating the proper protocol is essential to ensuring that the desired effect is achieved and modeled in the treatment planning system. The critical choice of printer material (since it determines the interaction with the radiation) will be discussed. Selection of 3D printer type, design methods, verification of dose calculation, and the printing process will be detailed to give the basis for establishing your own protocol for electron and photon fields. A practical discussion of likely obstacles that may be encountered will be included. The diversity of systems and techniques in proton facilities leads to different facilities having very different requirements for beam modifying hardware and quality assurance devices. Many departments find the need to design and fabricate facility-specific equipment, making 3D printing an attractive technology. 3D printer applications in proton therapy will be discussed, including beam filters and compensators, and the design of proton therapy specific quality assurance tools. Quality control specific to 3D printing in proton therapy will be addressed. Advantages and disadvantages of different printing technology for these applications will also be discussed. 3D printing applications using high-resolution radiology-based imaging data will be presented. This data is used to 3D print individualized physical models of patient’s unique anatomy for aid in planning complex and challenging surgical procedures. Methods, techniques and imaging requirements for 3D printing anatomic models from imaging data will be discussed. Specific applications currently being used in the radiology clinic will be detailed. Standardized phantoms for radiation therapy are abundant. However, custom phantom designs can be advantageous for both clinical tasks and research. 3D printing is a useful method of custom fabrication that allows one to construct custom objects relatively quickly. Possibilities for custom radiotherapy phantoms range from 3D printing a hollow shell and filling the shell with tissue equivalent materials to fully printing the entire phantom with materials that are tissue equivalent as well as suitable for 3D printing. A range of materials available for use in radiotherapy phantoms and in the case of phantoms for dosimetric measurements, this choice is critical. The necessary steps required will be discussed including: modalities of 3D model generation, 3D model requirements for 3D printing, generation of machine instructions from the 3D model, and 3D printing techniques, choice of phantoms material, and troubleshooting techniques for each step in the process. Case examples of 3D printed phantoms will be shown. Learning Objectives: Understand the types of 3D modeling software required to design your device, the file formats required for data transfer from design software to 3D printer, and general troubleshooting techniques for each step of the process. Learn the differences between materials and design for photons vs. electrons vs. protons. Understand the importance of material choice and design geometries for your custom phantoms. Learn specific steps of quality assurance and quality control for 3D printed beam filters and compensators for proton therapy. Learn of special 3D printing applications for imaging. Cunha: Research support from Phillips Healthcare.

60 APPLIED LIFE SCIENCES↗

MO-B-BRD-00: Clinical Applications of 3D Printing

This session is designed so that the learning objectives are practical. The intent is that the attendee may take home an understanding of not just the technology, but also the logistical steps necessary to execute these 3D printing techniques in the clinic. Four practical 3D printing topics will be discussed: (i) Creating bolus and compensators for photon machines; (ii) tools for proton therapy; (iii) clinical applications in imaging; (iv) custom phantom design for clinic and research use. The use of 3D printers within the radiation oncology setting is proving to be a useful tool for creating patient specific bolus and compensators with the added benefit of cost savings. Creating the proper protocol is essential to ensuring that the desired effect is achieved and modeled in the treatment planning system. The critical choice of printer material (since it determines the interaction with the radiation) will be discussed. Selection of 3D printer type, design methods, verification of dose calculation, and the printing process will be detailed to give the basis for establishing your own protocol for electron and photon fields. A practical discussion of likely obstacles that may be encountered will be included. The diversity of systems and techniques in proton facilities leads to different facilities having very different requirements for beam modifying hardware and quality assurance devices. Many departments find the need to design and fabricate facility-specific equipment, making 3D printing an attractive technology. 3D printer applications in proton therapy will be discussed, including beam filters and compensators, and the design of proton therapy specific quality assurance tools. Quality control specific to 3D printing in proton therapy will be addressed. Advantages and disadvantages of different printing technology for these applications will also be discussed. 3D printing applications using high-resolution radiology-based imaging data will be presented. This data is used to 3D print individualized physical models of patient’s unique anatomy for aid in planning complex and challenging surgical procedures. Methods, techniques and imaging requirements for 3D printing anatomic models from imaging data will be discussed. Specific applications currently being used in the radiology clinic will be detailed. Standardized phantoms for radiation therapy are abundant. However, custom phantom designs can be advantageous for both clinical tasks and research. 3D printing is a useful method of custom fabrication that allows one to construct custom objects relatively quickly. Possibilities for custom radiotherapy phantoms range from 3D printing a hollow shell and filling the shell with tissue equivalent materials to fully printing the entire phantom with materials that are tissue equivalent as well as suitable for 3D printing. A range of materials available for use in radiotherapy phantoms and in the case of phantoms for dosimetric measurements, this choice is critical. The necessary steps required will be discussed including: modalities of 3D model generation, 3D model requirements for 3D printing, generation of machine instructions from the 3D model, and 3D printing techniques, choice of phantoms material, and troubleshooting techniques for each step in the process. Case examples of 3D printed phantoms will be shown. Learning Objectives: Understand the types of 3D modeling software required to design your device, the file formats required for data transfer from design software to 3D printer, and general troubleshooting techniques for each step of the process. Learn the differences between materials and design for photons vs. electrons vs. protons. Understand the importance of material choice and design geometries for your custom phantoms. Learn specific steps of quality assurance and quality control for 3D printed beam filters and compensators for proton therapy. Learn of special 3D printing applications for imaging. Cunha: Research support from Phillips Healthcare.

60 APPLIED LIFE SCIENCES↗

MO-B-BRD-01: Creation of 3D Printed Phantoms for Clinical Radiation Therapy

This session is designed so that the learning objectives are practical. The intent is that the attendee may take home an understanding of not just the technology, but also the logistical steps necessary to execute these 3D printing techniques in the clinic. Four practical 3D printing topics will be discussed: (i) Creating bolus and compensators for photon machines; (ii) tools for proton therapy; (iii) clinical applications in imaging; (iv) custom phantom design for clinic and research use. The use of 3D printers within the radiation oncology setting is proving to be a useful tool for creating patient specific bolus and compensators with the added benefit of cost savings. Creating the proper protocol is essential to ensuring that the desired effect is achieved and modeled in the treatment planning system. The critical choice of printer material (since it determines the interaction with the radiation) will be discussed. Selection of 3D printer type, design methods, verification of dose calculation, and the printing process will be detailed to give the basis for establishing your own protocol for electron and photon fields. A practical discussion of likely obstacles that may be encountered will be included. The diversity of systems and techniques in proton facilities leads to different facilities having very different requirements for beam modifying hardware and quality assurance devices. Many departments find the need to design and fabricate facility-specific equipment, making 3D printing an attractive technology. 3D printer applications in proton therapy will be discussed, including beam filters and compensators, and the design of proton therapy specific quality assurance tools. Quality control specific to 3D printing in proton therapy will be addressed. Advantages and disadvantages of different printing technology for these applications will also be discussed. 3D printing applications using high-resolution radiology-based imaging data will be presented. This data is used to 3D print individualized physical models of patient’s unique anatomy for aid in planning complex and challenging surgical procedures. Methods, techniques and imaging requirements for 3D printing anatomic models from imaging data will be discussed. Specific applications currently being used in the radiology clinic will be detailed. Standardized phantoms for radiation therapy are abundant. However, custom phantom designs can be advantageous for both clinical tasks and research. 3D printing is a useful method of custom fabrication that allows one to construct custom objects relatively quickly. Possibilities for custom radiotherapy phantoms range from 3D printing a hollow shell and filling the shell with tissue equivalent materials to fully printing the entire phantom with materials that are tissue equivalent as well as suitable for 3D printing. A range of materials available for use in radiotherapy phantoms and in the case of phantoms for dosimetric measurements, this choice is critical. The necessary steps required will be discussed including: modalities of 3D model generation, 3D model requirements for 3D printing, generation of machine instructions from the 3D model, and 3D printing techniques, choice of phantoms material, and troubleshooting techniques for each step in the process. Case examples of 3D printed phantoms will be shown. Learning Objectives: Understand the types of 3D modeling software required to design your device, the file formats required for data transfer from design software to 3D printer, and general troubleshooting techniques for each step of the process. Learn the differences between materials and design for photons vs. electrons vs. protons. Understand the importance of material choice and design geometries for your custom phantoms. Learn specific steps of quality assurance and quality control for 3D printed beam filters and compensators for proton therapy. Learn of special 3D printing applications for imaging. Cunha: Research support from Phillips Healthcare.

60 APPLIED LIFE SCIENCES↗

Discrete Event Simulation-Based Timeline Validation Using R2U2

The Gateway Vehicle Systems Manager (VSM), the top-level software control system in a distributed, hierarchical Autonomous System Management Architecture is, like most modern spacecraft software control systems, heavily data-driven. For example, schedules (timelines) will be developed on the ground and, due to the high degree of autonomy, contain complex procedures involving conditional branching, variable timing, and resource contention resolution. In order to verify that an uploaded timeline will function correctly, it is necessary to explore the feasible set of possible executions. While it is possible to test a timeline using a mission simulation, the complexity of the system and duration of a timeline limits the number of trials and therefore the test coverage. To address this problem, the VSM team is using a discrete event system model that can rapidly generate from a timeline sets of event sequences using Monte Carlo techniques. To achieve rapid and trustworthy checking of the event sequences, we use an offline version of the runtime model checking tool R2U2. This presentation describes the approach the VSM team is using to implement the discrete event simulation and evaluate event sequences using R2U2. The presentation will discuss: 1. Description of the timelines by VSM in the context of VSM operations 2. Expansion of a timeline into a sequence of atomic events 3. Adjustment, in the Monte Carlo environment, of an event sequence to account for uncertainty, external events, and failures 4. Definition of R2U2 input and mission-time linear temporal logic files 5. Generation and use of R2U2 verdict sequences 6. Lessons learned and future work

Verification↗

The Uniform Methods Project: Smart Thermostat Evaluation Protocol

A smart thermostat is an internet-connected device that controls home heating, ventilation, and air-conditioning (HVAC) equipment and can automatically adjust temperature set points to optimize performance and achieve energy savings. Smart thermostat features often include two way communication, occupancy detection (such as geofencing and occupancy sensors), schedule learning, and seasonal optimization algorithms. Smart thermostats can control most conventional HVAC systems, including central air conditioners, heat pumps, and forced air furnaces. Several types of residential utility programs offer smart thermostats as replacements measures. Working with smart thermostat vendors, utilities can offer separate optimization programs to produce energy savings beyond those achieved by installing a smart thermostat. From an evaluation perspective, smart thermostat programs have several noteworthy features. First, the energy savings from a smart thermostat may change over the life of the device. As a smart thermostat is connected to the internet, original equipment manufacturers can update the thermostat software to improve the thermostat's energy efficiency. Likewise, users can adjust the thermostat settings and schedules over time in response to changes in weather, thermal comfort, energy prices, or preferences for energy efficiency. Additionally, many thermostat manufacturers offer seasonal optimization programs that recommend changes or make minor, automated adjustments to the thermostat settings to improve energy efficiency. These opt-in programs are now standard offerings for many smart thermostat manufacturers and provided at no additional cost to users. The potential for software updates and continuous optimization and the evolving nature of user interactions mean future energy savings may differ from first-year savings and the energy savings of smart thermostats may need to be evaluated more than once. Second, smart thermostats often have small unit energy savings relative to a home's total energy consumption, especially in comparison to whole- home retrofit programs. This can make it difficult to detect the smart thermostat savings in billing or advanced metering infrastructure (AMI) meter consumption data. For example, as cooling loads in many regions average about 20% of annual electricity consumption, smart thermostat savings of 10% of cooling energy use would equate to a 2% reduction in home electricity consumption. Evaluators should use regression analysis of whole-home billing consumption or advanced metering infrastructure (AMI) meter consumption data to evaluate smart thermostat savings because, as explained at greater length below , these data are usually available to evaluators and regression can control for the impacts of weather and other potentially confounding factors on a home's energy consumption. Finally, as with other energy efficiency programs, participation in smart thermostat programs is self-selective. As discussed at greater length below , smart thermostat participants tend to be, among other things, younger, higher-income, and more likely to adopt electric vehicles (EVs) and internet connected devices than nonparticipants. These differences are often unobservable to the evaluator and correlated with a home's energy consumption, creating the potential for bias in estimating savings. Due to the small unit savings of thermostats, errors and biases from self-selection that may not be very consequential when evaluating a whole- home retrofits (e.g., ±2% of home electricity consumption) can have a major impact when evaluating the savings and cost-effectiveness of smart thermostat programs. A percentage point change in the estimated savings could affect the cost-effectiveness of a program. This means it is important for evaluators to assess and to minimize the potential for error from selection bias in estimating smart thermostat program savings. The Uniform Methods Project provides model protocols for determining energy savings and demand reductions that result from specific energy efficiency measures implemented through state and utility programs. In most cases, the measure protocols are based on a particular option identified by the International Performance Verification and Measurement Protocol ; however, this work provides a more detailed approach to implementing that option. Each chapter is written by technical experts in collaboration with their peers, reviewed by industry experts, and subject to public review and comment. The UMP protocols can be used by utilities, program administrators, public utility commissions, evaluators, and other stakeholders for both program planning and evaluation.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗