Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “software quality, testing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6

True random number generation using the spin crossover in LaCoO 3

While digital computers rely on software-generated pseudo-random number generators, hardware-based true random number generators (TRNGs), which employ the natural physics of the underlying hardware, provide true stochasticity, and power and area efficiency. Research into TRNGs has extensively relied on the unpredictability in phase transitions, but such phase transitions are difficult to control given their often abrupt and narrow parameter ranges (e.g., occurring in a small temperature window). Here we demonstrate a TRNG based on self-oscillations in LaCoO 3 that is electrically biased within its spin crossover regime. The LaCoO 3 TRNG passes all standard tests of true stochasticity and uses only half the number of components compared to prior TRNGs. Assisted by phase field modeling, we show how spin crossovers are fundamentally better in producing true stochasticity compared to traditional phase transitions. As a validation, by probabilistically solving the NP-hard max-cut problem in a memristor crossbar array using our TRNG as a source of the required stochasticity, we demonstrate solution quality exceeding that using software-generated randomness.

97 MATHEMATICS AND COMPUTING↗

A Review of Candidates for a Validation Data Set for High-Assay Low-Enrichment Uranium Fuels

Many advanced reactor concept designs rely on high-assay low-enriched uranium (HALEU) fuel, enriched up to approximately 19.75% 235 U by weight. Efforts are underway by the US government to increase HALEU production in the United States to meet anticipated needs. However, very few data exist for validation of computational models that include HALEU, beyond a few fresh fuel benchmark specifications in the International Reactor Physics Experiment Evaluation Project. Nevertheless, there are other data with potential value available for developing into quality benchmarks for use in data- and software-validation efforts. This paper reviews the available evaluated HALEU fuel benchmarks and some of the potentially relevant benchmarks for fresh highly enriched uranium. It then introduces experimental data for HALEU fuel irradiated at Idaho National Laboratory, from relatively recent irradiation programs at the Advanced Test Reactor. Such data should be evaluated and, if valuable, collected into detailed benchmark specifications to meet the needs of HALEU-based reactor designers.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

LLM Benchmarking with LLaMA2: Evaluating Code Development Performance Across Multiple Programming Languages

The rapid evolution of large language models (LLMs) has opened new possibilities for automating various tasks in software development. This paper evaluates the capabilities of the LLaMA 2-70B model in automating these tasks for scientific applications written in commonly used programming languages. Using representative test problems, we assess the model's capacity to generate code, documentation, and unit tests, as well as its ability to translate existing code between commonly used programming languages. Our comprehensive analysis evaluates the compilation, runtime behavior, and correctness of the generated and translated code. Additionally, we assess the quality of automatically generated code, documentation, and unit tests. Here, our results indicate that while LLaMA 2-70B frequently generates syntactically correct and functional code for simpler numerical tasks, it encounters substantial difficulties with more complex, parallelized, or distributed computations, requiring considerable manual corrections. We identify key limitations and suggest areas for future improvements to better leverage AI-driven automation in scientific computing workflows.

97 MATHEMATICS AND COMPUTING↗

Image quality indicator and dechromator setup for static LANSCE proton radiography experiment

This document describes a static geometry consisting of a sphere of gold and Ti-6Al-4V alloy, along with an energy-loss matching dechromator composed of polycarbonate, to field with the X3 electromagnetic magnifying lens at LANSCE during the next beam cycle. This static configuration will allow us to: (1) characterize the system’s resolution at the material interface between the gold and titanium alloy, and test the image quality for fine features associated with the screws used to fasten the titanium alloy hemispheres; (2) tune our models of proton scattering and imaging for these materials; and (3) test a procedure to align the polycarbonate dechromator and the sphere within the proton beamline. The data will also be suitable for validation of the software options available for proton radiography simulations.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

TDCOSMO. X. Automated modeling of nine strongly lensed quasars and comparison between lens-modeling software

When strong gravitational lenses are to be used as an astrophysical or cosmological probe, models of their mass distributions are often needed. We present a new, time-efficient automation code for the uniform modeling of strongly lensed quasars with GLEE, a lens-modeling software for multiband data. By using the observed positions of the lensed quasars and the spatially extended surface brightness distribution of the host galaxy of the lensed quasar, we obtain a model of the mass distribution of the lens galaxy. We applied this uniform modeling pipeline to a sample of nine strongly lensed quasars for which images were obtained with the Wide Field Camera 3 of the Hubble Space Telescope. The models show well-reconstructed light components and a good alignment between mass and light centroids in most cases. We find that the automated modeling code significantly reduces the input time during the modeling process for the user. The time for preparing the required input files is reduced by a factor of 3 from ~3 h to about one hour. The active input time during the modeling process for the user is reduced by a factor of 10 from ~ 10 h to about one hour per lens system. This automated uniform modeling pipeline can efficiently produce uniform models of extensive lens-system samples that can be used for further cosmological analysis. A blind test that compared our results with those of an independent automated modeling pipeline based on the modeling software Lenstronomy revealed important lessons. Quantities such as Einstein radius, astrometry, mass flattening, and position angle are generally robustly determined. Other quantities, such as the radial slope of the mass density profile and predicted time delays, depend crucially on the quality of the data and on the accuracy with which the point spread function is reconstructed. Better data and/or a more detailed analysis are necessary to elevate our automated models to cosmography grade. Nevertheless, our pipeline enables the quick selection of lenses for follow-up and further modeling, which significantly speeds up the construction of cosmography-grade models. This important step forward will help us to take advantage of the increase in the number of lenses that is expected in the coming decade, which is an increase of several orders of magnitude.

79 ASTRONOMY AND ASTROPHYSICS↗

Towards Integrating Data Quality Assessments and Radiometer Uncertainty for Determining the Expanded Uncertainty of Three-Component Solar Radiation Measurements

Accurate solar irradiance data are fundamental for determining the design and performance characteristics of photovoltaic systems. The uncertainty of solar irradiance measurements depends on many factors including radiometer design, calibration, installation, maintenance, and operational environment. The key contributors to this uncertainty can be classified as the measurement uncertainty of a particular radiometer and the operational uncertainty determined for the time of measurement. Radiometer measurement uncertainty estimates (U R ) can be based on well-established methods used as part of the radiometer calibration process. Estimates of operational uncertainties (U o ) require consideration of additional site-specific factors that affect data quality. A method is needed for establishing the accuracy of solar irradiance data by integrating an existing data quality process and measurement uncertainty estimates for specific radiometers. An algorithm has been developed to integrate data quality analyses and measurement uncertainty estimates for three-component solar irradiance data: global horizontal (total hemispheric) irradiance, direct normal (beam) irradiance, and diffuse horizontal (sky) irradiance collected at one- to 60-minute intervals. The algorithm has been tested using one-minute irradiance measurements. The goal of the project is to distribute a user-friendly software package based on the new algorithm.

data integrity↗

MaPSA Quality Control and AI-Enhanced Grading For the CMS Phase-II Tracker Upgrade

The Compact Muon Solenoid (CMS) experiment will undergo changes as part of the Large Hadron Collider upgrade. The CMS tracker will be upgraded to cope with the new radiation environment and to provide tracking at the first level trigger. This upgrade features a new type of silicon module called PS Module, which combines a Pixel sensor and a Strip sensor in the same module. The pixel portion of the PS module has a sensor bump bonded to 16 Macro Pixel ASICs (MPA) to form a Macro Pixel Sub Assembly (MaPSA). At Fermilab, MaPSAs are tested for quality control before being assembled with the strip sensors, readout and service electronics to form a PS Module. All of this test data is stored in a centralized database, and is used to grade the final module to determine if it will be installed in the detector. The Phase II Outer Tracker Analyzer of Test Outputs (POTATO) is the software that processes this data and determines the module grades. Using recent technologies, an AI agent is being im plemented into POTATO in order to allow users to more efficiently sort through the large amounts of analysis data and ensure that only the user specified data is being considered. This poster will display the process of testing a MaPSA, how that test data is relevant to module assembly and grading, and how the POTATO grading tool is being improved with the use of an embedded AI agent.

Gzamouranis, Olivia [Purdue U.]↗

A microchip microcontroller-based transducer controller for non-contact scanning probe microscopy with phase-locked loop, amplitude, and Q control

An inexpensive yet versatile transducer controller for non-contact scanning probe microscopy (SPM) based on a PIC32 microcontroller from Microchip Technology, Inc is described. In addition to feedback control using the amplitude or phase of the signal from the non-contact transducer, the controller includes a phase-locked loop for frequency-shift feedback, as well as fixed-amplitude, quality factor (Q) control, and self-excitation modes. Apart from the input amplifiers, output buffers, and the Q-control circuit, all other functions of the controller are instantiated in software on the microchip, enabling rapid changes in operating parameters if needed. The controller communicates with a host personal computer via a simple serial connection. The controller has been tested with a quartz tuning-fork transducer but can be used with any oscillating non-contact transducer.

47 OTHER INSTRUMENTATION↗

Automating the Analysis of Large Language Models Responses through Zero-Shot Question Answering

Recent advancements in Large Language Models (LLMs) have shown significant potential in various applications, yet their evaluation, particularly in zero-shot question answering scenarios, remains a challenging task. In this study, our objective was to explore precision metrics for Large Language Models (LLM) and design and implement a software pipeline to automatically evaluate LLMs' outputs under zero-shot question answering. Zero-shot question answering involves a model providing answers to questions about topics it hasn't seen during training. It leverages the principles of zero-shot learning by relying on semantic understanding and generalization from related knowledge. The data used was metadata from medical databases on congenital heart disease. We explored eleven LLM metrics and selected three for our evaluation: BLEU, BERTScore, and MoverScore. BLEU calculates a score based on the overlap of n-grams (contiguous sequences of n items, typically words) between the machine-generated translation and the reference translations. Higher BLEU scores indicate better correspondence between the machine-generated and human-generated translations. BERTScore is a metric used to evaluate the quality of machine-generated text by measuring the similarity of token embeddings produced by BERT (Bidirectional Encoder Representations from Transformers) between the generated text and reference text. MoverScore is a metric that quantifies the dissimilarity between the distributions of word embeddings from machine-generated text and reference text, emphasizing semantic similarity over exact token overlap. We also introduced HBKI, a composite metric summarizing these approaches. We tested five models —GPT-3, Llama-2, Gemini 1.5 Pro, Solar 10.7B, and Mixtral-8x7b. Our software pipeline, designed and implemented using Object-Oriented Programming principles, allows users to customize the selection and extraction of features for topics of interest in their own research. Our results show that MoverScore delivered the most precise evaluation of the LLM's outputs, while Mixtral-8x7b achieved the best overall performance in extracting metadata from the databases.

97 MATHEMATICS AND COMPUTING↗

SAM Software Quality Assurance Plan Implementation and NQA-1 Assessment

The System Analysis Module (SAM) is an advanced and modern system analysis tool being developed at Argonne National Laboratory under the U.S. DOE Office of Nuclear Energy’s Nuclear Energy Advanced Modeling and Simulation (NEAMS) program. As a modern-day software, SAM development included efforts to follow best practices in software development. These best practices include version control using git, independent reviews of development activities, and detailed descriptions of developments and bug fixes using the GitLab issue and merge request system. In Fiscal Year 2023 (FY23), the SAM development team set out to formalize a Software Quality Assurance (SQA) program that allowed industry partners to credit the informal steps being taken by the SAM development team to ensure the quality of the software. As part of formalizing an SQA program, a SQA Plan (SQAP) was developed, implemented, and assessed. The SAM SQAP targets compliance with NQA-1-2008/2009 Addenda. The SQAP builds on the MOOSE SQAP and the Argonne SSQAPP while accounting for the needs of the multi-organization SAM development team. Modifications to the SAM repository structure, including a new testing system, updated test cases, and the development of an internal website, facilitate the implementation of the SAM SQAP. The initial assessment of the SAM SQAP indicated that the SAM SQAP was adequately and effectively implemented and additional work was required to improve the compliance of the SAM SQAP with NQA-1 2008/2009 Addenda. Several improvements to the SQAP have already been drafted to address these assessments and future work is planned to further improve the SAM SQAP in support of end-user needs.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Evaluating User Errors and Temporal Trends in Marine Fish Communities Using 360-Degree Underwater Photography

The use of environmental DNA (eDNA) sampling has been proposed as a complementary method to monitor fish species in marine environments, offering a non-invasive and potentially more efficient approach to marine species observations. eDNA monitoring could be especially useful in and around sites targeted for marine energy generation as these regions need regular monitoring that would be impractical with traditional techniques. Before we can fully rely upon eDNA, we must first verify its accuracy against other proven methods, such as the use of underwater photography. In this study, I deployed a 360-degree camera in the tidal channel of Sequim Bay once a month during several hours overlapping slack tide. I investigated how having multiple people identify and count fish on underwater images could affect the overall results. Using chi square tests in R, I compared my fish identifications and counts to those made by another intern on the same images recorded in August. I found significant differences in the number of species identified and the total individual counts between the two different datasets. I also tested the statistical differences in both Shannon diversity and Pielou evenness indices between the August, September, and November camera deployments using a Hutcheson t-test. Only one significant difference was found in the Shannon index comparisons, and none were found between the Pielou evenness comparisons. These findings show that if multiple identifiers are used to process underwater images, quality control checks must be made to reduce the potential for error. This also points toward the possibility to leverage more advanced image analysis processes, such as automated image analysis software. The findings from this study also show that the dynamics of marine fish communities can vary over a few months; however, further analysis is needed to determine the extent of the seasonal changes in Sequim Bay.

59 BASIC BIOLOGICAL SCIENCES↗

Monte Carlo N-Particle Transport Performance of Predicting Digital Radiographic IQI Inspection

The identification of porosity, geometric noncompliance, and other defect types are critical to the qualification of materials and components. X-ray radiographic nondestructive testing is a common industrial inspection method for process quality control and component qualification and certification. Digital radiography provides a quick and efficient alternative when compared to traditional film-based inspection. The quality of radiographic inspection is dependent on equipment specifications, such as the source spot size and detector pixel size, and the specific parameters selected for use for the radiographic technique. To evaluate if an x-ray system and technique is sufficient for a given requirement, a radiographic image quality indicator (IQI) can be used. Radiographic IQIs in hard to machine materials or hard to manufacture defects can be time consuming and expensive to manufacture. This study was conducted to evaluate current Savannah River National Laboratory (SRNL) x-ray imaging systems with a custom tantalum IQI and using Monte Carlo simulations to predict the performance of future systems. The tantalum IQI was tested using a Siefert Isovolt 420 keV x-ray tube with a Perkin Elmer XRD 1611 flat panel with 100-micron pixels. Using the Monte Carlo N-Particle transport software, the radiographic tally was used to simulate the photon flux through an identical tantalum IQI. These simulations provided a benchmark as to the best theoretical identification on a given system using our tantalum IQI. The simulations were refined to match SRNL’s current systems’ noise levels, leading to confidence in their ability to predict the performance of other systems that may be purchased and deployed in the future at the Savannah River Site. Future studies will be conducted to prove this research can be extended to artificially evaluate the ability for systems to identify critical defect sizes through x-ray radiographic inspection, drastically reducing the cost and time burdens of producing high-fidelity radiographic test articles.

digital X-ray radiography↗

MOOSE: A Modular Platform for Fission and Fusion Multiphysics

The Multiphysics Object-Oriented Simulation Environment (MOOSE) Framework, as well as MOOSE-based simulation tools, have accelerated the development of fission energy and advanced reactor technologies through the United States Department of Energy, Office of Nuclear Science, Nuclear Energy Advanced Modeling & Simulation (NEAMS) Program. MOOSE contains a complete platform of multiphysics simulation capabilities, capable of running on massively parallel systems, and is developed in an open-source manner with great attention paid to high-quality software quality assurance practices. This overall approach could greatly benefit the fusion energy community, which requires rapid design iteration and improvement in order to facilitate the successful development of fusion as an alternative energy source to fossil fuels. In the first half of this talk, applications of MOOSE and MOOSE-based tools for advanced reactor designs will be showcased, as well as MOOSE ecosystem infrastructure (such as the NEAMS Virtual Test Bed) that enables and accelerates fission reactor design. In the second half, a discussion of how the MOOSE approach to modeling and simulation is currently being applied internationally in fusion energy research and development at the United Kingdom Atomic Energy Authority will be discussed, and ongoing/future domestic research efforts will be highlighted.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

Improvements to In-Canal Assay of High Specific Activity 60Co at the ATR

Improvements to the quality of measurements in the assaying of High Specific Activity (HSA) cobalt 60 produced at the Advanced Test Reactor at the Idaho National Laboratory are presented. Such assays are conducted regularly between cycles at the ATR, and due to the radiological safety concerns, they are performed within the ATR canal. A standard ion chamber detector was placed in a collimated assay tube within the canal. Another tube, parallel to the detector, was used to position various cobalt 60 standards and targets. The collimated port between these tubes allows for the targets to be pulled across via a winch system. Previous measurements were performed at discrete 1.5-inch steps using a hand-cranked winch system. Charge was collected and integrated for 10 seconds at each discrete step during a scan. Two substantial improvements to the assay include an electronically controlled, motor-driven winch system and a software controlled continuous charge collection. Combined, these changes improved the quality of measurements as well as streamline the assay process. Further examination of these charge profiles indicated a reasonable amount of photon scattering, or shine, before a line-of-sight is established. This created inaccuracies in the reported activity of the cobalt targets. A series of MCNP-6.3 calculations were performed in an effort to quantify and correct for this shine effect. The results of the calculations and how they were employed to a create shine correction factor for the in-canal cobalt assay at the ATR are presented. The overall bias showed an overestimation of the HSA cobalt 60 activity by around 7-8%.

07 - ISOTOPES AND RADIATION SOURCES↗

Marine Algae Industrialization Consortium (MAGIC): Combining biofuel and high-value bioproducts to meet the RFS

The Marine Algae Industrialization Consortium (MAGIC) was formed to address pressing challenges in the commercialization of microalgae as a source of biofuel. The “Marine Algae Industrialization Consortium (MAGIC): Combining biofuel and high-value bioproducts to meet the RFS” project formally addressed two US Department of Energy Bioenergy Technologies Office (BETO) goals: (1) Model the sustainable supply of 1 million metric tonnes ash free dry weight (AFDW) cultivated algal biomass and (2) Demonstrate valuable co-products produced along with biofuel intermediates to increase value of algal biomass by 30%. To achieve these goals, the project demonstrated and validated high-value co-products to drive down the cost of biofuel by increasing the value of algae “co-products” towards increasing the selling price of total algae biomass as one of the key drivers of economics and adoption. This was accomplished through five core, interdependent tasks including: (1) strain selection to identify and deliver strains for mass culture, (2) mass culture using a hybrid cultivation system and following key operating parameters for downstream applications to provide algae feedstock, (3) recovery and conversion to evaluate two alternative methods to separate dry algae biomass into oil and residuals for downstream testing, (4) product assessment to determine biofuel, aquafeed or poultry feed product efficacy using algae biomass fractions as well as to provide critical performance data for valuation and (5) commercialization to use technoeconomic and life cycle assessments (TEA/LCA) as iterative design and assessment tools including consideration of target markets, competitors, and distribution channels to guide product assessment, development and valuation. A total of 46 peer-review publications, many open-access, provide detail of much of the work carried out and the results of the tasks. Additional reports and presentations provide other technical and public engagement material. At a high level, using a variety of approaches, more than 1000 marine microalgae strains were evaluated to ultimately identify the seven winners that were down-selected to be grown in mass culture. Strain selection demonstrated that there were no ‘super strains’ and that each candidate had strengths and limitations for specific products, growth conditions or operational considerations. Mass culture growth of these seven strains at >5000 L / 29 m 2 scale found that four them were suitable for product assessment. More than 250 kg of biomass was produced across hundreds of pond runs along with thousands of cultivation entries on the growth and biomass characteristics as well as environmental parameters. In the process, dozens of standard operating procedures were generated as was custom software to process and analyze cultivation data. Recovery and conversion of algae biomass demonstrated that a hexane solvent based extraction protocol was most effective at recovering oil (biocrude) from algae and four strains were processed to produce oil and lipid extracted algae (residuals) for downstream testing. Membrane-based oil separation was less successful, but may still be applicable to other commercial applications in the future. Product testing demonstrated that algae biocrude is of high quality and hydrotreating generated numerous fractions of high quality composition for fuel and lubricate based applications. Aquafeed studies performed at a variety of scales showed that both whole and defatted (lipid extracted algae) microalgae were suitable as a feed ingredient, but that the specifics of the fed animal and biochemical composition of the algae are critical factors when determining formulation. Similarly, poultry studies on whole and defatted microalgae generally showed positive outcomes on animal growth and health, with some microalgae providing enhanced nutritional composition of the animal product. Economic and life cycle assessments covered a wide range of possible commercialization and sustainability scenarios. Replacement value, improved product value added, consumer values marketing added valuation and improved animal health were considered as alternatives for microalgae valuation. Using the open pond system, algae productivity was identified as the key driver of commercialization economics, but combination of co-products (e.g. animal feed) with biofuel production substantially increased the total selling price of algae. Modeled microalgae selling price exceeded $\$$1500/tonne and could generate competitive biofuel selling prices below $\$$5 gallon gas equivalents using realistic algal productivities. Short (process scale) and longer (decadal trends) sustainability assessments show that marine microalgae can enhance the sustainability of energy production and lead to other realized benefits in water, fertilizer and land use for other sectors (e.g. agriculture). This project successfully demonstrated all of the components of an end-to-end process from mass microalgae cultivation and dewatering, to recovery and conversion of algae biomass components, to final product demonstration and process valuation; the combined results provide a framework for future commercialization of algae based biofuels.

09 BIOMASS FUELS↗

Automated segmentation and analysis of point clouds of pier foundations using Pier Inspection and Evaluation Report (PIER)

Pier foundations are commonly used in locations with unstable soil or where other types of foundations are unsuitable or cost prohibitive. A pier foundation consists of vertical columns to support the structure and elevate it above the ground. Common materials for pier foundations include masonry, concrete, timber, and steel. The methods for accurate placement of pier foundations have remained relatively unchanged for decades. For simple installations, construction chalk lines are used to layout the locations of piers to ensure accurate placement and elevation. For more complex installations, surveying instruments operated by trained professionals are employed to accurately locate piers and assess correct elevation before construction. After installation, another survey may need to be performed to assess the quality of the as-built foundation. However, with the advent of terrestrial laser scanners (TLS), the means now exist for contractors to conduct their own assessments of as-built foundations. The major barrier preventing contractors from performing their own assessments of as-built foundation quality is the segmentation and analysis of point cloud data, a skill that often requires a trained user. The objective of this research is to develop a software tool (PIER: Pier Inspection and Evaluation Report) to enable automated segmentation and analysis of point clouds of pier foundations. In this paper, the automated segmentation and analysis algorithms are detailed. A mockup lay out of pier foundations was built using concrete masonry units, and the algorithms were tested to evaluate performance. Limitations of the current algorithms and future research direction are discussed.

Turki, Amine [ORNL]↗

Maximizing Efficiency and Quality: Leveraging Automated Testing for Laboratory Commissioning

The traditional commissioning process uses sampling to select equipment for functional acceptance testing when large quantities of equipment are present. Although this approach is generally effective in identifying wide-spread issues, it has several shortcomings: it fails to evaluate equipment not included in the sample, provides only a one-time validation of equipment operation, and the standard documentation is a simple checklist of pass/fail questions. During the construction and commissioning process of the new Research and Innovation Laboratory (RAIL) in Golden, CO, the National Renewable Energy Laboratory team engaged Group14 Engineering to implement a Connected Commissioning process using fault detection and diagnostic software for automated functional acceptance testing. This presentation highlights the advantages offered by automated functional testing in this critical laboratory setting: (1) sampling 100% of BAS-connected equipment during functional testing, (2) testing results backed by data beyond the traditional pass/fail checklist, and (3) an automated test process that can be regularly executed by the building management team for ongoing commissioning throughout the life of the building. The presentation will also cover technical challenges associated with Connected Commissioning and the important conversations with key stakeholders that need to occur well before functional acceptance testing in order to successfully implement the automated testing processes.

automated testing↗

Verification and Validation of the Stakeholder Tool for Assessing Radioactive Transportation (START) – 22323

The U.S. Department of Energy (DOE) Office of Integrated Waste Management is planning for the eventual transportation, storage, and disposal of spent nuclear fuel (SNF) and high-level radioactive waste (HLW) from nuclear power plant and DOE sites. The Stakeholder Tool for Assessing Radioactive Transportation (START) is a web-based, geospatial decision-support tool developed for evaluating routing options and other aspects of transporting SNF and HLW, covering rail, truck, barge, and intermodal infrastructure and operations in the continental United States. The verification and validation (V&V) process is intended to independently assess START to provide confidence in the ability of START to accurately provide intended results. The V&V process checks the START tool using a variety of methods, ranging from independent hand calculations to comparison of START performance and results to those of other codes. The V&V activity was conducted independently from the START development team with opportunities to provide feedback and collaborate throughout the process. The V&V analyzed attributes of transportation routes produced by START, including route distance and both population and population density captured within buffer zones around routes. Population in the buffer zone, population density in the buffer zone, and route distance were all identified as crucial outputs of the START code and were subject to V&V tasks. Some of the improvements identified through the V&V process were standardizing the underlying population data in START, changing the projection of the population raster data, and changes to the methodology used for population density to improve its applicability for expected users. This collaboration also led to suggested improvements to some of the underlying shape file segments within START. A formal series of test routes went through the V&V process from START version 3.2.1 following the implementation of the recommended improvements to START. The V&V of the buffer zone population and distance reported in START were achieved by utilizing a custom workflow developed in QGIS. This platform was chosen because it is fully independent of START. The buffer zone population and route distances in START showed excellent agreement to independent V&V test results. Over 200 route test cases were run; in all cases, the percent difference in the population within the buffer zone was less than +/- 5%, and the majority of cases were below +/- 1%. These cases were also duplicated in ArcMap, which is Environmental Systems Research Institute’s (ESRI) GIS desktop application. The ArcMap results also show good agreement; however, they cannot be considered independent of START because START also uses ESRI software. The V&V tasks are currently addressing population density and estimated radiation dose in START as well as implementing further automation of the V&V process to allow larger suites of test routes to be run more easily and quickly by less experienced GIS users. These improvements to the V&V process will make it a valuable tool for the development team and the stakeholders so that each new iteration of START can easily be checked for quality and consistency. This paper and presentation will describe the V&V process, findings and implications, and ongoing V&V activities.

START, DOE, Transporation, dose, GIS↗