SEARCH · Engineering Papers

Results for “software testing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 829 records · Page 46

All sources NASA NTRS DOE OSTI

DOE OSTI2021

GDSA PFLOTRAN Development (FY2021)

The Spent Fuel & Waste Science and Technology (SFWST) Campaign of the U.S. Department of Energy (DOE) Office of Nuclear Energy (NE), Office of Spent Fuel & Waste Disposition (SFWD) is conducting research and development (R&D) on geologic disposal of spent nuclear fuel (SNF) and high-level nuclear waste (HLW). A high priority for SFWST disposal R&D is to develop a disposal system modeling and analysis capability for evaluating disposal system performance for nuclear waste in geologic media. This report describes fiscal year (FY) 2021 advances of the PFLOTRAN Development group of the SFWST Campaign. The mission of this group is to develop a geologic disposal system modeling capability for nuclear waste that can be used to probabilistically assess the performance of generic disposal concepts. In FY 2021, development proceeded along three main thrusts: software infrastructure, code performance, and process model advancement. Software infrastructure improvements included implementing an Agile software development framework and making improvements to the QA Test Suite. Code performance improvements included development of advanced linear and nonlinear solvers as well as design of flexible smoothing algorithms for capillary pressure functions. Process modeling advancements included the addition of flexible thermal conductivity function definitions and refinement of multi-continuum reactive transport to support Sandia’s participation in DECOVALEX

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

DOE OSTI2025

Control Strategies and Validation in the Hybrid Optimization and Performance Platform (HOPP)

The Hybrid Optimization and Performance Platform (HOPP) is a tool that simulates hybrid power plants in various configurations, and also calculates the financial feasibility of these plants. This report outlines an overview of HOPP and the energy storage dispatch strategies available. It then presents three case studies which demonstrate different applications of HOPP. The first case looks at the profitability of hybrid power plants in different locations in the USA. The second case examines the availability of hybrid power plants to provide energy reliability services. The third case presents a plant that produces both hydrogen and electricity, and demonstrates a dispatch strategy that chooses the most profitable energy vector based on price signals. The next section shows the validation of HOPP on operational data, using data from both unit-scale and utility-scale power plants. This validation process demonstrated that HOPP can simulate the power output of both wind and solar PV plants at both scales with comparable fidelity to an existing commercial software tool. Finally, HOPP is applied in a field test which applies an optimal dispatch strategy to a physical battery in a unit-scale hybrid plant at NREL. HOPP's optimal dispatch strategy, applied in a real-world setting, improved this hybrid plant's ability to meet a load signal while minimizing operational costs.

14 SOLAR ENERGY↗

DOE OSTI2023

PFLOTRAN Development FY2021

The Spent Fuel & Waste Science and Technology (SFWST) Campaign of the U.S. Department of Energy (DOE) Office of Nuclear Energy (NE), Office of Spent Fuel & Waste Disposition (SFWD) is conducting research and development (R&D) on geologic disposal of spent nuclear fuel (SNF) and high-level nuclear waste (HLW). A high priority for SFWST disposal R&D is to develop a disposal system modeling and analysis capability for valuating disposal system performance for nuclear waste in geologic media. This report describes fiscal year (FY) 2021 advances of the PFLOTRAN Development group of the SFWST Campaign. The mission of this group is to develop a geologic disposal system modeling capability for nuclear waste that can be used to probabilistically assess the performance of generic disposal concepts. In FY 2021, development proceeded along three main thrusts: software infrastructure, code performance, and process model advancement. Software infrastructure improvements included implementing an Agile software development framework and making improvements to the QA Test Suite. Code performance improvements included development of advanced linear and nonlinear solvers as well as design of flexible smoothing algorithms for capillary pressure functions. Process modeling advancements included the addition of flexible thermal conductivity function definitions and refinement of multi-continuum reactive transport to support Sandia’s participation in DECOVALEX. This report fulfills the GDSA PFLOTRAN Development Work Package Level 3 Milestone – PFLOTRAN Development, FY2021, M3SF-21SN010304072.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

DOE OSTI2026

Stereo-vision thermal imaging system for tracking flying animals in wind farm areas (CRADA #763) Abstract

CRADA 763: abstract ThermalTracker-3D (TT3D) is a stereo vision thermal imaging system that provides 3D flight information on detected birds, bats, and other flying targets. The system was initially developed for use in the siting and monitoring of offshore wind projects to establish pre-construction and operation collision risk data but can be applied to terrestrial wind energy projects as well as national security monitoring. This technology will reduce monitoring cost, decrease processing time, and provide more accurate data for wind energy developers/operators and regulatory agencies. While the current technology is at a high level of readiness, Technology Readiness Level (TRL) 7, there remain several barriers to commercialization, particularly around ease-of-use, that result in a low Adoption Readiness Level (ARL). The proposed work will advance commercialization readiness by streamlining calibration methods for built systems. This work will:1. 1. develop a software package for factory and dynamic calibration processes 2. test that package with existing prototype TT3D systems, and 3. conduct outreach with industry end-users.

ThermalTracker↗

DOE OSTI2021

Coordinating an operational data distribution network for CMIP6 data

The distribution of data contributed to the Coupled Model Intercomparison Project Phase 6 (CMIP6) is via the Earth System Grid Federation (ESGF). The ESGF is a network of internationally distributed sites that together work as a federated data archive. Data records from climate modelling institutes are published to the ESGF and then shared around the world. It is anticipated that CMIP6 will produce approximately 20 PB of data to be published and distributed via the ESGF. In addition to this large volume of data a number of value-added CMIP6 services are required to interact with the ESGF; for example the citation and errata services both interact with the ESGF but are not a core part of its infrastructure. With a number of interacting services and a large volume of data anticipated for CMIP6, the CMIP Data Node Operations Team (CDNOT) was formed. The CDNOT coordinated and implemented a series of CMIP6 preparation data challenges to test all the interacting components in the ESGF CMIP6 software ecosystem. This ensured that when CMIP6 data were released they could be reliably distributed.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

DOE OSTI2013

A Robust and Flexible Design for GCEP Unattended Online Enrichment Monitoring: An OLEM Collection Node Network

Oak Ridge National Laoratory (ORNL) has engineered an on-line enrichment monitor (OLEM) to continuously measure U-235 emissions from the UF6 gas flowing through a unit header pipe of a gas centrifuge enrichment plant (GCEP) as a component of the International Atomic Energy Agency s (IAEA) new generation of technology to support enrichment plant safeguards1. In contrast to other enrichment monitoring approaches, OLEM calibrates and corrects for the pressure and temperature dependent UF6 gas-density without external radiation sources by using the inherent unit header pipe pressure dynamics and combining U-235 gamma-ray spectrometery using a shielded NaI detector with gas pressure and temperature data near the spectrum measurement point to obtain the enrichment of the gas as a function of time. From a safeguards perspective, OLEM can provide early detection of a GCEP being misused for production of highly enriched uranium, but would not detect directly the isolation and use of a cascade within the production unit to produce HEU. OLEM may also reduce the number of samples collected for destructive assay and, if coupled with load cell monitoring, could support isotope mass balance verification and unattended cylinder verification. The earlier paper presented OLEM as one component along with shared load cells and unattended cylinder verification, in the IAEA emering toolbox for unattended instruments at GCEPs1 and described the OLEM concept and how previous modeling studies and field measurements helped confirm the viability of a passive on-line enrichment monitor for meeting IAEA objectives and to support the development of performance targets. Phase I of the United States Support Program (USSP) OLEM project completed a preliminary hardware, software and communications design; phase II will build and test field prototypes in controlled laboratory settings and then at an operational facility. That paper also discussed many of the OLEM collection node commercial off the shelf (COTS) components and summarized the OLEM collection node data security provisions. This paper will discuss a secure and redundant network of OLEM collection nodes, auxiliary detection units and supporting junction boxes distributed throughout a facility for monitoring enrichment on product, feed and tails unit header pipes; the purpose and capability of the built-in Electronic Optical Sealing System (EOSS) network gateway; and a network approach for obtaining reliable and authenticated pressure measurements.

Younkin, James R↗

DOE OSTI2015

Career Progression of Junior Professional Officers

The U.S. Support Program to IAEA Safeguards (USSP) has funded more than 25 Junior Professional Officer (JPO) positions in the IAEA Department of Safeguards since 2005. JPOs are college graduates with zero to two years’ work experience who work alongside experienced IAEA staff members for one to two years and assist with basic, yet essential work while obtaining valuable experience. They contribute to equipment development, testing, integration, open source information collection and analysis, and software and database development. This paper will study the trends in career progression for the JPOs who have completed assignments with the IAEA in the Department of Safeguards. Brookhaven National Laboratory, in its role in managing the USSP, has compiled information that can be analyzed for this purpose.

98 NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL P↗

DOE OSTI2021

A Predictive Model for Offgas Composition in Waste Glass Melters - Spring 2021 SULI General Audience Abstract

Approximately 56 million gallons of nuclear waste are currently stored in 177 underground storage tanks at the Hanford site in Washington state. A portion of the tank waste is targeted for immobilization by converting the nuclear waste to borosilicate glass through a process called vitrification. During vitrification, emissions are produced in addition to the intended glass product. These emissions, contained in a volume above the molten glass called the plenum, are composed of water vapor, air components, aerosols, particulates, and other gaseous species that result from evaporation, reactions, and air influx. The formation of these gasses causes the surface of the glass to foam. To reduce foaming, sucrose is added to the waste stream before the conversion to glass. Sucrose also reduces the formation of hazardous gasses in the plenum including nitrous oxides, NOx, and carboxides, COx. However, if excess sucrose is present, the reactions that produce the gasses in the plenum will be incomplete, and increased levels of NOx and COx can be observed. The Waste Treatment and Immobilization Plant at the Hanford site is equipped with the technology to treat the plenum emissions. Though many tests have been run to determine the presence of particulates in the plenum emissions, there is does not exist an established method to confidently predict the composition of the gaseous species in the plenum prior to treatment without physical testing. A model was created in the process modeling software Aspen Plus to predict the composition of NOx and COx in the plenum. The model employs a constant stirred tank reactor and user-defined reaction chemistry to determine the plenum composition. To verify the model, the offgas predicted by the model were compared to available offgas data for ten different feed compositions. It is demonstrated that the developed Aspen Plus model is capable of predicting the composition of NOx and COx in the plenum.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

DOE OSTI2025

Node-red Software Architecture For Soec System Control

The software workflows that have been developed are open-source and therefore can be accessed and utilized by researchers and students all around the world for free of cost. The software is flexible and easily modifiable to suit the needs for testing bench-scale to system-level setups, and is not limited to just hydrogen generation facilities. The web-interface of the code allows for easy remote access and management, which is especially useful for distributed systems. This is more challenging with traditional PLCs, which often require direct connections or specialized software, making remote troubleshooting more cumbersome. Additionally, since the underlying interface is open-source, with an active community contributing to its development, regular updates, new features, and a wealth of community support and shared solutions will keep improving the overall architecture without excessive fees for upgrades.

Shigrekar, Amey [Idaho National Laboratory (INL), ↗

DOE OSTI2020

Deep Learning for Automated Identification of Eels in Sonar Data

Freshwater eels, such as the American eel (Anguilla rostrata) present numerous challenges related to safe downstream fish passage at hydroelectric facilities. One of those challenges is effective monitoring of their abundance, movements, and behavior to facilitate design and operation of eel protection and passage facilities. A previous EPRI study documented the ability of human analysts to reliably identify American eels in data obtained with a 1100/1800 kHz, multibeam sonar. This report describes a project to develop deep learning (a subset of artificial intelligence) tools to automate the time-consuming, subjective process of eel identification in multibeam sonar data. The project exploited new data collected in the laboratory and the existing data from the prior EPRI field study to develop and test deep learning and other data analytic tools, including wavelet filtering, differencing for static object removal, and convolutional neural network analysis. The analysis of the laboratory data demonstrated feasibility of the approach, revealed object characteristics observed with the sonar that distinguish eels from similarly sized and shaped acoustic targets, and provided additional data for algorithm selection and training. Deep learning algorithms trained and tested on the laboratory data alone achieved accuracy rates of greater than 98% when classifying acoustic images of eels and similar-sized neutrally buoyant sticks. The algorithm trained and tested on the pre-existing field data alone, and yielded classification accuracy of 9.3% false positives and 13.3% false negatives when distinguishing between eels and sticks/PVC pipes based on video clips (i.e., multiple, consecutive images). This performance is comparable to the classification accuracy achieved by human analysts in the prior study. The deep learning algorithm trained on a combination of video clips obtained in the laboratory and the field and tested on video clips from the field, was able to distinguish eels from sticks and PVC pipes (a river debris analog) of similar size with 100% accuracy. Outreach to the hardware, software, and end-user communities early in the project helped to identify needs and specify the application space. Outreach to those communities at the end of the project communicated project results and opportunities for further development. The project achieved proof of concept for automated identification of eel in multibeam sonar data. Future work should focus on acquisition of additional data for more robust algorithm training and testing; modification of the software tools to accommodate multiple acoustic targets in the acoustic field at a given time; identification of additional object classes; incorporation of motion in the object identification and classification algorithms; operationalizing the software tools, including integration with other existing sonar data analysis tools; and partnering with hardware and software providers for distribution of the software tools with their commercial products.

13 HYDRO ENERGY↗

DOE OSTI2015

IBEX: An open infrastructure software platform to facilitate collaborative work in radiomics

Purpose: Radiomics, which is the high-throughput extraction and analysis of quantitative image features, has been shown to have considerable potential to quantify the tumor phenotype. However, at present, a lack of software infrastructure has impeded the development of radiomics and its applications. Therefore, the authors developed the imaging biomarker explorer (IBEX), an open infrastructure software platform that flexibly supports common radiomics workflow tasks such as multimodality image data import and review, development of feature extraction algorithms, model validation, and consistent data sharing among multiple institutions. Methods: The IBEX software package was developed using the MATLAB and C/C++ programming languages. The software architecture deploys the modern model-view-controller, unit testing, and function handle programming concepts to isolate each quantitative imaging analysis task, to validate if their relevant data and algorithms are fit for use, and to plug in new modules. On one hand, IBEX is self-contained and ready to use: it has implemented common data importers, common image filters, and common feature extraction algorithms. On the other hand, IBEX provides an integrated development environment on top of MATLAB and C/C++, so users are not limited to its built-in functions. In the IBEX developer studio, users can plug in, debug, and test new algorithms, extending IBEX’s functionality. IBEX also supports quality assurance for data and feature algorithms: image data, regions of interest, and feature algorithm-related data can be reviewed, validated, and/or modified. More importantly, two key elements in collaborative workflows, the consistency of data sharing and the reproducibility of calculation result, are embedded in the IBEX workflow: image data, feature algorithms, and model validation including newly developed ones from different users can be easily and consistently shared so that results can be more easily reproduced between institutions. Results: Researchers with a variety of technical skill levels, including radiation oncologists, physicists, and computer scientists, have found the IBEX software to be intuitive, powerful, and easy to use. IBEX can be run at any computer with the windows operating system and 1GB RAM. The authors fully validated the implementation of all importers, preprocessing algorithms, and feature extraction algorithms. Windows version 1.0 beta of stand-alone IBEX and IBEX’s source code can be downloaded. Conclusions: The authors successfully implemented IBEX, an open infrastructure software platform that streamlines common radiomics workflow tasks. Its transparency, flexibility, and portability can greatly accelerate the pace of radiomics research and pave the way toward successful clinical translation.

60 APPLIED LIFE SCIENCES↗

DOE OSTI2014

Initial experience of ArcCHECK and 3DVH software for RapidArc treatment plan verification

The purpose of this study was to perform delivery quality assurance with ArcCHECK and 3DVH system (Sun Nuclear, FL) and to evaluate the suitability of this system for volumetric-modulated arc therapy (VMAT) (RapidArc [RA]) verification. This software calculates the delivered dose distributions in patients by perturbing the calculated dose using errors detected in fluence or planar dose measurements. The device is tested to correlate the gamma passing rate (%GP) and the composite dose predicted by 3DVH software. A total of 28 patients with prostate cancer who were treated with RA were analyzed. RA treatments were delivered to a diode array phantom (ArcCHECK), which was used to create a planned dose perturbation (PDP) file. The 3DVH analysis used the dose differences derived from comparing the measured dose with the treatment planning system (TPS)-calculated doses to perturb the initial TPS-calculated dose. The 3DVH then overlays the resultant dose on the patient's structures using the resultant “PDP” beams. Measured dose distributions were compared with the calculated ones using the gamma index (GI) method by applying the global (Van Dyk) normalization and acceptance criteria, i.e., 3%/3 mm. Paired differences tests were used to estimate statistical significance of the differences between the composite dose calculated using 3DVH and %GP. Also, statistical correlation by means of logistic regression analysis has been analyzed. Dose-volume histogram (DVH) analysis for patient plans revealed small differences between treatment plan calculations and 3DVH results for organ at risk (OAR), whereas planning target volume (PTV) of the measured plan was systematically higher than that predicted by the TPS. The t-test results between the planned and the estimated DVH values showed that mean values were incomparable (p < 0.05). The quality assurance (QA) gamma analysis 3%/3 mm showed that in all cases there were only weak-to-moderate correlations (Pearson r: 0.12 to 0.74). Moreover, clinically relevant differences increased with increasing QA passing rate, indicating that some of the largest dose differences occurred in the cases of high QA passing rates, which may be called “false negatives.” The clinical importance of any disagreement between the measured and the calculated dose is often difficult to interpret; however, beam errors (either in delivery or in TPS calculation) can affect the effectiveness of the patient dose. Further research is needed to determinate the role of a PDP-type algorithm to accurately estimate patient dose effect.

61 RADIATION PROTECTION AND DOSIMETRY↗

DOE OSTI2022

Thermal Modelling of Advanced Test Reactor Fuel in a Generalized Dry Storage System

Star-CCM+, a computational fluid dynamics (CFD) software was used to conduct modeling and simulation of the thermal performance of a dry storage configuration consisting of Department of Energy Standardized Canisters (DOESCs) loaded with aluminum-clad spent nuclear fuel (ASNF.) The configuration includes nine DOESCs loaded with Advanced Test Reactor ASNF contained within a stainless steel overcanister centered in a ventilated, concrete overpack. The simulations were used to estimate the maximum temperatures reached by backfill gases inside the overcanister and DOESCs.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

DOE OSTI2016

Software Dependability for Digital Instrumentation and Control Systems in Nuclear Power Plants - Lessons Learnt from a DOE Sponsored Project

This paper discusses the lessons learnt from a sponsored study undertaken under the sponsorship of the Department of Energy by a team of scientists at the Ohio State University in collaboration with Technology Resources and experts around the world. This paper makes recommendations for future research and studies based on these lessons learnt. The purpose of the research was to provide a software dependability quantification framework that would allow its evaluation along the software development life cycle for safety critical instrumentation and control systems (I and C) such as those found in nuclear power plants. A typical example of safety critical system is the reactor protection system. The research funded by the Department of Energy built on preliminary research carried out for the US NRC that had focused solely on software reliability quantification. Quantification of software dependability is important for safety critical I and C because the regulatory framework in use in the United States for nuclear power plants is risk informed. Quantification of risk requires quantification of all contributors to risk and those include instrumentation and control systems. The purpose of the study was to help developers of software-based I and C with an early evaluation of the potential risk an application would represent at different stages of the software development life cycle by building gates at the end of each life cycle phase. Life cycle phases of software development comprehend phases such as requirements, design, coding and test. The gates' function would be to stop development from moving forward if the assessed dependability objective for that phase were not met. Gates thresholds were to be defined based on measurements collected during the life cycle phase under evaluation and phases that preceded it and an understanding of the mechanisms that would lead to threatening the dependability objectives. The information contained in the gates would in addition help steer the development towards areas that needed improvement. The following are four high level conclusions to our study. First the study demonstrates a lack of consensus on mechanisms that will threaten the dependability attributes. This lack of consensus should be resolved. The lack of consensus appears for all measures to the exception of reliability. Second, measures should be developed for software availability and safety. Third, for all dependability attributes (with the exception of reliability) consistency of the measurement threads should evaluated and measurement-based development gates derived. Finally our ability to perform such type of research depends in a significant manner on access to vendor and/or utility data. The licensing environment and the first-of-a-kind nature of current software-based applications for nuclear I and C make it difficult to collect the relevant data. An approach which would either increase access to data by providing 'immunity' to the vendor or utility or the development of a software-based I and C for the sole purpose of data collection for research should be investigated. (authors)

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

DOE OSTI2023

Improvements to Contributions from Neutron Inelastic Scattering for Next-Event Estimators in MCNP ® Software

An improvement to the handling of contributions from neutron inelastic scattering to next-event estimators has been implemented in the MCNP6 ® software for release with version 6.3.1. The kinematic equations that govern the outgoing energy of inelastic neutron scattering contributions to next-event estimators have two roots. Historically, the implementation in the MCNP coding has only used the upper root to the quadratic equation. This includes all versions predating version 6.3.1 including all versions of MCNP5 and MCNPX software. However, a review of the neutron next-event estimator physics has shown that this does not reproduce the track-length estimator results at low energies. Several examples are presented that test single neutron inelastic scattering reaction types, Level Scattering (Law 3), Tabulated Energy Angle (Law 61), and Kalbach-Mann distribution (Law 44). The test problems compare the track-length estimator (f4 tally) in the MCNP software, with the existing implementation of the neutron next-event estimator (f5 tally), and the modified changes to the neutron next-event estimator implementation. As the MCNP implementation ignores the lower root, the unmodified neutron next-event estimators will generally underestimate the lower energy contribution. However, a second issue with the Kalbach-Mann distribution (Law 44) implementation allows contributions to backward scattering in the center-of-mass frame that is not kinematically possible, thus overestimating backward scattering contributions. A third issue with the way the MCNP implementation handles floating point comparison for scattering directly ahead or directly backward in the center-of-mass frame generally leads to underestimation (except for backwards scattering for Law 44).

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

DOE OSTI2016

FFTF Passive Safety Test Data for Benchmarks for New LMR Designs

Liquid Metal Reactors (LMRs) continue to be considered as an attractive concept for advanced reactor design. Software packages such as SASSYS are being used to im-prove new LMR designs and operating characteristics. Significant cost and safety im-provements can be realized in advanced liquid metal reactor designs by emphasizing inherent or passive safety through crediting the beneficial reactivity feedbacks associ-ated with core and structural movement. This passive safety approach was adopted for the Fast Flux Test Facility (FFTF), and an experimental program was conducted to characterize the structural reactivity feedback. The FFTF passive safety testing pro-gram was developed to examine how specific design elements influenced dynamic re-activity feedback in response to a reactivity input and to demonstrate the scalability of reactivity feedback results to reactors of current interest. The U.S. Department of En-ergy, Office of Nuclear Energy Advanced Reactor Technology program is in the pro-cess of preserving, protecting, securing, and placing in electronic format information and data from the FFTF, including the core configurations and data collected during the passive safety tests. Benchmarks based on empirical data gathered during operation of the Fast Flux Test Facility (FFTF) as well as design documents and post-irradiation examination will aid in the validation of these software packages and the models and calculations they produce. Evaluation of these actual test data could provide insight to improve analytical methods which may be used to support future licensing applications for LMRs

Wootan, David W.↗

DOE OSTI2017

Computerized System for Safety Verification of External Beam Radiation Therapy Planning

Purpose: To report an assessment of in-house software, Verifier, developed to improve efficacy and efficiency of the radiation therapy (RT) treatment planning process and quality control review (QCR). Methods and Materials: Radiation therapy plan parameters retrieved from our treatment planning database are used by automated tests to give 75 types of warnings, such as prescription and plan discrepancies. The software is continuously updated on the basis of new issues, ideas, and planning policies. Verifier was retrospectively assessed (2007-2015) by examining impact on treatment plan revisions, frequency of quality improvement incident reports of avoidable RT plan-related safety events, unaddressed issues, and staff efficiency. Results: Plan revisions for specific issues declined dramatically in response to implementation of corresponding Verifier tests. Between 2012 and 2015 our institution's total rate of plan revisions dropped from 18.0% to 11.2%. Between 2008 and 2015 specific tests were added to Verifier while the rate of corresponding avoidable safety events was reduced from 0.34% to 0.00% over the same period. Simulations suggest Verifier saves approximately 2 to 5 minutes per QCR. Conclusions: The decrease in quantifiable metrics of plan revisions and incident reports suggests automatic RT plan-checking software enhances patient safety and clinical efficiency. Although only modest time savings may be gained using Verifier for the QCR itself, the greater impact on efficiency is through avoiding late-stage plan modifications and improving documentation via automation. We encourage other institutions to consider working toward adding similar technologies to enhance their RT quality assurance programs.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

DOE OSTI2023

HPC-driven computational reproducibility in numerical relativity codes: a use case study with IllinoisGRMHD

Abstract Reproducibility of results is a cornerstone of the scientific method. Scientific computing encounters two challenges when aiming for this goal. Firstly, reproducibility should not depend on details of the runtime environment, such as the compiler version or computing environment, so results are verifiable by third-parties. Secondly, different versions of software code executed in the same runtime environment should produceconsistent numerical results for physical quantities. In this manuscript, we test the feasibility of reproducing scientific results obtained using theIllinoisGRMHDcode that is part of an open-source community software for simulation in relativistic astrophysics, theEinstein Toolkit. We verify that numerical results of simulating a single isolated neutron star withIllinoisGRMHDcan be reproduced, and compare them to results reported by the code authors in 2015. We use two different supercomputers: Expanse at SDSC, and Stampede2 at TACC. By compiling the source code archived along with the paper on both Expanse and Stampede2, we find thatIllinoisGRMHDreproduces results published in its announcement paper up to errors comparable to round-off level changes in initial data parameters. We also verify that a current version ofIllinoisGRMHDreproduces these results once we account for bug fixes which have occurred since the original publication.

Astronomy & Astrophysics↗