Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Software Verification”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 199 records · Page 11
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Capability Enhancements for System-level thermal Hydraulic Modeling of Lead Fast Reactors

This project has focused on the use, assessment, and development of the SAS4A/SASSYS-1 (SAS) safety analysis software. Although SAS was originally intended as a safety analysis tool for Liquid Metal cooled Fast Reactors (LMFRs), which includes both Sodium Fast Reactors (SFRs) and Lead Fast Reactors (LFRs), the vast majority of its recent development and customization has been tailored to SFRs. In general, enhancements that are made to the software for SFRs are applicable to LFRs, however, the fuel composition and corrosive nature of lead requires careful consideration when performing safety analysis of an LFR. In this project an emphasis was placed on closing gaps that are associated with modeling LFRs using SAS. The principal objective of the project was to enhance the ability of SAS as a licensing tool for LFRs. This objective was to be accomplished through three tasks: 1) Enhance the ability of SAS to couple with external software; 2) Improve the underlying physics models in SAS, with priority given to models that are highly relevant to LFRs; 3) Extend the Verification and Validation (V&V) basis of the software

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2014

SU-E-T-608: Performance Comparison of Four Commercial Treatment Planning Systems Applied to Intensity-Modulated Radiotherapy

Purpose: To compare the performances of four commercial treatment planning systems (TPS) used for the intensity-modulated radiotherapy (IMRT). Methods: Ten patients of nasopharyngeal (4 cases), esophageal (3 cases) and cervical (3 cases) cancer were randomly selected from a 3-month IMRT plan pool at one radiotherapy center. For each patient, four IMRT plans were newly generated by using four commercial TPS (Corvus, Monaco, Pinnacle and Xio), and then verified with Matrixx (two-dimensional array/IBA Company) on Varian23EX accelerator. A pass rate (PR) calculated from the Gamma index by OminiPro IMRT 1.5 software was evaluated at four plan verification standards (1%/1mm, 2%/2mm, 3%/3mm, 4%/4mm and 5%/5mm) for each treatment plan. Overall and multiple pairwise comparisons of PRs were statistically conducted by analysis of covariance (ANOVA) F and LSD tests among four TPSs. Results: Overall significant (p>0.05) differences of PRs were found among four TPSs with F test values of 3.8 (p=0.02), 21.1(>0.01), 14.0 (>0.01), 8.3(>0.01) at standards of 1%/1mm to 4%/4mm respectively, except at 5%/5mm standard with 2.6 (p=0.06). All means (standard deviation) of PRs at 3%/3mm of 94.3 ± 3.3 (Corvus), 98.8 ± 0.8 (Monaco), 97.5± 1.7 (Pinnacle), 98.4 ± 1.0 (Xio) were above 90% and met clinical requirement. Multiple pairwise comparisons had not demonstrated a consistent low or high pattern on either TPS. Conclusion: Matrixx dose verification results show that the validation pass rates of Monaco and Xio plans are relatively higher than those of the other two; Pinnacle plan shows slight higher pass rate than Corvus plan; lowest pass rate was achieved by the Corvus plan among these four kinds of TPS.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2018

Test Harness Framework

The code facilitates running tests and then validating the results. The tests are normally attacks on a system and the verification is normally to validate protection software detected the attack, however the framework can be used in any area.

McCarty, MichaelV↗
DOE OSTI2012

An Integrated RFID and Barcode Tagged Item Inventory System for Deployment at New Brunswick Laboratory

New Brunswick Laboratory (NBL) has a numerous inventory containing thousands of plutonium and uranium certified reference materials. The current manual inventory process is well established but is a lengthy process which requires significant oversight and double checking to ensure correctness. Oak Ridge National Laboratory has worked with NBL to develop and deploy a new inventory system which utilizes handheld computers with barcode scanners and radio frequency identification (RFID) readers termed the Tagged Item Inventory System (TIIS). Certified reference materials are identified by labels which incorporate RFID tags and barcodes. The label printing process and RFID tag association process are integrated into the main desktop software application. Software on the handheld computers syncs with software on designated desktop machines and the NBL inventory database to provide a seamless inventory process. This process includes: 1) identifying items to be inventoried, 2) downloading the current inventory information to the handheld computer, 3) using the handheld to read item and location labels, and 4) syncing the handheld computer with a designated desktop machine to analyze the results, print reports, etc. The security of this inventory software has been a major concern. Designated roles linked to authenticated logins are used to control access to the desktop software while password protection and badge verification are used to control access to the handheld computers. The overall system design and deployment at NBL will be presented. The performance of the system will also be discussed with respect to a small piece of the overall inventory. Future work includes performing a full inventory at NBL with the Tagged Item Inventory System and comparing performance, cost, and radiation exposures to the current manual inventory process.

barcode↗
DOE OSTI2013

A Complex-Geometry Validation Experiment for Advanced Neutron Transport Codes

The Idaho National Laboratory (INL) has initiated a focused effort to upgrade legacy computational reactor physics software tools and protocols used for support of core fuel management and experiment management in the Advanced Test Reactor (ATR) and its companion critical facility (ATRC) at the INL.. This will be accomplished through the introduction of modern high-fidelity computational software and protocols, with appropriate new Verification and Validation (V&V) protocols, over the next 12-18 months. Stochastic and deterministic transport theory based reactor physics codes and nuclear data packages that support this effort include MCNP5[1], SCALE/KENO6[2], HELIOS[3], SCALE/NEWT[2], and ATTILA[4]. Furthermore, a capability for sensitivity analysis and uncertainty quantification based on the TSUNAMI[5] system has also been implemented. Finally, we are also evaluating the Serpent[6] and MC21[7] codes, as additional verification tools in the near term as well as for possible applications to full three-dimensional Monte Carlo based fuel management modeling in the longer term. On the experimental side, several new benchmark-quality code validation measurements based on neutron activation spectrometry have been conducted using the ATRC. Results for the first four experiments, focused on neutron spectrum measurements within the Northwest Large In-Pile Tube (NW LIPT) and in the core fuel elements surrounding the NW LIPT and the diametrically opposite Southeast IPT have been reported [8,9]. A fifth, very recent, experiment focused on detailed measurements of the element-to-element core power distribution is summarized here and examples of the use of the measured data for validation of corresponding MCNP5, HELIOS, NEWT, and Serpent computational models using modern least-square adjustment methods are provided.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2017

CASL Dakota Capabilities Summary

The Dakota software project serves the mission of Sandia National Laboratories and supports a worldwide user community by delivering state-of-the-art research and robust, usable software for optimization and uncertainty quantification. These capabilities enable advanced exploration and riskinformed prediction with a wide range of computational science and engineering models. Dakota is the verification and validation (V&V) / uncertainty quantification (UQ) software delivery vehicle for CASL, allowing analysts across focus areas to apply these capabilities to myriad nuclear engineering analyses.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

FY2021 Improvements to the New CTH Code Verification & Validation Test Suite

Over the past few years, the CTH multiphysics hydrocode has overhauled its software quality and testing processes, implementing current best practices in software quality and building a robust V&V test suite comprised of traditional hydrocode verification problems, including ASC Tri-Lab Test Suite and Enhanced Tri-Lab Test Suite problems, as well as validation problems for some of CTH’s most frequently used equations of state, materials models, and other key capabilities. Substantial progress towards building this new test suite was made in FY19 and FY20. In FY21, the test suite has been expanded to include verification and validation tests of the Steinberg-Guinan-Lund (ST) viscoplastic model and the Johnson Cook (JFRAC) fracture model. Additionally, two new verification tests were added, covering hydrodynamics and high explosive (HE) modeling capabilities: the Kidder Gaussian density problem and the Escape of HE Products (EHEP) problem from the Tri-Lab Test Suite. This report discusses each of these test problems in detail. Verification test results are compared to analytic solutions. Validation test results are compared to experimental data. Wherever possible, convergence or mesh refinement studies are included. Additionally, while implementing the Kidder verification problem, a bug was identified that affects the use of tables to initialize pressure or density in 1D or 2D calculations. A brief discussion of the bug and its fix is included. CTH demonstrates good performance overall on the new test suite problems. Simulation results showed good agreement with analytic solutions for the Kidder problem, with convergence rates ranging between 1.8 and sub-linear, and relatively good agreement for the EHEP problem, though convergence rates for pressure and density were nearly 0. The ST and JFRAC strain rate loading verification tests show good agreement with analytic solutions. Likewise, CTH simulation results show good agreement with experimental validation data, including Taylor rod impact testing, for the materials tested. Future V&V work will focus on adding 2D and 3D versions of existing verification tests as well as adding validation tests of other frequently used capabilities such as other fracture models.

42 ENGINEERING↗
DOE OSTI2023

CYVET: A Cyber-Physical Assurance Framework Based on a Semi-Supervised Vetting Approach

The project is designed to directly address the needs described in the Topic Area 4 (“Cybersecurity Verification and Validation”) of the CEDS’19 Research Call. The proposed CYVET system directly addresses the need to elevate the current industry capabilities to verify and validate OT cybersecurity and associated control system infrastructure. Currently, there is a significant gap in the energy sector’s capability in that regard during infrastructure improvement, equipment procurement, and compliance certification. CYVET provides that needed capability. CYVET is device and architecture agnostic and thus broadly applicable across the energy sector. The goal of this project is to develop and deliver a cybersecurity verification and validation framework testing capability to verify and validate OT equipment, software, and the underlying control system architecture. The primary project objectives are (i) Verification: the synthesis and reconciliation of standards and vendor supplied features, (ii) Validation: the generation, execution, and presentation of testing scripts of verified security features, and (3) Demonstration: apply the developed technology capabilities for verification and validation at a relevant end-user facility in the energy sector.

Lopez Jr, Juan↗
DOE OSTI2016

Rich-Text based PSA Model Transformation and Verification in RiskA

Nuclear Energy is one of the most possible alternative to fossil energy in large-scale. Nuclear safety is the lifeline of nuclear energy development, and has been an important part of national security, especially after the accident of Fukushima. On the basis of computer codes, reliability engineering and Probabilistic Risk Assessment (PSA) is effective methods to improve the safety of nuclear power plants. After many years of development, the methodologies, applications and relevant soft programs have been quite sufficient in the area of nuclear safety. However, it is still difficult for the current PSA technologies and software to meet the needs of dealing with the problems of quantitatively public risk assessment in the digital society. Especially, there are so many software that adopted different modeling and analyzing methods, which brings difficulties to the comparison, evaluation or verification for different software and models. By using its own academic advantages, the advanced nuclear energy research team, i.e., FDS Team has invested a few hundreds person-years of independent research and development of a reliability and probabilistic safety assessment software system RiskA. The system consists of three main functions: device reliability optimization management, probabilistic safety assessment and instantaneous risk monitoring, and the comprehensive public risk assessment in digital society. RiskA establishes an integrated risk evaluation system for the device reliability, nuclear power plant (NPP) risk monitoring and public risk assessment. Currently, RiskA has successfully been applied in more than 600 organizations and institutes from a variety of fields in 39 countries, involving nuclear, aerospace, defense and so on. For example, RiskA has been used in the safety analysis of International Thermonuclear Experimental Reactor (ITER), FDS series fusion reactor, and the Strategic Priority Research Program of Chinese Academy of Sciences 'the Future Advanced Nuclear Fission Energy - Accelerator Driven Sub-critical (ADS) transmutation system'. Also, RiskA has been used in public risk communication and risk assessment. Among them, Third Qinshan nuclear power plant Risk Monitor (TQRM), which is the China's first domestic nuclear safety analysis software used in the actual online operation, has been highly apprised by renowned experts in the field. Meanwhile, it has been identified as the first domestic nuclear power plant risk monitoring system with completely independent intellectual property rights, and has reached the international advanced level. (authors)

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2015

Automated Installation Verification of COMSOL via LiveLink for MATLAB

Verifying that a local software installation performs as the developer intends is a potentially time-consuming but necessary step for nuclear safety-related codes. Automating this process not only saves time, but can increase reliability and scope of verification compared to ‘hand’ comparisons. While COMSOL does not include automatic installation verification as many commercial codes do, it does provide tools such as LiveLink™ for MATLAB® and the COMSOL API for use with Java® through which the user can automate the process. Here we present a successful automated verification example of a local COMSOL 5.0 installation for nuclear safety-related calculations at the Oak Ridge National Laboratory’s High Flux Isotope Reactor (HFIR).

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2017

Initial Verification and Validation Assessment for VERA

The Virtual Environment for Reactor Applications (VERA) code suite is assessed in terms of capability and credibility against the Consortium for Advanced Simulation of Light Water Reactors (CASL) Verification and Validation Plan (presented herein) in the context of three selected challenge problems: CRUD-Induced Power Shift (CIPS), Departure from Nucleate Boiling (DNB), and Pellet-Clad Interaction (PCI). Capability refers to evidence of required functionality for capturing phenomena of interest while capability refers to the evidence that provides confidence in the calculated results. For this assessment, each challenge problem defines a set of phenomenological requirements against which the VERA software is assessed. This approach, in turn, enables the focused assessment of only those capabilities relevant to the challenge problem. The evaluation of VERA against the challenge problem requirements represents a capability assessment. The mechanism for assessment is the Sandia-developed Predictive Capability Maturity Model (PCMM) that, for this assessment, evaluates VERA on 8 major criteria: (1) Representation and Geometric Fidelity, (2) Physics and Material Model Fidelity, (3) Software Quality Assurance and Engineering, (4) Code Verification, (5) Solution Verification, (6) Separate Effects Model Validation, (7) Integral Effects Model Validation, and (8) Uncertainty Quantification. For each attribute, a maturity score from zero to three is assigned in the context of each challenge problem. The evaluation of these eight elements constitutes the credibility assessment for VERA.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
DOE OSTI2018

Updated Verification and Validation Assessment for VERA

The Virtual Environment for Reactor Applications (VERA) code suite is assessed in terms of capability and credibility against the Consortium for Advanced Simulation of Light Water Reactors (CASL) Verification and Validation Plan (presented herein) in the context of three selected challenge problems: CRUD-Induced Power Shift (CIPS), Departure from Nucleate Boiling (DNB), and Pellet-Clad Interaction (PCI). Capability refers to evidence of required functionality for capturing phenomena of interest while credibility refers to the evidence that provides confidence in the calculated results. For this assessment, each challenge problem defines a set of phenomenological requirements against which the VERA software is assessed. This approach, in turn, enables the focused assessment of only those capabilities relevant to the challenge problem. The evaluation of VERA against the challenge problem requirements represents a capability assessment. The mechanism for assessment is the Sandia-developed Predictive Capability Maturity Model (PCMM) that, for this assessment, evaluates VERA on 8 major criteria: (1) Representation and Geometric Fidelity, (2) Physics and Material Model Fidelity, (3) Software Quality Assurance and Engineering, (4) Code Verification, (5) Solution Verification, (6) Separate Effects Model Validation, (7) Integral Effects Model Validation, and (8) Uncertainty Quantification. For each attribute, a maturity score from zero to three is assigned in the context of each challenge problem. The evaluation of these eight elements constitutes the credibility assessment for VERA.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2019

Updated Verification and Validation Assessment for VERA

The Virtual Environment for Reactor Applications (VERA) code suite is assessed in terms of capability and credibility against the Consortium for Advanced Simulation of Light Water Reactors (CASL) Verification and Validation Plan (presented herein) in the context of three selected challenge problems: CRUD-Induced Power Shift (CIPS), Departure from Nucleate Boiling (DNB), and Pellet-Clad Interaction (PCI). Capability refers to evidence of required functionality for capturing phenomena of interest while credibility refers to the evidence that provides confidence in the calculated results. For this assessment, each challenge problem defines a set of phenomenological requirements against which the VERA software is assessed. This approach, in turn, enables the focused assessment of only those capabilities relevant to the challenge problem. The evaluation of VERA against the challenge problem requirements represents a capability assessment. The mechanism for assessment is the Sandia-developed Predictive Capability Maturity Model (PCMM) that, for this assessment, evaluates VERA on 8 major criteria: (1) Representation and Geometric Fidelity, (2) Physics and Material Model Fidelity, (3) Software Quality Assurance and Engineering, (4) Code Verification, (5) Solution Verification, (6) Separate Effects Model Validation, (7) Integral Effects Model Validation, and (8) Uncertainty Quantification. For each attribute, a maturity score from zero to three is assigned in the context of each challenge problem. The evaluation of these eight elements constitutes the credibility assessment for VERA.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2021

Considerations regarding the Use of Computer Vision Machine Learning in Safety-Related or Risk-Significant Applications in Nuclear Power Plants

With the advancements made to date in the field of artificial intelligence (AI), significant potential exists to utilize AI capabilities for nuclear power plant (NPP) applications. AI can replicate human decision making and it is usually faster and more accurate than humans. For implementations that impact critical NPP applications (e.g., safety-related or non-safety systems that potentially affect overall plant risk), a deeper safety analysis of the AI methods is necessary. AI applied to NPP operations could resemble the use of digital I&C (DI&C) because such applications involve digital computer hardware and custom-designed software that input plant data, execute complex software algorithms, and output the results to a system or licensed human operator to potentially provoke an action. For AI methods to be compliant with current safety requirements for DI&C, AI compatibility must be evaluated, and AI-related gaps may exist that prevent the prompt deployment of AI in NPPs. This effort aims to evaluate how example AI technologies align with the DI&C safety framework, and discusses how they could be analyzed, modeled, tested, and validated in a manner similar to typical DI&C technologies. Because AI is a broad field that encompasses areas such as machine learning (ML), natural language processing, and computer vision, this research focused on a subset of methods categorized as the computer vision ML (CVML) methods. This report explores two CVML use cases, gauge reading and fire watch, considered relevant to the DI&C standards, as they could play a safety-critical role. For the gauge reading use case, a CVML-enabled technology that can read gauges at oblique angles is utilized. For the fire watch use case, a CVML-enabled technology is utilized that migrates fire watch from a manual (human) approach to automated fire detection. These use cases are mainly intended to give context to the CVML system discussion. This effort assumes the worst-case scenario, with the CVML system being used to replace a safety-related or risk-significant system, thus requiring evaluation. Evaluating CVML against most of the relevant safety requirements for DI&C yielded several CVML-specific considerations due to the uniqueness of its characteristics in comparison with typical DI&C systems. For example, CVML models often employ commonly used (open-source) datasets, and it is not always possible to determine the level of overlap among open-source datasets. Therefore, the independence of the developed CVML models when demonstrating diversity is questionable, therefore creating vulnerability to common cause failure (CCF). The design verification process is also impacted since the data overlap could result in overestimation of the software validation and verification (V&V) performance results. Section 2 of this report evaluates a list of the identified CVML-specific characteristics and discusses the resulting considerations and potential solutions in the context of each referenced requirement. A summation is provided in Section 3. This report is not to be used as a guideline. It was developed to identify and consider issues in the implementation of ML technologies used to augment activities that may have a bearing on plant operation. The report draws parallels to the use of DI&C technologies, for which many standards are available to guide their use in nuclear plant operation. It considers the technologies and some of the potential implications of their use in safety-related applications but is not intended to address regulatory or licensing related issues.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2019

GALE 3.2 Verification Report

This report is documentation of the verification activities that have been performed on GALE-3.2 to ensure that the codes meet the requirements and specifications set forth by NRC and that the code meets its intended purpose. These activities are done in accordance with the GALE Software Quality Assurance Plan (PNNL-24249) and the GALE Configuration Management & Maintenance Plan (PNNL-24250). These two plans meet the requirements set for by both PNNL software quality assurance requirements and the NRC’s Software Quality Assurance Program and Guidelines (NUREG/BR-0167). The verification activities are documented to conform to Level 2 requirements of NUREG/BR-0167. Verifications were performed for both GALE-3.2 BWR and GALE-3.2 PWR Codes. The GALE 3.2 BWR and PWR codes are computerized mathematical models for calculating the releases of radioactive material in gaseous and liquid effluents. These computer codes are tools which can be used to determine conformance with the requirements of Appendix I to Title 10, Part 50 of the Code of Federal Regulations (10 CFR 50).

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2025

An Open-Source Python Package for CFD Solution Verification

Informed decision-making using computational fluid dynamics (CFD) results requires quantifying the errors and uncertainties of a simulation. Verification, validation, and uncertainty quantification (VVUQ) methods were developed to address this need and have matured. However, these VVUQ analyses are often non-trivial and require CFD analysts and practitioners to have specific skill sets. This has led to the uneven adoption of VVUQ analyses, in part, based on the availability of software tools to aid CFD analysts and practitioners. Solution verification, a procedure to evaluate the accuracy of a simulation by estimating potential errors arising from the computational model and computing the uncertainties without comparing to results from a physical system, is one of the lagging VVUQ analyses as the absence of software has forced CFD analysts and practitioners to develop their own codes or piece together incomplete software from across the internet. This work presents an opensource Python package, CFDverify, to lower the barrier of entry and fill in the technological gap in solution verification. CFDverify also provides a streamlined framework to remove some potential errors in post-processing CFD results. The hope is that CFDverify can improve the quality and quantity of CFD solution verification in scientific and research studies and attract interest in developing a communal tool. This paper describes the design, features, and an example use of CFDverify.

Weinmeister, Justin [ORNL] (ORCID:0000000160090237↗