Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Validation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 199 records · Page 11
All sourcesNASA NTRSDOE OSTI
DOE OSTI2019

Improvements to Bison Validation Base for LWR Fuel

The Bison fuel performance code is being developed by Idaho National Laboratory (INL) to analyze a wide range of fuel forms and operating conditions with the flexibility to conduct simulations in 1D through 3D geometries. Bison plays an important role in the development of advanced fuel with improved accident tolerance for existing light water reactors (LWRs), the improved understanding of mechanisms in fuel designs in a wider range of operating conditions, and the facilitation of future development of fuel for advanced reactor designs. Additionally, industry interest in advanced fuels has also driven Bison development for advanced fuels in the last few years. All of these factors combine to create a potential increase of new Bison users. These new users will potentially be interested in the Bison LWR validation base for examples of Bison's capabilities and how to use Bison. Current users also rely on the Bison validation base as a demonstration of the best practices settings and material behavioral models. Bison, both through direct updates to the Bison source code and to the nearly daily changes and improvements in the underlying MOOSE framework source code, undergoes nearly continuous changes. It is necessary that these source code changes are reflected with updated input files in the Bison validation base. Periodic updates of the validation case input files should also be accompanied by a review and revision of the validation case documentation on which our users rely. Therefore our efforts under this milestone were guided by two objectives: (1) Update the Bison LWR validation input files to use current Bison simulation settings, and (2) Enhance the ease-of-use for Bison through improved documentation These objectives build on the goals identified FY18 CASL milestone reports. During FY18 we addressed issues with the robustness of Bison and identified a need to migrate the assessment suite to the more reliable tensor mechanics module system and set a goal of standardizing the Bison input files. The FY18 CASL milestone report on updates to the Bison documentation system listed continued development of user-focused guides and examples along with revisions to the validation case documentation as future development goals. During FY19 we have separated these two objectives into four different focus areas for our work on this milestone. Our first two focus areas target the existing Bison validation suite: update the existing LWR validation cases and the accompanying documentation. The second two focus areas center on adding new capabilities: a new Bison LWR validation case and expansion of user support documentation. In the following report each of these four focus areas are discussed as a separate section. We conclude the report with a summary of potential future work to build on the efforts of this year.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2025

Progress Towards the Validation of a new RELAP5-3D model of the High Temperature Test Facility

Validation is a key step in the development of any type of systems model. As the next generation of reactors approaches, the need for codes that have been validated for these new types of systems continues to grow. An example of a prominent option is the Reactor Excursion Leak Analysis Program (RELAP5-3D), developed by Idaho National Laboratory. This code was developed for the purpose of systems level thermal-hydraulic modeling of light water reactors (LWRS) and postulated transients that can occur in LWRS.RELAP5-3D has been substantially validated against LWR data. Due to its long history as a reactor safety analysis tool, there has been an effort to adapt RELAP5-3D for the purposes of advanced reactor concepts such as prismatic high-temperature gas-cooled reactors (HTGRs). However, RELAP5-3D has not nearly been validated and verified for HTGRs to the degree of LWRs, warranting verification and validation opportunities with computational benchmarks and existing experimental facilities. Examples of such facilities include the modular high-temperature gas-cooled reactor (MHTGR) 350 and the high temperature engineering test reactor (HTTR) from Japan. The MHTGR 350 is a benchmark design concept for code-to-code verification purposes; therefore, it does not provide any experimental data for validation opportunities The HTTR provides useful multiphysics validation data but does not have the in-core instruments to generate thermal-hydraulic experimental data to help with RELAP5-3D validation. Consequently, a facility that could provide key in-core temperatures for thermal-hydraulic validation was still needed. The High Temperature Test Facility (HTTF) is an integral effects facility for HTGR thermal hydraulics developed and operated by Oregon State University. HTTF represents ¼ length scale of the General Atomics MHTGR and is rated for a total power of 2.2 MW. Axially, the core consists of an upper and lower reflector and 10 blocks, numbered from bottom to top (Block 1 is right above lower reflector). The core is heated via graphite resistive heater rods, with respective channels distributed throughout the core. The primary coolant is helium and heat can radiate out of the core to the reactor cavity cooling system (RCCS), which is cooled by water. The primary purpose of the facility is to investigate pressurized conduction cooldown (PCC) and depressurized conduction cooldown (DCC) transients, which are also referred to as the pressurized and depressurized loss of forced cooling respectively. Two experiments were chosen to perform the validation study with a RELAP5-3D model of HTTF. These experiments are PG-27 (PCC) and PG-29 (DCC). These were chosen based off of the quality of available experimental data before and during the experiment which led to their inclusion in the HTGR Thermal Hydraulics Benchmark.

22 - GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2018

Argonne Reactor Code Software Verification and Validation Plan for the Versatile Test Reactor

This Argonne Reactor Code (ARC) software has taken a pivotal role in the design work of the Versatile Test Reactor. The ARC software itself consists of several independent or semiindependent pieces of software that are used in concert to carry out specific analysis and design objectives. In this report, we discuss the validation that is needed for the ARC software overall and discuss available, usable experimental data and how it relates to the VTR project. We also discuss the verification that is required for the ARC software with respect to its usage in the VTR project and give some concrete examples as to what it must accomplish. A list of goals for the verification and validation work are presented as the general procedure that will be implemented for every individual piece of software in ARC. The ARC software has been continuously developed for over 50 years and many of the features used for the VTR project have thus been rigorously tested for accuracy during that timeframe. The ARC software is presently under source code control and undergoes nightly regression testing of not only the individual pieces, but the interconnected aspects that are common to most analysis projects. Its greatest weakness, from a software verification and validation perspective, is both a lack of documentation and outdated documentation. As an example, much of the previous validation work will require contemporary effort to re-evaluate given the updated base data libraries and modifications to the software that have occurred since. This requires new documentation to be produced to support the usage of the software in the VTR project. The bulk of the ARC software is focused on steady state neutronics and TH behavior of a reactor system. In this regard, we have identified the ZPPR, FFTF, and EBR-II programs as the best candidates for sources of validation data for the VTR project. The ZPPR-15 series in the ZPPR program was chosen as it is the best representation of the VTR fuel form and reactor core layout of all of the ZPPR measurements. Additional support for it comes from the fact that most of the data has already been processed into usable ARC models for VTR validation. The FFTF and EBR-II reactors were both functional reactors that provide reactor scale measurements. FFTF is clearly a closer match to VTR and its startup measurements would be a very valuable validation basis. EBR-II has invaluable burnup related measurements that will play a key role in the ARC software validation work. Each piece of ARC software will require a report detailing the software verification. The validation work will be logically split into the results of modeling each identified experimental resource (ZPPR, EBR-II, FFTF). The intended timeline for the VTR project is to have the ARC software verified and validated by the end of fiscal year 2020. This is incumbent on sufficient funding allocated to V&V activities and no change in programmatic priorities.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
DOE OSTI2024

Adaptive Protection and Validated Models to Enable Deployment of High Penetrations of Solar PV (PV-MOD)

The availability and validation of various PV models in commercial tools differ, with some models not yet thoroughly validated for advanced inverter functionalities and reliable performance under weak system conditions. Many existing models do not fully incorporate new inverter control functions, which can affect system stability. The increasing deployment of solar PV and other inverter-based resources (IBRs), including distributed energy resources (DERs), is influencing the reliable operation of protection schemes in distribution systems and microgrids. Emerging adaptive protection schemes (APS) offer new opportunities for protecting these systems during varying configurations and DER operating conditions, though their demonstration and validation remain limited. Adaptive protection schemes face similar challenges, as they are typically designed for specific configurations. There is a growing need for tools and methodologies to streamline the deployment of adaptive protection for safe and reliable DER integration. The project main objective was to develop and validate high-fidelity generic models of solar PV facilities for stability, protection, EMT, and QSTS analyses. This objective was achieved, and these models can now be integrated into commercial software tools, enabling utilities, vendors, and developers to study high-penetration PV systems more confidently. The project also demonstrated advanced applications of these models, including the design and deployment of adaptive protection schemes in high-penetration field applications and microgrids, supporting grid safety and reliability. Several milestones were reached by the end of the project. A sophisticated inverter test plan was developed, and inverters representative of the North American marketplace were selected. EPRI and NREL tested various inverters, conforming to IEEE standards. Improvements were made to existing generic models of IBR units, IBR plants, and aggregated feeders for various analyses. The first generic electromagnetic transient (EMT) model for a solar PV plant was developed, conforming to IEEE Std 2800™-2022 and validated against laboratory measurements of a 2.2 MVA large-scale battery energy storage system (BESS) inverter. That model was then used to produce reference responses illustrating examples of validated and verified IBR plant models that pass or fail tests for technical minimum capability and performance as specified in the IEEE standard. The developed, tested, and validated generic models can be used for transmission planning, stability assessments, expansion planning, and evaluating potential future IBR interconnection requirements. They can also support interconnection screens and conformity assessments of IBR plants, including solar PV. The project significantly contributed to the ongoing standardization and model-based representation and verification of IBR responses. The project further addressed challenges of common distribution protection schemes with increasing deployment of DER by developing, validating, and demonstrating adaptive protection schemes (APS) that can improve the reliable and safe integration of DER into distribution systems. New APS were designed using improved DER models for three common distribution systems: a radial feeder, a meshed network, and a microgrid. Modeling and hardware-in-the-loop (HIL) testing of the APS were conducted, successfully showing their effectiveness and selectivity. Proof-of-concept field demonstration was achieved for two APS, i.e., one on a radial feeder and another one in a microgrid. Field demonstration could not be achieved for the APS on a meshed network, primarily due apprehension of one utility partner and also due to limited access to the protective algorithms in the network protectors. Guidelines developed from the lessons learned in the project lay out the general process followed in the design, installation, and commissioning of APS for various distribution systems. Distribution utility partners’ apprehension about field demonstration of the new APS were addressed—with varying success—by taking a stepped risk-management approach of modeling of a wide range of sensitivities first, performing in-depth proof-of-concept testing in the laboratory including HIL next, and finally deliberately implementing and commissioning the actual protection equipment and algorithms into parts of—or in parallel operation to—the three real distribution systems. Future work should include pilot projects that further show the acceptable performance of the developed APS before these schemes be rolled out more widely. Inclusion of both utility and original equipment manufacturers (OEMs) in future projects could increase chances of successful field demonstration. Despite challenges in achieving the field demonstration goal of the project for all three APS, the research significantly contributed to the innovation of adaptive protection solutions for scalable and reliable DER integration into distribution systems. This project significantly enhances the understanding of the impact of using appropriate inverter models on distribution and transmission (T&D) systems. By addressing the limitations of existing generic models, the project introduces high-fidelity models for stability, protection, electromagnetic transient (EMT), and quasi-static time series (QSTS) analyses. These models, integrated into commercial software tools, enable utilities, vendors, and developers to confidently study high-penetration PV systems. The project also demonstrates advanced applications, including adaptive protection schemes (APS) for distribution systems and microgrids, ensuring grid safety and reliability. The technical effectiveness and economic feasibility of the methods are evident through the development and validation of sophisticated inverter test plans and the selection of representative inverters. Testing by EPRI and NREL on retail, commercial, and utility-scale inverters, conforming to IEEE standards, underscores the robustness of the models. Improvements to existing generic models for various analyses further enhance their validity and applicability. The project also identifies gaps in common distribution protection schemes and designed new APS using improved DER models, demonstrating their effectiveness through modeling and hardware-in-the-loop (HIL) testing. The project’s benefits to the public are manifold. By advancing the standardization and model-based representation of IBR response, it supports transmission planning, stability assessments, and future IBR interconnection requirements. The generic models can facilitate better communication between transmission planners and developers, supporting expected IBR plant capability and performance. Additionally, the development of APS for radial feeders, meshed networks, and microgrids supports the integration of distributed energy resources (DERs) into distribution systems, enhancing grid reliability and safety. The project’s emphasis on thorough testing and simplicity in design ensures practical and scalable solutions for DER integration.

14 SOLAR ENERGY↗
NASA NTRS1987

Design and validation of fault-tolerant flight systems

Flight systems must be validated to show that they are consistent with the requirements of their intended applications. While high reliability is difficult to validate, the additional complexity of fault tolerance further compounds the validation problem. The objective of NASA’s research is to develop a methodology for designing validatable fault-tolerant systems. Under the design-for-validation philosophy, emphasis is placed on developing validation methods that can be incorporated into the design process right from the start and design methods and guidance which, while incorporating fault tolerance, can assure validatability. This paper examines the statistical issues of validating highly reliable, fault tolerant system. There are many problems associated with traditional methods of designing and validating these potentially complex hardware and software systems. Useful design-for-validation methods, which include structured specification and design methodologies, mathematical proof techniques, analytical modeling, simulation and emulation, and physical testing, are discussed. Important design issues associated with fault tolerance are presented along with the related validation concerns which must be addressed. Experience has shown that synchronization and Byzantine resilience must accompany fault tolerance. Other design attributes associated with fault tolerance may be used by a designer on the basis of cost, weight, performance, and validation considerations.

Computer systems↗
NASA NTRS1990

CFD validation for aerodynamic flows - Challenge for the '90's

The process of computational fluid dynamics (CFD) validation is described from two perspectives, numerical error validation and physical model validation. Errors associated with time and space discretization, grid refinement, numerical dissipation, and level of solution convergence, are all possible sources of numerical error. The second aspect of CFD code validation is associated with the governing equations and the physical models, e.g., chemistry and turbulence models. This type of validation requires comparisons with carefully conducted experiments and is the primary aspect of the validation process discussed in this paper. Examples of validation are shown for selected Reynolds-averaged Navier-Stokes codes for conditions from transonic through hypersonic. Since code applications are becoming more complex, it no longer suffices to use data from surface or integral quantities alone to provide the required validation. Flow field surveys and experimental boundary condition measurements are emerging as critical data that must be obtained for CFD code validation. Progress in the overall level of validation is shown to be improving. However, future challenges remain, some of which are enumerated.

Marvin, Joseph G.↗
NASA NTRS1989

NASA's CFD Validation Program

With computational fluid dynamics (CFD) becoming a productive research and design tool, the requirement to validate CFD codes has grown significantly. NASA had emphasized CFD validation activities since 1986 when a separate work element was formed to fund experimental activities related to validation. NASA's CFD and CFD validation programs are closely coordinated to ensure that experimental data bases are available as soon as possible for validating codes. In response to industry and academic requirements, four levels of experimental research have been defined as part of CFD validation with NASA's Aeronautics Advisory Committee (AAC) support although only the fourth level actually has the detailed information necessary for validating codes. Critical flow physics especially turbulence modeling are key to improved CFD codes. NASA has focused additional resources on transition and turbulence physics to meet these requirements. With improved turbulence models, CFD codes will be more accurate, robust, and efficient. However, with the level of detailed information available from CFD codes, highly accurate and detailed experiments are required to capture the critical information for validating codes. Advanced instrumentation especially non-intrusive instrumentation is required to acquire this information in validation experiments. The CFD validation program is being coordinated and managed to address these critical activities. A list of experiments which are currently being supported at least partially are included.

Satran, Dale R.↗
NASA NTRS1996

Validation of NASA Thermal Ice Protection Computer Codes: Program Overview - Part 1

The Icing Technology Branch at NASA Lewis has been involved in an effort to validate two thermal ice protection codes developed at the NASA Lewis Research Center. LEWICE/Thermal (electrothermal deicing & anti-icing), and ANTICE (hot-gas & electrothermal anti-icing). The Thermal Code Validation effort was designated as a priority during a 1994 'peer review' of the NASA Lewis Icing program, and was implemented as a cooperative effort with industry. During April 1996, the first of a series of experimental validation tests was conducted in the NASA Lewis Icing Research Tunnel(IRT). The purpose of the April 96 test was to validate the electrothermal predictive capabilities of both LEWICE/Thermal, and ANTICE. A heavily instrumented test article was designed and fabricated for this test, with the capability of simulating electrothermal de-icing and anti-icing modes of operation. Thermal measurements were then obtained over a range of test conditions, for comparison with analytical predictions. This paper will present an overview of the test, including a detailed description of: (1) the validation process; (2) test article design; (3) test matrix development; and (4) test procedures. Selected experimental results will be presented for de-icing and anti-icing modes of operation. Finally, the status of the validation effort at this point will be summarized. Detailed comparisons between analytical predictions and experimental results are contained in the following two papers: 'Validation of NASA Thermal Ice Protection Computer Codes: Part 2- The Validation of LEWICE/Thermal' and 'Validation of NASA Thermal Ice Protection Computer Codes: Part 3-The Validation of ANTICE'

Miller, Dean↗
DOE OSTI2016

Validation and Spatiotemporal Analysis of CERES Surface Net Radiation Product

The Clouds and the Earth’s Radiant Energy System (CERES) generates one of the few global satellite radiation products. The CERES ARM Validation Experiment (CAVE) has been providing long-term in situ observations for the validation of the CERES products. However, the number of these sites is low and their distribution is globally sparse, and particularly the surface net radiation product has not been rigorously validated yet. Therefore, additional validation efforts are highly required to determine the accuracy of the CERES radiation products. In this study, global land surface measurements were comprehensively collected for use in the validation of the CERES net radiation (R n ) product on a daily (340 sites) and a monthly (260 sites) basis, respectively. The validation results demonstrated that the CERES R n product was, overall, highly accurate. The daily validations had a Mean Bias Error (MBE) of 3.43 W·m -2 , Root Mean Square Error (RMSE) of 33.56 W·m -2 , and R 2 of 0.79, and the monthly validations had an MBE of 3.40 W·m -2 , RMSE of 25.57 W·m -2 , and R 2 of 0.84. The accuracy was slightly lower for the high latitudes. Following the validation, the monthly CERES R n product, from March 2000 to July 2014, was used for a further analysis. The global spatiotemporal variation of the R n , which occurred during the measurement period, was analyzed. In addition, two hot spot regions, the southern Great Plains and south-central Africa, were then selected for use in determining the driving factors or attribution of the R n variation. We determined that R n over the southern Great Plains decreased by -0.33 W·m -2 per year, which was mainly driven by changes in surface green vegetation and precipitation. In south-central Africa, R n decreased at a rate of -0.63 W·m -2 per year, the major driving factor of which was surface green vegetation

54 ENVIRONMENTAL SCIENCES↗
NASA NTRSDate not supplied

TROPOMI Methane Validation in the GeoCarb Domain

Objective: GeoCarb validation •Purpose: •Do the four TCCON sites in the GeoCarb domain characterize satellite validation? •Method: •Assess the error estimates globally vs. in the GeoCarb domain •Assess the use of GML/DOE aircraft for CH4 validation •Systematic error estimate (from Kulawik, 2016): •+ regional bias (stdevof average bias versus validation at each station) •+ correlated error (stdev of daily average versus validation) •-co-location error (above quantities for model@satelliteminus model@valid) •-validation error (systematic error of validation estimates, e.g. model extension) Aircraft validation error •CH4 profile extension error estimate: 7 ppb •Extend aircraft observations to the top of the atmosphere with different model configurations •CAMS, scaling to stratosphere, then taking stratosphere •Extend by repeating top aircraft value to the troposphere, then taking CAMS •Extend with GEOS-Chem run (from 2010). •Difference CAMS-Scale vs. CAMS-Tropo: •3.6 +-5.5 ppb •Systematic error: 6.6 ppb •Extending aircraft to tropopause is blatantly wrong, but these two have the same stratosphere. •Difference CAMS-Scale vs. GEOS-Scale: •6.2 +-8.7 ppb •Do not consider bias since model is 10 years off. Systematic error ~8.7 ppb. •Estimate of systematic error from profile extension (possibly underestimated, resulting in possibly higher TROPOMI error estimate) •7 ppb •Would likely vary some by season, site, other conditions •Aircraft measurement error: 1 ppb •1 ppb from measurement error3

TROMPOMI↗
DOE OSTI2020

Development and validation of a 30-day mortality index based on pre-existing medical administrative data from 13,323 COVID-19 patients: The Veterans Health Administration COVID-19 (VACO) Index

Background Available COVID-19 mortality indices are limited to acute inpatient data. Using nationwide medical administrative data available prior to SARS-CoV-2 infection from the US Veterans Health Administration (VA), we developed the VA COVID-19 (VACO) 30-day mortality index and validated the index in two independent, prospective samples. Methods and findings We reviewed SARS-CoV-2 testing results within the VA between February 8 and August 18, 2020. The sample was split into a development cohort (test positive between March 2 and April 15, 2020), an early validation cohort (test positive between April 16 and May 18, 2020), and a late validation cohort (test positive between May 19 and July 19, 2020). Our logistic regression model in the development cohort considered demographics (age, sex, race/ethnicity), and pre-existing medical conditions and the Charlson Comorbidity Index (CCI) derived from ICD-10 diagnosis codes. Weights were fixed to create the VACO Index that was then validated by comparing area under receiver operating characteristic curves (AUC) in the early and late validation cohorts and among important validation cohort subgroups defined by sex, race/ethnicity, and geographic region. We also evaluated calibration curves and the range of predictions generated within age categories. 13,323 individuals tested positive for SARS-CoV-2 (median age: 63 years; 91% male; 42% non-Hispanic Black). We observed 480/3,681 (13%) deaths in development, 253/2,151 (12%) deaths in the early validation cohort, and 403/7,491 (5%) deaths in the late validation cohort. Age, multimorbidity described with CCI, and a history of myocardial infarction or peripheral vascular disease were independently associated with mortality–no other individual comorbid diagnosis provided additional information. The VACO Index discriminated mortality in development (AUC = 0.79, 95% CI: 0.77–0.81), and in early (AUC = 0.81 95% CI: 0.78–0.83) and late (AUC = 0.84, 95% CI: 0.78–0.86) validation. The VACO Index allows personalized estimates of 30-day mortality after COVID-19 infection. For example, among those aged 60–64 years, overall mortality was estimated at 9% (95% CI: 6–11%). The Index further discriminated risk in this age stratum from 4% (95% CI: 3–7%) to 21% (95% CI: 12–31%), depending on sex and comorbid disease. Conclusion Prior to infection, demographics and comorbid conditions can discriminate COVID-19 mortality risk overall and within age strata. The VACO Index reproducibly identified individuals at substantial risk of COVID-19 mortality who might consider continuing social distancing, despite relaxed state and local guidelines.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2016

Plan to Verify and Validate Multi-Hazard Risk-Informed Margin Management Methods and Tools

This report describes the development plan for verification and validation (V&V) of multihazard risk informed margin management (RIMM) methods and tools. The plan will include using new INL developed tool sets coupled with the new multi-partner External Hazards Experimental Group (EHEG) coordinated by the Idaho National Laboratory (INL) within the Risk-Informed Safety Margin Characterization (RISMC) technical pathway of the Light Water Reactor Sustainability Program. The RISMC pathway is developing a suite of new tools and methods for advanced evaluation of facility risk. The RISMC toolkit, includes tools and methods focused on evaluation of risk from external hazards (e.g, seismic and flooding events), which have been shown to be dominant risk contributors in probabilistic risk assessments performed for operating nuclear power plants (NPPs). The external events activity (EEA) within the RISMC pathway is tasked with developing tools and methods focused on evaluation of multi-hazard external risk (e.g, seismic and flooding events). The EHEG will provide technical expertise and experimental large-scale testing data needed for development and validation of tools and methods in the RISMC toolkit for external hazard safety evaluations. The EEA within RISMC has two key elements: (1) an organizational and research framework provided by INL and (2) a coordinated group of university and national laboratory partners with the complementary expertise and experimental capabilities needed to conduct large-scale external hazard-focused experiments. This cooperative group will allow INL to leverage a range of existing capabilities to meet the unique needs of RISMC tool and method development. In addition, the capabilities of the EHEG could be used to address needs of other national laboratories. Currently, there is limited data available for development and validation of the tools and methods being developed in the RISMC Toolkit. The EHEG is being developed to obtain high quality, large-scale experimental data validation of RISMC tools and methods in a timely and cost-effective way. The group of universities and national laboratories that will eventually form the EHEG (which is ultimately expected to include both the initial participants and other universities and national laboratories that have been identified) have the expertise and experimental capabilities needed to both obtain and compile existing data archives and perform additional seismic and flooding experiments. The data developed by EHEG will be stored in databases within RISMC. These databases will be used to validate the advanced external hazard tools and methods. The numerical tools under development in RISMC are used to evaluate the impacts that external hazards have on nuclear power plants. These analysis tools will be used to advanced current risk calculation processes and reduce uncertainty in these calculations. To have confidence in the predictive capability of these numerical tools it is important to verify and validate them. To verify and validate numerical tools the user must understand what physics is being used to represent the problem (i.e. constitutive models used to represent the soil and structure), the numerical solver approach used (i.e. finite element), what data already exists that could be used to validate the tools, and what data is needed for validation.

31274↗
DOE OSTI2021

A Verification and Validation Approach for COMSOL Multiphysics to Support High Flux Isotope Reactor (HFIR)

Over the last several decades, many reactors have successfully been converted from high enriched uranium (HEU) to low enriched uranium (LEU) fuels in United States in support of its global non-proliferation objectives. Of the reactors slated for conversion, five high-performance research reactors (HPRRs) remain. The high-flux isotope reactor (HFIR) at Oak Ridge National Laboratory (ORNL) is one of the five. Conversion of HFIR requires the qualification of a new fuel. To aid in the conversion process as well as improve safety margins, COMSOL Multiphysics was chosen to support and eventually supplement the steady-state heat-transfer code (SSHTC). In order to use COMSOL as a supplementary tool, i.e. one used for safety basis calculations, the code needs to undergo a verification and validation process. Verification and validation are important processes for the use of any software; however, physics-based solvers have a unique challenge, in that the validation is much more rigorous and its domain is only applicable to the general scope of the problem. This document details the verification and validation of COMOSL for analysis of HFIR. This report is intended for two different users: (1) current and future users of COMSOL for HFIR and (2) for individuals interested in assessing the scope of the validation of COMSOL for HFIR. This report is divided into six sections. The first of which details the process of verification and validation. Section one provides an overview of key concepts from ASME V&V 10, 20, and 40 and how these concepts are used to provide a validation of any multiphysics software. Section two covers the approach needed in order to validate multiphysics software for a given HFIR analysis. Parts three through five split up the key concepts highlighted in section two. The final section provides concluding remarks which demonstrate the scope and limitations of the validation study and how these may be expanded and improved.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
DOE OSTI2019

Photofission Validation Measurements for Active Interrogation Standards

Executive Summary PNNL has been developing technical standards for the detection of special nuclear material (SNM) in cargo containers using active interrogation approaches. Since the use of kilogram-scale quantities of SNM is impractical for standards testing, the work focused on designing surrogates that provided similar signature as the SNM. There are many active interrogation approaches to detect SNM; the standards development considered photofission, differential die-away and nuclear resonance fluorescence. Of these approaches, photofission currently carries the most interest. However, uncertainties in the photofission cross sections are as large as 50%, so that validation measurements are necessary to provide confidence that the surrogate will perform similarly to the SNM. This document describes approaches for conducting those validation measurements as well as provides a rough order-of-magnitude cost estimate for performing them. Photofission provides many observables for the detection of SNM. The standards development considered prompt neutrons, delayed neutrons and delayed gamma rays as possible detected particles. The surrogates, which consist primarily of depleted uranium (DU), were designed to reproduce the strength of these particles for a targeted quantity of 2 kg of HEU. Different surrogates are required for different measurement approaches because of the difference in cross section between 235 U and 238 U. For the standards validations, the relative strength of the detected particles for HEU and DU must be measured. To avoid measurements with kilogram-sized SNM, the validation approach is focused on two comparisons. In the first comparison, the relative rates for approximately 10 g samples of DU and HEU will be measured. This comparison will provide the basis for validating the relative strength of the detected particles for HEU and DU. In the second comparison, the relative rates for the approximately 10 g and 1 kg sample of DU will be measured. This comparison will enable the validation of secondary processes that occur after photofission that impact the measurement for kilogram-scale samples. For instance, for 2 kg of HEU, as much as 50% of the fission induced in the sample is induced from fission neutrons generated through photofission. There is a limited range of facilities in the U.S. that have both the necessary accelerator capability and the ability and willingness to handle SNM. Two such facilities, the Idaho Accelerator Center at Idaho State University and the Nuclear Engineering Program at the University of Michigan, have both expressed an interest in participating in such measurements. Both would need to amend their licensing to handle the 10 g of HEU proposed for these measurements. The rough order-of-magnitude cost estimate for all three validation measurements is $1.1M, spread over three years. Each of the three measurements could be performed separately, at approximately one-third of that cost. There are a number of critical assumptions in these cost estimate that will need to be confirmed before a more reliable cost estimate can be established. PNNL will manage the effort to ensure that the overall objective of the validation measurements can be achieved and will fabricate the 10 g samples of HEU and DU. Most the labor for the measurements will be provided by two collaborating universities as research opportunities for students. The successful completion of these validation measurements will reduce the uncertainties in the proposed standards from about 50% to 5%. The measurements would also provide some much-needed nuclear data on photofission.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
NASA NTRS2004

The Role of Structural Models in the Solar Sail Flight Validation Process

NASA is currently soliciting proposals via the New Millennium Program ST-9 opportunity for a potential Solar Sail Flight Validation (SSFV) experiment to develop and operate in space a deployable solar sail that can be steered and provides measurable acceleration. The approach planned for this experiment is to test and validate models and processes for solar sail design, fabrication, deployment, and flight. These models and processes would then be used to design, fabricate, and operate scaleable solar sails for future space science missions. There are six validation objectives planned for the ST9 SSFV experiment: 1) Validate solar sail design tools and fabrication methods; 2) Validate controlled deployment; 3) Validate in space structural characteristics (focus of poster); 4) Validate solar sail attitude control; 5) Validate solar sail thrust performance; 6) Characterize the sail's electromagnetic interaction with the space environment. This poster presents a top-level assessment of the role of structural models in the validation process for in-space structural characteristics.

Johnston, John D.↗
NASA NTRS2021

Validation Assessment for the Soil Moisture Active Passive (SMAP) Level 4 Carbon (L4_C) Data Product Version 5

The post-launch Cal/Val phase of the SMAP mission is guided by two primary objectives for each science product team: 1) to calibrate, verify, and improve the performance of the science algorithms, and 2) validate accuracies of the science data products as specified in the SMAP Level-1 mission science requirements. Algorithm science and product maintenance activities during the SMAP extended mission phase have also involved periodic algorithm calibration and product refinements to maintain or enhance product consistency and performance as well as science utility. This report provides an assessment of the latest (Version 5) SMAP Level 4 Carbon (L4_C) product. The L4_C Version 5 (v5) global record now spans more than six years (March 2015 – present) of SMAP operations and has benefited from five major reprocessing updates to the operational product. These reprocessing events and L4_C product release updates have incorporated various algorithm refinements and calibration adjustments to account for similar refinements to the upstream GEOS land model assimilation system, SMAP brightness temperatures, and MODIS vegetation inputs used for L4_C processing. The SMAP L4_C algorithms utilize a terrestrial carbon flux model informed by daily surface and root zone soil moisture information contributed from the SMAP Level 4 Soil Moisture (L4_SM) product along with optical remote sensing-based (e.g. MODIS-based) land cover and canopy fractional photosynthetic active radiation (fPAR), and other ancillary biophysical data. The carbon flux model estimates global daily net ecosystem CO2 exchange (NEE) and the component carbon fluxes, namely, vegetation gross primary production (GPP) and soil heterotrophic respiration (Rh). Other L4_C product elements include surface (~0-5 cm depth) soil organic carbon (SOC) stocks and associated environmental constraints to these processes, including soil moisture-related controls on GPP and ecosystem respiration (Kimball et al. 2014, Jones et al. 2017). The L4_C product addresses SMAP carbon cycle science objectives by: 1) providing a direct link between terrestrial carbon fluxes and underlying freeze/thaw and soil moisture-related constraints to these processes, 2) documenting primary connections between terrestrial water, energy and carbon cycles, and 3) improving understanding of terrestrial carbon sink activity. The SMAP L4_C algorithms and operational product are mature and at a CEOS Validation Stage 4 level (Jackson et al. 2012) based on extensive validation of the multi-year record against a diverse array of independent benchmarks, well characterized global performance, and systematic refinements gained from five major reprocessing events. There are no Level-1 mission science requirements for the L4_C product; however, self-imposed requirements have been established focusing on NEE as the primary product field for validation, and on demonstrating L4_C accuracy and success in meeting product science requirements (Jackson et al. 2012). The other L4_C product fields also have strong utility for carbon science applications (e.g., Liu et al. 2019, Endsley et al. 2020); however, analysis of these other fields is considered secondary relative to primary validation activities focusing on NEE. The L4_C targeted accuracy requirements are to meet or exceed a mean unbiased root-mean-square error (ubRMSE, or standard deviation of the error) for NEE of 1.6 g C m-2 d-1 and 30 g C m-2 yr-1, emphasizing northern (≥45°N) boreal and arctic ecosystems; this accuracy is similar to that of tower eddy covariance measurement-based observations (Baldocchi 2008). Methods used for the latest v5 L4_C product performance and validation assessment have been established from the SMAP Cal/Val plan and previous studies (Jackson et al. 2012, Jones et al. 2017) and include: 1) consistency evaluations of the product fields against earlier product releases (version 4 or earlier); 2) comparisons of daily carbon flux estimates with independent tower eddy 4covariance measurement-based daily carbon (CO2) flux observations from core tower validation sites (CVS); and 3) consistency checks against other global carbon products, including soil carbon inventory records, global GPP records derived from tower observation upscaling methods, and satellite-based observations of canopy solar induced chlorophyll fluorescence (SIF) as a surrogate for GPP. Metrics used to evaluate relative agreement between L4_C product fields and observational benchmarks include correlation (r-value), RMSE differences, bias and model sensitivity diagnostics. Following these validation criteria, the present report provides a validation assessment of the latest L4_C product release (v5). Detailed descriptions of the L4_C algorithm and additional global product accuracy and performance results are given elsewhere (Jones et al. 2017, Endsley et al. 2020). The v5 L4_C product replaces earlier product versions and continues to show: (i) accuracy and performance levels meeting or exceeding SMAP L4_C science requirements; (ii) improvement over the previous product version (version 4); and (iii) suitability for a diversity of science applications. Example L4_C applications from the recent literature include clarifying environmental trends and controls on the northern terrestrial carbon sink (Liu et al. 2019), diagnosing drought-related impacts on ecosystem productivity (Li et al. 2020), and regional monitoring of cropland conditions for projecting annual yields (Wurster et al. 2020). 2EXPECTED L4_C ALGORITHM AND PRODUCTPERFORMANCE The L4_C algorithm performance, including variance and uncertainty estimates of model outputs, was determined during the mission pre-launch phase through spatially explicit model sensitivity studies using available model inputs similar to those currently being used for operational production and evaluating the resulting model simulations over the observed range of northern (≥45 °N) and global conditions (Kimball et al. 2012, Entekhabi et al. 2014). The L4_C algorithm options were also evaluated during the mission prelaunch phase, including deriving canopy fPAR from lower order NDVI (Normalized Difference Vegetation Index) inputs in lieu of using MODIS (MOD15) fPAR; and including an explicit model representation of boreal fire disturbance recovery impacts. These results indicated that the L4_C accuracy requirements (i.e., NEE ubRMSE ≤ 30 g C m-2 yr-1or ≤ 1.6 g C m-2 d-1) could be met from the baseline algorithms over more than 82% and 89% of global and northern vegetated land areas, respectively (Yi et al. 2013, Kimball et al. 2014). The global L4_C algorithm error budget for NEE derived during the mission prelaunch phase indicated that the estimated NEE ubRMSE uncertainty is proportional to GPP and is therefore larger in higher biomass productivity areas, including forests and croplands (Kimball et al. 2014). Likewise, NEE ubRMSE uncertainty is expected to be lower in less-productive areas, including grasslands and shrublands. Expected model NEE ubRMSE levels were also generally within targeted accuracy levels for characteristically less-productive boreal and Arctic biomes, even though relative model error as a proportion of total productivity (NEE RMSE / GPP) may be large in these areas. The estimated NEE uncertainty was lower than expected in some warmer tropical high biomass productivity areas (e.g. Amazon rainforest) because of reduced low temperature and moisture constraints to the L4_C respiration calculations so that the bulk of model uncertainty is contributed by GPP in these areas. Model NEE uncertainty in the African Congo was estimated to

SMAP↗
DOE OSTI2015

Predicting Overall Survival After Stereotactic Ablative Radiation Therapy in Early-Stage Lung Cancer: Development and External Validation of the Amsterdam Prognostic Model

Purpose: A prognostic model for 5-year overall survival (OS), consisting of recursive partitioning analysis (RPA) and a nomogram, was developed for patients with early-stage non-small cell lung cancer (ES-NSCLC) treated with stereotactic ablative radiation therapy (SABR). Methods and Materials: A primary dataset of 703 ES-NSCLC SABR patients was randomly divided into a training (67%) and an internal validation (33%) dataset. In the former group, 21 unique parameters consisting of patient, treatment, and tumor factors were entered into an RPA model to predict OS. Univariate and multivariate models were constructed for RPA-selected factors to evaluate their relationship with OS. A nomogram for OS was constructed based on factors significant in multivariate modeling and validated with calibration plots. Both the RPA and the nomogram were externally validated in independent surgical (n=193) and SABR (n=543) datasets. Results: RPA identified 2 distinct risk classes based on tumor diameter, age, World Health Organization performance status (PS) and Charlson comorbidity index. This RPA had moderate discrimination in SABR datasets (c-index range: 0.52-0.60) but was of limited value in the surgical validation cohort. The nomogram predicting OS included smoking history in addition to RPA-identified factors. In contrast to RPA, validation of the nomogram performed well in internal validation (r{sup 2}=0.97) and external SABR (r{sup 2}=0.79) and surgical cohorts (r{sup 2}=0.91). Conclusions: The Amsterdam prognostic model is the first externally validated prognostication tool for OS in ES-NSCLC treated with SABR available to individualize patient decision making. The nomogram retained strong performance across surgical and SABR external validation datasets. RPA performance was poor in surgical patients, suggesting that 2 different distinct patient populations are being treated with these 2 effective modalities.

62 RADIOLOGY AND NUCLEAR MEDICINE↗
DOE OSTI2023

Empirical validation of building energy simulation model input parameter for multizone commercial building during the cooling season

This paper presents a critical advancement in Building Energy Modeling (BEM) through an empirical validation approach using a high-quality dataset from a multizone commercial office building in Oak Ridge, TN, USA. BEM is widely utilized in diverse construction applications, but its effectiveness relies on the accuracy of its predictions. The study focuses on empirical validation of input parameters in BEM, including building envelope data, infiltration modeling, and rooftop unit system performance curves. The validation of simulation input parameters leads to substantial improvements in the accuracy of simulation results. Notable both NMBE and cv (RMSE) values are reduced by 0.5 % for indoor air temperature and 17 % for indoor air relative humidity compared to the previous model. At the system level, both NMBE and cv (RMSE) values are reduced by 2 % for fan energy consumption and 4 % for cooling energy consumption, compared to the previous model. A literature review highlights a significant gap in empirical validation studies, which predominantly concentrate on either component-level or whole building validation. Furthermore, many studies employ simplified setups that may not faithfully represent the complexities of multizone commercial buildings. This paper distinguishes itself by emphasizing the critical importance of component-level input parameter validation. It underlines the need to validate data related to building envelope components and HVAC system performance curves, resulting in more accurate simulation outcomes. In conclusion, the utilization of actual multizone commercial building data enhances the study's practical relevance. In summary, this research underscores the pivotal role of input parameter validation in enhancing the accuracy and reliability of BEM.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗