Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “validation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 235 records · Page 13

Validation of RELAP-7 for Forced Convective and Natural Circulation Reactor Flows

New reactor designs such as small modular reactors and newly proposed passive safety systems rely on natural circulation two-phase flow cooling of the reactor core under accident conditions. Without a pump to drive the flow, the circulation rate is a strong function of void fraction. Under these conditions where the flow rate, pressure drop, and heat transfer are all closely coupled through the mixture density, the accurate prediction of void fraction is pivotal. Furthermore, while these passive systems boast a more reliable cooling option, the sensitivity of the system to two-phase flow instabilities is more likely than in forced convective cooling and can result in dangerous density wave or flow excursion conditions. Improvement of one-dimensional system codes and further development of three-dimensional code capability are all important efforts in addressing these issues. The proposed work is a critical step towards providing confidence through validation of the capability of thermal-hydraulic tools to meet the emerging challenges in reactor simulation. Historically, system analysis codes develop a stiffness over time due to fine-tuning of adjustable code parameters that result in a reluctance or inability to adopt improved models. Therefore, a rigorous effort should be invested in preparation for RELAP-7 to identify required modeling improvements of constitutive relations and validation of the code applicability to the wide range of possible reactor flow conditions. Part of this validation involves the uncertainty quantification which is a critical step in the Code Scaling, Applicability, and Uncertainty (CSAU) methodology and Best-Estimate Plus Uncertainty (BEPU) licensing method. BEPU allows recovery of safety margins and potentially optimizing plant performance, resulting in economic benefit for the reactor operators. Beginning with the collection of past experimental data covering prototypic LWR conditions, this document reviews three critical datasets available for the validation of system codes. The experimental methods, procedures, facilities, instrumentation, and supplementary information are consolidated to aid the validation efforts of this work and future projects. These three datasets (824 conditions) are chosen due to their wide acceptance and demonstrated application for code validation. Two experimental datasets from the Nuclear Power Engineering Corporation (NUPEC) span prototypic conditions of Boiling Water Reactors (BWR) and Pressurized Water Reactors (PWR). These datasets are labeled as the BFBT (BWR Full-size Fine-mesh Bundle Test) and the PSBT (PWR Subchannel and Bundle Test) datasets respectively. The third dataset is taken in an internally heated annulus that is scaled based on a BWR subassembly and includes two-phase flows important to post-blowdown analysis. Together, the datasets consider the full range of pressure and mass flux expected in LWR forced convective flows from steady-state to long term cooling conditions. Recognizing the deficiency of available data under low-pressure, low-flow, a new natural circulation dataset consisting of 107 conditions is obtained. With system pressures ranging between 145 and 950 kPa, natural circulation is driven by a heat flux up to 275 kW/m2 in a 3 m long heated region of the annulus test section which is followed by a 2 m long unheated region before entering a condenser. Stable natural circulation flow rates in this facility were measured to be between 182 and 590 kg/m2-s. Pressure and temperature are measured at ten locations in the facility. In addition, measurements of void fraction, interfacial area concentration, and gas velocity are obtained at five axial locations in the annulus test section. The RELAP5-3D code demonstrates good accuracy for void fraction predictions in BFBT and PSBT benchmarks, but shows relatively large discrepancy in the annulus experiments at lower pressures. In two-phase natural circulation, large oscillations are observed in many system parameters during the simulation of 19 experiment cases which were conducted at low pressures, and the model shows good predictive performance with the remaining 88 experiment cases. However, some discrepancies in void fraction and temperature predictions are found which may require improved constitutive modeling under these low-pressure, low-flow condition. The quantified uncertainty in void fraction predictions in three forced convection benchmarks due to uncertainties in boundary conditions and geometries are relatively small, except for a few cases in the annulus experiment. The output uncertainty of the natural circulation model is quantified by considering the measurement error in boundary conditions, and it is shown that the output uncertainty ranges are relatively small for the range of conditions considered. Through this work, a clear pathway is provided for validation of RELAP7 once the code is available.

42 ENGINEERING↗

Model validation and selection in metabolic flux analysis and flux balance analysis

13C-Metabolic Flux Analysis (13C-MFA) and Flux Balance Analysis (FBA) are widely used to investigate the operation of biochemical networks in both biological and biotechnological research. Both methods use metabolic reaction network models of metabolism operating at steady state so that reaction rates (fluxes) and the levels of metabolic intermediates are constrained to be invariant. They provide estimated (MFA) or predicted (FBA) values of the fluxes through the network in vivo, which cannot be measured directly. These fluxes can shed light on basic biology and have been successfully used to inform metabolic engineering strategies. Several approaches have been taken to test the reliability of estimates and predictions from constraint-based methods and to compare alternative model architectures. Despite advances in other areas of the statistical evaluation of metabolic models, such as the quantification of flux estimate uncertainty, validation and model selection methods have been underappreciated and underexplored. We review the history and state-of-the-art in constraint-based metabolic model validation and model selection. Applications and limitations of the χ 2 -test of goodness-of-fit, the most widely used quantitative validation and selection approach in 13C-MFA, are discussed, and complementary and alternative forms of validation and selection are proposed. A combined model validation and selection framework for 13C-MFA incorporating metabolite pool size information that leverages new developments in the field is presented and advocated for. Finally, we discuss how adopting robust validation and selection procedures can enhance confidence in constraint-based modeling as a whole and ultimately facilitate more widespread use of FBA in biotechnology.

59 BASIC BIOLOGICAL SCIENCES↗

Development, verification, and validation of comprehensive acoustic fluid-structure interaction capabilities in an open-source computational platform

The acoustic fluid-structure interaction (FSI) formulation is a practical numerical approach for the seismic analysis of fluid-filled tanks. However, there are no verification and validation studies reported in the literature that demonstrate the ability of an acoustic FSI numerical model to predict responses important to structural and mechanical design for intense translational and rotational earthquake inputs. Herein, an acoustic FSI formulation is implemented in the open-source Multiphysics Object-Oriented Simulation Environment (MOOSE), and is formally verified and validated using analytical solutions and code-to-code verification, and experimental data, respectively. The analytical solutions are for small amplitude, unidirectional seismic inputs. The code-to-code verification utilizes a previously verified and validated Arbitrary Lagrangian-Eulerian (ALE) numerical model in the commercial finite element code LS-DYNA. The validation studies utilize a comprehensive data set assembled from results of 3D earthquake-simulator tests of a fluid-filled vessel. The acoustic numerical model in MOOSE is verified and validated for hydrodynamic pressures and support reactions except for cases that involve significant convective response. For small amplitude inputs, numerically predicted wave heights match those of the analytical solutions. The numerical model is not verified and validated for wave height calculations under intense 3D seismic inputs. The run times for the acoustic FSI simulations in MOOSE are an order of magnitude, or more, shorter than for the corresponding ALE simulations in LS-DYNA. The utility of the MOOSE acoustic FSI implementation is demonstrated by seismic analysis of a building equipped with a fluid-filled, advanced nuclear reactor.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Validation of SAS4A/SASSYS-1 for predicting steady-state single-phase natural circulation

The validation of system analysis codes for nuclear reactor systems is required for the development and application of these computational tools. Designed as a comprehensive system analysis code for advanced nuclear reactors, SAS4A/SASSYS-1 requires validation of its physics model for capturing single-phase natural circulation behavior. To support the validation of SAS4A/SASSYS-1, high-precision experiments are performed capturing steady-state single-phase natural circulation on a scaled facility with comprehensive instrumentation. In this work, dedicated tests are performed quantifying the critical modeling parameters, and a single-phase natural circulation benchmark dataset is obtained with well-documented uncertainty and comprehensive facility description. The validation is then performed against the dataset examining the capability of SAS4A/SASSYS-1 in simulating steady-state single-phase natural circulation. The experimental facility is modeled in the candidate code. Solution verification is performed using Richardson-extrapolation-based estimators which quantify and restrict numerical errors from discretization. Input uncertainty provided by the benchmark dataset is forward propagated through the candidate code, quantifying the output uncertainty in a Monte Carlo approach. The composition of the output uncertainty is also quantified through a variance-based sensitivity analysis. With the uncertainty quantified for each individual condition, a detailed comparison between the simulation results and experimental data is performed covering the whole dataset. The results show consistent agreement for all primary parameters. The current validation activity provides a valuable benchmark dataset for the validation of system analysis codes in capturing single-phase natural circulation and demonstrates satisfactory prediction capability of SAS4A/SASSYS1 for steady-state single-phase natural circulation.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Enhancement of risk informed validation framework for external hazard scenario

In recent years, the U.S. Nuclear Regulatory Commission (USNRC) and the International Atomic Energy Agency (IAEA) have developed methodologies to assess the vulnerabilities of nuclear plants against site specific extreme hazards. In many cases, advanced simulation tools are being considered to simulate multi-physics, multi-scale phenomena and to evaluate vulnerability of nuclear facilities. The credibility of advanced simulation tools is assessed based on a formal verification, validation, and uncertainty quantification procedure. One of the key limitations in validation is the lack of relevant experimental data at system-level. This limitation leads to a decrease in the confidence of system-level risk predictions. Therefore, a robust validation framework is needed to formalize the confidence in predictive capability of advanced simulation results. Additionally, this study enhances the existing risk informed validation methodology, originally proposed by Kwag et al. [1] and Bodda et al. [2], by developing additional attributes and a new set of validation indicies for a complete and wider applicability of the framework. In this manuscript, the methodology to identify the critical path that leads to the system-level failure is illustrated. The overall validation is checked for completeness and consistency by comparing the critical path for both the system-level simulation and experimental models. The applicability of the code for an intended application is represented in terms of various maturity levels and helps in the process of decision making.

42 ENGINEERING↗

Validation of crystallographic models containing TLS or other descriptions of anisotropy

Guidelines and specific tests for validating macromolecular crystal structures that include TLS models are introduced. Validation may used to troubleshoot problems during refinement, to confirm the internal consistency of the model as part of deposition into the Protein Data Bank or to assess the plausibility of interpretating the boundary between two TLS groups as indicating a hinge point between structural domains. The use of TLS (translation/libration/screw) models to describe anisotropic displacement of atoms within a protein crystal structure has become increasingly common. These models may be used purely as an improved methodology for crystallographic refinement or as the basis for analyzing inter-domain and other large-scale motions implied by the crystal structure. In either case it is desirable to validate that the crystallographic model, including the TLS description of anisotropy, conforms to our best understanding of protein structures and their modes of flexibility. A set of validation tests has been implemented that can be integrated into ongoing crystallographic refinement or run afterwards to evaluate a previously refined structure. In either case validation can serve to increase confidence that the model is correct, to highlight aspects of the model that may be improved or to strengthen the evidence supporting specific modes of flexibility inferred from the refined TLS model. Automated validation checks have been added to the PARVATI and TLSMD web servers and incorporated into the CCP4i user interface.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

Understanding Dynamic Model Validation of a Wind Turbine Generator and a Wind Power Plant

Regional reliability organizations require power plants to validate the dynamic models that represent them to ensure that power systems studies are performed to the best representation of the components installed. In the process of validating a wind power plant (WPP), one must be cognizant of the parameter settings of the wind turbine generators (WTGs) and the operational settings of the WPP. Validating the dynamic model of a WPP is required to be performed periodically. This is because the control parameters of the WTGs and the other supporting components within a WPP may be modified to comply with new grid codes or upgrades to the WTG controller with new capabilities developed by the turbine manufacturers or requested by the plant owners or operators. The diversity within a WPP affects the way we represent it in a model. Diversity within a WPP may be found in the way the WTGs are controlled, the wind resource, the layout of the WPP (electrical diversity), and the type of WTGs used. Each group of WTGs constitutes a significant portion of the output power of the WPP, and their unique and salient behaviors should be represented individually. The objective of this paper is to illustrate the process of dynamic model validations of WTGs and WPPs, the available data recorded that must be screened before it is used for the dynamic validations, and the assumptions made in the dynamic models of the WTG and WPP that must be understood. Without understanding the correct process, the validations may lead to the wrong representations of the WTG and WPP modeled.

17 WIND ENERGY↗

Validation of Numerical Models of the Offshore Wind Turbine From the Alpha Ventus Wind Farm Against Full-Scale Measurements Within OC5 Phase III

The main objective of the Offshore Code Comparison Collaboration Continuation, with Correlation (OC5) project is validation of aero-hydro-servo-elastic simulation tools for offshore wind turbines (OWTs) through comparison of simulated results to the response data of physical systems. Phase III of the OC5 project validates OWT models against the measurements recorded on a Senvion 5M wind turbine supported by the OWEC Quattropod from the alpha ventus offshore wind farm. The following operating conditions of the wind turbine were chosen for the validation: (1) Idling below the cut-in wind speed; (2) Rotor-nacelle assembly (RNA) rotation maneuver below the cut-in wind speed; (3) Power production below and above the rated wind speed; and (4) Shutdown. A number of validation load cases were defined based on these operating conditions. The following measurements were used for validation: (1) Strains and accelerations recorded on the support structure; (2) Pitch, yaw, and azimuth angles, generator speed, and electrical power recorded from the RNA. Strains were not directly available from the majority of the OWT simulation tools. Therefore, strains were calculated based on out-of-plane bending moments, axial forces, and cross-sectional properties of the structural members. Also, a number of issues arose during the validation: (1) The need for a thorough quality check of sensor measurements; (2) The sensitivity of the turbine loads to the controller and airfoil properties, which were only approximated in the modeling approach; (3) The importance of estimating and applying an appropriate damping value for the structure; and (4) The importance of wind characteristics beyond turbulence on the loads. The simulation results and measurements were compared in terms of time series, discrete Fourier transforms, power spectral densities, probability density functions of strains and accelerometers. A good match was achieved between the measurements and models set up by OC5 Phase III participants.

17 WIND ENERGY↗

Electron Temperature Fluctuation Measurements and Transport Model Validation at Alcator C-Mod

The tokamak is a type of toroidal device used to confine a fusion plasma using large magnetic fields. Tokamaks and stellarators the leading devices for confining plasmas for fusion, and the capability to predict performance in these magnetically confined plasmas is essential for developing a sustainable fusion energy source. The magnetic configuration of tokamaks and stellarators does not exist in Nature, yet, the fundamental processes governing transport in fusion plasmas are universal – turbulence and instabilities, driven by inhomogeneity and asymmetry in the plasma, conspire to transport heat and particles across magnetic field lines and can play critical roles in impurity confinement and generation of intrinsic rotation. Turbulence exists in all plasmas, and in neutral fluids as well. The study of turbulence is essential to developing a fundamental understanding of the nature of the fourth state of matter, plasmas. Experimental studies of turbulence in tokamaks date back to early scattering observations from the late 1970s. Since that time, great advances in turbulence diagnostics have been made, all of which have significantly enhanced our knowledge and understanding of turbulence in tokamaks. Through comparisons with advanced gyrokinetic theory and turbulent-transport models a great deal of evidence exists to implicate turbulent-driven transport as an important mechanism determining transport in all channels: heat, particle and momentum However, prediction and control of turbulent-driven transport remains elusive. Key to development of predictive transport models for magnetically confined fusion plasmas is validation of the nonlinear gyrokinetic transport model, which describes transport due to turbulence. Validation of gyrokinetic codes must include detailed and quantitative comparisons with measured turbulence characteristics, in addition to comparisons with inferred transport levels and equilibrium profiles. For this reason, advanced plasma diagnostics for studying core turbulence are needed in order to assess the accuracy of gyrokinetic models for turbulent-driven particle, heat and momentum transport. New core turbulence diagnostics at the world-class tokamaks Alcator C-Mod at MIT and ASDEX Upgrade at the Max Planck Institute for Plasma Physics have been designed, developed, and operated over the course of this project. These new instruments are capable of measuring electron temperature fluctuations and the phase angle between density and temperature fluctuations locally and quantitatively. These new data sets from Alcator C-Mod and ASDEX Upgrade are being used to fill key gaps in our understanding of turbulent transport in tokamaks. In particular, this project has results in new results on the topics of the Transport Shortfall, the role of ETG turbulence in tokamak plasmas, profile stiffness, the LOC/SOC transition, and intrinsic rotation reversals. These data are used in a rigorous process of “Transport model validation”, and this group is a world-leader on using turbulence models to design new hardware and new experiments at tokamaks. A correlation electron cyclotron emission (CECE) diagnostic is an instrument used to measure micro-scale fluctuations (mm-scale, compared to the machine size of meters) of electron temperature in magnetically confined fusion plasmas, such as those in tokamaks and stellarators. These micro-scale fluctuations are associated with drift-wave type turbulence, which leads to enhanced cooling and mixing of particles in fusion plasmas and limits achieving the required temperatures and densities for self-sustained fusion reactions. A CECE system can also be coupled with a reflectometer system that measured micro-scale density fluctuations, and from these simultaneous measurements, one can extract the phase between the density (n) and temperature (T) fluctuations, creating an nT phase diagnostic. Measurements of the fluctuations and the phase angle between them are extremely useful for testing and validating predictive models for the transport of heat and particles in fusion plasmas due to turbulence. Once validated, the models are used to predict performance in ITER and other burning plasmas, such as the MIT ARC design. Most recently, data from the newly developed, so-called “CECE diagnostic” [Cima 1995, White 2008] and “nT phase angle measurements” [Haese 1999, White 2010] ]will be combined with data from density fluctuation diagnostics at ASDEX Upgrade to support a long-term program of physics research in turbulence and transport that will allow for more stringent testing and validation of gyrokinetic turbulent-transport codes. This work directly impacts the development of predictive transport models in the U.S. FES program, such as TGLF, developed by General Atomics, which are used to predict performance in ITER and other burning plasma devices as part of advancing the development of fusion energy sciences.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

MCNP5 Criticality Validation and Bias for Intermediate Enriched Uranium Systems

The purpose of this analysis is to validate the Monte Carlo N-Particle 5 (MCNP5) code Version 1.40 (LA-UR-03-1987, 2005) and its cross-section database for k-code calculations of intermediate enriched uranium systems on INTEL ® processor based PC's running any version of the WINDOWS operating system. Configurations with intermediate enriched uranium were modeled with the moderator range of 39≤HlFissile≤1438. See Table 2-1 for brief descriptions of selected cases and Table 3-1 for the range of applicability for this validation. A total of 167 input cases were evaluated including bare and reflected systems in a single body or arrays. The 167 cases were taken directly from the previous (Version 4C [Lan 2005]) validation database. Section 2.0 list data used to calculate k-effective (k eff ) for the 167 experimental criticality benchmark cases using the MCNP5 code v1.40 and its cross-section database. Appendix B lists the MCNP cross-section database entries validated for use in evaluating the intermediate enriched uranium systems for criticality safety. The dimensions and atom densities for the intermediate enriched uranium experiments were taken from NEA/NSC/DOC(95)03, September 2005, which will be referred to as the benchmark handbook throughout the report. For these input values, the experimental benchmark k eff is approximately 1.0. The MCNP validation computer runs ran to an accuracy of approximately ±0.001. For the cases where the reported benchmark k eff was not equal to 1.0000 the MCNP calculational results were normalized. The difference between the MCNP validation computer runs and the experimentally measured k eff is the MCNP5 v1.40 bias. The USLSTATS code (ORNL 1998) was utilized to perform the statistical analysis and generate an acceptable maximum k eff limit for calculations of the intermediate enriched uranium type systems.

42 ENGINEERING↗

OECD-NEA Expert Group on Multi-Physics Experimental Data, Benchmarks and Validation

High-fidelity, multi-physics modeling and simulation (M&S) tools are being developed and utilized for a variety of applications in nuclear science and technology and show great promise in their abilities to reproduce observed phenomena for many applications. Even with the increasing fidelity and sophistication of coupled multi-physics M&S tools, the underpinning models and data still need to be validated against experiments that may require a more complex array of validation data because of the great breadth of the time, energy and spatial domains of the physical phenomena that are being simulated. The Expert Group on Multi-Physics Experimental Data, Benchmarks and Validation (MPEBV) of the Nuclear Energy Agency (NEA) of the Organization for Economic Cooperation and Development (OECD) was formed to address the challenges with the validation of such tools. The work of the MPEBV expert group is shared among three task forces to fulfill its mandate and specific exercises are being developed to demonstrate validation principles for common industrial challenges. This paper describes the overall mission of the group, the specific objectives of the task forces, the linkages among the task forces, and the development of a validation exercise that focuses on a specific reactor challenge problem.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

Validation of Numerical Models of the Offshore Wind Turbine from the Alpha Ventus Wind Farm Against Full-Scale Measurements Within OC5 Phase III: Preprint

The main objective of the Offshore Code Comparison Collaboration Continuation, with Correlation (OC5) project, is validation of aero-hydro-servo-elastic simulation tools for offshore wind turbines (OWTs) through comparison of simulated results to the response data of physical systems. Phase III of the OC5 project validates OWT models against the measurements recorded on Senvion 5M wind turbine supported by the OWEC Quattropod from the alpha ventus offshore wind farm. The following operating conditions of the wind turbine were chosen for the validation: (1) Idling below the cut-in wind speed; (2) Rotornacelle assembly (RNA) rotation maneuver below the cut-in wind speed; (3) Power production below and above the rated wind speed; and (4) Shut-down. A number of validation load cases were defined based on these operating conditions. The following measurements were used for validation: (1) Strains and accelerations recorded on the support structure; (2) Pitch, yaw, and azimuth angles, generator speed, and electrical power recorded from the RNA. Strains were not directly available from the majority of the OWT simulation tools. Therefore, strains were calculated based on out-of-plane bending moments, axial forces, and geometrical properties of the structural members. Also, a number of issues arose during the validation: (1) The need for a thorough quality check of sensor measurements; (2) The sensitivity of the turbine loads to the controller and airfoil properties, which were only approximated in the modeling approach; (3) The importance of estimating and applying an appropriate damping value for the structure; and (4) The importance of wind characteristics beyond turbulence on the loads. The simulation results and measurements were compared in terms of time series, discrete Fourier transforms (DFTs), power spectral densities (PSDs), probability density functions (PDFs) of strains and accelerometers. A good match was achieved between the measurements and models set up by OC5 Phase III participants.

17 WIND ENERGY↗

Key Constructs for Reasonable Confidence in System Validation Programs

This paper presents four key constructs developed in preparing IEEE Std P2411TM, Human Factors Engineering Guide for the Validation of System Designs and Integrated Systems Operations at Nuclear Facilities for submission. These constructs reflect judgments that responsible parties make in order to bring validation to closure. Such judgments weigh competing concerns and cost-benefits for stakeholders. Improved processes and methods may support such judgments but cannot alone eliminate uncertainty. Stating and clarifying these constructs aims to reduce uncertainty in the validation process, to better prepare the implementers and reviewers of future validation programs, both in the nuclear industry and beyond. Reasonable Confidence – A proof standard, as used to reach conclusions in a legal case. Legal proof standards are bounded between “preponderance of evidence” and “beyond reasonable doubt”. Reasonable confidence is comparable to an intermediate legal proof standard of “clear and convincing evidence.” Representative Test Set – A collection of scenarios of sufficient variety to represent the anticipated range of operational conditions, events, evolutions, and activities for validating the system(s) under test. Dispositive vs. Diagnostic Criteria – Relevant performance criteria are placed in one of two categories. Dispositive criteria are assessed, without exception, to determine whether a validation test passes or fails overall. Diagnostic criteria are assessed, subject to justified exception, to evaluate the quality or degree of some particular aspect of performance. Repeatability – Consistency in achieving passing results is to be demonstrated for each scenario in the representative test set; thus, the minimum number of formal repetitions for each scenario is two.

99 GENERAL AND MISCELLANEOUS↗

Validation and Independent Uncertainty Analysis of the MIX-SOL-THERM-003 ICSBEP Benchmark

The International Criticality Safety Benchmark Evaluation Project (ICSBEP) was started in 1992 by the United States Department of Energy and later in 1995 became an international project with contributions from 22 countries. The project is now organized by the OECD (Organisation for Economic Co-operation and Development) Nuclear Energy Agency (NEA). In its most recent iteration, the ICSBEP handbook contains over five thousand evaluations of critical, near-critical, and subcritical experiments conducted in facilities all around the world. These benchmarks serve as valuable information for criticality safety engineers who can use them to validate calculation techniques and establish minimum subcritical margins for operations with fissionable materials. The benchmarks in the handbook are categorized by their fissile material composition, material form (oxide, solution, or metal), and fission energy spectra. This is especially useful for those looking for benchmarks similar to a system they are working on to compare methods and identify trends. The ICSBEP Handbook Uncertainty Guide is document outlining recommended practices and methods for determining uncertainties in these benchmarks. Quantifying these uncertainties thoroughly is crucial as it allows a higher degree of confidence that data used from them is valid and relevant. The guide stresses the importance of a thorough and well documented uncertainty analysis when evaluating an experiment. All measured values of a system, whether they be dimensions or material compositions, have a certain amount of uncertainty associated with them and can be analyzed one by one to determine their effects on the system. Many evaluated benchmarks in the handbook present this in detail, however some do not, mostly earlier evaluations performed in the 1990’s and early 2000’s. Recently at Los Alamos National Laboratory (LANL), the Nuclear Criticality Safety Division (NCSD) of LANL has been validating MCNP6.2 ® input files of criticality benchmarks for use by Whisper, a criticality safety code developed at LANL. This effort is also part of the OECD NEA Working Party on International Nuclear Data Evaluation Co-operation (WPEC) Subgroup 45, also known as Validation of Nuclear Data Libraries (VaNDaL). The goal of VaNDaL is to compile a set of validated simulation inputs for use in validating nuclear data and simulation codes. As part of these efforts, one of the benchmarks reviewed was the MIX SOL-THERM-003 ICSBEP benchmark. This paper provides an independent uncertainty analysis of this benchmark experiment.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Consideration for the application of artificial neural networks to criticality code validation - 14270

The artificial neural network (ANN) is a consistent example for non-parametric parameter estimation and regression, with powerful universal approximation properties, that is distribution independent. ANN application to criticality code validation centers on aligning mathematical relationships between code validation parameters and neural network prediction for those parameter values. Two specific goals are: (1) an appropriate ANN architecture to determine validation parameter values, and (2) bootstrapping individual ANNs as a 'Prediction Interval' (PI) technique to estimate the Calculation Margin and Upper Subcritical Limit. The individual ANN workflow is segregated into Training, Validation (network), and Testing Sets, where the network predictive properties and performance are expressed in the latter set. The Bootstrap ANN replicates a large set of individual ANNs (taken from the original data distribution) where (1-α/2)100% PI estimates can be determined. Results from the ANN method are compared to a variety of validation examples: a 'toy example' from ANSI/ANS-8.24, Appendix D, a variety of methods noted in a Nuclear Criticality Safety Division 2009 paper, and several validation campaigns for highly enriched uranium and plutonium systems. The ANN method is robust and can be applied to single or multivariate inputs. A trained ANN can provide both an average estimate (e.g., single value) and an estimate over a range of independent variable values where the kUSL can be formed in a regression relationship suitable for extrapolation. The input dependence has been tested, ranging from 15 to over 400 sample size distributions. An expanded Shapiro-Wilk test (for sample sizes up to 5000) was implemented to confirm that the ANN method is transparent to distribution function. The Principle Component Analysis method was applied as an input preprocessing step embedded within the ANN in order to reduce dimensionality and correlation for multivariable inputs. This can potentially aid in determining accurate margins over applying arbitrary margins. The automatic optimization of the Bootstrap ANN PI is left for future consideration. (authors)

42 ENGINEERING↗

Connect microstructure evolution to property degradation with validated simulation

Reliable material performance is required for plasma-facing material (PFM) candidates. Previous research has shown that plasma and neutron radiation exposure induces microstructural changes in PFMs; changes in thermal and electrical conductivities and in material hardening and embrittlement were also observed after neutron irradiation. These material property changes will negatively impact the performance of the PFMs in a fusion reactor. Despite the well-known connection between material microstructure, properties, and performance, there is a need for validated modeling capabilities connecting PFM property degradation with microstructural evolution under fusion-relevant conditions. We are developing a simulation capability to couple plasma-induced microstructural evolution to material property degradation. Our approach relies on deliberate mapping between individual simulation models and experimental characterization for validation. The open-source Multiphysics Object-Oriented Simulation Environment (MOOSE) software was used for this simulation capability development. A MOOSE phase-field model was coupled with the cluster dynamics code, Xolotl, to predict microstructural evolution. Microstructure characterization techniques, including scanning electron microscopy (SEM), transmission electron microscopy (TEM), and laser scanning confocal microscopy (LSCM) are used to validate these microstructural evolution simulations. Calculation of thermal and electrical conductivities with first principles simulations was performed for bulk material and for grain boundaries; these results are used within MOOSE models to calculate effective thermal and electrical conductivities as a function of grain characteristics. Thermoreflectance and four-probe techniques were employed to measure the thermal and electrical conductivities, respectively. A MOOSE crystal plasticity model was adapted to predict microstructure-sensitive deformation behavior, and X-ray diffraction (XRD) was used to collect bulk dislocation density data for validation. After individual simulation validation, these models are coupled to predict material property changes resulting from plasma exposure. We focused here on an experimental design to emphasize the separate effects of moderate thermal loads and plasma exposure using tungsten. Annealing of tungsten was performed under a protective environment for temperatures ranging from 500$^o$C to 1500$^o$C. The plasma exposure was completed in the Tritium Plasma Experiment at Idaho National Laboratory under a deuterium flux of 1e22 $\frac{D}{m^2s}$. This incremental approach is employed to build confidence in the modeling capability: separate-effects tests ensure that the models capture key mechanisms from single environmental conditions before predicting PFM property degradation under combined loads. We will show our early results from coupling these simulation models to predict PFM property changes from microstructural evolution. Comparisons of the simulation results with preliminary validation data will be discussed.

36 - MATERIALS SCIENCE↗

Evaluating wind speed and power forecasts for wind energy applications using an open-source and systematic validation framework

Building on the verification and validation work developed under the Second Wind Forecast Improvement Project, this work exhibits the value of a consistent procedure to evaluate wind power forecasts. We established an open-source Python code base tailored for wind speed and wind power forecast validation, WE-Validate. The code base can evaluate model forecasts with observations in a coherent manner. To demonstrate the systematic validation framework of WE-Validate, we designed and hosted a forecast evaluation benchmark exercise. We invited forecast providers in industry and academia to participate and submit forecasts for two case studies. We then evaluated the submissions with WE-Validate. Our findings suggest that ensemble means have reasonable skills in time series forecasting, whereas they are often inferior to single ensemble members in wind ramp forecasting. Adopting a voting scheme in ramp forecasting that allows ensemble members to detect ramps independently leads to satisfactory skill scores. Throughout this document, we also emphasize the importance of using statistically robust and resistant metrics as well as equitable skill scores in forecast evaluation.

17 WIND ENERGY↗

A New Generation of Crystallographic Validation Tools for the Protein Data Bank

This report presents the conclusions of the X-ray Validation Task Force of the worldwide Protein Data Bank (PDB). The PDB has expanded massively since current criteria for validation of deposited structures were adopted, allowing a much more sophisticated understanding of all the components of macromolecular crystals. The size of the PDB creates new opportunities to validate structures by comparison with the existing database, and the now-mandatory deposition of structure factors creates new opportunities to validate the underlying diffraction data. These developments highlighted the need for a new assessment of validation criteria. The Task Force recommends that a small set of validation data be presented in an easily understood format, relative to both the full PDB and the applicable resolution class, with greater detail available to interested users. Most importantly, we recommend that referees and editors judging the quality of structural experiments have access to a concise summary of well-established quality indicators.

59 BASIC BIOLOGICAL SCIENCES↗