Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “verification & validation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 415 records · Page 23

Verification and Validation of START: A Spent Nuclear Fuel Routing and Decision Support Tool

The Stakeholder Tool for Assessing Radioactive Transportation (START) is a web-based geospatial decision-support tool being developed by the US Department of Energy’s Office of Integrated Waste Management (IWM) to support federal interim storage for spent nuclear fuel (SNF) and associated transportation. START provides many functions for the IWM program including: serving as a communications tool for conveying geospatial data and information, an options analysis tool for exploring potential transport modes and routes for transporting SNF from nuclear power plants to future federal interim storage facilities, an emergency response planning tool for Tribes and States to identify training needs along potential SNF transport corridors, an environmental analysis tool for estimating potential radiation dose exposure from incident-free and incident-case SNF transport conditions, and a systems analysis support tool providing route-related inputs for system throughput analysis. As part of the START development process, a verification and validation (V&V) effort is being undertaken. In the initial V&V phase, several outputs of the START tool were checked such as the total distance, population and population densities within the buffer zone, and incident free dose. The V&V process is fluid as it will be utilized after each version change to ensure that the core functionalities of the tool are maintained and the results are consistent with the previous versions. Efforts have also been put into developing scripts to aid in the process of automating certain sections of the V&V work. As some of the V&V efforts use Environmental Systems Research Institute’s (ESRI) tools upon which the START framework is built, the START data were compared against outputs from tools like Quantum Geographic Information System (QGIS) for buffer zone populations and route lengths to ensure independence of the V&V process. Good agreement was observed between the START results and the independent V&V studies with the majority of the differences falling between 1% and 5% for populations within the buffer zone and route distance. This presentation describes the development and design of the START tool, V&V methods employed for various metrics of interest and their respective results, and future plans.

START, transportation, GIS, V&V↗

Unmanned Aircraft Systems Minimum Operations Performance Standards End-to-End Verification and Validation (E2-V2) Simulation

As Unmanned Aircraft Systems (UAS) make their way to mainstream aviation operations within the National Airspace System (NAS), research efforts are underway to develop a safe and effective environment for their integration into the NAS. Detect and Avoid (DAA) systems are required to account for the lack of "eyes in the sky" due to having no human on-board the aircraft. The current NAS relies on pilot's vigilance and judgement to remain Well Clear (CFR 14 91.113) of other aircraft. RTCA SC-228 has defined DAA Well Clear (DAAWC) to provide a quantified Well Clear volume to allow systems to be designed and measured against. Extended research efforts have been conducted to understand and quantify system requirements needed to support a UAS pilot's ability to remain well clear of other aircraft. The efforts have included developing and testing sensor, algorithm, alerting, and display requirements. More recently, sensor uncertainty and uncertainty mitigation strategies have been evaluated. This paper discusses results and lessons learned from an End-to-End Verification and Validation (E2-V2) simulation study of a DAA system representative of RTCA SC-228's proposed Phase I DAA Minimum Operational Performance Standards (MOPS). NASA Langley Research Center (LaRC) was called upon to develop a system that evaluates a specific set of encounters, in a variety of geometries, with end-to-end DAA functionality including the use of sensor and tracker models, a sensor uncertainty mitigation model, DAA algorithmic guidance in both vertical and horizontal maneuvering, and a pilot model which maneuvers the ownship aircraft to remain well clear from intruder aircraft, having received collective input from the previous modules of the system. LaRC developed a functioning batch simulation and added a sensor/tracker model from the Federal Aviation Administration (FAA) William J. Hughes Technical Center, an in-house developed sensor uncertainty mitigation strategy, and implemented a pilot model similar to one from the Massachusetts Institute of Technology's Lincoln Laboratory (MIT/LL). The resulting simulation provides the following key parameters, among others, to evaluate the effectiveness of the MOPS DAA system: severity of loss of well clear (SLoWC), alert scoring, and number of increasing alerts (alert jitter). The technique, results, and lessons learned from a detailed examination of DAA system performance over specific test vectors and encounter cases during the simulation experiment will be presented in this paper.

Ghatas, Rania W.↗

Verification and Validation Testing of the Parachute Decelerator System Prior to the First Supersonic Flight Dynamics Test for the Low Density Supersonic Decelerator Program

The Parachute Decelerator System (PDS) is comprised of all components associated with the supersonic parachute and its associated deployment. During the Supersonic Flight Dynamics Test (SFDT), for the Low Density Supersonic Decelerators Program, the PDS was required to deploy the supersonic parachute in a defined fashion. The PDS hardware includes three major subsystems that must function together. The first subsystem is the Parachute Deployment Device (PDD), which acts as a modified pilot deployment system. It is comprised of a pyrotechnic mortar, a Kevlar ballute, a lanyard actuated pyrotechnic inflation aid, and rigging with its associated thermal protection material (TPS). The second subsystem is the supersonic parachute deployment hardware. This includes all of the parachute specific rigging that includes the parachute stowage can and the rigging including TPS and bridle stiffeners for bridle management during deployment. The third subsystem is the Supersonic Parachute itself, which includes the main parachute and deployment bags. This paper summarizes the verification and validation of the deployment process, from the initialization of the PDS system through parachute bag strip that was done prior to the first SFDT.

Gallon, John C.↗

Verification and Validation Challenges for Adaptive Flight Control of Complex Autonomous Systems

Autonomy of aerospace systems requires the ability for flight control systems to be able to adapt to complex uncertain dynamic environment. In spite of the five decades of research in adaptive control, the fact still remains that currently no adaptive control system has ever been deployed on any safety-critical or human-rated production systems such as passenger transport aircraft. The problem lies in the difficulty with the certification of adaptive control systems since existing certification methods cannot readily be used for nonlinear adaptive control systems. Research to address the notion of metrics for adaptive control began to appear in the recent years. These metrics, if accepted, could pave a path towards certification that would potentially lead to the adoption of adaptive control as a future control technology for safety-critical and human-rated production systems. Development of certifiable adaptive control systems represents a major challenge to overcome. Adaptive control systems with learning algorithms will never become part of the future unless it can be proven that they are highly safe and reliable. Rigorous methods for adaptive control software verification and validation must therefore be developed to ensure that adaptive control system software failures will not occur, to verify that the adaptive control system functions as required, to eliminate unintended functionality, and to demonstrate that certification requirements imposed by regulatory bodies such as the Federal Aviation Administration (FAA) can be satisfied. This presentation will discuss some of the technical issues with adaptive flight control and related V&V challenges.

Nguyen, Nhan T.↗

Real-Time Simulation for Verification and Validation of Diagnostic and Prognostic Algorithms

To verify that a health management system (HMS) performs as expected, a virtual system simulation capability, including interaction with the associated platform or vehicle, very likely will need to be developed. The rationale for developing this capability is discussed and includes the limited capability to seed faults into the actual target system due to the risk of potential damage to high value hardware. The capability envisioned would accurately reproduce the propagation of a fault or failure as observed by sensors located at strategic locations on and around the target system and would also accurately reproduce the control system and vehicle response. In this way, HMS operation can be exercised over a broad range of conditions to verify that it meets requirements for accurate, timely response to actual faults with adequate margin against false and missed detections. An overview is also presented of a real-time rocket propulsion health management system laboratory which is available for future rocket engine programs. The health management elements and approaches of this lab are directly applicable for future space systems. In this paper the various components are discussed and the general fault detection, diagnosis, isolation and the response (FDIR) concept is presented. Additionally, the complexities of V&V (Verification and Validation) for advanced algorithms and the simulation capabilities required to meet the changing state-of-the-art in HMS are discussed.

Aguilar, Robet↗

Software Verification and Validation Plan Activities, 2011 – SAPHIRE 8 Quality Assurance

SV&V Updates through FY11 An official update was released in February 2011. This update was to add a new automated test, to revise the SV&V wording to indicate the operational life cycle that SAPHIRE 8 has entered, and to provide design documentation for Inspection Planning Reports (which evolved into the Plant Risk Information e-Book (PRIB)) and Analysis Speed Improvements. Another update to the SV&V will be released in conjunction with this report in September 2011 to include the following: two new automated tests, the inclusion of metrics into the testing regime and references to the addition of the Preliminary Design Review and Critical Design Review processes.

42 ENGINEERING↗

Stochastic Particle Real Time Analyzer (SPARTA) Validation and Verification Suite

This report presents the test cases used to verify, validate and demonstrate the features and capabilities of the first release of the 3D Direct Simulation Monte Carlo (DSMC) code SPARTA (Stochastic Real Time Particle Analyzer). The test cases included in this report exercise the most critical capabilities of the code like the accurate representation of physical phenomena (molecular advection and collisions, energy conservation, etc.) and implementation of numerical methods (grid adaptation, load balancing, etc.). Several test cases of simple flow examples are shown to demonstrate that the code can reproduce phenomena predicted by analytical solutions and theory. A number of additional test cases are presented to illustrate the ability of SPARTA to model flow around complicated shapes. In these cases, the results are compared to other well-established codes or theoretical predictions. This compilation of test cases is not exhaustive, and it is anticipated that more cases will be added in the future.

97 MATHEMATICS AND COMPUTING↗

Stochastic Particle Real Time Analyzer (SPARTA) Validation and Verification Suite

This report presents the test cases used to verify, validate and demonstrate the features and capabilities of the first release of the 3D Direct Simulation Monte Carlo (DSMC) code SPARTA (Stochastic Real Time Particle Analyzer). The test cases included in this report exercise the most critical capabilities of the code like the accurate representation of physical phenomena (molecular advection and collisions, energy conservation, etc.) and implementation of numerical methods (grid adaptation, load balancing, etc.). Several test cases of simple flow examples are shown to demonstrate that the code can reproduce phenomena predicted by analytical solutions and theory. A number of additional test cases are presented to illustrate the ability of SPARTA to model flow around complicated shapes. In these cases, the results are compared to other well-established codes or theoretical predictions. This compilation of test cases is not exhaustive, and it is anticipated that more cases will be added in the future.

97 MATHEMATICS AND COMPUTING↗

The NASA Orbital Debris Engineering Model 3.1: Development, Verification, and Validation

The NASA Orbital Debris Program Office has developed the Orbital Debris Engineering Model (ORDEM) primarily as a tool for spacecraft designers and other users to understand the long-term risk of collisions with orbital debris. The newest version, ORDEM 3.1, incorporates the latest and highest fidelity datasets available to build and validate representative orbital debris populations encompassing low Earth orbit (LEO) to geosynchronous orbit (GEO) altitudes for the years 2016-2050. ORDEM 3.1 models fluxes for object sizes > 10 μm within or transiting LEO and > 10 cm in GEO. The deterministic portion of the populations in ORDEM 3.1 is based on the U.S. Space Surveillance Network (SSN) catalog, which provides coverage down to approximately 10 cm in LEO and 1 m in GEO. Observational datasets from radar, in situ, and optical sources provide a foundation from which the model populations are statistically extrapolated to smaller sizes and orbit regions that are not well-covered by the SSN catalog, yet may pose the greatest threat to operational spacecraft. Objects in LEO ranging from approximately 5 mm to 10 cm are modeled using observational data from ground-based radar, namely the Haystack Ultrawideband Satellite Imaging Radar (HUSIR – formerly known as Haystack). The LEO population smaller than approximately 3 mm in size is characterized based on a reanalysis of in situ data from impacts to the windows and radiators of the U.S. Space Transportation System orbiter vehicle, i.e., the Space Shuttle. Data from impacts on the Hubble Space Telescope are also used to validate the sub-millimeter model populations in LEO. Debris in GEO with sizes ranging from 10 cm to 1 m is modeled using optical measurement data from the Michigan Orbital DEbris Survey Telescope (MODEST). Specific, major debris-producing events, including the Fengyun-1C, Iridium 33, and Cosmos 2251 debris clouds, and unique populations, such as sodium-potassium droplets, have been re-examined and are modeled and added to the ORDEM environment separately. The debris environment greater than 1 mm is forecast using NASA’s LEO-to- GEO ENvironment Debris model (LEGEND). Future explosions of intact objects and collisions involving objects greater than 10 cm are assessed statistically, and the NASA Standard Satellite Breakup Model is used to generate fragments from these events. Fragments smaller than 10 cm are further differentiated based on material density categories, i.e., high-, medium-, and low-density, to better characterize the potential debris risk posed to spacecraft. The future projection of the sub-millimeter environment is computed using a special small-particle degradation model where small particles are created from intact spacecraft and rocket bodies. This work discusses the development, features, and capabilities of the ORDEM 3.1 model; the ne new data analyses used to build the model populations; and sample verification and validation results.

Matney, M.↗

Assessment of verification and validation status - RELAP5-3D and RAVEN

The United States nuclear industry is facing a strong challenge to ensure maximum safety while enhancing economic benefit. Safety is a key parameter to all aspects related to light water reactor (LWR) nuclear power plants (NPPs), especially cost savings. Since the goal is to extend the lifetimes of these NPPs, the traditional deterministic safety concept may not guarantee a current economic asset. The Light Water Reactor Sustainability (LWRS) Program has been promoting a wide range of research and development (R&D) in this field to maximize the safety, economics, and performance of these NPPs through improved scientific understanding. One of the best practices to achieve this goal is to identify and optimize safety margins, which can lead to cost reduction. To do this, under the LWRS framework, the Risk-Informed Systems Analysis (RISA) Pathway will focus on the optimization of safety margin and minimization of uncertainties to ensure both safety and economics at the highest level. The RISA Pathway will provide enhanced capabilities for analyzing and characterizing LWR systems performance by developing and demonstrating methods, tools, and data to enable risk-informed margins management (RIMM). The goals of the RISA Pathway are twofold: (1) deploy the risk-informed tools and methods that enable better representation of safety margins and factors that contribute to cost and safety; and (2) conduct advanced risk assessment applications with industry to support margin management strategies that enable more cost-effective plant operation. The tools and methods provided by the RISA Pathway will support effective margin management for both active and passive safety systems, structures, and components (SSC) of an NPP. The tools and methods used in the RISA Pathway should have high confidence and highest technical maturity for and implementation to industry at its current setting. They should also have a capability to support risk-informed decision making for both probabilistic and deterministic elements of safety. The RISA Pathway will, therefore, perform a comprehensive assessment of verification and validation (V&V) status of RISA Toolkit to enhance credibility RISA Toolkit which be used by industry. This report summarizes assessment technical maturity of RISA thermal-hydraulics accident analysis code RELAP5-3D and multi-purpose probabilistic risk analysis tool RAVEN V&V status including specific information of the tool such as capability and features, quality assurance program, developer/independent V&V record, separation/integral tests history, user documents, and feedback.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

Performance and wake flow characterization of a 1:8.7-scale reference USDOE MHKF1 hydrokinetic turbine to establish a verification and validation test database

As hydrokinetic turbine technologies continue to advance towards commercialization, public datasets on the performance characteristics for these devices and their flow field effects are invaluable to advance our understanding of these technologies and to validate analytical and numerical models. The Applied Research Laboratory at The Pennsylvania State University (ARL Penn State) collaborated with Sandia National Laboratories and the University of California at Davis to design, fabricate (at a 1:8.7 scale), and experimentally test a novel hydrokinetic turbine rotor design to provide an open platform and dataset for further study and development. The water tunnel test of this three-bladed, horizontal-axis rotor recorded power production, blade loading, the near-wake flow, cavitation effects, and noise generation. These state-of-the-art measurements demonstrate much of the complex physics associated with the flow through an unducted, horizontal-axis turbine, and they elucidate the performance characteristics and flow field effects at an unprecedented fidelity, accuracy and resolution. Measurements of powering coefficients (power, torque and thrust) as a function of tip-speed-ratio were performed. The dataset also includes unsteady measurements of driveshaft loading, blade strain, tower pressures, and radiated noise. Detailed flow mapping using laser Doppler velocimetry, and planar and stereo particle image velocimetry includes measurements of mean velocity and Reynolds stresses. Although the wake measurements are limited to less than half a diameter, they reveal the complex flow patterns in the near-wake structure of the rotor. The full database, available at the United States Department of Energy’s marine and hydrokinetic data repository, includes tunnel and model Computer Aided Design geometry files and inflow data sufficient for a “Model-the-Test” computational Verification and Validation study.

13 HYDRO ENERGY↗

Development of the INL Thermal Energy Distribution System (TEDS) in the Modelica Eco-System for Validation and Verification

This report provides an overview on the development of the Thermal Energy Distribution System (TEDS) in the Modelica process model eco-system at Idaho National Laboratory as part of the integrated energy system initiative. Model development has led to the creation of a dynamic process model of the experimental TEDS facility housed within the Engineering Science Laboratory (ESL) at INL. The model was then used during the pre-construction phase of the experimental effort to inform experimental design (insulation requirements, bypass line placement, expected performance of components) and to test innovative control schemes prior to the initial operation. The TEDS model developed in Modelica includes the primary components of the TEDS experimental unit including: a 200kW Chromalox heater, a single-tank packed bed thermal energy storage system filled with 0.125 inch Alumina (Al 2 O 3 ), an ethylene-glycol to Therminol-66 heat exchanger, system piping, six control valves, and all associated temperature, pressure, and mass flow sensors. The model does not include nitrogen fill gas tanks, or associated overfill tanks, as these are not part of standard system control. Two simulation sets were run. The first is a five-hour test that operates as a shakedown test for the facility. This simulation puts the facility through all five potential operating modes and showcases the ability of valving, control sensors, and component controllers to meet the system demands. The second case imposes a typical summer day demand on the system from a region with mixed commercial and residential electrical needs where the generator alone cannot meet peak demand but instead requires the thermal storage unit to act as a peaking unit. Over the course of the fifteen-hour discharge cycle heat losses account for a heat loss of approximately 15°C in outlet temperature. Such heat losses would lead to a decrease in overall system efficiency and may lead to additional design changes for such designs in the future that may include a topping heater. Through commencement of this work, a systems level model of TEDS with associated control systems, sensors, piping diameters, and component capabilities has been created. This model has been utilized in the pre-experimental phase to inform system design, insulation thicknesses, and potential control schemes to operate the system effectively and safely. Once TEDS begins operation, this model will be refined, tuned, and used for validation and verification purposes.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Fully implicit 1D radiation hydrodynamics: Validation and verification

A fully implicit finite difference scheme has been developed to solve the hydrodynamic equations coupled with radiation transport. Solution of the time-dependent radiation transport equation is obtained using the discrete ordinates method and the energy flow into the Lagrangian meshes as a result of radiation interaction is fully accounted for. A tridiagonal matrix system is solved at each time step to determine the hydrodynamic variables implicitly. The results obtained from this fully implicit radiation hydrodynamics code in the planar geometry agrees well with the scaling law for radiation driven strong shock propagation in aluminium. For the point explosion problem the self similar solutions are compared with results for pure hydrodynamic case in spherical geometry. Results obtained when radiation interaction is also accounted agree with those of point explosion with heat conduction for lower input energies. Having, thus, benchmarked the code, self convergence of the method w.r.t. time step is studied in detail for both the planar and spherical problems. Spatial as well as temporal convergence rates are {approx_equal}1 as expected from the difference forms of mass, momentum and energy conservation equations. This shows that the asymptotic convergence rate of the code is realized properly.

97 MATHEMATICS AND COMPUTING↗

Simulation verification techniques study

Results are summarized of the simulation verification techniques study which consisted of two tasks: to develop techniques for simulator hardware checkout and to develop techniques for simulation performance verification (validation). The hardware verification task involved definition of simulation hardware (hardware units and integrated simulator configurations), survey of current hardware self-test techniques, and definition of hardware and software techniques for checkout of simulator subsystems. The performance verification task included definition of simulation performance parameters (and critical performance parameters), definition of methods for establishing standards of performance (sources of reference data or validation), and definition of methods for validating performance. Both major tasks included definition of verification software and assessment of verification data base impact. An annotated bibliography of all documents generated during this study is provided.

Schoonmaker, P. B.↗

Initial 1.5D BISON Simulation Verification and Validation CASL FY17 Letter Report

This milestone report describes the current efforts to improve the usability of the recently developed 1.5 dimensional capability in BISON, the initial verification of this BISON capability, and initial validation of the 1.5 dimensional models against existing 2 dimensional axisymmetric full length light water reactor (LWR) fuel rods. The 1.5 dimensional (1.5D) capability in BISON was developed to enable the simulation of LWR fuel rods in a less computationally demanding simulation formulation while retaining key physics of interest. BISON was originally developed to solve 2D axisymmetric and 3D models, and this original development focus allows BISON to examine a wider range of fuel types and geometrically complexity than can be solved with lower dimensional frameworks. As the set of problems which BISON is used to solve expands, the software design requirements continue to evolve. Coupling physics in a higher dimensional geometry is not always required to resolve the physics of interest, such as the modeling of radial heat transfer and radial displacement. BISON, built on the dimension-agnostic MOOSE framework, is able to solve problems in 3D, 2D, and 1D with only minor modifications to the material definition code so that the appropriate geometrical representation can be applied in different problems of interest. The motivation for the development and validation of the BISON 1.5D capability is the CASL program's need for a simplified and computationally efficient version of BISON for use in the large multi-physics VERA simulations which include hundreds of individual fuel rods. The original development of the BISON 1.5D simulation capability is described in the CASL FY17 Letter Report from January 2017, and this report documents the continuing work identified in the previous report. The improvements to the usability of the 1.5D capability through input file simplifications, the results of the verification simulations, the results of the initial validation against the 2D assessment cases in BISON, and the future development needs are included within this report.

1.5D Capability↗