Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Validate”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6
All sourcesNASA NTRSDOE OSTI
NASA NTRSDate not supplied

Identification and Study of Validation Level Test Cases for Computational Modeling of Non-Charring Ablators

Computational modeling of Thermal Protection System (TPS) materials, used for aerospace applications, provides numerous advantages in preliminary selection and design of a heatshield material and shape for atmospheric entry vehicles. However, to serve as a reliable tool for prediction of material thermal and ablative behavior, the modeling approach needs to be validated against real experimental and flight data, preferably at a range of applied conditions. The validation study is typically very complex as it requires reliable measured data not only for the material thermal response and surface recession, but also well characterized environmental conditions. The validation problem becomes even more complex when the material thermal response is dictated by multi-physics effects such as solid conduction, in-depth thermal decomposition, pyrolysis gas flow and chemical reactions. The multi-physics effects complicate not only the modeling effort, but also the experimental measurement for validation of various aspects of the highly coupled problem. In this study, an attempt is made to identify suitable experimental data that could serve as a source for validation of material thermal response modeling tools. To reduce the computational complexity, this study focuses only on non-charring ablators, where the material thermal response could be modeled with a single governing equation for solid conduction and the ablation is limited only to the surface of the material. With a well characterized and publicly available experimental data being sparse, the study is limited in presenting test cases for only three materials: camphor, graphite and FiberForm® in the sequence of increased modeling complexity. Graphite is a commonly used TPS material for aerospace applications, both for leading edges of high-speed vehicles and internal insulation of solid rocket motors. FiberForm® is a porous carbon pre-form used in preparation of the well known PICA material Tran et al. [1996]. Inclusion of camphor into the list is conditioned with the relative simplicity in modeling the material thermal and chemical response and the low-enthalpy flow environment. In addition, camphor has been used as a simple test material for study of flow transition behavior by Stock and Ginoux [1973] and assessment of a heatshield shape change at flight relevant conditions by Rotondi et al. [2022]. In this work, the identified experimental data was extracted from the public literature and test cases that yet have been published. As it was found from the review, not a single test case contains an exhaustive set of data that would validate every aspect of the material physics. However, in the data collected, various aspects of the material behavior can be still validated, such as surface and in-depth temperature, amount of recession and a shape change. The identified experimental data for each case is accompanied with a characterized flow environment and simulated boundary conditions predicted by a Data-Parallel Line Relaxation (DPLR) code Wright et al. [1998]. In addition, material thermal response numerical simulations in each test case are performed with Kentucky Aerothermodynamics and Thermal Response System (KATS-MR) Zibitsker et al. [2022] providing a comparative study and a sanity check for the proposed validation data. Sample results from the performed numerical study are shown below. Figure 1 shows distribution of surface heat flux and pressure values on a hemi-cylinder model made of FiberForm® and tested in HyMETS arc-jet facility. The results are shown for the high pressure condition among the two tests. Flow simulation was performed with DPLR code on a quarter of original geometry. In the figure, the quarter shape was mirrored across zx and xy planes to show the complete distribution. Figure 2 shows the material response results for the high pressure case (7500 Pa), simulated with KATS-MR and a comparison to the experimental data for the surface temperature and shape shape. The simulation was performed on a 2-D slice, extracted in the xy plane at the middle of the sample. Figure 3 shows the material response simulation for the low pressure case (3500 Pa) and a comparison to the experimental data for surface temperature and shape change.

ablation↗
DOE OSTI2025

Thermal Hydraulics Validation Activities in the ART-GCR Campaign

The Advanced Reactor Technologies - Gas Cooled Reactor Campaign (ART-GCR) is part of the United States Department of Energy - Office of Nuclear Energy's Advanced Reactor Technologies program aimed at developing a High Temperature Gas-cooled Reactor (HTGR) which will offer enhancements in safety and efficiency. The Design, Methods, and Validation arm of the ART-GCR program provides experience and advanced tools for HTGR design and analysis. The Methods and Validation arm is developing data to validate predictions of decay heat removal through an extensive experimental campaign the Natural Convection Shutdown Heat Removal Test Facility at Argonne National Laboratory. In-core thermal hydraulics validation is being facilitated through the OECD/NEA's Thermal Hydraulic Code Validation Benchmark for High Temperature Gas-Cooled Reactors using HTTF Data. That benchmark contains code-to-code and code-to-data comparisons for 3 sets of HTGR thermal hydraulics phenomena. This ongoing benchmark activity has 16 participants from 9 countries developing leading insights into HTGR thermal hydraulics code validation. The ART-GCR campaign is also working on the development of state-of-the-art MOOSE-based models of the High Temperature Engineering Test Reactor (HTTR). Through the OECD/NEA's HTTR Loss of Forced Cooling (LOFC) program, the ART-GCR campaign is developing a multiphysics transient code validation benchmark based on 3 HTTR experiments that will provide an opportunity for international collaboration on multiphysics code validation.

22 - GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2023

Validation of Microgrid Algorithms using At-Scale Simulation and Emulation Real-Time (ASSERT) Framework

This paper presents a novel framework for validating microgrid algorithms using a At-Scale Simulation and Emulation Real-Time (ASSERT) setup. The proposed universal microgrid validation framework was developed at Advanced Research on Integrated Energy Systems (ARIES) platform at National Renewable Energy Laboratory in Golden, Colorado, USA. As the number of intercommunicating devices grow in power systems, along with more integration of inverter-based resources, developing and validating microgrid operation and control algorithms is becoming complex and time-consuming task. The ASSERT framework is designed to provide a comprehensive, accurate, and efficient solution for validating microgrid algorithms. It combines at-scale simulations and real-time emulation to create a virtual environment that closely mimics the behavior of a physical microgrids which are characteristic of future smart distribution systems, such as: (1) increasing variability in the physical size of new energy technologies; (2) requiring control large numbers of interconnected devices, and (3) integrating diverse technologies that have not previously worked together. The paper describes the technical details of the ASSERT framework, its validation process, and its benefits over traditional validation methods. The framework's ability to validate the performance of various algorithms is demonstrated through a case study, which shows that it can effectively identify the weaknesses of an algorithm and optimize its performance. Overall, the ASSERT framework is a promising tool that can significantly enhance the development and validation of microgrid algorithms, leading to more reliable and efficient microgrid operation.

at-scale↗
DOE OSTI2020

Performing k eff Validation of As-Loaded Criticality Safety Calculations Using UNF-ST&DARDS: Applicable Experiment Selection

The general method for performing validation of as loaded criticality safety calculations using UNF ST&DARDS is presented in a paper by Clarity, which includes a description of the UNF-ST&DARDS system. Proof-of-principle analyses were performed in the summer of 2019 for MPC-32 dual purpose canisters (DPCs) containing pressurized water reactor (PWR) fuel assemblies. Summaries of these results are presented in this and a companion paper for this conference. The current paper describes the TSUNAMI-IP calculations performed to select applicable experiments for validation of 11 MPC-32 DPCs. The companion paper discusses the TSUNAMI-3D calculations used to generate sensitivity data to support the experiment selections discussed here. Experiment selection is based on the sensitivity/uncertainty (S/U) methods used to validate criticality safety calculations of as-loaded DPCs containing pressurized water reactor (PWR) spent nuclear fuel (SNF). This process has been demonstrated and is summarized in this paper. The approach is similar to that used in NUREG/CR-7109, which provides an approach for validation of PWR burnup credit (BUC), including major and minor actinides and major fission products. The premise of S/U-based validation is that applicable experiments—those having a similar bias to a given application system—will have similar sensitivities for each isotope and reaction in the two systems. It is assumed that cross sections with larger uncertainties are more likely to contain data errors which contribute to the bias. The integral index c k thus propagates the system sensitivities with the nuclear covariance data to calculate a correlation coefficient representing the similarity of the two systems. In this work, a c k value of 0.8 or higher is interpreted as identifying an experiment with sufficient similarity for use in validation. This paper presents a brief summary of the characteristics of the 11 MPC-32 DPCs used in the proof of-principle analysis for as-loaded criticality safety calculation validation and an overview of the critical experiment suite with which each of these DPC models was compared. A summary and discussion of c k results is also presented, followed by conclusions and a discussion of future work to be performed for validation of UNF ST&DARDS as-loaded criticality safety calculations.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2021

ARC Software Validation Work for the FFTF Reactor

Extensive efforts have been carried out at ANL for the verification and validation of the Argonne Reactor Codes (ARC) software package currently used for the design of Versatile Test Reactor (VTR). The ARC software package consists of steady state neutronics and thermal hydraulics modeling capabilities which are being used by the VTR program to develop most of the VTR reactor design details which will be part of the licensing application. It is anticipated that this software will continue to be used for the design work and for initial operations although additional software may be introduced at a later time. The validation work was focused primarily on obtaining validation data consistent with VTR and usable for the ARC software. Because no critical facilities or operating fast spectrum reactors are available to do experiments for the VTR, the next best option is to identify historical experimental data that can be used as validation data. Early on in VTR, the ZPPR-15 set of experiments was identified as good validation data because of 1) the availability and quality of the data, 2) existing staff that are already familiar with the experimental machine and measurements, 3) most of the ZPPR-15 loadings of interest have already been processed into ARC models, and 4) a full uncertainty quantification has already been done for several loadings of ZPPR-15. The FFTF startup and operations data was identified as the most consistent reactor type that has validation data usable for VTR. Finally, the EBR-II fuel depletion measurements were identified as the best available validation data for VTR. It is important to note that both the FFTF and EBR-II reactors typically come with higher uncertainties than the ZPPR. In the frame of the discussed verification and validation efforts, the present document discusses the analysis of selected FFTF measurements included in the benchmark specifications of the International Reactor Physics Experiment (IRPhE) handbook. The FFTF reactor core configurations from the benchmark specification are presented in Section 2. The analysis is performed with the use of the ARC code suite available at ANL for fast reactor studies and is discussed in Section 3. The reactor parameters from the benchmark include criticality, neutron spectra, effective delayed neutron spectra, control rod worth, isothermal temperature coefficient and low energy gamma-ray spectra. The calculated values and the comparison with the experimental data is discussed in Sections 4 to 9 for each considered reactor parameter. Finally, conclusions are presented in Section 10.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
NASA NTRS2000

Earth Science Enterprise Scientific Data Purchase Project: Verification and Validation

This paper presents viewgraphs on the Earth Science Enterprise Scientific Data Purchase Project's verification,and validation process. The topics include: 1) What is Verification and Validation? 2) Why Verification and Validation? 3) Background; 4) ESE Data Purchas Validation Process; 5) Data Validation System and Ingest Queue; 6) Shipment Verification; 7) Tracking and Metrics; 8) Validation of Contract Specifications; 9) Earth Watch Data Validation; 10) Validation of Vertical Accuracy; and 11) Results of Vertical Accuracy Assessment.

Jenner, Jeff↗
NASA NTRS2004

Review of TRMM/GPM Rainfall Algorithm Validation

A review is presented concerning current progress on evaluation and validation of standard Tropical Rainfall Measuring Mission (TRMM) precipitation retrieval algorithms and the prospects for implementing an improved validation research program for the next generation Global Precipitation Measurement (GPM) Mission. All standard TRMM algorithms are physical in design, and are thus based on fundamental principles of microwave radiative transfer and its interaction with semi-detailed cloud microphysical constituents. They are evaluated for consistency and degree of equivalence with one another, as well as intercompared to radar-retrieved rainfall at TRMM's four main ground validation sites. Similarities and differences are interpreted in the context of the radiative and microphysical assumptions underpinning the algorithms. Results indicate that the current accuracies of the TRMM Version 6 algorithms are approximately 15% at zonal-averaged / monthly scales with precisions of approximately 25% for full resolution / instantaneous rain rate estimates (i.e., level 2 retrievals). Strengths and weaknesses of the TRMM validation approach are summarized. Because the dew of convergence of level 2 TRMM algorithms is being used as a guide for setting validation requirements for the GPM mission, it is important that the GPM algorithm validation program be improved to ensure concomitant improvement in the standard GPM retrieval algorithms. An overview of the GPM Mission's validation plan is provided including a description of a new type of physical validation model using an analytic 3-dimensional radiative transfer model.

Smith, Eric A.↗
NASA NTRSDate not supplied

Validation of Decompression Sickness Risk Mitigation Protocols for Planetary Spaceflight Missions

BACKGROUND: Apollo missions used a 100% O2 cabin atmosphere which effectively eliminated the risk of decompression sickness (DCS) during extravehicular activity (EVA) on the moon. NASA’s future missions to the moon and Mars are expected to use nitrox gas mixtures of up to 34% O2, 66% N2, which will reduce flammability risk compared with Apollo, but will necessitate Oxygen prebreathe prior to EVA to reduce DCS risk to acceptable levels. Prebreathe protocols used on the space shuttle and International Space Station are validated for microgravity EVAs, but the significantly increased risk of DCS during equivalent ambulatory EVAs make these protocols inapplicable to planetary EVA. An “exploration atmosphere” of 56.5 kPa (8.2 psia), 34% O2, 66% N2 has been recommended by NASA as a compromise that balances prebreathe duration, hypoxia, and flammability risk, assuming a 29.6 kPa (4.3 psi) spacesuit. However, this atmosphere may not be used for vehicles that do not support frequent EVA, and with commercial providers and international providers expected to provide landers, pressurized rovers, habitats, and spacesuits, different combinations of vehicle and spacesuit atmospheres are possible and will each require validated prebreathe protocols. OVERVIEW: Key components of a multi-year strategic roadmap include: 1) Establish hypobaric chamber facility capable of supporting 8-person EVA prebreathe validation tests at saturation atmospheres up to 36% O2; 2) validate an EVA physical workload simulation for use during prebreathe validation testing; 3) validate the recommended “exploration atmosphere” prebreathe protocol; 4) validate prebreathe protocols for additional atmospheric combinations that bound the most likely potential operating ranges of future vehicles and spacesuits; and 5) update DCS risk estimation models based on results of prebreathe validation studies. DISCUSSION: Details and data from completion of the first two steps of the strategic roadmap will be presented; the third step is currently underway, with pilot results provided in a companion presentation. Steps four and five will require a multi-year series of chamber tests; collaborations are being pursued.

Andrew F. J. Abercromby↗
NASA NTRS2016

Experience with Aero- and Fluid-Dynamic Testing for Engineering and CFD Validation

Ever since computations have been used to simulate aerodynamics the need to ensure that the computations adequately represent real life has followed. Many experiments have been performed specifically for validation and as computational methods have improved, so have the validation experiments. Validation is also a moving target because computational methods improve requiring validation for the new aspect of flow physics that the computations aim to capture. Concurrently, new measurement techniques are being developed that can help capture more detailed flow features pressure sensitive paint (PSP) and particle image velocimetry (PIV) come to mind. This paper will present various wind-tunnel tests the author has been involved with and how they were used for validation of various kinds of CFD. A particular focus is the application of advanced measurement techniques to flow fields (and geometries) that had proven to be difficult to predict computationally. Many of these difficult flow problems arose from engineering and development problems that needed to be solved for a particular vehicle or research program. In some cases the experiments required to solve the engineering problems were refined to provide valuable CFD validation data in addition to the primary engineering data. All of these experiments have provided physical insight and validation data for a wide range of aerodynamic and acoustic phenomena for vehicles ranging from tractor-trailers to crewed spacecraft.

experiment↗
NASA NTRSDate not supplied

AVT-297 Development of a Framework for Validation of Computational Tools

Design of aerospace vehicles relies on system models. These models rely on validation data to ensure accurate representation of the system. This paper provides an introduction to the problem and an overview of a NATO STO Activity, AVT-297, which developed a framework to determine validation experiments required to support system level models. The goals of the research group were to develop a process to identify a validation database for intended uses of a vehicle system, demonstrate the process on vehicles and document the process. Applying this process to multiple intended uses and collecting this set of validation databases identifies a set of validation experiments that could be applied to the design of multiple vehicles, further reducing the cost and time to develop air and naval vehicles. Four subgroup teams focused on different aspects of the validation database process: two focused on different aspects of a missile, and two focused on different aspects of a fixed-wing mobility transport class vehicle. This resulted in three different processes to decompose the multi-disciplinary problem to identify critical validation data. All four of the teams demonstrated their process on a sample problem demonstrating the benefit of the process.

Validation↗
DOE OSTI2021

An Application for Validation of Power Distribution System Models in an ADMS Environment

An accurate model of a power distribution system is the foundation for model-based applications that ensure efficient and reliable grid operation in an advanced distribution management system (ADMS) environment. However, these models are error-prone and comprehensive model validation is challenging due to lack of standards-based systems, data originating from disparate databases and other sources, and the constantly evolving nature of modern power distribution systems. In this paper, a novel framework for comprehensive model validation is described. The proposed application, the Model Validator, ensures that a model is both consistent and feasible by validating the derivative static and operational network model. A modular architecture for the application has been implemented and integrated with an open-source standards-based platform for ADMS application development, GridAPPS-D, allowing new validation capability to be added with minimal time and effort. The Model Validator application is demonstrated on the IEEE 13-bus, 123-bus, and 8500-node test cases over three validation scenarios.

Poudel, Shiva↗
DOE OSTI2020

Certification Validation Libraries

“The code “Certificate Validation Libraries” provides the ability to validate x509 certificates in python using a simple library. These libraries validate certificates in 3 ways: 1) validation that certificate has not expired. 2) validation that certificate has not been revoked and 3) validation of certificate's digital signature. The benefits of this code is that one can properly validate x509 certificates, thus strengthening the security of systems using these libraries.”

Maerz, StefanA↗
DOE OSTI2019

Validation of RELAP-7 for Forced Convective and Natural Circulation Reactor Flows

New reactor designs such as small modular reactors and newly proposed passive safety systems rely on natural circulation two-phase flow cooling of the reactor core under accident conditions. Without a pump to drive the flow, the circulation rate is a strong function of void fraction. Under these conditions where the flow rate, pressure drop, and heat transfer are all closely coupled through the mixture density, the accurate prediction of void fraction is pivotal. Furthermore, while these passive systems boast a more reliable cooling option, the sensitivity of the system to two-phase flow instabilities is more likely than in forced convective cooling and can result in dangerous density wave or flow excursion conditions. Improvement of one-dimensional system codes and further development of three-dimensional code capability are all important efforts in addressing these issues. The proposed work is a critical step towards providing confidence through validation of the capability of thermal-hydraulic tools to meet the emerging challenges in reactor simulation. Historically, system analysis codes develop a stiffness over time due to fine-tuning of adjustable code parameters that result in a reluctance or inability to adopt improved models. Therefore, a rigorous effort should be invested in preparation for RELAP-7 to identify required modeling improvements of constitutive relations and validation of the code applicability to the wide range of possible reactor flow conditions. Part of this validation involves the uncertainty quantification which is a critical step in the Code Scaling, Applicability, and Uncertainty (CSAU) methodology and Best-Estimate Plus Uncertainty (BEPU) licensing method. BEPU allows recovery of safety margins and potentially optimizing plant performance, resulting in economic benefit for the reactor operators. Beginning with the collection of past experimental data covering prototypic LWR conditions, this document reviews three critical datasets available for the validation of system codes. The experimental methods, procedures, facilities, instrumentation, and supplementary information are consolidated to aid the validation efforts of this work and future projects. These three datasets (824 conditions) are chosen due to their wide acceptance and demonstrated application for code validation. Two experimental datasets from the Nuclear Power Engineering Corporation (NUPEC) span prototypic conditions of Boiling Water Reactors (BWR) and Pressurized Water Reactors (PWR). These datasets are labeled as the BFBT (BWR Full-size Fine-mesh Bundle Test) and the PSBT (PWR Subchannel and Bundle Test) datasets respectively. The third dataset is taken in an internally heated annulus that is scaled based on a BWR subassembly and includes two-phase flows important to post-blowdown analysis. Together, the datasets consider the full range of pressure and mass flux expected in LWR forced convective flows from steady-state to long term cooling conditions. Recognizing the deficiency of available data under low-pressure, low-flow, a new natural circulation dataset consisting of 107 conditions is obtained. With system pressures ranging between 145 and 950 kPa, natural circulation is driven by a heat flux up to 275 kW/m2 in a 3 m long heated region of the annulus test section which is followed by a 2 m long unheated region before entering a condenser. Stable natural circulation flow rates in this facility were measured to be between 182 and 590 kg/m2-s. Pressure and temperature are measured at ten locations in the facility. In addition, measurements of void fraction, interfacial area concentration, and gas velocity are obtained at five axial locations in the annulus test section. The RELAP5-3D code demonstrates good accuracy for void fraction predictions in BFBT and PSBT benchmarks, but shows relatively large discrepancy in the annulus experiments at lower pressures. In two-phase natural circulation, large oscillations are observed in many system parameters during the simulation of 19 experiment cases which were conducted at low pressures, and the model shows good predictive performance with the remaining 88 experiment cases. However, some discrepancies in void fraction and temperature predictions are found which may require improved constitutive modeling under these low-pressure, low-flow condition. The quantified uncertainty in void fraction predictions in three forced convection benchmarks due to uncertainties in boundary conditions and geometries are relatively small, except for a few cases in the annulus experiment. The output uncertainty of the natural circulation model is quantified by considering the measurement error in boundary conditions, and it is shown that the output uncertainty ranges are relatively small for the range of conditions considered. Through this work, a clear pathway is provided for validation of RELAP7 once the code is available.

42 ENGINEERING↗
NASA NTRS1999

EOS Terra Validation Program

The EOS Terra mission will be launched in July 1999. This mission has great relevance to the atmospheric radiation community and global change issues. Terra instruments include ASTER, CERES, MISR, MODIS and MOPITT. In addition to the fundamental radiance data sets, numerous global science data products will be generated, including various Earth radiation budget, cloud and aerosol parameters, as well as land surface, terrestrial ecology, ocean color, and atmospheric chemistry parameters. Significant investments have been made in on-board calibration to ensure the quality of the radiance observations. A key component of the Terra mission is the validation of the science data products. This is essential for a mission focused on global change issues and the underlying processes. The Terra algorithms have been subject to extensive pre-launch testing with field data whenever possible. Intensive efforts will be made to validate the Terra data products after launch. These include validation of instrument calibration (vicarious calibration) experiments, instrument and cross-platform comparisons, routine collection of high quality correlative data from ground-based networks, such as AERONET, and intensive sites, such as the SGP ARM site, as well as a variety field experiments, cruises, etc. Airborne simulator instruments have been developed for the field experiment and underflight activities including the MODIS Airborne Simulator (MAS), AirMISR, MASTER (MODIS-ASTER), and MOPITT-A. All are integrated on the NASA ER-2, though low altitude platforms are more typically used for MASTER. MATR is an additional sensor used for MOPITT algorithm development and validation. The intensive validation activities planned for the first year of the Terra mission will be described with emphasis on derived geophysical parameters of most relevance to the atmospheric radiation community. Detailed information about the EOS Terra validation Program can be found on the EOS Validation program homepage i/e.: http://ospso.gsfc.nasa.gov/validation/valpage.html).

Starr, David↗
NASA NTRS2003

Validation and Error Characterization for the Global Precipitation Measurement

The Global Precipitation Measurement (GPM) is an international effort to increase scientific knowledge on the global water cycle with specific goals of improving the understanding and the predictions of climate, weather, and hydrology. These goals will be achieved through several satellites specifically dedicated to GPM along with the integration of numerous meteorological satellite data streams from international and domestic partners. The GPM effort is led by the National Aeronautics and Space Administration (NASA) of the United States and the National Space Development Agency (NASDA) of Japan. In addition to the spaceborne assets, international and domestic partners will provide ground-based resources for validating the satellite observations and retrievals. This paper describes the validation effort of Global Precipitation Measurement to provide quantitative estimates on the errors of the GPM satellite retrievals. The GPM validation approach will build upon the research experience of the Tropical Rainfall Measuring Mission (TRMM) retrieval comparisons and its validation program. The GPM ground validation program will employ instrumentation, physical infrastructure, and research capabilities at Supersites located in important meteorological regimes of the globe. NASA will provide two Supersites, one in a tropical oceanic and the other in a mid-latitude continental regime. GPM international partners will provide Supersites for other important regimes. Those objectives or regimes not addressed by Supersites will be covered through focused field experiments. This paper describes the specific errors that GPM ground validation will address, quantify, and relate to the GPM satellite physical retrievals. GPM will attempt to identify the source of errors within retrievals including those of instrument calibration, retrieval physical assumptions, and algorithm applicability. With the identification of error sources, improvements will be made to the respective calibration, assumption, or algorithm. The instrumentation and techniques of the Supersites will be discussed. The GPM core satellite, with its dual-frequency radar and conically scanning radiometer, will provide insight into precipitation drop-size distributions and potentially increased measurement capabilities of light rain and snowfall. The ground validation program will include instrumentation and techniques commensurate with these new measurement capabilities.

Bidwell, Steven W.↗
NASA NTRS2014

Observations on CFD Verification and Validation from the AIAA Drag Prediction Workshops

The authors provide observations from the AIAA Drag Prediction Workshops that have spanned over a decade and from a recent validation experiment at NASA Langley. These workshops provide an assessment of the predictive capability of forces and moments, focused on drag, for transonic transports. It is very difficult to manage the consistency of results in a workshop setting to perform verification and validation at the scientific level, but it may be sufficient to assess it at the level of practice. Observations thus far: 1) due to simplifications in the workshop test cases, wind tunnel data are not necessarily the “correct” results that CFD should match, 2) an average of core CFD data are not necessarily a better estimate of the true solution as it is merely an average of other solutions and has many coupled sources of variation, 3) outlier solutions should be investigated and understood, and 4) the DPW series does not have the systematic build up and definition on both the computational and experimental side that is required for detailed verification and validation. Several observations regarding the importance of the grid, effects of physical modeling, benefits of open forums, and guidance for validation experiments are discussed. The increased variation in results when predicting regions of flow separation and increased variation due to interaction effects, e.g., fuselage and horizontal tail, point out the need for validation data sets for these important flow phenomena. Experiences with a recent validation experiment at NASA Langley are included to provide guidance on validation experiments.

Computational Fluid Dynamics↗
NASA NTRS2015

A Vehicle Management End-to-End Testing and Analysis Platform for Validation of Mission and Fault Management Algorithms to Reduce Risk for NASAs Space Launch System

The engineering development of the National Aeronautics and Space Administration's (NASA) new Space Launch System (SLS) requires cross discipline teams with extensive knowledge of launch vehicle subsystems, information theory, and autonomous algorithms dealing with all operations from pre-launch through on orbit operations. The nominal and off-nominal characteristics of SLS's elements and subsystems must be understood and matched with the autonomous algorithm monitoring and mitigation capabilities for accurate control and response to abnormal conditions throughout all vehicle mission flight phases, including precipitating safing actions and crew aborts. This presents a large and complex systems engineering challenge, which is being addressed in part by focusing on the specific subsystems involved in the handling of off-nominal mission and fault tolerance with response management. Using traditional model-based system and software engineering design principles from the Unified Modeling Language (UML) and Systems Modeling Language (SysML), the Mission and Fault Management (M&FM) algorithms for the vehicle are crafted and vetted in Integrated Development Teams (IDTs) composed of multiple development disciplines such as Systems Engineering (SE), Flight Software (FSW), Safety and Mission Assurance (S&MA) and the major subsystems and vehicle elements such as Main Propulsion Systems (MPS), boosters, avionics, Guidance, Navigation, and Control (GNC), Thrust Vector Control (TVC), and liquid engines. These model-based algorithms and their development lifecycle from inception through FSW certification are an important focus of SLS's development effort to further ensure reliable detection and response to off-nominal vehicle states during all phases of vehicle operation from pre-launch through end of flight. To test and validate these M&FM algorithms a dedicated test-bed was developed for full Vehicle Management End-to-End Testing (VMET). For addressing fault management (FM) early in the development lifecycle for the SLS program, NASA formed the M&FM team as part of the Integrated Systems Health Management and Automation Branch under the Spacecraft Vehicle Systems Department at the Marshall Space Flight Center (MSFC). To support the development of the FM algorithms, the VMET developed by the M&FM team provides the ability to integrate the algorithms, perform test cases, and integrate vendor-supplied physics-based launch vehicle (LV) subsystem models. Additionally, the team has developed processes for implementing and validating the M&FM algorithms for concept validation and risk reduction. The flexibility of the VMET capabilities enables thorough testing of the M&FM algorithms by providing configurable suites of both nominal and off-nominal test cases to validate the developed algorithms utilizing actual subsystem models such as MPS, GNC, and others. One of the principal functions of VMET is to validate the M&FM algorithms and substantiate them with performance baselines for each of the target vehicle subsystems in an independent platform exterior to the flight software test and validation processes. In any software development process there is inherent risk in the interpretation and implementation of concepts from requirements and test cases into flight software compounded with potential human errors throughout the development and regression testing lifecycle. Risk reduction is addressed by the M&FM group but in particular by the Analysis Team working with other organizations such as S&MA, Structures and Environments, GNC, Orion, Crew Office, Flight Operations, and Ground Operations by assessing performance of the M&FM algorithms in terms of their ability to reduce Loss of Mission (LOM) and Loss of Crew (LOC) probabilities. In addition, through state machine and diagnostic modeling, analysis efforts investigate a broader suite of failure effects and associated detection and responses to be tested in VMET to ensure reliable failure detection, and confirm responses do not create additional risks or cause undesired states through interactive dynamic effects with other algorithms and systems. VMET further contributes to risk reduction by prototyping and exercising the M&FM algorithms early in their implementation and without any inherent hindrances such as meeting FSW processor scheduling constraints due to their target platform - the ARINC 6535-partitioned Operating System, resource limitations, and other factors related to integration with other subsystems not directly involved with M&FM such as telemetry packing and processing. The baseline plan for use of VMET encompasses testing the original M&FM algorithms coded in the same C++ language and state machine architectural concepts as that used by FSW. This enables the development of performance standards and test cases to characterize the M&FM algorithms and sets a benchmark from which to measure their effectiveness and performance in the exterior FSW development and test processes. This paper is outlined in a systematic fashion analogous to a lifecycle process flow for engineering development of algorithms into software and testing. Section I describes the NASA SLS M&FM context, presenting the current infrastructure, leading principles, methods, and participants. Section II defines the testing philosophy of the M&FM algorithms as related to VMET followed by section III, which presents the modeling methods of the algorithms to be tested and validated in VMET. Its details are then further presented in section IV followed by Section V presenting integration, test status, and state analysis. Finally, section VI addresses the summary and forward directions followed by the appendices presenting relevant information on terminology and documentation.

Trevino, Luis↗
NASA NTRS2020

Hardware Verification and Validation for a Navigation Sensor Software Model in Support of Flight Vehicle Performance Analysis

… or, “It’s in the details, how to make complicated software perform like complicated hardware.” In attempts to minimize development time and quickly build an operational vehicle, NASA’s Space Launch System (SLS) has had to be intentional about integrated testing. Constraints on budget and schedule have required balance between testing needs and the desire for an integrated flight vehicle as soon as possible. To provide key insights early in design and analysis cycles, a large amount of effort has shifted into maturing and validating models at the component level with integrated testing as a means to validate their integration. In terms of SLS Navigation, this, and the model-based design approach have pushed explicit requirements for sensor models to be validated against flight hardware to high precision. This paper covers the approach taken to verify and validate the models for the two key navigation sensors on the SLS vehicle, the Redundant Inertial Navigation Sensor and the Rate Gyro Assembly. These models are used in performance evaluation, fault detection, and operations development extensively. Using a mix of data from hardware vendor documentation and testing reports, limited in-house testing, and integration activities, these models were able to be validated against flight hardware at multiple levels, from the internal software design to statistical behavior at the raw sensor and integrated box levels. The high level of insight into the hardware elements is instrumental to support flight certification activities and building confidence in SLS Navigation capability. Focused testing enabled additional insight and validation that proved invaluable and the resulting insights were used to focus and mature models. Additionally, of having validated performance-based hardware models enables a wide breadth of activities including detailed fault detection studies and integration into future vehicle frameworks, such as an upper stage and provide a valuable asset to continued SLS analysis and design.

Evan J Anzalone↗