Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Validation Studies”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4

CFD Validation Study of a Hypersonic Cone-Slice-Flap Variable Geometry Configuration

Model validation is the process of determining the degree of accuracy between physical reality and the model. The result of model validation can either be used to improve the model through calibration or quantify the model-form uncertainty. This work focuses on providing the model-form uncertainty through an area metric for a hypersonic cone-slice-flap variable geometry configuration given uncertainty in both the simulation and experimental data. The research here compares two different turbulence models for the simulations. For a variable geometry, performing uncertainty quantification to capture the model-form uncertainty on every configuration is computationally challenging. This work lays out a procedure that can give an accurate representation of the model-form uncertainty using a small number of high-fidelity runs and many low-fidelity runs on multiple configurations. Running this comparison provides a quantifiable measurement for the accuracy of each turbulence model for this type of design. The high-fidelity CFD solver used was VULCAN-CFD and the low-fidelity results came from Cart3D. The experimental data came from the 20-Inch Mach 6 Tunnel located at NASA Langley Research Center. The present work showed that the using both the Spalart and Allamaras and Menter Shear-Stress Transport turbulence models overpredicted the drag and lift coefficient, while underpredicting the pitching moment coefficient. The model-form uncertainty estimate resulted in up to a 13.6% change in the total uncertainty for the drag coefficient, up to a 57.4% change in total uncertainty for the lift coefficient, and up to a 100% change in total uncertainty for the pitching moment coefficient.

Laura M. White↗

CFD Validation Study of a Hypersonic Cone-Slice-Flap Variable Geometry Configuration

Model validation is the process of determining the degree of accuracy between physical reality and the model. The result of model validation can either be used to improve the model through calibration or quantify the model-form uncertainty. This work focuses on providing the model-form uncertainty through an area metric for a hypersonic cone-slice-flap variable geometry configuration given uncertainty in both the simulation and experimental data. For a variable geometry, performing uncertainty quantification to capture the model-form uncertainty on every configuration is computationally challenging. This work lays out a procedure that can give an accurate representation of the model-form uncertainty using a small number of high-fidelity runs and many low-fidelity runs on multiple configurations. Running this comparison provides a quantifiable measurement for the accuracy of each turbulence model for this type of design. The high-fidelity CFD solver used was VULCAN-CFD and the low-fidelity results came from Cart3D. The experimental data came from the 20-Inch Mach 6 Tunnel located at NASA Langley Research Center. The present work showed that the turbulence simulation overpredicted the drag and lift coefficient, while underpredicting the pitching moment coefficient. The model-form uncertainty estimate resulted up to a 13.6% change in the total uncertainty for the drag coefficient, up to a 57.4% change in total uncertainty for the lift coefficient, and up to a 100% change in total uncertainty for the pitching moment coefficient.

Laura White↗

Quantitative Exposure Assessment Professional Judgment Validation Study

Professional judgement is a key element of an exposure assessment program. Industrial hygienists frequently rely on it to make judgements about the acceptability of occupational exposures to chemical, biological, and physical agents. Often this must be done with little or no quantitative sampling data. This is especially true in the case of research and development, where activities may be short duration and non-routine. In this situation there are very limited opportunities for sampling, and therefore, professional judgement becomes a major tool industrial hygienists rely upon. One of the limitations of professional judgement is that exposures may be misclassified. That is, an exposure deemed to be unacceptable may in fact be acceptable and unnecessary effort and expense may be incurred as a result. On the other hand, an exposure may be judged acceptable when it is in fact unacceptable, resulting in an overexposure and potential injury or illness. It is therefore crucial that professional judgement be validated. In this study we evaluated 106 acceptable exposure determinations with retrospective quantitative exposure monitoring to validate the performance of Sandia National Laboratory industrial hygienists at performing exposure determinations with professional judgement.

54 ENVIRONMENTAL SCIENCES↗

PNNL Report: New Study Validates Color-Rendition Specification Criteria Based on TM-30

Pacific Northwest National Laboratory (PNNL) has released the results of a study – published in Lighting Research & Technology – that validates color-preference specification criteria based on ANSI/IES TM-30-18. The result provides additional data to support TM-30 Annex E (in press), which will include tiers of recommended specification criteria for different goals (color preference, color vividness, and color fidelity), among other information related to color rendition specification. Standardized methods for evaluating the color rendition of light sources have progressed rapidly in the past five years, beginning with the publication of IES TM-30-15, followed by the adoption of CIE 224:2017, and culminating with the issuance of ANSI/IES TM-30-18 (which revised TM-30-15, made it a national standard, and unified the calculation framework of CIE 224 and IES TM-30). Importantly, none of these documents provided guidance on how to specify color rendition using the included measures; that is, no value or combination of values was identified that leads to any desired quality of the light. Dozens of experiments over the past 60 years have examined subjective evaluations of color quality—such as preference, naturalness, or vividness—although the focus has traditionally been on developing a single-number color quality metric rather than specification criteria. Further, many of these pre-LED and pre-TM-30 studies were limited in their ability to vary the lighting characteristics, preventing them from generating a complete and accurate picture of what influences color rendering quality in architectural spaces. A renewed emphasis in this area of research over the past four years has yielded a growing consensus about how measures of color fidelity (Rf), gamut area (Rg), and red chroma (Rcs,h1) can be used in combination to effectively describe several important perceptual attributes. In the new study, which was the third in a series conducted at PNNL, 90 lighting conditions with systematically varied color rendition and chromaticity were evaluated by 25 participants, who provided ratings for normalness, saturation, preference, and acceptability. The lighting conditions were specifically designed to test the bounds of previously proposed color preference criteria, while simultaneously verifying the applicability of the criteria across different values of CCT and Duv. In line with the preceding work, there were substantial differences in ratings based on color rendition, but minimal differences based on chromaticity. The tested criteria performed well, although after the experiment, some small adjustments in lower limits provided more effective divisions between tiers of performance. The adjusted criteria are: • Level 1: Rf = 78, Rg = 95, -1% = Rcs,h1 = 15% • Level 2: Rf = 74, Rg = 92, -7% = Rcs,h1 = 19% • Level 3: Rf = 70, Rg = 89, -12% = Rcs,h1 = 23%. These criteria were concurrently checked against the results from other, independent experiments, and will be included in ANSI/IES TM-30 Annex E as the specifications for color preference. The new criteria will be a valuable tool for improving lighting quality and balancing the tradeoff between color rendition and energy efficiency.

Royer, Michael P.↗

EOS Airborne Program for Validation Studies of EOS Data Products

An overview of the NASA EOS (Earth Observing System) Validation Program will be given with special emphasis on synergy of airborne field missions and EOS satellite missions. Presentation will describe objectives of program and recent field campaigns for validation of standard scientific data products from EOS Terra. Airborne simulators developed for Terra validation will also be described. Plans for future field activity and airborne simulators, especially for EOS Aqua validation and also encompassing EOS Aura validation, will also be briefly presented.

Starr, David OC.↗

A laboratory validation study of the time-lapse oscillatory pumping test for leakage detection in geological repositories

Geologic repositories are extensively used for disposing byproducts in mineral and energy industries. The safety and reliability of these repositories are a primary concern to environmental regulators and the public. Time-lapse oscillatory pumping test (OPT) has been introduced recently as a pressure-based technique for detecting potential leakage in geologic repositories. By routinely conducting OPT at a number of pulsing frequencies, an operator may identify the potential repository anomalies in the frequency domain, alleviating the ambiguity caused by reservoir noise and improving the signal-to-noise ratio. Building on previous theoretical and field studies, this work performed a series of laboratory experiments to validate the concept of time-lapse OPT using a custom made, stainless steel tank under relatively high pressures. The experimental configuration simulates a miniature geologic storage repository consisting of three layers (i.e., injection zone, caprock, and above-zone aquifer). Here results show that leakage in the injection zone led to deviations in the power spectrum of observed pressure data, and the amplitude of which also increases with decreasing pulsing frequencies. The experimental results are further analyzed by developing a 3D flow model, using which the model parameters are estimated through frequency domain inversion.

58 GEOSCIENCES↗

Validation study of GENE on ASDEX Upgrade using perturbative thermal diffusivity measured with partial sawtooth heat pulses

Perturbative thermal diffusivity is measured with partial sawtooth-generated heat pulses on ASDEX Upgrade for the first time, and these measurements are used to validate the first nonlinear ion-scale gyrokinetic simulation that agrees with experimentally measured perturbative diffusivity. Recent work on Alcator C-Mod and DIII-D has shown that ion-scale gyrokinetics cannot adequately describe certain plasma conditions, and that in these instances multi-scale simulations may resolve the observed discrepancies. This work presents the first measurements of electron perturbative thermal diffusivity with partial sawteeth on ASDEX Upgrade (measuring values between 2 and 9 m 2 s -1 ), and compares these measurements to those made with more established modulated electron cyclotron heating measurements, finding good agreement within experimental uncertainty. Perturbative diffusivity is found to scale inversely with collisionality across ASDEX Upgrade and Alcator C-Mod plasmas. Finally, perturbative diffusivity is used as a validation constraint in a study with the gyrokinetic code GENE, showing the first instance where an ion-scale gyrokinetic simulation can simultaneously match the experimental ion and electron heat fluxes and the perturbative thermal diffusivity. Here these results indicate that multi-scale effects may only be important in some plasmas, and the collisionality and ratio of high to low wavenumber linear growth rates may distinguish these cases.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

A Validation Study of Merging and Spacing Techniques in a NAS-Wide Simulation

In November 2010, Intelligent Automation, Inc. (IAI) delivered an M&S software tool to that allows system level studies of the complex terminal airspace with the ACES simulation. The software was evaluated against current day arrivals in the Atlanta TRACON using Atlanta's Hartsfield-Jackson International Airport (KATL) arrival schedules. Results of this validation effort are presented describing data sets, traffic flow assumptions and techniques, and arrival rate comparisons between reported landings at Atlanta versus simulated arrivals using the same traffic sets in ACES equipped with M&S. Initial results showed the simulated system capacity to be significantly below arrival capacity seen at KATL. Data was gathered for Atlanta using commercial airport and flight tracking websites (like FlightAware.com), and analyzed to insure compatible techniques were used for result reporting and comparison. TFM operators for Atlanta were consulted for tuning final simulation parameters and for guidance in flow management techniques during high volume operations. Using these modified parameters and incorporating TFM guidance for efficiencies in flowing aircraft, arrival capacity for KATL was matched for the simulation. Following this validation effort, a sensitivity study was conducted to measure the impact of variations in system parameters on the Atlanta airport arrival capacity.

Glaab, Patricia C.↗

CFD Validation Studies for Hypersonic Flow Prediction

A series of experiments to measure pressure and heating for code validation involving hypersonic, laminar, separated flows was conducted at the Calspan-University at Buffalo Research Center (CUBRC) in the Large Energy National Shock (LENS) tunnel. The experimental data serves as a focus for a code validation session but are not available to the authors until the conclusion of this session. The first set of experiments considered here involve Mach 9.5 and Mach 11.3 N, flow over a hollow cylinder-flare with 30 deg flare angle at several Reynolds numbers sustaining laminar, separated flow. Truncated and extended flare configurations are considered. The second set of experiments, at similar conditions, involves flow over a sharp, double cone with fore-cone angle of 25 deg and aft-cone angle of 55 deg. Both sets of experiments involve 30 deg compressions. Location of the separation point in the numerical simulation is extremely sensitive to the level of grid refinement in the numerical predictions. The numerical simulations also show a significant influence of Reynolds number on extent of separation. Flow unsteadiness was easily introduced into the double cone simulations using aggressive relaxation parameters that normally promote convergence.

Gnoffo, Peter A.↗

CFD Validation Studies for Hypersonic Flow Prediction

A series of experiments to measure pressure and heating for code validation involving hypersonic, laminar, separated flows was conducted at the Calspan-University at Buffalo Research Center (CUBRC) in the Large Energy National Shock (LENS) tunnel. The experimental data serves as a focus for a code validation session but are not available to the authors until the conclusion of this session. The first set of experiments considered here involve Mach 9.5 and Mach 11.3 N2 flow over a hollow cylinder-flare with 30 degree flare angle at several Reynolds numbers sustaining laminar, separated flow. Truncated and extended flare configurations are considered. The second set of experiments, at similar conditions, involves flow over a sharp, double cone with fore-cone angle of 25 degrees and aft-cone angle of 55 degrees. Both sets of experiments involve 30 degree compressions. Location of the separation point in the numerical simulation is extremely sensitive to the level of grid refinement in the numerical predictions. The numerical simulations also show a significant influence of Reynolds number on extent of separation. Flow unsteadiness was easily introduced into the double cone simulations using aggressive relaxation parameters that normally promote convergence.

Gnoffo, Peter A.↗

Expert system verification and validation study. Delivery 3A and 3B: Trip summaries

Key results are documented from attending the 4th workshop on verification, validation, and testing. The most interesting part of the workshop was when representatives from the U.S., Japan, and Europe presented surveys of VV&T within their respective regions. Another interesting part focused on current efforts to define industry standards for artificial intelligence and how that might affect approaches to VV&T of expert systems. The next part of the workshop focused on VV&T methods of applying mathematical techniques to verification of rule bases and techniques for capturing information relating to the process of developing software. The final part focused on software tools. A summary is also presented of the EPRI conference on 'Methodologies, Tools, and Standards for Cost Effective Reliable Software Verification and Validation. The conference was divided into discussion sessions on the following issues: development process, automated tools, software reliability, methods, standards, and cost/benefit considerations.

French, Scott↗

Expert system verification and validation study. ES V/V guidelines/workshop conference summary

The intent of the workshop was to start moving research on the verification and validation (V&V) of knowledge based systems (KBSs) in the direction of providing tangible 'products' that a KBS developer could use. In the near term research will focus on identifying the kinds of experiences encountered during KBS development of 'real' KBSs. These will be stored in a repository and will serve as the foundation for the rest of the activities described here. One specific approach to be pursued is 'benchmarking'. With this approach, a KBS developer can use either 'canned' KBSs with seeded errors or existing KBSs with known errors to evaluate a given tool's ability to satisfactorily identify errors.

French, Scott W.↗

Lessons Learned from CFD Validation Study of Protuberance Heating

The objectives of this presentation are: (1) Share lessons learned from a recent exercise in CFD validation of protuberance heating (2) Impact of experimental data reduction assumptions and techniques on validation activity (3) Advanced data reduction techniques may provide useful data from non-typical test methods (4) Significance of the recovery factor for high-speed flows (5) Show typical results of the Lag turbulence model on protuberances (6) Introduce and inform the listener of a protuberance heating dataset which will soon be available for comparison

Oliver, Brandon↗

Expert system verification and validation study: ES V/V Workshop

The primary purpose of this document is to build a foundation for applying principles of verification and validation (V&V) of expert systems. To achieve this, some V&V as applied to conventionally implemented software is required. Part one will discuss the background of V&V from the perspective of (1) what is V&V of software and (2) V&V's role in developing software. Part one will also overview some common analysis techniques that are applied when performing V&V of software. All of these materials will be presented based on the assumption that the reader has little or no background in V&V or in developing procedural software. The primary purpose of part two is to explain the major techniques that have been developed for V&V of expert systems.

French, Scott↗

Prospective, Multicenter Validation Study of Magnetic Resonance Volumetry for Response Assessment After Preoperative Chemoradiation in Rectal Cancer: Can the Results in the Literature be Reproduced?

Purpose: To review the available literature on tumor size/volume measurements on magnetic resonance imaging for response assessment after chemoradiotherapy, and validate these cut-offs in an independent multicenter patient cohort. Methods and Materials: The study included 2 parts. (1) Review of the literature: articles were included that assessed the accuracy of tumor size/volume measurements on magnetic resonance imaging for tumor response assessment. Size/volume cut-offs were extracted; (2) Multicenter validation: extracted cut-offs from the literature were tested in a multicenter cohort (n=146). Accuracies were calculated and compared with reported results from the literature. Results: The review included 14 articles, in which 3 different measurement methods were assessed: (1) tumor length; (2) 3-dimensonial tumor size; and (3) whole volume. Study outcomes consisted of (1) complete response (ypT0) versus residual tumor; (2) tumor regression grade 1 to 2 versus 3 to 5; and (3) T-downstaging (ypT<cT). In the multicenter cohort, best results were obtained for the validation of the whole-volume measurements, in particular for the outcome ypT0 (accuracy 44%-80%), with the optimal cut-offs being 1.6 cm{sup 3} (after chemoradiation therapy) and a volume reduction of Δ80% to 86.6%. Accuracies for whole-volume measurements to assess tumor regression grade 1 to 2 were 52% to 61%, and for T-downstaging 51% to 57%. Overall accuracies for tumor length ranged between 48% and 53% and for 3D size measurement between 52% and 56%. Conclusions: Magnetic resonance volumetry using whole-tumor volume measurements can be helpful in rectal cancer response assessment with selected cut-off values. Measurements of tumor length or 3-dimensional tumor size are not helpful. Magnetic resonance volumetry is mainly accurate to assess a complete tumor response (ypT0) after chemoradiation therapy (accuracies up to 80%).

62 RADIOLOGY AND NUCLEAR MEDICINE↗

Validation Study for Crediting Chlorine in Criticality Analyses for US Spent Nuclear Fuel Disposition

Spent nuclear fuel (SNF) management practices in the United States rely on dry storage systems that include both canister- and cask-based systems. The United States Department of Energy Used Fuel Disposition Campaign is examining the feasibility of direct disposal of dual-purpose (storage and transportation) canisters (DPCs) in a geological repository. One of the major technical challenges for direct disposal is the ability to demonstrate the subcriticality of the DPCs loaded with SNF for the repository performance period (e.g., 10,000 years or more) as the DPCs may undergo degradation over time. Specifically, groundwater ingress into the DPC (i.e., flooding) could allow the system to achieve criticality in scenarios where the neutron absorber plates in the DPC basket have degraded. However, as was shown by Banerjee et al., some aqueous species in the groundwater provide noticeable reactivity reduction for these systems. For certain amounts of particular aqueous species (e.g., chlorine, lithium) in the groundwater, subcriticality can be demonstrated even for DPCs with complete degradation of the neutron absorber plates or a degraded fuel basket configuration. It has been demonstrated that chlorine is the leading impurity, as indicated by significant neutron absorption in the water that is available in reasonable quantities for the deep geological repository media under consideration. This paper presents the results of an investigation of the available integral experiments worldwide that could be used to validate DPC disposal criticality evaluations, including credit for chlorine. Due to the small number of applicable critical configurations, validation through traditional trending analysis was not possible. The bias in the eigenvalue of the application systems due only to the chlorine was calculated using TSURFER analysis and found to be on the order of 100 percent mille (1 pcm = 10 -5 k eff ). This study investigated the design of a series of critical configurations with varying amounts of chlorine to address validation gaps. Such integral experiments would support the crediting of the chlorine neutron-absorption properties in groundwater and the demonstration of subcriticality for DPCs in deep geologic repositories with sufficient chlorine availability.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

Expert system verification and validation study. Phase 2: Requirements identification. Delivery 1: Updated survey report

The purpose is to report the state-of-the-practice in Verification and Validation (V and V) of Expert Systems (ESs) on current NASA and Industry applications. This is the first task of a series which has the ultimate purpose of ensuring that adequate ES V and V tools and techniques are available for Space Station Knowledge Based Systems development. The strategy for determining the state-of-the-practice is to check how well each of the known ES V and V issues are being addressed and to what extent they have impacted the development of Expert Systems.

Source record↗