Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “VALIDATION”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

WE-Validate: An Open-Source Framework For Wind Power Validation

Grid operators rely on historical weather time series at existing and planned wind power plants to make informed decisions when planning for a future power grid with very high penetration of renewable power. While synthetic wind power time series have been developed based on historical weather models, their validation with actual power production data remains complex due to variations in modeling practices and methodologies. This paper introduces the WE-Validate framework, originally designed for wind speed validation and now enhanced for wind power validation with a graphical user interface to support users with minimal programming experience. Validation of wind power with WE-Validate is based on robust metrics consisting of RMSE, centered RMSE, average bias, average percent bias, mean absolute error, mean absolute percent error, cross correlation, and calculation of ramping magnitude, rate, and duration. This paper showcases WE-Validate with validation of synthetically derived power for a wind plant in Washington state for one month in 2018. Validation of the synthetic power from two comparison data sets compared with observations shows both comparison series have strong correlation with observed across weekly and monthly aggregations while suffering from persistent negative bias. The suite of metrics within WE-Validate facilitates immediate insight into the utility of the comparison data sets through compression across multiple axes. This user-friendly, open-source tool can be extended beyond wind power, making it a valuable resource for system planners and operators in different domains.

Moncheur de Rieudotte, Malcolm P.↗
NASA NTRS1989

Design for validation: An approach to systems validation

Every complex system built is validated in some manner. Computer validation begins with review of the system design. As systems became too complicated for one person to review, validation began to rely on the application of adhoc methods by many individuals. As the cost of the changes mounted and the expense of failure increased, more organized procedures became essential. Attempts at devising and carrying out those procedures showed that validation is indeed a difficult technical problem. The successful transformation of the validation process into a systematic series of formally sound, integrated steps is necessary if the liability inherent in the future digita-system-based avionic and space systems is to be minimized. A suggested framework and timetable for the transformtion are presented. Basic working definitions of two pivotal ideas (validation and system life-cyle) are provided and show how the two concepts interact. Many examples are given of past and present validation activities by NASA and others. A conceptual framework is presented for the validation process. Finally, important areas are listed for ongoing development of the validation process at NASA Langley Research Center.

Carter, William C.↗
DOE OSTI2024

Validation Assessment of Turbulent Reacting Flow Model Using The Area-Validation Metric on Medium-Scale Methanol Pool Fire Results

Accident analysis and ensuring power plant safety are pivotal in the nuclear energy sector. Significant strides have been achieved over the past few decades regarding fire protection and safety, primarily centered on design and regulatory compliance. Yet, after the Fukushima accident a decade ago, the imperative to enhance measures against fire, internal flooding, and power loss has intensified. Hence, a comprehensive, multilayered protection strategy against severe accidents is needed. Consequently, gaining a deeper insight into pool fires and their behavior through extensive validated data can greatly aid in improving these measures using advanced validation techniques. A model validation study was performed at Sandia National Laboratories in which a 30-cm diameter methanol pool fire was modeled using the SIERRA/Fuego turbulent reacting flow code. This validation study used a standard validation experiment to compare model results against, and conclusions have been published. The fire was modeled with a Large Eddy Simulation (LES) turbulence model with subgrid turbulent kinetic energy closure. Combustion was modeled using a strained laminar flamelet library approach. Radiative heat transfer was accounted for with a model utilizing the gray-gas approximation. In the present study, additional validation analysis is performed using the area validation metric (AVM). These activities are done on multiple datasets involving different variables and temporal/spatial ranges and intervals. In conclusion, the results provide insight into the use of the area validation metric on such temporally varying datasets and the importance of physics-aware use of the metric for proper analysis.

98 NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL P↗
NASA NTRS2004

Can One Satellite Data Set Validation Another? Validation of Envisat SCIAMACHY Data by Comparisons with NOAA-16 SBUV/2 and ERS-2 GOME

Validation of satellite data remains a high priority for the construction of climate data sets. Traditionally ground based measurements have provided the primary comparison data for validation. For some atmospheric parameters such as ozone, a thoroughly validated satellite data record can be used to validate a new instrument s data product in addition to using ground based data. Comparing validated data with new satellite data has several advantages; availability of much more data, which will improve precision, larger geographical coverage, and the footprints are closer in size, which removes uncertainty due to different observed atmospheric volumes. To demonstrate the applicability and some limitations of this technique, observations from the newly launched SCIAMACHY instrument were compared with the NOM-16 SBW/2 and ERS-2 GOME instruments. The SBW/2 data had all ready undergone validation by comparing to the total ozone ground network. Overall the SCIAMACHY data were found to low by 3% with respect to satellite data and 1% low with respect to ground station data. There appears to be seasonal and or solar zenith angle dependences in the comparisons with SBW/2 where differences increase with higher solar zenith angles. It is known that accuracies in both satellite and ground based total ozone algorithms decrease at high solar zenith angles. There is a strong need for more accurate measurement from and the ground under these conditions. At the present time SCIAMACHY data are limited and longer data set with more coverage in both hemispheres is needed to unravel the cause of these differences.

Hilsenrath, E.↗
NASA NTRS2008

In-Flight Validation of Mid and Thermal Infrared Remotely Sensed Data Using the Lake Tahoe and Salton Sea Automated Validation Sites

The presentation includes an introduction, Lake Tahoe site layout and measurements, Salton Sea site layout and measurements, field instrument calibration and cross-calculations, data reduction methodology and error budgets, and example results for MODIS. Summary and conclusions are: 1) Lake Tahoe CA/NV automated validation site was established in 1999 to assess radiometric accuracy of satellite and airborne mid and thermal infrared data and products. Water surface temperatures range from 4-25C.2) Salton Sea CA automated validation site was established in 2008 to broaden range of available water surface temperatures and atmospheric water vapor test cases. Water surface temperatures range from 15-35C. 3) Sites provide all information necessary for validation every 2 mins (bulk temperature, skin temperature, air temperature, wind speed, wind direction, net radiation, relative humidity). 4) Sites have been used to validate mid and thermal infrared data and products from: ASTER, AATSR, ATSR2, MODIS-Terra, MODIS-Aqua, Landsat 5, Landsat 7, MTI, TES, MASTER, MAS. 5) Approximately 10 years of data available to help validate AVHRR.

Lake Tahoe↗
NASA NTRS2007

Initial Retrieval Validation from the Joint Airborne IASI Validation Experiment (JAIVEx)

The Joint Airborne IASI Validation Experiment (JAIVEx) was conducted during April 2007 mainly for validation of the Infrared Atmospheric Sounding Interferometer (IASI) on the MetOp satellite, but also included a strong component focusing on validation of the Atmospheric InfraRed Sounder (AIRS) aboard the AQUA satellite. The cross validation of IASI and AIRS is important for the joint use of their data in the global Numerical Weather Prediction process. Initial inter-comparisons of geophysical products have been conducted from different aspects, such as using different measurements from airborne ultraspectral Fourier transform spectrometers (specifically, the NPOESS Airborne Sounder Testbed Interferometer (NAST-I) and the Scanning-High resolution Interferometer Sounder (S-HIS) aboard the NASA WB-57 aircraft), UK Facility for Airborne Atmospheric Measurements (FAAM) BAe146-301 aircraft insitu instruments, dedicated dropsondes, radiosondes, and ground based Raman Lidar. An overview of the JAIVEx retrieval validation plan and some initial results of this field campaign are presented.

Zhou, Daniel K.↗
NASA NTRS2017

Validation and Scaling of Soil Moisture in a Semi-Arid Environment: SMAP Validation Experiment 2015 (SMAPVEX15)

The NASA SMAP (Soil Moisture Active Passive) mission conducted the SMAP Validation Experiment 2015 (SMAPVEX15) in order to support the calibration and validation activities of SMAP soil moisture data products. The main goals of the experiment were to address issues regarding the spatial disaggregation methodologies for improvement of soil moisture products and validation of the in situ measurement upscaling techniques. To support these objectives high-resolution soil moisture maps were acquired with the airborne PALS (Passive Active L-band Sensor) instrument over an area in southeast Arizona that includes the Walnut Gulch Experimental Watershed (WGEW), and intensive ground sampling was carried out to augment the permanent in situ instrumentation. The objective of the paper was to establish the correspondence and relationship between the highly heterogeneous spatial distribution of soil moisture on the ground and the coarse resolution radiometer-based soil moisture retrievals of SMAP. The high-resolution mapping conducted with PALS provided the required connection between the in situ measurements and SMAP retrievals. The in situ measurements were used to validate the PALS soil moisture acquired at 1-km resolution. Based on the information from a dense network of rain gauges in the study area, the in situ soil moisture measurements did not capture all the precipitation events accurately. That is, the PALS and SMAP soil moisture estimates responded to precipitation events detected by rain gauges, which were in some cases not detected by the in situ soil moisture sensors. It was also concluded that the spatial distribution of the soil moisture resulted from the relatively small spatial extents of the typical convective storms in this region was not completely captured with the in situ stations. After removing those cases (approximately10 of the observations) the following metrics were obtained: RMSD (root mean square difference) of0.016m3m3 and correlation of 0.83. The PALS soil moisture was also compared to SMAP and in situ soil moisture at the 36-km scale, which is the SMAP grid size for the standard product. PALS and SMAP soil moistures were found to be very similar owing to the close match of the brightness temperature measurements and the use of a common soil moisture retrieval algorithm. Spatial heterogeneity, which was identified using the high-resolution PALS soil moisture and the intensive ground sampling, also contributed to differences between the soil moisture estimates. In general, discrepancies found between the L-band soil moisture estimates and the 5-cm depth in situ measurements require methodologies to mitigate the impact on their interpretations in soil moisture validation and algorithm development. Specifically, the metrics computed for the SMAP radiometer-based soil moisture product over WGEW will include errors resulting from rainfall, particularly during the monsoon season when the spatial distribution of soil moisture is especially heterogeneous.

SMAPVEX15↗
DOE OSTI2022

Dark Energy Survey Year 3 results: Cosmology from combined galaxy clustering and lensing validation on cosmological simulations

In this work, we present a validation of the Dark Energy Survey Year 3 (DES Y3) 3×2-point analysis choices by testing them on Buzzard2.0, a new suite of cosmological simulations that is tailored for the testing and validation of combined galaxy clustering and weak-lensing analyses. We show that the buzzard2.0 simulations accurately reproduce many important aspects of the DES Y3 data, including photometric redshift and magnitude distributions, and the relevant set of two-point clustering and weak-lensing statistics. We then show that our model for the 3×2-point data vector is accurate enough to recover the true cosmology in simulated surveys assuming the true redshift distributions for our source and lens samples, demonstrating robustness to uncertainties in the modeling of the nonlinear matter power spectrum, nonlinear galaxy bias, and higher-order lensing corrections. Additionally, we demonstrate for the first time that our photometric redshift calibration methodology, including information from photometry, spectroscopy, clustering cross-correlations, and galaxy–galaxy lensing ratios, is accurate enough to recover the true cosmology in simulated surveys in the presence of realistic photometric redshift uncertainties.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2024

Dark Energy Survey Year 3 results: Simulation-based cosmological inference with wavelet harmonics, scattering transforms, and moments of weak lensing mass maps. Validation on simulations

Beyond-two-point statistics contain additional information on cosmological as well as astrophysical and observational (systematics) parameters. In this methodology paper we provide an end-to-end simulationbased analysis of a set of Gaussian and non-Gaussian weak lensing statistics using detailed mock catalogs of the Dark Energy Survey (DES). Here, we implement: 1) second and third moments; 2) wavelet phase harmonics (WPH); 3) the scattering transform (ST). Our analysis is fully based on simulations, it spans a space of seven $νw$CDM cosmological parameters, and it forward models the most relevant sources of systematics of the data (masks, noise variations, clustering of the sources, intrinsic alignments, and shear and redshift calibration). We implement a neural network compression of the summary statistics, and we estimate the parameter posteriors using a likelihood-free-inference approach. We validate the pipeline extensively, and we find that WPH exhibits the strongest performance when combined with second moments, followed by ST, and then by third moments. The combination of all the different statistics further enhances constraints with respect to second moments, up to 25 percent, 15 percent, and 90 percent for S 8 , Ω m , and the figure-of-merit FoM S8;Ωm , respectively. We further find that non-Gaussian statistics improve constraints on w and on the amplitude of intrinsic alignment with respect to second moments constraints. The methodological advances presented here are suitable for application to Stage IV surveys from Euclid, Rubin-LSST, and Roman with additional validation on mock catalogs for each survey. In a companion paper we present an application to DES Year 3 data.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2022

Validation of MPACT BWR capabilities against critical experiments

Over the years, significant validation work for the neutronics code MPACT has been performed against zero-power critical benchmarks and measured data from operating nuclear power plants. As a part of the Modeling and Analysis of Exelon BWRs for Eigenvalue and Thermal Limits Predictability project, new validation efforts relevant to boiling water reactor (BWR) core applications have been performed. This paper presents the results of the critical experiment portion of the BWR validation efforts for MPACT. The Kritz-4 experiments and IPEN/MB-01 BWR-relevant configurations are modeled with MPACT. The Kritz-4 BWR critical experiments were performed at both cold and hot conditions, which is a unique feature among other critical experiment facilities. The MPACT results with linear source (LS) Method of Characteristics (MOC) have very good agreement with the measured criticality. When using the 60-group BWR library with LS and P2 scattering, the maximum k{sub eff} error is 109 pcm, and the average cold-hot bias is only -0.11 pcm/K. The fission rate distributions of MPACT are also verified with Serpent calculations. For the IPEN facility, two configurations are considered: one with a large central void, and one with a cruciform control rod. The MPACT k{sub eff} errors are less than 50 pcm for all IPEN problems. (authors)

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
NASA NTRS1995

Upper Atmosphere Research Satellite Validation Workshop III: Temperature and Constituents Validation

The Upper Atmosphere Research Satellite (UARS) was launched in September 1991. Since that time data have been retrieved continuously from the various instruments on the UARS spacecraft. These data have been processed by the respective instrument science teams and subsequently archived in the UARS Central Data Handling Facility (CDHF) at the NASA Goddard Space Flight Center, Greenbelt, Maryland. This report contains the proceedings from one of the three workshops held to evaluate the progress in validating UARS constituents and temperature data and to document the quality of that data. The first workshop was held in Oxford, England, in March 1992, five and one-half months after UARS launch. The second workshop was held in Boulder, Colorado in October 1992. Since launch, the various data have undergone numerous revisions. In many instances these revisions are a result of data problems identified during the validation workshops. Thus, the formal validation effort is a continually ongoing process.

Grose, William L.↗
NASA NTRS2017

Validation of SMAP Surface Soil Moisture Products with Core Validation Sites

The NASA Soil Moisture Active Passive (SMAP) mission has utilized a set of core validation sites as the primary methodology in assessing the soil moisture retrieval algorithm performance. Those sites provide well calibrated in situ soil moisture measurements within SMAP product grid pixels for diverse conditions and locations.The estimation of the average soil moisture within the SMAP product grid pixels based on in situ measurements is more reliable when location specific calibration of the sensors has been performed and there is adequate replication over the spatial domain, with an up-scaling function based on analysis using independent estimates of the soil moisture distribution. SMAP fulfilled these requirements through a collaborative CalVal Partner program.This paper presents the results from 34 candidate core validation sites for the first eleven months of the SMAP mission. As a result of the screening of the sites prior to the availability of SMAP data, out of the 34 candidate sites 18 sites fulfilled all the requirements at one of the resolution scales (at least). The rest of the sites are used as secondary information in algorithm evaluation. The results indicate that the SMAP radiometer-based soil moisture data product meets its expected performance of 0.04 cu m/cu m volumetric soil moisture (unbiased root mean square error); the combined radar-radiometer product is close to its expected performance of 0.04 cu m/cu m, and the radar-based product meets its target accuracy of 0.06 cu m/cu m (the lengths of the combined and radar-based products are truncated to about 10 weeks because of the SMAP radar failure). Upon completing the intensive CalVal phase of the mission the SMAP project will continue to enhance the products in the primary and extended geographic domains, in co-operation with the CalVal Partners, by continuing the comparisons over the existing core validation sites and inclusion of candidate sites that can address shortcomings.

SMAP↗
NASA NTRSDate not supplied

Oceanic Validation of IMERG Version 7 with the GPM Validation Network

- To validate Version 7 of the Integrated Multi-satellitE Retrievals for the Global Precipitation Measurement (GPM) mission (IMERG) over tropical and high-latitude oceans using the GPM Validation Network (VN). - To trace errors from the Level-3 IMERG V07 product back through to the input Level-2 Goddard Profiling Algorithm (GPROF) V07 product for the GPM Microwave Imager (GMI).

GPM↗
DOE OSTI2023

Validation study of sodium pool fire modeling efforts in $\mathrm{MELCOR}$ and $\mathrm{SPHINCS}$ codes

Discharge of sodium coolant into containment from a sodium-cooled fast reactor vessel can occur in the event of a pipe leak or break. In this situation, some of the liquid sodium droplets discharged from the coolant system will react with oxygen in the air before reaching the containment. This phase of the event is normally termed the sodium spray fire phase. Unreacted sodium droplets pool on the containment floor where continued reaction with containment atmospheric oxygen occurs. This phase of the event is normally termed the sodium pool fire phase. Both phases of these sodium-oxygen reactions (or fires) are important to model because of the heat addition and aerosol generation that occur. Any fission products trapped in the sodium coolant may also be released during this progression of events, which if released from containment could pose a health risk to workers and the public. The paper describes progress of an international collaborative research in the area of the sodium fire modeling in the sodium-cooled fast reactors between the United States and Japan under the framework of the Civil Nuclear Energy Research and Development Working Group. In this collaboration between Sandia National Laboratories and Japan Atomic Energy Agency, the validation basis for and modeling capabilities of sodium spray and pool fires in MELCOR of Sandia National Laboratories and SPHINCS of Japan Atomic Energy Agency are being enhanced. Here this study documents MELCOR and SPHINCS sodium pool fire model validation exercises against the JAEA’s sodium pool fire experiments, F7-1 and F7-2. The proposed enhancement of the sodium pool fire models in MELCOR through addition of thermal hydraulic and sodium spreading models that enable a better representation of experimental results is also described.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2024

Design and Validation of a High-Throughput Reductive Catalytic Fractionation Method

Reductive catalytic fractionation (RCF) is a promising method to extract and depolymerize lignin from biomass, and bench-scale studies have enabled considerable progress in the past decade. RCF experiments are typically conducted in pressurized batch reactors with volumes ranging between 50 and 1000 mL, limiting the throughput of these experiments to one to six reactions per day for an individual researcher. Here, we report a high-throughput RCF (HTP-RCF) method in which batch RCF reactions are conducted in 1 mL wells machined directly into Hastelloy reactor plates. The plate reactors can seal high pressures produced by organic solvents by vertically stacking multiple reactor plates, leading to a compact and modular system capable of performing 240 reactions per experiment. Using this setup, we screened solvent mixtures and catalyst loadings for hydrogen-free RCF using 50 mg poplar and 0.5 mL reaction solvent. The system of 1:1 isopropanol/methanol showed optimal monomer yields and selectivity to 4-propyl substituted monomers, and validation reactions using 75 mL batch reactors produced identical monomer yields. To accommodate the low material loadings, we then developed a workup procedure for parallel filtration, washing, and drying of samples and a 1H nuclear magnetic resonance spectroscopy method to measure the RCF oil yield without performing liquid-liquid extraction. As a demonstration of this experimental pipeline, 50 unique switchgrass samples were screened in RCF reactions in the HTP-RCF system, revealing a wide range of monomer yields (21-36%), S/G ratios (0.41-0.93), and oil yields (40-75%). These results were successfully validated by repeating RCF reactions in 75 mL batch reactors for a subset of samples. We anticipate that this approach can be used to rapidly screen substrates, catalysts, and reaction conditions in high-pressure batch reactions with higher throughput than standard batch reactors.

BIOMASS FUELS,INORGANIC, ORGANIC, PHYSICAL, AND AN↗
DOE OSTI2025

Large-scale experimental validation of thermochemical water-splitting oxides discovered by defect graph neural networks

Thermochemical water-splitting (TCH) based on 2-step thermal redox cycles in metal oxides is a promising approach to generating H 2 , but state-of-the-art (SOTA) CeO 2 has several practical limitations, which has motivated continued materials discovery efforts in this field. Here, in this study, we improve upon a SOTA defect graph neural network (dGNN) surrogate model's oxygen vacancy predictions and combine them with materials project phase diagrams to down-select and discover structurally diverse, experimentally known metal oxides whose TCH performance was previously unknown. Amongst twelve candidates selected based on our high-throughput screening and down-selection criteria, we achieved ∼80% accuracy in identifying materials with stable redox cycling and hydrogen production in stagnation flow reactor water-splitting experiments. Closer to 100% accuracy can be achieved if higher-accuracy, hybrid DFT-predicted vacancy formation energies were computed and used in lieu of the most uncertain dGNN-based screening predictions, as they correct false positives to true negatives. Notably, two discovered candidates, Sr 3 PrMn 2 O 8 and Ba 2 Fe 2 O 5 , display hydrogen yields greater than CeO 2 under specific redox conditions. In conclusion, these results demonstrate our ability to computationally predict and experimentally validate promising candidate TCH materials that have the potential to compete with CeO 2 .

08 HYDROGEN↗