Results for “Validation” — Page 25

DOE OSTI2019

Validating Performance Models for Hybrid Power Plant Control Assessment

The need for simple, but accurate performance models of wind turbine generators (WTGs), photovoltaic (PV) plants, and battery energy storage systems (BESS) for various hybrid power plant (HPP) studies motivates the present work. Particularly, the development and verification stage of HPP controls requires reduced-order models to minimize the complexity and computation effort of simulation platforms. In this paper, such models are proposed, and the most essential parts of the models are validated through field measurements. The models target power system integration studies involving active and reactive power, as well as frequency and voltage regulation where detailed models, as proposed in the standards, can be cumbersome. Field measurements of two Vestas WTGs, one 1-MW PV plant, and one 1-MW/1-MWh BESS are used for model validation. The results show that the WTG and PV performance models correctly estimate the power generation variability according to fluctuations in wind speed and solar irradiance. The BESS performance model provides satisfactory results related to grid-forming control performance and estimation of state-of-charge. The presented validation work enables using the proposed performance models for power system studies and HPP control design in all model-based design stages, that is, preliminary analysis, design, verification, and validation with a high level of confidence.

24 POWER TRANSMISSION AND DISTRIBUTION↗

DOE OSTI2016

197 candidates and 104 validated planets in K2's first five fields

We present 197 planet candidates discovered using data from the first year of the NASA K2 mission (Campaigns 0–4), along with the results of an intensive program of photometric analyses, stellar spectroscopy, high-resolution imaging, and statistical validation. We distill these candidates into sets of 104 validated planets (57 in multi-planet systems), 30 false positives, and 63 remaining candidates. Our validated systems span a range of properties, with median values of R {sub P} = 2.3 R{sub ⊕}, P = 8.6 days, T{sub eff} = 5300 K, and Kp = 12.7 mag. Stellar spectroscopy provides precise stellar and planetary parameters for most of these systems. We show that K2 has increased by 30% the number of small planets known to orbit moderately bright stars (1–4 R {sub ⊕}, Kp = 9–13 mag). Of particular interest are 76 planets smaller than 2 R {sub ⊕}, 15 orbiting stars brighter than Kp = 11.5 mag, 5 receiving Earth-like irradiation levels, and several multi-planet systems—including 4 planets orbiting the M dwarf K2–72 near mean-motion resonances. By quantifying the likelihood that each candidate is a planet we demonstrate that our candidate sample has an overall false positive rate of 15%–30%, with rates substantially lower for small candidates (<2R{sub ⊕}) and larger for candidates with radii >8R{sub ⊕} and/or with P<3 days. Extrapolation of the current planetary yield suggests that K2 will discover between 500 and 1000 planets in its planned four-year mission, assuming sufficient follow-up resources are available. Efficient observing and analysis, together with an organized and coherent follow-up strategy, are essential for maximizing the efficacy of planet-validation efforts for K2, TESS, and future large-scale surveys.

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2026

Validation of the NLR Pumped Storage Hydropower Cost Model

The National Laboratory of the Rockies (NLR) first released its pumped storage hydropower (PSH) cost model in 2023 as the most detailed bottom-up PSH cost model available to the public. It is available both as a spreadsheet and an interactive web tool, enabling users with a variety of PSH interests to transparently characterize costs of alternative PSH sites and designs. The PSH cost model cannot replace detailed site-level studies and design, but it is important to validate it against other industry PSH cost estimates. The initial model methodology report validated the cost model for a single proposed site, the Eagle Mountain Project in California. This slide deck documents an expanded validation exercise using cost data from six other sites: Goldendale (Washington), Seminoe (Wyoming), Gordon Butte (Montana), Swan Lake (Oregon), White Pine (Oregon), and Lewis Ridge (Kentucky). It compares itemized costs from Federal Energy Regulatory Commission (FERC) applications and other reported costs with NLR PSH cost model outputs after customizing inputs for each site. The validation exercise finds that the NLR model's conservative indirect cost assumptions often drive overall cost overestimation, with direct cost comparisons typically agreeing more closely. All cost model estimates are well within an Association for the Advancement of Cost Engineering (AACE) Class 5 estimation range (-50% to +100%), with five within the AACE Class 4 range (-30% to +50%) and four being within 15%. This result is considered reasonable performance for a parametric model applied at a preliminary design stage.

13 HYDRO ENERGY↗

DOE OSTI2026

H2@Scale - Validating an Electrolysis System with High Output Pressure: Cooperative Research and Development Final Report, CRADA Number CRD-18-00741

Electrolysis has been a commercially available product for a while and electrolyzers have been a proven capability to provide additional benefits (e.g. controllable load for grid services) in addition to production of hydrogen. The hydrogen output is typically compressed for storage and dispensing. Compression adds cost and decreases system reliability. Honda’s electrolyzer systems have been developed to include electrochemical compression to leverage the production system itself for at least partial compression. In this project, the team will evaluate Honda’s PEM based electrochemical compression system. The system is capable of compressing hydrogen up to 70 MPa electrochemically. Validation testing is the next step to accelerate this technology into the marketplace, as the validation will provide needed data under a variety of operation conditions and controls. These operating conditions and controls are based on over a decade of NLR research and development with low-temperature electrolysis. The validation testing will include preparing NLR’s site for third party evaluation, benchmark testing of Honda’s stack and system, and simulating operation connected to renewables or in a grid service profile. NLR’s Energy System Integration Lab will be the location for the electrolyzer validation research and integrated into the Hydrogen Infrastructure Test & Research Facility (HITRF). This will build into the existing retail style hydrogen fueling station for a fully integrated experimental setup.

08 HYDROGEN↗

NASA NTRS2001

A semi-automatic method for left ventricle volume estimate: an in vivo validation study

This study aims to the validation of the left ventricular (LV) volume estimates obtained by processing volumetric data utilizing a segmentation model based on level set technique. The validation has been performed by comparing real-time volumetric echo data (RT3DE) and magnetic resonance (MRI) data. A validation protocol has been defined. The validation protocol was applied to twenty-four estimates (range 61-467 ml) obtained from normal and pathologic subjects, which underwent both RT3DE and MRI. A statistical analysis was performed on each estimate and on clinical parameters as stroke volume (SV) and ejection fraction (EF). Assuming MRI estimates (x) as a reference, an excellent correlation was found with volume measured by utilizing the segmentation procedure (y) (y=0.89x + 13.78, r=0.98). The mean error on SV was 8 ml and the mean error on EF was 2%. This study demonstrated that the segmentation technique is reliably applicable on human hearts in clinical practice.

NASA Discipline Cardiopulmonary↗

NASA NTRS2004

Mars Exploration Rover Mission: Entry, Descent, and Landing System Validation

System validation for a Mars entry, descent, and landing system is not simply a demonstration that the electrical system functions in the associated environments. The function of this system is its interaction with the atmospheric and surface environment. Thus, in addition to traditional test-bed, hardware-in-the-loop, testing, a validation program that confirms the environmental interaction is required. Unfortunately, it is not possible to conduct a meaningful end-to-end test of a Mars landing system on Earth. The validation plan must be constructed from an interconnected combination of simulation, analysis and test. For the Mars Exploration Rover mission, this combination of activities and the logic of how they combined to the system's validation was explicitly stated, reviewed, and tracked as part of the development plan.

system validation↗

NASA NTRS2017

Development of Level 2 Calibration and Validation Plans for GOES-R; What is a RIMP?

Calibration and Validation (CalVal) plans for Geostationary Operational Environmental Satellite version R (GOES-R) Level 2 (L2) products were documented via Resource, Implementation, and Management Plans (RIMPs) for all of the official L2 products required from the GOES-R Advanced Baseline Imager (ABI). In 2015 the GOES-R program decided to replace the typical CalVal plans with RIMPs that covered, for a given L2 product, what was required from that product, how it would be validated, and what tools would be used to do so. Similar to Level 1b products, the intent was to cover the full spectrum of planning required for the CalVal of L2 ABI products. Instead of focusing on step-by-step procedures, the RIMPs concentrated on the criteria for each stage of the validation process (Beta, Provisional, and Full Validation) and the many elements required to prove when each stage was reached.

Validation↗

NASA NTRS2017

Development of Level 2 Calibration and Validation Plans for GOES-R; What is a RIMP?

Calibration and Validation (CalVal) plans for Geostationary Operational Environmental Satellite version R (GOES-R) Level 2 (L2) products were documented via Resource, Implementation, and Management Plans (RIMPs) for all of the official L2 products required from the GOES-R Advanced Baseline Imager (ABI). In 2015 the GOES-R program decided to replace the typical CalVal plans with RIMPs that covered, for a given L2 product, what was required from that product, how it would be validated, and what tools would be used to do so. Similar to Level 1b products, the intent was to cover the full spectrum of planning required for the CalVal of L2 ABI products. Instead of focusing on step-by-step procedures, the RIMPs concentrated on the criteria for each stage of the validation process (Beta, Provisional, and Full Validation) and the many elements required to prove when each stage was reached.

Validation↗

NASA NTRS2017

Development of Level 1b Calibration and Validation Readiness, Implementation and Management Plans for GOES-R

A complement of Readiness, Implementation and Management Plans (RIMPs) to facilitate management of post-launch product test activities for the official Geostationary Operational Environmental Satellite (GOES-R) Level 1b (L1b) products have been developed and documented. Separate plans have been created for each of the GOES-R sensors including: the Advanced Baseline Imager (ABI), the Extreme ultraviolet and X-ray Irradiance Sensors (EXIS), Geostationary Lightning Mapper (GLM), GOES-R Magnetometer (MAG), the Space Environment In-Situ Suite (SEISS), and the Solar Ultraviolet Imager (SUVI). The GOES-R program has implemented these RIMPs in order to address the full scope of CalVal activities required for a successful demonstration of GOES-R L1b data product quality throughout the three validation stages: Beta, Provisional and Full Validation. For each product maturity level, the RIMPs include specific performance criteria and required artifacts that provide evidence a given validation stage has been reached, the timing when each stage will be complete, a description of every applicable Post-Launch Product Test (PLPT), roles and responsibilities of personnel, upstream dependencies, and analysis methods and tools to be employed during validation. Instrument level Post-Launch Tests (PLTs) are also referenced and apply primarily to functional check-out of the instruments.

Validation↗

NASA NTRS2019

Validation of CERES Flight Model 5 In-Orbit Calibrations Using Lunar Observations

Scientific studies require radiation fluxes over the Earth to be accurate within 1% for shortwave fluxes and 0.5% for outgoing longwave fluxes. The validation of in-orbit instrument performance requires both stability in calibration source and also calibration corrections to compensate for instrument changes. The Moon offers an external source whose signal variance is predictable and non-degrading. CERES detectors register the signal output from the entire face of the Moon. Lunar observations performed by CERES Flight Models (FM) 1 through 4 have been successful in assisting validation of radiances to the required accuracy. CERES Flight Model 5 (FM-5) is on Suomi/NPP spacecraft, orbiting since October 2011. This paper uses lunar measurements to validate detector output of FM-5. These measurements are adjusted to remove orbital effects due to variations in distance between Moon and Sun, distance between the satellite and the Moon and lunar phase angle. These effects create a total variation in lunar irradiance of 20% in the total channel and 8% in the shortwave channel. The change in orientation of the Moon as seen by the detector is called libration and causes variations of about 1% of the irradiance. A consistent dataset spanning at least 2 years in length is required to remove variations due to libration. The major uncertainties remaining in the measurements are assumed to be due to changes of the spectral responses of the channels due to degradation of optical surfaces in orbit. The results demonstrate that lunar observations can be used to validate FM-5 measurements.

Clouds and Earth Radiant Energy System↗

NASA NTRSDate not supplied

Verification and Validation of Safety-Critical Aircraft Systems Operating under Off-Nominal, Contingency, and Emergency Conditions

Verification and validation (V&V) of safety-critical technologies developed for loss of control (LOC) prevention and recovery and other aviation safety concerns pose significant challenges. Aircraft LOC can result from a wide spectrum of hazards, often occurring in combination, which cannot be fully replicated during evaluation. Technologies developed for LOC prevention and recovery must therefore be effective under a wide variety of hazardous and uncertain conditions, and the verification and validation of these technologies must provide some measure of assurance that the new vehicle safety technologies do no harm (i.e., that they themselves do not introduce new safety risks). V&V technologies must also enable the identification of system limitations and constraints, as well as enable the identification of safe and unsafe operating conditions (and their boundaries). Additionally, the V&V of complex, increasingly autonomous systems is a fundamental concern. Scalable, reproducible and cost-effective techniques for the assurance of safety critical systems during their design and operation is a key barrier to fielding new systems or updating current systems. Moreover, these techniques need to provide artifacts that enable a comprehensive evidence-based approach to certification. This briefing summarizes research performed under NASA’s Aviation Safety Program and follow-on research for the V&V of safety-critical aircraft system technologies developed for LOC prevention and recovery and increasingly autonomous systems, and for a broad assurance capability in both current and emerging aviation applications. Note that, in this briefing, the term “validation” refers to a confirmation that the system implementation (e.g., algorithms etc.) is performing the intended function(s), as well as an affirmation of effectiveness in these functions. “Verification” refers to a confirmation that the system implementation in the software and hardware meets its (hopefully validated) specifications (e.g., correctly executes algorithms as designed).

Validation↗

NASA NTRS2020

The GPM Ground Validation Program

We present a detailed overview of the structure and activities associated with the NASA-led ground-validation component of the NASA-JAXA Global Precipitation Measurement (GPM) mission. The overarching philosophy and approaches for NASA's GV program are presented with primary focus placed on aspects of direct validation and a summary of physical validation campaigns and results. We describe a spectrum of key instruments, methods, field campaigns and data products developed and used by NASA's GV team to verify GPM level-2 precipitation products in rain and snow. We describe the tools and analysis framework used to confirm that NASA's Level-I science requirements for GPM are met by the GPM Core Observatory. Examples of routine validation activities related to verification of Integrated MultisatellitE Retrievals for GPM (IMERG) products for two different regions of the globe (Korea and the US) are provided, and a brief analysis related to IMERG performance in the extreme rainfall event associated with Hurricane Florence is discussed.

precipitation↗

NASA NTRS2021

The Salinity Pilot-Mission Exploitation Platform (Pi-MEP): A Hub for Validation and Exploitation of Satellite Sea Surface Salinity Data

The Pilot-Mission Exploitation Platform (Pi-MEP) for salinity is an ESA initiative originally meant to support and widen the uptake of Soil Moisture and Ocean Salinity (SMOS) mission data over the ocean. Starting in 2017, the project aims at setting up a computational web-based platform focusing on satellite sea surface salinity data, supporting studies on enhanced validation and scientific process over the ocean. It has been designed in close collaboration with a dedicated science advisory group in order to achieve three main objectives: gathering all the data required to exploit satellite sea surface salinity data, systematically producing a wide range of metrics for comparing and monitoring sea surface salinity products’ quality, and providing user-friendly tools to explore, visualize and exploit both the collected products and the results of the automated analyses. The Salinity Pi-MEP is becoming a reference hub for the validation of satellite sea surface salinity missions by providing valuable information on satellite products (SMOS, Aquarius, SMAP), an extensive in situ database (e.g., Argo, thermosalinographs, moorings, drifters) and additional thematic datasets (precipitation, evaporation, currents, sea level anomalies, sea surface temperature, etc.). Co-localized databases between satellite products and in situ datasets are systematically generated together with validation analysis reports for 30 predefined regions. The data and reports are made fully accessible through the web interface of the platform. The datasets, validation metrics and tools (automatic, user-driven) of the platform are described in detail in this paper. Several dedicated scientific case studies involving satellite SSS data are also systematically monitored by the platform, including major river plumes, mesoscale signatures in boundary currents, high latitudes, semi-enclosed seas, and the high-precipitation region of the eastern tropical Pacific. Since 2019, a partnership in the Salinity Pi-MEP project has been agreed between ESA and NASA to enlarge focus to encompass the entire set of satellite salinity sensors. The two agencies are now working together to widen the platform features on several technical aspects, such as triple-collocation software implementation, additional match-up collocation criteria and sustained exploitation of data from the SPURS campaigns

ocean↗

NASA NTRSDate not supplied

GeoCarb L2 Validation- Albedo Error Propagation into XCO2 in the Near Infrared

GeoCarb validation activities Pre-launch validation •Thermovac: Nov-Jan, 2022/3: Edwards/Dryden will move to Palo Alto Nov-March 2022/3 •Day-in-the life simulation-2022 •Simulations near Dallas, TX •Validation of TROPOMI in GeoCarb domain-2022 2GeoCarb post-launch GeoCarb will have gas precision of: •0.3% (1.2 ppm) for XCO2 •0.6% (11 ppb) for XCH4 •10% or 12 ppb for XCO GeoCarb SIF with precision of better than 0.75 W/m2/um/sr GeoCarb will utilize TCCON, GML/DOE aircraft, AirCore, and any validation site with systematic errors better than GeoCarb’s

GeoCarb↗

NASA NTRS2022

Validation of IMERG Oceanic Precipitation over Kwajalein

The integrated Multi-satellitE Retrievals for GPM (IMERG) Version V05B and V06B precipitation products from the Global Precipitation Measurement (GPM) mission are validated against ground-based observations from the Kwajalein Polarimetric S-band Weather Radar (KPOL) deployed at Kwajalein Atoll in the central Pacific Ocean. Such a validation is particularly important as comprehensive surface measurements over the oceans are practically infeasible, which hampers the identification of possible errors, and improvement of future versions of IMERG and other satellitebased retrieval algorithms. The V05B and V06B IMERG products are validated at their native 0.1°, 30 min resolution from 2014 to 2018 based on both volumetric and categorical metrics. This validation study indicates that precipitation rates from both IMERG V05B and V06B are underestimated with respect to radar surface estimates, but the underestimation is much reduced from V05B IMERG V06B outperforms V05B with reduced systematic bias and improved precipitation detectability. The IMERG performance is further traced back to its individual sensors and morphing-based algorithms. The overall underestimation in V05B is mainly driven by the negative relative biases from morphing-based algorithms which are largely corrected in V06B. Imagers perform generally better than sounders because of the usage of low-frequency channels in imagers which can better detect emission signals by the hydrometeors. Among imagers, the GPM Microwave Imager (GMI) and Advanced Microwave Scanning Radiometer Version 2 (AMSR2) are the best, followed by Special Sensor Microwave Imager/Sounder (SSMIS). Among sounders, the Microwave Humidity Sounder (MHS) is the best, followed by Advanced Technology Microwave Sounder (ATMS) and the Sounder for Atmospheric Profiling of Humidity in the Intertropics by Radiometry (SAPHIR) for V06B. Among all categories, morph-only and IR+morph only perform better than SAPHIR. SAPHIR shows the worst performance among all categories, likely due to its limited channel selection. It is envisaged that these results will improve our understanding of IMERG performance over oceans and aid in the improvement of future versions of IMERG.

KPOL↗

NASA NTRSDate not supplied

SPHINX: An SEP Model Validation Infrastructure developed through Community Challenges and the SEP Scoreboards

Solar Energetic Particle (SEP) events are interesting from a scientific perspective as they are the product of a broad set of physical processes from the corona out through the extent of the heliosphere, and provide insight into processes of particle acceleration and transport that are widely applicable in astrophysics. From the operations perspective, SEP events pose a radiation hazard for aviation, electronics in space, and human space exploration, in particular for missions outside of the Earth’s protective magnetosphere including to the Moon and Mars (Whitman et al 2022). For these reasons, SEP modelers have developed a rich and diverse set of models with a wide variety of aims. Some models probe the basic physics at the heart of particle acceleration and transport. Others produce fast statistical forecasts or employ disruptive new techniques like Machine Learning with the goal to assist end users in making operational decisions. To enable a consistent and quantitative understanding of SEP model performance, a generalized, automated validation infrastructure, called SPHINX, is being developed at NASA SRAG in close collaboration with NASA CCMC, NASA M2M, NOAA SWPC, and BIRA-IASB. This infrastructure has been built up through a multi-year community challenge. Starting in 2018 at the SHINE workshop, an effort was launched through SHINE, ISWAT, and ESWW to encourage quantitative, comprehensive, and consistent validation of SEP models. This effort has defined a set of challenge SEP events with the aim of generating quantitative comparisons between forecasts and observations and a set of challenge “non-events” to assess false alarms. In 2023, these challenge lists have been extended to statistically significant numbers with a prescribed set of rules for producing forecasts and supported through the dedicated SEPVAL working meetings. The participation of the research community has allowed the infrastructure to validate all the types of outputs being produced by SEP models. In parallel, the SPHINX code is being applied to real time forecasts submitted to the SEP Scoreboards, ensuring that the validation infrastructure can interpret forecasts produced in an operational scenario and provide metrics meaningful for operations. Upon completion, SPHINX and its interactive user interface, SPHINX-Web, will be made available for public use.

space weather↗

NASA NTRSDate not supplied

Pretest Simulations of a Supersonic Mixing and Combustion Validation Experiment to Assess Sensitivities

The reliance on CFD simulations to develop, design, and optimize scramjet systems (or components) has become commonplace. This reliance inevitably hinges on the ability of the computational analyst to quantify the level of confidence in their computational results. Unfortunately, nearly all the measured data available for this assessment comes from antiquated experimental datasets, or from tests that focused on the extraction of scramjet system (or component) performance. The objective of a CFD validation experiment is to quantify the predictive accuracy of one or more of the CFD physics submodels, implying that other uncertainties related to replicating the facility flow environment (e.g., knowledge of boundary conditions) must be minimized to the extent possible. This inevitably places stringent requirements on the quality and quantity of measurements taken to accurately specify inflow, outflow, and surface conditions for the CFD simulations; in addition to the measurements taken for the validation of physics submodels. This places additional demands on the experimental process above and beyond those for test article performance assessment. A recent high speed code credibility workshop series sponsored by AFRL identified a gap in existing validation data for fundamental assessments of turbulent mixing and combustion CFD closure models at scramjet engine relevant conditions. To address this gap, engineers at AFRL have designed a coaxial jet flame configuration that will be tested at two facilities (Research Cell 19 at the Air Force Research Lab, and at Purdue University). The effort described here documents pretest simulations of this validation experiment with the goal of fleshing out the extent of the facility flowpath that must be included to adequately reproduce the facility test section flow environment. The findings indicate that the flow around the support structure for the fuel injection centerbody upstream of the facility nozzle generates disturbances that persist throughout the nozzle expansion process; corrupting the azimuthal symmetry that was desired in the fuel/air mixing region of the test section. Simulations without this support structure maintained a high degree of azimuthal symmetry up until the fuel injection plane. However, even in this scenario the azimuthal symmetry was not maintained once the centerbody boundary layer transitioned to a wake flow downstream of the fuel injection plane.

CFD↗

NASA NTRSDate not supplied

Pretest Simulations of a Supersonic Mixing and Combustion Validation Experiment to Assess Sensitivities

The reliance on CFD simulations to develop, design, and optimize scramjet systems (or components) has become commonplace. This reliance inevitably hinges on the ability of the computational analyst to quantify the level of confidence in their computational results. Unfortunately, nearly all the measured data available for this assessment comes from antiquated experimental datasets, or from tests that focused on the extraction of scramjet system (or component) performance. The objective of a CFD validation experiment is to quantify the predictive accuracy of one or more of the CFD physics submodels, implying that other uncertainties related to replicating the facility flow environment (e.g., knowledge of boundary conditions) must be minimized to the extent possible. This inevitably places stringent requirements on the quality and quantity of measurements taken to accurately specify inflow, outflow, and surface conditions for the CFD simulations; in addition to the measurements taken for the validation of physics submodels. This places additional demands on the experimental process above and beyond those for test article performance assessment. A recent high speed code credibility workshop series sponsored by AFRL identified a gap in existing validation data for fundamental assessments of turbulent mixing and combustion CFD closure models at scramjet engine relevant conditions. To address this gap, engineers at AFRL have designed a coaxial jet flame configuration that will be tested at two facilities (Research Cell 19 at the Air Force Research Lab, and at Purdue University). The effort described here documents pretest simulations of this validation experiment with the goal of fleshing out the extent of the facility flowpath that must be included to adequately reproduce the facility test section flow environment. The findings indicate that the flow around the support structure for the fuel injection centerbody upstream of the facility nozzle generates disturbances that persist throughout the nozzle expansion process; corrupting the azimuthal symmetry that was desired in the fuel/air mixing region of the test section. Simulations without this support structure maintained a high degree of azimuthal symmetry up until the fuel injection plane. However, even in this scenario the azimuthal symmetry was not maintained once the centerbody boundary layer transitioned to a wake flow downstream of the fuel injection plane.

CFD↗