Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data validation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2

Kepler Data Validation Time Series File: Description of File Format and Content

The Kepler space mission searches its time series data for periodic, transit-like signatures. The ephemerides of these events, called Threshold Crossing Events (TCEs), are reported in the TCE tables at the NASA Exoplanet Archive (NExScI). Those TCEs are then further evaluated to create planet candidates and populate the Kepler Objects of Interest (KOI) table, also hosted at the Exoplanet Archive. The search, evaluation and export of TCEs is performed by two pipeline modules, TPS (Transit Planet Search) and DV (Data Validation). TPS searches for the strongest, believable signal and then sends that information to DV to fit a transit model, compute various statistics, and remove the transit events so that the light curve can be searched for other TCEs. More on how this search is done and on the creation of the TCE table can be found in Tenenbaum et al. (2012), Seader et al. (2015), Jenkins (2002). For each star with at least one TCE, the pipeline exports a file that contains the light curves used by TPS and DV to find and evaluate the TCE(s). This document describes the content of these DV time series files, and this introduction provides a bit of context for how the data in these files are used by the pipeline.

DV Time Series↗

Kepler Data Validation II–Transit Model Fitting and Multiple-Planet Search

This paper discusses the transit model-fitting and multiple-planet search algorithms and performance of the Kepler Science Data Processing Pipeline, developed by the Kepler Science Operations Center (SOC). Threshold crossing events (TCEs), which are transit candidate events, are generated by the Transiting Planet Search (TPS) component of the pipeline and subsequently processed in the data validation (DV) component. The transit model is used in DV to fit TCEs to characterize planetary candidates and to derive parameters that are used in various diagnostic tests to classify them. After the signature associated with the TCE is removed from the light curve of the target star, the residual light curve goes through TPS again to search for additional TCEs. The iterative process of transit model fitting and multiple-planet search continues until no TCE is generated from the residual light curve or an upper limit is reached. The transit model-fitting and multiple-planet search performance of the final release (9.3, 2016January) of the pipeline is demonstrated with the results of the processing of four years (17 quarters) of flight data from the primary Kepler Mission. The transit model-fitting results are accessible from the NASA Exoplanet Archive. The final version of the SOC codebase is available through GitHub.

Threshold crossing events (TCEs↗

SAGE II aerosol data validation - Comparative studies of SAGE II and SAM II data sets

Data from the Stratospheric Aerosol and Gas Experiment (SAGE II) satellite are compared with data from the Stratospheric Aerosol Measurement (SAM II) satellite. Both experiments produce aerosol extinction profiles by measuring the attenuation of solar radiation during each sunrise and sunset observed by the satelltie. The SAGE II obtains profiles at 1.02 microns and three smaller wavelengths, whereas the SAM II measures at only one radiometric channel at 1.0 microns. It is found that the differences between the two sets of data are generally within the error bars associated with each measurement. In addition, the sunrise and sunset data from SAGE II are analyzed.

Yue, G. K.↗

NASA TLA workload analysis support. Volume 2: Metering and spacing studies validation data

Four sets of graphic reports--one for each of the metering and spacing scenarios--are presented. The complete data file from which the reports were generated is also given. The data was used to validate the detail task of both the pilot and copilot for four metering and spacing scenarios. The output presents two measures of demand workload and a report showing task length and task interaction.

Sundstrom, J. L.↗

Assessing the Relative Performance of Microwave-based Satellite Rain Rate Retrievals using TRMM Ground Validation Data

Space-borne microwave sensors provide critical rain information used in several global multi-satellite rain products, which in turn are used for a variety of important studies, including landslide forecasting, flash flood warning, data assimilation, climate studies, and validation of model forecast of precipitation. This study employs four years (2003-2006) of satellite data to assess the relative performance and skill of SSM/I (F13, F14 and F15), AMSU-B (N15, N16 and N17), AMSR-E (AQUA) and the TRMM Microwave Imager (TMI) in estimating surface rainfall based on direct instantaneous comparison with ground-based rain estimates from Tropical Rainfall Measuring Mission (TRMM) Ground Validation (GV) sites at Kwajalein, Republic of the Marshall Islands (KWAJ) and Melbourne, Florida (MELB). The relative performance of each of these satellites is examined via comparisons with GV radar-based rain rate estimates. Because underlying surface terrain is known to affect the relative performance of the satellite algorithms, the data for MELB was further stratified into ocean, land and coast categories using a 0.25 terrain mask. Of all the satellite estimates compared in this study, TMI and AMSR-E exhibited considerably higher correlations and skills in estimating/observing surface precipitation. While SSM/I and AMSU-B exhibited lower correlations and skills for each of the different terrain categories, the SSM/I absolute biases trended slightly lower than AMSRE over ocean, where the observations from both emission and scattering channels were used in the retrievals. AMSU-B exhibited the least skill relative to GV in all of the relevant statistical categories, and an anomalous spike was observed in the probability distribution functions near 1.0 mm hr-1. This statistical artifact appears to be related to attempts by algorithm developers to include some lighter rain rates, not easily detectable by its scatter-only frequencies. AMSU-B, however, agreed well with GV when the matching data was analyzed on monthly scales. These results signal developers of global rainfall products, such as the TRMM Multi-Satellite Precipitation Analysis (TMPA), and the Climate Data Center s Morphing (CMORPH) technique, that care must be taken when incorporating data from these input satellite estimates in order to provide the highest quality estimates in their products.

Wolff, David B.↗

Assessing the Relative Performance of Microwave-Based Satellite Rain Rate Retrievals Using TRMM Ground Validation Data

Space-borne microwave sensors provide critical rain information used in several global multi-satellite rain products, which in turn are used for a variety of important studies, including landslide forecasting, flash flood warning, data assimilation, climate studies, and validation of model forecasts of precipitation. This study employs four years (2003-2006) of satellite data to assess the relative performance and skill of SSM/I (F13, F14 and F15), AMSU-B (N15, N16 and N17), AMSR-E (Aqua) and the TRMM Microwave Imager (TMI) in estimating surface rainfall based on direct instantaneous comparisons with ground-based rain estimates from Tropical Rainfall Measuring Mission (TRMM) Ground Validation (GV) sites at Kwajalein, Republic of the Marshall Islands (KWAJ) and Melbourne, Florida (MELB). The relative performance of each of these satellite estimates is examined via comparisons with space- and time-coincident GV radar-based rain rate estimates. Because underlying surface terrain is known to affect the relative performance of the satellite algorithms, the data for MELB was further stratified into ocean, land and coast categories using a 0.25 terrain mask. Of all the satellite estimates compared in this study, TMI and AMSR-E exhibited considerably higher correlations and skills in estimating/observing surface precipitation. While SSM/I and AMSU-B exhibited lower correlations and skills for each of the different terrain categories, the SSM/I absolute biases trended slightly lower than AMSRE over ocean, where the observations from both emission and scattering channels were used in the retrievals. AMSU-B exhibited the least skill relative to GV in all of the relevant statistical categories, and an anomalous spike was observed in the probability distribution functions near 1.0 mm/hr. This statistical artifact appears to be related to attempts by algorithm developers to include some lighter rain rates, not easily detectable by its scatter-only frequencies. AMSU-B, however, agreed well with GV when the matching data was analyzed on monthly scales. These results signal developers of global rainfall products, such as the TRMM Multi-Satellite Precipitation Analysis (TMPA), and the Climate Data Center s Morphing (CMORPH) technique, that care must be taken when incorporating data from these input satellite estimates in order to provide the highest quality estimates in their products. 3

Wolff, David B.↗

Assessing the Relative Performance of Microwave-Based Satellite Rain Rate Retrievals Using TRMM Ground Validation Data

Space-borne microwave sensors provide critical rain information used in several global multi-satellite rain products, which in turn are used for a variety of important studies, including landslide forecasting, flash flood warning, data assimilation, climate studies, and validation of model forecasts of precipitation. This study employs four years (2003-2006) of satellite data to assess the relative performance and skill of SSM/I (F13, F14 and F15), AMSU-B (N15, N16 and N17), AMSR-E (Aqua) and the TRMM Microwave Imager (TMI) in estimating surface rainfall based on direct instantaneous comparisons with ground-based rain estimates from Tropical Rainfall Measuring Mission (TRMM) Ground Validation (GV) sites at Kwajalein, Republic of the Marshall Islands (KWAJ) and Melbourne, Florida (MELB). The relative performance of each of these satellite estimates is examined via comparisons with space- and time-coincident GV radar-based rain rate estimates. Because underlying surface terrain is known to affect the relative performance of the satellite algorithms, the data for MELB was further stratified into ocean, land and coast categories using a 0.25deg terrain mask. Of all the satellite estimates compared in this study, TMI and AMSR-E exhibited considerably higher correlations and skills in estimating/observing surface precipitation. While SSM/I and AMSU-B exhibited lower correlations and skills for each of the different terrain categories, the SSM/I absolute biases trended slightly lower than AMSR-E over ocean, where the observations from both emission and scattering channels were used in the retrievals. AMSU-B exhibited the least skill relative to GV in all of the relevant statistical categories, and an anomalous spike was observed in the probability distribution functions near 1.0 mm/hr. This statistical artifact appears to be related to attempts by algorithm developers to include some lighter rain rates, not easily detectable by its scatter-only frequencies. AMSU-B, however, agreed well with GV when the matching data was analyzed on monthly scales. These results signal developers of global rainfall products, such as the TRMM Multi-Satellite Precipitation Analysis (TMPA), and the Climate Data Center s Morphing (CMORPH) technique, that care must be taken when incorporating data from these input satellite estimates in order to provide the highest quality estimates in their products.

Wolff, David B.↗

Interactive Framework to Support Open Model-Data Validation Efforts at the CCMC

Validation of models using observation data has been a central activity at the Community Coordinated Modeling Center (CCMC) over more than a decade. The Comprehensive Assessment of Models and Events using Library Tools (CAMEL) framework is a database-driven implementation of the interactive analysis of model-data comparisons that simultaneously provides a view across a multitude of locations and time periods. CAMEL thus extends the single-location or single-trajectory timeseries data comparison capabilities provided by the CCMC online visualization that formed the backbone of initial validation efforts. We demonstrate CAMEL capabilities for an example application to study Neutral Density in the upper atmosphere among comparisons in the heliosphere, the Earth’s radiation belt, magnetosphere, and ionosphere-thermosphere-mesosphere domains.

Lutz Rastaetter↗

Verification and Validation Data for Computational Unsteady Aerodynamics

Computational Unsteady Aerodynamics computer codes are being increasingly used. In order to validate their results they must be tested against valid experimental data. The present report aims at collecting reliable experimental data on unsteady aerodynamics and presenting them in a form which permits use for verification of codes. For ease of handling, the data are also presented in machine readable form (CD-ROM). Data on increasingly complex generic forms were selected and the following categories are covered: flutter, buffet, stability and control, dynamic stall, cavity flows, store separation. Computational solutions are included in order to permit evaluation of codes and analysis of solutions which differ from experimental data.

Applied Vehicle Technology Panel (AVT)Task Group↗

Spectrally and Radiometrically Stable Wide-Band on Board Calibration Source for In-Flight Data Validation in Imaging Spectroscopy Applications

The quality of the quantitative spectral data collected by an imaging spectrometer instrument is critically dependent upon the accuracy of the spectral and radiometric calibration of the system. In order for the collected spectra to be scientifically useful, the calibration of the instrument must be precisely known not only prior to but during data collection. Thus, in addition to a rigorous in-lab calibration procedure, the airborne instruments designed and built by the NASA/JPL Imaging Spectroscopy Group incorporate an on board calibrator (OBC) system with the instrument to provide auxiliary in-use system calibration data. The output of the OBC source illuminates a target panel on the backside of the foreoptics shutter both before and after data collection. The OBC and in-lab calibration data sets are then used to validate and post-process the collected spectral image data. The resulting accuracy of the spectrometer output data is therefore integrally dependent upon the stability of the OBC source. In this paper we describe the design and application of the latest iteration of this novel device developed at NASA/JPL which integrates a halogen-cycle source with a precisely designed fiber coupling system and a fiber-based intensity monitoring feedback loop. The OBC source in this Airborne Testbed Spectrometer was run over a period of 15 hours while both the radiometric and spectral stabilities of the output were measured and demonstrated stability to within 1% of nominal.

visible spectrometers↗

Atmospheric Correction at AERONET Locations: A New Science and Validation Data Set

This paper describes an AERONET-based Surface Reflectance Validation Network (ASRVN) and its dataset of spectral surface bidirectional reflectance and albedo based on MODIS TERRA and AQUA data. The ASRVN is an operational data collection and processing system. It receives 50x50 square kilometer subsets of MODIS L1B data from MODAPS and AERONET aerosol and water vapor information. Then it performs an accurate atmospheric correction for about 100 AERONET sites based on accurate radiative transfer theory with high quality control of the input data. The ASRVN processing software consists of L1B data gridding algorithm, a new cloud mask algorithm based on a time series analysis, and an atmospheric correction algorithm. The atmospheric correction is achieved by fitting the MODIS top of atmosphere measurements, accumulated for 16-day interval, with theoretical reflectance parameterized in terms of coefficients of the LSRT BRF model. The ASRVN takes several steps to ensure high quality of results: 1) cloud mask algorithm filters opaque clouds; 2) an aerosol filter has been developed to filter residual semi-transparent and sub-pixel clouds, as well as cases with high inhomogeneity of aerosols in the processing area; 3) imposing requirement of consistency of the new solution with previously retrieved BRF and albedo; 4) rapid adjustment of the 16-day retrieval to the surface changes using the last day of measurements; and 5) development of seasonal back-up spectral BRF database to increase data coverage. The ASRVN provides a gapless or near-gapless coverage for the processing area. The gaps, caused by clouds, are filled most naturally with the latest solution for a given pixels. The ASRVN products include three parameters of LSRT model (k(sup L), k(sup G), k(sup V)), surface albedo, NBRF (a normalized BRF computed for a standard viewing geometry, VZA=0 deg., SZA=45 deg.), and IBRF (instantaneous, or one angle, BRF value derived from the last day of MODIS measurement for specific viewing geometry) for MODIS 500m bands 1-7. The results are produced daily at resolution of 1 km in gridded format. We also provide cloud mask, quality flag and a browse bitmap image. The new dataset can be used for a wide range of applications including validation analysis and science research.

Wang, Yujie↗

MISR Science Data Validation Plan Summary Charts

The purpose of the MISR experiment is to acquire systematic multi-angle imagery for global monitoring over a multi-year period of top-of-atmosphere and surface albedos and to measure the shortwave radiative properties of aerosols, clouds, and surface scenes.

MISR science data validation plan summary charts↗

Synthesis of Multispectral Bands from Hyperspectral Data: Validation Based on Images Acquired by AVIRIS, Hyperion, ALI, and ETM+

Multispectral data requirements for Earth science applications are not always studied rigorously studied before a new remote sensing system is designed. A study of the spatial resolution, spectral bandpasses, and radiometric sensitivity requirements of real-world applications would focus the design onto providing maximum benefits to the end-user community. To support systematic studies of multispectral data requirements, the Applications Research Toolbox (ART) has been developed at NASA's Stennis Space Center. The ART software allows users to create and assess simulated datasets while varying a wide range of system parameters. The simulations are based on data acquired by existing multispectral and hyperspectral instruments. The produced datasets can be further evaluated for specific end-user applications. Spectral synthesis of multispectral images from hyperspectral data is a key part of the ART software. In this process, hyperspectral image cubes are transformed into multispectral imagery without changes in spatial sampling and resolution. The transformation algorithm takes into account spectral responses of both the synthesized, broad, multispectral bands and the utilized, narrow, hyperspectral bands. To validate the spectral synthesis algorithm, simulated multispectral images are compared with images collected near-coincidentally by the Landsat 7 ETM+ and the EO-1 ALI instruments. Hyperspectral images acquired with the airborne AVIRIS instrument and with the Hyperion instrument onboard the EO-1 satellite were used as input data to the presented simulations.

Blonksi, Slawomir↗

Availability of High Quality TRMM Ground Validation Data from Kwajalein, RMI: A Practical Application of the Relative Calibration Adjustment Technique

Since the Tropical Rainfall Measuring Mission (TRMM) satellite launch in November 1997, the TRMM Satellite Validation Office (TSVO) at NASA Goddard Space Flight Center (GSFC) has been performing quality control and estimating rainfall from the KPOL S-band radar at Kwajalein, Republic of the Marshall Islands. Over this period, KPOL has incurred many episodes of calibration and antenna pointing angle uncertainty. To address these issues, the TSVO has applied the Relative Calibration Adjustment (RCA) technique to eight years of KPOL radar data to produce Ground Validation (GV) Version 7 products. This application has significantly improved stability in KPOL reflectivity distributions needed for Probability Matching Method (PMM) rain rate estimation and for comparisons to the TRMM Precipitation Radar (PR). In years with significant calibration and angle corrections, the statistical improvement in PMM distributions is dramatic. The intent of this paper is to show improved stability in corrected KPOL reflectivity distributions by using the PR as a stable reference. Inter-month fluctuations in mean reflectivity differences between the PR and corrected KPOL are on the order of 1-2 dB, and inter-year mean reflectivity differences fluctuate by approximately 1 dB. This represents a marked improvement in stability with confidence comparable to the established calibration and uncertainty boundaries of the PR. The practical application of the RCA method has salvaged eight years of radar data that would have otherwise been unusable, and has made possible a high-quality database of tropical ocean-based reflectivity measurements and precipitation estimates for the research community.

Marks, David A.↗

Retrievals of Cloud Droplet Size from the RSP Data: Validation Using in Situ Measurements

We present comparisons of cloud droplet size distributions retrieved from the Research Scanning Polarimeter (RSP) data with correlative in situ measurements made during the North Atlantic Aerosols and Marine Ecosystems Study (NAAMES). This field experiment was based at St. Johns airport, Newfoundland, Canada with the latest deployment in May - June 2016. RSP was onboard the NASA C-130 aircraft together with an array of in situ and other remote sensing instrumentation. The RSP is an along-track scanner measuring polarized and total reflectances in9 spectral channels. Its unique high angular resolution allows for characterization of liquid water droplet size using the rainbow structure observed in the polarized reflectances in the scattering angle range between 135 and 165 degrees. A parametric fitting algorithm applied to the polarized reflectances provides retrievals of the droplet effective radius and variance assuming a prescribed size distribution shape (gamma distribution). In addition to this, we use a non-parametric method, Rainbow Fourier Transform (RFT), which allows us to retrieve the droplet size distribution (DSD) itself. The latter is important in the case of clouds with complex structure, which results in multi-modal DSDs. During NAAMES the aircraft performed a number of flight patterns specifically designed for comparison of remote sensing retrievals and in situ measurements. These patterns consisted of two flight segments above the same straight ground track. One of these segments was flown above clouds allowing for remote sensing measurements, while the other was at the cloud top where cloud droplets were sampled. We compare the DSDs retrieved from the RSP data with in situ measurements made by the Cloud Droplet Probe (CDP). The comparisons show generally good agreement with deviations explainable by the position of the aircraft within cloud and by presence of additional cloud layers in RSP view that do not contribute to the in situ DSDs. In the latter case the distributions retrieved from the RSP data were consistent with the multi-layer cloud structures observed in the correlative High Spectral Resolution Lidar (HSRL) profiles. The comparison results provide a rare validation of polarimetric droplet size retrieval techniques, which can be used for analysis of satellite data on global scale.

cloud droplets↗

Metric analysis and data validation across FORTRAN projects

The desire to predict the effort in developing or explaining the quality of software has led to the proposal of several metrics. As a step toward validating these metrics, the Software Engineering Laboratory (SEL) has analyzed the software science metrics, cyclomatic complexity, and various standard program measures for their relation to effort (including design through acceptance testing), development errors (both discrete and weighted according to the amount of time to locate and fix), and one another. The data investigated are collected from a project FORTRAN environment and examined across several projects at once, within individual projects and by reporting accuracy checks demonstrating the need to validate a database. When the data comes from individual programmers or certain validated projects, the metrics' correlations with actual effort seem to be strongest. For modules developed entirely by individual programmers, the validity ratios induce a statistically significant ordering of several of the metrics' correlations. When comparing the strongest correlations, neither software science's E metric cyclomatic complexity not source lines of code appears to relate convincingly better with effort than the others.

Basili, Victor R.↗

Retrievals of Cloud Droplet Size from the Research Scanning Polarimeter Data: Validation Using In Situ Measurements

We present comparisons of cloud droplet size distributions (DSDs) retrieved from the research scanning polarimeter (RSP) data with correlative in situ measurements made during the North Atlantic Aerosols and Marine Ecosystems Study (NAAMES). The airborne portion of this field experiment was based out of St. John's airport, Newfoundland, Canada with the focus of this paper being on the deployment in May - June 2016. RSP was onboard the NASA C-130 aircraft together with an array of in situ and other remote sensing instrumentation. The RSP is an along-track scanner measuring the polarized and total reflectance in 9 spectral channels. Its uniquely high angular resolution allows for characterization of liquid water droplet sizes using the rainbow structure observed in the polarized reflectance over the scattering angle range from 135 to 165.degrees The rainbow is dominated by single scattering of light by cloud droplets, so its structure is characteristic specifically of the droplet sizes at cloud top (within unit optical depth into the cloud, equivalent to approximately 50m). A parametric fitting algorithm applied to the polarized reflectance provides retrievals of the droplet effective radius and variance assuming a prescribed size distribution shape (gamma distribution). In addition to this, we use a non-parametric method, the Rainbow Fourier Transform (RFT), which allows us to retrieve the droplet size distribution itself. The latter is important in the case of clouds with complex microphysical structure, or multiple layers of cloud, which result in multi-modal DSDs. During NAAMES the aircraft performed a number of flight patterns specifically designed for comparisons between remote sensing retrievals and in situ measurements. These patterns consisted of two flight segments above the same straight ground track. One of these segments was flown above clouds allowing for remote sensing measurements, while the other was near the cloud top where cloud droplets were sampled. We compare the DSDs retrieved from the RSP data with in situ measurements made by the Cloud Droplet Probe (CDP). The comparisons generally show good agreement (better than 1 micron for effective radius and in most cases better than 0.02 for effective variance) with deviations explainable by the position of the aircraft within the cloud, or by the presence of additional cloud layers between the cloud being sampled by the in situ instrumentation and the altitude of the remote sensing segment. In the latter case, the multi-modal DSDs retrieved from the RSP data were consistent with the multi-layer cloud structures observed in the correlative High Spectral Resolution Lidar (HSRL) profiles. The results of these comparisons provide a rare validation of polarimetric droplet size retrieval techniques, demonstrating their accuracy and robustness and the potential of satellite data of this kind on a global scale.

Remote Sensing↗

MODIS Observation of Aerosols over Southern Africa During SAFARI 2000: Data, Validation, and Estimation of Aerosol Radiative Forcing

Aerosol properties, including optical thickness and size parameters, are retrieved operationally from the MODIS sensor onboard the Terra satellite launched on 18 December 1999. The predominant aerosol type over the Southern African region is smoke, which is generated from biomass burning on land and transported over the southern Atlantic Ocean. The SAFARI-2000 period experienced smoke aerosol emissions from the regular biomass burning activities as well as from the prescribed burns administered on the auspices of the experiment. The MODIS Aerosol Science Team (MAST) formulates and implements strategies for the retrieval of aerosol products from MODIS, as well as for validating and analyzing them in order to estimate aerosol effects in the radiative forcing of climate as accurately as possible. These activities are carried out not only from a global perspective, but also with a focus on specific regions identified as having interesting characteristics, such as the biomass burning phenomenon in southern Africa and the associated smoke aerosol, particulate, and trace gas emissions. Indeed, the SAFARI-2000 aerosol measurements from the ground and from aircraft, along with MODIS, provide excellent data sources for a more intensive validation and a closer study of the aerosol characteristics over Southern Africa. The SAFARI-2000 ground-based measurements of aerosol optical thickness (AOT) from both the automatic Aerosol Robotic Network (AERONET) and handheld Sun photometers have been used to validate MODIS retrievals, based on a sophisticated spatio-temporal technique. The average global monthly distribution of aerosol from MODIS has been combined with other data to calculate the southern African aerosol daily averaged (24 hr) radiative forcing over the ocean for September 2000. It is estimated that on the average, for cloud free conditions over an area of 9 million square kin, this predominantly smoke aerosol exerts a forcing of -30 W/square m C lose to the terrestrial surface and -10 W/square m at the top of the atmosphere (TOA). While cooling the surface and Earth system, the difference of 20 W/square m is energy that heats the atmosphere.

Ichoku, Charles↗