SEARCH · Engineering Papers

Results for “data quality”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 235 records · Page 13

All sources NASA NTRS DOE OSTI

NASA NTRS2000

SeaWiFS Postlaunch Calibration and Validation Analyses

The effort to resolve data quality issues and improve on the initial data evaluation methodologies of the SeaWiFS Project was an extensive one. These evaluations have resulted, to date, in three major reprocessings of the entire data set where each reprocessing addressed the data quality issues that could be identified up to the time of the reprocessing. Three volumes of the SeaWiFS Postlaunch Technical Report Series (Volumes 9, 10, and 11) are needed to document the improvements implemented since launch. Volume 10 continues the sequential presentation of postlaunch data analysis and algorithm descriptions begun in Volume 9. Chapter 1 of Volume 10 describes an absorbing aerosol index, similar to that produced by the Total Ozone Mapping Spectrometer (TOMS) Project, which is used to flag pixels contaminated by absorbing aerosols, such as, dust and smoke. Chapter 2 discusses the algorithm being used to remove SeaWiFS out-of-band radiance from the water-leaving radiances. Chapter 3 provides an itemization of all significant changes in the processing algorithms for each of the first three reprocessings. Chapter 4 shows the time series of global clear water and deep-water (depths greater than 1,000m) bio-optical and atmospheric properties (normalized water-leaving radiances, chlorophyll, atmospheric optical depth, etc.) based on the eight-day composites as a check on the sensor calibration stability. Chapter 5 examines the variation in the derived products with scan angle using high resolution data around Hawaii to test for residual scan modulation effects and atmospheric correction biases. Chapter 6 provides a methodology for evaluating the atmospheric correction algorithm and atmospheric derived products using ground-based observations. Similarly, Chapter 7 presents match-up comparisons of coincident satellite and in situ data to determine the accuracy of the water-leaving radiances, chlorophyll a, and K(490) products.

Hooker, Stanford B.↗

NASA NTRS2008

Sensor Selection and Data Validation for Reliable Integrated System Health Management

For new access to space systems with challenging mission requirements, effective implementation of integrated system health management (ISHM) must be available early in the program to support the design of systems that are safe, reliable, highly autonomous. Early ISHM availability is also needed to promote design for affordable operations; increased knowledge of functional health provided by ISHM supports construction of more efficient operations infrastructure. Lack of early ISHM inclusion in the system design process could result in retrofitting health management systems to augment and expand operational and safety requirements; thereby increasing program cost and risk due to increased instrumentation and computational complexity. Having the right sensors generating the required data to perform condition assessment, such as fault detection and isolation, with a high degree of confidence is critical to reliable operation of ISHM. Also, the data being generated by the sensors needs to be qualified to ensure that the assessments made by the ISHM is not based on faulty data. NASA Glenn Research Center has been developing technologies for sensor selection and data validation as part of the FDDR (Fault Detection, Diagnosis, and Response) element of the Upper Stage project of the Ares 1 launch vehicle development. This presentation will provide an overview of the GRC approach to sensor selection and data quality validation and will present recent results from applications that are representative of the complexity of propulsion systems for access to space vehicles. A brief overview of the sensor selection and data quality validation approaches is provided below. The NASA GRC developed Systematic Sensor Selection Strategy (S4) is a model-based procedure for systematically and quantitatively selecting an optimal sensor suite to provide overall health assessment of a host system. S4 can be logically partitioned into three major subdivisions: the knowledge base, the down-select iteration, and the final selection analysis. The knowledge base required for productive use of S4 consists of system design information and heritage experience together with a focus on components with health implications. The sensor suite down-selection is an iterative process for identifying a group of sensors that provide good fault detection and isolation for targeted fault scenarios. In the final selection analysis, a statistical evaluation algorithm provides the final robustness test for each down-selected sensor suite. NASA GRC has developed an approach to sensor data qualification that applies empirical relationships, threshold detection techniques, and Bayesian belief theory to a network of sensors related by physics (i.e., analytical redundancy) in order to identify the failure of a given sensor within the network. This data quality validation approach extends the state-of-the-art, from red-lines and reasonableness checks that flag a sensor after it fails, to include analytical redundancy-based methods that can identify a sensor in the process of failing. The focus of this effort is on understanding the proper application of analytical redundancy-based data qualification methods for onboard use in monitoring Upper Stage sensors.

Garg, Sanjay↗

NASA NTRS2000

SeaWiFS Postlaunch Calibration and Validation Analyses

The effort to resolve data quality issues and improve on the initial data evaluation methodologies of the Sea-viewing Wide Field-of-view Sensor (SeaWiFS) Project was an extensive one. These evaluations have resulted, to date, in three major reprocessings of the entire data set where each reprocessing addressed the data quality issues that could be identified up to the time of each reprocessing. The number of chapters (21) needed to document this extensive work in the SeaWiFS Postlaunch Technical Report Series requires three volumes. The chapters in Volumes 9, 10, and 11 are in a logical order sequencing through sensor calibration, atmospheric correction, masks and flags, product evaluations, and bio-optical algorithms. The first chapter of Volume 9 is an overview of the calibration and validation program, including a table of activities from the inception of the SeaWiFS Project. Chapter 2 describes the fine adjustments of sensor detector knee radiances, i.e., radiance levels where three of the four detectors in each SeaWiFS band saturate. Chapters 3 and 4 describe the analyses of the lunar and solar calibration time series, respectively, which are used to track the temporal changes in radiometric sensitivity in each band. Chapter 5 outlines the procedure used to adjust band 7 relative to band 8 to derive reasonable aerosol radiances in band 7 as compared to those in band 8 in the vicinity of Lanai, Hawaii, the vicarious calibration site. Chapter 6 presents the procedure used to estimate the vicarious calibration gain adjustment factors for bands 1-6 using the waterleaving radiances from the Marine Optical Buoy (MOBY) offshore of Lanai. Chapter 7 provides the adjustments to the coccolithophore flag algorithm which were required for improved performance over the prelaunch version. Chapter 8 is an overview of the numerous modifications to the atmospheric correction algorithm that have been implemented. Chapter 9 describes the methodology used to remove artifacts of sun glint contamination for portions of the imagery outside the sun glint mask. Finally, Chapter 10 explains a modification to the ozone interpolation method to account for actual time differences between the SeaWiFS and Total Ozone Mapping Spectrometer (TOMS) orbits.

Hooker, Stanford B.↗

NASA NTRSDate not supplied

TPSAS-NF1676L-32668-DND

The Stratospheric Aerosol and Gas Experiment (SAGE) III is a solar occultation instrument installed on the International Space Station (ISS) that has been providing vertically resolved measurements of aerosol extinction, ozone concentrations, and other trace gases. With a legacy going back to the late 1970s, SAGE ozone measurements have been a vital contributor to international ozone assessments and analyses on stratospheric ozone trends. As the newest SAGE instrument, SAGE III/ISS will continue to provide high precision profiles of ozone that will help determine if the Montreal Protocol has been successful in enabling the recovery of the ozone layer. However, as is a necessary step for all new instruments, a rigorous validation of the data quality is required to determine if the data is sufficient for incorporation into trend analyses. Herein we discuss an initial evaluation of the data quality of vertical profiles of ozone with a focus on internal consistency for the first validation version (v5.1) derived from SAGE III/ISS measurements.

R Damadeo↗

NASA NTRS2020

Precision Assessment of the HPLC Phytoplankton Pigment Dataset Analyzed by NASA to Quantify Global Variability in Support of Ocean Color Remote Sensing

The ability to generate chlorophyll a (Chl a) assessments from ocean color orbital sensors, such as VIIRS and MODIS, that satisfy the requirements to be climate-quality data record (CDR) quality is contingent in part on the quality of the in situ ground or sea truth observations that serve as datasets for vicarious calibration and algorithm validation activities. NASA has a mandate to collect, analyze, and distribute in situ data of the highest possible quality with documented uncertainties and in keeping with established performance metrics. Using a dataset of over 18,000 HPLC phytoplankton pigment samples representing water collected in all major ocean basins analyzed a central laboratory (Field Support Group (FSG) of the Ocean Ecology Laboratory (OEL) at NASA Goddard Space Flight Center (GSFC)), we performed an assessment of the global precision among sample replicates of Chl a as well as major accessory pigments. We investigated the impacts of filtration volume, water basin, collection technique, pigment concentration, and different filtration volumes for replicate filters on replicate filter precision, as well as investigating any pigment-specific differences. Our results quantify sample variability with the goal of understanding any systemic biases or biogeographic influences.

Thomas, Crystal S.↗

NASA NTRS1986

A modeling analysis program for the JPL Table Mountain Io sodium cloud data

Progress and achievements in the second year are discussed in three main areas: (1) data quality review of the 1981 Region B/C images; (2) data processing activities; and (3) modeling activities. The data quality review revealed that almost all 1981 Region B/C images are of sufficient quality to be valuable in the analyses of the JPL data set. In the second area, the major milestone reached was the successful development and application of complex image-processing software required to render the original image data suitable for modeling analysis studies. In the third area, the lifetime description of sodium atoms in the planet magnetosphere was improved in the model to include the offset dipole nature of the magnetic field as well as an east-west electric field. These improvements are important in properly representing the basic morphology as well as the east-west asymmetries of the sodium cloud.

Smyth, W. H.↗

NASA NTRS2010

Applications of OMI Tropospheric NO(Sub 2) Data: Air Quality Trends in the US and Lightning NO(x) Source Strength

This talk will be presented in two parts: 1) an analysis of tropospheric column NO2 trends in the eastern half of the United States over the period 2005 to 2009 and 2) estimation of lightning NO(x) production rates based on OMI observations and lightning flash rate data. The air quality trends in the eastern US will be determined for specific subregions using tropospheric column NO2 data from OMI for 2005 through 2008 and from GOME-2 for 2007 through 2009. This period is characterized by significant NO(x) emission reductions at power plants within most of this region. The air quality trends will be compared with those estimated from continuous emission monitoring data from the power plants compiled by the US Environmental Protection Agency. OMI NO2 data have also been used to estimate lightning NOx production per flash in selected storms near Costa Rica and Panama during the 2007 NASA TC4 field campaign and over the continental US, Gulf of Mexico, and western Atlantic during the summers of 2005 and 2006. The lightning signal is extracted from the OMI data through a custom retrieval in which an NO2 profile representative of convective outflow is used in the airmass factor calculation and the background NO2 column is subtracted from the tropospheric column. When combined with NO(x)/NO2 ratios from the NASA GMT model and observed flash rates, the resulting estimates of NO(x) production per flash are comparable to those estimated obtained from analyses of aircraft data and cloud-resolving modeling.

Pickering, Kenneth↗

NASA NTRS1984

Selection of a map grid for data analysis and archival

Arguments for selection of a map grid are reiterated by illustrating the quantitative effects on data quality caused by using different grids. It is shown that the use of the rectangular latitude-longitude grid actually degrades data quality, increases stored data volume, and increases the complexity of data manipulation. Use of an equal-area grid is shown to be a proper way to aggregate data, but such grids are thought to be inconvenient. A solution to this dilemma is proposed by showing that the analysis and archival map grids need not be the same. The results of a proper analysis of the data on an equal-area grid can be remapped to the more 'convenient' rectangular latitude-longitude grid without loss of quality.

Rossow, W. B.↗

NASA NTRS2018

Challenges in Development of Online Visualization and Analysis Tools for Satellite Data

Over the years, various online visualization and analysis tools have been developed to facilitate satellite data access and help scientific users around the world to conduct research and develop applications (e.g., data product evaluation, what-if questions, etc.). For those who are new to satellite data products, using them can be a daunting task due to many obstacles in data processing such as data formats, complex data structures, special software packages, unfamiliar terminology, etc., especially when one is not sure whether a dataset is suitable for his/er research project. Even for experienced users, developing software for data processing and analysis can be a costly and time-consuming task. Online visualization tools can overcome many of these difficulties and allow users to focus on scientific questions. For example, Giovanni (the Geospatial Interactive Online Visualization and Analysis Infrastructure, https://giovanni.gsfc.nasa.gov), developed by the NASA Goddard Earth Sciences Data and Information Services Center (GES DISC), allows access over 1900 satellite and model variables in 82 measurement groups of 8 disciplines without downloading data and software. Main features include basic functions for data analysis and visualization, data provenance, output data in different formats (ASCII, NetCDF, GeoTIFF), and more. Over the years, ~1700 peer-reviewed publications in different disciplines have been benefited from Giovanni in research activities (e.g. initial investigation, what-if questions, product evaluation). Despite the success of online visualization and analysis tools, challenges and new opportunities still exist and more can be done with new requirements and technology. Examples are: a) how to increase the efficiency of dataset search by enhancing intuitive aspects; b) how to facilitate interdisciplinary research; c) how to provide data quality information; d) how to engage users to participate in data quality assessment; and more. NASA Earth Observing System Data and Information System (EOSDIS) satellite-based data products are processed at various levels ranging from Level 0 to Level 4. While most users use data products at higher levels (Level-3 and 4), products at lower levels are still important for case studies, algorithm development, ground validation, etc. In this presentation, we will use Giovanni as an example to present and discuss challenges and near-future opportunities for satellite data online visualization and analysis tools.

Liu, Zhong↗

NASA NTRS1977

Land use change detection with LANDSAT-2 data for monitoring and predicting regional water quality degradation

The author has identified the following significant results. Comparison between LANDSAT 1 and 2 imagery of Arkansas provided evidence of significant land use changes during the 1972-75 time period. Analysis of Arkansas historical water quality information has shown conclusively that whereas point source pollution generally can be detected by use of water quality data collected by state and federal agencies, sampling methodologies for nonpoint source contamination attributable to surface runoff are totally inadequate. The expensive undertaking of monitoring all nonpoint sources for numerous watersheds can be lessened by implementing LANDSAT change detection analyses.

Macdonald, H.↗

NASA NTRS2020

Improving Quantitative Laboratory Analysis of Phycobiliproteins to Provide High Quality Validation Data for Ocean Color Remote Sensing Algorithm

Identification and characterization of phytoplankton communities and their physiology is a primary aim of NASA's PACE satellite mission. The concentration and composition of phytoplankton pigments modulate the spectral distribution of light emanating from the ocean, which is measured by ocean color satellites, and thus provide critical information on phytoplankton community composition and physiological parameters. One diagnostic class of pigments not routinely well-characterized is the phycobiliproteins (PBPs), and NASA has a requirement to collect and distribute high quality in situ data in support of data product validation activities for ocean color missions. Phycobiliproteins are light-harvesting proteins that are the predominant photosynthetic pigments in some classes of phytoplankton including cyanobacteria, such as Synechococcus, Trichodesmium, and Microcystis. With the advance of hyperspectral ocean color sensors such as on PACE (expected to launch in late 2022), it is essential that we implement routine analysis of PBPs that satisfies several considerations: reproducible, high extraction efficiency for a variety of environments, and Suitable for large scale analysis. Published techniques for PBP analysis vary in recommendations for: collection, extraction, disruption mode, and analysis; evidence suggests the variation in results may depend at least in part on the species and even strain(s) of interest. Experiments that tested variations in these parameters have drawn very different conclusions regarding extraction efficiency and reproducibility. Cyanobacteria are more difficult to extract than other PBP-containing algae such as cryptophytes, but can be important primary producers. We used a cryptophyte (Rhodomonas salina) and cyanobacterium (Synechococcus sp.) to compare extraction efficiencies of water samples concentrated via centrifugation to filtered samples using two different extraction buffers (phosphate and asolectin-CHAPS). Samples were analyzed on a fluorometer configured for PC and PE detection. The results have important implications for collection and storage of samples for routine analysis; some previous studies (although not all) have suggested that filtered samples have a much lower extraction efficiency than whole water samples.

Kenemer, Christopher↗

NASA NTRS1993

Results of the second Round Robin on opening-load measurement conducted by ASTM Task Group E24.04.04 on crack closure measurement and analysis

A second experimental Round Robin on the measurement of the crack opening load in fatigue crack growth tests has been completed by the ASTM Task Group E24.04.04 on Crack Closure Measurement and Analysis. Fourteen laboratories participated in the testing of aluminum alloy compact tension specimens. Opening-load measurements were made at three crack lengths during constant Delta K, constant stress ratio tests by most of the participants. Four participants made opening-load measurements during threshold tests. All opening-load measurements were based on the analysis of specimens compliance behavior, where the displacement/strain was measured either at the crack mouth or the mid-height back face location. The Round Robin data were analyzed for opening load using two non-subjective analysis methods: the compliance offset and the correlation coefficient methods. The scatter in the opening load results was significantly reduced when some of the results were excluded from the analysis population based on an accept/reject criterion for raw data quality. The compliance offset and correlation coefficient opening load analysis methods produced similar results for data populations that had been screened to eliminate poor quality data.

Phillips, E. P.↗

DOE OSTI2025

Anomaly Detection in Seismic Data with Deep Learning: Application for Instrument Failure Detection and Forecasting

Seismic data quality assessment (QA) is the first and one of the most important steps before conducting any further data analysis. Traditional methods involve checking various metrics, such as spike detection and power spectral density, by setting strict thresholds or comparing data against synthetic benchmarks. However, these approaches often rely on pre-existing knowledge and assumptions about data anomalies, leading to potential misclassification of unusual cases. Here, in this study, we propose a deep autoencoder model, an unsupervised learning approach that evaluates data quality without making assumptions about normal and anomalous data, which can be used to identify deviations in recorded data that may indicate nascent instrument failure. We test the model with the U.S. International Monitoring System (IMS) seismic stations and demonstrate the capability of detecting anomalies on a monthly scale. This could prompt station operators to examine potential problems early, allowing sufficient time for instrument maintenance to prevent data outages. In addition, we use a new manually selected testing dataset to compare our model performance against two supervised machine learning (ML) approaches and a standard QA package, as baseline models. When applied to the dataset containing known data anomalies, performance of the supervised and unsupervised ML approaches is similar, with an accuracy of 88.1% for our model compared to ∼90% for the supervised ML approach and 78.2% for the standard QA package. Our model outperforms the baseline models when applied to new stations, where new types of data anomalies can be station-specific and not included in the training dataset. Finally, we show model transferability by training the model with data from the Global Seismograph Network only and applying it to the IMS network data. The results suggest that our model is generalizable and can be applied to new stations with good accuracy.

Lin, Jiun-Ting [Lawrence Livermore National Labora↗

NASA NTRS2010

HOPE: An On-Line Piloted Handling Qualities Experiment Data Book

A novel on-line database for capturing most of the information obtained during piloted handling qualities experiments (either flight or simulated) is described. The Hyperlinked Overview of Piloted Evaluations (HOPE) web application is based on an open-source object-oriented Web-based front end (Ruby-on-Rails) that can be used with a variety of back-end relational database engines. The hyperlinked, on-line data book approach allows an easily-traversed way of looking at a variety of collected data, including pilot ratings, pilot information, vehicle and configuration characteristics, test maneuvers, and individual flight test cards and repeat runs. It allows for on-line retrieval of pilot comments, both audio and transcribed, as well as time history data retrieval and video playback. Pilot questionnaires are recorded as are pilot biographies. Simple statistics are calculated for each selected group of pilot ratings, allowing multiple ways to aggregate the data set (by pilot, by task, or by vehicle configuration, for example). Any number of per-run or per-task metrics can be captured in the database. The entire run metrics dataset can be downloaded in comma-separated text for further analysis off-line. It is expected that this tool will be made available upon request

Jackson, E. B.↗

DOE OSTI2024

MIBiG 4.0: advancing biosynthetic gene cluster curation through global collaboration

Specialized or secondary metabolites are small molecules of biological origin, often showing potent biological activities with applications in agriculture, engineering and medicine. Usually, the biosynthesis of these natural products is governed by sets of co-regulated and physically clustered genes known as biosynthetic gene clusters (BGCs). To share information about BGCs in a standardized and machine-readable way, the Minimum Information about a Biosynthetic Gene cluster (MIBiG) data standard and repository was initiated in 2015. Since its conception, MIBiG has been regularly updated to expand data coverage and remain up to date with innovations in natural product research. Here, we describe MIBiG version 4.0, an extensive update to the data repository and the underlying data standard. In a massive community annotation effort, 267 contributors performed 8304 edits, creating 557 new entries and modifying 590 existing entries, resulting in a new total of 3059 curated entries in MIBiG. Particular attention was paid to ensuring high data quality, with automated data validation using a newly developed custom submission portal prototype, paired with a novel peer-reviewing model. MIBiG 4.0 also takes steps towards a rolling release model and a broader involvement of the scientific community. MIBiG 4.0 is accessible online at https://mibig.secondarymetabolites.org/.

59 BASIC BIOLOGICAL SCIENCES↗

DOE OSTI2025

ARM Cloud and Precipitation Measurements and Science Group (CPMSG) 2024 Workshop Report

The mission of the U.S. Department of Energy (DOE) Atmospheric Radiation Measurement (ARM) user facility is to improve the understanding and representation of cloud and aerosol processes and their interaction with the Earth's surface in Earth system models (ESMs) by providing comprehensive field observations and supporting advanced data analytics. The ARM Cloud and Precipitation Measurements and Science Group (CPMSG) was chartered in March 2019 to help improve the performance and scientific impact of ARM measurements of clouds and precipitation. The group aims to identify and address gaps in measurement capabilities, maximize the scientific impact of ARM data, and effectively serve the scientific community. To achieve these goals, the group includes experts in cloud and precipitation science, as well as representatives from ARM infrastructure, including instrument mentors, engineers, data quality officers, and data product translators. Prior to CPMSG, early discussions on cloud and precipitation measurements primarily focused on improving radar systems, but have since evolved to include a broader scope involving radiometers and other instruments. Since its formation, the CPMSG has gathered feedback using science traceability matrices. CPMSG aims to keep these as living documents to show the measurement needs, scientific drivers, roadblocks, maturity of measurements and retrievals, and pathways to model improvements. The group meets quarterly to discuss and prioritize measurement and operational improvements.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2025

ARM Cloud and Precipitation Measurements and Science Group (CPMSG) 2024 Workshop Report

54 ENVIRONMENTAL SCIENCES↗

NASA NTRS1982

LANDSAT-D assessment system library computer compatible tape (LASLIB-CCT/LAS-CCT)

The format for computer compatible tapes containing thematic mapper archival data and product data generated by the LANDSAT-D Assessment System (LAS). These data tapes are engineering products developed primarily for LAS internal use to assist in the early evaluation of thematic mapper data quality and ground data processing algorithms during the limited period of one year following the launch of LANDSAT-D.

Source record↗