Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “sensor data validation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 685 records · Page 38
All sourcesNASA NTRSDOE OSTI
NASA NTRS2010

A Modular Instrumentation System for NASA's Habitat Demonstration Unit

NASA's human spaceflight program is focused on developing technologies to expand the reaches of human exploration and science activities beyond low earth orbit. A critical aspect of living in space or on planetary surfaces is habitation, which provides a safe and comfortable space in which humans can live and work. NASA is seeking out the best option for habitation by exploring several different concepts through the Habitat Demonstration Unit (HDU) project. The purpose of this HDU is to develop a fully autonomous habitation system that enables human exploration of space. One critical feature of the HDU project that helps to accomplish its mission of autonomy is the instrumentation system that monitors key subsystems operating within a Habitat configuration. The following paper will discuss previous instrumentation systems used in analog habitat concepts and how the current instrumentation system being implemented on the HDU1-PEM, or pressurized excursion module, is building upon the lessons learned of those previous systems. Additionally, this paper will discuss the benefits and the limitations of implementing a wireless sensor network (WSN) as the basis for data transport in the instrumentation system. Finally, this paper will address the experiences and lessons learned with integration, testing prior to deployment, and field testing at the JSC rock yard. NASA is developing the HDU1-PEM as a step towards a fully autonomous habitation system that enables human exploration of space. To accomplish this purpose, the HDU project is focusing on development, integration, testing, and evaluation of habitation systems. The HDU will be used as a technology pull, testbed, and integration environment in which to advance NASA's understanding of alternative mission architectures, requirements, and operations concepts definition and validation. This project is a multi-year effort. In 2010, the HDU1-PEM will be in a pressurized excursion module configuration, and in 2011 the module will be reconfigured for a pressurized core module configuration. Each year the HDU configurations will undergo testing at NASA's Desert Research and Technology Studies (D-RaTS) in Arizona [1]. As part of this project, a modular instrumentation system is developed to meet the monitoring needs of the HDU subsystems and to integrate with the current command and data handling infrastructure that has been developed for the project. The main objective of this study is to provide for the monitoring needs of the HDU. The requirements necessary to meet this objective are developed by working with the subsystem managers of the HDU to understand their monitoring needs. Additionally, the instrumentation system design leverages knowledge and lessons learned from previous studies, such as the inflatable habitat health monitoring system that was deployed in Antarctica [2], the integrated health monitoring system developed for NASA's Microhab [3], and the JSC Lunar Habitat Wireless Testbed to demonstrate a "standardsbased" approach to a wireless instrumentation system [4]. The HDU also requires flexibility in reconfiguration options, and it is necessary to demonstrate and evaluate a modular approach to an instrumentation system. Thus, the instrumentation system is designed in two parts: the primary system employs a standard WSN configuration, and the secondary system employs a wired USB hub. The WSN design provides for reconfiguration or replacement of sensors due to malfunctions or upgrades by using a wireless node that accepts ten instrument inputs and wirelessly transmits the data to the command and data handling system. The USB hub is necessary for those instruments that operate using a wired USB connection, although the design attempts to limit the amount of sensors that need to be wired connections.

Rojdev, Kristina↗
NASA NTRS2003

Using X-band Weather Radar Measurements to Monitor the Integrity of Digital Elevation Models for Synthetic Vision Systems

Synthetic Vision Systems (SVS) provide pilots with displays of stored geo-spatial data representing terrain, obstacles, and cultural features. As comprehensive validation is impractical, these databases typically have no quantifiable level of integrity. Further, updates to the databases may not be provided as changes occur. These issues limit the certification level and constrain the operational context of SVS for civil aviation. Previous work demonstrated the feasibility of using a realtime monitor to bound the integrity of Digital Elevation Models (DEMs) by using radar altimeter measurements during flight. This paper describes an extension of this concept to include X-band Weather Radar (WxR) measurements. This enables the monitor to detect additional classes of DEM errors and to reduce the exposure time associated with integrity threats. Feature extraction techniques are used along with a statistical assessment of similarity measures between the sensed and stored features that are detected. Recent flight-testing in the area around the Juneau, Alaska Airport (JNU) has resulted in a comprehensive set of sensor data that is being used to assess the feasibility of the proposed monitor technology. Initial results of this assessment are presented.

Young, Steve↗
NASA NTRS1977

Reliability measurement during software development

Measurement of software reliability was carried out during the development of data base software for a multi-sensor tracking system. Every run made during this project was scored as success or failure, and supporting data were collected on forms for further analysis. The failure ratio (number of failures per calendar interval divided by total number of runs) and failure rate (number of failures divided by CPU time for the interval) were found to be consistent measures, on a month-to-month basis as well as from module to module, and therefore considered valid indicators of reliability in this environment. Trend lines could be established from these measurements that provide good visualization of the progress on the job as a whole as well as on individual modules. Over one-half of the observed failures were due to factors associated with the specific run submission rather than with the code proper.

Hecht, H.↗
NASA NTRS1979

Application of satellite time transfer in autonomous spacecraft clocks

The conceptual design of a spacecraft clock that will provide a standard time scale for experimenters in future spacecraft., and can be sychronized to a time scale without the need for additional calibration and validation is described. The time distribution to the users is handled through onboard computers, without human intervention for extended periods. A group parallel binary code, under consideration for onboard use, is discussed. Each group in the code can easily be truncated. The autonomously operated clock not only achieves simpler procedures and shorter lead times for data processing, but also contributes to spacecraft autonomy for onboard navigation and data packetization. The clock can be used to control the sensor in a spacecraft, compare another time signal such as that from the global positioning system, and, if the cost is not a consideration, can be used on the ground in remote sites for timekeeping and control.

Chi, A. R.↗
NASA NTRS1996

SeaWiFS Technical Report Series: SeaWiFS Calibration and Validation Quality Control Procedures - Volume 38

This document provides five brief reports that address several quality control procedures under the auspices of the Calibration and Validation Element (CVE) within the Sea-viewing Wide Field-of-view Sensor (SeaWiFS) Project. Chapter 1 describes analyses of the 32 sensor engineering telemetry streams. Anomalies in any of the values may impact sensor performance in direct or indirect ways. The analyses are primarily examinations of parameter time series combined with statistical methods such as auto- and cross-correlation functions. Chapter 2 describes how the various onboard (solar and lunar) and vicarious (in situ) calibration data will be analyzed to quantify sensor degradation, if present. The analyses also include methods for detecting the influence of charged particles on sensor performance such as might be expected in the South Atlantic Anomaly (SAA). Chapter 3 discusses the quality control of the ancillary environmental data that are routinely received from other agencies or projects which are used in the atmospheric correction algorithm (total ozone, surface wind velocity, and surface pressure; surface relative humidity is also obtained, but is not used in the initial operational algorithm). Chapter 4 explains the procedures for screening level-, level-2, and level-3 products. These quality control operations incorporate both automated and interactive procedures which check for file format errors (all levels), navigation offsets (level-1), mask and flag performance (level-2), and product anomalies (all levels). Finally, Chapter 5 discusses the match-up data set development for comparing SeaWiFS level-2 derived products with in situ observations, as well as the subsequent outlier analyses that will be used for evaluating error sources.

Hooker, Stanford B.↗
NASA NTRS1998

Automated System Checkout to Support Predictive Maintenance for the Reusable Launch Vehicle

The Propulsion Checkout and Control System (PCCS) is a predictive maintenance software system. The real-time checkout procedures and diagnostics are designed to detect components that need maintenance based on their condition, rather than using more conventional approaches such as scheduled or reliability centered maintenance. Predictive maintenance can reduce turn-around time and cost and increase safety as compared to conventional maintenance approaches. Real-time sensor validation, limit checking, statistical anomaly detection, and failure prediction based on simulation models are employed. Multi-signal models, useful for testability analysis during system design, are used during the operational phase to detect and isolate degraded or failed components. The TEAMS-RT real-time diagnostic engine was developed to utilize the multi-signal models by Qualtech Systems, Inc. Capability of predicting the maintenance condition was successfully demonstrated with a variety of data, from simulation to actual operation on the Integrated Propulsion Technology Demonstrator (IPTD) at Marshall Space Flight Center (MSFC). Playback of IPTD valve actuations for feature recognition updates identified an otherwise undetectable Main Propulsion System 12 inch prevalve degradation. The algorithms were loaded into the Propulsion Checkout and Control System for further development and are the first known application of predictive Integrated Vehicle Health Management to an operational cryogenic testbed. The software performed successfully in real-time, meeting the required performance goal of 1 second cycle time.

Patterson-Hine, Ann↗
NASA NTRS2004

Physical Validation of TRMM TMI and PR Monthly Rain Products Over Oklahoma

The Tropical Rainfall Measuring Mission (TRMM) provides monthly rainfall estimates using data collected by the TRMM satellite. These estimates cover a substantial fraction of the earth's surface. The physical validation of TRMM estimates involves corroborating the accuracy of spaceborne estimates of areal rainfall by inferring errors and biases from ground-based rain estimates. The TRMM error budget consists of two major sources of error: retrieval and sampling. Sampling errors are intrinsic to the process of estimating monthly rainfall and occur because the satellite extrapolates monthly rainfall from a small subset of measurements collected only during satellite overpasses. Retrieval errors, on the other hand, are related to the process of collecting measurements while the satellite is overhead. One of the big challenges confronting the TRMM validation effort is how to best estimate these two main components of the TRMM error budget, which are not easily decoupled. This four-year study computed bulk sampling and retrieval errors for the TRMM microwave imager (TMI) and the precipitation radar (PR) by applying a technique that sub-samples gauge data at TRMM overpass times. Gridded monthly rain estimates are then computed from the monthly bulk statistics of the collected samples, providing a sensor-dependent gauge rain estimate that is assumed to include a TRMM equivalent sampling error. The sub-sampled gauge rain estimates are then used in conjunction with the monthly satellite and gauge (without sub- sampling) estimates to decouple retrieval and sampling errors. The computed mean sampling errors for the TMI and PR were 5.9% and 7.796, respectively, in good agreement with theoretical predictions. The PR year-to-year retrieval biases exceeded corresponding TMI biases, but it was found that these differences were partially due to negative TMI biases during cold months and positive TMI biases during warm months.

Fisher, Brad L.↗
NASA NTRS1993

AVIRIS calibration and application in coastal oceanic environments

The Airborne Visible-Infrared Imaging Spectrometer (AVIRIS) is a test-bed for future spacecraft sensors such as the High-Resolution Imaging Spectrometer and the Moderate-Resolution Imaging Spectrometers planned for the Earth Observing System. To use this sensor for ocean applications, S/N was increased by spatial averaging of images. Post-flight recalibration was accomplished by measuring in situ the water-leaving radiance at flight time, modeling its transmission to the aircraft, and adding modeled atmospheric radiance to that value. The preflight calibration curve was then adjusted until aircraft and modeled total radiance values matched. Water-leaving radiance values from the recalibrated AVIRIS imagery were consistent with in situ data supporting the validity of the approach. Imagery of the absorption coefficient at 415 nm and backscattering coefficient at 671 nm were used to depict the dissolved and particulate constituents of the Tampa Bay plume during late, ebb-tidal conditions.

Carder, Kendall L.↗
NASA NTRS1992

AVIRIS calibration and application in coastal oceanic environments

The Airborne Visible-Infrared Imaging Spectrometer (AVIRIS) is a test-bed for future spacecraft sensors such as the High-Resolution Imaging Spectrometer and the Moderate-Resolution Imaging Spectrometers planned for the Earth Observing System. To use this sensor for ocean applications, S/N was increased by spatial averaging of images. Post-flight recalibration was accomplished using in situ the water-leaving radiance measured at flight time, modeling radiance transmission to the aircraft, and adding modeled atmospheric radiance to that value. The preflight calibration curve was then adjusted until aircraft and modeled total radiance values matched. Water-leaving radiance values from the recalibrated AVIRIS imagery were consistent with in situ data supporting the validity of the approach. Imagery of the absorption coefficient at 415 nm and backscattering coefficient at 671 nm were used to depict the dissolved and particulate constituents of an ebb-tidal esturance plume on the East coast of Florida.

Carder, Kendall L.↗
NASA NTRS2001

A Model-based Health Monitoring and Diagnostic System for the UH-60 Helicopter

Model-based reasoning techniques hold much promise in providing comprehensive monitoring and diagnostics capabilities for complex systems. We are exploring the use of one of these techniques, which utilizes multi-signal modeling and the TEAMS-RT real-time diagnostic engine, on the UH-60 Rotorcraft Aircrew Systems Concepts Airborne Laboratory (RASCAL) flight research aircraft. We focus on the engine and transmission systems, and acquire sensor data across the 1553 bus as well as by direct analog-to-digital conversion from sensors to the QHuMS (Qualtech health and usage monitoring system) computer. The QHuMS computer uses commercially available components and is rack-mounted in the RASCAL facility. A multi-signal model of the transmission and engine subsystems enables studies of system testability and analysis of the degree of fault isolation available with various instrumentation suites. The model and examples of these analyses will be described and the data architectures enumerated. Flight tests of this system will validate the data architecture and provide real-time flight profiles to be further analyzed in the laboratory.

Patterson-Hine, Ann↗
NASA NTRS2011

Magnetospheric Multiscale (MMS) Mission Attitude Ground System Design

This paper describes the attitude ground system (AGS) design to be used for support of the Magnetospheric MultiScale (MMS) mission. The AGS exists as one component of the mission operations control center. It has responsibility for validating the onboard attitude and accelerometer bias estimates, calibrating the attitude sensors and the spacecraft inertia tensor, and generating a definitive attitude history for use by the science teams. NASA's Goddard Space Flight Center (GSFC) in Greenbelt, Maryland is responsible for developing the MMS spacecraft, for the overall management of the MMS mission, and for mission operations. MMS is scheduled for launch in 2014 for a planned two-year mission. The MMS mission consists of four identical spacecraft flying in a tetrahedral formation in an eccentric Earth orbit. The relatively tight formation, ranging from 10 to 400 km, will provide coordinated observations giving insight into small-scale magnetic field reconnection processes. By varying the size of the tetrahedron and the orbital semi-major axis and eccentricity, and making use of the changing solar phase, this geometry allows for the study of both bow shock and magnetotail plasma physics, including acceleration, reconnection, and turbulence. The mission divides into two phases for science; these phases will have orbit dimensions of l.2xl2 Earth radii in the first phase and l.2x25 Earth radii in the second in order to study the dayside magnetopause and the nightside magnetotail, respectively. The orbital periods are roughly one day and three days for the two mission phases. Each of the four MMS spacecraft will be spin stabilized at 3 revolutions per minute (rpm), with the spin axis oriented near the ecliptic north pole but tipped approximately 2.5 deg towards the Sun line. The main body of each spacecraft will be an eight-sided platform with diameter of 3.4 m and height of 1.2 m. Several booms are attached to this central core: two axial booms of 14.9 m length, two radial magnetometer booms of 5 m length, and four radial -wire booms of 60 m length. Attitude and orbit control will use a set of axial and radial thrusters. A four-head star tracker and a slit-type digital Sun sensor (DSS) provide input for attitude determination. In addition, an accelerometer will be· used for closed-loop orbit maneuver control. The primary AGS product will be a daily definitive attitude history. Due to power limitations; the star tracker and accelerometer data will not be available at all times. However, tracker data from at least 10 percent of each orbit and continuous DSS data will be provided. An extended Kalman filter (EKF) will be used to estimate the three-axis attitude (i.e., spin axis orientation and spin phase) and rotation rate for all times when the tracker data is valid. For other times, the attitude is generated by assuming a constant angular momentum vector in the inertial frame. The DSS sun pulse will provide a timing signal to maintain an accurate spin phase. There will be times when the Sun is occulted and DSS data is not available. If this occurs at the start or end of a definitive attitude product, then the spin phase will be extrapolated using the mean rate determined by the EKF.

Sedlak, Joseph E.↗
NASA NTRS2010

Magnetospheric Multiscale (MMS) Mission Attitude Ground System Design

This paper describes the attitude ground system (AGS) design to be used for support of the Magnetospheric MultiScale (MMS) mission. The AGS exists as one component of the mission operations control center. It has responsibility for validating the onboard attitude and accelerometer bias estimates, calibrating the attitude sensors and the spacecraft inertia tensor, and generating a definitive attitude history for use by the science teams. NASA's Goddard Space Flight Center (GSFC) in Greenbelt, Maryland is responsible for developing the MMS spacecraft, for the overall management of the MMS mission, and for mission operations. MMS is scheduled for launch in 2014 for a planned two-year mission. The MMS mission consists of four identical spacecraft flying in a tetrahedral formation in an eccentric Earth orbit. The relatively tight formation, ranging from 10 to 400 km, will provide coordinated observations giving insight into small-scale magnetic field reconnection processes. By varying the size of the tetrahedron and the orbital semi-major axis and eccentricity, and making use of the changing solar phase, this geometry allows for the study of both bow shock and magnetotail plasma physics, including acceleration, reconnection, and turbulence. The mission divides into two phases for science; these phases will have orbit dimensions of 1.2 x 12 Earth radii in the first phase and 1.2x25 Earth radii in the second in order to study the dayside magnetopause and the nightside magnetotail, respectively. The orbital periods are roughly one day and three days for the two mission phases. Each of the four MMS spacecraft will be spin stabilized at 3 revolutions per minute (rpm), with the spin axis oriented near the ecliptic north pole but tipped approximately 2.5 deg towards the Sun line. The main body of each spacecraft will be an eight-sided platform with diameter of 3.4 m and height of 1.2 m. Several booms are attached to this central core: two axial booms of 14.9 m length, two radial magnetometer booms of 5 m length, and four radial wire booms of 60 m length. Attitude and orbit control will use a set of axial and radial thrusters. A four-head star tracker and a slit-type digital Sun sensor (DSS) provide input for attitude determination. In addition, an accelerometer will be used for closed-loop orbit maneuver control. The primary AGS product will be a daily definitive attitude history. Due to power limitations, the star tracker and accelerometer data will not be available at all times. However, tracker data from at least 10 percent of each orbit and continuous DSS data will be provided. An extended Kalman filter (EKF) will be used to estimate the three-axis attitude (i.e., spin axis orientation and spin phase) and rotation rate for all times when the tracker data is valid. For other times, the attitude is generated by assuming a constant angular momentum vector in the inertial frame. The DSS sun pulse will provide a timing signal to maintain an accurate spin phase. There will be times when the Sun is occulted and DSS data is not available. If this occurs at the start or end of a definitive attitude product, then the spin phase will be extrapolated using the mean rate determined by the EKF.

Sedlak, Joseph E.↗
NASA NTRS1977

Analytic investigation of the AEM-A/HCMM attitude control system performance

The Heat Capacity Mapping Mission (HCMM), scheduled for launch in 1978, will be three-axis stabilized relative to the earth in a 600-kilometer altitude, polar orbit. The autonomous attitude control system consists of three torquing coils and a momentum wheel driven in response to error signals computed from data received from an infrared horizon sensor and a magnetometer. This paper presents a simple model of the attitude dynamics and derives the equations that determine the stability of the system during both attitude acquisition (acquisition-mode) and mission operations (mission-mode). Modifications to the proposed mission-mode control laws which speed the system's response to transient attitude errors and reduce the steady-state attitude errors are suggested. Numerical simulations are performed to validate the results obtained with the simple model.

Lerner, G. M.↗
NASA NTRS1992

Postflight aerothermodynamic analysis of Pegasus(tm) using computational fluid dynamic techniques

The objective was to validate the computational capability of the NASA Ames Navier-Stokes code, F3D, for flows at high Mach numbers using comparison flight test data from the Pegasus (tm) air launched, winged space booster. Comparisons were made with temperature and heat fluxes estimated from measurements on the wing surfaces and wing-fuselage fairings. Tests were conducted for solution convergence, sensitivity to grid density, and effects of distributing grid points to provide high density near temperature and heat flux sensors. The measured temperatures were from sensors embedded in the ablating thermal protection system. Surface heat fluxes were from plugs fabricated of highly insulative, nonablating material, and mounted level with the surface of the surrounding ablative material. As a preflight design tool, the F3D code produces accurate predictions of heat transfer and other aerodynamic properties, and it can provide detailed data for assessment of boundary layer separation, shock waves, and vortex formation. As a postflight analysis tool, the code provides a way to clarify and interpret the measured results.

Kuhn, Gary D.↗
NASA NTRS1993

Kalman filter estimation of attitude and gyro bias with the QUEST observation model

The loss function for the Wahba attitude estimation problem employs unit vector observations with scalar weights. It is usually associated with the QUEST observation model, where the actual sensor noise is assumed the same for all components of the observed vector, regardless of the position in the field of view. The QUEST model has the great advantage of being sensor-independent except for the scalar parameter that characterizes the sensor errors. Although efficient algorithms for solving the Wahba problem exist, extending these algorithms to estimate gyro biases or sensor alignments has had mixed success. However, it is straightforward to estimate bias and alignment parameters with a Kalman filter. This paper investigates the use of an extended Kalman filter for the attitude and gyro bias that incorporates the QUEST observation model, to be referred to as the Unit Vector Filter (UVF). The UVF results are compared with those from a more conventional filter, the Real-Time Sequential Filter (RTSF), for which the residual is the two-dimensional projection of the unit vector onto the plane perpendicular to the sensor boresight. The RTSF is similar in design to that used by Multimission Modular Spacecraft for onboard attitude determination. An apparent obstacle to the use of unit vectors as measurements is their singular noise covariance matrix. Shuster has shown that this problem should not affect filter performance. The UVF and RTSF are tested using actual flight data from the Extreme Ultraviolet Explorer (EUVE). It is found that these filters generate nearly identical attitude and gyro bias estimates, thus validating the use of unit vectors and the QUEST noise model.

Sedlak, J.↗
NASA NTRS1989

SSME fault monitoring and diagnosis expert system

An expert system, called LEADER, has been designed and implemented for automatic learning, detection, identification, verification, and correction of anomalous propulsion system operations in real time. LEADER employs a set of sensors to monitor engine component performance and to detect, identify, and validate abnormalities with respect to varying engine dynamics and behavior. Two diagnostic approaches are adopted in the architecture of LEADER. In the first approach fault diagnosis is performed through learning and identifying engine behavior patterns. LEADER, utilizing this approach, generates few hypotheses about the possible abnormalities. These hypotheses are then validated based on the SSME design and functional knowledge. The second approach directs the processing of engine sensory data and performs reasoning based on the SSME design, functional knowledge, and the deep-level knowledge, i.e., the first principles (physics and mechanics) of SSME subsystems and components. This paper describes LEADER's architecture which integrates a design based reasoning approach with neural network-based fault pattern matching techniques. The fault diagnosis results obtained through the analyses of SSME ground test data are presented and discussed.

Ali, Moonis↗
NASA NTRS2020

Field Evaluation of Portable Soil Water Content Sensors in a Sandy Loam

Ground observations are critical in the validation of soil water content (SWC) estimates from both satellites and land surface models. Portable SWC sensors provide useful information to determine the amount of SWC in the topsoil layer for various applications; however, these probes are not accurate without site‐specific correction. In the present study, we examined and compared six different types of portable electromagnetic (EM) SWC sensors, including multiple sensors made by the same manufacturers, for a total of 16 EM‐based SWC probes equipped with portable data loggers. All SWC probes met the target accuracy after onsite correction—the RMSD was <0.025 m(exp 3) m(exp -3). Using the two‐sample t-tests, we observed that SWC data obtained from similar electrode lengths and from different manufacturers showed similar distributions over time with the same mean. Furthermore, using the maximize R method to combine SWC data from two different types of sensors increased the accuracy of the results. When datasets from two different types of sensors were combined, the Pearson's correlation coefficient (R value) and RMSD values were improved. The average R value improved from .930 to .945, and the RMSD decreased from 0.036 m(exp 3) m(exp -3) to 0.018 m(exp 3) m(exp −3). These results indicate that, along with site‐specific correction, synergetic use of multiple manufacturers’ EM‐based SWC probes can improve the R value and reduce systematic bias.

soil moisture data↗
NASA NTRS2022

Cloud Mask Intercomparison eXercise (CMIX): An evaluation of cloud masking algorithms for Landsat 8 and Sentinel-2

Cloud cover is a major limiting factor in exploiting time-series data acquired by optical spaceborne remote sensing sensors. Multiple methods have been developed to address the problem of cloud detection in satellite imagery and a number of cloud masking algorithms have been developed for optical sensors but very few studies have carried out quantitative intercomparison of state-of-the-art methods in this domain. This paper summarizes results of the first Cloud Masking Intercomparison eXercise (CMIX) conducted within the Committee Earth Observation Satellites (CEOS) Working Group on Calibration & Validation (WGCV). CEOS is the forum for space agency coordination and cooperation on Earth observations, with activities organized under working groups. CMIX, as one such activity, is an international collaborative effort aimed at intercomparing cloud detection algorithms for moderate-spatial resolution (10–30 m) spaceborne optical sensors. The focus of CMIX is on open and free imagery acquired by the Landsat 8 (NASA/USGS) and Sentinel-2 (ESA) missions. Ten algorithms developed by nine teams from fourteen different organizations representing universities, research centers and industry, as well as space agencies (CNES, ESA, DLR, and NASA), are evaluated within the CMIX. Those algorithms vary in their approach and concepts utilized which were based on various spectral properties, spatial and temporal features, as well as machine learning methods. Algorithm outputs are evaluated against existing reference cloud mask datasets. Those datasets vary in sampling methods, geographical distribution, sample unit (points, polygons, full image labels), and generation approaches (experts, machine learning, sky images). Overall, the performance of algorithms varied depending on the reference dataset, which can be attributed to differences in how the reference datasets were produced. The algorithms were in good agreement for thick cloud detection, which were opaque and had lower uncertainties in their identification, in contrast to thin/semi-transparent clouds detection. Not only did CMIX allow identification of strengths and weaknesses of existing algorithms and potential areas of improvements, but also the problems associated with the existing reference datasets. The paper concludes with recommendations on generating new reference datasets, metrics, and an analysis framework to be further exploited and additional input datasets to be considered by future CMIX activities.

Sergii Skakun↗