Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Systematic error”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,657 records · Page 92

SU-E-J-103: Setup Errors Analysis by Cone-Beam CT (CBCT)-Based Imaged-Guided Intensity Modulated Radiotherapy for Esophageal Cancer

Purpose: To quantify setup errors by pretreatment kilovolt cone-beam computed tomography(KV-CBCT) scans for middle or distal esophageal carcinoma patients. Methods: Fifty-two consecutive middle or distal esophageal carcinoma patients who underwent IMRT were included this study. A planning CT scan using a big-bore CT simulator was performed in the treatment position and was used as the reference scan for image registration with CBCT. CBCT scans(On-Board Imaging v1. 5 system, Varian Medical Systems) were acquired daily during the first treatment week. A total of 260 CBCT scans was assessed with a registration clip box defined around the PTV-thorax in the reference scan based on(nine CBCTs per patient) bony anatomy using Offline Review software v10.0(Varian Medical Systems). The anterior-posterior(AP), left-right(LR), superiorinferior( SI) corrections were recorded. The systematic and random errors were calculated. The CTV-to-PTV margins in each CBCT frequency was based on the Van Herk formula (2.5Σ+0.7σ). Results: The SD of systematic error (Σ) was 2.0mm, 2.3mm, 3.8mm in the AP, LR and SI directions, respectively. The average random error (σ) was 1.6mm, 2.4mm, 4.1mm in the AP, LR and SI directions, respectively. The CTV-to-PTV safety margin was 6.1mm, 7.5mm, 12.3mm in the AP, LR and SI directions based on van Herk formula. Conclusion: Our data recommend the use of 6 mm, 8mm, and 12 mm for esophageal carcinoma patient setup in AP, LR, SI directions, respectively.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

Presearch Data Conditioning in the Kepler Science Operations Center Pipeline

We describe the Presearch Data Conditioning (PDC) software component and its context in the Kepler Science Operations Center (SOC) pipeline. The primary tasks of this component are to correct systematic and other errors, remove excess flux due to aperture crowding, and condition the raw flux light curves for over 160,000 long cadence (~thirty minute) and 512 short cadence (~one minute) targets across the focal plane array. Long cadence corrected flux light curves are subjected to a transiting planet search in a subsequent pipeline module. We discuss the science algorithms for long and short cadence PDC: identification and correction of unexplained (i.e., unrelated to known anomalies) discontinuities; systematic error correction; and excess flux removal. We discuss the propagation of uncertainties from raw to corrected flux. Finally, we present examples of raw and corrected flux time series for flight data to illustrate PDC performance. Corrected flux light curves produced by PDC are exported to the Multi-mission Archive at Space Telescope [Science Institute] (MAST) and will be made available to the general public in accordance with the NASA/Kepler data release policy.

Twicken, Joseph D.↗

Type Ia Supernova Growth-rate Measurement with LSST Simulations: Intrinsic Scatter Systematics

Measurement of the growth rate of structures (fσ 8 ) with Type Ia supernovae (SNe Ia) will improve our understanding of the nature of dark energy and enable tests of general relativity. In this paper, we generate simulations of the 10 yr SN Ia data set of the Rubin-LSST survey, including a correlated velocity field from an N-body simulation and realistic models of SNe Ia properties and their correlations with host-galaxy properties. We find, similar to SN Ia analyses that constrain the dark energy equation-of-state parameters w 0 w a , that constraints on fσ 8 can be biased depending on the intrinsic scatter of SNe Ia. While for the majority of intrinsic scatter models we recover fσ 8 with a precision of ∼13%–14%, for the most realistic dust-based model, we find that the presence of non-Gaussianities in Hubble diagram residuals leads to a bias on fσ 8 of ∼ −20%. When trying to correct for the dust-based intrinsic scatter, we find that the propagation of the uncertainty on the model parameters does not significantly increase the error on fσ 8 . We also find that while the main component of the error budget of fσ 8 is the statistical uncertainty (>75% of the total error budget), the systematic error budget is dominated by the uncertainty on the damping parameter, σ u , that gives an empirical description of the effect of redshift space distortions on the velocity power spectrum. Our results motivate a search for new methods to correct for the non-Gaussian distribution of the Hubble diagram residuals, as well as an improved modeling of the damping parameter.

Carreres, Bastien [Duke Univ., Durham, NC (United ↗

A Practical Methodology for Quantifying Random and Systematic Components of Unexplained Variance in a Wind Tunnel

This paper documents a check standard wind tunnel test conducted in the Langley 0.3-Meter Transonic Cryogenic Tunnel (0.3M TCT) that was designed and analyzed using the Modern Design of Experiments (MDOE). The test designed to partition the unexplained variance of typical wind tunnel data samples into two constituent components, one attributable to ordinary random error, and one attributable to systematic error induced by covariate effects. Covariate effects in wind tunnel testing are discussed, with examples. The impact of systematic (non-random) unexplained variance on the statistical independence of sequential measurements is reviewed. The corresponding correlation among experimental errors is discussed, as is the impact of such correlation on experimental results generally. The specific experiment documented herein was organized as a formal test for the presence of unexplained variance in representative samples of wind tunnel data, in order to quantify the frequency with which such systematic error was detected, and its magnitude relative to ordinary random error. Levels of systematic and random error reported here are representative of those quantified in other facilities, as cited in the references.

Deloach, Richard↗

Errors in radial velocity variance from Doppler wind lidar

A high-fidelity lidar turbulence measurement technique relies on accurate estimates of radial velocity variance that are subject to both systematic and random errors determined by the autocorrelation function of radial velocity, the sampling rate, and the sampling duration. Our paper quantifies the effect of the volumetric averaging in lidar radial velocity measurements on the autocorrelation function and the dependence of the systematic and random errors on the sampling duration, using both statistically simulated and observed data. For current-generation scanning lidars and sampling durations of about 30 min and longer, during which the stationarity assumption is valid for atmospheric flows, the systematic error is negligible but the random error exceeds about 10%.

54 ENVIRONMENTAL SCIENCES↗

Shock compression response of highly reactive Ni + Al multilayered thin foils

The shock-compression response of Ni + Al multilayered thin foils is investigated using laser-accelerated thin-foil plate-impact experiments over the pressure range of 2 to 11 GPa. The foils contain alternating Ni and Al layers (parallel but not flat) of nominally 50 nm bilayer spacing. The goal is to determine the equation of state and shock-induced reactivity of these highly reactive fully dense thin-foil materials. The laser-accelerated thin-foil impact set-up involved combined use of photon-doppler-velocimetry to monitor the acceleration and impact velocity of an aluminum flyer, and VISAR interferometry was used to monitor the back free-surface velocity of the impacted Ni + Al multilayered target. The shock-compression response of the Ni + Al target foils was determined using experimentally measured parameters and impedance matching approach, with error bars identified considering systematic and experimental errors. Meso-scale CTH shock simulations were performed using real imported microstructures of the cross-sections of the multilayered Ni + Al foils to compute the Hugoniot response (assuming no reaction) for correlation with their experimentally determined equation of state. It was observed that at particle velocities below ∼150 m/s, the experimentally determined equation of state trend matches the CTH-predicted inert response and is consistent with the observed unreacted state of the recovered Ni + Al target foils from this velocity regime. At higher particle velocities, the experimentally determined equation of state deviates from the CTH-predicted inert response. A complete and self-sustained reaction is also seen in targets recovered from experiments performed at these higher particle velocities. The deviation in the measured equation of state, to higher shock speeds and expanded volumes, combined with the observation of complete reaction in the recovered multilayered foils, confirmed via microstructure characterization, is indicative of the occurrence of shock-induced chemical reaction occurring in the time-scale of the high-pressure state. TEM characterization of recovered shock-compressed (unreacted) Ni + Al multilayered foils exhibits distinct features of constituent mixing revealing jetted layers and inter-mixed regions. These features were primarily observed in the proximity of the undulations present in the alternating layers of the Ni + Al starting foils, suggesting the important role of such instabilities in promoting shock-induced intermetallic-forming reactions in the fully dense highly exothermic multilayered thin foils.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Scalable measurement error mitigation via iterative bayesian unfolding

Measurement errors are a significant obstacle to achieving scalable quantum computation. To counteract systematic readout errors, researchers have developed postprocessing techniques known as measurement error mitigation methods. However, these methods face a tradeoff between scalability and returning nonnegative probabilities. In this paper, we present a solution to overcome this challenge. Our approach focuses on iterative Bayesian unfolding, a standard mitigation technique used in high-energy physics experiments, and implements it in a scalable way. We demonstrate our method on experimental Greenberger-Horne-Zeilinger state preparation on up to 127 qubits and on the Bernstein-Vazirani algorithm on up to 26 qubits. Compared to state-of-the-art methods (such as M3), our implementation guarantees valid probability distributions, returns comparable or better-mitigated results, and does so without a noticeable time and memory overhead. Published by the American Physical Society 2024

Pokharel, Bibek (ORCID:0000000342841353)↗

Analysis of Integrated Vapor Transport Biases

Abstract High-fidelity analyses and forecasts of integrated vapor transport (VT) are central to the study of Earth’s hydrological cycle as well as high-impact phenomena such as monsoons and atmospheric rivers. The impact of the in-line analysis correction-based additive inflation (ACAI) on IVT biases and forecast errors is examined within the Navy Earth System Prediction Capability (Navy ESPC) global coupled system. The ACAI technique uses atmospheric analysis corrections from the data assimilation system to approximate model bias and as a representation of stochastic model error to simultaneously reduce systematic and random errors and improve ensemble performance. ACAI reduces the global average magnitude of the 7- and 14-day IVT bias by 16%–17% during Northern Hemisphere summer, reaching 70% reductions in some tropical regions. The global average IVT bias reduction is similar to the bias reduction for low-level wind speed bias and considerably smaller than the bias reduction in total precipitable water. The localized regions where ACAI increases IVT bias occur where the control IVT biases change sign and structure with increasing forecast lead time, such as the South Asian monsoon region. Substituting analyzed wind or moisture fields for the forecast fields when calculating the forecast IVT confirms that, on average, wind errors dominate the IVT error calculation in the tropics, although wind and moisture error contributions are comparable in the extratropics. The existence of regions where using either analyzed winds or analyzed moisture increases IVT bias or mean absolute error reveals areas with compensating errors.

Reynolds, Carolyn A.↗

The AAMP5/AAMP-FV project

This presentation describes a project, formal verification of the microcode in the AAMP5 microprocessor, conducted to explore how formal techniques for specification and verification could be introduced into an industrial process. Sponsored by the Systems Validation Branch of NASA Langley and by Collins Commercial Avionics, a division of Rockwell International, it was conducted by Collins and the SRI International Computer Science Laboratory. The project consisted of specifying in the PVS language developed by SRI a portion of a Rockwell proprietary microprocessor, the AAMP5, at both the instruction set and register-transfer levels and using the PVS theorem prover to prove the microcode correct for a representative subset of instructions. While this presentation includes a brief technical overview, its emphasis is on the lessons learned in using PVS for an example of this size and the implications for using formal methods in an industrial setting. The central result of this project was to demonstrate the feasibility of formally specifying a commercial microprocessor and the use of mechanical proofs of correctness to verify microcode. This is particularly significant since the AAMP5 was not designed for formal verification, but to provide a more than three fold performance improvement, by pipelining instruction execution, while remaining object code compatible with the earlier AAMP2. As a consequence, the AAMP5 is one of the most complex microprocessors to which formal methods have been applied. Another key result was the discovery of both actual and seeded errors. Two actual microcode errors were discovered and corrected during development of the formal specification, illustrating the value of simply creating a precise specification. Two seeded errors were systematically uncovered while doing correctness proofs. One of these was an actual error that had been discovered after first fabrication but left in the microcode provided to SRI. The other error was designed to be unlikely to be detected by walkthroughs, testing, or simulation. Several other results emerged during the project, including the ease with which practicing engineers became comfortable with PVS, the need for libraries of general purpose theories, the usefulness of formal specification in revealing errors, the natural fit between formal specification and inspections, the difficulty of selecting the best style of specification for a new problem domain, the high level of assurance provided by proofs of correctness, and the need to engineer proof strategies for reuse.

Miller, Steven P.↗

Improving Multiday Solar Wind Speed Forecasts

We analyze the residual errors for the Wang-Sheeley-Arge (WSA) solar wind speed forecasts as a function of the photospheric magnetic field expansion factor (fp) and the minimum separation angle (d) in the photosphere between the footpoints of open field lines and the nearest coronal hole boundary. We find the map of residual speed errors are systematic when examined as a function of fp and d. We use these residual error maps to apply corrections to the model speeds. We test this correction approach using 3-day lead time speed forecasts for an entire year of observations and model results. Our methods can readily be applied to develop corrections for the remaining WSA forecast lead times which range from 1 to 7 days in 1-day increments. Since the solar wind density, temperature, and the interplanetary magnetic field strength all correlate well with the solar wind speed, the improved accuracy of solar wind speed forecasts enables the production of multiday forecasts of the solar wind density, temperature, pressure, and interplanetary field strength, and geophysical indices. These additional parameters would expand the usefulness of Air Force Data Assimilative Photospheric Flux Transport-WSA forecasts for space weather clients.

H A Elliot↗

A Framework for Estimating Global River Discharge From the Surface Water and Ocean Topography Satellite Mission

The Surface Water and Ocean Topography (SWOT) mission will vastly expand measurements of global rivers, providing critical new data sets for both gaged and ungaged basins. SWOT discharge products (available approximately 1 year after launch) will provide discharge for all river that reaches wider than 100 m. In this paper, we describe how SWOT discharge produced and archived by the US and French space agencies will be computed from measurements of river water surface elevation, width, and slope and ancillary data, along with expected discharge accuracy. We present for the first time a complete estimate of the SWOT discharge uncertainty budget, with separate terms for random (standard error) and systematic (bias) uncertainty components in river discharge time series. We expect that discharge uncertainty will be less than 30% for two-thirds of global reaches and will be dominated by bias. Separate river discharge estimates will combine both SWOT and in situ data; these “gage-constrained” discharge estimates can be expected to have lower systematic uncertainty. Temporal variations in river discharge time series will be dominated by random error and are expected to be estimated within 15% for nearly all reaches, allowing accurate inference of event flow dynamics globally, including in ungaged basins. We believe this level of accuracy lays the groundwork for SWOT to enable breakthroughs in global hydrologic science.

Michael Durand↗

Three Decades of Precision Orbit Determination Progress, Achievements, Future Challenges and its Vital Contribution to Oceanography and Climate Research

Although satellite altimetry has been around for thirty years, the last fifteen beginning with the launch of TOPEX/Poseidon (TP) have yielded an abundance of significant results including: monitoring of ENS0 events, detection of internal tides, determination of accurate global tides, unambiguous delineation of Rossby waves and their propagation characteristics, accurate determination of geostrophic currents, and a multi-decadal time series of mean sea level trend and dynamic ocean topography variability. While the high level of accuracy being achieved is a result of both instrument maturity and the quality of models and correction algorithms applied to the data, improving the quality of the Climate Data Records produced from altimetry is highly dependent on concurrent progress being made in fields such as orbit determination. The precision orbits form the reference frame from which the radar altimeter observations are made. Therefore, the accuracy of the altimetric mapping is limited to a great extent by the accuracy to which a satellite orbit can be computed. The TP mission represents the first time that the radial component of an altimeter orbit was routinely computed with an accuracy of 2-cm. Recently it has been demonstrated that it is possible to compute the radial component of Jason orbits with an accuracy of better than 1-cm. Additionally, still further improvements in TP orbits are being achieved with new techniques and algorithms largely developed from combined Jason and TP data analysis. While these recent POD achievements are impressive, the new accuracies are now revealing subtle systematic orbit error that manifest as both intra and inter annual ocean topography errors. Additionally the construction of inter-decadal time series of climate data records requires the removal of systematic differences across multiple missions. Current and future efforts must focus on the understanding and reduction of these errors in order to generate a complete and consistent time series of improved orbits across multiple missions and decades required for the most stringent climate-related research. This presentation discusses the POD progress and achievements made over nearly three decades, and presents the future challenges, goals and their impact on altimetric derived ocean sciences.

Luthcke, Scott↗

Improved Understanding of Coupled Water and Carbon Cycle Processes through Machine Learning Approaches

Focal Area(s): This white paper addresses how explainable machine learning (ML) algorithms can improve insights gained from complex data (Focal Area 3). We will also address how ML approaches related to sensor compression and low energy AI hardware can be used for efficient data acquisition (Focal Area 1). Science Challenge: We will focus on the coupled water cycle and carbon cycle processes in terrestrial ecosystems and terrestrial-aquatic interfaces. Thus, our approaches will be centered around several data-model integration challenges indicated in EESD strategic plans for two science focus areas: Terrestrial Ecosystem Science and Hydrobiogeochemistry. We can use AI to correct systematic model errors due to biases in either data or model structure/processes. Error patterns in model predictions usually vary by region, model, season etc. But there are systematic patterns in them. e.g., soil moisture tends to be overestimated in the arid western continental United States underestimated in wetter eastern USA; some land surface models tend to underestimate moisture in wet seasons and overestimate in dry seasons. Moreover, the consequences of extreme events (e.g., drought, extreme flooding, storm surges associated with tropical storms, hurricanes) on carbon cycle processes are not well represented in ecosystem and Earth system models. Redox-sensitive processes (e.g., rapid oxidation/reduction of iron and other redox-sensitive elements in soil microsites subjected to fluctuated hydrology) in terrestrial-aquatic interfaces further challenge model predictions of hot-spots (and hot-moments) due to poor understanding of underlying mechanisms. Thus, AI/ML approaches can be used to learn patterns in the data and model errors and use them to build model equations and correct process-based model errors.

54 ENVIRONMENTAL SCIENCES↗

SU-E-J-177: A Computational Approach for Determination of Anisotropic PTV Margins Based On Statistical Shape Analysis for Prostate Cancer Radiotherapy

Purpose: Our aim of this study was to propose a computational approach for determination of anisotropic planning target volume (PTV) margins based on statistical shape analysis with taking into account time variations of clinical target volume (CTV) shapes for the prostate cancer radiation treatment planning (RTP). Methods: Systematic and random setup errors were measured using orthogonal projection and cone beam computed tomography (CBCT) images for data of 20 patients, who underwent the intensity modulated radiation therapy for prostate cancer. The low-risk, intermediate-risk, and high-risk CTVs were defined as only a prostate, a prostate plus proximal 1-cm seminal vesicles, and a prostate plus proximal 2-cm seminal vesicles, respectively. All CTV regions were registered with a reference CTV region with a median volume to remove the effect of the setup errors, and converted to a point distribution models. The systematic and random errors for translations of CTV regions were automatically evaluated by analyzing the movements of centroids of CTV regions. The random and systematic errors for shape variations of CTV regions were obtained from covariance matrices based on point distributions for the CTV contours on CBCT images of 72 fractions of 10 patients. Anisotropic PTV margins for 6 directions (right, left, anterior, posterior, superior and inferior) were derived by using Yoda’s PTV margin model. Results: PTV margins with and without shape variations were 5.75 to 8.03 mm and 5.23 to 7.67 mm for low-risk group, 5.87 to 8.33 mm and 5.23 to 7.67 mm for intermediate-risk group, and 5.88 to 8.25 mm and 5.29 to 7.82 mm for highrisk group, respectively. Conclusion: The proposed computational approach could be feasible for determination of the anisotropic PTV margins with taking into account CTV shape variations for the RTP.

60 APPLIED LIFE SCIENCES↗

TH-CD-207A-11: Sensitivity Analysis of Action Limits for Real-Time EPID-Based Delivery Verification System Using Artificial Clinical Relevant Error Simulations

Purpose: A real-time patient treatment delivery verification system using EPID (Watchdog) has been developed as an advanced patient safety tool. In a pilot study data was acquired for 119 prostate and head and neck (HN) IMRT patient deliveries to generate body-site specific action limits using statistical process control. The purpose of this study is to determine the sensitivity of Watchdog to detect clinically significant errors during treatment delivery. Methods: Watchdog utilizes a physics-based model to generate a series of predicted transit cine EPID images as a reference data set, and compares these in real-time to measured transit cine-EPID images acquired during treatment using chi comparison (4%, 4mm criteria) after the initial 2s of treatment to allow for dose ramp-up. Four study cases were used; dosimetric (monitor unit) errors in prostate (7 fields) and HN (9 fields) IMRT treatments of (5%, 7%, 10%) and positioning (systematic displacement) errors in the same treatments of (5mm, 7mm, 10mm). These errors were introduced by modifying the patient CT scan and re-calculating the predicted EPID data set. The error embedded predicted EPID data sets were compared to the measured EPID data acquired during patient treatment. The treatment delivery percentage (measured from 2s) where Watchdog detected the error was determined. Results: Watchdog detected all simulated errors for all fields during delivery. The dosimetric errors were detected at average treatment delivery percentage of (4%, 0%, 0%) and (7%, 0%, 0%) for prostate and HN respectively. For patient positional errors, the average treatment delivery percentage was (52%, 43%, 25%) and (39%, 16%, 6%). Conclusion: These results suggest that Watchdog can detect significant dosimetric and positioning errors in prostate and HN IMRT treatments in real-time allowing for treatment interruption. Displacements of the patient require longer to detect however incorrect body site or very large geographic misses will be detected rapidly.

60 APPLIED LIFE SCIENCES↗

Analytic Perturbation Method for Estimating Ground Flash Fraction from Satellite Lightning Observations

An analytic perturbation method is introduced for estimating the lightning ground flash fraction in a set of N lightning flashes observed by a satellite lightning mapper. The value of N is large, typically in the thousands, and the observations consist of the maximum optical group area produced by each flash. The method is tested using simulated observations that are based on Optical Transient Detector (OTD) and Lightning Imaging Sensor (LIS) data. National Lightning Detection NetworkTM (NLDN) data is used to determine the flash‐type (ground or cloud) of the satellite-observed flashes, and provides the ground flash fraction truth for the simulation runs. It is found that the mean ground flash fraction retrieval errors are below 0.04 across the full range 0‐1 under certain simulation conditions. In general, it is demonstrated that the retrieval errors depend on many factors (i.e., the number, N, of satellite observations, the magnitude of random and systematic measurement errors, and the number of samples used to form certain climate distributions employed in the model).

Koshak, William↗

Implication of spot position error on plan quality and patient safety in pencil-beam-scanning proton therapy

Purpose: To quantitatively and systematically assess dosimetric effects induced by spot positioning error as a function of spot spacing (SS) on intensity-modulated proton therapy (IMPT) plan quality and to facilitate evaluation of safety tolerance limits on spot position. Methods: Spot position errors (PE) ranging from 1 to 2 mm were simulated. Simple plans were created on a water phantom, and IMPT plans were calculated on two pediatric patients with a brain tumor of 28 and 3 cc, respectively, using a commercial planning system. For the phantom, a uniform dose was delivered to targets located at different depths from 10 to 20 cm with various field sizes from 2{sup 2} to 15{sup 2} cm{sup 2}. Two nominal spot sizes, 4.0 and 6.6 mm of 1 σ in water at isocenter, were used for treatment planning. The SS ranged from 0.5 σ to 1.5 σ, which is 2–6 mm for the small spot size and 3.3–9.9 mm for the large spot size. Various perturbation scenarios of a single spot error and systematic and random multiple spot errors were studied. To quantify the dosimetric effects, percent dose error (PDE) depth profiles and the value of percent dose error at the maximum dose difference (PDE [ΔDmax]) were used for evaluation. Results: A pair of hot and cold spots was created per spot shift. PDE[ΔDmax] is found to be a complex function of PE, SS, spot size, depth, and global spot distribution that can be well defined in simple models. For volumetric targets, the PDE [ΔDmax] is not noticeably affected by the change of field size or target volume within the studied ranges. In general, reducing SS decreased the dose error. For the facility studied, given a single spot error with a PE of 1.2 mm and for both spot sizes, a SS of 1σ resulted in a 2% maximum dose error; a SS larger than 1.25 σ substantially increased the dose error and its sensitivity to PE. A similar trend was observed in multiple spot errors (both systematic and random errors). Systematic PE can lead to noticeable hot spots along the field edges, which may be near critical structures. However, random PE showed minimal dose error. Conclusions: Dose error dependence for PE was quantitatively and systematically characterized and an analytic tool was built to simulate systematic and random errors for patient-specific IMPT. This information facilitates the determination of facility specific spot position error thresholds.

60 APPLIED LIFE SCIENCES↗

Consistent evaluation of GOSAT, SCIAMACHY, carbontracker, and MACC through comparisons to TCCON

Consistent validation of satellite CO 2 estimates is a prerequisite for using multiple satellite CO 2 measurements for joint flux inversion, and for establishing an accurate long-term atmospheric CO 2 data record. We focus on validating model and satellite observation attributes that impact flux estimates and CO 2 assimilation, including accurate error estimates, correlated and random errors, overall biases, biases by season and latitude, the impact of coincidence criteria, validation of seasonal cycle phase and amplitude, yearly growth, and daily variability. We evaluate dry air mole fraction (X CO 2 ) for GOSAT (ACOS b3.5) and SCIAMACHY (BESD v2.00.08) as well as the CarbonTracker (CT2013b) simulated CO 2 mole fraction fields and the MACC CO 2 inversion system (v13.1) and compare these to TCCON observations (GGG2014). We find standard deviations of 0.9 ppm, 0.9, 1.7, and 2.1 ppm versus TCCON for CT2013b, MACC, GOSAT, and SCIAMACHY, respectively, with the single target errors 1.9 and 0.9 times the predicted errors for GOSAT and SCIAMACHY, respectively. When satellite data are averaged and interpreted according to error 2 = a 2 + b 2 / n (where n are the number of observations averaged, a are the systematic (correlated) errors, and b are the random (uncorrelated) errors), we find that the correlated error term a = 0.6 ppm and the uncorrelated error term b = 1.7 ppm for GOSAT and a = 1.0 ppm, b = 1.4 ppm for SCIAMACHY regional averages. Biases at individual stations have year-to-year variability of ~ 0.3 ppm, with biases larger than the TCCON predicted bias uncertainty of 0.4 ppm at many stations. Using fitting software, we find that GOSAT underpredicts the seasonal cycle amplitude in the Northern Hemisphere (NH) between 46–53° N. In the Southern Hemisphere (SH), CT2013b underestimates the seasonal cycle amplitude. Biases are calculated for 3-month intervals and indicate the months that contribute to the observed amplitude differences. The seasonal cycle phase indicates whether a dataset or model lags another dataset in time. We calculate this at a subset of stations where there is adequate satellite data, and find that the GOSAT retrieved phase improves substantially over the prior and the SCIAMACHY retrieved phase improves substantially for 2 of 7 sites. The models reproduce the measured seasonal cycle phase well except for at Lauder125 (CT2013b), Darwin (MACC), and Izana (+ 10 days, CT2013b), as for Bremen and Four Corners, which are highly influenced by local effects. We compare the variability within one day between TCCON and models in JJA; there is correlation between 0.2 and 0.8 in the NH, with models showing 10–100 % the variability of TCCON at different stations (except Bremen and Four Corners which have no variability compared to TCCON) and CT2013b showing more variability than MACC. This paper highlights findings that provide inputs to estimate flux errors in model assimilations, and places where models and satellites need further investigation, e.g. the SH for models and 45–67° N for GOSAT

54 ENVIRONMENTAL SCIENCES↗