Self-calibrating optical galaxy cluster selection bias using cluster, galaxy, and shear cross-correlations
ABSTRACT The clustering signals of galaxy clusters are powerful tools for self-calibrating the mass–observable relation and are complementary to cluster abundance and lensing. In this work, we explore the possibility of combining three correlation functions – cluster lensing, the cluster–galaxy cross-correlation function, and the galaxy autocorrelation function – to self-calibrate optical cluster selection bias, the boosted clustering and lensing signals in a richness-selected sample mainly caused by projection effects. We develop mock catalogues of redMaGiC-like galaxies and redMaPPer-like clusters by applying halo occupation distribution models to N-body simulations and using counts-in-cylinders around massive haloes as a richness proxy. In addition to the previously known small-scale boost in projected correlation functions, we find that the projection effects also significantly boost three-dimensional correlation functions to scales of 100 $h^{-1} \, \rm Mpc$. We perform a likelihood analysis assuming survey conditions similar to the Dark Energy Survey and show that the selection bias can be self-consistently constrained at the 10 per cent level. We discuss strategies for applying this approach to real data. We expect that expanding the analysis to smaller scales and using deeper lensing data would further improve the constraints on cluster selection bias.