Beyond Fisher forecasting for cosmology
The planning and design of future experiments rely heavily on forecasting to assess the potential scientific value provided by a hypothetical set of measurements. The Fisher information matrix, due to its convenient properties and low computational cost, provides an especially useful forecasting tool. However, the Fisher matrix only provides a reasonable approximation to the true likelihood when data are nearly Gaussian distributed and observables have nearly linear dependence on the parameters of interest. Also, Fisher forecasting techniques alone cannot be used to assess their own validity. Thorough sampling of the exact or mock likelihood can definitively determine whether a Fisher forecast is valid, though such sampling is often prohibitively expensive. Here we propose a simple test, based on the Derivative Approximation for likelihoods (DALI) technique, to determine whether the Fisher matrix provides a good approximation to the exact likelihood. We show that the Fisher matrix becomes a poor approximation to the true likelihood in regions where two-dimensional slices of level surfaces of the DALI approximation to the likelihood differ from two-dimensional slices of level surfaces of the Fisher approximation to the likelihood. We demonstrate that our method accurately predicts situations in which the Fisher approximation deviates from the true likelihood for various cosmological models and several data combinations, with only a modest increase in computational cost compared to standard Fisher forecasts.