In this paper, we introduce a Gaussian process based moving horizon estimation (MHE) framework. The scheme is based on offline collected data and offline hyperparameter optimization. In particular, compared to standard MHE schemes, we replace the mathematical model of the system by the posterior mean of the Gaussian process. To account for the uncertainty of the learned model, we exploit the posterior variance of the learned Gaussian process in the weighting matrices of the cost function of the proposed MHE scheme. We prove practical robust exponential stability of the resulting estimator using a recently proposed Lyapunov-based proof technique. Finally, the performance of the Gaussian process based MHE scheme is illustrated via a nonlinear system.