Geomagnetic data is vital for predicting earthquakes and magnetic storms. In this regard, a new Bayesian exponential regularized tensor completion framework for sparse geomagnetic data, i.e. BERTC, is proposed to address this problem in the study. First, the spatiotemporal geomagnetic data is reshaped into a 3D tensor with days and hours that features random missing elements. Second, a Gibbs sampling algorithm is developed to achieve probabilistic inference on matrices' factors and corresponding parameters in this model. Thus, the sparse tensor can be gradually optimized to fill the missing entries during iterations. Third, an exponential regularizer is proposed to reduce oscillations before and after iterations to enhance imputation quality further. Finally, the derived factor matrices are aggregated from Gibbs sampling to complete the sparse tensor. Numerical geomagnetic datasets from 13 cities are employed, and extensive comparison experiments are conducted to evaluate the imputation performance of the BERTC. The results show the superiority of the proposed BERTC compared to the state-of-the-art methods in terms of imputation accuracy, with an approximate improvement of the imputation accuracy as no less than 20%.