Feature selection is usually a necessary step for data mining and machine learning. Currently, secure machine learning, especially in privacy preservation, has attracted much attention. However, feature selection with privacy preservation is still a new issue, especially for ensemble feature selection. In this paper, a differentially private ensemble feature selection algorithms is presented. The basic idea behind the proposed algorithm is the output perturbation where the density of perturbation noise depends on the privacy degree and sensitivity of original feature selection algorithm. Besides the theoretical proof, the experimental results also demonstrated their high performance under certain privacy preservation degree.