In this paper, cement-based piezoelectric composite material is introduced into the periodic structure. A type of one-dimensional phononic crystal structure was proposed to overcome the difficulty of opening the low-frequency band gap. By the finite element method, the band gap structure was calculated. The influencing factors of the band gap were analyzed. The results show that the lower the density of hard clay, the higher the cut-off frequency of the band gap, while the higher the density of cement-based piezoelectric composite material, the lower the starting frequency of the band gap. The width of band gap also closely related to the lattice constant of the structure and the filling rate of material.