We investigate the tensor force (TF) effect %in the residual interaction on the Gamow-Teller (GT) transitions in four magic nuclei, $^{48}$Ca, $^{90}$Zr, $^{132}$Sn and $^{208}$Pb. The TF is taken into account by using the Br\"uckner $G$-matrix theory with the charge-dependent (CD) Bonn potential as the residual interaction of charge-exchange quasiparticle random phase approximation (QRPA). We found that particle-particle ($p-p$) tensor interaction does not affect the GT transitions because of the closed shell nature in the nuclei, but repulsive particle-hole ($p-h$) residual interaction for the $p-h$ configuration of spin-orbit partners dominates the high-lying giant GT states for all of the nuclei. It is also shown that appreciable GT strengths are shifted to lower energy region by the attractive $p-h$ TF for the same $j_\pi=j_\nu$ configuration, and produce the low-lying GT peak about 2.5 MeV in $^{48}$Ca. Simultaneously, in $^{90}$Zr and $^{132}$Sn, the low-energy strength appears as a lower energy shoulder near the main GT peak. On the other hand, the shift of the low-lying GT state is not seen clearly for $^{208}$Pb because of the strong spin-orbit splitting of high $j$ orbits, which dominates the GT strength.