We studied helicity resolved Raman scattering in PtSe$_2$ flakes with different thicknesses. The peak amplitude of helicity-switched Raman scattering is significantly larger than that of helicity-conserved scattering for the in-plane $E_{g}$ mode, consistent with the Raman tensor analyses and conservation law of angular momentum. The peak amplitude of the helicity-switched $E_{g}$ mode is larger for the thinner flakes. In addition, we find Raman peaks near the energy levels of infrared (IR)-active $E_u$ and $A_{2u}$ modes, only for monolayer and a few-layers thick flakes. Interestingly, these peaks manifest themselves only for helicity-switched Raman scattering; they are nearly absent for helicity-conserved scattering.