Sapphire substrate is the most commonly used in semiconductor industry for GaN-based light emitting diodes (LEDs). Chemical mechanical polishing (CMP) is one of the most effective methods to achieve atomic-scale smooth surface. Sapphire is very difficult to process due to its brittleness and high hardness. In order to improve the processing efficiency of sapphire CMP, ZnO was used as an additive in this paper. Because of the insolubility of ZnO in the slurry, the effect of ZnO on the removal rate is not obvious. Through the preparation process optimization of mixing surfactant, deionized water and ZnO with ultrasonic dispersion, the dispersion effect of ZnO in sapphire slurry was obviously enhanced, and the CMP processing efficiency of sapphire substrate was effectively improved. Meanwhile, the action mechanism of ZnO was investigated in depth.