Prebreakdown time delay is one of the key breakdown characteristics which can reflect the insulation and discharge performances of dielectric liquids. In this article, the investigation of the voltage-varying characteristics of prebreakdown time delays and the corresponding voltage polarity effects for underwater millisecond discharges have been carried out. The results indicated that the voltage polarity effect of total time delay is determined by formation time delay at low applied voltage; meanwhile, determined by propagation time delay at high applied voltage. It was further found that the liquid phase disturbance induced by Joule heat during the prebreakdown process has a remarkable influence on the streamer propagation. As a result, the abnormal voltage-varying characteristics and polarity effects variation of total and propagation time delays were observed for the first time. Finally, a simple prediction model of prebreakdown time delays was proposed and compared with the classical Martin fitting formula. The findings could give a new insight into the liquid breakdown phenomena under long pulses and provide a practical tool for the related engineering applications.