Due to the existence of system error or the difference in reference coordinates for imaging, there is a significantly spatial mismatch among video SAR frames, which will affect the performance of target localization and tracking greatly. To solve this problem, a PCE-RPM-Net that can achieve video SAR inter-frame images registration is proposed in this paper. PCE-RPM-Net is mainly composed of U-Net and RPM-Net, in which U-Net is adopted to extract features of SAR images to generate point clouds and RPM-Net is used to align the point clouds. Experimental results show that the PCE-RPM-Net has higher registration accuracy and faster speed compared with the classical traditional algorithm.