Very-high-resolution (VHR) satellite image contains reliable various information over large areas, so that, it has been used as key data in the field of remote sensing. Image registration must be conducted to effectively use the multitemporal VHR satellite images. Conjugate points (CPs) extracted from the same region between images are required to perform image registration. However, outliers included in the CPs cause distortion when they were used for the image registration. Here we propose a deep learning-based technique to effectively remove the outliers. A Siamese network was built as a purpose of an outlier removal, and the network was trained using data based on the patch pirs centered on each CP. Experimental results demonstrate that the proposed method can remove outliers more effectively than a random sample consensus (RANSAC) technique thus and achieves improved registration accuracy.