An image restoration and evaluation algorithm based on saliency detection is proposed in this paper. Firstly, the saliency map corresponding to the original image is extracted based on the local feature and regional feature, and then the binary image corresponding to the saliency map is obtained by threshold segmentation method, so as to obtain the salient object image and background image. Then the salient object and background images which are only disturbed by noise are prepared. Finally, through the quality evaluation index to compare the restoration effect of only restoring the salient object and only restoring the background in the whole degraded image, it is concluded that the salient object has more important influence on the image quality.