In this study, we propose a method for detecting changes on the outer surface of pipes using inspection videos captured by an inspection robot. It is critical to detect anomalies on the outer surface of pipes during patrol inspections. Anomalies are defined as deviations from the normal state and should be detected as areas that have changed from the normal state. Therefore, for appropriate maintenance of the plants, it is crucial to perform change detection by comparing videos that capture the past normal state with those capturing the current state. The problem with detecting changes from videos is deciding which frame to compare. We therefore propose sequential filtering to determine image pairs based on the position of the images and their similarity. We then apply a deep learning method to perform change detection. An indoor simulated plant environment has been constructed to test the efficacy of the proposed method. Experiments and evaluation results showed that the proposed method outperformed an autoencoder. The proposed method also achieved an F1 score of 0.880 for change detection in the inspection videos by introducing sequential filtering, which prevented mismatching of image pairs and reduced computational costs.