Fine grained image classification is a very popular research topic in the fields of computer vision and pattern recognition in recent years. At present, fine-grained image classification by deep learning is mainly based on a single image. The network cannot find the subtle differences between similar image pairs, so it is unable to find the high discriminative regions accurately and efficiently. In this paper, an image classification method based on channel interaction mechanism is proposed. By selecting image feature pairs with high similarity, features of the two images are subtracted to extract the discriminative semantic features in the image pair. Then, the vector is used for classification. This method can learn the interaction between two images in a dynamic way. The comparison with traditional CNN on different fine-grained image data sets show the effectiveness of this method.