This paper proposes a classification-assisted deep sparse network (CDSRC) model to achieve the purpose of image classification. The proposed algorithm consists of four parts: Encoder, Self-representer, Decoder and Classifier. The Encoder part can extract the high-level feature map of the input image, and Self-representer can establish the representational relationship between the test set and the training set image, so as to reconstruct the image of the test set. The Decoder can restore the reconstructed sample to the original image in the form of deconvolution, which is used to supervise the Self-representer. Next, the Encoder can effectively extract the feature map of the original image and the reconstruction of the test set sample. In addition, in order to increase the robustness of image recognition, a Classifier part is added after the Encoder. The Classifier is mainly used to classify training samples while extracting features in the training phase. This will increase the feature similarity of images of the same category, increase the difference of image features of different categories, and reduce the noise interference formed by individual samples. After the algorithm training is completed, the test sample is imported into the Encoder to extract the feature map, the feature map is combined with the sparse matrix of the Self-representer part, and then the test sample category is predicted. Experiments show that the algorithm(CDSRC) in this paper has better results than the SRC-related algorithms that have been proposed.