Since the global outbreak of COVID-19, the epidemic has had a great impact on people's lives and the world economy. Diagnosis of COVID-19 using deep learning has become increasingly important due to the inefficiency of traditional RT - PCR test. However, training deep neural networks requires a large amount of manually labeled data, and collecting a large number of COVID-19 CT images is difficult. To address this issue, we explore the effect of Pretext-Invariant Representation Learning (PIRL) using unlabeled datasets to pre-train the network on classification results. In addition, we also explore the prediction effect of PIRL combined with transfer learning (TF). According to the experimental results, applying the TF-PIRL prediction model constructed in this paper to COVID-19 diagnosis, the accuracy and AUC are 0.7734 and 0.8556 respectively, which outperform the network training from scratch, transfer learning-based network training and PIRL-based network training.