Diabetic retinopathy (DR) is a vision-impairing disease affecting millions of people worldwide. Early diagnosis is crucial for the treatment and prevention of this pathology. In this study, we propose a new model called BERTImages, based on BERT (Bidirectional Encoder Representations from Transformers), for the detection and classification of diabetic retinopathy. Our model incorporates processing of spatial relationships and contextual features of retinal images. Experiments conducted on the APTOS Kaggle dataset show promising results for the detection and classification of diabetic retinopathy. Compared with other deep learning models, our method achieves the best rates in terms of recall, precision and F1 score.