In this paper, a deep learning and adaptive block-based chaotic encryption algorithm for atrial fibrillation medical image is proposed. Firstly, we use 2D Sine Logistic chaos system to generate two security sequences with good chaotic characteristics. Then the image is divided into fixed size image blocks, and the maximum pixel difference and variance of the image blocks are calculated. Finally, chaotic sequence 1 is used for ciphertext feedback encryption of smooth blocks, and chaotic sequence 2 is used for plaintext feedback encryption of complex blocks. So, the encrypted image is obtained. The RBF network is used to predict the chaotic sequence, and the predictive key stream is obtained. Experiment results show that the proposed algorithm has high encryption efficiency, and the encryption speed is about 1 time higher than that of the existing methods. The new algorithm is suitable for real-time encryption of medical images with large amount of data.