Circular RNAs (circRNAs) play a critical role in gene regulation and association with diseases due to their specialized structure, which is formed as a closed loop structure during a non-canonical splicing process where the donor site back-spliced to an upstream acceptor site. As fundamental work to clarify their functions and mechanisms, a large number of computational methods for predicting circRNA formation have been proposed, among which, in particular, deep learning is utilized to capture relevant patterns from raw RNA sequences and model their interactions to facilitate prediction. However, these methods fail to fully utilize the important characteristics of back-splicing events, i.e., the positional information of the splice sites and the interaction features of its flanking sequences, for prediction. To this end, we hereby propose a novel approach called SIDE for predicting circRNA back-splicing events using only nucleotide sequences. Our model employs a dual encoder to capture global and interactive features of the sequence, and then a decoder designed by the contrastive learning to fuse out discriminative features improving the prediction of circRNAs formation. Empirical results on three real-world datasets have shown the effectiveness of SIDE. Our code is publicly available at https://github.com/scu-kdde/Bioinfo-SIDE-2023.