Steady-state visual evoked potentials (SSVEP) systems are considered ideal for practical brain-computer interfaces, with a focus on precise decoding of multi-class datasets in short time. Convolutional Correlation Analysis (Conv-CA) combines convolutional neural networks with canonical correlation analysis, extracting features from electroencephalography signals and template signals in two branches. These features are then classified using a correlation layer. This paper introduces residual connections to the network, proposing the improved Residual-based Convolutional Correlation Analysis (ResConv-CA) model. All experiments were validated on a 40-class public dataset, and considering the 140ms visual delay of SSVEP, we selected an effective data length of [0.64s, 5.64s]. In experiments, the ResConv-CA model showed increased average classification accuracy and improved accuracy for challenging subjects. This confirms the improved stability and accuracy of the enhanced model with the addition of residual connections.