In daily life, expression recognition is often used in medicine, transportation, education and other fields. In this paper, a multi-scale fusion face expression recognition algorithm based on VGGNet19 is proposed to address the problem of low recognition rate that occurs in existing convolutional neural networks for face expression recognition. The improved Xception network architecture is fused with VGGNet19 to improve the feature extraction capability. In addition, for VGGNet19, only one fully-connected layer is retained to reduce the number of parameters and improve computer efficiency, and a dropout strategy is introduced before the fully-connected layer to prevent overfitting, and finally the cross-entropy function is combined with Softmax for classification recognition. The final cross-entropy function was combined with Softmax for classification and finally obtained an accuracy of 89.83% on the RAF-DB dataset.