Facial expression recognition(FER) has been improved with convolutional neural network(CNN) in the last few years. In particular, the annotation of facial expressions is more complicated. Annotating each facial expression is prohibitively time-consuming and difficult task for psychologists. Therefore, each FER database only contains limited labeled samples, thus it is prone to overfitting when using CNN. In this paper, an effective neural network for FER is proposed, named Ghost-based Convolutional Neural Network (GCNN), which is to overcome overfitting. Ghost-module architecture utilizes a series of low-cost linear transformations to reduce parameters and generate more feature maps. With the Ghost-module architecture, GCNN can extract and classify the facial expression features effectively. Experiments show that this method beats all advanced methods on three FER databases, including RAF-DB, FER2013 and FERPlus databases.