Remote sensing technology is widely used in agriculture monitoring because of the advantages of large-area simultaneous observation, low cost and dynamic monitoring of time and space. However, a manual visual interpretation method is often used to extract the information behind the remote sensing images, which is time and labor consuming. Moreover, handcraft features such as texture and structure of crop images are applied to classify crop planting area while these features are not robust. In order to reduce the cost and enhance the classification accuracy, we improved the state-of-the-art image semantic segmentation network SegNet in crop planting area classification which can speed up the convergence and reduce the model size largely with small gains in accuracy. The experiment result shows that the classification accuracy of the improved SegNet is slightly increased compared with SegNet, and the computational cost (FLOPs) of the improved SegNet is much less than SegNet.