Shadow play is an attractive part of Chinese traditional folk art however the recognition of shadow play characters is a tough work for an ordinary people. Recent year, because neural networks have a preeminent performance on object classification, we try to use different CNN architectures to solve such task. Commonly, big dataset is necessary for training CNN but our original dataset is quite small. To solve the problem, we did data augmentation by graphic transformation. After comparing the accuracy of different models, experimental results show that ResNet can achieve a high classification effect in the shadow puppetry character dataset.