Discovering domain generation algorithms (DGAs) used to build command and control (C&C) infrastructures of botnets is crucial for recognizing botnets. Recent studies in DGA detection benefit from deep learning, such as convolutional neural network (CNN) and long short-term memory neural network (LSTM). However, these studies need massive supervised data to train their models, while obtaining enough labeled samples is consistently time-consuming and labor-intensive. In this paper, we propose a deep learning model, called PEPC, to detect and classify DGA domain names with only a small dataset. PEPC consists of two modules: (1) the pre-trained embeddings (PTE) module to quantify domain names to numeric vectors; and (2) the deep parallel convolutional neural networks (DPCNN) module to better extract features of vectors for prediction. Comparing our model with the 5 common deep learning-based DGA detection approaches, results show that our model yields an average improvement of 10 F1 points, while it requires just 30 training samples for each class. Significantly, PTE can help models achieve better detection and classification performances on small training samples.