Understanding the semantic concept recognition in the learning brain benefits disorder treatment and high-quality education. This paper provides a study on identifying different semantic concepts from fMRIs to investigate their different brain activities. In this study, we extracted the functional connection matrixes from fMRIs after their pre-proceedings. To recognize the semantic concepts, we employed two deep learning models, i.e., LeNet and ResNet, compared to the classic support vector machine. Experimental evaluations were conducted on a publicly available dataset to recognize the word classes, including Nonu, verb, and adjective. The results manifest that two deep models perform much better than SVM in accuracy, precision, and recall, whereas ResNet is better than LeNet. That is to say, there exists a different pattern between different concepts. In addition, we also used the mentioned models to identify the different concepts for each individual, indicating their different patterns in individuals. This study contributes to understanding human cognition and language processing and puts forward prospects for disorder treatment and education design.