With rapid developments of artificial intelligence (AI) techniques, biometrical recognition will be further extended to the field of hand gesture recognition. It’s expected that common hand gesture actions in the daily life will be able to be smartly recognized by an AI system to further promote communication between persons and persons (or machines). This paper presents a hand number gesture recognition system where the popular deep learning model technique, the convolution neural network (CNN), is employed for classifications of hand number gesture images. In this work, recognition performance evaluations are done by different system settings including mainly CNN input images, CNN structures and fully connected layer (FC) parameters of CNN. In hand number gesture experiments with various system settings, the setting that the input hand number gesture image with segmentations of region of interest (ROI) combined with the specific CNN model containing two FC layers can have the most satisfactory performance, achieving the validation accuracy of 92.3% and the test accuracy of 81.3%.