Handwritten Character Recognition (HCR) is convenient for people to save handwriting information in digital form in a short time. However, most current works can only recognize a single type of character, which is not enough for practical use. Moreover, the input is relatively limited since most of the works do not support Red, Green, and Blue (RGB) input or high-resolution input. Furthermore, the published code-based recognition process is unfriendly to non-programmer users. In this work, an improved recognition system of handwritten characters, integers, and symbols is constructed by applying a deep learning algorithm called convolutional neural networks (CNN). The proposed CNN model employs the structure of the MobileNet. To design this model, Tensorflow is adopted to achieve the stated goal. At the same time, a rather huge training dataset is applied to ensure a relatively decent accuracy. Lastly, we compile our model with an optimizer that implements the Adam algorithm and set the epochs to 10 when we fit the model. The experimental results indicate that the proposed method achieved 0.88 for accuracy on the testing data, which is satisfactory in a real application. Besides a Graphical User Interface (GUI) has been formed so that even non-programmer users can easily use it with their electronic devices.