Face reflects the emotions in mind and hence by analyzing facial expressions one can read the emotions of others. Emotion recognition is a challenging task. Inspired by the sophisticated human nervous system, the Artificial Neural Network (ANN) is an efficient machine learning tool to handle this kind of difficult problems. Hence in this paper, an efficient ANN based human emotion recognition system using the features of discriminative facial patches is proposed. First of all, the facial landmark points are detected from the facial expression images and some active patches are considered. Then, dimensionality reduction is performed to obtain the discriminative facial patches and the features are extracted from these patches. Local Binary Pattern (LBP) and Histogram of Oriented Gradients (HOG) features are extracted as the feature vectors. The experiments on JAFEE datasets are made to recognize the six universal expressions (anger, fear, disgust, happiness, sadness and surprise). Finally, the extracted features are classified using both Support Vector Machine (SVM) and ANN classifiers. Results show that recognition of human expressions with ANN using HOG features has a better recognition rate compared to that using appearance features.