In this paper, we present an improved hybrid technique for spontaneous micro-expression based human emotion recognition. Micro-expressions are the involuntary gestures that people often try to conceal and suppress. But these expressions can be important tools for potential humanitarian applications such as lie detection, enhancing relationships, national security, clinical therapy and psychiatry. An effective micro-expression recognition technique requires feature extraction and classification. We select and extract features in the first phase using optimal Firefly algorithm. The second phase follows improved swarm optimized functional link artificial neural network (ISO-FLANN) algorithm for feature classification. Finally, emotion recognition is done taking weighted outputs of the classified features. To implement the proposed system, CASME II database of micro-expression collection has been used. Performance of the proposed system is evaluated presenting important figures of merit. We report an overall recognition rate of 68.65% which is superior to the state of the art methods. The proposed method is a simulation based study of human micro-expressions and thus adds useful contribution to the field of human computer interaction.