Motor functions of human hand during daily living activities involve multiple finger movements, which has not yet been fully explored for electromyogram (EMG) based prosthesis control. This paper presents a framework based on forearm muscle synergy for recognition of finger movement using four channel EMG. With five normal-limbed subjects, synergy of four forearm muscles was estimated for five finger movements through non-negative matrix factorization of EMG feature. Using leave-one-patient-out cross-validation, radial basis function support vector machine was implemented for recognition of finger movements. The framework exhibited an average recognition rate of 97%. This study offers feasibility of a finger movement recognition framework based on the inherent physiological mechanism of muscle synergy, which has potential for dexterous finger movement control in prosthetic hands.