In this paper, an unknown system dynamic estimator (USDE) with only partial model information is proposed to design a feedback controller for a flexible joint manipulator. In this framework, both the unmodeled dynamics of the system and the effect of flexible joints are estimated and a diagonal matrix is constructed to reduce the computational burden in multi-degree-of-freedom robot control. Furthermore, the USDE-based feedback controller is designed to guarantee the tracking performance. Theoretical analysis is studied in terms of the Lyapunov theory. Finally, comparative numerical simulation results are provided based on Baxter robotic system to demonstrate the feasibility and effectiveness of the proposed method.