Vehicle trajectory prediction is a vital component for autonomous vehicles environment perception system, which remains a challenging problem due to the high density, frequent occlusions with interaction, and the randomicity of surrounding objects’ short-term path. In this paper, we propose a triplet refinement LSTM network for vehicle trajectory prediction. A novel LSTM module is proposed to capture historical information interaction between current object and surrounding objects. Furthermore, we derive a Graph Convolutional Attention (GAT) module to model the spatial interactions between object groups. Then, we present a temporal attention module to estimate the importance of objects’ behavior fragment. Finally, an end-to-end framework is composed by above three components. The quantitative and qualitative experiments on D 2 -City dataset proved that our algorithm can achieve competitive performance than state-of-the-art methods.