Recent research shows that the combination of reinforcement learning (RL) with traditional control method can be an effective tool for designing near optimal feedback controller for dynamic systems. In this paper, a vehicle-following control based on reinforcement learning is proposed, in which pairs of the input-output of model predictive control (MPC) are chosen as offline-learning data. Through continuous iterations of actor-network and critic-network, the longitudinal vehicle-following controller can be obtained. Simulation results illustrate that proposed learning-based predictive control (LPC) can improve the computational efficiency, and obtain a better performance of policy optimization.