The trajectory generated by GMR-GP cannot go through the first via-point precisely. Therefore, this paper designs an optimization method of Learning from Demonstration based on incremental GMR-GP. First, an incremental GMR-GP algorithm is designed. The advantage of the incremental GMR-GP is that the starting point of the mean trajectory can gradually approach to the real observation point while maintaining the true intention of the teaching action as much as possible. Then, an incremental GMR-GP based on importance weighting is proposed, which makes an important distinction among the new trajectories generated by the incremental GMR-GP. The generated trajectory further improves the reservation of the prior trajectory on the true teaching intention, and the uncertainty is reduced. Moreover, the availability of the proposed method is validated and analyzed by performing a series of numerical simulations and Baxter robot experiments. The results indicate that the proposed method can provide reliable solutions, which can go through the first via-point more precisely while retaining the true demonstrating intent as much as possible.