In recent years, virtual reality (VR) has emerged as a novel paradigm for immersive applications for rehabilitation purposes. When designing a virtual rehabilitation application, difficulty customization and motivation improvement are two major concerns and are essential for ensuring a desirable rehabilitation outcome. It is significant to explore the impact of task difficulty on user experiences in virtual rehabilitation. This paper proposes a VR rehabilitation system for walking tasks with difficulty customization based on performance judgment (loose, standard, strict, and adaptive modes). In addition, this paper investigates the influence of walking task difficulties on users' task performance and user experience with sense of agency and emotion as two indexes, through an experimental study of users' subjective feelings and objective measured data. The results show that the task difficulties designed in this study have a noticeable influence on task performance and user experience. The proposed system and experimental study can provide a strong empirical basis for building a more effective and user-centered VR rehabilitation training system.