Reconstructing the retinogeniculate visual pathway with the pituitary tumor compression is a difficult task. The tumor not only compresses the retinogeniculate visual pathway to produce a large angular deviation, but also interferes with the diffusion magnetic resonance image signal. In order to solve the above problems, the anatomical prior knowledge and fiber direction distribution are used in this study. We first manually drawn regions of interest as anatomical priors. The regions of interest are located in the eyeball and optic tract. Then, we purpose a mathematical model that combine the anatomical prior knowledge and fiber direction distribution. The mathematical model is an Markov decision process, which can be solved by reinforcement learning. In the comparison experiments, the purposed method shows high performance on retinogeniculate visual pathway reconstruction. Furthermore, only our method is able to tracking retinogeniculate visual pathway with the pituitary tumor.