The flexible job shop scheduling problem (FJSP) is significant for realistic manufacturing. However, the job processing time usually is uncertain and changeable during manufacturing. This paper presents a multi-objective FJSP with fuzzy processing time (MOFFJSP) for optimizing the makespan and total machine workload as objectives. To solve the MOFFJSP, a MOEA/D based on reinforcement learning named RMOEA/D is proposed. RMOEA/D can be featured as: (i) an initial strategy with three rules is used to get a high-quality initial population; (ii) a parameter adaption strategy based on Q-learning is proposed to guide the population choose the best parameter to increase diversity; (iii) a variable neighborhood search based on reinforcement learning is designed to lead the solution to choose the right local search method; and (iv) an elite archive is used to improve the usage rate of the abandoned historical solution. RMOEA/D is compared with five well-known realted methods, i.e., MOEA/D, NSGA-II, MOEA/D-M2M, NSGA-III and IAIS on three benchmark suites. The results show that RMOEA/D outperforms these five state-of-art algorithms.