In order to meet the requirements of the sixth generation mobile communication (6G) system for high speed and large capacity of optical communication network, further improving the bandwidth, response rate and amplification of optical fiber amplifier in optical transmission network has become the focus of current research. Tellurite fiber is used as fiber gain medium, and a modified particle swarm optimization algorithm is proposed, which dynamically adjusts the individual velocity, position and inertial weight values during the iterative process to obtain higher convergence speed and enhance the global search capability. The algorithm is applied to optimize, analyze and verify the configuration of each pump light parameter of Raman fiber amplifier, and finally, the average gain of the backward-pumping Raman fiber amplifier is up to 15.9320dB and the gain fluctuation is about 0.3620dB. The results show that the modified particle swarm optimization algorithm can optimize the Raman fiber amplifier pump power and wavelength accurately and quickly, and can improve the performance of Raman amplifier effectively, which has a certain reference significance for the design of Raman fiber amplifier in the future.