With the rapid development of wireless power transmission (WPT) technology and the demand for a wide range of applications, the common two-coil system is becoming increasingly mature. However, the monotonous structure of two-coil systems limits their development. By incorporating relay coils, the distance of power transmission, efficiency, and electromagnetic field (EMF) leakage can be effectively improved. In this paper, the three-coil system is analyzed, and the relay coil parameters are optimized by an evolutionary algorithm. After that, based on the coil optimization, the compensation capacitance of the relay coil is analyzed and an analytical formula of the compensation capacitance of the optimized relay coil is obtained. The results show that the parameter-optimized system reduces current by 86.9%, increases efficiency by 10%, and suppresses EMF leakage by 18 dB. Meanwhile, the capacitor optimization further improves the transmission efficiency of the system.