With the fast increasing of mobile devices and energy consumption, energy efficiency became a popular topic in recently. Fortunately, the emergence of various communication services and development of the reinforcement learning provided a new chance to increase the energy efficiency. In this paper, we analyze the delay insensitive service. And, the optimal problem of the energy efficiency is also formulated under the perfect channel states information (CSI) and statistical CSI assumptions, respectively. Furthermore, in order to overcome the limited of partial CSI, we proposed an energy efficient transmission policy based on the reinforcement learning. Numerical results show that the performance of the proposed policy is obvious better than the performance of statistical CSI and approaches to the performance of perfect CSI effectively.