Cell-free massive MIMO (CF-mMIMO) is a crucial technology for 6G. For improving computing efficiency and saving transmission cost, we consider the mobile edge compting (MEC)-empowered CF-mMIMO system, which enables the computing tasks of the system to be processed and calculated on multiple edge servers, rather than always on the cloud server (CS). An optimization problem for the system is formulated, which reduces the energy consumption by jointly optimizing offloading decisions, and the allocation of both the communication resources and computing resources. To address this intricate optimization problem, we propose an optimization scheme based on the twin delayed deep deterministic policy gradient (TD3) algorithm. The TD3 algorithm achieves rapid convergence and stable energy metrics by continuously interacting with the system. Simulation results show that the TD3-based scheme has the fastest convergence speed and outperforms the baseline.