In the last decade, deep reinforcement learning (DRL) algorithms have been employed in the design of energy management strategy (EMS) for hybrid electric vehicles (HEVs). Investigation of the real-time applicability of DRL algorithms as an EMS is critical in terms of training time, fuel savings, and state-of-charge (SOC) sustainability. To this end, we propose a twin delayed deep deterministic policy gradient (TD3) algorithm that is an improved version of the deep deterministic policy gradient (DDPG) algorithm for HEV fuel savings. Compared to the existing Q-learning-based reinforcement learning and the deep Q-network-based and DDPG-based deep reinforcement algorithms, the proposed TD3 provides stable training efficiency, promising fuel economy, and a lower variation range of SOC charge sustainability under various drive cycles.