We revisit von Neumann’s algorithm for generating exponentially distributed variable. This algorithm requires$e^{2}/(e-1)\approx 4.30$ uniform deviates from (0,1) on average to generate an exponentially distributed variable. In 2016, the early rejection was suggested by Karney to use in von Neumann’s algorithm for lowering the expected number of uniform deviates to $el(\sqrt{e}-1)\approx 4.19$. In this paper, we give a new parameter setting for the early rejection step, which can help reduce the expected number to a minimum of 4. The experimental results also show that our improved version of von Neumann’s algorithm can be slightly more efficient than the version presented by Karney especially for software implementations.