To solve the dilemma introduced by rate-distortion optimization (RDO) in H.264 rate control, the linear model of MAD prediction is applied. However, this model suits image characteristics locally but in some cases it could be inefficient. If there is a low spatial correlation (i.e., the encoder is processing pictures with varying characteristics), the MAD estimated by the model does not stand for the complexity resulting in the improper quantization parameter (QP) value and image quality degradation. In this paper, the mean square of AC coefficients is introduced into R-D model instead of MAD. Experiments show that the proposed rate control scheme performs well when compared with JVT-G012.