In this paper, a fast wavelet based background modeling is introduced, which need only a few image sequences. The building of our background modeling mainly concludes four steps. The first step is getting the first-layer wavelet coefficients from background image. Secondly, separate the three high-frequency region into several 2*2 blocks, and calculate the mean and difference of every block. Then, connect and label the blocks with similar mean and difference, and re-calculate then mean and difference with same label. At last, combined the pixels information of wavelet low-frequency with its corresponding positions the eigenvector matrix of background is constructed. In addition, after iterative training, it can also be applied to fixed camera scenes.