Vertical vibration model of strip mill is established by considering piecewise nonlinear constraint arising from hydraulic cylinder. The approximate analytical solution of the model is obtained by incremental harmonic balance (IHB) method. The harmonic frequencies not conducive to stability are produced because the nonlinearity of the piecewise constraint. Bifurcation behavior of the system changes with external excitation amplitude and frequency ratio, intermittent chaos and periodic window are appeared alternately, the chaos region enlarged with the increasing piecewise interval. Compared with the piecewise linear constraint, the region of chaotic motion is widened obviously under the nonlinear constraint. Moreover, the critical condition of bifurcation and chaos behavior can be obtained from maximum Lyapunov exponent (MLE).