The seal structure is an efficient way to reduce leakage and improve rotor stability in gas turbine machines. Various seals have been proposed to achieve this goal in the last century. Labyrinth seals (LS), one of those seals, are widely used in industry due to their relatively simple structure. This paper studied whirl radius dependence of hole diaphragm labyrinth seal (HDLS), which designed by adding hole-diaphragms in LS. A series of working conditions of whirl frequencies, rotation speeds and whirl radius rates were considered in simulations. The characteristic parameters in the rotordynamic model of the seal rotor system were calculated. The results show that a better stiffness performance of HDLS was obtained than those of LS and diaphragm labyrinth seal (DLS) on the small whirl radius. The effective damping of HDLS is slightly better than that of LS but moderately worse than that of DLS at higher rotation speeds. The adaptability under multiple working conditions of HDLS represents its feasibility, which can be used in seal-rotor systems. The analysis of the new type of labyrinth seal proposed in this paper confirmed some advantages in gas turbine and steam turbine applications and is worthy of further study.