As the application scenarios of multi-rotor Unmanned Aerial Vehicles (UAVs) become more and more extensive, the field of target tracking and accurate landing has become one of the research hotspots. In order to realize the accurate autonomous landing of UAV, considering the increasingly mature visual navigation technology, this paper proposed a hierarchical landmark based on nested rectangles and its relative positioning algorithm, and designed a set of autonomous landing system for the mobile platform of quadrotor UAV. The landmark has a high recognition rate of different heights and good expansibility. The algorithm has high computational efficiency and low performance requirements for image processing equipment, thus the system has good accuracy and robustness. The simulation and flight experiment show that the proposed algorithm can effectively realize the accurate autonomous landing of UAV.