While urbanization promotes both economic development and job opportunities, it also exerts enormous pressures on the urban land resource. Increasing urban land use efficiency (ULUE) is important towards achieving the sustainable development of China. This paper theoretically analyzes the impact of industrial agglomeration on ULUE and its spillover effects. Furthermore, the influence of industrial agglomeration is empirically studied by a spatial panel model based on Landsat-TM/ETM remote sensing image data and socioeconomic data of Chinese cities at prefecture-level and above for 2000, 2005, 2010, and 2015. The following results were obtained: First, from the perspective of the underlying theoretical mechanism, industrial agglomeration can increase ULUE through labor market externalities, technology externalities, and capital externalities; moreover, spillover effects of industrial agglomeration on ULUE of neighboring cities occurred through spatial interaction among local governments and the spatial flow of production factors. Second, ULUE showed strong spatial correlation, and both specialized and diversified agglomerations exerted significant positive spillover effects on ULUE. Third, the direct effects of specialized agglomeration were significantly positive for northeastern, central, and northwestern China; the direct effects of diversified agglomeration were significantly positive for northwestern China; the indirect effects of specialized agglomeration were significantly negative for eastern China and significantly positive for northwestern and southwestern China; the indirect effects of diversified agglomeration were significantly negative for eastern China and significantly positive for northwestern China. Considering these results, China’s future industrial development should be based on sub-regional characteristics. More targeted industrial agglomeration policies are therefore needed to improve ULUE and support the economic development of the Chinese society.