To address the issue of inefficient autonomous exploration, this paper proposes a heuristic planning framework for efficient exploration. We extract recently changed spatial voxels from a 3D model based on an octree map, extract frontiers, and introduce an improved K-means-based frontier clustering algorithm to achieve efficient clustering of frontiers. Simultaneously, we employ the Dynamic Window Approach (DWA) for collision-free tracking of local frontier cluster points, while concurrently growing RRT in the global scope to generate paths for the global frontier cluster points. Furthermore, in order to mitigate the myopic behavior of exploration strategies, an utility function is designed that simultaneously considers information gain and path length. A heuristic algorithm, Lin-Kernighan heuristic (LKH), is utilized to solve the Traveling Salesman Problem (TSP) within this context. We compare our proposed algorithm with several advanced algorithms in multiple simulated environments. Experimental results demonstrate that the proposed algorithm achieves approximately 28.5% faster exploration efficiency compared to state-of-the-art algorithms.