Feature-based visual odometry suffers from weak feature extraction ability, insufficient image feature exploration, and instability under variations of illumination and viewpoint. To address these issues, this paper proposes a visual odometry algorithm based on handcrafted and learned CNN feature points. Firstly, a KHNet feature detector is constructed by combining a detector-descriptor network. Secondly, a geometric constraint is established by using a bidirectional nearest neighbour matching method for feature matching and pose estimation. Finally, local window bundle adjustment optimization is performed on the newly generated keyframes. Experimental results show that the proposed method is more robust to variations of illumination and viewpoint compared to visual odometry based solely on handcrafted or learned CNN feature points. Moreover, the accuracy of both absolute and relative pose errors is improved, demonstrating the feasibility and robustness of the proposed method.