In this paper, we present a method for pose-estimate-based target tracking (PBTT) that enables the performance of autonomous aerial manipulation operations in unstructured environments using fully on-board computation for both UAV localization and target tracking. The PBTT method does not depend on extracting traditional visual features (e.g. using SIFT, SURF, ORB, etc.) on or near the target. Instead, the algorithm combines input from an RGB-D camera and the UAV’s position estimator (which utilizes a downward-facing optical flow camera for horizontal localization) to track a target point selected by a human operator. The effectiveness of the PBTT method is evaluated through several autonomous flight tests performed with the Interacting-Boomcopter (I-BC) UAV platform in unstructured environments and in the presence of light wind disturbances.