Holoscopic imaging is a promising technique that captures full-colour spatial images using a single aperture. It uses a micro-lens array to view the scene at different angles and record 4D information on a two-dimensional surface, making it useful for depth estimation. However, current disparity estimation methods suffer from poor performance in texture-less regions. This paper proposes a novel method to reduce the disparity error in these regions by directly labelling and grouping elemental images from a Holoscopic image. The proposed approach involves extracting a subset of viewpoint images from the Holoscopic image and subjecting them to conventional image segmentation. Labels are then applied to the elemental images corresponding to each segmented object using viewpoint images/elemental image pixels mapping. Content-based image retrieval is also employed to improve segmentation. The proposed technique has wide applications for 3D imaging, including augmented and virtual reality, inspection, robotics, security, and entertainment.