Environmental disturbances may lead to misidentification, missed detection and false alarm of visible light perception system for unmanned ships. In this paper, effects of environmental disturbance factors such as rain, fog, droplet and noise on the visible light perception system of unmanned ships are assessed. By analyzing and specifying evaluation methods for environmental factors, experiments were conducted using the VOC2007 dataset with three target recognition algorithms, YOLO, Faster R-CNN, and SSD. The precision and recall results of three algorithms before and after disturbances injection indicate that the SSD algorithm exhibits better resilience to environmental factors. This evaluation method is demonstrated to be effective and applicable for Unmanned ships.