Recently, the self-driving system for automobiles and the driving support system have been actively developed. In the shipping industry, the research related to autonomous and remotely controlled ships has been carried out to reduce seafarer's work load, support the seafarer’s shortage, decrease the operational cost and improve the working environment. However, many autonomous ship studies do not consider the recognition problem of other ships and obstacles using cameras during the night navigation.