Food safety has always been the focus of attention. In order to ensure the food hygiene of operational restaurants, the kitchen of restaurants is required to equip with monitoring to prevent safety and hygiene problems caused by non-standard operations. The traditional monitoring method is to view the surveillance video by manpower, which is extremely labor-consuming and time-consuming, with low accuracy and poor operability. Aiming at this problem, this paper proposes a system with a deep learning method to automatically identify the violation of kitchen staff. The system will use the camera in the kitchen scene to detect and identify a variety of non-standard behaviors affecting food safety in the collected video stream, including not wearing chef's hat and smoking, so as to realize intelligent supervision of kitchen food safety and achieve good results.