To solve the problem of low precision of weed identification and segmentation in a complex field environment, a weed segmentation and recognition method based on an improved Mask R-CNN model was proposed. CBAM (Convolutional Block Attention Module) attention mechanism is added to enable the model to better focus on effective features, strengthen the model’s learning of weed contour, texture, color, and other features, suppress invalid features, and improve the efficiency of the model feature learning while increasing less computational effort. The depth separable convolution is used to replace the traditional convolution operation. Its first channel convolution and point convolution can reduce the number of parameters without omitting any features and improve the model operation speed. The data input stream structure of the model is improved, the size of the convolution kernel is reduced, and the learning ability of the model for small targets is increased. When IoU is 0.5, mAP is 0.948 on the beet and miscellaneous vegetable dataset using this model, which is significantly improved compared with the model before improvement. The results show that this method can effectively identify weeds and segment weed contours in complex environments.