Benefiting from a large amount of labeled data, deep learning has attracted widespread attention for its excellent performance. This great success is inseparable from the support of the supervision of massive data. However, in industrial defect detection scenarios, the cost of collecting and labeling large amounts of data is quite high. Therefore, this paper explores a limited-data regime to build a more efficient defect detection model. To this end, we propose a defect-based location augmentation method to increase the diversity of training data and improve the performance of defect detection models. The scheme based on defect location augmentation greatly increases the diversity of training data, which alleviates the overfitting problem of insufficient training data. Experimental results show that the proposed method can well recognize the defect types of industrial components when only one shot defect data is available.