Phase-sensitive optical time-domain reflectometer technology can transform the fiber-optic cable into a large-scale sensor array for distributed acoustic sensing (DAS), which is an emerging infrastructure for the Internet of Things. However, it is limitated in event recognition capability, which is a major factor preventing its practical field application. This article proposes a perturbation recognition algorithm based on GASF-ConvNeXt-TF with fast process and high recognition accuracy. First, Gramian angular summation field (GASF) algorithm is used to encode external disturbance signal to transform the 1-D time-series signal into a more concentrated 2-D image feature. Then the convolutional neural network model ConvNeXt_tiny network is applied as the classifier. In order to prevent the weight gradient from oscillating back and forth during network training process, a cosine annealing algorithm is introduced to control the decay of the learning rate. Meanwhile, transfer learning is used to further optimize the network model, resulting in higher classification accuracy and faster convergence. Finally, two different experimental scenarios are arranged in a total length of 2.2 km of optical fiber cable, and six different disturbance events (shaking, kicking, knocking, trampling, wheel rolling, and impacting) are set. Different from previous perimeter security disturbance identification experiments, not only single-point disturbance recognition is performed but also two points disturbances are simultaneously recognized, and all have good overall identification accuracy. The overall recognition accuracy of the six disturbance events in single and multiple points experiments are 99.3% and 98.3%, respectively, with an average recognition time of 0.103 s. The proposed technique has potential application in infrastructures structure health monitoring, such as factories, airports, energy pipeline, and highway.