Fire detection technology has been researched and developed for decades. However, in videos and complex scenes, it still lacks fast recognition of fire’s existence. The traditional model of fire recognition still needs a large number of samples and time-consuming machine learning progress. Meanwhile, the uncertain shape of fire leads to the reduction of accuracy using CNN. Based on the above problems, we establish a novel method based on I-Vector. We use an adapted I-Vector algorithm to extract the time sequence feature vector on the fire and its surroundings and train a G-PLDA classifier to recognize the dynamic occurrence of fire more quickly and accurately. This model requires fewer samples and a shorter learning time while obtaining an accuracy similar to the traditional fire recognition models, which provides a new effective solution for rapid analysis of whether there is a fire in the video scene. In addition, the algorithm has a sound universality and is easy to deploy in the application fields of video fire supervision, UAV fire inspection, and other related fields.