In an earlier study, we proposed a method to estimate the bolus velocity from a swallowing ultrasound video, which is considered an effective indicator for the evaluation of swallowing function to prevent aspiration pneumonia. However, this method requires a manual setting of the timing at which the bolus appears in the field of view. Therefore, in this study, we propose a method to automatically detect the timing of bolus inflow by extracting the bolus inflow region to detect the moving object. By applying this method to videos in which swallowing occurs, we were able to detect the frame number at which the bolus inflow occurred with an average error of 1.5 frames. Considering the conventional visual inspection results as the baseline, we expect that this method will detect the timing with the same accuracy, even in the data of subjects with lesser esophageal wall movement during swallowing than that observed in healthy subjects.