Rapid intensification (RI) of tropical cyclones (TC) remains a major challenge for extreme weather forecasting. In this paper, we present a model named TEMPO-RI (mulTi-task spatio-tEmporal Model of troPical cyclOne Rapid Intensification forecasting) to provide a 24-hour forecast for RI by using spatial-temporal reanalysis data from the past 24 hours. The model consists of 3DCNN (3-Dimensional Convolutional Neural Network), 2DCNN (2-Dimensional Convolutional Neural Network), and BiGRU (Bidirectional Gate Recurrent Unit). Specifically, the 3DCNN and 2DCNN are used for extracting atmospheric features, oceanic features, and domain knowledge from spatial reanalysis data. The BiGRU is used to learn the short-term dependencies in the time series of the TC from both forward and backward directions. Here, we also consider RI forecasting as a joint task by predicting whether it will intensify rapidly and the intensity of RI samples at the same time to tackle the problems of the imbalanced RI datasets. Our model with the joint task shows a 44.0% and 53.3% improvement in the probability of detection (POD) and Peirce Skill Score (PSS) respectively, and a 42.9% reduction in false alarm ratio (FAR) compared to the benchmark in an imbalanced dataset for RI: no-RI (1: 9). Compared to the NHC (National Hurricane Center) Operational RI Consensus Forecast, our POD and PSS are 9.1% and 16.9% higher than the operational forecast, respectively, and the FAR is 45.1% lower.