Tendency forecasting of infectious diseases, such as COVID-19, is urgently required to evaluate outbreak risk and control decisions. Although transmission models based on natural factors like virus propagation, temperature, and human modality are studied carefully, social factors cause high flexibility on dynamic propagation change under actual virus spreading conditions. We propose a time-variant relevance-based infected recovered extreme learning machine to generate a quantitative forecasting model with social factors. Also, embedded distance is used to measure the similarity and realize flexible forecasting based on social impactors. We investigated the age structure and the medical supply under the COVID-19 pandemic with nonidentical open-source data We found that embedded distance with the proposed model is highly consistent with projection accuracy, and the proposed method can achieve higher accuracy than existed methods. Based on the forecasting model, age distribution and medical supply make a difference in COVID-19 transmission. Areas with the middle proportion of the aged population face higher outbreaking risks, and sufficient medical supply control the infection speed efficiently within three weeks. This study provides an efficient projection of dynamic transmission under the social impact on infectious diseases pandemics.