Abstract Purpose Whole grains have recently been promoted as beneficial to diabetes prevention. However, the evidence for the glycemic benefits of whole grains seems to conflict between the cohort studies and randomized control trials (RCTs). To fill the research gap, we conducted a meta-analysis to determine the effects of whole grains on diabetes prevention and to inform recommendations. Methods We searched PubMed, Clarivate Web of Science, and Cochrane Library until March 2024. We used the risk ratio (RR) of type 2 diabetes to represent the clinical outcomes for cohort studies, while the biomarkers, including fasting blood glucose and insulin, HbA1C, and HOMA-IR, were utilized to show outcomes for RCTs. Dose-response relationships between whole grain intakes and outcomes were tested with random effects meta-regression models and restricted cubic splines models. This study is registered with PROSPERO, CRD42021281639. Results Ten prospective cohort studies and 37 RCTs were included. Cohort studies suggested a 50 g/day whole grain intake reduced the risk of type 2 diabetes (RR = 0.761, 95% CI: 0.700 to 0.828, I 2 = 72.39%, P