In most sub-Saharan African countries, staple cereal grains harbor many fungi and some produce mycotoxins that negatively impact health and trade. Maize and three small grain cereals (sorghum, pearl millet, and finger millet) produced by smallholder farmers in Zimbabwe during 2016 and 2017 were examined for fungal community structure, and total aflatoxin (AF) and fumonisin (FM) content. A total of 800 maize and 180 small grain samples were collected at harvest and during storage from four agroecological zones. Fusarium spp. dominated the fungi associated with maize. Across crops, Aspergillusflavus constituted the main Aspergillus spp. Small grain cereals were less susceptible to both AF and FM. AF (52%) and FM (89%) prevalence was higher in maize than in small grains (13–25% for AF and 0–32% for FM). Less than 2% of small grain samples exceeded the EU regulatory limit for AF (4 µg/kg), while