The possible mechanism of metal toxicity is oxidative stress and mitochondrial DNA damage. However, the association between environmental metal exposure and mitochondrial DNA damage and the potential mediators remain unclear. In this study, a total of 155 participants from general population were enrolled. In 2016, we measured participants’ blood and urinary chromium, cadmium, manganese, and lead concentrations, serum malondialdehyde (MDA) concentration, urinary 8-hydroxy-2 deoxyguanosine (8-OHdG) concentration, serum activity of superoxide dismutase (SOD), catalase (CAT), and glutathione peroxidase (GSH-Px). In 2018, we measured participants’ serum mitochondrial DNA polymerase γ (mtDNApolγ) activity, serum mitochondrial transcription factor A (mtTFA) activity, and mitochondrial DNA copy number (mtDNACN). We used general liner regression model and mediation analysis to explore the associations and potential mediation effects, respectively. Blood cadmium was associated with GSH-Px (βDoubling increase = 12.48), blood lead was associated with SOD (βDoubling increase = 5.41), and blood chromium was associated with mtDNApolγ (βDoubling increase = − 0.54; βTertile 3 vs 1 = − 0.24) and mtDNACN (βDoubling increase = − 0.22). Urinary cadmium was associated with mtDNApolγ (βTertile 3 vs 1 = 0.22) and 8-OHdG (βDoubling increase = 0.24). We did not find a significant mediation effect of oxidative stress on the association between metal exposure and mtDNA damage, but we found that mtDNApolγ accounted for 16% of the association between blood Cr and mtDNACN (P = 0.1). The findings suggest that metal exposure was associated with oxidative stress and mtDNA damage, and mtDNA transcription replicase might mediate the association between metal exposure and mtDNA damage.