We study diffractive vector meson production in ultra-peripheral collisions (UPCs) of heavy nuclei, utilizing a theoretical framework based on the Color Glass Condensate (CGC) formalism. We focus on Au+Au, U+U, Ru+Ru, Zr+Zr, and Pb+Pb collisions, examining the transverse momentum dependence of vector meson production cross-sections and ${\rm cos(2\Delta\Phi)}$ asymmetries in the decay product distributions to explore the role of nuclear geometry. The angular modulation is due to the linear polarization of the incoming photons and quantum interference effects. We extract nuclear radii and find them to be consistent with experimental data from the STAR collaboration. The amplitudes of the ${\rm cos(2\Delta\Phi)}$ modulation in the cross-section and the extracted radii depend on the nuclear geometry. This dependence is dominated by the geometry-dependent variation of the minimum impact parameter required for ultra-peripheral collisions.