Seismic studies show two antipodal regions of lower shear velocity at the core-mantle boundary (CMB) called Large Low Shear Velocity Provinces (LLSVPs). They are thought to be thermally and chemically distinct, and therefore might have a different density and viscosity than the ambient mantle. Employing a composite rheology, using both diffusion and dislocation creep, we investigate the influence of grain size evolution on the dynamics of thermo-chemical piles in evolutionary geodynamic models. We consider a primordial layer and a time-dependent basalt production at the surface to dynamically form the present-day chemical heterogeneities, similar to earlier studies, e.g., by Nakagawa and Tackley (2014). We perform a parameter study which includes different densities and viscosities of the imposed primordial layer. Further, we test the influence of yield stress and parameters of the grain size evolution equation on the dynamics of piles and their interaction with the ambient mantle. Our results show that, relative to the ambient mantle, grain size is higher inside the piles, but due to the large temperature at the CMB, the viscosity is not remarkably different from ambient mantle viscosity. We further find, that although the average viscosity of the detected piles is buffered by both grain size and temperature, grain size dominates the viscosity development. However, depending on the convection regime, in the ambient mantle, viscosity can be dominated by temperature. All pile properties, except for temperature, show a self-regulating behaviour: although grain size, density and viscosity decrease when downwellings or overturns occur, these properties quickly recover and return to values prior to the downwelling. We compute the necessary recovery time and find, that it takes approximately 400 Myr for the properties to recover after a resurfacing event. Extrapolating to Earth-values, we estimate a much smaller recovery time. We observe that dynamic recrystallisation counteracts grain growth in the piles when the lithosphere is weakened and forms downwellings. Venus-type resurfacing episodes reduce the grain size in piles and ambient mantle to few millimetres. More continuous mobile-lid type downwellings limit the grain size to a centimetre. Consequently, we find that grain size-dependent viscosity does not increase the resistance of thermo-chemical piles to downgoing slabs. Mostly, piles deform in grain size- sensitive diffusion creep but they are not stiff enough to counteract the force of downwellings. Hence, we conclude that the location of subduction zones could be responsible for the location and stability of the thermo-chemical piles of the Earth because of dynamic recrystallisation. [ABSTRACT FROM AUTHOR]