Ab initio simulations of a range of interferometric experiments are used to identify a strong dependence on multiphoton phase shifts in above-threshold ionization. A simple rule of thumb for interaction phase shifts is derived to explain both the conservation of photoelectron yield and its absolute CEP-dependence. For instance, it is found that interferometric above-threshold ionization experiments are shifted by $\pi/4$ relative to RABBIT experiments, and that there is no RABBIT-term in a laser-assisted photoionization experiment with odd and even harmonics. Thus, our work helps to resolve the issues of interpretation of quantum dynamics in attosecond and free-electron laser sciences.