We present a new framework to study the time variation of fundamental constants in a model-independent way. Model independence implies more free parameters than assumed in previous studies. Using data from atomic clocks based on $^{87}$Sr, $^{171}$Yb$^+$ and $^{133}$Cs, we set bounds on parameters controlling the variation of the fine-structure constant, $\alpha$, and the electron-to-proton mass ratio, $\mu$. We consider variations on timescales ranging from a minute to almost a day. In addition, we use our results to derive some of the tightest limits to date on the parameter space of models of ultralight dark matter and axion-like particles.