However, this is only one perspective, and changing $\alpha$ will change other realms of physics differently. In deeply relativistic quantum regimes, for example, it makes much more sense to make $\hbar$ and $c$ the benchholders of your physics, and you switch from atomic units to natural units; here $\alpha$ is more meaningful as a measure of the Coulomb constant $e^2/4\pi/\varepsilon_0$$e^2/4\pi\varepsilon_0$, which determines the strength of the electromagnetic interaction between two elementary charges.
This is a different perspective, one among many others, and; they are all equally valid, and for each a change in $\alpha$ will affect physics in different ways. For all of them you need to choose some dimensionless ratio as a hallmark that physics has changed - but the reference quantity in that ratio will be different, depending on what physics you're exploring, so you always need to say what it is you're keeping constant.