The problem is that any statement that a dimensionful physical constant has changed is meaningless if you do not state what it is you are keeping constant. When you do state it, and you work it out in full, it turns out you're measuring the change of a dimensionless physical constant.
Suppose, for example, that I claim "the speed of light has increased by 10% since last year". My experimental evidence for this is simple: I have a ruler and a clock, and every day I use them (the exact same ruler and clock) to measure the speed of light. I then calculate the time $T$ - as a quantity, in seconds, given by my clock - taken by light to cross the ruler, and I divide it into the length $L$ of the ruler. Over a year, this quantity $L/T$ increases by 10%.
For the metrologically minded, let's take a look at my experimental apparatus:
My ruler consists, essentially, of a bunch of atoms in a linear arrangement. For example, I could make my ruler by putting a bunch of hydrogen atoms in a line, separated from each other by one Bohr radius (which can be experimentally determined, at least in principle, as the distance at which the ground state charge density decreases to $e^{-2}$ of the maximum). If my ruler is 1m long, it will have $N=1\:\mathrm m/a_0\approx1.89\times10^{10}$ atoms in it.
Moreover, since it's the same ruler, and because I'm lazy, I don't recalibrate it to the SI every day before I measure (which would have catastrophic consequences for my result!). Instead, what I do before I measure is look at all the bonds to ensure they're still one Bohr radius long, and re-count the atoms to ensure the ruler still has the exact same number $N$ of atoms.
I should also note that, despite sounding esoteric, this ruler is close to the best possible model for an actual physical ruler made of platinum or whatever, since its length is governed by the same physics (nonrelativistic QM plus electrostatics, a.k.a. chemistry) that governs the size of all everyday objects. It's been distilled into a form that's clearly definable, but the essence of the definition is along the lines of "this ruler here, so long as the ends don't get worn out and it doesn't get bent and there's no monkey business with thermal expansion or whatever".
My clock is a caesium clock as per the SI second. That is, it has a bunch of caesium atoms which are placed in a specific state (technically, a coherent superposition of their ground state at and the first hyperfine excited state) which emits microwave radiation. The clock measures this radiation and counts the number of maxima; every 9,192,631,770 cycles it increases the counter by 1 second.
With this apparatus, then, I observe the measured speed of light to change. What does this result mean? The easiest interpretation is simple:
- The speed of light has changed. Light simply travels faster than it did last year. That's pretty mysterious, but then the result is pretty weird too.
However, there are other possible interpretations for the result. For example,
the size of the hydrogen atoms might have changed. That is, the speed of light is still the same, but for some mysterious reason all my hydrogen atoms are 10% bigger than they were. This is not that crazy at all: the Bohr radius is determined by the Schrödinger equation under the electrostatic force, so their constants $\hbar,m_e$ and $e^2$ determine $a_0$ to be $a_0=\hbar^2/m_ee^2$. If any of those changed - say, all electrons are suddenly 10% lighter, which is as mysterious as light being faster - then I'd see exactly the same result I do observe.
It's important to note that this is the reading of my result that would come from a strict application of the current SI system of units, since the SI meter is defined as the distance covered by light in 1s as given by my clock. However, if my ruler has "shrunk" then so have I (as I'm made of atoms) and so has every piece of equipment in my lab and elsewhere on Earth. I could then speak of the "mysterious growth of the SI meter" equally meaningfully.
Alternatively,
- the caesium atoms could be getting progressively more sluggish. They could simply no longer be giving out microwave peaks and troughs as fast as they used to (or as I'd see if I teleported myself to the past), so even though the speed of light is the same, the length of the ruler is the same, and the traversal time is the same, the counter on the clock now reads 10% less seconds than it would have a year ago.
These three explanations are all equally mysterious, or equally reasonable, as each other. Moreover, from my experiment I have no way to test which is right, as I have no way of teleporting myself back to the past to compare my (possibly) engorged hydrogens or my (allegedly) sluggish caesiums with their previous versions, in the same way that I cannot set up a race between my (supposedly) faster light and the light of yesteryear. It is clear that something in physics has indeed changed, but you can pan the change into different factors depending on your perspective.
As it turns out, of course, the thing in physics which has changed, and which is the only thing that can unambiguously be said to have changed, is the fine structure constant. This $\alpha$, as any atomic physicist will tell you, is one over the speed of light in atomic units, which is exactly what we're measuring. More specifically,
$$\alpha=\frac{e^2}{\hbar c},$$
and $e^2/\hbar$ is easily seen to be the atomic unit of velocity. This abstract 'atomic unit' is of course a very physical quantity: it is, within a constant, calculable factor, the mean velocity of any electron around its atom.
Now, as it happens, this crazy experiment I proposed is indeed being performed. The actual experimental realization does not rely on meter-long rulers or on external clocks, but it relies instead on the natural length and time scales of ytterbium ions, which of course are completely determined by the same length and time scales as all atomic physics.
Atoms, it turns out, have natural built-in velocimeters for measuring the speed of light: their kinetic energies $p^2/2m$ change by relativistic corrections on the order of $p^4/8m^3c^2$, and the different electronic and spin currents have magnetic interactions on the order of $p/c$. Different states respond differently, and even in different directions, to these perturbations, so it's possible to monitor $c$ by observing the precise location of the different energy levels. (For more information, see NPL | Physics | PRL | arXiv.)
There is, so far, no observed change in $\alpha$. But if there is, we simply won't be able to tell whether a change in $\alpha=e^2/\hbar c$ is because of a change in the speed of light $c$, the size $\hbar$ of the fundamental phase-space cell, or the strength $e^2$ of the electrostatic interaction, or of the finely-tuned joint changes of these constant that would implement the three implementations stated above. Those individual changes have no meaning by themselves.
Finally, some useful references for further reading are
How fundamental are fundamental constants? M.J. Duff, Contemp. Phys. 56 no. 1, 35-47 (2014), arXiv:1412.2040,
which supersedes arXiv:hep-th/0208093 (Comment on time-variation of fundamental constants, 2002), and
Trialogue on the number of fundamental constants. M.J. Duff, L.B. Okun and G. Veneziano, J. High Energy Phys. 03 (2002) 023, arXiv:physics/0110060.