It's actually an ingenious, but relatively simple bit of physics and engineering.
It works by compensating for the linear thermal expansion of the pendulum rod, utilizing the thermal expansion of the mercury but in the opposite direction and thus preserving the position of the center of gravity.
Note that the period of the (compound) pendulum is given by $$T=2\pi\sqrt{\frac{I}{mgL}}$$ where $L$ is the distance to the center of mass of the rod from its pivot point, and $I$ is its moment of inertia also about the pivot point. This means the period varies if this distance varies.
With the adding of mercury tubes, as the temperature rises for example, this will cause expansion in the pendulum rod downward $^1$. But at the same time, this temperature rise causes the expansion of the mercury in the tube that moves the mercury upward.
The exact opposite effect happens for temperature drops. That is, the pendulum arm decreases its length, so that $L$ decreases, while the mercury in the tube decreases its height in the other direction.
The net result is that the arm keeps a constant location of the center of gravity, and therefore keeps a constant period.
$^1$ The rod is fixed to a pivot at the opposite end, so even though the whole thing may expand, it's the change in the distance $L$ from the pivot that's important.