In this old answer of Steve Allen's, he quotes this nice passage
Imagine for a moment what would happen if, just as a practical joke, someone found a way to stop all atomic clocks, just for a short time. This would cause such a tremendous disturbance in world affairs that the loss of TAI would be a totally insignificant matter! Furthermore, when it came to setting it up again, the phase of TAI could be retrieved to within a few tenths of a microsecond by observations of rapidly rotating pulsars...
-- Claude Audoin & Bernard Guinot, p. 252, sec. 7.3.1 of "The Measurement of Time: Time, Frequency and the Atomic Clock", Cambridge University Press, 2001
I'd like to explore this scenario in a bit more detail. Specifically, suppose that our practical joker (let's call him Richard for definiteness's sake) managed to stop all running time standards for a well-defined time $\tau$ and then set them running again. Richard then challenges us to determine $\tau$. To keep things simple, Richard has agreed to provide, if we ask him nicely, an upper bound $T$ on $|\tau|$ - but, again, he challenges us to accept the largest, loosest bound that we can.
As Audoin and Guinot point out, you can use pulsar observations to constrain $\tau$. Suppose you can observe a pulsar that oscillates regularly at a period $T_1$ of 10s. Then, if you know that $|\tau|<T_1/2$, you can match the phase of the observed oscillations with the historical record, you can determine $\tau$ to the same precision to which you know $T_1$.
If $\tau$ is not smaller than $T_1$, of course, this won't work, as you won't know how many periods will have elapsed. However, if you have more than one pulsar, you can significantly extend the range of $\tau$ that you'd be able to pin down. Indeed, if pulsar 2 has a period $T_2$ of 11s, then it goes in and out of phase with pulsar 1 over a period of 110s, so we can recover $\tau$ if we're guaranteed that $|\tau|<$55s. Similarly, if you add a third pulsar with period 13s, the range goes up to (13x11x10)/2 s=715s - much longer than the individual periods of the three pulsars.
In real life, of course, we know a good many pulsars, so you could do quite a bit with their signals to extend the range of $\tau$s that we can recover from. (On the other hand, they do oscillate very rapidly, so even if the skip is thousands of periods long you're only on the few-second regime. But then their periods can be known to high accuracy, which lengthens the period of collective oscillations between two or more pulsars - i.e. it lengthens their effective LCM once you factor in uncertainties in the periods.) Also, as Chris White points out, pulsars can have glitches, which poses a new problem of its own. (But then, you'd need to have glitches in a substantial fraction of known pulsars to really compromise your ability to recover.) And presumably there's more things I haven't considered (what are they?).
So, my question is: using currently known data on observed pulsars and their periods, what's the longest $\boldsymbol\tau$ from which we could recover? Is it long enough that we could use longer-period sources like the period decay of neutron binaries to get further fixes on $\tau$? With what sort of accuracy could we recover $\tau$?
If this has already been explored in the literature then I'm happy to take a reference, but otherwise I'm looking for detailed and in-depth treatments that teach me lots of physics. I'll chip in some rep carrots to spice things up in a few days.
Also, to be clear - answers need not exclusively use pulsars. If there is some other method that can constrain $\tau$, using either other types of astronomical sources or using earthbound phenomena and experiments, then I'll be happy to learn about those!