A lot of textbooks and exam boards claim that light incident at exactly the critical angle is transmitted along the media boundary (i.e. at right-angles to the normal), but this seems to violate the principle of reversibility in classical physics. How would a photon or ray travelling in the reverse direction "know" when to enter the higher refracting medium? It can't know, so I conclude that such light is simply reflected?
Is this correct?
When one is exactly at the critical angle, the light behaves in a way that may be interpreted as "something in between" refraction and reflection: it continues in a direction that is tangent to the boundary of the mediums.
When the angle is smaller than the critical angle, we get refraction. At the critical angle, $\theta_2$ of the refraction becomes 90 degrees, so we get the tangent propagation. At angles larger than the critical ones, there is a discontinuity: the equation for $\theta_2$ (arcsine of something) has no solutions which is why we get a total internal reflection.
There is nothing about these facts that would contradict reversibility or time-reversal symmetry of the laws of physics. If we time reverse the behavior at the critical angle, it indeed looks like the light must "randomly pick" a moment at which it enters the medium with the higher refraction index and there isn't any unique way to pick the preferred moment.
But that's not a problem because the probability that the direction of light is "exactly" tangent to the boundary is zero. In a real-world situation, the light will be a superposition of beams with angles $\theta_2=\pi/2-\epsilon$ for various small values of $\epsilon$, and for any nonzero $\epsilon$, the light will know very well when it hits the boundary. So your problem only occurs at a negligible, "measure zero" portion of the situation, so it is at most a "measure zero" problem. When one adds the appropriate degree of realism and specifies the precise angles and deviations from the "idealized model", the problem goes away.
It's more correct to say that light rays split into reflected and refracted subcomponents at boundaries, rather than reflect or refract. Figuring this out is done by rigorously looking at the boundary conditions of the electric and magnetic fields at the interface. The result ends up being polarization-dependent, and is known as Fresnel's equations.
Per Fresnel's equations, as expected, some of the light is transmitted at the boundary, and some is reflected at the boundary, even below the critical angle. At the point that you hit the critical angle and the light ray "goes horizontal", the transmission coefficient goes down to zero. So, while there is a "solution" that moves horizontally, exactly 0% of the incident light ray will actually be doing this. The reflection coefficient hits 100% exactly at the critical angle.
More rigorously, consider that an analysis of the boundary conditions at the interface requires that the two polarizations of light have reflectances:
$$R_{s} = \left(\frac{n_{1}\cos\theta_{i} -n_{2}\cos\theta_{t}}{n_{1}\cos\theta_{i}+n_{2}\cos\theta_{t}}\right)^{2}$$
$$R_{p} = \left(\frac{n_{1}\cos\theta_{t} -n_{2}\cos\theta_{i}}{n_{1}\cos\theta_{t}+n_{2}\cos\theta_{i}}\right)^{2}$$
Then, consider that we're at the critical angle, which tells us that $\theta_{t} = \pi/2$ and $n_{2} = n_{1}\sin\theta_{i}$ (I henceforth remove the $i$ subscript). Then, we have:
$$R_{s} = \left(\frac{n_{1}\cos\theta-0}{n_{1}\cos\theta+0}\right)^{2}=1$$
and
$$R_{p} = \left(\frac{0-n_{2}\cos\theta}{0+n_{2}\cos\theta}\right)^{2}=1$$
By conservation of energy, this means that the transmission coefficients are therefore both zero, and no light is refracted at this angle.
While the existing answers are not wrong, they brush over a good deal of relevant detail and it wouldn't come amiss to set the record straight on a few counts.
In particular, as regards time-reversibility, this diagram is hugely misleading:
Why is it misleading? Well, because it assumes that you can work in the ray approximation, and this fails when you're at the critical angle. All optics is wave optics, and that reduces to ray optics in many situations, but this isn't quite one of them.
So, let's back out a little and get back to the wave-optics view of reflection and refraction, which is normally known as the Fresnel equations formalism. Here, what we try to do is to set up a boundary between two media, and 'shoot' from below with an incoming plane wave, $$ \mathbf E_\mathrm{in}(\mathbf r,t) = \mathrm{Re}\mathopen{}\left( \mathbf E_{0,\mathrm{in}} e^{i(\mathbf k_\mathrm{in}\cdot \mathbf r- \omega t)}\right)\mathclose{}, $$ and then we look for a corresponding set of reflected and transmitted waves, $\mathbf E_\mathrm{re}(\mathbf r,t)$ and $\mathbf E_\mathrm{tr}(\mathbf r,t)$, such that the combined fields ($\mathbf E_\mathrm{in}(\mathbf r,t)+\mathbf E_\mathrm{re}(\mathbf r,t)$ in medium $1$ and $\mathbf E_\mathrm{tr}(\mathbf r,t)$ in medium $2$) will be a solution to the Maxwell equations.
Now, the gory details are provided in plenty of textbooks, so I won't repeat them here (but see e.g. §7.3 in Jackson for more details). It boils down to this, though:
So, what's the deal with total internal reflection? Well, when we say that $$k_{z,\mathrm{tr}}^2 +k_x^2 = \frac{n_2^2}{n_1^2}\left(k_x^2+k_z^2\right),$$ if $k_x$ is large enough with respect to $k_z$, and $n_2$ is too small with respect to $n_1$, there's a chance that the right-hand side will be smaller than $k_x$, and the only way to accommodate that is to have a negative $k_{z,\mathrm{tr}}^2$. Does this mean that we need to throw the whole formalism out the window? Not really - it just means that on the other side of a total internal reflection we will find an evanescent wave, which decays exponentially away from the boundary.
Thus, if you write $$ k_{z,\mathrm{tr}} = \sqrt{\frac{n_2^2-n_1^2}{n_1^2}k_x^2+\frac{n_2^2}{n_1^2}k_{z,\mathrm{in}}^2} = i\sqrt{\frac{n_1^2-n_2^2}{n_1^2}k_x^2-\frac{n_2^2}{n_1^2}k_{z,\mathrm{in}}^2} = i\kappa_{z}, $$ and it turns out that you get an imaginary $k_{z,\mathrm{tr}}$, then the transmitted field reads $$ \mathbf E_\mathrm{tr}(\mathbf r,t) = \mathrm{Re}\mathopen{}\left( \mathbf E_{0,\mathrm{tr}} e^{-\kappa_{z} z}e^{i(k_x x- \omega t)}\right)\mathclose{}, $$ where what used to be an oscillatory $e^{ik_z z}$ has turned into the decaying exponential $e^{-\kappa_{z} z}$.
This is best displayed graphically: the animations below show the incident, reflected, transmitted, and total fields, over the course of a cycle, both for normal refraction,
and for total internal reflection:
Note, in particular, that the field does not stop sharply at the boundary, but that it extends a little bit beyond, with an smooth decay to zero away from the border.
So, what happens exactly at the critical angle? This is where it gets fun: the critical angle is the point strictly between positive and imaginary in $k_{z,\mathrm{tr}}$, and that turns out to be exactly zero: that is, the transmitted field has a zero $k_z$ component, $$ \mathbf E_\mathrm{tr}(\mathbf r,t) = \mathrm{Re}\mathopen{}\left( \mathbf E_{0,\mathrm{tr}} e^{i(k_x x- \omega t)}\right)\mathclose{}, $$ and this means that it neither oscillates nor decays away from the boundary, and it fills all of the $n_2$ space with a plane wave propagating strictly parallel to the boundary between the media:
This feels highly counter-intuitive at first, because this solution calls into being this infinite amount of energy that's not even flowing away from the boundary. However, it's important to keep in mind that, because we started our calculation with a plane wave of infinite energy, we decidedly stepped into unphysical territory right from the beginning, and we have no right to complain about these kinds of features.
This is also why the initial diagram is wrong: in proper plane-wave refraction at the critical angle, you don't have a single beam striking the boundary at a single place; instead, you have a plane wave inciding on the entire boundary at all times, interfacing with a (non-)evanescent wave on the other side. Thus, if you stop time and make it go backwards, the incident beam will turn into a plane wave that does not 'originate' from anywhere in the beam. (And, also: to really say that you've 'turned time backwards', you also need to supply the time-reversed version of the reflected beam, which supplies all of the energy flux that gets transferred to the time-reversed incident beam.)
OK, so that's what plane-wave optics has to say, and it's pretty ugly and not very physical at all. How does this connect with reality, and why are we allowed to use a formalism that produces so many unusable answers?
This comes down to the foundations for ray optics, which we can build out of plane-wave optics by considering a spread over wavevectors, of the form $$ \mathbf E(\mathbf r,t) = \int f(\theta,\phi) \mathbf E_\mathbf{k} e^{i(\mathbf k\cdot\mathbf r-\omega t)} \mathrm d\Omega, $$ where $f(\theta,\phi)$ is a distribution function that's tightly bunched around the intended incidence angle $\theta=\theta_\mathrm{in}$. This is what Luboš means when he says that this is "not a problem because the probability that the direction of light is "exactly" tangent to the boundary is zero": we're integrating over a continuous spread of $\theta$s, and we can drop individual points without affecting the value of the integral. This is what happens to the points at exactly $\theta=\theta_c$: they happen in a set of measure zero w.r.t. the integration, so we can pretty much just ignore what happens with them.
And finally, to come back to the question: what happens in a physical experiment, if you shine a beam at exactly the critical angle, i.e. something like this,
Image source: Getty, obviously. I couldn't find one without a watermark. If you do, please tell me.
and then you try and reverse time? Here the image makes it clear that the integral above is what's going on: if the center of the beam is at exactly the critical angle, then half of its energy is in modes at higher angles (which completely reflect internally), and half of its energy is at angles that are almost-but-not-quite at the critical angle, and these come away at a finite transmission angle away from $90$°. This must be the case: the only way to step away from that is to come in with a mode with an infinite transverse spread at the boundary; this is the wave version of the uncertainty principle at work.
And, if you want to time-reverse this, then you fall back into a linear combination of the two easier cases: you need to provide a beam at grazing-but-nonzero incidence from the transmission side, which has a well-defined incidence point, and everything is fine. (And, similarly, you also need to provide the time-reversed version of the reflected beam, and this is what will be putting in most of the energy into the time-reversed incident beam. If not, then you will get a reflection of the time-reversed transmitted beam, which is completely absent in the non-time-reversed version.)
Mathematica code for the animations available
through Import["http://halirutan.github.io/Mathematica-SE-Tools/decode.m"]["https://i.sstatic.net/jZ55H.png"]
