We know that a rainbow occurs due to the refraction and reflection of light inside raindrops. We also know that inside spherical raindrops total internal reflection is not possible. So some light is always refracted straight out of the raindrop.
So doesn't that mean we should be able to observe the rainbow if we looked directly facing the sun as well? I have attached a diagram to clarify my idea.
TL;DR: A rainbow is only visible due to the focusing effects of raindrops, and rays do not get focused unless they reflect internally at least once.
The key point that gets lost in most introductory-level discussions of rainbows is that it's not just that different colors get bent by different angles in the drops. It's also that there's a particular angle where rays of a given angle get concentrated. For an illustration of this, see Emilio Pisanty's excellent answer to this question, and Ruslan's fascinating ray-traced renderings of rainbow droplets in this Q&A.
Specifically, to see a definite color at a definite angle, you need the rays of that color hitting a particular droplet to be concentrated at that angle. I'll crib Emilio's diagram here to illustrate this:
In particular, let $\theta$ be the angle of incidence of a light ray hitting a droplet. In the diagram above, this angle goes from $\theta = 0$ near the center of the sphere up to $\theta \approx 90^\circ$ for rays hitting near the top of the sphere. Let $\delta$ be the total angle a given light ray is deflected before it exits the sphere. In the diagram above, the fact that $\delta$ has an extremum when viewed as a function of $\theta$ leads to the fact that a whole bunch of light rays of that color get concentrated there.
To calculate how this works, we can apply Snell's Law to find $\delta$ as a function of $\theta$. If we go through the geometry, we find that the total deflection for a ray that is not internally reflected is $$ \delta = 2 \theta - 2 \sin^{-1} \left( \frac{\sin \theta}{n} \right). $$ (To see this, note that the complicated inverse-sine expression in brackets is just the angle of refraction $\phi$ inside the sphere, and the ray gets deflected by an angle of $\theta - \phi$ in each of its two refractions.) With some basic calculus, it can be shown (it's a good exercise — try it out!) that this function never has a maximum or minimum for any $\theta$ so long as $n > 1$. So the rays of a particular color don't get concentrated at any particular angle, and they get washed out, as the OP of this question was concerned about.
However, if we do the same thing for a ray that is reflected once internally, we find that $$ \delta = 2 \pi + 2 \theta - 4 \sin^{-1} \left( \frac{\sin \theta}{n} \right) $$ and this function does have a minimum when $\theta$ satisfies $n^2 = 1 + 3 \cos^2 \theta$.1 This means that we get a concentration of rays at a particular color if there's an internal reflection, and we can see a rainbow in that direction. The fact that the position of the maximum depends on $n$, coupled with the fact that $n$ varies slightly for water over the visual spectrum, means that we end up with different colors getting concentrated at different angles, and so we see a rainbow.
1 More generally, if there are $m$ internal reflections, $\delta$ will be minimized when $\theta$ satisfies $n^2 = 1 + m(m+2) \cos^2 \theta$.
Yes, but they're difficult to observe, and they're not as simple as your diagram.
As Michael Seifert points out, the no-reflections path does not produce a rainbow; however, some light will reflect more than once internally, and this can produce additional rainbows. You may have personally seen a double rainbow; this is produced by light that reflects twice before exiting the water droplets, and while it's dimmer than the main rainbow it's still at least partially visible to the naked eye under good conditions. Further reflections produce successively fainter rainbows, and both the third- and fourth-order rainbows are within about 45 degrees of the sun (source: https://en.wikipedia.org/wiki/Rainbow#Higher-order_rainbows). It takes near-perfect conditions to see these rainbows against the glare of looking towards the sun, but it is possible and there have been reputable sightings.
