I'd like to add a little to Steve B's answer, which is the one I can best understand so far. His crucial point is that "tapering" (smooth refractive index variation) makes no difference to the deflexion - it's only the beginning and ending values of the refractive index throughout the deflexion that determine the change in angle. To convince yourself of this, think of a ray running through a stack of thin, homogeneous layers and apply Snell's law to each. You'll find that all the "intermediate" indices and refraction angles cancel out.
Now I think on relativistic theoretical grounds we can fairly confidently say (with a possible loophole) that here on Earth we are not immersed in any refracting medium:
- Recall that $c$, from relativity's standpoint, is no longer primarily thought of as the speed of light, it is simply a parameter with dimensions of speed that defines what reality is in a whole class (as $c$ ranges over $[0,\infty)$) of possible relativities that are all compatible with the postulates of special relativity and which generalize the Galilean transformation (which is simply what we get as $c\to\infty$ and Galilean relativity is indeed compatible with the basic symmetry and group theoretic postulates of special relativity). That sentence is a bit "dense" so see my answer here for more info. So we now need to find out by experiment what value of $c$ defines our Universe - which of this whole family of relativities we actually live in, if indeed $c$ is finite (infinite $c$ yields Galilean relativity);
- From this modern standpoint, therefore, we do not identity $c$ by measuring the speed of light in a vacuum, we instead look for examples of velocities which transform under switches between inertial frames in the peculiar way that this group theoretic formulation of relativity foretells if the parameter $c$ is finite. This behavior is glaringly obviously different from the Galilean transformation. So it's not about its value, nor what it is associated with. It's about finding the kind of weird frame independence that we found in the Michelson-Morley experiment. Light shows exactly this behavior. One of the things that SR foretells is that it is only and precisely the class of massless particles that have this behavior. So now we interpret the MM experiment as yielding two independent results thus: Result 1: we've (i) found the weird, finite $c$ behavior shown by the speed of light, (ii) SR foretells that there can only be one such $c$ so now we know that our universe has a finite $c$ and it is not the special case of Galilean relativity that holds. Result 2: Light travels at this speed $c$ and so light must be propagated by a massless particle. Note that, would the MM experiment have turned out differently today, we'd interpret it not as telling against SR, but as yielding mass bounds on light.
Moreover, any kind of "medium" breaks the symmetries that underly special relativity, unless the medium does not interact with the massless particles we are using as our yardstick to measure $c$ with. Velocities of light in optical mediums transform under boosts quite differently from $c$. The matter of the medium throws up a "preferred" reference frame because the atoms / molecules in it "hold on to the light" for small time intervals in a process of cyclic absorption and re-emission that begets a "slowing of light" and thus an effective refractive index. The delays are nonzero and thus are dilated under boosts, leading to a dependence on frame of the refractive index.
Simply put, were we in any medium that interacts with light, we would almost certainly know it were there through Michelson-Morely kinds of experiments. We would get a non null result to the MM experiment i.e. the Earth's motion would affect the measured lightspeed, but we still wouldn't see the Galilean relativity that Michelson and Moreley foresaw. We'd see something in between, and we could even get good estimates on $c$ and how much greater than the speed of light it were. So we can be fairly sure that dark matter, if we're in it, has a refractive index of 1 and, if it does interact with light and the EM field, it does so in a weird and subtle way which we haven't yet worked out how to detect.
On Copernican grounds, we can argue that it is unlikely that the deep space between stars has a nonunity refractive index if, as we know, the space around our Earth almost certainly does not. It would be pretty weird if our little patch of space were so special that only here can lightspeed assume the universal constant $c$. So both here and in deep space, we have good reason to assume $n=1$ and that dark matter, if it is there, does not change this fact.
This leaves the last possibility of a layer towards the Solar System's edge of nonunity refractive index between us and the far stars. Okay, so now we return to Steve B's answer. This means that there must be no angular deflexion as in your "sunset" effect. There could be, though, a sideways shift of a ray through the medium depending on direction (see my drawing). Now, our galaxy is rather thick compared with the Solar System, so the sideways shift wouldn't make any difference to our ability to see stars. The source I could dig up The cosmoquest.org forum tells me that the angle between the Solar and Galactic ecliptic is about 60 degrees, hence my drawing.
Let's suppose we have a star that is such that its rays pass through the layer unshifted at some time in the year. Six months later there is a sideways deflexion of the ray $d$. If the medium were perfectly unchanging and were thick enough, we would see the inclination $\theta_s(t)$ of the star change with time $t$ in a way that would not agree with a simple triangulation without a within-year-varying horizontal shift $d(t)$. Snell's law within the medium would show itself. The effect would be subtle, but eventually by fitting inclination versus time models assuming simple triangulation to many stars, we'd see deviations that couldn't be explained any other way. In contrast, simple triangulation models do give accurate fits to the observed $\theta_s(t)$. Moreover, depending on what the medium is, its distribution might shift randomly with time, giving further distortions that we would detect in the end.
So it's not altogether impossible that we live in a light-interacting medium, but its highly unlikely we are immersed in it, an extremely good guess that the distant stars are not immersed in it and, if there are light-interacting medium at the edge of the Solar System, it's likely to be either very thin or with a refractive index very, very near to 1.
Update for Questions:
Wait a minute. It occurs to me that for what I proposed, my refractive index numbers were way off in my calculators. I gave the extra solar medium an index of 1000000 thinking this would speed up light. But isn't that backwards? Wouldn't my index be less than one in order to account for faster light? What should the index be for a faster speed? A number smaller than one but greater than zero, or a negative number? Neither seem to work in the calculators.
The only way that the magnitude of the refractive index's real part can be less than one is within the narrow wavelength bands of so-called anomalous dispersion that happen for some materials. This means that the phase velocity is more than $c$ in these narrow bands. The signal propagation velocity on the other hand cannot be greater than $c$ without violating special relativity. See the "Group and Phase Velocity" section of the "Dispersion" Wikipedia page Anomalous dispersion, with its drastic dependence on wavelength, is really noticeable. We'd therefore see only very narrow parts of the spectrum of one star owing to the frequency dependence of $d(f)$. Negative real-part refractive index with magnitude more than one can also happen, but, as far as we know, it does not arise in nature and only arises in sophisticated metamaterials. Of course, maybe dark matter has this behaviour too. But even so, it shouldn't change my or Steve B's arguments. In a negative index material rays stay on the same side of the unit normal, so it's pretty bizzare indeed. see the "Negative Index Metamaterials" Wikipedia page.