To properly answer this, one first needs to define how the observer is "moving" or "not moving" relative to the black hole and a "fixed" coordinate system. Let's first assume that the observer is, in a practical sense, "stationary," as in is somehow able to fix their position in space so that the black hole's singularity is not moving relative to them, nor are outside objects. This would require a force by you to counter gravity. Because once you're inside the event horizon this is impossible (since your counter-velocity would need to be faster than light), we're just using this concept as a thought experiment for the moment.
With that, an observer "falling into a black hole" will not actually notice much of a difference, even once they pass the event horizon. Assuming there's no accretion disk (so we're not worried about superheated gas and dust blocking your view), your view looking out on the universe would be fairly unchanged from what you may have thought initially.
In general, there will be two effects. First, objects would be generally blueshifted since the black hole's gravity is going to be pulling the light; again, this is only if you are "standing still." Second, you will get visual distortions due to the gravity, with features appearing somewhat squished as though you're looking through a concave lens.
Now let's say you're actually falling into the black hole and you are past the event horizon. Since you are now moving along a free-fall geodesic, the light from other objects would no longer be blue-shifted because you would be in the same inertial reference frame. As you get closer to the singularity, the outside universe would appear more and more compressed (lensed). Also, the singularity would grow larger.
That last part actually confused me at first, so I'm copying Andrew's explanation directly: "Imagine living on a sphere (heh). That is, imagine an existence confined strictly to the closed, two-dimensional surface of a sphere. Pick a point to stand on/in. No matter which direction you send out a beam of light, it will eventually cross the geometric point on the exact opposite side of your sphere. So no matter which direction you look in, it's towards that single geometric point. Similarly, every line of sight from a point of view within the event horizon points towards the singularity." So even though it's a point, you're still going to see it no matter where you look.