Is the orientation of rotation described by the Terrell-Penrose effect dependent upon the direction of travel? (2D for simplicity) The title, I'm sure, is confusing, so let me try to elucidate what I mean.
Imagine the 2D scenario pictured below, where a rocket traveling only with an x-component at a constant velocity close to the speed of light in the reference frame of an observer on Earth is heading towards a box falling from the sky above it perpendicular to the rocket (at a constant velocity much less than the speed of light).

It is known by the Terrell-Penrose effect that this box will appear to an observer on the rocket rotationally distorted rather than contracted in length (in short, the length contraction predicted by SR "disappears" since light has a finite speed). But what is the orientation of this rotation? This is the heart of my question.
My thought is to use the Lorentz transformation equations in order to see what things look like from the perspective of the rocket. Taking Earth as the stationary reference frame $S$ and the rocket as moving reference frame $S'$ with velocity $v$ close to $c$, we get
$$\Delta t' = \gamma(\Delta t-\frac{v}{c^2}\Delta x)$$
Now take a snapshot at $t=0$, which we will define as the time at which the box crosses the gridline below it (assume the rocket is still to the right of the box at $t$). In the Earth reference frame, both the left side and right side of the box (points A and B) cross this line simultaneously; in other words, $\Delta t=0$. But this is not true in the reference frame of the rocket due to the effects of SR. We have some difference $\Delta t'$ between points A and B crossing the line, so the box will appear to be rotated since one of the sides crosses before the other. Letting L be the length of the box, we see:
$$\Delta t' = \gamma(\Delta t-\frac{v}{c^2}L) = -\gamma\frac{v}{c^2}L \neq 0$$
Now which point crosses first in the reference frame of the rocket? Is it not always the point closest to it, no matter which way the rocket is traveling, since light travels at an equal speed $c$ from both ends of the box, so the closest side's light will always reach the rocket first? Wouldn't the box always be rotated in the reference frame of the rocket in the orientations shown below? I thought this would be a simple consequence of the fact that time dilation is not dependent on the direction of relative motion.

I'm not sure exactly where I'm going wrong with my logic here. I was marked off on an exam on a similar problem for this way of thinking, so I know I am wrong, but where? (No feedback was given other than marking me down.)
Any help would be great. I honestly am just really fascinated by this phenomenon and want to know the correct way of thinking about it. I think this is a really important and fundamental topic that I need to know well for future studies. Thanks!
 A: You seem to have a very fundamental misunderstanding of an observer in special relativity. In special relativity, an observer should not be thought of as an actual person located at one point in space. Rather, an observer is something that is everywhere and "instantaneously" knows all events at all points in spacetime.
The word "observe" has this precise meaning in special relativity. What one observes does not include the travel time of light to an actual person. What you observe is not what you actually see. For example, the apparent speed of jets can be faster than light (in fact, up to several times faster) because of the travel time of light from the jet to Earth. Their actual speed is, of course, slower than light. Therefore, it would be incorrect to say that they are observed to be faster than light.
Likewise, the Penrose-Terrell effect arises because of the travel time of light to the eyes of a person. The Lorentz transformations, being relationships between different observations, do not predict this effect. Furthermore, there is no need to use them because everything is computed in one frame. You just need to compare the paths of light from different corners of the object. Length contraction doesn't matter and can be included independently afterwards if you wish.
A: The easiest way, by far, to think about Penrose-Terrell rotation is in the rest frame of the object being looked at, not the rest frame of the rocket.
In that frame, light from the object fills spacetime with a static light field. What you see at a spacetime point is determined only by your $x,y,z$ coordinates in that frame. Moreover, it's just what your nonrelativistic spatial intuition tells you that you should see. For example, if the object is a six-sided die, and you are located somewhere on a line through opposite corners of the die, your intuition is that you should see something like this:

and that intuition is correct. The only effect of your speed is to distort this two-dimensional, projected image by Doppler shift and aberration. The image may be redder and larger or bluer and smaller, and the line segments may be distorted to circular arcs, but you'll see the same basic arrangement of lines and pips.
In your example, supposing your rocket ship passes below the box, you will see the bottom and right of the box for half of the trip, then briefly just the bottom (very briefly considering the speed), then the bottom and left.
When you pass below the box, the box will appear, due to aberration, to be still far in front of you, not at a right angle to your motion. When, later, it appears to be at a right angle to your motion, you actually passed it some time ago, so you'll see the bottom and left of it. That's the Penrose-Terrell "rotation". It's really an optical illusion. If you start at rest (in the earth frame) far to the right of the box, accelerate to near-light speed, then decelerate to a stop after the flyby at a similar distance to the left, then the time at which you can only see the bottom of the box will be around the halfway point of the trip by your own clock. Only the apparent position of the box in the sky might lead you to believe that you aren't passing it at that time.
A: First of all, the rotation you are depicting in your question is not the terrell penrose effect. You are describing the "rotation" of the box along the axis coming out of the screen. The terrell penrose effect would have the box "apparently" rotating along an axis parallel to y axis ( and a miniscule , negligible rotation along an axis parallel to x axis , as box is falling at speed way less than c) , when " seen " by an observer in the frame of the rocket.

Now which point crosses first in the reference frame of the rocket? Is
it not always the point closest to it, no matter which way the rocket
is traveling, since light travels at an equal speed c from both ends
of the box, so the closest side's light will always reach the rocket
first?

No, which point will cross first depends on which way the rocket is travelling. You might need to brush up on relativity of simultaneity. "Leading clocks lag" . Which end of the box is the leading clock depends on direction of motion of the spacecraft.
Wouldn't the box always be rotated in the reference frame of the rocket in the orientations shown below? I thought this would be a simple consequence of the fact that time dilation is not dependent on the direction of relative motion.
