Another way to look at it--We actually only "See" in two dimensions. Each of our eyes only has the ability to process xy coordinates, not z (A one-eyed person has no depth perception). Our brain diffs the two 2d images from our eyes to give us a good guess at a z coordinate (Since it's not true 3d perception our brain can be tricked here, hence illusions! Also our 2d vision can be obstructed, true 3d vision would not be).

In the same way as our brain "emulates" 3d vision, it diffs what is going on right now from what went on a minute ago (or a day ago or a year ago) to give you an understanding of the t axis. It is no less valid than our view of the z coordinate our brains construct.

Another way to look at it--We actually only "See" in two dimensions. Each of our eyes only has the ability to process xy coordinates, not z (A one-eyed person has no depth perception). Our brain diffs the two 2d images from our eyes to give us a good guess at a z coordinate (Since it's not true 3d perception our brain can be tricked here, hence illusions! Also our 2d vision can be obstructed, true 3d vision would not be).

In the same way as our brain "emulates" 3d vision, it diffs what is going on right now from what went on a minute ago (or a day ago or a year ago) to give you an understanding of the t axis. It is no less valid than our view of the z coordinate our brains construct, but unlike 3d it's not helpful to visualize the t dimension so our brain doesn't do that, instead it makes the information available in other ways.