It is all in what you want to describe mathematically. You can have an N dimensional space and yes, you could "visualize" the analogue of two dimensions going into three. These are Euclidean spaces , i.e. the metric is ds*2=dx*2+dy*2+dz*2+.... up to N terms.
Time is the fourth dimension in current physics because we are attempting to describe and predict motions and interactions of matter and light in a mathematical manner, and the equations are such that they simplify when time is assumed to be the fourth dimension in what is called a pseudo Euclidean space., and in our case dt**2 has a negative sign.
It is what the physics comes out with that makes time the fourth dimension.