Does the $\rm s$ squared in acceleration $\rm m/s^2$ have a geometrical interpretation? Is there a geometric interpretation of $\rm s^2$ in $\rm m/s^2$ as an actual square? Is it right to see the time forming a square like there is an actual square of side 2 in 2 $\rm m^2$?
If that's wrong, Why is that an invalid interpretation?
 A: In SI units, no, but in geometrized units like we often use in general relativity there is a geometric interpretation of acceleration.
Specifically, in geometrized units $L/T^2 \rightarrow L/L^2 = 1/L$. Geometrically this gives a curvature which has units of inverse length. So the acceleration of an object, interpreted geometrically, is the curvature of that object's worldline in spacetime.
Note, this does not give a geometric interpretation specifically of the $T^2$, but only of the overall acceleration. It is generally not fruitful to pick apart units like that and ask about the significance of part of a unit in the context of a whole unit. The $L$ in $L/T^2$ also does not have an interpretation as a length by itself since any characteristic length will also require specification of time.
A: The unit $\mathrm{m/s^2}$ represents metres-per-second added/removed per second, so $$\mathrm{\left[\frac{m}{s^2}\right]=\left[\frac{m}{s}/s\right]}.$$
Now, in classical non-relativistic mechanics space is three-dimensional while time is one-dimensional. With space being three-dimensional, we can see 1D, 2D og 3D space-property combinations. Area, such as $A=L\cdot B$, is a 2D space-property where the 1D lengths $L$ and $B$ are measured along two perpendicular spatial dimensions. The derived area unit $\mathrm{m^2}$ thus denotes the size of a span out into a 2D plane of spatial geometry.
With time we have just one dimension to deal with. So I'm not sure how a unit like $\mathrm{s^2}$ could be interpretted as an abstract time-version of an "area" - some kind of 2D "time"? There aren't two dimensions to span into, so even trying any interpretation like this seems void. Apart from this dimensional issue, the unit bundle $\mathrm{m/s^2}$ is derived not from two distinct time dimensions, but from time representations along the same dimension (how many metres you move per second can be changes each second along the same timeline, so to speak).
So, no, I don't see any significant geometric-like intepretation of $\mathrm{s^2}$. I would just stick with the idea that the $\mathrm{m/s^2}$ is nothing but a mathematically more convenient way to write $\mathrm{m/s/s}$. Since it also happens to be mathematically consistent, this is the notation that has become consensus and standard use.
