Just as a quick example: say that the dot product were not invariant under transformations. Then let's say that we have two reference frames, A and B, where reference frame B is rotated and displaced with respect to A and which moves at a constant speed w.r.t. A (where $v\ll c$).
Then the researcher in A wants to calculate the gravitational attraction between two known masses. For this he seperates the masses by a rod which seperates the masses and which extends from the origin of his reference frame to some point $\vec r={(x,y,z)}$. The gravitational strength is proportional to the length of this rod. How does he measure the length? He takes the dot product of $\vec r$ and takes the square root of the resulting number.
Now the researcher in B wants to know the length and the resulting gravitational attraction from the known masses, but since he is far away from the rod he can't measure the length directly. So he says to researcher A: ok, I can't see the rod, but there is a set of rules which tell me how to calculate where your origin is and where the point $\vec r$ is viewed from my frame, from this I can calculate the length. These rules are of course the galilei transformations.
After he has done the calculation he finds out that the rod is shorter in his frame since the dot product is not invariant, and because the rod has a different length the gravitational attraction between the masses is different. This is nonsensical! The gravitational attraction between two masses should not depend on the reference frame.
This is why vectors and transformations and tensors and what not exist: such that the laws which govern one frame of reference are also valid in any other frame which is obtained from the first by a transformation.