Yes, we are allowed to translate vectors. To put it simply, think of coordinate geometry. When you shift the origin, it doesn't effect the orientation and length of a line segment. Similarly, in vectors, translation doesn't effect the vector as a vector is defined by its magnitude and direction. You can move it to anywhere in space and it will remain the same, provided these 2 properties don't change.
In the example you gave in the problem, although force vector doesn't change and is translated, the position vector of the point of application changes . b and a are completely different vectors in terms of magnitude and direction and hence their respective vector products with force vector are different and hence, the torque is different.
To summarize, translation of vectors means that the vector doesn't change if you move it to different points in space. However, the torque on a body also depends upon the position vector from centre of mass, which changes in this case. This doesn't contradict translation of vectors, since vector product depends upon both the vectors, not only force.
Update-Point of application matters only because position vector depends upon it and hence torque.