In a basic friction problem with Block $A$ sliding on top of Block $B$ , the direction of the friction force is usually explained as being simply the opposite of the direction of motion. So if Block $A$ is sliding to the right, the friction force is pointing to the left. But this reasoning implicitly assumes that we are calculating friction force from the reference frame of Block $B$. What if we instead look at the problem from the reference frame of Block $A$ ? To Block $A$ , it looks as if Block $B$ is sliding to the left, so an observer on Block $A$ would say that there is a friction force which, to oppose the direction of motion, points to the right.
It seems counterintuitive and probably wrong for the direction of friction force to depend on reference frame like this. Where is the flaw in the reasoning above? Are the two reference frames described above not exactly equivalent in a way that leads to the force changing directions?