if person A is outside a pool and person B is inside a pool with goggles, won't the light beam travel closer to the normal inside the pool but then away from the normal when it enters the goggles (due to air) so then the light beam appears to travel in a straight line (even though it zigzags?)
The answer depends on the angle of the goggles with respect to the surface of the pool. What you say is true when the two are parallel, but you did not specify that. Otherwise the two water interfaces (with the air, and with the mask) will act like the (not parallel) faces of a prism, and the light will not return to the original direction inside the mask.
Simple diagram to clarify (not exact - and with rather high apparent refractive index):
On the left, the ray does not return to the same direction; on the right it does.