You're probably perceiving it incorrectly. The sky is not a flat surface; it looks like the inside of a sphere, and we tend to perceive it as a flattened sphere, with the zenith "closer" than the horizon. This perception is reinforced by the appearance of the daytime sky, in which overhead clouds really are much closer than clouds near the horizon. That could throw off what you think you're seeing. Tracing a straight line across the sky can be difficult, especially near the horizon.
The perpendicular bisector should, and I believe does, point directly at the Sun.
Try holding a rigid rod, like a yard or meter stick, out at arm's length, intersecting both the Sun and the Moon, and observe the angle at which the rod crosses the moon. You could also use a length of string stretched taut between your hands.
Atmospheric refraction can shift the perceived position of the Sun or Moon by about one width (half a degree); that's not enough to explain what you're seeing.
If the Moon is very close to the horizon, refraction might distort its visible shape enough to explain the perceived anomaly, but that would be noticeable enough that I don't think it's what you're referring to.