Consider diagram A. In diagram A a car is at the top of a cliff. Gravity is pulling it down, but it does't move since its on a flat surface. Even though it would be at a lower potential at point B, it can't get to point B because the force of gravity is perpendicular to the direction it can move.
Now consider diagram B. In diagram B, an electron is at the negative terminal. The electrical field is pulling in towards the positive terminal, but it can't get there since the conductor is perpendicular to it. Even though it would be at a lower potential at point B, it can't get to point B because the field is perpendicular to the conductor.
Yet, somehow the in the second diagram/scenario, the current does flow. Why? Gravity and electromagnetism are similar and the situations above are similar yet there is movement in the first one but no movement in the second. Why?