Why does the direction of flow of charge depend on electric potential difference? Why does electric charge flow direction depend on the potential difference and not the charge difference between two points?
 A: Systems try and minimise their potential energy.
So if a charge is able to move from a position where it has a high potential energy to a position where it has a lower potential energy it will do so.
Here is an example where there is a charge difference and no flow of charges:

The metallic sphere is initially uncharged and because it is a conductor the potential of all parts of the sphere are the same.
When the negative rod is brought close to the metallic sphere electrons in the metal sphere move away from the rod and leave a surplus of positive charge close to the rod.
The electrons moved so as to reduce the potential energy of the system (rod and sphere) as compared with the potential energy the system would have had if the charges had not moved.
Note that the negative rod had tried to make the region of the sphere closest have a lower potential than that region which is furthest from the sphere.
The rod set up a potential difference across the metal sphere which become zero due to the redistribution of the electrons.  
You will note that the charge distribution on the metallic sphere is asymmetric but the charges do not move because in that position the potential energy has been minimized and the potential of the metallic sphere is the same everywhere.  
As the negative rod is removed there is a potential difference across the metal sphere and the electrons move to reduce the potential difference until it becomes zero.
A: Because the potential difference is dependent on the amount of charge and hence the "charge difference" of the two points.
The  electrostatic potential $$\vec{E}(\vec{r}')=\int \rm{d} \vec{r} \rho(\vec{r}) \frac{\vec{r}-\vec{r}'}{|\vec{r}-\vec{r}'|^3}$$ is dependent on the charge density $\rho(\vec{r}) $ which may be different along a line connecting two points A and B. The potential between those two points would be represented as a line integral $$V=-\int_c \vec{E} \cdot\rm {d}  \vec{l}$$ where $c$ is a path connecting the two points. 
As in gravitational fields the objects in electrical fields tend to move from higher to lower potential to lower their energies.
A: The electric field, ${\bf E}$ is related to the gradient of the potential, ${\bf E} = -\nabla  V$ and the force on a charge $q$ is ${\bf F}= q {\bf E}$. This means positive charges move in the direction of decreasing potential and negative charges move in the direction of increasing potential. The source of an electric field is due to charge separation of equal amounts of positive and negative charge, so in a sense, the charge $q$ will move between two places where there is a difference of charges: moving away from the charge of the same sign towards the charge of the opposite sign.
A: The unit of electric potential (volt) has charge as a unit within it. 
V = Joule / Coulumb 
The difference in charge between two points determines how much energy you (would need to move them together) or (could get out of moving them apart). That is why we call it electric potential. 
This also determines the direction that charge will want to move in. 
A: The answer you want , I think can be found using energy considerations. Thomsons theorem states that given a certain total charge on a number of surfaces, the configuration of the system such that the field energy is minimised requires each of the surfaces to be an equipotential. For more rigorous mathematical proofs you can google for many papers on this theorem and its extensions. Hope this helps.
A: Others have given good answers in terms of math and electromagnetics. I think it may help to have a conceptual answer too. 
Let's use an analogy between electric charge and the simple mechanics of water. 
In a water system, potential energy is due to gravity. Water raised up high in a water tower or a lake on a hill has higher potential energy than water at ground level. In a water system, voltage is analogous to the height difference in the two bodies of water, and charge is analogous to the amount of water. 
Let's say you connected a water tower to a tank at ground level with a pipe. When you open the valve on the pipe, which way will the water flow? Down of course. 
But note I didn't have to tell you which tank was larger. Whether the tank in the tower has more water or less than the tank at ground level, the water always flows down. That's because without outside influence, the water will always flow from the higher potential to the lower. 
The same is true in an electrical system. The flow of charge is always from higher to lower potential. The amount of charge on either end will affect the rate of flow and other interesting things, but not the direction of flow. 
Water flows downhill. Charge also flows "downhill" in terms of potential. 
A: 
Why does electric charge flow direction depend on the potential difference [...]

Because a system will always want to minimize it's energy. Just like a book falls to lower height (lower gravitational potential energy), a charge will move towards a point which it is repelled the least from (lowest electrical potential energy).

[...] and not the charge difference between two points?

Untrue. Their sign difference does matter! Different signs attract, equal signs repel.
Now, their difference in magnitude seems not to matter - at first sight.
But imagine having 10 negative charges at one point and just 2 negative charges at another point. Then a new added negative charge will be repelled more from the first point than from the latter and will of course rather move towards the latter - which is exactly what potential energy means.
So charge magnitude does matter on direction, and is the reason for potential energy.
