Why should Fermi levels be equal? Fermi level is defined as the energy level where the probability of finding an electron is 50%. Why should Fermi levels of two metals/semiconductors in contact be the same? Why don't the energy levels where the probability of finding and electron is x% line up?
 A: The Fermi level of a solid (metal, semiconductor, etc.) is synonymous to the total chemical potential of a body, which is a thermodynamic quantity but also plays a role in statistical physics as the reference energy of the Fermi-Dirac energy distribution. It is a thermodynamic law that upon contact of two systems the total chemical potentials align, so that the combined system has again one constant total chemical potential. This is similar to the temperatures of two systems equilibrating when brought into contact. 
A: Because being the Fermi level in one metal higher than the other, it means that electrons in the metal with the highest $E_F$ are able to go to lower energy states in the other metal, but not the other way around. Thus electrons will flow until both metals(or semiconductors, or combination) have the same Fermi energy, breaking this flow. Also you could think that at 0 Kelvin the probability of being  above $E_F$ is 0%.
Is very similar to connecting to water containers, one being a little more high than the other, so that the water flows until the height is the same.
