In metals, Fermi energy is the energy of the fastest moving electron but why doesn't this same definition hold good for semiconductors?
Because the Fermy energy is not really the energy of the fastest electron(altougth for understanding how it works, it is not that bad start, it can be seen that way for metals, for example). It is more like the energy under which all the levels are filled. For semiconductors, as they have band structure, the energies that an electron can take are limited to some levels or bands. Then the Fermi energy is just a limit. As the electrons are forbiden to have certain energies, and if the Fermi energy lays in this frobiden region it just means than all levels under it will be filled with electrons, but no electrons will have fermi enrgy unless it is crossing one available band. It's kinda more complicated, see http://hyperphysics.phy-astr.gsu.edu/hbase/solids/band.html for more information.