If you put the two poles of the electric source close together and put a carbon filament between them, perhaps you will see a plasma arc. The carbon has gone, but for a powerful enough source the flow of electrons does not stops (until the source is not exhausted). The potential difference (the voltage) is responsible for the velocity of the flowing electrons (and the area of the electrodes is somehow responsible for the possible amount of flowing electrons and the volume of the arc). Then higher the potential difference then higher the flow velocity. The same happens in a wire. And the higher velocity led to higher frequency of collisions with the hull atoms and the other electrons.
Short answer to your question: In situations of different electric potentials it changes the velocity of the involved charges. In a high potential difference the velocity of the involved electrons is higher than in a lower potential difference, where the velocity is lower. The drift velocity may be higher / lower too, but this is not a must. It depends from the electric resistance of the wires material. So for metals - since the material heats up - it increases and for - heated up by more potential difference (higher voltage) - semiconductors it decreases.