According to the Drude Model, we have that $$\vec{J}=\frac{ne^2 \tau}{m_e}\vec{E}$$ where $\tau$ is the mean free time of each electron and $n$ is the free electron density. This is simply ohms law and hence the Drude model provides a microscopic explanation for Ohms Law. The Drude model also implies that the mean free time $\tau$ and the mean free distance $\lambda$ are independent of the applied electric field provided the range of applied fields is not too great (this is because thermal velocities are much greater than drift velocities and so $\tau$ and $\lambda$ are determined by the temperature and not the field). My issue is that in a real resistor, if we double the applied voltage and hence double the electric field present inside the resistor, then the energy dissipated within the resistor is quadrupled because the power has a quadratic dependence on V ($P=\frac{V^2}{R}$). But according to the Drude model, $\tau$ and $\lambda$ are independent of the field, so if we double $\vec{E}$ then $\tau$ and $\lambda$ remain constant. This means that we have exactly the same amount of collisions per unit time (despite doubling the field) and we also have the electrons travelling the same distance per unit time (despite doubling the field). But this is clearly inconsistent with the fact that the power has a quadratic dependence on the electric field/Voltage. We should expect that doubling $\vec{E}$ should either double $\lambda$ or half $\tau$ (thereby doubling the amount of collisions per unit time) in order to get the quadratic dependence of power on E. So what is up with this?
How do we reconcile the fact that power dissipation is quadratically dependent on $\vec{E}$ with the Drude model given that the Drude model implies/assumes that $\tau$ and $\gamma$ are independent of the applied electric field?
Any help on this issue would be most appreciated.