Where is the misconception here?
The misconception is here:
When we have a battery with zero internal resistance, it means that
the power it provides (or the power it takes when current flows in the
other direction) is given by $VI=I^2R=0$.
This isn't correct. The power delivered by the voltage source (battery) to the load is simply the product of the voltage across and the current through the load.
$$P_L = V_L I_L$$
If the voltage source has internal resistance $r$, the power dissipated by that internal resistance is
$$P_r = I^2_Lr$$
Note that this isn't the power the source provides (as you state above) but rather the power that isn't available to the load since it is power that is lost to heating of the voltage source.
If the load is a resistor with resistance $R$, and the voltage source has an open circuit voltage $V$, then the load current $I_L$ is given by
$$I_L = \frac{V}{R + r}$$
and then
$$P_L = I^2_L R = V^2\frac{R}{(R + r)^2}$$
$$P_r = I^2_L r = V^2\frac{r}{(R + r)^2}$$
In the ideal case that $r = 0$, it follows that
$$P_L = \frac{V^2}{R}$$
$$P_r = 0$$
and so there is zero power dissipated by the source but non-zero power delivered by the source. For the case that the load is a short circuit $(R = 0)$
$$P_L = 0$$
$$P_r = \frac{V^2}{r}$$
and so there is zero power delivered by the source but non-zero power dissipated by the source.
See that the case that both $R$ and $r$ are zero (ideal voltage source with ideal short circuit load) is not defined (division by zero).