The identification of $N$ with $-i\partial_\phi$ is mathematicaly inconsistent as $N$ cannot take negative values. As a consequence it is not surprising that there are some paradoxes.

For example, from $[\phi, \hat N]=i$ we can derive an uncertainly relation $\Delta N \Delta \phi\ge 1/2$, but when $\Delta N$ is small (zero in your fixed number state) this implies $\Delta \phi>2\pi$, which is not possible as $\phi$ is an angle.

The identification of $N$ with $-i\partial_\phi$ is mathematicaly inconsistent as $N$ cannot take negative values. As a consequence it is not surprising that there are some paradoxes.

For example, from $[\phi, \hat N]=i$ we can derive an uncertainly relation $\Delta N \Delta \phi\ge 1/2$, but when $\Delta N$ is small (zero in your fixed number state) this implies $\Delta \phi>2\pi$, which is not possible as $\phi$ is an angle with bounded range.