There is a heuristic way to look at this.
The Dirac delta function corresponds to a spike when its argument is zero. You can view it as the limit of a sequence of Gaussian functions whose areas are all one but whose width goes to zero. The derivative of a Gaussian function looks like this:
So in the limit, the derivative of the Dirac function is something like an up spike infinitesimally to the left of the origin followed by a down spike infinitesimally to the right. So the matrix elements you're looking at aren't actually diagonal, they're infinitesimally off-diagonal.
These kinds of heuristics can be useful, but they can also be dangerous, so don't take what I say too literally.
Another way to look at this is to approach the derivative Dirac delta via a discretisation. If the wavefunction is represented by a vector of equally spaced samples, the derivative can be represented by central differences. Assuming periodic boundary conditions we get a matrix like:
0 & 1 & 0 & 0 & \ldots & -1 \\
-1 & 0 & 1 & 0 & \ldots & 0 \\
0 & -1 & 0 & 1 & \ldots & 0 \\
0 & 0 & -1 & 0 & \ldots & 0 \\
& & \vdots \\
1 & 0 & 0 & 0 & \ldots & 0 \\
We have 1's just above the diagonal and -1's just below. As the discretisation gets finer we get a matrix where they entries are more and more concentrated near the diagonal even though all of the non-zero terms are actually off-diagonal. In the limit you can again imagine something that is infinitesimally off-diagonal.