We know that $$[x,p]=i\hbar. $$
Consider now the diagonal element in the momentum representation, $$\langle p'|[x,p]|p'\rangle=i\hbar\langle p'|p'\rangle=i\hbar\delta(0).$$
But the LHS = $$\langle p'|xp-px|p'\rangle=\langle p'|xp'-p'x|p'\rangle=p'(\langle p'|x|p'\rangle-\langle p'|x|p'\rangle)=0.$$
What went wrong?
The wrong step was in assuming that those expressions are well-defined in the first place.
The "generalized" bras/kets $|p\rangle$ and $|x\rangle$ are not true members of the Hilbert space, and expressions like $\langle p'| (\ldots)|p'\rangle$ are inherently ill-defined, regardless of what's in the middle. When working with these rather singular objects and the delta functions to which they are intimately related, you must have different labels on the bra and ket, e.g. $$\langle p|p'\rangle = \delta(p-p')$$
Moreover, the expression $[X,P] = i\hbar \mathbb I$ is not actually true, because the right-hand side can act on any bra or ket while the left-hand side can act only on a limited domain. Treating this as a strict and literal equality can lead to some technical issues such as the one you are currently running into.
For perfectly good reasons, physicists often dispense with the rigorous treatment of distributions, generalized vectors, and rigged Hilbert spaces in favor of a more practical and down-to-Earth approach to quantum mechanics. However, this comes at the cost of having to deal with some opaque, seemingly ad-hoc rules like these to keep you out of the trap that you have fallen into.
The issue you have (as mentioned in comments and @J. Murray) is that none of the things you are evaluating are defined. We instead calculate $\langle p'| [ X , P ] | p \rangle$. This is manipulated as $$ \langle p' | [ X , P ] | p \rangle = \langle p' | X P - P X | p \rangle = ( p - p' ) \langle p' | X | p \rangle $$ We now use the fact that $$ X | p \rangle = - i \hbar \frac{d}{dp} | p \rangle $$ Then, $$ \langle p' | [ X , P ] | p \rangle = - i \hbar ( p - p' ) \frac{d}{dp} \langle p' | p \rangle = - i \hbar ( p - p' ) \frac{d}{dp}\delta ( p - p' ) $$ Now you see why you get infinity if you simply set $p=p'$ without careful consideration. This result is consistent with the commutator since $$ \langle p' | [ X , P ] | p \rangle = i \hbar \langle p' | p \rangle = i\hbar \delta ( p - p') $$ I'll leave it to you to verify that both results are equal.
