The above question arose when I read Stephen Barnett's and Paul Radmore's Methods in Theoretical Quantum Optics. In Appendix A (on quadrature eigenstates), the following line of argumentation appears:
Step 1
$\hat{x}$ and $\hat{y}$ are two hermitian operators whose commutator is $$\left[ \hat{x}, \hat{y} \right] = i$$ The eigenvectors of $\hat{x}$ are denoted $| x \rangle$. Their properties are $$\hat{x} | x \rangle = x | x \rangle \qquad \text{and} \qquad \langle x | x' \rangle = \delta( x-x' ).$$ Arbitrary vectors $|\Psi\rangle$ are represented in the eigenbasis of $\hat{x}$ by their wavefunction $$\Psi(x) \equiv \langle x | \Psi \rangle.$$ Obviously, the action of $\hat{x}$ on the wavefunction is multiplication with $x$, i.e. $$\langle x | \hat{x} | \Psi \rangle = x \Psi(x).$$
Step 2
In order to determine the action of $\hat{y}$ in the eigenbasis of $\hat{x}$, one considers \begin{align} \langle x | \left[ \hat{x}, \hat{y} \right] | x' \rangle &= i \, \langle x | x' \rangle = i \, \delta( x-x' ), \\ \langle x | \left[ \hat{x}, \hat{y} \right] | x' \rangle &= \langle x | \left( \hat{x}\hat{y} - \hat{y} \hat{x} \right) | x' \rangle = (x - x') \, \langle x | \hat{y} | x' \rangle. \end{align} Rearranging terms yields $$\langle x | \hat{y} | x' \rangle = i \frac{\delta( x-x' )}{(x - x')} = -i \, \delta'( x-x' ).$$
EDIT
Chiral Anomaly suggested that the above equation is invalid because it involves division by zero. I have come to believe that it is in fact valid and the division by zero is to be meant only symbolically (like the delta function is not an actual function). It can easily be shown that $$x \, \delta'(x) = - \delta(x)$$ c.f. https://en.wikipedia.org/wiki/Dirac_delta_function#Distributional_derivatives, and through substitution one obtains $$ (x-x') \, \delta'(x-x') = \delta(x-x').$$ Knowing this, it seems reasonable to conclude from $$(x-x') \, \left( i \, \langle x | \hat{y} | x' \rangle \right) = \delta(x-x')$$ that $$i \, \langle x | \hat{y} | x' \rangle = \delta'(x-x') \qquad \text{i.e.} \qquad \langle x | \hat{y} | x' \rangle = -i \delta'(x-x')$$ This "proof" hinges on the unqiueness of the distribution $d(x-x')$ satisfying $(x-x') \, d(x-x') = \delta(x-x')$ (which I believe should not be an issue). However, as far as I see there is no division by zero involved.
The action of $\hat{y}$ on the wavefunction follows via $$\langle x | \hat{y} | \Psi \rangle = \int_{-\infty}^\infty dx' \; \langle x | \hat{y} | x' \rangle \, \Psi(x') = -i \, \partial_x \Psi(x).$$
I am confused!
To conclude, the action of the operator $\hat{y}$ in the eigenbasis of $\hat{x}$ was derived solely from the commutator $[\hat{x}, \hat{y}]$. But there is surely more than one operator with the specified commutator! For example, $$[\hat{x}, \hat{y} + \lambda \hat{x}] = [\hat{x}, \hat{y}] \qquad \forall \lambda \in \mathbb{C},$$ i.e. I can shift $\hat{y}$ by any amount of $\hat{x}$ without changing the commutator. The action of that shifted operator on the wavefunction will be $$\langle x | \hat{y} | \Psi \rangle = -i \, \partial_x \Psi(x) + \lambda \, x \, \Psi(x),$$ which contradicts the conclusions from step 2.
Where is the mistake?
PS: Motivation
To add a bit of context, I will explain how I came about the text summarized above. The following is not necessary to understand the question.
I was trying to determine the scalar product between a quadrature eigenstate and a number state of a single electromagnetic field mode. The quadrature eigenstate $|x\rangle$ is defined by $$ \hat{x} |x\rangle \equiv \left( \hat{a} + \hat{a}^\dagger \right) |x\rangle = x |x\rangle,$$ while the number eigenstate is given by $$\hat{a} \hat{a}^\dagger |n\rangle = n |n\rangle.$$ To evaluate $\langle x | n \rangle$, one can use (as Barnett and Radmore do) that $$\hat{a} \hat{a}^\dagger = A \hat{x}^2 + B \hat{y}^2$$ with $\hat{x} \propto \hat{a} + \hat{a}^\dagger$ (position operator of quantum harmonic oscillator) and $\hat{x} \propto i \, \left( \hat{a} - \hat{a}^\dagger \right)$ (momentum operator). When the action of $\hat{x}$ and $\hat{y}$ on the "position" eigenstates is known, the wavefunctions of the number eigenstates can be obtained as the solution of an ordinary differential equation. The authors cite here the well-known result for the quantum harmonic oscillator. However, unlike in the case of the quantum harmonic oscillator, it seems unwarranted to just postulate a specific representation of the momentum operator, since there is no correspondence rule available.
