Is obtaining the coordinate representation of momentum operator from commutator more fundamental than generator of translation

Question

Related post: What is the most general expression for the coordinate representation of momentum operator?

There are two methods of obtaining the coordinate representation of momentum in quantum mechanics. $$ \langle x|p|y \rangle = -i \delta'(x-y). \tag{1} $$

The first one is from the canonical commutator $$[x,p]=i \tag{2} $$ As shown in Dirac's principles of quantum mechanics 4th edition section 22, actually there is an ambigulity in this procedure. $$ \langle x|p|y \rangle = -i \delta'(x-y) + F' \delta (x-y) \tag{3} $$ where $F':=dF/dx$ is a derivative of a general function $F(x)$ (I slightly modify the equation in Dirac's book).

Eq. (3) satisfies the commutator (1), but implies arbitary value of expectation of momentum. As noticed by Dirac, this ambigulity can be removed from a local phase factor $\psi \rightarrow e^{-iF(x)} \psi$.

The second one is regarding momentum operator is the generator of translation, as given in Sakurai's modern quantum mechanics 1st edition p54,

$$ ( 1- \frac{ i p \Delta x'}{\hbar} ) | \alpha \rangle = \int dx' \mathcal{F} (\Delta x') | x ' \rangle \langle x | \alpha \rangle $$ $$ = \int dx' | x' + \Delta x' \rangle \langle x' | \alpha \rangle $$ $$ = \int dx' | x' \rangle \langle x' - \Delta x' | \alpha \rangle $$ $$ = \int dx' | x' \rangle \left( \langle x'| \alpha \rangle - \Delta x' \frac{ \partial}{\partial x'} \langle x' | \alpha \rangle \right). \tag{1.7.15} $$

Comparision both sides yields

$$ p | \alpha \rangle = \int dx' | x' \rangle \left( - i \frac{ \partial}{\partial x'} \langle x'| \alpha \rangle \right) \tag{1.7.16} $$ $$ \langle x' | p | \alpha \rangle = - i \frac{ \partial }{ \partial x'} \langle x'| \alpha \rangle \tag{1.7.17} $$

It seems that the arbitrariness as Eq. (3) is hidden in the second approach.

My questions are:

(i) Did Dirac already discover the origin of gauge invariance? If we do not consider from the commutator, I may say, there is no gauge invariance. A local phase factor would modify the expectation value of momentum, therefore we can only have global phase factor corresponds to the same state. However, since local phase factor comes from commutator, it has to be a redundency.

(ii) Is the first method, from commutator, more fundamental than translation generator? Since we can find gauge invariance from the first method.

Answer 1

We interpret OP's question (v4) as:

How do we recover the phase ambiguity from the generator of translation method in Ref. 1?

Recall that an eigenvector for an operator can be rescaled with a non-zero multiplicative factor. The main point is that the position eigenket $| x \rangle$, which satisfies

$$\tag{A} \hat{x}| x \rangle~=~ x| x \rangle, $$

can always be redefined with an $x$-dependent phase factor without destroying the normalization condition

$$\tag{B} \langle x | x^{\prime} \rangle ~=~\delta(x-x^{\prime}).$$

So the phase ambiguity is encoded in the different choices of position eigenkets $| x \rangle$. See also e.g. this, this, and this related Phys.SE posts.

References:

1. J.J. Sakurai, Modern Quantum Mechanics, 1994, p. 54.