Dirac being opaque and hard to follow? Well I never...

In Chapter 10 Dirac argues on physical grounds that the eigenkets of an observable must form a complete set. His argument goes that say we have an observable with eigenstates $|\varepsilon\rangle$ and some general state $|P\rangle$. Then I can write $|P\rangle$ as \begin{equation}|P\rangle = \sum a_\varepsilon|\varepsilon\rangle + \sum b_\gamma |\gamma\rangle \end{equation} Where $|\gamma\rangle$ are some state that cannot be written as a combination of the $|\varepsilon\rangle$s. Unlike Dirac I have normalized my eigenstates so that $\langle \varepsilon|\varepsilon\rangle = 1$ by pulling out a complex factor $a_\gamma$ and have not bothered to write down the integral, as the argument is essentially the same and it just complicates things. Now if we make a measurement of $\varepsilon$ for this state there is a probability $|b_\gamma|^2$ that we will find the system in each of the states $|\gamma\rangle$. But $|\gamma\rangle$ does not correspond to any allowed value of $\varepsilon$, so clearly does not make sense for the result of a measurement of $\varepsilon$! Therefore $b_\gamma = 0$ for all $\gamma$ and the $|\varepsilon\rangle$s form a complete set.

In the part you are asking about he is trying to prove that this way of writing $|P\rangle = \sum a_\varepsilon|\varepsilon\rangle$ is unique, i.e. for a given state $|P\rangle$ there is only one possible set of coefficients $a_\varepsilon$. Now we have from orthogonality that \begin{equation}\langle\varepsilon^\prime|\varepsilon\rangle = \left\{\begin{array}{lc} 1& \varepsilon = \varepsilon^\prime \\ 0 & \varepsilon\ne\varepsilon^\prime\end{array}\right.\end{equation} Let us say that there is another set of coefficients $a_\varepsilon^\prime$ such that $|P\rangle = \sum a_\varepsilon^\prime|\varepsilon\rangle$. Subtracting these two expressions for $|P\rangle$ we find \begin{equation} 0 = \sum (a_\varepsilon - a_\varepsilon^\prime)|\varepsilon\rangle\end{equation} Multiplying by $\langle\varepsilon^\prime|$ we find that all the terms go to 0 except \begin{equation} 0 = (a_{\varepsilon^\prime}-a_{\varepsilon^\prime}^\prime)\end{equation} So it terns out $a_{\varepsilon^\prime} = a_{\varepsilon^\prime}^\prime$ and since this must be true for each $\varepsilon$, the expression for $|P\rangle$ was unique.

With the integral included the discrete coefficients are replaced by functions in the integral $a(\varepsilon)$ and the normalisation is $\langle\varepsilon^\prime|\varepsilon\rangle = \delta(\varepsilon^\prime - \varepsilon)$ but otherwise the argument is unchanged.

Dirac being opaque and hard to follow? Well I never...

In Chapter 10 Dirac argues on physical grounds that the eigenkets of an observable must form a complete set. His argument goes that say we have an observable with eigenstates $|\varepsilon\rangle$ and some general state $|P\rangle$. Then I can write $|P\rangle$ as \begin{equation}|P\rangle = \sum a_\varepsilon|\varepsilon\rangle + \sum b_\gamma |\gamma\rangle \end{equation} Where $|\gamma\rangle$ are some state that cannot be written as a combination of the $|\varepsilon\rangle$s. Unlike Dirac I have normalized my eigenstates so that $\langle \varepsilon|\varepsilon\rangle = 1$ by pulling out a complex factor $a_\gamma$ and have not bothered to write down the integral, as the argument is essentially the same and it just complicates things. Now if we make a measurement of $\varepsilon$ for this state there is a probability $|b_\gamma|^2$ that we will find the system in each of the states $|\gamma\rangle$. But $|\gamma\rangle$ does not correspond to any allowed value of $\varepsilon$, so clearly does not make sense for the result of a measurement of $\varepsilon$! Therefore $b_\gamma = 0$ for all $\gamma$ and the $|\varepsilon\rangle$s form a complete set.

Edit: It is also worth noting that requiring an operator to have a complete set of real, orthogonal, eigenvalues is equivalent to requiring the operator to be Hermitian, which is how this requirement on observables is normally stated.

In the part you are asking about he is trying to prove that this way of writing $|P\rangle = \sum a_\varepsilon|\varepsilon\rangle$ is unique, i.e. for a given state $|P\rangle$ there is only one possible set of coefficients $a_\varepsilon$. Now we have from orthogonality that \begin{equation}\langle\varepsilon^\prime|\varepsilon\rangle = \left\{\begin{array}{lc} 1& \varepsilon = \varepsilon^\prime \\ 0 & \varepsilon\ne\varepsilon^\prime\end{array}\right.\end{equation} Let us say that there is another set of coefficients $a_\varepsilon^\prime$ such that $|P\rangle = \sum a_\varepsilon^\prime|\varepsilon\rangle$. Subtracting these two expressions for $|P\rangle$ we find \begin{equation} 0 = \sum (a_\varepsilon - a_\varepsilon^\prime)|\varepsilon\rangle\end{equation} Multiplying by $\langle\varepsilon^\prime|$ we find that all the terms go to 0 except \begin{equation} 0 = (a_{\varepsilon^\prime}-a_{\varepsilon^\prime}^\prime)\end{equation} So it terns out $a_{\varepsilon^\prime} = a_{\varepsilon^\prime}^\prime$ and since this must be true for each $\varepsilon$, the expression for $|P\rangle$ was unique.

With the integral included the discrete coefficients are replaced by functions in the integral $a(\varepsilon)$ and the normalisation is $\langle\varepsilon^\prime|\varepsilon\rangle = \delta(\varepsilon^\prime - \varepsilon)$ but otherwise the argument is unchanged.