$\phi(x)|0\rangle$ is not the state of a particle (I stress that $\phi(x)|0\rangle$ is a one-particle state) with position $x$ (when the temporal component of $x$ is zero in particular).
The situation is different form the momentum representation. Indeed, $a_p^\dagger|0\rangle$ is a momentum-defined one-particle state.
The position representation of the particles of QFT is a quite delicate issue. It is still unclear and actually there are a number of no-go theorems about its existence either in terms of projection valued measures or POVMs.
An apparent statndard definition of the position representation is the famous Newton-Wigner one, however it is plagued by a number of issues concerning locality.
A modern treatise on the issues about the position representation and the various no-go theorems, in relativistic QM can be found here and more recently here.