I would expect the translation operator to be more specific than what you describe. Specifically, I would expect the translation operator to change a state localised within some region $\Omega$ to a state localised within a region $\Omega + \epsilon$.
If you manage to single out which operators measureNote that in ordinary quantum mechanics, the probablityprobability that a state is localised within a certainthe region omega can be written as $\langle 1_\Omega\rangle$, where $1_\Omega(x)$ is 1 if $x\in \Omega$ and 0 otherwise. In quantum field theory, you will need to combine this function with a density operator, since particle number can vary.
If you manage to single out the operators $1_\Omega$, you can use them to construct a coordinate system. So whether you start with a coordinate system, or with a sensible notion of translation operator, you end op in the same situation in the end.