In chapter 15.2 of Peskin, the comparator is defined, as some object $U\left(y,\,x\right)$ which transforms as: $$ U\left(y,\,x\right) \mapsto V\left(y\right) U\left(y,\,x\right) \left[V\left(x\right)^\dagger\right] $$
where $V\left(x\right)\in SU\left(2\right)^{\mathbb{R}^4}$.
This object is introduced mainly in order to be able to define the covariant derivative, but its main defining property is that the Fermion field transforms then as:
$$ U\left(y,\,x\right)\psi\left(x\right)\mapsto V\left(y\right)U\left(y,\,x\right)\psi\left(x\right)$$ so we basically have a way to make the Fermion field transform as if it were at $y$ instead of at $x$.
At some stage Peskin states it is reasonable to that this comparator be unitary. My question is why? What breaks down, and where, if we don't assume that?