Why is this not a realisable operation on a quantum system? Let $\rho = \begin{bmatrix}\ 1&0 \\ 0&0 \end{bmatrix}$, $\rho' = \begin{bmatrix}\ 0&0 \\ 0&1 \end{bmatrix}$, $\rho'' = \dfrac{1}{2}\begin{bmatrix}\ 1&1 \\ 1&1 \end{bmatrix}$ (all density operators).
Consider a physical operation $\phi$ such that $\phi(\rho) = \rho$, $\phi(\rho') = \rho'$, $\phi(\rho'') = \dfrac{1}{5}\begin{bmatrix}\ 4&2 \\ 2&1 \end{bmatrix}$.
Why is $\phi$ not a realisable physical operation? It certainly preserves trace and positivity...
 A: Your map fails to be completely positive.  If you apply it to half of a maximally entangled state $(|0\rangle|0\rangle+|1\rangle|1\rangle)/\sqrt{2}$, you can easily see that $\phi(\rho)=\rho$ and $\phi(\rho')=\rho'$ imply that $\phi(|0\rangle\langle1|) = \alpha |0\rangle\langle1|$ and $\phi(|1\rangle\langle0|) = \alpha^* |1\rangle\langle0|$ for the resulting state to be positive (with $|\alpha|\le1$).  However, this is incompatible with the last condition.
A: A physical quantum operation ${\cal E}$ can be described as a map between the set of density operators of the form
$${\cal E}(\rho) =\sum_k E_k \rho E_k^{\dagger}, \qquad  \sum_kE_k^{\dagger}E_k \leq {\bf 1},$$
cf. Ref. 1. As Norbert Schuch correctly notes this implies that a physical quantum operation ${\cal E}$ must be a completely positive map. In this answer we note  that OP's example fails to be quantum operation for an even more elementary reason: Its image (of the Bloch ball $B^3$) doesn't lie inside the Bloch ball
$$ \rho~=~\frac{1}{2}({\bf 1} + \sigma), \qquad \sigma = \sum_{i=1}^3x^i\sigma_i, \qquad \sum_{i=1}^3(x^i)^2\leq 1.$$
In detail, let 
$$\rho^{+} ~=~ \begin{bmatrix}\ 1&0 \\ 0&0 \end{bmatrix}, \quad 
\rho^{-} ~=~ \begin{bmatrix}\ 0&0 \\ 0&1 \end{bmatrix}, \quad 
\rho_{1} ~=~ \frac{1}{2}\begin{bmatrix}\ 1&1 \\ 1&1 \end{bmatrix}, \quad 
\rho_{2} ~=~ \frac{1}{5}\begin{bmatrix}\ 4&2 \\ 2&1 \end{bmatrix},$$
with 
$$ {\cal E}(\rho^{\pm})~=~\rho^{\pm}, \qquad {\cal E}(\rho_1)~=~\rho_2. $$
In other words, the North and the South pole of the Bloch sphere $S^2$ are fixed points, and the pure state $(1,0,0)$ is mapped to the pure state $(\frac{4}{5},0,\frac{3}{5})$ in the $xz$ plane.
It follow from linearity that
$$ {\cal E}({\bf 1})~=~{\bf 1}, \qquad {\cal E}(\sigma_3)~=~\sigma_3, \qquad {\cal E}(\sigma_1)~=~\frac{4}{5}\sigma_1+\frac{3}{5}\sigma_3. $$
In other words, the great circle in the $xz$ plane is mapped to an ellipse in the $xz$ plane, which spends half the time outside (and half the time inside) the great circle. So the image of $ {\cal E}$ doesn't lie inside the Bloch ball as it should.
References:


*

*M.A. Nielsen and I.L. Chuang, Quantum Computation and Quantum Information, (2011) Section 8.2.

