You are right!
If you consider whatever is producing the $\vec{B}$ field as part of the system then the whole thing is symmetric under time reversal. It is only when the $\vec{B}$ field is considered as something external (on which the time-reversal operator does not act) that the symmetry is broken.
It doesn't seem physical for K to act on one part of the system.
I don't think a 'universal' time-reversal operation is any more (or less) physical than one which acts on only a certain subsystem (can you make time run backwards?).
In practice, what time-reversal symmetry means is something like this. (I will give a classical description for ease of intuition, a quantum version can be formulated along similar lines.) Suppose I have an interacting system of $N$ particles which I prepare in state $(\vec{q}_1(0),\vec{p}_1(0))$, and after time $t$ is found to be in state $(\vec{q}_1(t),\vec{p}_1(t))$. I now prepare a second system with identical particles in state $(\vec{q}_2(0),\vec{p}_2(0))=(\vec{q}_1(t),-\vec{p}_1(t))$ (i.e. with the same positions and opposite velocities of the final state of system 1). If the interaction is time-reversal symmetric then after running the second experiment for time $t$ it is found that $(\vec{q}_2(t),\vec{p}_2(t))=(\vec{q}_1(0),-\vec{p}_1(0))$, so that the dynamics of the second experiment are exactly the reverse of those of the first experiment.
Now, if I know that the interaction is electromagnetic, and there are no external forces, then I know the results must obey time-reversal symmetry (as the laws of electrodynamics have this symmetry). However, if I choose to impose an external magnetic field on the first experiment then when I perform the second experiment I have the choice of whether to reverse this field or not.
- If I choose to reverse the field then the results show time-reversal symmetry.
- If I choose not to reverse the field then the results do not show time-reversal symmetry.
Statements 1 and 2 seem equally meaningful (and equally 'physical') to me.