You're confusing two phenomena here.
I have read that there is no force on Q from the shell no matter where Q is inside the shell ('there will be a large force from a few electrons pulling the charge one way, and a smaller force but from more electrons pulling the charge the other way').
This is (a paraphrase of) the usual argument for the fact that $\vec{E}$ vanishes inside a spherical shell with a uniform charge distribution. If we were to place a charge $Q$ inside the shell, and held all the charges on the shell fixed (for example, if it was insulating), then $Q$ would feel no force.
Critically, the usual argument relies on the fact that the charge distribution is uniform over the surface of the shell. If the charge on the shell isn't evenly distributed, the electric field inside the shell will not be zero.
But I have read that then the electric field from Q induces a charge (the electrons in the shell rearrange themselves to neutralise the electric field from Q) after which Q will feel an attractive/repelling force from the shell.
If, on the other hand, the sphere is conductive, the charges can move around on it. Placing the charge $Q$ inside the sphere then causes them to experience a force, and they will redistribute themselves so that they are in equilibrium with $Q$ and with the other charges on the sphere.
It doesn't seem to make sense as the force per charge (E-field) inside the shell was zero before we added Q, and then we put a charge of 1Q in there, so Q should feel zero force from the shell?
The key difference here is that you're talking about two different distributions of charge on the shell. Since the shell charges are in different places here, we shouldn't expect the forces on $Q$ to be the same in each case.
As an analogy: suppose there is a charged bead $q$ that can slide along a rod placed along the $x$-axis. I then fix a second charge $Q$ in place somewhere along the $y$-axis. At the moment I put $Q$ in place, it feels some particular force. But $q$ can slide along the rod, and so it gets pushed away from $Q$, which causes the force on $Q$ to change.