If I hold a pencil at its end and spin it, throwing it upwards, it will spin about its end, but will soon start spinning around its center. How is this?

I would draw the following torque diagram:

 - Object: uniform thin rod with length $\ell$ and moments $I_{center}=\frac 1 {12} m\ell^2$ and $I_{end}=\frac 1 3 m\ell^2$)
 - Center of rotation some small distance $d$ from the end
 - Torque $mg$ downward, at center of mass, with $\theta = 90°$ and $r = \frac \ell 2 - d$
 - Possibly wind resistance $D$ upward, at center of mass, with $\theta = -90°$ and $r = \frac \ell 2 - d$

Thus, $\tau = \sum {r \times F} = \sum {rF~sin\theta} = \left (\frac \ell 2 - d\right)(mg - D)$. I could see how this might cause it to spin, but how does the center of rotation to move?