How does time dilation work without a privileged reference frame?

As I understand special relativity, light travels at the same speed in all reference frames.

What I fail to understand is why time dilation would occur in one reference frame, but not by an equal amount in all other reference frames, at the same time.

To put it another way, if I am sitting on earth, observing a space ship that is flying at a speed near $c$, I should observe him moving through time slower than me. However, looking out his window, he should observe me traveling relative to him at a speed near $c$ and should likewise see me moving through time slower than himself.

What am I missing from my understanding of special relativity?

-
 possible duplicate of Time Dilation - how does it know which FoR to age slower? – Qmechanic♦ Dec 29 '11 at 16:27