The answer is neither smaller, as is the common assumption, nor the same, as in the answer Cristi Stoica gave (although her logic gets us closer to the actual solution). In fact, as an observer pulls away from a flat mirror, his image actually gets larger! The reason for this is that the observer (like the rest of us) is not a simple 2-dimensional line, but a 3-dimensional solid.
If instead of drawing the observer as a line, we draw him as a circle, it becomes pretty easy to see that as the observer pulls back, the sight lines (tangents) from the eye to the edge of the reflected image "roll back" along the edge of the observer's body. The size of the image at the mirror's surface gets bigger (albeit not by an easily observable amount after a short distance).
At distance d=0 from the mirror, the reflected image size = 0 (you never get that in real life, because your nose keeps your eyes from getting right up to the mirror's surface.
At distance d=infinity, the reflected image size is one-half the subject's height.