Sorry if this is a trivial question.

Why does gravity act at the center of mass?

If we a solid $E$, doesn't gravity act at all points $(x,y,z)$ in $E$? Why then when we do problems we only only consider the weight force from the center of mass?

Thanks in Advance.