First of all the torque caused by gravity on an object is not $\vec{r}_{cm}\times M\vec{g}$ it is only in uniform gravitational filed see the torque on each infinitesimal element of the object , $dm$ , is $d\vec{\tau} = \vec{r}\times\vec{F}_{g}$ where $\vec{F}_{g} $ is the gravitational force on $dm$ which in general depends on the inverse square of the element's distance to the object excerting the gravitational force on it , e.g. the earth but considering a uniform gravitational filed so $\vec{F}_{g} $ would be $dm$ times $\vec{g}$ ,the torque would be :
$$d\vec{\tau} = \vec{r}\times{dm\vec{g}}$$
now summing (integrating) over all these elements of the object we would have (note that $\vec{g}$ is constant):
$$\int{d\vec{\tau}} = \left ( \int {\vec{r}dm} \right )\times\vec{g}$$
the first term in the right side is just $M\vec{r}_{cm}$ therefore the total gravitational torque on the body , $\vec{\tau}_{g}$ , would be :
$$\vec{\tau}_{g} = \vec{r}_{cm}\times{M\vec{g}}$$
That's why they call it "center of gravity" since in general it does not coincide with the center of mass of the body , but since $\vec{F}_{g}$ does not change much in a (sufficiently small) object it's a pretty good approximation to write $ \vec{\tau}_{g} =\vec{r}_{cm}\times{M\vec{g}}$ when calculating the torque caused by gravity.
Now back to your question, you can choose the origin for calculating torque and angular momentum to be any ("inertial") point in space that you want , but since the pivot force is unknown it is better to eliminate its effect in the net torque by choosing our origin right there ($\vec{r}_{pivot} = 0$ so ... ) now the only force that has a torque is gravity , which is $\vec{\tau}_{g} =\vec{r}_{cm}\times{M\vec{g}}$ now $\vec{r}_{cm}$ is nonzero so the only possible way for zero torque is for $\vec{r}_{cm}$ and $M\vec{g}$ (force of gravity ) to be on the same line (forming a zero angle) by hanging the objects from two different points you will find two intersecting lines that will intersect at the center of gravity.