Imagine your rod is made up from lots of little bricks stacked on each other. Then the total potential energy is the sum of the potential energy of each brick.
The diagram shows the rod and one of the bricks of size $dx$ and at a height $x$. If $\rho$ is the mass per unit length, then the mass of the brick is $\rho \space dx$, and the potential energy is $\rho \space dx \space g \space x$. To get the total potential energy we just have to sum up the potential energies of all the bricks. To do the sum we let $dx$ go to zero and replace the sum with an integral so:
$$ \begin{align}
U &= \rho \space g \int_0^h dx . x \\
&= \rho \space g \left[\frac{x^2}{2}\right]_0^h \\
&= \frac{\rho g h^2}{2}
\end{align} $$
And since $\rho h$ is just the mass $m$ this gives us:
$$ U = \frac{mgh}{2} $$
As you say, this is the same result you get by just considering the centre of mass, but note that we've got this result without involving the centre of mass at all. I think this is a better way to understand why the potential energy is $mgh/2$ without just invoking the centre of mass and waving your arms in the air.