Gravitational force between rod and a point mass? Gravitational force could be easily calculated by dividing the rod into very small segments, calculating the individual forces between small segment of rod and point mass and simply integrating the total force for a complete rod.
But my question is that why cannot we assume that the whole mass of the rod to be concentrated at the center of mass of the rod, thus considering it as a point mass and directly calculating the force by Newtons Gravitational formula?
 A: Because Newtons law of gravitation gives the first result if applied correctly (as you do in the first paragraph)!
Newtons law of gravitation is not about the centre of mass! The reduction to the centre of mass is only valid for spherically symmetric objects (due to the shell theorem) or as an approximation in the limit of large distances (due to the multipole expansion, where large means large compared to the extents of the interacting mass distributions).
A: The easiest way to see it is to replace the rod with two point masses connected by a rigid bar. This will simulate two opposite small bits of the rod.  Let all three masses have mass $m$, the distance from the single mass to the center of the bar $R$ and the distance from the center of the bar to one of the masses be $r$.  Assume the bar is oriented along the line from the center to the single mass.  Then $$F=Gm^2\left(\frac 1{(R-r)^2}+\frac 1{(R+r)^2}\right)$$ while lumping the two masses at the center of the bar gives $$F=Gm^2\left(\frac 2{R^2}\right)$$ and we have $$\left(\frac 1{(R-r)^2}+\frac 1{(R+r)^2}\right)=\frac{2R^2+2r^2}{(R^2-r^2)^2}\gt \frac 2{R^2}$$ The point is that the force on the close mass increases more than the force on the far mass decreases.  If you do the calculation with the bar perpendicular to the line to the single mass, you will find the force is reduced because the distance to the two masses is $\sqrt{R^2+r^2}$ instead of $R$  
If $r \ll R$ these effects become small and may be able to be ignored in practical calculation.  The corrections fall off as $\frac 1{R^3}$ or faster.  From far enough away, you can just lump all the mass at the cm and consider a body a point mass.
